After the first five shooters scored, goaltender Manny Legace stopped Fredrik Modin to end the game. Legace finished with 19 saves.
Trailing 3-2 late in the third period, Martin Rucinsky stripped the puck from defenceman Ron Hainsey next to the Columbus goal and scored into an open net for his 10th to tie it with 7:29 remaining.
Bill Guerin and Keith Tkachuk also tallied for the Blues, who have points in 10 of their last 12 games. Rucinsky also had an assist.
Dan Fritsche, Aaron Johnson and Manny Malhotra scored for Columbus. Anson Carter added two assists in his first game back after missing seven with a shoulder injury.
St. Louis had tied it 2-2 early in the third period on Tkachuk's redirection, but Malhotra responded just over 2 minutes later - on a centring pass from Carter - from the slot on the power play.
Columbus scored first midway through a low-key opening period - even though there were two goals.
David Vyborny cut across the high slot and slid a pass to Fritsche, charging to the net. Fritsche redirected the puck past Legace for his eighth to make it 1-0.
Guerin made it 1-1 at 15:14 on the power play, taking Stempniak's feed at the low, left circle and scoring over a sprawled Fredrik Norrena. The goal was Guerin's 18th after tallying just 13 last season.
Johnson converted a one-timer from the slot on a pass from Carter behind the net to make it 2-1 at 13:29. Blues defenceman Jamie Rivers overskated the puck, leaving Carter wide open.
The teams looked disinterested most of first two periods - registering only 12 shots apiece - until Columbus' go-ahead goal.
Tkachuk scored his 12th on St. Louis' first shot of the third period at 1:07. He has four goals in his last six games.
Notes: St. Louis coach Andy Murray - 7-4-3 with the Blues - was one of two candidates to interview in November to be the Blue Jackets' coach. Columbus ownership selected Ken Hitchcock, who has guided the team to a 11-9-3 mark. ... Stempniak extended his point-streak to a career-high seven games. ...The Blue Jackets played without their top two goal-scorers - Rick Nash and Sergei Fedorov - who are injured.