The fifth straight win by the Canadiens (40-31-6) ended the Rangers' five-game winning run and moved Montreal into a tie in the Eastern Conference with seventh-place Tampa Bay at 86 points, one behind the sixth-place Rangers.
Steve Begin, Michael Ryder and Tomas Plekanec, with his 20th of the season, also scored in the second period and Mark Streit got one in the third for Montreal, which has won seven of its last eight games.
Michael Nylander and Paul Mara scored in the first frame and Thomas Pock and Matt Cullen had goals in the third for New York (39-29-9), which suffered only its fifth regulation loss in 23 matches.
Lundqvist, the first star in the NHL last week, went into the game with a .979 save percentage in his last five starts, but allowed four goals on only 15 shots. He got the hook at 9:47 of the middle period in favour of Steve Valiquette, who is filling in for injured back-up Kevin Weekes.
Kovalev got it started for Montreal when he slammed a shot in from in front of the crease on a power play 2:17 into the second period while taking a stick in the face from Cullen.
Begin elected to shoot on a two-on-one only 32 seconds later and beat Lundqvist between the pads. Ryder scored on his own rebound at 4:36 and Plekanec finished a rush with Andrei Kostitsyn at 9:47 to end Lundqvist's night.
Kovalev was battling in front of the Rangers net on a power play at 14:09 when Saku Koivu's pass went off a skate and off the big Russian for Montreal's fifth goal.
The Rangers roared back with two goals in the third when Thomas Pock's floater from the point went in off Montreal centre Maxim Lapierre at 6:23 and Marek Malik's point shot deflected off Cullen at 7:38.
But Streit cooled the comeback when his change-up from the slot went in off Valiquette's stick at 13:03. Koivu got his fourth assist of the game on the goal.
The two teams meet again on April 5 in New York in the next-to-last game of the season for both clubs.
The noise was at playoff level among the 21,273 spectators as the two teams came out hitting, but Montreal took three straight minor penalties and it cost them.
On a two-man advantage, Nylander shot the rebound of Brandan Shanahan's point shot into an open side at 11:55.
With 45 seconds left in the period, Sean Avery hit a goalpost and the puck went to Mara at the point for a wrist shot that went in through a crowd in front of Jaroslav Halak.
The Rangers end a four-game road trip Saturday in Philadelphia. Montreal plays Friday night in Ottawa and Saturday night at home against Buffalo.
Notes - Brad Isbister was scratched for the Rangers. Karel Rachunek, Fedor Tyutin and Marcel Hossa are all out with sprained MCLs. Weekes is day-to-day with a quadriceps injury. . . Montreal scratched Sergei Samsonov, Josh Gorges and Garth Murray. Alexander Perezhogin (concussion) and Cristobal Huet (hamstring) are sidelined.