Henrik Lundqvist blocked Langenbrunner's drive with his glove, but the puck bounced behind him and went in before he could recover. Petr Prucha had a chance to tie in the first extra round, but fired a shot off the crossbar.
Zach Parise forced overtime and completed New Jersey's rally from a 2-0 deficit with his 25th goal, scoring with 7:13 left in regulation. Brian Rafalski scored in the second period for the Devils, who won the season series 5-2-1 - including two shootout victories.
Marcel Hossa and Prucha scored for the Rangers, 11th in the Eastern Conference and five points behind Carolina and the playoff cutoff.
Martin Brodeur earned his 38th win while making his 31st straight start for the Devils, who beat New York 2-1 on Tuesday.