Coming off consecutive losses to Atlanta, the Panthers improved to 1-4-1 on the road and 5-5-1 overall. New York, idle since winning a shootout in Toronto on Saturday night, lost for the fifth time in its last seven games to fall to 4-5-0.
Juraj Kolnik and Gregory Campbell also scored for Florida.
The 41-year-old Belfour, 2-2-0 in his first season with Florida, allowed Petr Prucha's power-play score early in the second period and Martin Straka's goal off an uncontested close-range shot with 6:50 left in the game.
Stumpel opened the scoring with a power-play goal in the first period and sealed the victory with 3:29 remaining with his fifth goal of the season.
Kolnik gave the Panthers a two-goal lead at 6:32 of the third period after stealing Rangers defenceman Michal Rozsival's backhand pass attempt along the boards. Kolnik broke in alone on goalie Henrik Lundqvist, faked him out of position and easily forehanded the puck into the empty net for his fourth goal of the season.
Campbell gave Florida a 2-1 lead at 4:10 of the second period, tipping Rostislav Olesz's high shot past Lundquist just 36 seconds after Prucha tied it with his first goal of the season. Prucha, a 30-goal scorer last season as a rookie, beat Belfour off Brendan Shanahan's rebound a second after a two-man advantage ended.
Two nights after a special-teams meltdown in a 6-3 home loss to Atlanta, the Panthers killed Joe Nieuwendyk's early hooking penalty and opened the scoring on Stumpel's power-play goal. On Monday night, Florida allowed five goals in seven short-handed situations and failed to score on six power-play chances.
Stumpel scored with New York's Karel Rachunek off for hooking, firing a low snap shot through Lundqvist's legs for his fourth goal of the season.
The Panthers also killed Steve Matador's elbowing penalty that carried into the second period and Gary Roberts' roughing infraction early in the second, but Roberts was only a step on the ice and Olli Jokinen still off for tripping when Prucha scored.
Lundqvist made 20 saves for the Rangers. Last season, he was 3-0 against Florida, including his only two shutouts.
Notes: Florida right wing Todd Bertuzzi missed his fourth straight game because of back spasms. ... After playing six of their first nine games at home, the Rangers will play nine of their next 11 on the road, starting Saturday night in Phoenix with the first of four straight games on a West Coast swing. ... The Panthers will face New Jersey on Thursday night and complete their three-game trip Saturday night against the New York Islanders.