Washington won for the third time in four games and snapped New York's four-game road winning streak.
Matt Pettinger also scored for the Capitals, whose last four victories have come against teams that totalled 100 or more points last season.
Michael Nylander had a goal for the Rangers, who had at least a point in five straight games.
Washington outshot New York 17-5 in the opening period and took a 1-0 lead on Gordon's goal at 11:59. Gordon picked up a puck that skipped off defenceman Sandis Ozolinsh's stick deep in the Rangers zone, skated in on goal and scored on his third whack in front of the net.
New York's offence, unable to establish any momentum while the Rangers were killing six straight penalties, started putting some pressure in the second period with 16 shots at Capitals goalie Olaf Kolzig.
Most were routine, but Kolzig sprawled along the goal line with 7:07 remaining in the second, finally gloving the puck after a goal-mouth scramble, then swept aside Matt Cullen's drive at the left post with 3:14 left.
Ovechkin's goal made it 2-0 early in the third period. With the Rangers' Peter Prucha bearhugging him at the top of the slot, Ovechkin took a lead pass from Dainius Zubrus and somehow managed to uncork a low shot that skittered between Lundqvist's legs at 4:53.
Pettinger increased the lead with 9:06 left, scoring from between the circles after a Gordon feed sprung him for a dash along the boards.
With Zubrus off for tripping, Nylander ruined Kolzig's bid for his 34th career shutout, scoring from the left side with 1:51 left.
Notes: Capitals enforcer Donald Brashear finally picked up his first fighting major in the team's 16th game. Brashear pummelled Colton Orr 2:34 into the first period. ... Capitals rookie D John Erskine assisted on Ovechkin's goal, his first NHL point. ... Ozolinsh was briefly shaken up early in the third period when he was driven into the goal, dislodging it from its magnetic moorings, but returned to the game.