Crosby fired a pass from the right-wing boards, while Rangers captain Jaromir Jagr sat in the penalty box, that struck someone in front and ricocheted in. Earlier the 19-year-old phenom set up three goals for Pittsburgh, which had six goals on 22 shots.
Jagr was left hanging his head while Crosby happily skated across the ice and looked up at the board to see how timely his goal was.
The Penguins went 1-1 at home to start the season, each a shutout, before winning their first road game. Pittsburgh finished last in the Eastern Conference last season with only 10 wins away from the Steel City.
Jordan Staal, the No. 2 pick in this year's draft, and Kris Letang each scored their first NHL goals for Pittsburgh, while Ryan Whitney scored his first two of the season and Michel Ouellet added one.
Marc-Andre Fleury allowed goals to Jagr, Matt Cullen, Adam Hall, Michael Nylander - each scoring their second of the season - and Brendan Shanahan but still made 37 saves against the Rangers, who recorded three goals on 18 shots in the second period.
After a scoreless first period, the teams combined for six goals in the second with Pittsburgh getting three on only seven shots.
Whitney gave the Penguins a 5-4 lead 1:01 into the third, again with Jagr in the box. Nylander tied it at 6:09 with another power-play goal, this time on a 5-on-3.
Whitney scored again 53 seconds later, but Shanahan redirected a pass from Jagr in front at 11:12 to make it 5-5 with his 602nd goal, 14th-best in NHL history.
Neither team led by more than one goal and each found the touch on the power play. Pittsburgh finished 4-for-9, while New York was 3-for-9.
Before the Rangers' struggling power play broke out it went from unproductive to detrimental on its fourth opportunity.
Staal broke up Jagr's pass at the Pittsburgh blue-line and outraced defenceman Michal Rozsival up the ice on a breakaway. He made a series of forehand-to-backhand shifts and scored against Henrik Lundqvist 3:23 into the frame.
Jagr made up for his turnover 1:22 later, taking advantage of a 4-on-4 situation. It took only 1:50 more for Cullen to give New York its first lead of the night, 2-1.
Cullen, a teammate of Staal's brother Eric last season with the Stanley Cup champion Carolina Hurricanes, rushed the puck through the Penguins zone and squeezed a shot through Fleury's pads at 6:35.
Letang tied it again with Pittsburgh's first power-play goal of the season after 11 failed chances. The Penguins gained a 5-on-3 advantage and needed 20 seconds to make the Rangers pay.
Hall snapped the Rangers' power-play drought with 4:52 remaining in the period, giving New York their second man-advantage goal in 23 chances and a 3-2 lead.
Ouellet got Pittsburgh even 47 seconds later with his second of the season, another power-play goal.
Notes: Rangers coach Tom Renney inserted physical RW Colton Orr into the lineup for the first time this season in place of rookie LW Nigel Dawes. ... Penguins LW Nils Ekman played despite a hamstring injury. ... The Rangers selected another Staal brother, defenceman Marc, with the 12th overall pick in the 2005 draft.