Vitaly Vishnevski, Niclas Havelid and Bobby Holik also scored for Atlanta. Erik Christensen and Ryan Whitney tallied for Pittsburgh.
The Thrashers have had two five-game winning streaks this month, the longest in the franchise's short history. Pittsburgh has been streaking for a month, losing four, winning four and now losing five.
Kozlov got the game-winner, his 15th of the season, about seven minutes into the third period on a wrist shot from along the goal line that got past goalie Marc-Andre Fleury inside the near post. Kozlov has two goals and four assists in three games against Pittsburgh this season.
The Penguins took a 1-0 lead late in the first period on Christensen's sixth goal as he took a rebound off the post and shoved it toward the net, where it deflected off both goalie Kari Lehtonen's skates and stick before sliding across the goal line.
Lehtonen stopped 35 of 37 shots and has turned away 162 of 173 during the current winning streak.
Atlanta, coming off a 2-1 victory over Tampa Bay on Tuesday on Jim Slater's goal in the final 10 seconds, came back in the second period as Marian Hossa set up goals about eight minutes apart by Vishnevski, his third, and Havelid, his first since March 1. Hossa assisted on three of Atlanta's four goals.
Pittsburgh's Sidney Crosby, the NHL's leading scorer, went scoreless for the second night in a row after scoring in his previous 10 games. The Penguins are 0-28-1 the last two seasons when Crosby doesn't score.
Holik added an empty-net goal off Hossa's pass with 19 seconds remaining.