J.P. Vigier also scored and Kari Lehtonen turned aside 18 shots in the Thrashers' win, their sixth in the first eight games.
Atlanta has picked up 13 of a possible 16 points, with its only regulation loss coming on a last-second goal by defending Stanley Cup champion Carolina.
Kovalchuk, whose lone goal coming into the night had been an overtime winner, finally got one in regulation early in the third. Last season's 52-goal scorer iced the win with three seconds remaining with an empty-netter, assisted by Hossa.
Hossa broke a scoreless tie 3 1/2 minutes into the second after being awarded a penalty shot. He set up the play by stealing the puck just inside the blue line and getting hooked from behind by Ruslan Salei.
With another crack at Alex Auld, Hossa drifted leisurely to his right and fired the puck past the goalie's right pad after luring him across the crease. It was the second penalty-shot score of Hossa's career and his team-leading eighth goal of the season.
Kovalchuk gave the Thrashers a 3-1 lead with a big assist from Bobby Holik, who powered past Bryan Allen along the boards and spotted Kovalchuk drifting alone in between the faceoff circles.
While Hossa and Kovalchuk are expected to score for the Thrashers, their other goal came from an unlikely source.
With the checking line on, Eric Boulton picked off the puck at centre ice and bulled his way toward the net. Auld stopped the shot, but fell over backward without controlling the rebound. Vigier knocked the puck under Auld's right leg just 1:04 after Hossa's goal.
It was Vigier's first point of the season, much less goal.
Chris Gratton and Joe Nieuwendyk scored for the Panthers, who also knocked a couple of shots off the post.
After Atlanta jumped to a 2-0 lead, Gratton scored with 1:43 left in the second after Juraj Kolnik hustled for a puck sliding wide of the net and flipped it to his teammate all alone in front of the net.
Just seconds after Florida clanked its second one off the post, Allen fired a slap shot from the point that ricocheted perfectly off the boards behind the net to Stephen Weiss at the opposite post.
He redirected the puck to Nieuwendyk, who one-timed it past Lehtonen for his third goal of the season.
Notes: Florida RW Todd Bertuzzi missed his second straight game because of back spasms. ... Thrashers D Steve McCarthy took a shot off his left arm less than two minutes into the game and hobbled to the bench, but quickly returned after receiving treatment in the tunnel. ... Atlanta played the 500th game in team history.