Patrice Bergeron scored the winner with 6:28 remaining when he redirected a slap shot from Brad Stuart past Senators goalie Martin Gerber. Petr Tenkrat also scored for the Bruins.
P.J. Axelsson had two assists for Boston, which allowed fewer than five goals for the first time in five games.
Chris Neil's second of the night tied the game 3-3 when he redirected a shot from Peter Schaefer past Bruins goalie Tim Thomas 1:25 into the third period.
Neil had cut the Bruins' lead to 3-2 with 53 seconds left in the second period by beating Thomas with a backhander. Jason Spezza also scored for the Senators.
With the Bruins ahead 2-1, Murray scored his 300th career NHL goal, wristing a shot past Gerber with 2:36 remaining in the second period.
Murray gave the Bruins a 1-0 lead when he tipped in a rebound from a Zdeno Chara slap shot 23 seconds into the second period.
Spezza's goal was his eighth and he leads Ottawa with 21 points.
Schaefer had two assists for Ottawa, which went 1-3 on its four-game road trip.
Tenkrat scored his first goal of the season for Boston after being recalled from Providence earlier in the day.
Notes: Ottawa opened its all-time series against Boston with 17 consecutive losses and a 22-game winless streak. ... Murray is in his 17th season and leads the Bruins with eight goals. ... Marc Savard has 12 points in the last six games. ... Spezza's power-play goal was Ottawa's fifth in 48 opportunities. ... Boston won its first matchup with Ottawa this season when Zdeno Chara scored late in the third period for a 2-1 victory.