Tyler Kennedy and Jordan Staal scored with the man advantage in the second period to give the Penguins a 3-2 lead. Goligoski, who opened the scoring at even strength in the first, and Chris Kunitz added power-play goals in the third.
Marc-Andre Fleury, who made 20 saves, crossed his arms as his teammates gathered around him at the end of the game.
"I thought that's how you celebrate in Montreal," said Fleury, who grew up in nearby Sorel, Quebec.
Canadiens goalie Carey Price struck a similar pose after Montreal's shootout win over Pittsburgh last Thursday.
"That was a push in the back for us and we made sure we could come back and put it in his face," said Kris Letang, who got his third assist on Pittsburgh's fifth goal.
The Penguins won without Crosby in the lineup for the first time in four games since their captain was sidelined by a concussion.
"He's the best player in the game," Goligoski said. "We have to make sure we're playing the right way, maybe a little more so when he's not in the lineup. It's tough without him but if we're playing our game we can still win a lot of hockey games."
David Desharnais scored his first NHL goal to give the Canadiens a short-lived 2-1 lead early in the second. Tomas Plekanec scored his 13th of the season to tie it at 1 at 13:15 of the first, just 18 seconds after Goligoski opened the scoring with his first goal of the game.
Price stopped 31 shots for Montreal, which ended a three-game winning streak. That run began with the Canadiens goalie stopping five straight Penguins shooters in Thursday's tiebreaker.
Montreal's Benoit Pouliot was the other hero for the homeside in that game when he delivered the shootout winner in the Penguins' first game of the season without Crosby, who leads the league with 66 points despite missing the last four games.
Pouliot, whose goal gave the Canadiens a win over the Rangers one night earlier in New York, played the goat in the rematch against the Penguins.
Staal, who also had two assists, scored his first of the season with 10.9 seconds left in the second, an unassisted effort as Pittsburgh took advantage of a too-many-men call against Montreal late in the period.
Goligoski got his second of the game -- his ninth of the season -- 2:33 into the third after Pouliot was called for hooking.
Pouliot appeared to have drawn the Canadiens within one at 4:40, but referee Justin St. Pierre immediately waved off the goal and sent the Montreal left wing back to the penalty box for slashing Tyler Kennedy.
"It wasn't only Benoit, there were others who took penalties, too," Canadiens coach Jacques Martin said. "It was a lack of a discipline in the penalties and a lack of discipline in our penalty killing."
Goligoski got his third point of the game when Kunitz deflected Kris Letang's shot past Price at 11:29.
"We just weren't sharp," Montreal captain Brian Gionta said. "We took a few too many penalties and you get out of the rhythm of the game, you get out of the flow of the game, and obviously we weren't very sharp on the PK."
Penguins RW Arron Asham returned after sitting out Monday's loss in Boston because of illness. ... Canadiens LW Michael Cammalleri missed his second game in a row because of the flu. ... D Alexandre Picard was left out of Montreal's lineup. ... Martin used his timeout prior to a faceoff in Montreal's zone 6:46 into the second.