"That's the best feeling in all of sports, especially after a night like tonight," he said. "Long game, I know everyone's tired on both sides."
Alexei Ramirez also went deep, and the White Sox came away with the win after wiping out an early three-run deficit and then rallying again in a game that lasted 3 hours, 52 minutes.
Robertson got the final out in the eighth with two on and the Yankees up 5-4, but he couldn't hold it in the ninth.
Dayan Viciedo led off with a single, and Dunn then won it when he drove an 0-2 pitch well beyond the wall in right-center for his eighth home run. It was the 10th time he has ended a game by homering, and the first for him since July 4, 2013, against Baltimore.
"Really, I wasn't looking for anything," Dunn said. "It's 0-2. I figured he was probably trying to get a ground ball. First of all, I didn't want to chase a bad pitch. He's got two really, really good pitches. You just try to swing at a strike."
Dunn's drive made a winner of Daniel Webb (4-0), who pitched 1 1/3 scoreless innings, and gave Robertson (0-1) his first blown save in 10 chances. It also put the White Sox at .500 at 25-25.
Derek Jeter singled twice, walked and scored while tying former White Sox great Luis Aparicio for second all time in games played at shortstop with 2,583.
"I guess it is ironic that we're here," Jeter said. "It's hard to believe when you think about the history of the game that there is only one guy that's played more games. It's something I'm proud of. I take pride in doing my job and being available to play every day, and to have your name in company with him is pretty special."
Ramirez and Adam Eaton each had three hits for Chicago, and the White Sox won for the fourth time in five games.
The Yankees looked as though they might be on their way to an easy win when Brian McCann hit a three-run homer off Hector Noesi in the first. It was a good start for New York after managing just one baserunner in six innings against Chris Sale in Thursday's loss, but things quickly took a difficult turn.
The Yankees gave up unearned runs in the first and fourth, and Chicago took a 4-3 lead on Ramirez's two-run homer off Hiroki Kuroda in the fifth.
Roberts then scored when Petricka bounced a pitch to Ellsbury, who then drove in Soriano with a sacrifice fly to center to give the Yankees a 5-4 lead.
Kuroda allowed four runs (two earned) and eight hits in 4 2/3 innings. He is 0-7 in 11 road starts since a win at Texas on July 25.
Noesi, a former Yankees prospect who got dealt to Seattle in the Michael Pineda trade in January 2012, allowed three runs and four hits. He walked five but was in line for his first win since he beat Minnesota while with the Mariners on May 6, 2012, before the bullpen gave away the lead.
"I don't think about that," he said. "That's bad for me. This is a new year. I'm trying to get my first win and I will get it."
Jeter and Aparicio trail Omar Vizquel (2,709) in games played at shortstop. ... The White Sox plan to honor Jeter before Sunday's game with a video tribute and commemorative gifts. ... Roberts came up limping after he fouled a pitch off his knee in the fourth. ... Sale had no setbacks after a dominant return to the rotation on Thursday. Manager Robin Ventura said the left-hander was feeling "great" a day after retiring 18 of 19 batters. Sale had missed nearly five weeks because of a flexor strain in his pitching arm.
Giancarlo Stanton returns from quad injury as Yankees' DH for Game 5
Giancarlo Stanton, who has been out since Game 1 of the ALCS with a quad injury, will return to the Yankees' lineup as the designated hitter for Game 5 on Friday.
CC Sabathia's career done as Yankees add Ben Heller to ALCS roster
CC Sabathia was removed from the Yankees' ALCS roster, meaning his career in pinstripes is over. He was replaced on the roster by right-hander Ben Heller.
Braves' Freeman has arthroscopic elbow surgery
Braves 1B Freddie Freeman is expected to be back for the start of spring training after arthroscopic elbow surgery on Wednesday.
Bryce Harper: No jealousy to see Nationals in World Series
Don't expect Bryce Harper to be envious about his former team, the Nationals, reaching the World Series. "I'm so happy for them," he told The Athletic.
Stephen A.: Nats are finding life is better without Harper
Stephen A. Smith and Max Kellerman react to the Nationals reaching the World Series after Bryce Harper left for the Phillies.
How the new dead ball is faking out players, fans and even cameramen
On the field and in the stands, how players and others react to MLB's newly juiceless baseballs off the bat tells a story.