Archer, though, settled down and Rays fought back to get their second victory in eight games with the Yankees this season.
Evan Longoria hit a tiebreaking sacrifice fly during a two-run eighth inning and the Rays rallied late to beat the Yankees 4-2 on Tuesday night.
"The way the game started I don't know if we saw an ending like that," Rays manager Kevin Cash said. "It's a credit to Arch, keeping his cool, keeping us right there in the ballgame. Extremely impressive outing."
Archer gave up two runs and seven hits in seven innings. He retired 15 of 16 over his final five innings.
"Honestly not thinking about the situation," Archer said. "Just thinking about executing my pitches. I just stayed in the present moment and was able to work through that first."
Kevin Kiermaier drew a one-out walk in the eighth from Nathan Eovaldi (3-1) and went to third on Steven Souza Jr.'s single. Dellin Betances entered and gave up Longoria's sac fly. Souza scored on Betances' wild pitch to make it 4-2.
"Dellin wasn't fully rested, and I was trying to stay away from using him but we got in a situation where we needed a strikeout and that's why I put him in," Yankees manager Joe Girardi said.
Tampa Bay tied it at 2 on David DeJesus' two-run single in the seventh.
Eovaldi gave up four runs and six hits over 7 1/3 innings.
Ellsbury went 2 for 3 against Archer, and has 12 hits in 18 at-bats off him.
Alex Rodriguez went 0 for 3 with a walk. A-Rod still has reached base safely in 12 of 23 plate appearances this year in five road games against the Rays, with three homers, five RBIs and eight runs scored.
The announced attendance was 10,417, which marks the second straight day that the turnout was the lowest for a Rays' home game against the Yankees. Monday's series opening crowd was 10,619.
Yankees: Rodriguez was in the lineup after having some left hamstring soreness in Monday night's 11-5 win over the Rays. "Worry about it a little bit, but we'll manage it," Girardi said. ... LHP Chris Capuano (quadriceps) allowed two runs and seven hits in six innings for Triple-A Scranton/Wilkes-Barre.
Rays: LHP Matt Moore (elbow surgery) will have his third batting practice session Wednesday. ... Closer Jake McGee (elbow surgery) will pitch for Triple-A Durham Wednesday and could return this weekend.
Girardi and several players, including Rodriguez and Ellsbury, talked on the phone from the manager's office with Yogi Berra on his 90th birthday Tuesday.
"He was happy to talk to all of us," Girardi aid. "He just told us to keep it going. It was good."
Yankees INF Stephen Drew took grounders at third base before game and will likely get time there backing up Chase Headley. Rodriguez has been the primary fill-in at third, starting two games at the position. Drew has started at second base and shortstop this year.
Death of Tampa Bay Rays bullpen catcher Jean Ramirez ruled a suicide
The death of Tampa Bay Rays bullpen catcher Jean Ramirez near his home in Fort Worth, Texas, has been ruled a suicide.
Brad Ausmus joins Oakland A's as bench coach for first-time manager Mark Kotsay
The A's have hired Brad Ausmus as the team's bench coach under first-time manager Mark Kotsay. Ausmus previously was the manager for the Tigers and Angels.
Buster Olney's Top 10s for 2022: Ranking MLB's best teams
With moves left to be made once the lockout ends, Buster Olney rates baseball's best teams as 2022 begins.
MLB lockout: Are the owners and players talking? Should baseball fans be worried? Here's the latest
Nearly a month and a half since the lockout began, it's time for another update on where baseball's immediate and long-term future stands.
Atlantic League ends trials with 61-foot mounds, robo-umps for balls and strikes
The Atlantic League is restoring its pitching mound to 60 feet, 6 inches from home plate and returning strike zone judgment to umpires after experimenting with the rules since 2019.
Travis Snider retires after 8 big league seasons
Travis Snider says he has retired after eight major league seasons.