Rivera homered and drove in four runs, Erasmo Ramirez allowed one hit over five innings and the Rays beat the Yankees 6-1 on Thursday night.
After losing the series opener 11-5, the Rays battled back to win the final three games.
"We played really good baseball the last three nights," Tampa Bay manager Kevin Cash said.
Yankees starter Chase Whitley (1-2) left with two on and two outs in the second due to a right elbow injury. He was replaced by Esmil Rogers, who gave up a three-run homer on his first pitch to the light-hitting Rivera that put Tampa Bay up 4-0.
"I was looking for something in the strike zone so I could put a good swing on it," Rivera said.
Whitley will undergo imaging tests and further examination Friday.
"Obviously a concern about this," Yankees manager Joe Girardi said. "It's in that area you don't want to talk about."
Ramirez (1-1) didn't allow a hit after Jacoby Ellsbury opened the first with a single. The right-hander struck out four and walked two.
A-Rod got his 1,000th RBI with the Yankees on his 663rd career homer.
The Rays have given up two runs or fewer 19 times this season, tops in the majors.
Rivera, who entered hitting .126 with one homer and five RBIs, made it 5-0 on an RBI single off Rogers in the fourth. Rivera's career high for RBIs is five set on April 28, 2014, for San Diego against San Francisco.
"We needed that, Rene needed that," Cash said.
James Loney opened the scoring with a first-inning RBI single off Whitley, charged with three runs, two hits and two walks.
Girardi said left-hander Chris Capuano, who is coming back from a quadriceps injury and pitched six innings for Triple-A Scranton/Wilkes-Barre on Tuesday night, will take Whitley's spot in the rotation.
Whitley has been filling the spot that opened when Masahiro Tanaka was injured.
"It's frustrating but I've been dealing with (tightness) for a while," Whitley said. "Of course there's a concern but I'll know more about it tomorrow."
The announced crowd was 11,977. The four-game set drew just 44,937.
Yankees: Girardi said INF Brendan Ryan (calf) left an extended spring training game early due to heat exhaustion. ... Tanaka (right wrist and forearm) will have his second bullpen session Friday.
Andriese was optioned to Triple-A Durham after the game to make room for closer Jake McGee, who is coming back from offseason elbow surgery and was reinstated from the 15-day disabled list.
Rodriguez has 1,990 RBIs, which is two away from tying Babe Ruth for fourth place all-time since RBIs became an official stat in 1920. ... Four of his nine homers this season have come off Tampa Bay pitching. ... He has 51 homers in 212 games against the Rays. ... His solo homer stopped a stretch of 26 consecutive innings without an extra-base hit for the Yankees.
Death of Tampa Bay Rays bullpen catcher Jean Ramirez ruled a suicide
The death of Tampa Bay Rays bullpen catcher Jean Ramirez near his home in Fort Worth, Texas, has been ruled a suicide.
Brad Ausmus joins Oakland A's as bench coach for first-time manager Mark Kotsay
The A's have hired Brad Ausmus as the team's bench coach under first-time manager Mark Kotsay. Ausmus previously was the manager for the Tigers and Angels.
Buster Olney's Top 10s for 2022: Ranking MLB's best teams
With moves left to be made once the lockout ends, Buster Olney rates baseball's best teams as 2022 begins.
MLB lockout: Are the owners and players talking? Should baseball fans be worried? Here's the latest
Nearly a month and a half since the lockout began, it's time for another update on where baseball's immediate and long-term future stands.
Atlantic League ends trials with 61-foot mounds, robo-umps for balls and strikes
The Atlantic League is restoring its pitching mound to 60 feet, 6 inches from home plate and returning strike zone judgment to umpires after experimenting with the rules since 2019.
Travis Snider retires after 8 big league seasons
Travis Snider says he has retired after eight major league seasons.