"We're going to go by committee right now," Maddon told reporters.
Maddon said he doesn't plan on appointing a new closer and that the role will be spread among several pitchers, giving him "the ability to utilize the whole bullpen." Jake McGee and Juan Carlos Oviedo are among the candidates who could get regular time in the role.
Balfour entered Monday 0-2 with a 6.46 ERA in 24 games and had nine saves in 11 opportunities. He has walked 20 batters in 23 2/3 innings.
"It just hasn't worked out," Maddon said, adding that he expects Balfour will get a chance to close some games later this season.
Balfour was one pitch away from taking the Rays to the bottom of the ninth with the game still scoreless, but Brad Miller laced a triple into the right-field corner. After a walk to Willie Bloomquist, Endy Chavez bounced a single into left field to drive in the first run.
"I think I upset the baseball gods or did something wrong," Balfour said Sunday, when he struck out the first two batters he faced before giving up four hits and two walks. "I've never given up five runs in my career in an inning and I've done it twice this year."
The Associated Press contributed to this report.