Niemann was struck on the ankle by Adam Lind's drive with runners at first and second in the first inning. He recovered to make the out, but walked gingerly around the mound and was checked by the trainer before finishing the inning.
Niemann came out and warmed up to start the second, but was removed after a conference on the mound with the trainer and manager Joe Maddon. Left-hander Cesar Ramos came on and earned the win while Niemann went to the hospital for X-rays.
"It's not easy to lose a pitcher of his caliber," Maddon said.
The Rays did not put a timetable on Niemann's return, but the fracture will likely take four to six weeks to heal.
Maddon said the Rays will call up a reliever Tuesday to help out until Niemann's spot in the rotation comes up again, adding that he'd prefer not to move Wade Davis back to the rotation.
Niemann missed more than six weeks last season with a sore lower back, going 11-4 after his return in late June.
Information from The Associated Press was used in this report.