The Yankees announced the move two days after German struggled for a third consecutive start in Friday's 5-2 loss to the Indians.
German, 26, who was tested at Progressive Field on Saturday by Indians team physicians after cutting short a between-starts throwing session, will undergo an MRI on Monday.
"We're hoping that when we get back to New York and do some tests that it is something that's minor and doesn't hold him back for long," Yankees manager Aaron Boone said.
The Dominican Republic-born German, speaking through an interpreter, told reporters Sunday that he first started feeling the discomfort in his hip during his start two Sundays ago at Kansas City. He couldn't pinpoint the exact moment he was hurt, but said he began feeling badly after that game.
During that outing, German allowed seven hits and nine runs in five innings. He also gave up four home runs en route to earning a no-decision after the Yankees lost in extra innings.
"It's the kind of injury that as you progress through the game, it gets worse," German said. "The beginning, not so much, but toward the later innings it adds on and it starts taking away strength from you from executing the pitches and commanding the pitches.
"I've been able to manage it in between starts. It's just that after the outing [Friday], it was hurting more than usual."
The right-hander allowed four runs in six innings in Friday's start. It was in the second inning when he said his hip really began bothering him. Across his past three starts, German has an 8.59 ERA. He has also given up seven home runs in those outings, compared with five through his first 10 appearances of the season.
Before his recent struggles, German had been a stabilizing force in New York's injury-plagued starting rotation. He is 9-2 with a 3.86 ERA in 13 games -- 12 starts -- this season and is tied with Justin Verlander, Hyun-Jin Ryu and Lucas Giolito for the major league lead in wins.
"It's a little frustrating," German said. "This is exactly what I was trying to prevent, trying to work on this and trying to prevent it from becoming a bigger thing. And now the IL is going to stop (me) from helping the team, and pitching, which is what I want to do."
German said he didn't tell the Yankees about his injury until Saturday because he thought it was something that he could manage and pitch through.
"In between starts it would kind of go away," German said. "So I didn't think it was something that would put me out of the rotation. I thought it was something I would deal with and get over it."
Until he signed with the Atlanta Braves on Thursday, left-handed starter Dallas Keuchel had been in talks with the Yankees about potentially signing with them as a free agent. Keuchel and the Yankees were reportedly less than $2 million apart on a potential offer.
Keuchel could have added some depth to a thin Yankees staff.
"We always want our players to treat whatever's going on with them if there's any kind of issues," Boone said. "But also understand major league sports, there's little things that you deal with that to you as a player is not that big of a thing that you know you're going to have to deal with when you get out there."
The Yankees recalled left-hander Stephen Tarpley from Triple-A Scranton/Wilkes-Barre to replace German on the 25-man roster. He got the save Sunday in the Yankees' 7-6 victory over the Indians in 10 innings.
German, 26, becomes the latest Yankees starting pitcher to spend time on the IL. Luis Severino has not pitched yet this season because of a shoulder injury, while James Paxton and CC Sabathia also have had IL stints over the first two months of the season.
Once the Yankees get past the Subway Series with the New York Mets on Monday and Tuesday, they'll have a better idea of how to manage their pitching staff in the coming days. It's likely they will go with another bullpen day whenever German's spot in the rotation arrives next.