Manager Buck Showalter said Castillo was taken to a hospital for tests.
"He's at the hospital," Showalter told reporters after the game. "Doctor on duty looked at him, and they decided to get some tests. They are conducting those tests right now. He had the same issue before, so [head athletic trainer] Richie [Bancells] and everybody wanted to be real cautious with it."
Caleb Joseph replaced Castillo.
Earlier this year, Castillo required a similar hospital trip. He spent time in the emergency room May 30 and landed on the disabled list with a testicular injury after a pitch that hit New York Yankees shortstop Didi Gregorius caromed and hit Castillo in the groin.