Terms were not disclosed, but a source told ESPN NFL Insider Adam Caplan that Herremans received a one-year deal that could be worth as much as $3.5 million.
Herremans will try to bring some stability to the Colts' offensive line. Injuries and poor play caused Indianapolis to use 11 different starting lineups along the offensive line last season. Herremans has started at every offensive line position except center in his career.
The 32-year-old right guard had recently been released after 10 seasons with the Eagles.
"It was a different experience for me going through trying to find a whole new team. Indy seems like a good fit for me and a good opportunity to start," he told ESPN's Josina Anderson. "I want to compete for a Super Bowl. I've been close before but never quite close enough.
"I had a good time speaking with their offensive line coach and their general manager Ryan Grigson. I know him from when he was in Philadelphia. It was good to reconnect. I have a good feeling about it. I'm really excited to get started."
Grigson is the Eagles' former director of player personnel.
Originally a fourth-round pick in the 2005 draft out of Saginaw Valley State, Herremans was due to make $4 million in 2015 before his release by Philadelphia.
He was, along with outside linebacker Trent Cole, the longest-tenured player on the Eagles' roster. Both players were released this offseason.
Herremans missed the second half of the 2014 season after tearing a biceps muscle during the Eagles' Oct. 26 game in Arizona. He was placed on injured reserve.
Before that, Herremans started seven games at right guard and one game at right tackle for the Eagles.
ESPN.com Colts reporter Mike Wells and Eagles reporter Phil Sheridan contributed to this report.