The team announced that he receives a three-year contract with a team option in 2023. Girardi will be introduced at a news conference Monday.
"I'm excited for this next chapter of my career," Girardi said in a statement. "The Phillies have a strong commitment to winning from the owners to the front office to the players to the fans. It's something that I've seen up close for the last 30 years of my baseball career. I played against the great Phillies players of the early '90s from Dutch Daulton to John Kruk to Dave Hollins, and I managed against their teams during the incredible run they had from 2008 to 2011. To have my name now associated with this great franchise is something that I couldn't be happier about."
Girardi, who will wear No. 25, replaces Gabe Kapler, who was fired after two disappointing seasons. The Phillies went into last season with high hopes after signing Bryce Harper but still missed the playoffs, finishing at 81-81 and in fourth place in the National League East.
The Phillies haven't had a winning season since 2011 when they finished a run of five straight NL East titles, two pennants and one World Series championship under Charlie Manuel
Girardi also interviewed with the New York Mets, who still have an opening, and the Chicago Cubs, who went with David Ross. The Phillies reportedly also talked to former major league managers Buck Showalter and Dusty Baker.
"Joe brings high character and a tremendous work ethic to his position, and he is a proven winner," Phillies general manager Matt Klentak said. "I look forward to working with him and I believe that he is the right manager to lead our team to the next level."
Phillies CEO John Middleton fired Kapler, overruling Klentak and team president Andy MacPhail. Both men face a critical offseason trying to address the team's needs. The Phillies sorely lack pitching, especially starters.
"Matt did a great job running the search, culminating with the three exceptional candidates we interviewed," Middleton said. "Ultimately, we all agree that Joe is the right person to lead our team, and I am excited to welcome him to the Phillies."
The 55-year-old Girardi managed 10 years with the Yankees from 2008 to 2017, winning a World Series championship in 2009. He had a 910-710 record over that span.
"He's going to represent their franchise well," Yankees general manager Brian Cashman said Thursday. "He's been a winner his entire career, so I expect nothing but the same to continue there in Philadelphia. I wish him luck. I'd rather it not be in the American League East. I guess that's the biggest compliment I could give."
Girardi began his managerial career with the Marlins in 2006 but was fired after one season despite going 78-84 and being voted NL Manager of the Year -- the only manager in the history of the award to win it with a losing record. He succeeded Joe Torre after the 2007 season and spent a decade in pinstripes, leading New York to its 27th World Series title, beating the Phillies in six games in 2009.
As a player, Girardi was a catcher in the big leagues from 1989 to 2003 and hit. 267. He won the World Series with the Yankees in 1996, 1998 and 1999 and was an All-Star for the Chicago Cubs in 2000.
Information from The Associated Press was used in this report.