Klentak, 36, replaces Ruben Amaro Jr., who was fired in September with the Phillies on the way to a major league-worst 63-99 record and a second straight last-place finish in the National League East.
The Phillies did not immediately confirm the hiring, which was first reported by the Boston Globe.
Klentak, a Massachusetts native, is a former shortstop and baseball team captain at Dartmouth College. He began his executive career with the Colorado Rockies organization, and has previously worked with Andy MacPhail, the Phillies' new president of baseball operations, in Major League Baseball's central office and the front office of the Baltimore Orioles.
Upon parting ways with Amaro, MacPhail and Phillies part-owner John Middleton indicated they would be looking at GM candidates with strong analytical backgrounds. Tampa Bay's Chaim Bloom and Oakland's Dan Kantrovitz, Ivy League graduates in their 30s, also made it to the final round of the interview process before the Phillies decided on Klentak.
Before landing the Phillies' job, Klentak interviewed for the vacant GM position in Anaheim. The Angels went outside the organization and hired former New York Yankees assistant GM Billy Eppler to fill the role.