Owner Jim Irsay announced the moves Monday, one day after the team finished the year 2-14 without Manning. Coach Jim Caldwell was retained, but Irsay said that evaluation was still ongoing.
Strangely enough, Manning, whose season-ending neck injury and subsequent surgeries sent the Colts down the path to the No. 1 pick, was meeting with Bill Polian when Polian was summoned away for a meeting with Irsay on Monday afternoon.
"I was stunned, I was surprised, I was saddened and just disappointed that it all happened this way," Manning told ESPN senior NFL analyst Chris Mortensen by telephone. "I was actually meeting with Bill in the training room and we were talking about my rehab schedule, what I could and couldn't do in the facility under the new (collective bargaining agreement) and my checkpoints when he was summoned to see Irsay. He actually chuckled and said, 'We'll see.' "
Manning said he and Polian later had a very emotional meeting when the quarterback learned that the man who drafted him with the No. 1 pick in 1998 had been let go.
"Bill and I and Chris Polian has been here the whole time, too, had a great ride with tremendous highs and it makes me very sad that this ends on such a negative note," said Manning. "I knew when the Colts hired Bill with his track record in Buffalo and Carolina, that I would be very happy to be a part of an organization he ran. And that proved to be true. I knew he would hire the best coaches, provide the best possible surroundings, give me great teammates and he did that and I will be forever indebted to him."
Since making Manning the top overall selection, the Colts have celebrated 141 regular-season wins, 11 playoff appearances, six division titles, two AFC championships and one Super Bowl title. During that span, there have been only three head coaches, two offensive coordinators and one family making the personnel moves.
"I used to think nobody hated losing more than I did or was more miserable after a loss. I decided it was a tie between me and Bill," Manning told Mortensen. "And I stay in the building pretty late and he was always here working, too. Personally, my goodness, so many thoughts. He was one of the last guys to leave my wedding with Ashley (his wife). This is very, very tough."
But without its franchise quarterback and star player at the helm this season, the losses came in bunches for Indianapolis.
"I felt that it was time for a change, that there was a need for a change," Irsay said Monday, flanked by his three daughters. "Bill had entered a role where he was less involved, but still quite a bit involved because of the lockout and Peyton's injury and the losing streak. He was around a lot more than he probably anticipated or I did. But it really was a question about both situations. I thought that it was time to change the personnel department on the football side of things that wasn't involved with the coaching."
Bill Polian declined to comment when The Associated Press contacted him by phone, but did issue a statement through the team.
"I'm grateful for all the support the fans have shown us in good times and bad," Polian said. "Indianapolis has been a wonderful place to live and work. Most of all, I would like to thank the players coaches and staff who have played the pivotal role in this magnificent journey. I will miss them all."
In the end, it was the Polians' inability to find a suitable backup for Manning that doomed the Colts.
With Manning struggling to come back from May surgery on his neck, Indy brought 17-year veteran Kerry Collins out of retirement in late August with a $4 million contract. He didn't last a month.
Collins was replaced by Curtis Painter, who started well but struggled badly after Bill Polian said he felt "vindicated" by Painter's success.
The Colts lost their first 13 games, finished with the second-fewest victories since the team moved to Indianapolis in 1984 and wound up with the No. 1 draft pick.
Bill Polian repeatedly said he should be blamed if there was a talent deficiency.
Chris Polian, who was named the team's GM four years ago but didn't start making the day-to-day decisions until this season, got caught in the crossfire. He was seen leaving the team complex Monday afternoon, just about the time word leaked of the firings.
"To think that just less than less than two years ago we were getting ready to go to Miami and play in a Super Bowl, and we were cleaning off the confetti from the turf in Lucas Oil Stadium, it has been a very sharp decline," Irsay said. "Even after going 10-6 last year, if people had said you would have the No. 1 draft pick and go 2-14, nobody would have believed it."
Stanford quarterback Andrew Luck is the presumed No. 1 pick in April's draft and there has been question as to whether Manning and Luck could co-exist on the same roster.
For his part, Manning said Sunday, "I think I can co-exist with any player I've ever played with. I think I've always been a good teammate in that way."
However, questions remain about Manning's health and the viability of taking a quarterback with the No. 1 pick. Manning, who signed a five-year, $90 million deal in July, never played after undergoing his third neck surgery in 19 months, a fusion of two vertebrae to fix a damaged nerve that was causing weakness in his throwing arm.
He did, however, start throwing to teammates in mid-December and received good reviews from both Bill Polian, who was watching, and running back Joseph Addai, who was catching balls. Doctors familiar with the procedure who did not treat Manning say he should return to a high level of play now that the fusion has healed.
If he is healthy, Irsay has promised to bring back Manning regardless of the $28 million bonus he is owed in March.
Manning told Mortensen on Monday that he has not yet spoken to Irsay about his future with the franchise.
"That's what Bill and I were talking about when he was pulled away -- I asked him about my physical checkpoints," the four-time league MVP said. "I wanted to know if he thought Feb. 7 or 8 (was) a good time for the next one and he told me just make it March 1st. Honestly, I don't know. my rehab is a marathon. Some days are real good, some days are just OK and some days aren't so good. I'm going to just keep grinding but now is the time to pay tribute to Bill and not talk about me."
While Manning is the face of the franchise, the Polians' dismissal caught many Colts players off-guard.
"Shock, man, just shock," safety Antoine Bethea said after finding out the Polians were gone. "I knew there were going to be some things happening, but I didn't expect them to come so soon."
Irsay has not established a timeline for finding a new general manager, though he said he would start contacting other GMs and teams Monday night.
No decision on Caldwell is expected to be made until after the GM is hired, though players have overwhelmingly supported him.
"One thing about Jim is that he's a stand-up guy, and guys respect that," cornerback Jerraud Powers said. "He's a great coach, a good guy, and it was unfortunate situation this year. But he never lost the locker room."
Irsay said there's a good chance Caldwell will be back, but for how long remains to be seen.
A team source told ESPN's Ed Werder that Caldwell met with his staff following the firings and informed them they were being retained but there were no guarantees at this time. According to the source, those involved in the talk understood Caldwell's words to to mean the Colts' next general manager would decide their fate.
Besides deciding Caldwell's future, whoever replaces Polian faces an abundance of decisions in a major rebuilding effort.
If Manning returns, Irsay might be more likely to bring back former Pro Bowlers Robert Mathis, Jeff Saturday and Reggie Wayne, all of whom are now in their 30s, and keep other key high-priced cogs in the Colts' success such as Brackett and safety Melvin Bullitt.
But the Polians will no longer be making those decisions.
"Bill and I had a chance to have a second meeting and to have some tears and a hug, and that was good," Irsay said. "It's extremely difficult. Walking out of the locker room in Jacksonville that was a hard walk, and it's hard because my affection for them is deep."
Information from ESPN senior NFL analyst Chris Mortensen, ESPN's Ed Werder and The Associated Press contributed to this report.