The veteran trio played at the 2008 Beijing Games under Goorjian, with the legendary coach returning 12 years later with the men's program still in search of an elusive medal.
"I thought I was the right guy for the position. These guys have been working for 12 years. Do they want somebody to come in that they have no rapport with at all?" Goorjian told ESPN.
Bogut would retire prior to the games, leaving the 'core-five' of Mills, Ingles, Matthew Dellavedova and Aron Baynes missing a key piece as they entered what would likely be their last campaign together as a group.
"Bogut didn't play, Aron Baynes obviously got hurt but before that in Toronto he virtually didn't play all season," Goorjian said. "Then you've got Delly who's in that leadership group and didn't play. When we got to that camp, you could see, they're pros but they were underdone."
While Mills, Ingles and Dellavedova remained major cogs in the Boomers' success, Tokyo doubled as a transition to a bright future for the national program, with Matisse Thybulle, Dante Exum and Josh Green forming the nucleus of the next wave.
One player who didn't make the final twelve was Josh Giddey, who was later selected by the Oklahoma City Thunder with the sixth pick in the NBA Draft. Goorjian recalled agonising over the decision to leave Giddey on the outer for Tokyo.
"The toughest part is always selection. He had a lot on his plate at the time. It's such a fine line, there was a lot on his plate during the camp," he said.
"People talk about Nathan Sobey, it wasn't that. It came down to Josh Green or Josh Giddey. They're both young, they both have a huge future moving forward. When we played those four days leading up, Josh Green had a better camp, it's as simple as that.
"When he was told he wasn't in the 12, it had to be very disappointing for him and he handled it. He was disappointed no doubt, but he was in for the next camp and had moved forward. He was more locked in; he had a great three or four days training and then got an opportunity against Nigeria to play big minutes."
Giddey saw a team-high 24 minutes of action in the 108-69 win over Nigeria, tallying 14 points to go along with four rebounds, three assists and a block. It was an eye-catching performance with impeccable timing on the eve of the draft.
"(Nigeria) had 8-10 NBA players, they are very athletic, and they came after him and he shined. I was really happy that he got touched by our leadership group with the Boomers," he said. "He's a huge, huge piece moving forward when you look at Josh Green, you look at Matisse Thybulle and you throw Josh Giddey in there."
Thybulle, who quickly became a fan favourite was often tasked with the top defensive assignment, replaced Dellavedova in the starting line-up for the bronze medal game to guard Slovenian superstar Luka Doncic. Displaying dazzling perimeter defence and highlight reel plays above the rim on offence, Goorjian praised Thybulle's ability to acclimatise in the Boomers system on and off the court.
For Exum, the formula was simple. Stay healthy. Yet, the 26-year-old would exceed all tempered expectations, improving with each game to become an integral part of the rotation.
"Here's a guy who doesn't have his NBA contract, he's on a limb. He's not set for the following year; he's been injured, and he was totally committed to this. He was fearless," Goorjian said of Exum. "He practiced, he went in that lane, he put his shoulder down and went hard, he didn't protect himself, he put his body on the line.
"I just have so much admiration for him. He is healthy now, I'm just hoping like Jock Landale, he can get a nice contract out of this situation from the Boomers because he really is a pro."
It would be premature to suggest Mills and Ingles won't make Paris 2024, they likely will, but importantly it feels like the program is ready to transition to the next wave as the pursuit for gold continues.
"It's an aspect of this that was really strong with Patty Mills and the leadership group," Goorjian said. "It was important to them that when they move this team doesn't go back culturally or performance wise."
While looking to the future is a tantalising exercise, the standard has now been set, with Goorjian's completing the job he signed up for, delivering a historic medal while helping to foster the next wave of Boomers on the road to Paris.
"If we had lost to Slovenia I had nothing for that. It would have broken my heart. Those two guys, what they have put into this, this was it. There's no it'll be ok or pass the buck to someone else, it sits with me. They put faith in me to deliver," he said. "They're the ones that performed, it's about them but I felt unbelievable pressure leading into these Olympics and unbelievable pressure when we got to that point.
"We won. The embrace with Patty was sincere. The embrace with Joe was sincere. It was like nothing I've ever felt in my life before and I know I'm going to feel it for the rest of my life. When I see those guys which I know I will, you'll always have that."