After the Crimson failed to earn a bid in 2011, Swegan kept on following the team and sought out head coach Tommy Amaker again last month.
And when Harvard won the Ivy League title outright with Penn's loss at Princeton on March 6, the 86-year-old Swegan suddenly became in demand.
Now Harvard is giving back to him, 66 years after he scored the final basket in a Final Four loss to Ohio State.
Swegan told ESPN.com on Tuesday night that a collection of three or four Harvard alumni, who call themselves "Friends of Harvard Basketball,'' are going to fly Swegan to Albuquerque for the NCAA tournament second-round game against Vanderbilt on Thursday.
"There are three or four guys who are underwriting the whole thing,'' said Swegan. "I'm sort of their guest. I'm going, and I'll stay the whole time.''
Swegan, who lives near Youngstown, Ohio, said a recent ESPN.com story allowed him to reconnect with the only other three living members of the '46 team.
"I reconnected with them,'' said Swegan. "The rest are deceased. The ones that are still alive, we've all connected by phone but none of them can make it. I'll be the only one from that team there.''
Swegan said he was hoping that Harvard would play closer to his home -- in Pittsburgh, Columbus, Louisville or Nashville. Harvard is in the East Region, but in a pod in Albuquerque. If Harvard upset Vanderbilt in a first-round game, it would play the winner of Wisconsin-Montana in the third round.
The Crimson's arrival in the NCAA tournament was celebrated in Washington, too. President Barack Obama went to Harvard Law School and has strong ties to the school with a close friend's son on the squad.
"But I think beating Vanderbilt, after they have been on this run [SEC tournament title] is just too much of a stretch,'' Obama said. "But I'll be rooting for Harvard.''
Arne Duncan, the U.S. Secretary of Education, played at Harvard in the mid-1980s.
"It's impossible to be first in anything at Harvard,'' said Duncan. "They've done it.''
Duncan said the difference with Harvard is that there is now commitment to the institution.
"The president [at Harvard] is 100 percent behind Coach Amaker,'' Duncan said. "There is now prestige and status with the program. That wasn't always the case. I don't have answer why it wasn't there in the past. The building blocks are now there to lead to success.''
Duncan said part of the reason he chose Harvard was the hope that he would be on the team that got the Crimson to the NCAA tournament. But that didn't happen.
"My dream was to be on the first team that went in a long time and go to the tournament,'' Duncan said. "But it never happened. The commitment has always been there for football and ice hockey.''
And now the Crimson have added a steal in recruiting for 2013 with power forward Zena Edosomwan out of North Hollywood, Calif. Getting the 6-foot-8 forward to commit was viewed as a major coup.
"I couldn't be more thrilled for Coach Amaker and the players,'' Duncan said. "They've worked so hard for this and came about as close as you can last year. Princeton hit a great shot [to beat Harvard in a one-game playoff and clinch the Ivy League's automatic bid last season] and broke their heart.''
But Harvard came back this season and won 26 games (12 in the Ivy) to earn a No. 12 seed.
And Swegan will be there to watch how it unfolds.
"I'm looking forward to this,'' Swegan said. "There are going to be so many good teams there.''
• Syracuse did fine without Fab Melo in a win at Cincinnati in late January. More C.J. Fair may make the Orange quicker and even more versatile offensively. The shot-blocking will fade, but the offense won't without Melo.
• I'm not surprised UCLA coach Ben Howland is coming back. The recruiting class is too good next season. And athletic director Dan Guerrero hired Howland. He has to give him one more year to get the Bruins back on track.
• It wasn't a shocker to see South Carolina's Darrin Horn and SMU's Matt Doherty were fired Tuesday. Former Boston College coach Al Skinner has a home in South Carolina and is trying to make a play for the Gamecocks gig. He could do for South Carolina what Bobby Cremins did for College of Charleston. Of course, Wichita State's Gregg Marshall is likely first up on the docket. But Marshall may feel he can get a better job with the Shockers' success. SMU should go get my ESPN colleague Fran Fraschilla. But if the Mustangs choose not to or if Fraschilla isn't interested, look to Marquette associate head coach Tony Benford to make a strong play. He played in the old Southwest Conference at Texas Tech and has deep Texas ties.
• I was waiting for the official story about Rhode Island contacting Wagner to talk with Danny Hurley. If Hurley takes the job, it contradicts what he said about Wagner being the perfect place to coach because there is a strong connection to the kids at a real local, grassroots level.
• Guess who is ready to be a head coach? Former Kansas star and current Jayhawks assistant Danny Manning. If Shaka Smart left VCU for Illinois, Manning would go heavily after that job. If Marshall left, he would do the same at Wichita State. And he wouldn't mind getting a look from Southern Illinois, as well.