Now, USC may have that national title stripped after the school was rocked by severe sanctions handed down by the NCAA on Thursday.
Though the BCS may force Southern California to vacate its national championship from that 2004 season, Smith says the NCAA and BCS can never truly take away a championship he helped the Trojans win.
"Can't take my ring away, can't take my tape away, can't take my memories away," Smith said of always having evidence of that game.
Smith and Giants cornerback Terrell Thomas were teammates at USC during the time for which the school is being punished for lack of institutional control. USC was spanked with a two-year bowl ban, four years' probation, loss of scholarships and forfeiture of 14 victories for improper benefits to Reggie Bush dating back to the Trojans' national championship won during the 2004 season.
The NCAA ruled that Bush was ineligible beginning at least by December 2004 for receiving lavish gifts from two sports marketers hoping to sign him.
Smith and Thomas were surprised by the severity of the punishment, and both players said they were not aware of any wrongdoing.
"Not at all," Thomas said when asked if he knew of any improper behavior when he played at USC from 2003 to 2007. "I never thought anything of it because they were investigating for so long. I thought nothing would ever come out of it. I hope USC does a great job of appealing it and hopefully it will change the outcome."
Smith doesn't want to see USC's 2004 national championship erased from the history books. Smith, who was at USC from 2003-06, enjoyed a monster game during USC's 55-19 Orange Bowl victory over Oklahoma. Smith had seven receptions for 113 yards and three touchdowns. Thomas also played in that game, totaling two tackles.
The BCS presidential oversight committee will meet soon to discuss whether USC will be stripped of that title.
Both Giants said no matter what happens, they won't feel like that national title has been taken away from them.
"No, because we still went out and won it," Smith said. "It just kind of [stinks] because that was one of my best games as a Trojan. If they take away the record books, I don't want some kid who is looking 10 years from now to not see my name on it."
Thomas said that he hopes the school's appeal will help reduce the loss of 30 football scholarships over three years.
"It is unfortunate," Thomas said. "My main thing is, suspend the school but don't take away the scholarships. That is affecting the kids. That is 30 kids that don't have scholarships to go to college now from all over the country and that is unfair.
"The NCAA needs to do something about the ruling because they make so much money off of us. I am not saying that [college players] should receive anything, but at the same time something has to give in the middle. The institute is not losing, USC is not losing, football is not losing, it's the kids that are losing."
Smith was asked what he will do if he was asked to give back his national championship ring.
"If they do, I will say I lost it," Smith said.