"I wasn't trying to be a wise guy," Wolf told ESPN's Chris Mortensen. "I realize he has a real tough job. It was just a passing moment, and obviously, it was picked up on a microphone. I basically meant, 'Congrats on making a decision.' That's why the owners are paying him all that money.
"But it's still a tough job, and I congratulated him on making that decision, although I don't think it was anything profound. It happened so quickly. It's not me saying I'm for or against his decision. That's not for me to say."
During Thursday night's gold jacket ceremony ahead of this weekend's Hall of Fame induction in Canton, Ohio, Wolf was shown telling Goodell, "Way to go on that Brady thing."
Wolf slapped Goodell on the shoulder with his left hand as he delivered the remark, and Goodell's reaction was to let out a hearty laugh -- a response heavily criticized on social media Friday.
Goodell upheld Brady's four-game suspension for his role in Deflategate. They are due to meet in a Manhattan federal court Wednesday for a court-mandated settlement hearing.
Wolf will be inducted into the Hall as a contributor Saturday.
Information from ESPN.com's Mike Reiss was used in this report.