MOBILE, Ala. -- Sean Payton made his first public remarks since being reinstated from his season-long suspension, saying he wished he had managed his coaching staff better and that he has no desire to speak with former defensive coordinator Gregg Williams.
The New Orleans Saints coach returned to work Wednesday a day after his suspension for his role in the team's bounty program was lifted, watching Senior Bowl practice.
"There's certain things that, you know, as you have a chance to look back on it that I would want to look back on differently and I would want to handle differently, with the challenges of managing a staff," Payton said.
Williams was suspended indefinitely by the NFL for masterminding the team's bounty program with the team during the 2009-11 seasons.
Payton said he met for four-and-a-half hours with commissioner Roger Goodell on Monday, and Williams' status was not discussed.
"I have no interest in talking to Gregg," he said.
Williams has not been reinstated by the NFL, and his status will not be addressed until another team is interested in hiring him, sources told ESPN. It was reported in early January that Williams would not return to the Rams, where he had been hired before his suspension.
Payton said he and Goodell agreed not to discuss specifics when asked whether there was an organized bounty system.
"We're at a point where it's time for closure. It's time for us as a team, it's time for us as a league, to take this next step forward," Payton said.
He said it was hard not being able to talk with his colleagues and players on the Saints during his suspension.
"I think the hardest part, though, was not football. The hardest part was so many of these people we're talking about -- [general manager] Mickey Loomis, [owner Tom] Benson, a number of the players -- you're used to talking to them on a regular basis," he said. "So the difficult part for me was not having the personal interaction. Not necessarily football-related, but just as you would as a friend. That was what I found to be more difficult than football."
Earlier this month, the Saints signed Payton to a five-year contract extension that runs through the 2017 season.
The deal is expected to pay Payton more than $8 million annually, which should establish him as the NFL's highest-paid coach, a league source told ESPN.
Payton said Wednesday that he never planned to leave the Saints.
"There was no way I was going to another team," he said.
The coach watched the Senior Bowl's North squad practice from the stands, chatting with assistant head coach Joe Vitt and general manager Mickey Loomis.
The NFL suspended Payton, Vitt, Loomis and four current or former Saints players, including linebacker Jonathan Vilma, after an investigation found the club had a performance pool offering cash rewards for key plays, including big hits. The player suspensions eventually were overturned.
Payton said he wasn't involved in the appeals process.
"I followed the process but I wasn't really privy to how those appeals went, how the meetings went, the specifics, and I'm still not," he said.
Payton said his situation was different because as head coach, "It all falls under your umbrella." He said he wouldn't allow himself to get "disappointed or upset, and I was just going to be patient to keep waiting and making sure I did everything the right way."
He said the challenges of rebuilding after a losing season will be painful and that his return won't guarantee an immediate return to winning.
Payton said the Saints' struggles defensively and running the ball early in the season, along with problems with special teams at times, made it tough to win -- with or without the bounty scandal.
Payton, citing advice from Bill Parcells, noted that his return won't mean an automatic return to big success. It was the Saints' first losing season since 2007
"Coach Parcells said to me that this mind-set that you're back and all of a sudden you're back to winning 11, 12, 13 games doesn't exist, because you very well could win five the next year," he said. "He's right. We've got a lot of things we've got to correct to get to where we want to go. That's just the truth."
He said he hadn't started talking to Loomis regarding any possible changes in the coaching staff .
The defense allowed 7,042 yards, the most in a season under first-year coordinator Steve Spagnuolo but also improved enough to later record the club's first shutout in 17 years.
Information from ESPN NFL Insider Ed Werder and The Associated Press was used in this report.