"I don't want to say it was a joke, but I just never saw Trestman as being a head coach," Brown said Wednesday on "The Waddle & Silvy Show" on ESPN 1000.
Brown had posted nine straight 1,000-yard receiving seasons with the Oakland Raiders entering 2002, the year Trestman took over as the team's offensive coordinator. In that 2002 season, Jerry Rice emerged as the team's leading receiver and Brown finished with 930 yards.
"The year he took over as offensive coordinator is also the year my reign with the Raiders ended because he made Jerry the No. 1 receiver instead of myself," Brown said. "The year before I made the Pro Bowl and caught [91 passes for 1,165 yards] ... and the year afterwards, the year he takes over I think I came like 50 yards from catching 1,000 yards in 10 or 11 straight seasons.
"Hey, look, I'm not a selfish player, but come on, if I put the work in, make this happen for me. We had some interesting words about that part of it."
That 2002 season might have been the end of Brown's run as the team's top receiver, but it also ended with a Super Bowl appearance. The Raiders, who lost to the Tampa Bay Buccaneers in Super Bowl XXXVII, led the NFL in total yards and had the top-ranked passing game led by quarterback Rich Gannon, who won the MVP.
Oakland was one of Trestman's eight stops in the NFL as an assistant. He has spent the past five years in the CFL as head coach of the Montreal Alouettes.
Brown said there's no question Trestman can run an effective offense. It's running an NFL locker room that gives Brown pause.
"He's a really smart guy, knows football like the back of his hand. That's not the issue," Brown said. "But there's a lot more to coaching than just X's and O's. You have to be able to deal with players and that's the reason [former Raiders coach] Bill Callahan hasn't succeeded as a head coach because he's a smart guy, knows football like the back of his hand, but when it comes to leading me, it's a totally different intangible you have to have. It's going to be very interesting to see how [Trestman] and Jay Cutler get along, that's for sure."