"He don't want to meet with me," Oakley told The Undefeated's Mike Wise. "Man won't even shake my hand."
Oakley has been critical of the Knicks over the years, specifically taking on Dolan's stewardship as owner at times. But Oakley told The Undefeated he was really hurt when Dolan refused to shake his hand as Oakley approached him at the 2014 All-Star Game in New Orleans.
Since then, he said, it has been an even more acrimonious relationship.
"Whenever I go to the Garden now, security has been told to inform him immediately," Oakley said. "Last year against Charlotte, I went to a game and a bunch of security people came up to me right away and told me not to walk around, stay where I am. He just doesn't want me in the building."
Oakley was arrested and charged with three counts of assault after an altercation in the stands near Dolan at Madison Square Garden on Wednesday night.
Oakley said he was particularly wounded by the team's statement Wednesday night about the altercation and his arrest. The statement ended with: "He is a great Knick, and we hope he gets help soon."
"Right there, attacking my character like that -- not makin' it about the moment but somethin' bigger that's wrong with me? -- I couldn't believe it," Oakley said. "I don't know how you do that. I don't."
Oakley later told Fox 5 in New York that he had "a couple" of drinks before the game -- but none while at MSG.
The Knicks released a second statement on Thursday afternoon:
"There are dozens of security staff, employees and NYPD that witnessed Oakley's abusive behavior. It started when he entered the building and continued until he was arrested and left the building. Every single statement we have received is consistent in describing his actions. Everything he said since the incident is pure fiction."
Oakley contested that statement too.
"They got cameras following me all the way down to my seat," he told Fox 5. "I didn't get up until they walked over to me. So it can always [be] a story after a story, you know how it goes. PR stuff. But I'm not looking for PR. I just want to be treated right."
Wednesday's incident occurred during a stoppage in play, as Kristaps Porzingis was shooting free throws in the first quarter against the visiting Los Angeles Clippers. A police source told ESPN that Oakley had purchased a seat a few rows behind Dolan and was making comments at the Knicks owner, who eventually had security escort Oakley from the arena.
"Well, first of all, I want to say I'm sorry to all the fans of the Garden, all the fans," Oakley said Thursday on ESPN's Stephen A. Smith radio show. "It was just a bad scene. I love New York, gave my whole heart. I walked into the Garden, simply, I was there four minutes. I was talking to people. One of the guys said someone is watching you. They have to tell Dolan anytime I walk into the Garden. Then I'm told I have to leave. Orders you have to leave the building."
Knicks president Phil Jackson came out to try to calm down Oakley, who was yelling about Dolan and telling security he wasn't going to move as they handcuffed him outside the tunnel leading to the court. At one point, Oakley was on the ground surrounded by police and Madison Square Garden security.
According to the NYPD, Oakley, 53, punched three male MSG employees -- all of whom suffered minor injuries and refused medical attention at the scene -- as he was being escorted out. He was charged with three misdemeanor counts of assault, all third degree, and one count of criminal trespass, a third-degree misdemeanor.
Oakley was given a desk appearance ticket and will be required to appear in front of a judge at a later date. The court date is scheduled for March, a source told ESPN.
Oakley confirmed on Smith's radio show that he spoke to his close friend Michael Jordan on Wednesday night after the incident. Friends of Oakley had reached out to Jordan in an effort to connect the two to help calm Oakley down, sources told ESPN.
"It was just a conversation," Oakley said. "A couple of my guys, really good friend of mine reached out. They wanted to see what was going on. I don't know why it is so touchy when I come to the Garden."
James also showed his support for Oakley in an Instagram post Wednesday night, then did so again to conclude a brief postgame meeting with media Thursday night.
"Charles Oakley for president," he said.
Oakley played for the Knicks from 1988 to 1998, helping them reach the NBA Finals, but he has a poor relationship with the team because of his criticism of Dolan, the Madison Square Garden chairman.
"I've been saying the same thing since '88, when I came to New York," Oakley said on Smith's radio show. "I ain't critical, man. You are more critical. He gets mad at the headlines, I don't write the headlines. What am I supposed to say, they are goin' to win 60 [games]?"
ESPN's Ian Begley, Ohm Youngmisuk, Ramona Shelburne and The Associated Press contributed to this report.