Taylor, who played with the Jaguars from 1998 until being released after the 2008 season, challenged Jaguars fans during a conference call on Wednesday by saying the team probably would be better off moving.
"Sometimes the truth hurts," said Taylor, who gets his first crack at his old team after missing 10 games with an ankle injury when he and the New England Patriots host the Jaguars Sunday. "It can be played either way in that the fans want the team to stay there, they have to show that they want the team to stay there."
Taylor, participated fully in practice Wednesday for the first time since being sidelined, said the league has made it clear a team eventually will be in Los Angeles and because of attendance problems, the Jaguars might be the choice.
"I don't know what's going to be a decision maker or breaker but you look at average attendance and whatever have you at the end of the day they're going to find a team to take out there," he said. "I don't want to offend anyone. Never do, but you've got to be realistic."
Jaguars coach Jack Del Rio said his former back has always been one to speak his mind.
"Fred has always been a great interview because he speaks straight from the heart and he's not trying to be politically correct," Del Rio said. "That's what he feels and it's entertaining reading."
As far as any potential move goes, Del Rio said he will support whatever decision team owner Wayne Weaver makes.
"I'm an employee of his," Del Rio said. "Whatever he wants to do, my personal feelings are not part of it, my duty is to do the best I can in my role as head coach."
Before suggesting a possible move to Los Angeles, Taylor found himself in a sparring match with Del Rio.
Taylor, who is Jacksonville's all-time leading rusher with 11,271 yards, drew a rebuff from Del Rio after a Florida Times-Union report. Taylor said there is plenty of veteran leadership leading the way in New England, more film watching and quizzes with the Patriots than he experienced with the Jaguars.
"I saw [Taylor's comments], and I said that's really great because he pointed out the leadership there shows the way and I think he was referring to veteran players," Del Rio said. "So, that's what we're looking to instill in our veteran players, and evidently he wasn't that veteran leader here having our guys watch the tape. So, we're going to work on getting the veteran leaders that are now here to have our guys watching more tape because he provided the blueprint."
Even though both Del Rio and Taylor say they have no hard feelings, the player said the comment stung.
"A smidgen," he said. "A little piece of me didn't" like it.
Taylor explained he didn't mean the Patriots had a hold on all film watching and the quizzes come from coach Bill Belichick's style. He also insisted he wasn't taking shots at Del Rio.
"Never, never, never," he said. "I would never take a shot at him. You never burn bridges. Why would I burn bridges? I would never say that even if I felt that way. I have no problems with Jack, I really don't."
Taylor says him the tone of his comments were lost in the translation.
"It's kind of like a text message," Taylor explained. "He didn't hear my tone, he didn't know if I was laughing or joking. He didn't hear any of that."
The Associated Press contributed to this story.