Tennessee (17-11, 8-7 SEC) is at risk of missing the NCAA tournament for the third straight year since Martin started as coach, putting him on the hot seat as the Vols enter their final three games, including against Stallings' Commodores on Saturday.
"I think for people to speculate on his job is absurd," Stallings said Thursday before practice, according to The Tennessean.
The Volunteers, who played in the NIT the past two years, are currently among the "last four in" the NCAA tournament field, according to ESPN's Joe Lunardi's Bracketology.
Martin took over for Bruce Pearl, whose three-year suspension for lying to the NCAA has nearly expired. An online petition calling for Tennessee to rehire Pearl has more than 29,000 signatures.
The Vols reached the NCAA tournament in all six of Pearl's seasons. But Stallings said for Tennessee to give up on Martin so soon after hiring him would be "ridiculous."
"Hopefully, the powers that be over at Tennessee will tune those idiots out and give [Martin] the kind of time he deserves to do the job he needs to do," Stallings said.
Pearl, now an ESPN analyst, wouldn't rule out a return to coaching in a recent interview with USA Today.
"I don't know," Pearl said. "I'm working hard to put myself in position if the opportunity presented itself. But I can tell you that if I don't ever coach again, it won't be something that will haunt me. I don't feel like I've gotta get back into coaching to prove anything to anybody."
It would cost Tennessee $1.56 million to buy out Martin's contract this season, The Tennessean reported this week. That's not insignificant, considering the athletic department has paid more than $15 million in buyouts to former personnel over the past decade, including $5 million to former football coach Derek Dooley.
Stallings said he believes Tennessee will make the NCAA tournament. The Vols' remaining regular-season schedule consists of the home game against Vanderbilt (15-12), a game at Auburn (13-13), and at home against fellow NCAA bubble team Missouri (19-9).
"It used to be in this business guys got four or five years to show what they could do," said Stallings, who is in his 14th season at Vanderbilt. "The only thing that's changed is people thinking that they ought to have more of a say in that and then weak athletic administrations giving in to that kind of pressure."