Valanciunas was called for a flagrant foul 1 after he took down James on the play that drew a loud cheer from the sellout crowd of 19,800.
"The game is more important than trying to deliver a hard foul," said James, kicked in the groin by Houston's James Harden on Sunday. "It's the old elementary school house rule that the second guy always gets caught. You just relax and play the game."
James bristled when Valanciunas offered to help him up -- and later when asked about his response.
"Why would I need his help (getting) up?" James asked. "My teammates will come get me. That's all that matters."
The foul was upgraded to flagrant following a video review.
"I wasn't trying to hurt him, I was just trying to stop him from dunking the ball," Valanciunas said.
James finished four rebounds shy of a triple-double to help Cleveland win the season series against the Raptors with its third win in four meetings.
"That could be a factor, the way the East is shaping up," said Cavaliers coach David Blatt, who called the victory "extremely important."
Cleveland (39-24) also moved into a virtual tie with Toronto (38-23) and Chicago (38-23) for second place in the East. All three teams trail Atlanta by 10 1/2 games.
Kyrie Irving scored 26 points for Cleveland, Kevin Love added 22 points and 10 rebounds, J.R. Smith had 15 points and James Jones 14. The Cavaliers won their second straight and improved to 9-8 in the second game of back-to-backs.
Valanciunas had 26 points and 11 rebounds for Toronto, Lou Williams scored 21 of his 26 points in the fourth quarter, and DeMar DeRozan had 25 points. They lost for the sixth time in seven games.
Toronto All-Star point guard Kyle Lowry was rested for the third straight game.
Williams opened the fourth with a 3 and three free throws, bringing Toronto within four at 84-80. Williams made three more free throws at 9:22, and a 3 by Terrence Ross at 8:51 cut the gap to one at 91-90.
After a timeout by Cleveland, Smith missed a 3 and Williams hit one at the other end, giving Toronto its first lead since the opening quarter.
"They just shot the light outs," Blatt said.
Greivis Vasquez tied it at 97 by making one of two at the line at 5:51, but a 3 by James Jones and a pair of free throws by James put Cleveland up 102-97 just over a minute later.
James made 3-pointers on either side of a layup by Valanciunas as Cleveland took at 108-101 lead with 3:26, leading to a Toronto timeout.
Williams hit a 3 and Valanciunas tipped in his own miss as Toronto cut it to 108-106 at 2:25, but Smith answered with a 3, Cleveland's 15th of the game, at 2:07. After Irving made a pair of free throws, James sealed it with an acrobatic driving layup that put Cleveland up 115-106 with 55 seconds left.
"We showed poise and maturity when we had to," Blatt said, "because that game easily could have gotten away from us."
NOT ON THE RAY-DAR
Blatt said he wasn't troubled by the news that 10-time All-Star Ray Allen, who was part of Boston's NBA title in 2008 and Miami's in 2013, has decided not to play for an NBA team this season. Rumors had linked Allen with Cleveland and former Miami teammate James. "I never really tried to look at it as something that was a definitive option because it really wasn't ever clear if he was going to play at all," Blatt said.
QUITE THE QUARTER
Williams set a Raptors record with his 21-point fourth. Five Raptors had scored 20 points in a quarter. Alan Anderson was the last to do it, scoring 20 in the third against New York on March 23, 2013.
Cavaliers: Cleveland has three sets of back-to-back games remaining. ... The Cavaliers are 10-3 against Atlantic Division opponents, including a 6-1 mark on the road. ... Cleveland's 15 3-pointers were the most by a Toronto opponent this season.
Raptors: DeRozan didn't score in the fourth quarter, missing two field goal attempts. ... Vasquez had 13 assists. ... The Raptors posted their 30th sellout of the season, the third time in team history they've done so. Toronto also sold out 30 home games in 2001-02 and 2007-08.
Cavaliers: at Hawks on Friday night
Raptors: at Hornets on Friday night.
Paul Pierce, Chris Bosh, Chris Webber, Ben Wallace headline 2021 class for Basketball Hall of Fame
Paul Pierce, Chris Bosh, Chris Webber and Ben Wallace were among those selected as part of the 2021 class for the Naismith Memorial Basketball Hall of Fame on Sunday.
Bosh thankful for HOF election
Chris Bosh reflects on his career and getting elected to the Naismith Memorial Basketball Hall of Fame.
Basketball Hall of Fame: Kobe Bryant, Tim Duncan, Kevin Garnett headline Class of 2020
Here's everything you need to know about the star-studded Class of 2020.