Kelly completed a deflected 66-yard touchdown pass and added two more long ones in the fourth quarter, and the Rebels held on to beat No. 2 Alabama 43-37 on Saturday night.
The Rebels (3-0, 1-0 Southeastern Conference) left the topsy-turvy game with their first victory in Bryant-Denny Stadium since 1988, and second straight in a one-sided series. The Crimson Tide (2-1, 0-1) had been 25-1 in Tuscaloosa against Ole Miss and had never lost two straight in the series.
Both games went down to the wire and were settled with the help of Rebels interceptions.
"We certainly think we're at a point now where it's not a shock," Mississippi coach Hugh Freeze said. "I still say it would be an upset in most people's eyes to do it here.
"I don't think it surprises the people in our locker room. Our pregame was as short as it's ever been. I said, 'You're good enough to win.'"
The Rebels made two final defensive stands, the first when Tony Bridges intercepted a deep ball from Jake Coker, who came off the bench to lead a comeback from a 30-10 deficit. Ole Miss scored 24 points off five Alabama turnovers.
Kelly delivered big play after big play following an ineffective first half, none wilder than a fling and a prayer in the third quarter, a 66-yard deflection caught by Quincy Adeboyejo for a touchdown.
Kelly bobbled a high snap and lofted the ball toward a crowd as he was going down before a swarm of Tide defenders. Adeboyejo grabbed a deflected ball and raced toward the end zone in front of Alabama's student section.
"That happened in my prayer time today," Freeze said. "That's the only thing I can say about that."
Kelly said he figured the 6-foot-3 Adeboyejo could "jump the highest out of anybody, and I just gave him a chance."
"Right after that you think, 'Hey, we're winning this,'" he said. "And that's exactly how we all thought, that this was our game."
Alabama had won 17 straight at home coming into the game to tie Baylor for the longest active streak in FBS, another milestone snapped by a feisty Ole Miss team that's established itself as an SEC contender under Freeze.
"We thought last year when we played we gave some things away, and I really don't want to take anything away from Ole Miss," Tide coach Nick Saban said. "They played a really good game, but are you going to be a giver or are you going to be a taker? If you're going to be a great competitor, you've got to take what you want.
"I don't know if there's anybody we can beat if we're going to give away  points, I really don't."
Coker came up with big plays, and big mistakes. He led two touchdown drives in a span of 2:03 late in the fourth quarter, aided by an onside kick. Derrick Henry scored on a 2-yard run and Richard Mullaney caught a 2-yard touchdown pass. Coker also was picked off by Bridges downfield with 2:36 left.
Coker was 21-of-45 passing for 201 yards with three touchdowns and two interceptions. Henry ran for 127 yards.
Coker, meanwhile, ran seven times for 58 yards.
"We had some things in the game plan early that the quarterback was going to pull the ball on and do some things, so we thought Cooper's speed would be sort of a change of pace for them, a little bit of an element of surprise," Saban said.
Alabama got one more chance with 31 seconds left after the defense stuffed Jeremy Liggins on fourth-and-1 from the Tide 31. Replay officials said Jaylen Walton's elbow touched the ground just shy of a first down on the previous play.
Four straight incompletions, including a drop, ended the rally hopes for Alabama.
Texas Tech RB SaRodorick Thompson, charged with street racing, will still play vs. Texas
Texas Tech running back SaRodorick Thompson, charged Monday with street racing, is still expected to play Saturday against Texas in the Red Raiders' Big 12 opener.
Texas Tech reports zero COVID-19 positive tests in week after game
Texas Tech's football team had no positives COVID-19 tests among 360 conducted for during three testing periods last week, after its first game.
Justin Fuente hopes Virginia Tech 'able to play' NC State this week
Virginia Tech coach Justin Fuente said Monday that he hopes his team will finally be able to play its opener against NC State this week, though it all depends on testing. "Taking it day by day," he said.
The coronavirus and college sports: NCAA reopening plans, latest news, program cuts, more
After first postponing its college football season, the Big Ten is now set to begin play the weekend of Oct. 24. Here's the latest on how the coronavirus pandemic is affecting the college sports world.
Dave Aranda optimistic Baylor will play Big 12 opener against Kansas
Baylor coach Dave Aranda expressed optimism Monday that his team will be able to play its Big 12 opener against Kansas on Saturday after having to call off a game last weekend because of positive COVID-19 tests that affected a position group.
Tennessee starting Jarrett Guarantano at QB, deliberating on backup
Jarrett Guarantano, who lost his starting job last season and even received death threats, has had a full offseason to better learn coordinator Jim Chaney's offense.