Kelly completed a deflected 66-yard touchdown pass and added two more long ones in the fourth quarter, and the Rebels held on to beat No. 2 Alabama 43-37 on Saturday night.
The Rebels (3-0, 1-0 Southeastern Conference) left the topsy-turvy game with their first victory in Bryant-Denny Stadium since 1988, and second straight in a one-sided series. The Crimson Tide (2-1, 0-1) had been 25-1 in Tuscaloosa against Ole Miss and had never lost two straight in the series.
Both games went down to the wire and were settled with the help of Rebels interceptions.
"We certainly think we're at a point now where it's not a shock," Mississippi coach Hugh Freeze said. "I still say it would be an upset in most people's eyes to do it here.
"I don't think it surprises the people in our locker room. Our pregame was as short as it's ever been. I said, 'You're good enough to win.'"
The Rebels made two final defensive stands, the first when Tony Bridges intercepted a deep ball from Jake Coker, who came off the bench to lead a comeback from a 30-10 deficit. Ole Miss scored 24 points off five Alabama turnovers.
Kelly delivered big play after big play following an ineffective first half, none wilder than a fling and a prayer in the third quarter, a 66-yard deflection caught by Quincy Adeboyejo for a touchdown.
Kelly bobbled a high snap and lofted the ball toward a crowd as he was going down before a swarm of Tide defenders. Adeboyejo grabbed a deflected ball and raced toward the end zone in front of Alabama's student section.
"That happened in my prayer time today," Freeze said. "That's the only thing I can say about that."
Kelly said he figured the 6-foot-3 Adeboyejo could "jump the highest out of anybody, and I just gave him a chance."
"Right after that you think, 'Hey, we're winning this,'" he said. "And that's exactly how we all thought, that this was our game."
Alabama had won 17 straight at home coming into the game to tie Baylor for the longest active streak in FBS, another milestone snapped by a feisty Ole Miss team that's established itself as an SEC contender under Freeze.
"We thought last year when we played we gave some things away, and I really don't want to take anything away from Ole Miss," Tide coach Nick Saban said. "They played a really good game, but are you going to be a giver or are you going to be a taker? If you're going to be a great competitor, you've got to take what you want.
"I don't know if there's anybody we can beat if we're going to give away  points, I really don't."
Coker came up with big plays, and big mistakes. He led two touchdown drives in a span of 2:03 late in the fourth quarter, aided by an onside kick. Derrick Henry scored on a 2-yard run and Richard Mullaney caught a 2-yard touchdown pass. Coker also was picked off by Bridges downfield with 2:36 left.
Coker was 21-of-45 passing for 201 yards with three touchdowns and two interceptions. Henry ran for 127 yards.
Coker, meanwhile, ran seven times for 58 yards.
"We had some things in the game plan early that the quarterback was going to pull the ball on and do some things, so we thought Cooper's speed would be sort of a change of pace for them, a little bit of an element of surprise," Saban said.
Alabama got one more chance with 31 seconds left after the defense stuffed Jeremy Liggins on fourth-and-1 from the Tide 31. Replay officials said Jaylen Walton's elbow touched the ground just shy of a first down on the previous play.
Four straight incompletions, including a drop, ended the rally hopes for Alabama.
Auburn right guard Brandon Council expected out for season with knee injury
Auburn right guard Brandon Council is expected to miss the rest of the season after sustaining a "pretty severe knee injury'' in Saturday's win over Ole Miss.
The coronavirus and college sports: NCAA reopening plans, latest news, program cuts, more
Postponed and canceled games, positive tests sending coaches and players into isolation ... yes, it's still 2020. Here's the latest on how the coronavirus pandemic is affecting the college sports world.
Wisconsin QB Graham Mertz awaiting confirmation of positive COVID-19 test, per reports
Wisconsin quarterback Graham Mertz, who had a record-setting first career start Friday, tested positive for COVID-19 over the weekend, according to multiple reports, and will take a PCR test to confirm the result.
Ga'Quincy McKinstry, ESPN's No. 38 overall recruit, commits to Alabama football
Alabama's No. 1-ranked recruiting class added another ESPN 300 prospect on Sunday when defensive back Ga'Quincy McKinstry announced his commitment to the Crimson Tide. He's ranked No. 38 overall by ESPN.
Indiana Hoosiers jump into AP Top 25 poll, Ohio State Buckeyes move up to No. 3
Indiana jumped into The Associated Press college football poll Sunday after pulling off the weekend's most dramatic upset, and Ohio State moved up to No. 3 following a dominant season debut.
AP Top 25 college football poll reaction: What's next for each ranked team?
The Big Ten returned and it did not disappoint, with some incredible performances and epic finishes. What does it mean for the AP Top 25? Let's break down what's next for each ranked team.