Wilson shot a 2-under 70 in the final round on the Old Course at St. Andrews as McIlroy came up just short after a bogey on the 17th hole. Wilson finished at 17-under 271, with McIlroy (68) in a three-way tie for second with Richie Ramsay of Scotland and Tommy Fleetwood of England.
The 34-year old Wilson, currently ranked 792nd in the world, had finished second nine times in his 227 previous tour events, but lost his card two years ago and was competing this week on an invitation.
"It's all really hard to believe as I have dreamed of this moment many times in my career, and in the circumstances it has happened I can't believe it," said Wilson, who started the day with a three-shot lead. "So it's a dream come true to win a tournament as St. Andrews because three, four weeks ago my career was looking pretty grim. It's just amazing what is possible playing professional golf."
McIlroy started with a double bogey but then made four straight birdies and picked up two more shots after the turn before putting from off the green and into the infamous Road Hole bunker at the 17th. At the first hole, his ball spun back off the green into the Swilcan Burn.
"I feel I cost myself the tournament in the space of 20 yards at the front of the green at the first and over at the Road Hole bunker, with both not too far away from each other," McIlroy said. "They were the only two mistakes I made all day."
Ramsay had a two-shot lead after birdies at 14 and 15 but then handed the shots back with bogeys the next two to shoot 67. Fleetwood had a chance to force a playoff but missed a six-foot putt at the 18th to settle for a 68.
Chris Doak of Scotland was alone in fifth place, another shot behind.