Kelly hit his approach to 4 feet on the par-4 18th to set up the winning putt. The 55-year-old from Wisconsin has nine victories on the 50-and-over tour after winning three times on the PGA Tour.
He broke out of a putting slump with help from Steve Stricker, his Madison, Wisconsin, friend and neighbor who tied for eighth.
"It felt good all week. I just had to keep my patience in there. That was what sent me through, besides obviously Steve's putting lesson again. But really is the patience. Because of that, because I was able to make some putts, that really carried me through."
Kelly returned from a lightning delay to play the final three holes of regulation, making a birdie on the par-3 17th. Triplett birdied the par-4 16th and closed with two pars.
"Just a good old-fashioned afternoon Midwest thunder boomer pop-up," Kelly said about the storm. "It didn't last long, luckily. Didn't even think we were going to need to hit balls, but I'm glad I did because I found out just how old I am on that first drive in the playoff."
The 60-year-old Triplett won the most recent of his eight senior titles in 2019.
Alker followed his second-round 64 with a 69. The Senior PGA Championship winner last week in Michigan, the 50-year-old from New Zealand has four victories in his past 13 events.
The 64-year-old Langer had four back-nine birdies in a 68.
Brett Quigley was fifth at 15 under after a 70.
Stricker returned after having to withdraw from the Senior PGA because of a positive COVID-19 test result. He won the major in Alabama in his previous start.
Stricker will host the American Family Insurance Championship next week in Madison. Kelly is the two-time defending champion.