The contract, which comes with an option for 2018, goes into effect after his current deal expires at the end of this season. The deal is worth $40 million, according to media reports.
Executive vice president Dan Duquette said he figured there was no time like the present to lock up a two-time Gold Glove winner at a key infield position.
"Why not now, really?" Duquette said. "When we looked at the market, we thought the best chance to sign J.J. was before he went to free agency because he's distinguished himself as one of the top shortstops in the league. And J.J. let us know that he wanted to continue his career here in Baltimore."
Hardy has been a fixture at shortstop for the Orioles since 2011, playing solid defense and providing decent hitting in a lineup filled with sluggers.
Although he hit only nine homers this season, Hardy totaled 77 long balls over the previous three years and won Gold Gloves in 2012 and '13.
Free agency was tempting, but in the end, Hardy said, he wanted to stay with the Orioles.
"It crossed my mind, but at the same time, this is where I wanted to be," he said. "There's a lot of uncertainty in free agency. You don't know what's going to happen. I knew that I liked it here and I liked playing with all my teammates. Bottom line, it comes down to winning, and I think we've got a good thing here."
Hardy, 32, batted .268 with 52 RBIs in 141 games during the regular season, then went 3-for-10 in Baltimore's three-game sweep of Detroit in the AL Division Series.
In the field, he ranked second among AL shortstops with 99 double plays turned, was third in fielding percentage (.978) and assists (394), and was fifth in putouts (187).
Hardy is a two-time All-Star, in 2007 with Milwaukee and last year as a member of the Orioles.
"I hope J.J. ends his career here," Orioles manager Buck Showalter said. "It makes us a better club, obviously. He's been solid. You never take for granted what J.J. brings."
Teammate Adam Jones welcomed the signing before the team had formally announced it.
"J.J. has been one of the best shortstops in baseball the last three, four years, and he's deserving," Jones said. "He could have gone to free agency and gotten more money, but it's not about the money. He's still going to be eating good, and he's happy here.
"And happiness, at the end of the day, will make you happy."
Hardy had previously discussed an extension with the team during spring training, but the two sides couldn't reach an agreement before the regular season began. He earned $21 million over the past three years.
ESPN MLB Insider Jim Bowden, ESPN.com's Jerry Crasnick and The Associated Press contributed to this report.