Gostkowski accomplished the feat late in the third quarter of Sunday's game against the Jacksonville Jaguars. The only time Gostkowski missed a PAT in his career came in Week 17 of the 2006 season, his rookie year.
The previous record of 422 was held by Matt Stover.
"That's not something I try to keep up with, but it was cool," Gostkowski said after the game. "It's always fun to do well and win, just being on a good team that scores a lot of points. When you kick, you're the beneficiary of opportunity. I've had a lot of opportunities to kick a lot of field goals, a lot of extra points."
Gostkowski's milestone was acknowledged by the Patriots during the game, as a picture of Gostkowski was shown on the video board with a description of what he had accomplished. That led to a loud cheer from the home crowd.
The NFL has moved the extra point back to the 15 yard-line this year, making it a more competitive play. That change makes it more challenging for those looking to break Gostkowski's record.
Gostkowski's milestone came on the same day that quarterback Tom Brady reached the 400-touchdown mark.
"That takes a lot of games, a lot of snaps to do that, and it’s certainly a tribute to both of them, the durability, longevity and obviously consistent production that both of them have had," coach Bill Belichick said. "So, I feel really fortunate to have coached and had guys like that on our team, and those two guys in particular, they had a good day as they’ve had for many, many days in their career. I’m really fortunate to have those guys on this team."