"I enjoyed a fulfilling 15-year major-league career, and each season brought experiences and friends that I will cherish forever," the 40-year-old relief pitcher said in a statement
released by the Yankees.
"But being able to be a part of four world championship teams with the Yankees, while playing in a place like Yankee Stadium in front of the greatest fans in the world -- that time will always hold a special place in my heart," he said.
The tall right-hander with the funky, sidearm delivery played for the Yankees from 1996-2000, winning four World Series titles. He returned for part of the 2003 season.
Nelson was 0-1 with a 3.38 ERA in six games for the Chicago White Sox last year, missing most of the season because of an injured right elbow.
He had a 48-45 career record with 33 saves and was an All-Star with the Seattle Mariners in 2001. Nelson pitched for Seattle from 1992-95, 2001-3 and in 2005. He also played for Texas.
Nelson has appeared in 53 postseason games, second on the career list for pitchers behind former Yankees teammate Mariano Rivera.
The practice of signing with a former team and then retiring is unusual in baseball but more common in the NFL.
New York also agreed to a minor league contract with catcher Todd Pratt, who hit .207 in 135 at-bats last season for Atlanta.