"We don't know when we are going to get [Aceves] back," Joe Girardi said.
Aceves was eligible to return from the disabled list on Tuesday, but a bulging disk forced him to return to New York instead to see doctors. With Gaudin on board, Sergio Mitre could pitch in some shorter relief roles.
Gaudin finished last season with the Yankees and was with the team in spring training, before being released on March 25.
He signed with Oakland and went 0-2 with an 8.83 ERA in 12 appearances before being released by the A's last Friday. He last pitched on May 15.
Gaudin was on New York's postseason roster in 2009, throwing one scoreless inning in the ALCS.
To make room for Gaudin, left-hander Boone Logan was optioned to Triple-A Scranton/Wilkes-Barre. To make room on the 40-man roster, right-hander Shane Lindsay was designated for assignment.
Information from The Associated Press was used in this report.