The deal was agreed to earlier this week, but Wednesday marked the end of the NBA's trade moratorium.
Anderson, a 6-foot-10 forward that can shoot the 3-pointer, completed his fourth NBA season and third in Orlando. He was a coveted free agent after posting a career-best 16.1 points and 7.7 rebounds, and being named the league's most improved player.
The Magic had the opportunity to match New Orleans' four-year, $36 million offer sheet, but declined.
Orlando gets back the 6-10 Ayon, who completed his rookie NBA season after playing professionally in Spain. He started 24 games, averaging 5.9 points and 4.9 rebounds.