The deal includes $12 million in the first year of the contract, the source told Schefter.
Anderson, 26, got off to a slow start for the Jets in 2019, but he picked it up in late November and finished with 52 receptions for 779 yards and two touchdowns. He averaged 15.0 yards per catch, 20th among qualifying receivers.
Anderson and Walker played at Temple when Rhule was the Owls' head coach, while Whitehead played there when Rhule was an assistant.
With the Panthers, Anderson will be paired with 2018 first-round pick DJ Moore, who had 87 catches for 1,175 yards last season. The combo, along with running back Christian McCaffrey, will give new offensive coordinator Joe Brady and quarterback Teddy Bridgewater the potential for a potent offense.
Always known as a vertical threat, Anderson diversified his game down the stretch and demonstrated a newfound aggressiveness on contested balls. He was the Jets' most dangerous perimeter target from 2017 to 2019, totaling 18 touchdowns in that span -- including nine of 30 or more yards.
Anderson's departure hurts the Jets because they don't have any proven outside receivers on the roster. Their most accomplished receiver is Quincy Enunwa, but he's rehabbing from a serious neck injury and still hasn't been medically cleared for football activities.
The Jets also have Vyncint Smith (22 career receptions) and recently signed Josh Doctson, a former Washington Redskins first-round pick who washed out with them and the Minnesota Vikings. The team is covered in the slot with Jamison Crowder, who had a career-high 78 catches last season, but the lack of a perimeter threat means the Jets could look to pick a receiver early in the draft.
Anderson made the Jets as an undrafted rookie out of Temple in 2016, impressing the coaches with his raw speed. At the same time, his off-the-field habits raised concerns within the organization.
He was arrested twice in less than a year, both times near his home in South Florida, but he avoided a league suspension and jail time. Both incidents, the latter of which occurred in January 2018, involved altercations with police. He was charged with several felonies, but the only one that stuck was a misdemeanor charge for reckless driving.
Anderson, who played last season on a one-year, $3 million contract (the amount of his RFA tender), has 207 receptions for 3,059 yards and 20 touchdowns in his career.
ESPN's Rich Cimini and David Newton contributed to this report.