St. John's coach Mike Anderson announced the move on Thursday of the Baltimore native who played in 90 games and made 67 starts for the Scarlet Knights.
Mathis averaged 8.3 points and helped Rutgers advance to the NCAA Tournament Round of 32 in its first appearance since 1991. He shot 38.3% and averaged 2.8 boards in 21.3 minutes per game and finished in double figures 10 times, highlighted by a 25-point game in a win over Purdue.
Mathis will have two years of eligibility remaining.
"He has the ability to score and defend at a high level and he is a great fit for our program," Anderson said. "Plus, he's a veteran and a proven winner at the collegiate and high school levels. That's invaluable when you're establishing a winning culture."
As a sophomore, Mathis started 31 games, averaging 7.4 points and 3.5 rebounds. He averaged 8.5 points as a freshman.
St. John's posted a 16-11 record this past season.