The university announced Ray's hiring Sunday and scheduled a press conference for Monday morning.
The 40-year-old has been the top assistant at Clemson for the past two seasons. The Tigers won 22 games and went to the NCAA tournament in 2011. He also spent four years as an assistant at Purdue under Matt Painter, where the Boilermakers made the NCAA tournament every season -- including two Sweet 16 appearances.
Ray also spent seven seasons at Indiana State and two seasons at Northern Illinois.
Ray replaces Rick Stansbury, who announced his retirement in March after 14 seasons leading the program. The details of Ray's contract were not immediately released. Stansbury was making nearly $1.5 million per year.
Mississippi State athletic director Scott Stricklin said in a statement that Ray fit the model he wanted as a head coach. Ray is the first African-American men's basketball coach at the school.
"He is bright, enthusiastic, disciplined and is a man of integrity," Stricklin said. "He has served with some of the top head and assistant coaches in college basketball and will bring a piece of all of them to our head coaching position."
Ray inherits a Mississippi State program that's had several defections after finishing a disappointing 21-12 season that ended in the first round of the National Invitation Tournament. Juniors Renardo Sidney and Arnett Moultrie left the program to pursue professional careers while freshman Deville Smith left to transfer to another program.
The Bulldogs should still have a decent nucleus led by Rodney Hood, who averaged 10.3 points and 4.8 rebounds as a freshman. Guard Jalen Steele and forward Wendell Lewis also are expected to return after having productive seasons.
Clemson coach Brad Brownell said Ray was well-respected in the coaching profession and would do well at Mississippi State.
"Rick is a terrific coach in every sense of the word," Brownell said. "He is a consummate professional. He is a relentless recruiter, has worked with some of the best X's and O's coaches in the business, and is unbelievably bright."
Said Ray in a statement: "I have so much appreciation for Brad Brownell. I learned a great deal underneath him in my two years. I learned more about being a basketball coach, how to build a program, and how to treat people."
"The biggest memory from Clemson will be the bonds that I had with our players in such a short amount of time," he added. "I can't say enough about how good our basketball players were as people."
Information from The Associated Press was used in this report.