Menzies will replace Chris Beard, who left the Rebels less than three weeks after being hired and will instead coach Texas Tech.
The 2015 Western Athletic Conference coach of the year, Menzies had coached the Aggies for nine years and completed his fifth straight 20-win season this year and seventh in nine years.
"Marvin is an experienced head coach with UNLV and West Coast roots who knows what it takes to build and maintain a winning college basketball program," UNLV president Len Jessup said in a statement. "He is a dynamic recruiter and proven leader who understands the proud tradition of UNLV basketball, and we look forward to welcoming him back to campus."
Menzies spent the 2004-05 season as an assistant with the Rebels under Lon Kruger before leaving to serve as an assistant under Rick Pitino in Louisville. After two years with Pitino, Menzies took over at New Mexico State.
"We are looking forward to bringing Marvin Menzies back to UNLV as our men's basketball head coach," athletic director Tina Kunzer-Murphy said in a statement. "He has spent almost a decade as a Division I head coach, has won regular-season and conference tournament championships and has led his team to multiple NCAA tournament appearances. He has connections to our university and our community, and respects the tradition of Runnin' Rebel basketball.