Herman will receive a five-year, $6.75 million deal from Houston.
"This is a tremendous opportunity for my family and I to come back to Houston and lead one of the top programs in the country," Herman, who has 11 years experience coaching at the college level in Texas, said in a statement. "I am looking forward to working with one of the top athletic directors in the country as well as Chancellor Khator in a partnership that will make the city of Houston and the great state of Texas proud."
Houston fired coach Tony Levine last week.
No. 4 Ohio State plays No. 1 Alabama in a CFP semifinal Jan. 1 at the Allstate Sugar Bowl. If Ohio State wins, the Buckeyes will play Oregon or Florida State for the national championship on Jan. 12 in Arlington, Texas.
Houston plays Pittsburgh in the Armed Forces Bowl on Jan. 2, and defensive coordinator David Gibbs will serve as the Cougars' interim coach.
The 39-year-old Herman, a Cincinnati native, spent the first 10 years of his career coaching in the state of Texas before becoming offensive coordinator at Iowa State (2009-11).
He has been quarterbacks coach and playcaller for Ohio State teams that have gone 36-3 the past three seasons. The Buckeyes rank fifth in the nation in yards per play this season and finished fifth in yards per play last season.
Under Herman, Buckeyes quarterback Braxton Miller was the Big Ten Player of the Year in 2012 and 2013. Redshirt freshman quarterback J.T. Barrett stepped in this season after Miller was injured in the preseason, and Herman helped developed him into a Heisman Trophy contender. Barrett set a conference record for touchdowns responsible for this season before going down with a season-ending injury against Michigan.
Last week, Herman won the Broyles Award as the top assistant coach in college football.
The Cougars were 21-17 under Levine, including bowl appearances the past two seasons, and finished 7-5 this season.
But with a new on-campus stadium and a roster that seemed to be one of the most talented in the American Athletic Conference, more was expected from the Cougars.
The Associated Press contributed to this report.