Schiano, 49, also will serve as the team's associate head coach. Schiano replaces Chris Ash, who was introduced as head coach at Rutgers earlier this week.
Before coaching in the NFL, Schiano led a massive rebuild at Rutgers, taking one of the worst programs in major college football and turning it into a consistent winner. From 2001 to '11, the Scarlet Knights went 68-67 under Schiano, going to bowl games in six of his last seven seasons.
He parlayed the success into the Buccaneers job, going 11-21 before being fired after the 2013 season. He has been serving as an NFL analyst for ESPN.
"Greg Schiano is an excellent coach," Ohio State coach Urban Meyer said. "He is someone I have known for quite some time now and someone who is going to align with our staff extremely well. I think he will be outstanding as a coach and mentor in our program, and I am pleased to have him on our staff."
Schiano was on South Carolina's interview list for its head-coaching position, which eventually went to Will Muschamp. Schiano also was linked to the head-coaching opening at Miami, sources told ESPN. The Hurricanes hired former Georgia coach Mark Richt.
Before taking over at Rutgers in his home state of New Jersey, Schiano was defensive coordinator at Miami under Butch Davis. He also spent three seasons as an assistant with the Chicago Bears and five years working for Joe Paterno at Penn State.
Ash spent two years in Columbus working as co-coordinator with Luke Fickell and helping the Buckeyes win the national title in 2014. He built the defense into a top-20 unit nationally.
Ohio State went 11-1 this season, boasting its best defense since 2010. The Buckeyes are tied for fourth in the nation in yards allowed per play (4.39) this season. They are 49-4 in four seasons under Meyer.
The Buccaneers allowed the second-fewest rushing yards per game (96.3) under Schiano during the 2012 and 2013 seasons, but Tampa Bay allowed the most passing yards per game (267.6) during his two seasons with the team.
ESPN Stats & Information and The Associated Press contributed to this report.