Longtime assistant Greg Gard is taking over as interim head coach.
"After months of conversation with Barry Alvarez and his administrative staff, as well as my wife, Kelly, I have decided that now is the right time to step down from the head coaching position here at Wisconsin," Ryan said in a statement released after the game.
Ryan coached Wisconsin to a school-record 36 wins last season and a loss in the national championship game against Duke. Wisconsin is 7-5 this season, one more loss than the Badgers had all last season.
Ryan, who turns 68 on Sunday, had said in a statement in June that he would coach this season and then step down.
"This was a decision months in the making," Ryan said. "I brought this up to Barry back in April. He advised me to take some time to think it over, and I appreciated that. But in recent weeks, I have come to the conclusion that now is the right time for me to retire and for Greg Gard to have the opportunity to coach the team for the remainder of the season. I discussed this with Barry, and I appreciate him giving me the space to make this decision."
Ryan said in his postgame news conference that it was extremely hard telling his players of the decision.
"It is so emotional right now, and I'm trying to hold this together," Ryan said.
Ryan said that part of the reason he came back for this season was to give Gard time to be with his father, Glen, who was diagnosed with cancer in the spring and died this fall. Ryan said the demands on Gard's time to fly around the country meeting with doctors and helping his dad put his retirement on the back burner. Following Glen Gard's death, Ryan then spoke with Alvarez and decided the semester break would be a good time to step down.
Ryan did not take questions after his announcement, saying he had to go meet with some of the staff because he didn't want to inform them prior to the game.
The development with Ryan seemed to take the players by surprise, but forward Nigel Hayes said he noticed something different about the coach before the victory over the Islanders.
Hayes said he never makes eye contact with Ryan as the lineup is announced. But Tuesday night, he did.
"It was like the saddest look I've ever seen him have," Hayes said. "I guess now it makes sense why he looked like that."
"We've got to remember he's an old guy," Hayes also said. "He's been yelling and screaming for a long time. That takes a toll on your health."
Ryan is the winningest coach in Wisconsin history, with 364 wins over 14-plus seasons. Ryan led the Badgers to the NCAA tournament every year of his tenure, entering this season. Before Ryan came to Madison, Wisconsin had been to the tournament seven times in school history. Wisconsin won seven Big Ten titles under Ryan, and his .717 winning percentage (172-68) in conference play is the best in Big Ten history.
This is tough to watch..Thank you Coach Ryan for molding me into a young man. Wish I could argue with you one more time. All love to Pops!!— Sam Dekker (@dekker) December 16, 2015
Gard was in his 23rd season as an assistant under Ryan and his 15th with the Badgers. Alvarez said he will evaluate how Gard works with the team and then make a decision at the end of the year on what the school wants to do.
"I've never had more than a one-year contract in my entire career, so for me it's never been about the pressure in that way," said Gard, who recently turned 45. "For me, it's about putting these young men in the best position to have success as student athletes."
Gard pledged Ryan won't "vanish off planet Earth, and I'll make sure I still have his phone number."
Information from ESPN Stats & Information and The Associated Press was used in this report.