Herrmann, 48, replaces Bill Sheridan, who was promoted Monday to replace Steve Spagnuolo as defensive coordinator.
Before joining the Jets, Herrmann spent 20 years at Michigan, including a nine-year stint as defensive coordinator. During that time, Sheridan was a member of his staff.
"I've always been very much aware of Jim Herrmann and the job that he has done," Coughlin said Thursday. "He was the defensive coordinator at the University of Michigan for a lot of years and did an outstanding job. I was very impressed by not only the type of defense that they played but the aggressive style that they played. And he is a part of the Bo Schembechler tree. I have always been a great admirer of Bo Schembechler."
In 2007, Herrmann worked with a Jets defense that finished seventh in the NFL against the run (94.9 yards per game) and 16th overall (329.4).
Herrmann said that his biggest adjustment will be learning the Giants' terminology on defense.
"It is learning a new language," he said. "A lot of it will probably be similar. A lot of it is the same, just a little tweak here or a tweak there, but, yeah, it is a language. It is like learning Spanish. You have to be able to go in and talk the language and then be able, once the players come in, to talk the correct language so everybody is on the same page."
He also has to get to know his players.
"I know what I have heard from Bill," said Herrmann, who spent three seasons with the Jets. "As defensive coaches, we don't really watch other defenses. So I am looking forward to watching film once we come back in as a staff and getting to see how they are as players, and then once they come in get a chance to meet them and start anew."
The Giants also said Thursday that linebacker and long snapper Zak DeOssie has been selected by NFC coach Andy Reid as a "need" player for the Pro Bowl.
"I am the luckiest kid on the planet," said DeOssie, who long snaps for Pro Bowl punter Jeff Feagles and will probably handle the snaps for PATS by John Carney, another Giants teammate. "I'm sitting at home and the next thing I know I am going to be playing in the Pro Bowl. It is very, very exciting."
The Giants also signed defensive tackle Anthony Bryant who has played in nine NFL regular-season games with Tampa Bay, Detroit and Miami since 2005.