Graham, 43, joins newly hired head coach Josh McDaniels in Las Vegas after spending the past two seasons as the Giants' defensive coordinator under Joe Judge.
Graham replaces Gus Bradley, who is heading to the Indianapolis Colts to be their defensive coordinator after one season with the Raiders.
The Raiders have also hired a new quarterbacks coach in former New England Patriots quality control/QB coach Bo Hardegree and are retaining receivers coach Edgar Bennett, sources told ESPN.
McDaniels, in his introductory news conference on Monday, was asked what defensive scheme he planned on using with the Raiders.
"Most of it today is actually built out of the nickel anyway," he said. "We talk a lot about 4-3, 3-4, [but] 85 percent of the game is now 'sub' defense, so your decision-making process might be a little different than it was 15 years ago, when you were playing a lot more base defense."
Graham ran a 3-4 base with the Giants, while the Raiders have run a 4-3 scheme.
"We'll make those considerations and determine how we want to go forward here as we kind of build the staff and move along with the team, but I think right now it's built in that mode," McDaniels said.
"But like I said, most of these games you're playing a four-man line and nickel defense anyway, so that's almost become your base.
"So, the vision for us is going to need to be -- there's five starting defensive backs now, and there's two linebackers and so on and so forth. We'll do the due diligence as we hire our staff and then evaluate our team and decide what's best as we move forward."
Graham was with the Patriots from 2009-2015, serving as a defensive assistant as well as the defensive line and linebackers coach in his time there, which coincided with McDaniels' return to New England in 2012.
Under Bradley, the Raiders defense improved from the No. 25 overall defense in 2020 to No. 17 last season, but was last in the NFL in red-zone defense under Bradley (83.3% touchdowns and 6.29 points per red-zone-zone trip) and was next-to-last in red-zone takeaways per red-zone drive.
Raiders Pro Bowl defensive end Maxx Crosby was asked after the AFC's practice on Thursday what he wanted to see in a defensive coordinator.
"I love Gus and I feel like he has us going in the right direction," Crosby said. "Obviously, I don't know, I don't have any control of that, but we'll see. We'll have to see."
Graham also worked with new Giants head coach Brian Daboll from their time together (2013-15) in New England. Daboll said the team had hoped to keep Graham if he did not land a head-coaching job elsewhere.
"I've had a good relationship with Pat for some time in this league," Daboll said at his introductory news conference earlier this week. "He's very diligent. He's smart. I think the players respect him. He understands different defenses and I have a good working relationship with him. I did when I was back at New England.
"Certainly, we hope that he has an opportunity to become a head coach. I think that's everybody's dream, but selfishly, I would love him to be here. He offers a lot to our program. I think he'd be a great support system for me and I'm hoping that that works out."
The Giants' defense was ranked 12th in the NFL in 2020 under Graham. It was 21st this past season. Graham is held in high regard by the Giants organization and had interviewed for its head-coaching position.
NFL Network first reported Graham's hire by the Raiders.
ESPN's Jordan Raanan contributed to this report.