Wheeler visited the Raiders on Thursday and agreed to the deal a day later.
Wheeler played 13 games for Indianapolis last season, making 80 tackles and recording one sack. He played four seasons overall with the Colts with 182 tackles and two sacks.
"We're excited to add another veteran to complement our talented linebacking group," general manager Reggie McKenzie said. "He gives us a lot of athleticism and versatility. He's a player that can do a lot of things."
McKenzie said earlier in the week that strongside linebacker was one of the final starting positions that needed to be filled this offseason.
A hole was created there earlier this month when the Raiders released Kamerion Wimbley. Wheeler is not the pass rusher that Wimbley is but is considered to be a better run defender, which is a big need in Oakland.
Information from The Associated Press was used in this report.