Caldwell turned the Ravens' offense around in 2012, when he replaced Cam Cameron as offensive coordinator in December. Under Caldwell, the Ravens averaged 31 points and 410.3 yards in the playoffs.
He didn't have the same success with the Ravens this past season. In his first full season as the Ravens' playcaller, the offense ranked 29th in total yards (307.4) and 25th in scoring (20.0 points).
If Caldwell leaves, the Ravens will have a decision on how to replace him. The only coach on staff who has experience as an NFL offensive coordinator is wide receivers coach Jim Hostler, who was the San Francisco 49ers' offensive coordinator in 2007. The Ravens will likely have to look outside the organization and could consider someone like Norv Turner if he is released from his contract with the Cleveland Browns.