He has the most to lose since coverages that were focused on Steve Smith, often leaving the middle of the field open, could be shifted to the tight end.
But Olsen isn't concerned -- or if he is he's not showing it.
"I know everyone at one point was kind of panicking," Olsen recently told the Charlotte Observer at a screening of the movie "Draft Day." "Would it have been nice to have those [receivers] back? Of course.
"But I think we’ve signed a lot of guys that can fill a lot of those roles. We’re putting it together. It’s hard to judge a team in March. When the season gets closer, that will be a better example of what our team is.”
That left them without a receiver on the roster that caught a pass last season.
They have since signed Jerricho Cotchery (Pittsburgh), Jason Avant (Philadelphia) and Tiquan Underwood (Tampa Bay), but none of those receivers has put up statistics close to what Smith did during his career. None are considered a No. 1 receiver. Most were a second, third or even fourth option last season.
Again, Olsen isn't concerned even though he now will become a central piece in game plans to stop the Carolina passing attack. If anything he's optimistic because the team signed Baltimore free agent tight end Ed Dickson, opening up the possibility of more two-tight-end sets.
Quarterback Cam Newton threw for a career-high 4,051 yards as a rookie in 2011 with Olsen and Jeremy Shockey running a lot of two-tight-end sets. They combined for 82 catches for 995 yards and nine touchdowns.
"There is a plan," Olsen said. "We have to trust in that. Mr. Gettleman's done an awesome job since he’s gotten here in a short time putting pieces in place to fill holes. And doing so with guys other people maybe overlooked. Last year, a lot of the guys that came in were in that type of situation and were huge parts of our team.”