The team officially placed McCown on injured reserve on Wednesday and re-signed wide receiver Terrelle Pryor, who has been with the team in training camp.
"I don't want to head down this slope," he said. "I'm going to reserve talk for the quarterback stuff until [Wednesday]."
Pettine said he will decide who starts without influence from the front office.
Manziel was the third-team quarterback against the Ravens because he had been disciplined for his decisions during the team's bye week and because he lied to the coaching staff about a video that appeared of him partying in Austin, Texas.
When the Browns moved Manziel from starter to third team, the indications were that it would be the way they would approach the rest of the season. Within a couple of days, though, Pettine and offensive coordinator John DeFilippo said they could envision Manziel playing again for the Browns this season.
Davis was the backup and nearly guided the Browns to a win over Baltimore after replacing McCown in the fourth quarter. Davis threw a 42-yard touchdown pass and had the Browns in position to try a game-winning 51-yard field goal. The kick, though, was blocked, and the Ravens returned it for a touchdown on the last play of the game.
"You're poised for thrill of victory, and then you're tasting agony of defeat all within a span of 10 to 15 seconds," Pettine said. "I can't imagine too many game-ending scenarios that would make you feel worse at the end than that."
Pettine said he would "take everything into consideration" when deciding which quarterback starts.
"Make the decision for this week that's best for the team," he said.
McCown's fracture will not need surgery. He played well for the Browns, but he struggled with a concussion, a rib injury and now the collarbone injury during the season.
"I cannot say enough positive things about Josh McCown, what he's done for this football team and what he means to this football team," Pettine said.
In eight games, McCown completed 186 of 292 passes for 2,109 yards with 12 touchdowns and four interceptions. His final QB rating was 93.3.