Torrealba said he planned to appeal the penalty. That would allow him to keep playing until a hearing was held.
"Three days is way too much. I didn't mean to touch him," Torrealba said.
Torrealba was ejected in the ninth inning of Monday's 6-3 home loss to the Blue Jays for arguing with umpire Larry Vanover after taking a called third strike. He also was fined an undisclosed amount.
Major League Baseball said the penalty was to start Wednesday, when the Padres played Toronto, unless he appealed.
Torrealba said the brim of his helmet made contact with Vanover. "I wanted to let him know what I was thinking. He was really inconsistent. I don't think it was fair.
"I understand they're going to miss calls -- they're human like we are. But when you ask, they sometimes talk to you like you're a little kid. They are so defensive now. We've been having some really bad umpires and some really bad calls."
Torrealba is hitting .277 in 32 games this season as he splits time with Nick Hundley.