Judon, a Pro Bowl linebacker, was removed from the game after referee Brad Allen announced the Pro Bowl linebacker made "intentional" contact with the official.
With 5½ minutes remaining, Judon got in the middle of a fight between Ravens cornerback Marcus Peters and Steelers wide receiver Diontae Johnson. In trying to stop being held back from behind by a coach, Judon swung his arm forward and hit the right arm of Patterson, who immediately pulled backward.
"I would never intentionally make contact with an official," Judon said in a statement after the game. "I was attempting to free my arm as I was being held back, and I inadvertently contacted the official's arm. My emotions were running high in the moment, and I take full responsibility for what happened. I need to do a better job of keeping my cool and not doing anything that negatively affects my team."
In a pool report, NFL head of officiating Al Riveron said it's up to the official to decide whether the level of contact "rises to a certain level" to warrant an ejection.
"In the official's opinion, that was enough to eject him," Riveron said.
Judon became Baltimore's third starter to leave in the first half. The Ravens lost two offensive linemen -- Ronnie Stanley and Tyre Phillips -- to ankle injuries in the first quarter. Coach John Harbaugh said after the game that Stanley's injury will sideline him for the rest of the season.
Harbaugh supported Judon in that the contact with the official wasn't intentional.
"I'm very confident that Matthew didn't touch an official on purpose," Harbaugh said.