Gilberry sacked Manziel deep in the backfield on the Browns' second drive Sunday and displayed the quarterback's signature move to the FirstEnergy Stadium crowd.
Manziel said he "didn't even notice" the taunting, but Bengals players said their gestures were premeditated.
"He kind of brought that on himself," Gilberry said. "He's a great athlete, a good quarterback, but today was more about us than him."
Later in the game, Bengals linebacker Rey Maualuga got in on the action by flashing money signs in Manziel's face after batting down a pass. The gesture cost the Bengals a 15-yard unsportsmanlike conduct penalty and gave the Browns new life. Two costly Bengals flags helped the Browns move into field goal range on the drive, including an offside play negating an interception.
Reserve defensive tackle Brandon Thompson also flashed the money sign as he got up after sacking Manziel in the third quarter.
Manziel struggled mightily in his starting debut, completing 10 of 18 passes for just 80 yards and two interceptions in Cleveland's 30-0 loss to rival Cincinnati.
"It was tough," he said. "When you get beat, you get beat and you take it like a man. I felt like it was a fail on my part. "
Browns coach Mike Pettine did not mention the taunting but said that Manziel "played like a rookie."
"(Manziel) didn't play well," Pettine said. "Looked like a rookie, played like a rookie."
Cincinnati defensive tackle Domata Peko said Browns fans were flashing the money signs to the Bengals' team bus as it pulled up to the stadium. The Bengals didn't like that, and it showed.
"This ain't college," Peko said. "This is the NFL. You have grown men that are going to be chasing for you, not college kids."
The Browns recorded three-and-outs on Manziel's first two drives as an NFL starter. On Gilberry's sack, Manziel appeared ready to hand off to running back Isaiah Crowell but kept the ball, allowing Gilberry to quickly pounce.
Manziel started the game 1-of-3 for 4 yards and an interception in his first three drives, including an accurate throw on the run to receiver Andrew Hawkins that was dropped and a pass to Hawkins by the sideline that cornerback Dre Kirkpatrick picked off. He later was picked off in the end zone by Pacman Jones.
ESPN.com Browns reporter Pat McManamon and The Associated Press contributed to this report.