Finnegan set it off by pushing up Johnson's face mask at the line of scrimmage. Johnson ripped off Finnegan's helmet and landed at least two punches to Finnegan's head and neck. Finnegan tore off Johnson's helmet before players and referees intervened.
"He kept doing little things and I told him: 'Just because you're frustrated, you need to stop what you're doing,'" Johnson said. "I guess he thought it was funny."
The usually soft-spoken Johnson caught nine passes and a touchdown before being ejected.
"It's just a buildup of things over plays," Johnson said. "I just lost my cool."
Finnegan did not speak to reporters after the game.
"It's not good for the game, no," said Titans coach Jeff Fisher, co-chairman of the NFL's competition committee. "You don't want to see that stuff."
The game was halted for more than five minutes as officials sorted out the chaos. Johnson received a standing ovation as security guards escorted him off the field. Finnegan taunted booing fans as he exited out the opposite tunnel.
Fisher said he was confused by the ejection for his player as Finnegan said he never took a swing.
Johnson apologized for the brawl after the game.
"I would like to apologize to the organization, our owner, my teammates," he said. "What happened out there wasn't me. I just lost my cool and I wish I could take back what happened but I can't. I'm pretty sure I'll be disciplined for it, I'll have to deal with it from there."
The two had been pushing and jawing earlier in the game, and Johnson has called Finnegan's antics annoying in the past.
Last season, Johnson was fined $7,500 for taking Finnegan to the ground by the facemask during a scuffle after a play in a 34-31 Texans win.
Johnson said this week: "That's the past, I'm over that. That's way behind me, I don't think about that."
Texans owner Bob McNair was dubious when asked if he feared a suspension for Johnson.
"I don't know why there would be," McNair said. "The DB was all over him and he's the one who initiated it and he'd been doing it the play before and the whole game. He just went a little too far and Andre's the one who was on camera when the action heated up. That's not Andre."
The incident came at the end of a tumultuous week for the Titans, who've lost four in a row.
Titans rookie quarterback Rusty Smith was intercepted three times in his first career start.
Smith replaced Houston native Vince Young, who argued with Titans coach Jeff Fisher after he was taken out of last week's loss to Washington with a thumb injury. Young was placed on injured reserve and apologized to Fisher via text message. Young did not join his team for Sunday's game.
The Texans led 17-0 at the time of the ejections and won 20-0.
Information from ESPN.com's Paul Kuharsky and The Associated Press contributed to this report.