Brown scored touchdowns of 3 yards and 1 yard, and Gray ran it in from 17 yard for the Longhorns (4-5, 3-3 Big 12), who got their sixth straight win over the Red Raiders.
Swoopes completed 13 of 25 passes for 228 yards, hitting Jaxon Shipley for a 4-yard score.
It was the first 100-yard rushing game for the Longhorns this season.
John Harris finished with a career-high 165 yards receiving. It was his third game this season of 100-plus yards receiving.
The loss assured Texas Tech (3-6, 1-5) a fifth straight losing Big 12 record with three conference games remaining.
Texas Tech backup quarterback Pat Mahomes left the game after getting leveled by Quandre Diggs. True freshman walk-on Vincent Testaverde Jr., son of Heisman Trophy winner Vinny Testaverde, finished the game.
Brown moved him into 12th place in school history for career yards rushing.
The win keeps both teams in contention for a bowl but the road is more difficult now for the Red Raiders, who need to win out against Oklahoma in two weeks, Iowa State and Baylor. Texas plays West Virginia, Oklahoma State and TCU.
De'Andre Washington finished with 97 yards on 16 carries for Texas Tech, which scored just one offensive touchdown.
Testaverde came in after Mahomes got hit by Diggs and fumbled. Mahomes got up wobbly and was taken to the locker room, apparently for concussion testing.
Mahomes started his first game in place of Davis Webb, who reinjured his left ankle in last week's loss to TCU.
Testaverde led the Red Raiders on a 75-yard drive, capped by a 6-yard touchdown by Kenny Williams. It was Williams' first rushing touchdown of the season; he was the Red Raiders leading rusher in 2013 and 2012 but switched to linebacker to start this year.
Defensive end Branden Jackson scored a touchdown for Texas Tech in the first quarter after Swoopes fumbled deep in Longhorn territory. It was the first fumble recovery for a touchdown for the Red Raiders since Will Smith in a win over Texas State last season.
Texas Tech is now 1-12 in November games dating back to 2011.
NCAA to allow transfers to play college football, basketball without sitting out a season, reports say
The NCAA Division I Council voted Wednesday to change the long-standing rule that forces transfer athletes in football and basketball to sit out a season at their new school, a report said.
Kimes tabs Lance as the QB with the highest ceiling
Mina Kimes explains why she believes that Trey Lance is the most exciting quarterback in the draft.
Florida State unveils elaborate football facility mock-up
Florida State unveiled new renderings for a long-anticipated stand-alone football facility that is part of a $100 million athletic department campaign.
Chauncey Golston's NFL draft profile
Check out the best highlights from Iowa DE Chauncey Golston's college career.
Marlon Tuipulotu's NFL draft profile
Check out the best highlights from USC DT Marlon Tuipulotu's college career.
Janarius Robinson's NFL draft profile
Check out the best highlights from Florida State DE Janarius Robinson's college career.