Brady, the No. 13 seed who was the runner-up in this year's Australian Open and a semifinalist at last year's US Open, asked for medical assistance before opting to stop the match.
She told reporters afterward that she was hampered by plantar fasciitis and a bone bruise in her left foot.
"I wasn't even really running for balls, just because I was in so much pain," said Brady, who said she had considered not even entering the Grand Slam event. "I wasn't serving at full capacity.''
The 24th-seeded Gauff, a 17-year-old based in Florida, will next face Ons Jabeur of Tunisia.
She has been as far as the fourth round at Wimbledon and the Australian Open but has never been past that stage at a Grand Slam tournament. She lost in the second round in her Roland Garros main-draw debut last year.
"Obviously, she's an incredible talent,'' the 26-year-old Brady said of Gauff. "She's so young. ... She's definitely continuing to climb the ranks, improve her game. She's really good. Yeah, I wish her all the best.''
Muchova had 23 winners compared to 22 for Stephens, but the Czech player made 32 unforced errors to 25 for the unseeded American.
Stephens lost the final at Roland Garros in 2018 after winning her only Grand Slam title at the US Open the year before.
She next faces former French Open doubles champion Barbora Krejcikova, who matched her best performance in singles at a Grand Slam by reaching the fourth round.
Kenin is chasing her first title at Roland Garros after losing last year's final to Iga Swiatek.
The No. 28-seeded Pegula rallied from a 0-3 start to win the first set, but Kenin began stepping into the court to take charge of rallies, especially with a backhand that produced two dozen winners. She hit 48 winners to 18 for Pegula.
"I'm proud of myself for just fighting out there and battling out," Kenin said.
Kenin had 10 double faults and was broken five times but held in her final four service games to close out the win. The 2020 Australian Open champion, who was sidelined by an appendectomy in February, improved to 10-8 this year.
"This whole year hasn't been so great in terms of my tennis," Kenin said. "I've had some early round exits. I'm just happy that I'm finally finding my rhythm and playing some good tennis again."
Swiatek rallied from a slow start for her win Saturday. Down a break in the opening set she regrouped and beat Anett Kontaveit 7-6 (4), 6-0.
Swiatek has won 20 consecutive sets at Roland Garros and that streak was in jeopardy when she trailed 4-2. She dominated from there and lost only 12 points in the second set.
"It's good to have matches like that, because it keeps you down to earth, and you have to be careful on every point,'' Swiatek said. "I'm just happy that I'm able to play really solid in really important moments.''
The eighth-seeded Swiatek next faces 18-year-old Ukrainian Marta Kostyuk, who has reached the round of 16 at a major event for the first time.
Krejcikova used a heavy forehand to upset fifth-seeded Elina Svitolina of Ukraine 6-3, 6-2 on Court Philippe-Chatrier. She thrust both hands in the air and shouted with joy after winning on her first match point.
Krejcikova also reached the fourth round in singles at Roland Garros last year.
The unseeded Krejcikova's best singles performance at any other major was reaching the second round at the Australian Open.
She is a former top-ranked doubles player with titles from the French Open and Wimbledon in 2018.
Information from The Associated Press was used in this report