The 15th-seeded Stephens made 35 unforced errors, 10 more than Halep, and simply could not hold serve, getting broken in six of nine games.
The match was tied at 4-all in the first set when Halep took control by winning the next four games.
"It's always tough playing someone who is really consistent and has a lot of confidence and just plays a solid game," said Stephens, who is based in Florida.
"I didn't execute my game as well as I thought I could," Stephens continued. "That's a little bit disappointing, but obviously that happens."
Stephens was broken in six of the nine games she served and made 35 unforced errors, 10 more than Halep.
"I dominated the match, I think," Halep said. "I didn't serve very well, but I tried to return better -- and I did."
Halep, the highest remaining seed, reached the quarterfinals at Roland Garros for the first time.
The 28th-seeded German was broken three times in the first set, once in the second and three more times in the third. But she was able to earn seven breaks of her own -- all in the final two sets -- to advance to the quarterfinals at Roland Garros for the second time in her career.
"I was really relieved that I picked myself up after I was up 4-2 in the third and I started playing a little passively," Petkovic said. "I wasn't going for my shots anymore. I was sort of hoping that she's going to miss. She's too good for that.
"After going down 5-4 in the third, I just sort of told myself, 'OK, listen, you have to play aggressive. You have to go for your shots, and nobody's going to give you the quarterfinals just because you're nice, so go gain it.'"
Bertens, a Dutchwoman ranked No. 148, had been bidding to become the lowest-ranked woman to reach the French Open quarterfinals since the field at the clay-court major was expanded to 128 players in 1983.
After splitting the first two sets, the players exchanged breaks quickly and often in the third. Each of the first five games went to the returning player, with Petkovic finally holding to lead 4-2.
Both players were broken once more, but Petkovic was able to hold in the last game with a forehand winner on match point.
"It was super tough, because I didn't know her at all. I just stalked her on YouTube yesterday the whole day, and I was already impressed yesterday," Petkovic said. "I think she's going to be a force on the WTA when she gets fitter and recovers from her injuries that she's had."
Petkovic reached the quarterfinals at Roland Garros in 2011. She reached the same stage at the Australian Open and US Open that year for her best Grand Slam performances.
The Associated Press contributed to this report.