Owner of six major trophies but never the champion at Roland Garros, Djokovic is trying to become the eighth man to complete a career Grand Slam.
He's been to the semifinals five of the past seven years in Paris, losing to Rafael Nadal in the 2012 final.
Djokovic needed only about 1½ hours to eliminate Chardy and improve to 9-0 against the Frenchman. Djokovic has won all 22 sets they've played.
Federer, the 2009 French Open champion, has not reached a Grand Slam final since winning his 17th major title at Wimbledon in 2012.
The fourth-seeded Federer played in two clay-court tournaments before coming to Roland Garros. He reached the final in Monte Carlo but lost his first match in Rome. He missed the Madrid tournament because of the birth of his second set of twins, Leo and Lenny.
No. 8 Milos Raonic of Canada, No. 6 Tomas Berdych of the Czech Republic, No. 18 Ernests Gulbis of Latvia, No. 25 Marin Cilic of Croatia, No. 27 Roberto Bautista Agut of Spain, No. 29 Gilles Simon of France and No. 31 Dmitry Tursunov of Russia also advanced to the third round.
One match that carried over from the first round ended quickly Wednesday with Steve Johnson of the United States advancing. The American saved two match points on Tuesday in the third set before completing a 4-6, 6-7 (7), 7-6 (3), 6-3, 6-3 win over Laurent Lokoli of France.
The match had been suspended because of darkness on Tuesday night with Johnson leading 3-1 in the fifth.