Djiby Baye Fall scored on a bizarre play in front of the Columbus goal in which he had two chances to head the ball in and made the most of the second effort for the game's only goal.
Crew SC fielded a strong side and were a constant menace to Cincinnati goalkeeper Mitch Hildebrandt, who made a string of fine saves in the second half to preserve the win for his team.
And NASL club Miami FC were next to get in on upsetting an in-state top-flight rival, knocking off Orlando City 3-1.
Stefano Pinho's hat trick staked the visiting side to a 3-0 lead, before Giles Barnes pulled one back for Orlando in the 79th minute.
Sacramento Republic also taught their MLS rivals a lesson, downing Real Salt Lake 4-1 at Papa Murphy's Park.
First-half goals from Trevin Caesar, Agustin Cazarez and Jeremy Hall handed the hosts an unassailable 3-1 lead at the break, while Danny Barrera made sure of the win with a fine finish 20 minutes from time.
A rotated New England Revolution XI cruised to a 3-0 win against 10-man Rochester Rhinos at Providence College.
Teal Bunbury opened the scoring for the home team from the penalty spot minutes after Rochester's Joseph Farrell was sent off after receiving a second yellow card.
Rapid-fire second-half goals from Donnie Smith and Zachary Herivaux wrapped up proceedings on the night, with the Revolution advancing to the fifth-round.
Daniel Royer did the honors for the hosts at Red Bull Arena, pouncing on a rebound from a Bradley Wright-Phillips shot and firing past Sean Johnson for the game's decisive goal.
The Philadelphia Union ran out 3-1 winners against the Harrisburg City Islanders in their fourth-round matchup at Talen Energy Stadium.
Marcus Epps added to the hosts' lead just two minutes after the break, leaving Harrisburg City to chase a pair of goals to equalize that never came.
Defending U.S.O.C. champions FC Dallas survived a scare from the Tulsa Roughnecks, needing a stoppage-time own goal from the USL club to secure a 2-1 win.
Coy Craft started the scoring for Dallas in the 31st minute, smashing home a Ryan Hollingshead assist from outside the penalty area.
But Tulsa's Kosuke Kimura levelled matters in the 59th minute and the visitors looked set to force extra time before Mou Jadama knocked the ball into his own net to let Dallas off the hook.
In another nervy finish, the Chicago Fire held on for a 1-0 win against St. Louis FC, which finished the match with 10 men.
Luis Solignac scored a first-half goal and Dragan Stojkovic was sent off for a hard foul in the 80th minute that earned him a second yellow card, further hampering the home team's chances of getting back in the match.
Sporting Kansas City had four players on the scoresheet in a runaway 4-0 win against fellow MLS club Minnesota United.
Ike Opara and Gerso had the Western Conference leaders up 2-0 at the break before Dom Dwyer and Daniel Salloi added second-half goals to seal the deal for SKC.
The Houston Dynamo pulled off a 3-2 win against North Carolina FC, but needed extra time, a prolonged weather delay and a late winner from Memo Rodriguez to do it.
The game was tied at 2-2 in the 88th minute when lightning in the area forced the match to be delayed for an extended period.
Rodriguez teed up a poor clearance from NCFC in the second extra-time period and fired it past Brian Sylvestre to spared the Dynamo blushes.
Atlanta United was made to work for passage to the next round in a 3-2 win against the Charleston Battery.
Forrest Lasso got Charleston out to an early lead, heading home from close range in the third minute to put his team up 1-0.
But the visiting Battery struck right back to retake the lead just one minute later, as Tah Anunga set up Dante Marini for an unstoppable 20-yard shot that gave his team a 2-1 lead at half-time.
Martinez knocked home the equalizer for Atlanta just after the break, heading a Brandon Vazquez cross into the upper-left corner of goal to make it 2-2.
And Vazquez was the man of the hour yet again, completing the rally for Atlanta and knocking home a Mikey Ambrose assist from very close range to put Tata Martino's team back on top for good.
In Wednesday's late games, the LA Galaxy secured a 3-1 win over Orange County SC, while goals from Jackson Yueill and Cordell Cato ensured the San Jose Earthquakes overcame the San Francisco Deltas 2-0.