Kyle Walker's first-half goal against his former club was enough to secure the three points for Pep Guardiola's side.
City climb up to seventh in the table while Chris Wilder's side continue their winless run and remain in the bottom three.
The visitors went ahead after 28 minutes when Walker unleashed low drive into the bottom corner.
Kevin De Bruyne almost doubled City's lead 10 minutes after the break but his shot went just wide.
With 20 minutes to go, John Lundstram missed United's best chance to equalise after his close range shot went over the bar.
The hosts only managed to register a single shot on target throughout the 90 minutes.
"I'm happy to get on the scoresheet and obviously happy to get the three points, which was the main thing coming up here," Walker told BT Sport after the game.
"They're well drilled. Everyone knows their position and they're a tough team to break down, but luckily enough I came up with the goal.
"My mum and dad live here, so if I'd celebrated I'd have got a lot of stick. I'm a Sheffield United fan, so I couldn't celebrate.
"I'm enjoying my football. We don't sit back and defend. We want to score three or four, but it's great to get the clean sheet on the board as well."
City boss Guardiola reiterated his desire to stay with the Premier League club and ruled out a potential return to his former club Barcelona when his contract expires at the end of the season.
"My career as a manager in Barcelona is over," Guardiola said postmatch. "There are incredible people who can be in charge.
"Ronald Koeman is an excellent manager. So it's over [for me]. I will come back to watch a game, but it's done. I am happy here, I have the desire to do well and this is the most important thing.
"Now there will be elections in Barcelona and hopefully they can choose the right person and the right board to maintain this incredible club as it is, at the highest level."