Southampton, thrashed 9-0 by Leicester City last week and beaten 3-1 by City in the League Cup in midweek, took a shock lead when James Ward-Prowse scored in the 13th minute.
City spent the rest of the game battering at Southampton's defence to little effect but finally equalised in the 70th minute when Aguero turned in Walker's cross.
Southampton dug deep trying to hang on for an unlikely point but were breached again when Walker slid the ball home in the 87th minute.
"It was a crazy game to play, scoring was one of those crazy moments. It's always hard playing against 11 behind the ball," said Walker, who scored his first goal of the season.
"We kept knocking on the door and thankfully for me the door came open and I was the one that got the goal."
Pep Guardiola told the BBC after the match: "We did everything, it was so difficult to attack when they defend with 11 players."
"We did everything we could and in the end our intensity and our players paid off. We made a good comeback. Everyone plays the way we want to play. They are top quality players, they are fast players, it was our job to try and beat Southampton and we did.
"It is not about what Southampton do or play but it is about how we reacted to that. Today we have to do everything. We have to take the rhythm that we needed forward in every sense of the world from how we pick the ball out of the net to how we take throw ins or corners."