Vladislav Namestnikov scored twice in the final four minutes, while also adding an assist. Jesper Fast and Boo Nieves staked the Rangers to a 2-0 lead in the first period, and Jimmy Vesey and Pavel Buchnevich also scored.
New York improved to 8-5-1 in its past 14 and earned its 58th point to move into 12th in the Eastern Conference standings.
Buffalo squandered a chance to close the gap on idle Pittsburgh in the race for the Eastern Conference's eighth and final playoff spot. The Sabres have 63 points and remain four behind the Penguins.
With the Rangers up 3-2, Georgiev was particularly sharp in the opening six minutes of the third period. First, he got across to his left to stop Rasmus Dahlin's one-timer 58 second in. Some five minutes later, the goalie squared up in his crease to stop Tage Thompson's shot from the slot.
Georgiev won his third straight start. He was coming off a 55-save performance in a 4-1 win over Toronto on Sunday.
Buchnevich put New York up 4-2 by tapping in Kevin Hayes' pass into the open left side with 4:01 left. Namestnikov then scored into an open net off the ensuing faceoff after Ullmark misplayed the puck. It caromed off Ullmark's stick and toward the front of the net to Namestnikov.
The Sabres closed their seven-game homestand at 3-3-1 while continuing a lengthy string of inconsistent outings.
Coming off a 3-1 victory over the New York Islanders on Tuesday, Buffalo was trying to win consecutive games for the first time since Dec. 11-13.
Skinner cut the Rangers' lead to 2-1 by one-timing in Jason Pominville's pass into the left circle 53 seconds into the second period.
The Rangers responded 54 seconds later, when Vesey jammed a puck inside the right post off a pass from Buchnevich.
Skinner then scored a power-play goal with 3:04 left. Dahlin corralled a bouncing puck and fed Skinner at the top of the right circle, and he knuckled in a shot that fooled Georgiev.
LW Connor Brickley made his Rangers debut a month after he was acquired in a trade that sent Cole Schneider to Nashville. Brickley played in place of rookie Filip Chytil, who was benched because of what coach David Quinn described as steadily declining play. ... Sabres D Zach Bogosian returned after missing two games with a lower-body injury. ... Quinn is pleased with Vesey's physical play of late, which is noticeably getting under the skin of opponents. "It's a funny sport. When you (tick) people off, they tend to come after you after the whistle," Quinn said. "So he's (ticking) people off with his play."
Rangers: At Pittsburgh Penguins on Sunday.
Sabres: Open three-game road trip at New Jersey on Sunday.
More AP NHL: www.apnews.com/NHL and www.twitter.com/AP-Sports