No penalty was called on the play and Anderson, who is listed day to day, had to be helped to the locker room.
"My heart sank into my stomach when it happened," said defenseman Marc Methot, who got tangled up with Kreider on the play. "It's the last thing we need. He's basically been the heart and soul of our team, or at least a large part of it. It's unfortunate, but it's the luck we're getting right now as a team."
The 31-year-old Anderson leads the NHL with a 1.57 goals-against average and a .950 save percentage, but Senators coach Paul MacLean says the only thing the team can do is look at the injury as yet another opportunity for someone else in the organization.
That appears to be goaltender Robin Lehner, who will be recalled from Binghamton of the American Hockey League and is expected to arrive in Ottawa on Friday.
"It's creating great opportunities for young players," MacLean said. "We've always talked about what great depth we have in this organization and how many good young players that we have playing in Binghamton and we're finding out that we do have a lot of great young players because now they're all playing here in Ottawa and they're finding a way to win a game."
Anderson didn't speak to reporters after the game, but later provided an update on his Twitter account.
"My ankle's gonna be ok, be back (as soon as possible)," he wrote.
Lost in the wake of Anderson's injury, was the fact the Senators won their third straight game.
Daugavins appeared to fan on his winning shootout attempt that fooled Rangers goalie Henrik Lundqvist, and the puck went into the net between the legs.
Kreider then stepped up needing to score, but couldn't beat Bishop.
"I was actually a little lucky," Daugavins said with a smile.
Silfverberg and Mika Zibanejad scored for the Senators in regulation. Anderson stopped all 21 shots he faced before being replaced by Bishop, who allowed two goals on 13 shots.
Callahan and Ryan McDonagh scored 51 seconds apart in the third period for the Rangers. Lundqvist made 35 saves.
Bishop disliked seeing Anderson get hurt, but added there was little time to dwell on it.
"You want to know if he's OK at first, but once the puck drops you've got to do your job," Bishop said. "It's one of those things that as cold as you can be once the puck drops, you're in the game."
The Rangers tied it 1-1 at 10:44 of the third period on a power-play goal by Callahan and then quickly took the lead on McDonagh's shot from the point.
Callahan's goal was his fourth of the season. McDonagh netted his second.
Zibanejad tied the game with a power-play goal from in close at 13:30.
"We got that first one and then we were able to get the lead and at least we got a point," Rangers defenseman Marc Staal said. "We found a way to get back in the game."
Thursday was the first meeting between the teams since the Rangers eliminated the Senators in Game 7 of last year's Eastern Conference quarterfinal series.
The Senators opened the scoring at 6:48 of the first period on a short-handed goal by Silfverberg. The Swedish rookie beat New York defenseman Michael Del Zotto in a race for the puck in the Rangers end, looked up as if he was going to pass, but instead beat Lundqvist with a backhand into the top corner.
"It gives me a lot of confidence," Silfverberg said. "(Lundqvist) is one of the best goalies in the league and he's one of the best goalies Sweden ever had."
Del Zotto then had a great chance to tie it on the power play, only to be stopped by a sprawling forward Daniel Alfredsson in the Ottawa crease.
The game slowed down in the second with neither team generating many chances. Silfverberg had a decent opportunity early in the period, but Lundqvist made a good save.
In addition to from Karlsson and Spezza, Ottawa was without LW Milan Michalek (knee, day to day), C Peter Regin (day to day), D Jared Cowen (hip, out for season), RW Guillaume Latendresse (whiplash, day to day). D Mike Lundin and C Jim O'Brien were healthy scratches. ... The Rangers were missing RW Rick Nash (undisclosed, day to day), C Darroll Powe (concussion, indefinitely), and D Michael Sauer (concussion, indefinitely). D Steve Eminger and RW Arron Asham were healthy scratches.