Columbus tied the game with less than three minutes left in regulation, and the Rangers lost in extra time for the fifth time in their last six home games.
"It's really frustrating. I don't know what to say anymore," said Lundqvist, who made 25 saves. "That's how it's been going the last few weeks. It's only on us."
The Rangers took a 3-2 lead on Chris Kreider's second goal of the game at 12:06 of the third period and appeared to go ahead by two several minutes later on a goal by Ryan Strome, but Columbus coach John Tortorella challenged the play and it was ruled offside.
Blue Jackets defenseman Zach Werenski then tied it at 17:40 when he scored his seventh of the season after the puck went back to him at the point and he fired it past Lundqvist.
The 20-year-old Dubois won it with his 14th goal of the season.
"We found a way," Tortorella said.
Backup goaltender Joonas Korpisalo finished with 19 saves for Columbus.
The teams played a generally haphazard and at times listless game after the three-day Christmas break. And once again, the victory margin was narrow. Sixteen of the last 18 games between the Rangers and Blue Jackets have been decided by one or two goals.
"I thought both teams didn't have a whole lot of juice," said Tortorella, the former Rangers coach. "We have to try to keep getting better each day."
Columbus struck first when Oliver Bjorkstrand scored at 9:17 of the first. Bjorkstrand took a pass from Boone Jenner and beat Lundqvist with a shot from the faceoff circle to the goaltender's right. Cam Atkinson also had an assist.
Jimmy Vesey tied it for the Rangers at 13:59 with his 10th of the season. Vesey took a pass from Kevin Hayes on a 2-on-1 break before tapping the puck past Korpisalo. With the assist, Hayes extended his point streak to eight games. Vladislav Namestnikov also assisted.
Columbus went ahead 2-1 when David Savard's long-range shot from the right circle beat Lundqvist at 16:08. The goal was the second for the Blue Jackets' defenseman. Bjorkstrand and defenseman Scott Harrington assisted.
The Rangers tied it with a power-play goal by Kreider at 1:45 of the second. Kreider deflected a soft shot from the point by Mats Zuccarello. Hayes also added another assist. The 26-year-old forward leads the Rangers with 31 points.
Hayes has five goals and 10 assists during his eight-game scoring streak and also has nine points in his last nine games against Columbus.
"When you lose, no one is happy," Hayes said. "You want to get two points every night."
The Rangers were coming off a 3-2 home shootout loss to Philadelphia on Sunday. Starting with a Dec. 2 shootout loss to Winnipeg, their home woes have escalated and also included overtime losses to Arizona and Vegas.
The Rangers have points in seven straight home games but only two of them wins, going 2-0-5 during that stretch. New York is 3-6-5 overall since Nov. 23.
And with a 4-10-2 road record, they are facing seven of their next nine games away from home -- where they haven't won in regulation yet this season.
"We certainly can't let this get in the way of us moving on," coach David Quinn said. "The bottom line is we had chances to do things right. That's the only thing we can control. We didn't execute what we were supposed to do."
Korpisalo made his first start since Dec. 11, when he lost 3-2 to Vancouver. Columbus had been 5-0-1 since behind starter Sergei Bobrovsky.
"We were a little inconsistent. It was one of those games where it was lucky for us to get the last goal," Bjorkstrand said. "These are huge points. They will mean a lot at the end of the season."
Korpisalo also played in the previous game between the Rangers and Blue Jackets when New York won 5-4 in a shootout at Columbus on Nov. 10. ... The Rangers and Blue Jackets meet again Jan. 13 in Columbus and April 5 at Madison Square Garden. ... The Rangers scratched defenseman Tony DeAngelo and injured forward Jesper Fast. ... New York again played without defenseman Kevin Shattenkirk, sidelined with a shoulder injury sustained Dec. 10 against Tampa Bay. ... Columbus scratched defenseman Dean Kukan and forward Lukas Sedlak.
Blue Jackets: Host the Toronto Maple Leafs on Friday.
Rangers: Visit the Nashville Predators on Saturday.
More AP NHL: https://apnews.com/tag/NHL and https://twitter.com/AP-Sports