Houston, which has finished with the worst record in the majors each of the past three seasons, won a series at Yankee Stadium for the first time in four tries.
The fourth-place Astros improved to 4-7 in the Bronx and will go for a three-game sweep Thursday afternoon.
"We haven't been giving up toward the end of the game. We just keep grinding out at-bats, and things are working out for us," Grossman said.
Stephen Drew hit his first homer for New York, and Ellsbury drove in a run with a bunt single. But the Yankees (63-61) lost for the seventh time in nine games and dropped five games out of a playoff spot.
They are 2 for 17 with runners in scoring position during the series and have totaled 20 runs in their last nine games, never scoring more than four in that stretch.
"We've got to score more than two runs. It's hard to win when you only score two runs," manager Joe Girardi said. "Our pitching has been pretty good, but that's a pretty tall task."
Ellsbury flied out against Jose Veras with two on to end it. New York stranded 10 in all.
"I knew it was in the park. It was a great sinker," Veras said.
Feldman (7-9) matched a career high by throwing 121 pitches and worked out of trouble to last 6 2/3 innings. He gave up two runs and eight hits, striking out a season-best seven.
The right-hander tossed 6 2/3 shutout innings to beat the Yankees on opening day. Houston had lost his previous six road starts, but he entered with a 2.30 ERA during four straight quality starts against New York.
"He's a veteran guy and he's been around the game for a long time. He knows how to pitch," Fowler said.
Houston smacked five straight one-out singles, four off Rogers, in a span of 10 pitches.
"We have relied on the home run quite a bit, and today was a prime example of keeping the line moving," Astros manager Bo Porter said. "I felt like guys had great at-bats."
No. 9 batter Jake Marisnick tied it with a single to left, Grossman put the Astros up 4-2, and Altuve added an RBI single as the boos increased for Rogers.
"When the other guys are used a lot, guys have to step up," Girardi said. "It just didn't happen tonight."
Kevin Chapman struck out Ellsbury with runners at second and third to end the bottom half. Veras worked the ninth for his first save since Sept. 2, 2013, with Detroit.
"It was nice to see Chapman come in, probably the biggest part of the game, and pick me up. I owe him a dinner or something like that," Feldman said.
Seemingly resigned to trying small ball, the Yankees went 4 for 5 on stolen base attempts in the first five innings. But one pitch after Chase Headley was caught at second in the fourth, Drew hit his fifth home run of the season. The other four came with Boston before a July 31 trade.
Astros: Regular closer Chad Qualls was unavailable because of a stiff back. He is day to day.
Yankees: Slumping DH-RF Carlos Beltran sat out because of pain in his troublesome right elbow and was examined by a doctor. He had his third cortisone shot and is day to day. "Hopefully, this is the last one," said Beltran, who hopes to return to the lineup on Friday. "At the end of the year I have to address it and get the surgery to get it over with, but right now I'm just trying to finish the season." ... RHP David Phelps (elbow inflammation) played catch and will try to get ready to come off the disabled list as a reliever, Girardi said.
LHP Dallas Keuchel (10-8, 3.11 ERA) makes his first career start against the Yankees in the series finale. RHP Brandon McCarthy is 4-2 with a 2.30 ERA in seven outings for New York since he was acquired from Arizona in early July.
HERE COMES TROUBLE
Fowler is 9 for 20 (.450) with six extra-base hits against the Yankees this year.
Yankees LF Brett Gardner tumbled head over heels over a low retaining wall to make a courageous catch in the ninth.
Donaldson provides context to his 'Jackie' comments to Anderson
Josh Donaldson explains his reasoning behind calling Tim Anderson "Jackie Robinson" during the White Sox and Yankees game.
Varsho's double scores 3 in 10th in Dbacks' 7-6 win vs Cubs
— Daulton Varsho drilled a two-out, three-RBI double in the 10th inning then scored on a throwing error by Andrelton Simmons to give the Arizona Diamondbacks to a 7-6 comeback win over the Chicago Cubs on Saturday.
Musgrove sharp, Machado homers as Padres top Giants 2-1
— Joe Musgrove pitched seven scoreless innings of four-hit ball to remain unbeaten, Manny Machado homered and the San Diego Padres beat the San Francisco Giants 2-1 on Saturday.
Daulton Varsho dashes around the bases for Little League HR in the 10th
Daulton Varsho rips one into deep right field in the 10th inning scoring Ketel Marte, Jake McCarthy and himself.
Schwindel, Wisdom go back to back in the 2nd
Frank Schwindel and Patrick Wisdom put up back-to-back home runs in the second inning against the Diamondbacks.
Devers hits 2 HRs; Red Sox rally past Mariners 6-5
— Rafael Devers hit two homers, Christian Vázquez had a tiebreaking single in the eighth inning and Boston’s bullpen held the Seattle Mariners scoreless for six innings, lifting the Red Sox to a 6-5 victory on Saturday.