NL manager Bruce Bochy asked Ramirez to be the replacement for the Philadelphia Phillies' Placido Polanco (back injury), but Ramirez opted to stick with his plan to spend the break at home in the Dominican Republic.
Ramirez had told reporters on Saturday night that he wasn't sure if he'd go if he was asked.
"I don't know what I'd say," Ramirez said Saturday. "My family has already gone to the Dominican Republic. I'm planning to go there and join them as well."
Ramirez's representative confirmed Ramirez would fly from Pittsburgh to the Dominican Republic.
Ramirez, who was an All-Star in 2005 and 2008, is batting .300 with 15 homers and 51 RBIs this season.
Fifteen players have withdrawn from the All-Star Game in recent days. Six are starting pitchers who started Sunday and therefore were ineligible, according to the rules, to pitch in the All-Star Game. But the remainder of the withdrawals came as a result of injuries.
Bruce Levine covers the Cubs for ESPNChicago.com and ESPN 1000.