The Rays would not elaborate on Rasmus' decision, saying the team was "respecting the privacy of Colby and his family."
The Rays do not expect him to return, sources tell ESPN.
Rasmus, who signed a one-year, $5 million deal with the Rays as a free agent in January, has played only 37 games this season after undergoing hip surgery in October.
"The Rays fully support Colby's decision," the team said in a statement. "... We are thankful for his contributions to our team. And we wish him and his family the best as they move forward."
Rasmus, 30, was hitting .281 with nine homers and 23 RBIs this season. He hit 165 home runs while playing for Houston, Toronto and St. Louis over the previous eight seasons.