Low, 61, who was contracted until 2022, will leave the German FA after 17 years with the national team and 15 years as the head coach. He has coached them for 189 games, having won 120 and lost 31. The news means Germany will require a new manager to take them through to the 2022 World Cup in Qatar.
Low first joined the team in 2004 as assistant to Jurgen Klinsmann before taking over in 2006 and led them to the Euro 2008 final before winning the World Cup six years later.
"Joachim Low will vacate his position as the Bundestrainer after the 2021 European Championship," a statement on the DFB website read. "The Bundestrainer asked to cancel his contract, valid until 2022, immediately after the tournament. The DFB has agreed to it."
Low added: "I take this step very consciously, full of pride and enormous gratitude, but at the same time continue to be very motivated when it comes to the upcoming European Championship tournament.
"Proud, because it is something very special and an honour for me to be involved in my country. And because I have been able to work with the best footballers in the country for almost 17 years and support them in their development.
"I have great triumphs with them and painful defeats, but above all many wonderful and magical moments -- not just winning the 2014 World Cup in Brazil. I am and will remain grateful to the DFB, which has always provided me and the team with an ideal working environment."
An exciting semifinal run with big wins over England and Argentina at the 2010 World Cup were the first clear signs of Low's style that would conquer the world four years later.
Their 7-1 demolition of hosts Brazil in the semifinal of the 2014 tournament stunned the football world and was arguably the best performance by Low's team with its quick passing style that dismantled the Brazilian defence in a sensational first half.
Doubts over Low's future grew following a 6-0 defeat to Spain in the UEFA Nations League in November. They had begun following Germany's failure to recover from the 2018 World Cup exit at the group stages, a first for Low and his staff.
"I was able to work with the best players of the country for nearly 17 years and be part of their development," Low said and highlighted the "many wonderful and magic moments" he had with the team.
DFB president Fritz Keller said that Low's announcement now made it possible "to announce his successor with the requisite time, with calmness and with sound judgement."
Liverpool coach Jurgen Klopp and Bayern Munich's Hansi Flick have been touted as potential successors in recent months, while Germany great Lothar Matthaus was suggested as an ideal fit for the position by former teammate Mehmet Scholl. Germany under-21 coach Stefan Kuntz has also been linked with the job in the past.
Asked about Klopp's chance of replacing Low, RB Leipzig coach Julian Nagelsmann told a news conference on Tuesday: "Klopp is, of course, an exceptional coach but I think Liverpool and Jurgen Klopp are content with their partnership, so I don't know if that's a good question for me. You should ask that to Jurgen."
Euro 2020 was postponed until 2021 because of the coronavirus pandemic, but the tournament kept its original name. UEFA said last month the tournament would go ahead, although doubts remain about which countries will host it.
Germany face France, Hungary and Poland in their Euro group matches in June.
Information from Reuters was used in this report.