It is the second consecutive World Cup where Germany have exited at the group stage after they finished bottom of their group in Russia in 2018
Japan's controversial 2-1 victory against Spain would have eliminated Germany anyway, but Hansi Flick's side needed three points to give themselves any chance of reaching the last 16.
They completed their part of the bargain, but it was to no avail as the four-time champions finished third in the group and were eliminated on goal difference.
It could have been a different story after Japan's winning goal in the other Group E match was initially ruled out because the officials thought the ball had gone out of play before it was put into the net, but the decision was overturned by VAR.
Germany were in complete control and in front through Serge Gnabry's 10th-minute header, only for Costa Rica to turn the match on its head to raise the remarkable possibility of Spain also going out.
Yeltsin Tejeda equalised after Germany keeper Manuel Neuer pushed out Kendall Waston's header in the 58th minute, before Neuer was deemed to have the final touch for the second goal 12 minutes later when Juan Pablo Vargas bundled in from close range.
At that stage, Costa Rica were going through with Japan, and Spain were heading for the exit.
It would have capped one of the most astonishing nights in World Cup history, but Germany substitute Kai Havertz restored a measure of normality with two clinical finishes to extinguish Costa Rica's hopes of springing a major shock.
Substitute Niclas Fullkrug added a last-minute fourth to confirm victory, but it will be of no consolation as Germany failed to reach the knockout stage of the tournament for the second time in succession.
This was also a match that made history as an all-female officiating team took charge of a men's World Cup game for the first time, with France's Stephanie Frappart refereeing, joined by assistant referees Neuza Back from Brazil and Karen Diaz Medina from Mexico.
Germany's second successive exit at the group stage will come as another shock to their system and they can have no complaints as they leave Qatar.
The skids were under their campaign when they lost 2-1 to Japan in their opener and, despite showing real character to come from behind to draw 1-1 with Spain, the first-game defeat left them vulnerable.
A Germany win looked a formality as they controlled the first half hour with embarrassing ease and led through Gnabry's strike.
Costa Rica were overpowered but Germany's complacency gave them an entry into the game and suddenly it was they who looked like qualifying as they went ahead, although it did not last long.
Germany have some fine players in the brilliant Jamal Musiala, who was outstanding and hit the post twice in the second half, but this World Cup may spell the end for the older generation such as goalkeeper Neuer and forward Thomas Muller. Neuer was uncertain and could have done better with both goals while Muller missed an easy headed first half chance and was substituted after the break.
There is certainly likely to be a lengthy inquest into another failure at the finals.
Costa Rica looked for all the world like they were heading for a heavy defeat, in the manner of their 7-0 loss to Spain, in the opening half as they were just unable to deal with the Germans.
Only the brilliance of Keylor Navas' goalkeeping kept Germany at bay, but what resilience Costa Rica showed to hang in and take their chances when they were presented with them.
And the world of football watched in anticipation of a gripping finale after Neuer's own goal put them 2-1 up, leaving Luis Suarez Fernandez's side on course for a remarkable qualification and Germany going out along with Spain.
The dream only lasted three minutes, but would have felt like an age for those two giants of the game.
Germany had too much quality for Costa Rica in the end, but what a superb effort this was and they have a victory over group winners Japan to savour on their sixth appearance at the World Cup.
-- Courtesy of BBC Sport