Ollie Watkins has scored in each of his last six Premier League away appearances for Aston Villa
The Foxes looked set to secure a valuable point in their first game since sacking Brendan Rodgers despite Kiernan Dewsbury-Hall's red card.
Ollie Watkins marked his 100th Premier League appearance by slotting Villa ahead with his eighth goal in 10 appearances after being sent through by Emiliano Buendia.
Villa had only conceded once in their previous five games but were undone by Harvey Barnes' excellent finish after he had shrugged off Ashley Young and cut in from the wing following Wout Faes' raking pass.
Dewsbury-Hall was sent off midway through the second half for a second bookable offence but the Foxes held on until substitute Bertrand Traore curled in Villa's 87th-minute winner.
The drama continued when Leicester were awarded a 92nd-minute penalty - only for referee Graham Scott to overturn his decision after looking at a replay on the pitchside monitor.
The alarm bells are ringing at the King Power, seven years after Leicester's extraordinary Premier League title triumph.
The Foxes are without a manager and remain in the relegation zone, two points from safety, after conceding a late winner for the second successive game.
Last Saturday they were sunk by Jean-Philippe Mateta's 94th-minute goal in Rodgers' final game in charge.
First-team coach Adam Sadler and goalkeeping coach Mike Stowell took charge against Villa but Leicester's season of hard-luck stories continued, this time by Traore's finish from outside the box.
Their hopes were raised by the awarding of a late penalty after a challenge between Watkins and substitute Patson Daka, only for referee Scott to change his mind after watching the replay following a VAR intervention.
With just nine games remaining, Leicester's fate remains in their own hands with five matches against teams near the bottom - starting with Saturday's visit of 18th-placed Bournemouth.
Whether they will have appointed a permanent replacement for Rodgers by then remains to be seen, but the Foxes desperately need to start picking up points.
"Until we hear different, Mike and I will continue to give everything for the club," said Sadler.
"We have been here a long time, we know the people inside out, they know us - so if it continues to be us, we will give everything for the club."
Leicester can only hope their new permanent appointment will make as positive an impact as Unai Emery has at Villa.
They were 17th when they sacked Steven Gerrard and replaced him with the former Arsenal boss last October.
Under Emery's guidance, Villa have won 10 of their 16 Premier League games and their latest success lifted them to seventh, with a place in Europe now a real possibility.
"Now we can add another step of thinking we can be a candidate to be in a European position - but it is going to be difficult because Brighton and Brentford are playing very well," said Emery.
"Chelsea and Liverpool have to be in Europe next year, they will fight for it, but we can have ambition and motivation to try to be a candidate."
Villa's sixth Premier League away win in eight games under the Spaniard epitomised the spirit and confidence flowing through this Villa team, while Watkins continues to impress in front of goal.
He has not appeared for England since March 2022 but was name-checked by Gareth Southgate when he announced his Euro 2024 qualifiers squad and he will be hard to ignore if he carries on his impressive scoring form.
-- Courtesy of BBC Sport