As the game looked to be heading to extra time, substitute Ndiaye weaved past a couple of defenders before beating Fraser Forster with a low strike at the near post.
Tottenham, who beat Chelsea at the weekend to boost their hopes of a top-four finish in the Premier League, looked off the pace as they struggled to test Blades goalkeeper Wes Foderingham.
Richarlison blasted a shot over the bar from close range but failed to make an impact, forcing Spurs to turn to Harry Kane as the second half wore on, but even the club's all-time top scorer was unable to really test a fired-up Sheffield United.
Kane had just the one real chance to score when he rose unmarked to meet a cross in the final few seconds but somehow headed wide.
The Blades - who are chasing promotion to the Premier League - then held firm to secure a famous win.
Their reward is an all-Championship tie with Blackburn in the last eight.
The win was all the more impressive for Sheffield United, who are second in the Championship, after manager Paul Heckingbottom made eight changes with a sickness bug having affected his side.
The Blades had also received an additional setback in the hours before the game when Heckingbottom received confirmation of a touchline ban for his sending-off against Middlesbrough last month, meaning he had to watch Wednesday's game from the stands.
Spurs had also heavily rotated their side, with six changes from their 2-0 win against Chelsea at the weekend - their second successive Premier League victory that boosted hopes of a top-four finish.
Kane was named on the bench but they still had plenty of quality in their first XI, with £60m-signing Richarlison and Son Heung-min in attack.
But they struggled to trouble Sheffield United's disciplined defence, with Son and Richarlison wasteful with the few chances the visitors had.
Sheffield United grew in confidence as the game wore on and were not unnerved by the introduction of Kane just after the hour mark.
But having not tested Forster for 78 minutes it felt like a moment of magic would be needed to decide the tie, and that duly arrived when Ndiaye showed superb footwork to evade challenges inside the box to fire home the winner.
-- Courtesy of BBC Sport