Rasmus Hjolund and Scott McTominay celebrate
United made a flying start and looked to have wrapped the tie up when a smooth finish from Bruno Fernandes and Kobbie Mainoo's first senior goal gave them a 2-0 lead inside 13 minutes.
Alejandro Garnacho also hit the woodwork as United dominated but the team 73 places beneath them on the league ladder, roared on by passionate Welsh followers inside a packed Rodney Parade, staged a stirring fightback.
Bryn Morris' long-range effort glanced off Lisandro Martinez's head past United's debutant keeper Altay Bayindir after 36 minutes and Newport really started to dream of causing a sensation two minutes after the break.
Adam Lewis' cross allowed Will Evans to steal in ahead of Raphael Varane to put Newport level as the hordes of 'The Amber Army' celebrated wildly.
United were stunned but managed to regain enough composure to regain the lead in the 68th minute, as Antony reacted first to beat Newport goalkeeper Nick Townsend after Luke Shaw's effort came back off the post.
Rasmus Hojlund made the result safe for Erik ten Hag's team, finishing from close range after substitute Omari Forson had a shot blocked by Townsend.
This time the hosts had no response, leaving United to be grateful for their escape and the chance to look forward to a fifth-round tie at either Bristol City or Nottingham Forest.
-- Courtesy of BBC Sport