Mason Mount scored his first Chelsea goal of the season after Tyrone Mings' mistake
The England international was gifted an opener when Tyrone Mings' header fell to him for an easy finish after six minutes.
And in front of the watching England boss Gareth Southgate, Mount added a second from a 25-yard free-kick - but again he was helped after Villa keeper Emiliano Martinez was wrong-footed.
Villa, who had been unbeaten in four games, did everything but score in the first half but found Chelsea goalkeeper Kepa Arrizabalaga in inspired form as he denied Danny Ings, John McGinn and Jacob Ramsey.
Potter, who is now six games unbeaten since taking over as Chelsea boss last month, and whose side have won the last five, gambled by playing forward Raheem Sterling at right wing-back following a knee injury to Reece James, which is set to keep the defender out of the World Cup.
Potter said afterwards that James would be in a knee brace for four weeks.
But after seeing Villa dominate the first half without finding the net, Potter moved Sterling up front and enjoyed a more comfortable second half, which kept Chelsea in fourth, four points behind third-placed Tottenham with a game in hand.
Villa might have been let down by two costly errors, and were booed at the final whistle, but manager Steven Gerrard would have been encouraged by his side's first-half display, if not their finishing.
Potter had already experimented with Sterling as a wing-back, opting for the England international on the left in his first game after joining the club last month, a draw against RB Salzburg.
But against Villa it backfired as the hosts tore down the flanks, with Sterling leaving the right side of the defence exposed.
After Mount was gifted his goal by Mings, Chelsea were left chasing the ball. Marc Cucurella struggled as Ings, Ollie Watkins and Leon Bailey caused problems right across the attack, helped by the fact that McGinn was doubling up on Villa's left side.
The defence was indebted to Arrizabalaga, who made six first-half saves, including a brilliant stop to deny Ings' header from point-blank range.
Five minutes before half-time, Potter admitted his tactical plan had failed by moving Ruben Loftus-Cheek into the right wing-back position instead of Sterling.
And after Cesar Azpilicueta and Kalidou Koulibaly replaced Cucurella and Kai Havertz at the break, Chelsea looked a lot more solid. They could have gone further ahead when Sterling, restored to the attack, headed wide and had another effort saved by Martinez.
Mount, who is in talks about a new Chelsea contract, killed the game off with a well-taken free-kick, but again he had a helping hand when Martinez shifted to his right.
The Chelsea forward's strike went to the goalkeeper's left, but crossed the line close to the middle of the goal.
In addition to the boos at the final whistle, Gerrard was also jeered as he made his way down the tunnel.
But after the manager asked for recent "frustration" about his team's results to be channelled in the right way, the hosts answered with a first half that saw them also hit the woodwork twice.
How they did not score was largely down to Arrizabalaga, who, in addition to a brilliant save from Ings, made an extraordinary double save to stop McGinn and then Ramsey after Bailey hit the bar with an earlier header.
With Ings and Bailey in the starting line-up in place of Philippe Coutinho and Emiliano Buendia, there was plenty of joy down the flanks and the hosts were ably helped by Douglas Luiz in midfield.
But Mings' error was inexplicable as he mistimed his header under no pressure as Ben Chilwell's flick looped up to him in the penalty area.
Villa might argue that Chilwell was lucky to stay on the pitch after a robust tackle on Ramsey, for which he was only booked.
In the second half, there were fewer chances as Chelsea tightened up and took control of possession. Arrizabalaga denied McGinn once more after 62 minutes, and within three minutes there was no coming back once Martinez misjudged Mount's free-kick.
-- Courtesy of BBC Sport