Damian Penaud, whose father Alain played 32 for France between 1992 and 2000, scored France's first try as the hosts reeled in the All Blacks
The hosts had to fight deep into the final quarter as tries from New Zealand wing Mark Telea inside the opening minutes of both halves threatened to wreck the opening-day party.
Ultimately, though, Thomas Ramos' relentless boot and a well-worked try from Damian Penaud edged them clear of the enterprising, if fading All Blacks.
A 73rd-minute penalty from Ramos finally put France out of seven-point range on the scoreboard and the home fans could celebrate in comfort as their players hunted for a crowning try.
They found it with three minutes left as Melvyn Jaminet gathered fellow replacement Maxime Lucu's teasing chip.
France will continue their Pool A campaign on Thursday against Uruguay before meetings with Namibia and Italy.
Victories in all three would guarantee them top spot and a last-eight meeting with the runners-up from Pool B, which contains Ireland, South Africa and Scotland.
New Zealand, who had won all 31 of their previous pool-stage game at World Cups, will know from painful experience that all is not lost.
Four years ago, at Japan 2019, they beat South Africa in the pick of the pool-stage matches, only to watch the Springboks become the first team to lose their opening game and go on to lift the William Webb Ellis Trophy.
France's players stayed out and and soaked up the atmosphere of blaring Euro-disco and late summer heat after the final whistle.
In seven weeks it will be autumn, but the stage will be the same. The result and, perhaps more importantly, the resolve will harden belief that this golden generation could be the one to finally lift the silverware that has escaped them in three previous finals.
-- Courtesy of BBC Sport