Purpose Recent global-scale analysis showed the extent of inequality in terms of carbon emissions related to overall consumption, with richer households emitting significantly more greenhouse gases than poorer ones. While socio-economic status is a known determinant of food consumption, and despite the urgent need to move towards more sustainable diets, very few studies have explored socio-economic differences regarding the environmental impacts of diets. The objective of the present study was to compare the environmental impacts of French adults' diets according to food insecurity (FI) status and income level. Methods The environmental impacts of diets of a representative sample of adults living in France (n = 1964) were assessed using data from the last National Individual Food Consumption Survey (INCA3) and the Agribalyse® v3.0.1 environmental database. Fifteen impact indicators were estimated, including climate change, eutrophication (freshwater, marine, terrestrial), resource depletion (energy, minerals, water), and the single EF score. First, the mean diet-related impact (per day per person) was estimated for each environmental indicator by decile of environmental impact. Second, the environmental impacts of diets of individuals living in food-insecure households (severe and moderate FI, as measured by the Household Food Security Survey Module) were compared with those of individuals living in food-secure households, the latter being divided by income decile. Differences in environmental impacts of diets (total and by food group) between these 12 sub-populations were tested by ANOVA after adjustment for age, gender, energy intake and household size. Results The 10% of the population with the highest environmental impact has a mean impact approximately 3–6 times higher than the 10th with the lowest environmental impact, depending on the indicator. Individuals living in households with severe and moderate FI represented 3.7% and 6.7% of the studied population, respectively. Results showed a high variability in impacts within each of the 12 sub-population and no difference in environmental impacts of diets between sub-populations, except for water use (p < 0.001) and freshwater eutrophication (p = 0.02). The lowest water use and freshwater eutrophication were observed for individuals living in households with severe FI and the highest for high-income sub-populations, with differences mainly explained by the level of fruit and vegetable intakes and the type of fish consumed, respectively. Low-income populations, in particular individuals living in households with severe FI, had relatively high intakes of ruminant meat but for most indicators, the high environmental impact of this food group was offset by low consumption of other high impacting food groups (e.g., fruits and vegetables), and/or by high consumption of low impacting food groups (e.g., starches), resulting in no difference in the impact at the diet level. Conclusion While there is a high inter-individual variability in the environmental impacts of diets, this variability was not related to income level or FI status for most indicators, except higher water use and freshwater eutrophication in higher-income populations. Overall, our results underline the importance of considering individual dietary patterns and thinking at the whole diet level, and not only considering specific food or food groups impacts, when designing educational tools or public policies to promote more sustainable diets.