Field margins are major habitats for biodiversity conservation and ecosystem functioning in agricultural landscapes, but biotic homogenization of plant communities threatens their ecological and agronomic functions. Our objective is to determine the drivers of plant diversity in field margins for conservation and restoration purposes. To do so, we assessed the effects of field margin structure and long-term management over 20 years (1995–2015) on the taxonomic and functional α- and β-diversity, and the functional composition of herbaceous plant communities. In 2015, we surveyed 302 field margins in bocage landscapes of Brittany, northwestern France. Results were very similar between taxonomic and functional diversity but revealed important discrepancies between the drivers of α- and β-diversity. Deep ditches, mowing and grazing increased α-diversity but did not affect β-diversity. Denser hedgerows had lower α-diversity than other field margins but strongly contributed to β-diversity by harbouring more unique sets of species or life strategies. Long-term herbicide spraying in field margins and cropping intensity in adjacent habitats did not affect α-diversity, but had more complex effects on β-diversity and selected for common weeds. All in all, preservation of dense hedgerows, abandonment of herbicide spraying, and protection against agrochemical drifts are key measures to prevent the establishment of common weeds and biotic homogenization of herbaceous plant communities in field margins. Above all, our study shows how important it is to go beyond α-diversity to make robust conservation and restoration decisions.