Finely regulated fatty acid (FA) metabolism within ovarian follicles is crucial to follicular development and influences the quality of the enclosed oocyte, which relies on the surrounding intra-follicular environment for its growth and maturation. A growing number of studies have examined the association between the lipid composition of follicular compartments and oocyte quality. In this review, we focus on lipids, their possible exchanges between compartments within the ovarian follicle and their involvement in different pathways during oocyte final growth and maturation. Lipidomics provides a detailed snapshot of the global lipid profiles and identified lipids, clearly discriminating the cells or fluid from follicles at distinct physiological stages. Follicular fluid appears as a main mediator of lipid exchanges between follicular somatic cells and the oocyte, through vesicle-mediated and non-vesicular transport of esterified and free FA. A variety of expression data allowed the identification of common and cell-type-specific actors of lipid metabolism in theca cells, granulosa cells, cumulus cells and oocytes, including key regulators of FA uptake, FA transport, lipid transformation, lipoprotein synthesis and protein palmitoylation. They act in harmony to accompany follicular development, and maintain intra-follicular homeostasis to allow the oocyte to accumulate energy and membrane lipids for subsequent meiotic divisions and first embryo cleavages.