Genetic parameters for 18 fatty acids or groups of fatty acids (FA), milk production traits, and somatic cell score (SCS) were estimated by restricted maximum likelihood with a repeatability animal model, using 45,259 test-day records from the first lactations of 13,677 Alpine and Saanen goats. Fatty acid data were collected as part of an extensive recording scheme (PhénoFinLait), and sample testing was based on mid-infrared spectra estimates. The total predicted FA content in milk was approximately 3.5% in Alpine and Saanen goats. Goat milk fat showed similar saturated FA to cattle and sheep, but higher contents of capric (C10:0) FA (~9.7g/100g of milk fat). Heritability estimates ranged from 0.18 to 0.49 for FA and estimates were generally higher when FA were expressed in g/100g of milk fat compared with g/100g of milk. In general, the 3 specific short- and medium-chain goat FA, caproic acid (C6:0), caprylic acid (C8:0), and especially capric (C10:0) acid, had among the highest heritability estimates (from 0.21 to 0.37; average of 0.30). Heritability estimates for milk yield, fat and protein contents, and SCS were 0.22, 0.23, 0.39, 0.09, and 0.24, 0.20, 0.40, and 0.15, in Alpine and Saanen goats, respectively. When FA were expressed in g/100g of milk, genetic correlations between fat content and all FA were high and positive. Genetic correlations between the fat content and FA groups expressed in g/100g of fat led to further investigation of the association between fat content and FA profile within milk fat. Accordingly, in both Saanen and Alpine breeds, no significant genetic correlations were found between fat content and C16:0, whereas the correlations between fat content and specific goat FA (C6:0 to C10:0) were positive (0.17 to 0.59). In addition, the genetic correlation between fat content and C14:0 was negative (-0.17 to -0.35). The values of the genetic correlations between protein content and individual FA were similar, although genetic correlations between protein content and FA groups were close to zero. Genetic correlations of milk yield or SCS with the FA profile were weak. Results for genetic parameters for FA, however, should be further validated, because the low predicting ability of certain FA using mid-infrared spectra and the limited calibration data set might have resulted in low accuracy. In conclusion, our results indicated substantial genetic variation in goat milk FA that supported their amenability for genetic selection. In addition, selection on protein and fat contents is not expected to have an undesirable effect on the FA profile in regard to specificity of goat products and human health.