The consumption of sugar, salt, and fat in infancy may influence later health. The objective of this study was to describe the frequency of use of added sugar, salt, and fat during the complementary feeding period and the associated infant caregiving practices. Data were obtained from a monthly questionnaire filled by parents for 10,907 infants from the French Etude Longitudinale Française depuis l’Enfance (ELFE) cohort. A score of frequency of use (SU) for added sugar, salt, and fat (oil, margarine, butter, and/or cream) was calculated from the age at complementary feeding introduction (CFI) to the 10th month. Associations between the SU of each added ingredient with infant feeding and caregiving practices were studied with multivariable linear regressions adjusted for familial characteristics. Only 28% of the parents followed the recommendation of adding fat and simultaneously not adding sugar or salt. Breastfeeding mothers were more prone to add sugar, salt, and fat than non-breastfeeding mothers. CFI before four months was positively associated with the SU of added sugar and salt and negatively associated with the SU of added fat. The use of commercial baby food was negatively related to the SU of added salt and fat. The use of these added ingredients was mainly related to breastfeeding, age at CFI, and use of commercial food, and it was independent of the household socioeconomic characteristics.