It is unclear whether unemployment exposure, as well as working conditions, can have sustained effects on the health of retirees who are no longer exposed. The aim of the present study is to investigate this issue in 29,281 French retirees from the CONSTANCES cohort in whom the prevalence of suboptimal self-rated health, disability for routine tasks, cardiovascular diseases and cancers is assessed according to lifetime exposure to unemployment and prior working conditions. The analyses are performed retrospectively using multivariable logistic regression models with adjustment for potential confounders such as sex, birth year, parental histories of cardiovascular disease and cancer, social position, retirement age and duration. High lifetime exposure to unemployment is associated with an increased prevalence of suboptimal self-rated health (adjusted odds ratio (95% CI), 1.39 (1.23–1.57)), disability for routine tasks (1.41 (1.26–1.57)) and several cardiovascular diseases including stroke (1.66 (1.19–2.31)), myocardial infarction (1.65 (1.18–2.31)) and peripheral arterial disease (2.38 (1.46–3.90)). Bad prior working conditions are associated with an increased prevalence of disability for routine tasks (1.17 (1.04–1.33)) and cancers (1.27 (1.04–1.54)), notably prostate cancer (1.60 (1.01–2.64)). These findings suggest that unemployment and working conditions have long-term health effects that may cumulate over lifetime, emphasizing that risk evaluation and preventive strategies in retirees, as in workers, should take into account the life-course of individuals in addition to traditional risk factors.