Animal welfare is increasingly recognised as an integral component of livestock system sustainability, thereby necessitating to be incorporated into on-farm sustainability assessments but this may involve practical challenges. In the EU INTAQT project, sustainability, including environmental impacts, intrinsic quality of products and animal welfare, was assessed on farm across diverse farming systems for dairy cows (71 farms), fattening beef (51 farms) and broilers (57 farms) from 10 countries. Welfare was assessed at farm level according to AssureWel protocol for cattle and EBENE® protocol for broilers and the observers were trained online. The principal component analysis of animal-based measures (ABMs) from the cattle welfare assessments highlighted variability across countries and therefore systems, but the effect could be linked to the observers, since observer, country and sometimes systems were confounded. ANOVA and variance tests for each ABM (ABM ~ Observer) showed that for ABMs requiring observation (e.g., Lameness, Hairless patches) the means and the variances depended on the observers (p < 0.05). In broilers, we did not observe variations between observers. The difference between cattle and broilers could be explained by ABMs easier to observe for broilers than for cattle where ABMs seem to be harder to standardise. Though statistical analysis can be performed per country, this observer bias in cattle assessments highlights practical difficulties and limitations in integrating animal welfare into on-farm sustainability evaluations, particularly in a multi-country European project. Common guidelines on how and what to include for animal welfare in sustainability assessments, extensive training of observers and a good repeatability between them are pre-requisites to ensure data quality in multi-partner projects. More generally, the risk of over simplifying each dimension by increasing the complexity of livestock sustainability concept and therefore its on-farm assessment needs to be considered for future global sustainability evaluations.