In a context where sustainability assessments are increasingly popular, this perspective article discusses the influence of methodological choices on measurements of farm sustainability. We build the argumentation on the premises that sustainability is a multi-dimensional concept that can be measured through an indicator approach and use examples from a case study analysis of seven European dairy farms. Specifically, the article demonstrates how and why indicator selection, estimation methods, and reporting frameworks can influence measured performance and thereby affect wider sustainability conclusions about production systems and practice change. Overall, we highlight that while in practical terms, methodological choices are necessary to conduct a farm sustainability assessment, important limitations can arise from the process. Of particular concern are farm conclusions and recommendations that lead to perverse outcomes and generate further sustainability issues outside of study scope. Practical guidance is provided to aid methodological choices with a more comprehensive and critical view of farm sustainability assessments. Importantly, we call for a more upfront recognition of methodological shortcomings in farm analyses.