Despite extensive literature on the complex nature of empowerment, current efforts to measure women’s empowerment in the agricultural development sector are largely limited to assessing visible forms of agency. We take a critical look at current efforts to measure women’s empowerment at the individual/household level through standardized tools. We examine the results of a household survey conducted in Nepal using the Women’s Empowerment in Agriculture Index (WEAI), which was developed as a monitoring and evaluation tool for the Feed the Future Initiative. Our interpretation of the results is informed by qualitative fieldwork conducted in the same region. In our quantitative analysis, we regress correlates of empowerment identified in the literature, such as age, education, household wealth, income, and household composition, on individual empowerment as measured by the WEAI. While several factors associated with women’s empowerment are significantly associated, household composition and intra-household relationships, which we expected to be essential factors in the local context, appear to be unrelated to the WEAI empowerment score. A measure of critical consciousness tested alongside the WEAI instrument appears instead to be closely associated with these factors. Our qualitative findings reveal that there is a discrepancy between local meanings of empowerment and definitions of empowerment defined in terms of agency. Based on these results, we suggest that improvements in measurement may be possible if approaches that measure power predominantly in terms of agency or decision-making were to include critical consciousness in their framework.