We compare two different elicitation methods for measuring risk attitudes on a sample of French farmers. We consider the lottery tasks initially proposed by Holt and Laury (Econ Rev 92:1644-1655, 2002) and by Eckel and Grossman (Evol Hum Behav 23:281-295, 2002; J Econ Behav Org 68:1-7, 2008). The main empirical result from this within-subject study is that risk preference measures are affected by the type of mechanism used. We first show that this risk preference instability can be related to non-expected utility preferences of farmers. Using a risk-taking psychometric questionnaire, we then demonstrate that risk preferences of farmers are context-dependent. This may be another explanation of the observed risk preference instability.