Participatory plant breeding (PPB) has received much attention in recent decades for its ability to develop varieties adapted to the diversity of farm conditions and to farmers' needs and practices. Specific methodological issues arise when working with on-farm experiments, one being the implementation of an experimental design that matches farmers' constraints and objectives, while allowing for accurate statistical analyses of the data. We took the example of a French PPB case on bread wheat, in which farmers, facilitators and researchers have co-constructed an experimental design that meets their needs, but is very unbalanced and required the development of Bayesian statistical models to compare populations on-farm, over environments and analyze their sensitivity to environments. Through a simulation study, we investigated the effects of different characteristics of the experimental design on the behavior of two of these Bayesian models to identify the range of values that are most appropriate and give recommendations for decentralized experiments. We analyzed the estimates obtained by the models using different simulated datasets that differed by the values of the experimental design's parameters. While within-environment population effects were well estimated even with few replicated controls, replicating populations of interest rather than controls within environments and including enough environments provided more power to detect significant differences. Population effects and sensitivities over environments were mainly impacted by the number of replications of populations across environments, therefore effort should be made to repeat populations in more environments if the aim is to characterize their behavior under various environmental conditions.