Spatiotemporal variations of natural selection may influence the evolution of various features of organisms such as local adaptation or specialisation. This article develops a method for inferring how selection varies between locations and between generations from phenotypic data. It is assumed that generations are non-overlapping and that individuals reproduce by selfing or asexually. A quantitative genetics model taking account of the effects of stabilising natural selection, the environment and mutation on phenotypic means and variances is developed. Explicit results on the evolution of populations are derived and used to develop a Bayesian inference method. The latter is applied to simulated data and to data from a wheat participatory plant breeding programme. It has some ability to infer evolutionary parameters, but estimates may be sensitive to prior distributions, for example when phenotypic time series are short and when environmental effects are large. In such cases, sensitivity to prior distributions may be reported or more data may be collected.