Accurate and timely observations of wheat phenology and, particularly, of heading date are instrumental for many scientific and technical domains such as wheat ecophysiology, crop breeding, crop management or precision agriculture. Visual annotation of the heading date in situ is a labour-intensive task that may become prohibitive in scientific and technical activities where high-throughput is needed. This study presents an automatic method to estimate wheat heading date from a series of daily images acquired by a fixed RGB camera in the field. A convolutional neural network (CNN) is trained to identify the presence of spikes in small patches. The heading date is then estimated from the dynamics of the spike presence in the patches over time. The method is applied and validated over a large set of 47 experimental sites located in different regions in France, covering three years with nine wheat cultivars. Results show that our method provides good estimates of the heading dates with a root mean square error close to 2 days when compared to the visual scoring from experts. It outperforms the predictions of a phenological model based on the ARCWHEAT crop model calibrated for our local conditions. The potentials and limits of the proposed methodology towards a possible operational implementation in agronomic applications and decision support systems are finally further discussed.