We investigate the evolutionary dynamics of a population structured in phenotype, subjected to trait dependent selection with a linearly moving optimum and an asexual mode of reproduction. Our model consists of a non-local and non-linear parabolic PDE. Our main goal is to measure the history of traits when the population stays around an equilibrium. We define an ancestral process based on the idea of neutral fractions. It allows us to derive quantitative information upon the evolution of diversity in the population along time. First, we study the long-time asymptotics of the ancestral process. We show that the very few fittest individuals drive adaptation. We then tackle the adaptive dynamics regime, where the effect of mutations is asymptotically small. In this limit, we provide an interpretation for the minimizer of some related optimization problem, an Hamilton Jacobi equation, as the typical ancestral lineage. We check the theoretical results against individual based simulations.