There is an increasing concern about the potential for size-selective harvest to impair population persistence. Yet little is known about the relative contribution of the evolutionary (shifts in life history) and demographic effects (decreased population density and size truncation) of harvesting to changes in fitness. Using medaka (Oryzias latipes), we experimentally investigated the fitness consequences of antagonistic size-dependent selection under contrasted levels of density (low versus high) and size structure (uniform versus truncated). The size-dependent selection generated large- and small-breeder lines with fast growth and late maturity versus slow growth and early maturity life histories, respectively. A decrease in density had a positive effect on almost all fitness components, while size truncation only had a positive effect on fish growth. Small-breeder fish grew slower and had a greater probability of reproducing, at least when considering small-sized females. The number of larvae and juveniles did not differ between the two lines. Finally, the positive effect of decreased density on population asymptotic growth rate was less pronounced in small-breeders than in large-breeders of medaka. These findings stress the importance of considering the ramifications of fishing-adapted life history to population persistence in the light of density-dependent population dynamics.