Environmental change often combined with selective harvesting has profound and diverse impacts on marine fish populations. Unlocking the biological consequences of these effects on wild fish is notoriously challenging, especially in highly productive but naturally variable systems with uncertain futures such as Eastern Boundary current systems. Here, we developed otolith increment-based growth chronologies covering half a century (53 years) for a small pelagic fish (Atlantic horse mackerel, Trachurus trachurus) in the northern limb of the Canary current upwelling system. We used increasingly complex mixed-effects models to partition individually resolved growth variation among intrinsic (Age and Age-at-capture) and extrinsic (biotic and abiotic factors) sources in four complementary data sets: a general population chronology, and three chronologies derived from age groups that reflect ontogenetic habitat shifts. First, we investigated the timing and scale of growth phase shifts and assessed the effects of extrinsic factors on inter-annual growth variation. Second, we quantified among and within cohort growth variability over time. Our results provided strong evidence for inter-annual SST and primary productivity variation impacting on Atlantic horse mackerel growth. We also identified phase shifts in growth that point to larger ecosystem-wide changes (regime shifts), potentially driven by large-scale climatic indices, such as North Atlantic Oscillation and East Atlantic pattern. Cohort-dependent growth effects likely reflect persistent environmental influences and density dependence. Further, we found evidence for carryover effects in growth whereby a poor start in life tended to persist despite compensatory growth being observed in some individuals. We show how population productivity can be impacted by multiple, interacting environmental and biotic factors leading to potential ecosystem regime shifts. Such information is key to understand recruitment dynamics and population persistence, and will have important implications for fisheries management and to those seeking to understand the effects of large-scale climate change on marine productivity.