Globally, agricultural soil management leads to soil organic carbon (SOC) losses, which contribute to increase atmospheric CO2 concentrations and thereby climate change. Grassland introduction into cropping phases (ley grasslands) was suggested as an appropriate management strategy to reduce these losses. Here we examine the impact of ley grassland durations in crop rotations on soil organic carbon in temperate climate from 2005 to 2100. We considered two IPCC scenarios, RCP4.5 and RCP8.5, with and without atmospheric CO2 enhancements. We used the DailyDayCent model and a long-term field experiment to show that ley grasslands increase SOC storage by approximately 10 Mg ha−1 over 96 years compared with continuous cropping. Surprisingly, extending ley duration from 3 to 6 years does not enhance SOC, while it had a positive effect on plant residue accumulation in soil. Furthermore, in comparison with non-renewed grasslands, those renewed every three years demonstrated a notable increase in SOC storage, by 0.3 Mg ha−1 yr−1. These results may be explained by the enhanced input of root C in young grassland systems and its preferential contribution to soil organic matter formation. We concluded that management of root C inputs by ley grassland ploughing and renewal intervals is crucial for maximizing SOC stocks in agricultural soils, through balancing biomass carbon inputs during regrowth and carbon losses through soil respiration.