Crop residues are a crucial ecological niche with a major biological impact on agricultural ecosystems. In this study, we used a combined diachronic and synchronic field experiment based on wheat-oilseed rape rotations to test the hypothesis that plant is a structuring factor of microbial communities in crop residues, and that this effect decreases over time with their likely progressive degradation and colonisation by other microorganisms. We characterised an entire fungal and bacterial community associated with 150 wheat and oilseed rape residue samples at a plurennial scale by metabarcoding. The impact of plant species on the residue microbiota decreased over time and our data revealed turnover, with the replacement of oligotrophs, often plant-specific genera (such as pathogens) by copiotrophs, belonging to more generalist genera. Within a single cropping season, the plant-specific genera and species were gradually replaced by taxa that are likely to originate from the soil. These changes occurred more rapidly for bacteria than for fungi, known to degrade complex compounds. Overall, our findings suggest that crop residues constitute a key fully-fledged microbial ecosystem. Taking into account this ecosystem, that has been neglected for too long, is essential, not only to improve the quantitative management of residues, the presence of which can be detrimental to crop health, but also to identify groups of beneficial microorganisms. Our findings are of particular importance, because the wheat-oilseed rape rotation, in which no-till practices are frequent, is particularly widespread in the European arable cropping systems.