While breeding for improved immunity is essential to achieve sustainable fruit production, it also requires to account for genotype-by-environment interactions (G × E), which still represent a major challenge. To tackle this issue, we conducted a comprehensive study to identify genetic markers with main and environment-specific effects on pest and disease response in peach (Prunus persica) and apricot (Prunus armeniaca). Leveraging multienvironment trials (MET), we assessed the genetic architecture of resistance and tolerance to seven major pests and diseases through visual scoring of symptoms in naturally infected core collections, repeated within and between years and sites. We applied a series of genome-wide association models (GWAS) to both maximum of symptom severity and kinetic disease progression. These analyses lead to the identification of environment-shared quantitative trait loci (QTLs), environment-specific QTLs, and interactive QTLs with antagonist or differential effects across environments. We mapped 60 high-confidence QTLs encompassing a total of 87 candidate genes involved in both basal and host-specific responses, mostly consisting of the Leucine-Rich Repeat Containing Receptors (LRR-CRs) gene family. The most promising disease resistance candidate genes were found for peach leaf curl on LG4 and for apricot and peach rust on LG2 and LG4. These findings underscore the critical role of G × E in shaping the phenotypic response to biotic pressure, especially for blossom blight. Last, models including dominance effects revealed 123 specific QTLs, emphasizing the significance of non-additive genetic effects, therefore warranting further investigation. These insights will support the development of marker-assisted selection to improve the immunity of Prunus varieties in diverse environmental conditions.