Deciphering the genetic architecture of quantitative traits through genome-wide association mapping on core collections paves the way for breeding adapted crops. We constructed the G2P-SOL pepper world core collection, a set of 423 Capsicum spp. accessions of great genotypic and phenotypic variability, representing the diversity of more than 10,000 pepper accessions from 10 major genebanks. To tackle the modulation of complex quantitative traits by the environment and by the genotype-by-environment (GxE) interaction effects, a highly diverse subset of 350 C. annuum accessions was extensively phenotyped in a multi-location effort. Multi-environment genome-wide association studies (GWAS) were conducted for 23 agronomic traits pertaining to fruit flavor, color, size and shape, and to plant productivity, vigor and precocity. Robust QTLs and environment-specific QTLs were thus detected, and 97 well-known and new genes potentially underlying 53 of the most high-confidence and robust QTLs were highlighted. These results assert the usefulness and universality of the G2P-SOL core collection, which will be available upon request for the pepper community, alongside the genotypic and phenotypic data collected as part of the G2P-SOL project. This unique resource will be a key asset in accelerating gene discovery, developing genetic markers for marker-assisted selection, and breeding pepper varieties adapted to various climatic environments.