The adaptation of populations to local environments often relies on the selection of optimal values for one or several polygenic phenotypes. The architecture of these phenotypes is complex, and the majority of loci associated to them in GWAS are located in regulatory regions. The evolution of such phenotypes may thus involve changes in gene expression regulation. In the last few years, systems biology offered a new perspective on the molecular bases of complex phenotypes, shifting from a gene-centric to a network-centric view. However, the link between the structure of gene regulatory networks, the phenotype architecture and its evolution is still unclear. To tackle this question, we investigated several tissue-specific eQTL networks derived from the GTEX dataset. We showed that such networks are highly structured, with tissue-specific modules (groups of SNPs associated to groups of genes) delineating and isolating different biological functions within an interconnected network. We also found that SNPs explaining the heritability of one phenotype are clustered within a few tissue-specific modules that regulate functions related to the phenotype. These SNPs tend to be local hubs – SNPs that are holding together a module and tend to fall in tissue-specific activated regulatory elements. Finally, we showed that local hubs are also enriched in signature of polygenic selection signals, while global hubs tend to be enriched in negative selection signals. Altogether, these results provide a model to understand how polygenic phenotypes, while being determined by many genes, often pleiotropic and affecting other phenotypes through the regulatory network, can evolve.