Understanding genomic signatures of divergent selection underlying long-term adaptation in populations located in heterogeneous environments is a key goal in evolutionary biology. In this study, we investigated neutral, adaptive and deleterious genetic variation using 7,192 SNPs in 31 Lake Trout (Salvelinus namaycush) populations (n = 673) from Quebec, Canada. Average genetic diversity was low, weakly shared among lakes, and positively correlated with lake size, indicating a major role for genetic drift subsequent to lake isolation. Putatively deleterious mutations were on average at lower frequencies than the other SNPs, and their abundance relative to the entire polymorphism in each population was positively correlated with inbreeding, suggesting that the effectiveness of purifying selection was negatively correlated with inbreeding, as predicted from theory. Despite evidence for propounced genetic drift and inbreeding, several outlier loci were associated with temperature and found in or close to genes with biologically relevant functions notably related to heat stress and immune responses. Outcomes of gene–temperature associations were influenced by the inclusion of the most inbred populations, in which allele frequencies deviated the most from model predictions. This result illustrates challenge in identifying gene–environment associations in cases of high genetic drift and restricted gene flow and suggests limited adaptation in populations experiencing higher inbreeding. We discuss the relevance of these findings for the conservation and management, notably regarding stocking and genetic rescue, of Lake Trout populations and other species inhabiting highly fragmented habitats