Habitat loss and fragmentation often result in small, isolated populations vulnerable to environmental disturbance and loss of genetic diversity. Low genetic diversity can increase extinction risk of small populations by elevating inbreeding and inbreeding depression, and reducing adaptive potential. Due to their linear nature and extensive use by humans, freshwater ecosystems are especially vulnerable to habitat loss and fragmentation. Although the effects of fragmentation on genetic structure have been extensively studied in migratory fishes, they are less understood in low-mobility species. We estimated impacts of instream barriers on genetic structure and diversity of the low-mobility river blackfish (Gadopsis marmoratus) within five streams separated by weirs or dams constructed 45–120 years ago. We found evidence of small-scale (<13 km) genetic structure within reaches unimpeded by barriers, as expected for a fish with low mobility. Genetic diversity was lower above barriers in small streams only, regardless of barrier age. In particular, one isolated population showed evidence of a recent bottleneck and inbreeding. Differentiation above and below the barrier (FST = 0.13) was greatest in this stream, but in other streams did not differ from background levels. Spatially explicit simulations suggest that short-term barrier effects would not be detected with our data set unless effective population sizes were very small (<100). Our study highlights that, in structured populations, the ability to detect short-term genetic effects from barriers is reduced and requires more genetic markers compared to panmictic populations. We also demonstrate the importance of accounting for natural population genetic structure in fragmentation studies.