In complex eukaryotes, transposable elements (TEs), previously considered as junk DNA, are more and more acknowledged as genome shapers and as a source of gene innovation, genome plasticity and genome divergence. Fungi are simple and easy-to-manipulate eukaryotic organisms, for which ever-increasing genome information indicates that many plant-associated fungi have a tendency to have expanded genomes. This increase in genome size is mostly driven by TE expansion that eventually shapes adaptive regions of the genome. Such genome regions harbour genes involved in niche adaptation and favour accelerated evolution of these genes. The recent rise of comparative genomics in fungi now allows the use of phylogeny to date TE invasion and proliferation in genomes. This in turn provides inferences about the impact of TEs on speciation and on the rise of better-adapted species. Here, focusing on plant-associated fungi, we review our current level of knowledge on how TEs may have contributed to speciation, the rise of two-speed genomes and the shaping of specific genome environments. Moreover, we consider the role of TEs in gene duplication and diversification that contribute to both adaptation to new hosts and adaptation to host resistance genes in gene-for-gene systems.