Agricultural intensification not only increases food production but also drives widespread biodiversity decline. Increasing landscape heterogeneity has been suggested to increase biodiversity across habitats, while increasing crop heterogeneity may support biodiversity within agroecosystems. These spatial heterogeneity effects can be partitioned into compositional (land‐cover type diversity) and configurational heterogeneity (land‐cover type arrangement), measured either for the crop mosaic or across the landscape for both crops and semi‐natural habitats. However, studies have reported mixed responses of biodiversity to increases in these heterogeneity components across taxa and contexts. Our meta‐analysis covering 6397 fields across 122 studies conducted in Asia, Europe, North and South America reveals consistently positive effects of crop and landscape heterogeneity, as well as compositional and configurational heterogeneity for plant, invertebrate, vertebrate, pollinator and predator biodiversity. Vertebrates and plants benefit more from landscape heterogeneity, while invertebrates derive similar benefits from both crop and landscape heterogeneity. Pollinators benefit more from configurational heterogeneity, but predators favour compositional heterogeneity. These positive effects are consistent for invertebrates and vertebrates in both tropical/subtropical and temperate agroecosystems, and in annual and perennial cropping systems, and at small to large spatial scales. Our results suggest that promoting increased landscape heterogeneity by diversifying crops and semi‐natural habitats, as suggested in the current UN Decade on Ecosystem Restoration, is key for restoring biodiversity in agricultural landscapes.