Nitrous oxide (N2O) is a major radiative forcing and stratospheric ozone depleting gas emitted from terrestrial and aquatic ecosystems. It can be transformed to N2 by bacteria and archaea harboring the nitrous oxide reductase (N2OR), which is the only known N2O sink in the biosphere. Despite its crucial role in mitigating N2O emissions, knowledge of the N2OR in the environment remains limited. Here, we report a comprehensive phylogenetic analysis of the nosZ gene coding the N2OR in genomes retrieved from public databases. The resulting phylogeny revealed two distinct clades of nosZ, with one unaccounted for in studies investigating N2O reducing communities. Examination of N2OR structural elements not considered in the phylogeny revealed that the two clades differ in their signal peptides, indicating differences in the translocation pathway of the N2OR across the membrane. Sequencing of environmental clones of the previously undetected nosZ lineage in various environments showed that it is widespread and diverse. Using quantitative PCR, we demonstrate that this clade was most often at least as abundant as the other, thereby more than doubling the known extent of the overall N2O reducing community in the environment. Furthermore, we observed that the relative abundance of nosZ from either clade varied among habitat types and environmental conditions. Our results indicate a physiological dichotomy in the diversity of N2O reducing microorganisms, which might be of importance for understanding the relationship between the diversity of N2O reducing microorganisms and N2O reduction in different ecosystems.