Organic agriculture is widely accepted as a strategy to reduce the environmental impacts of food production and help achieve global climate and biodiversity targets. However, studies concluding that organic farming could satisfy global food demand have overlooked the key role that nitrogen plays in sustaining crop yields. Using a spatially explicit biophysical optimization model that accounts for crop growth nitrogen requirements, we show that, in the absence of synthetic nitrogen fertilizers, the production gap between organic and conventional agriculture increases as organic agriculture expands globally (with organic producing 36% less food for human consumption than conventional in a fully organic world). Yet, by targeting both food supply (via a redesign of the livestock sector) and demand (by reducing average per capita caloric intake), public policies could support a transition towards organic agriculture in 40-60% of the global agricultural area even under current nitrogen limitations thus helping to achieve important environmental and health benefits. The ability of organic agriculture to feed the world population remains contentious. A biophysical optimization model indicates the extent to which nitrogen availability could limit the expansion of organic production systems and what measures could alleviate this limitation.