Banana crops in the Caribbean are characterized by the use of high rates of nitrogen (N) fertilization which causes severe environmental damages. The aim of this study was to assess the fertilizer N use efficiency (NUE) of banana crops in the field. To do so, a field trial was carried out during the first (GS1) and the fourth (GS4) growing seasons of banana crops, and the fate of a 15 N-labeled pulse applied late in the growing season (flowering stage) was determined. At harvest, NUE (average 24% 15 N applied) and the total recovery of fertilizer 15 N in the soil-plant system (i.e., 40% in GS1 and 62% in GS4) were low. Low NUE resulted mainly from the dilution in a large soil mineral N pool derived from earlier applications of the labeled-N fertilizer applied at flowering, combined with leaching caused by numerous high-intensity rainfall events (>20 mm d −1). Crop residues from previous cycles present at time of fertilizer application in the fourth growing season, promoted fertilizer N immobilization, which in turn favored fertilizer N recovery by decreasing N leaching. The results suggest that N fertilization after the first season could be reduced by 30% (i.e., −90 kg N ha −1) corresponding to the suppression of two applications from flowering to harvest with the current fertilizer management, as available N derived from earlier applications is sufficient to meet plant requirements.