Phenomic prediction (PP), a novel approach utilizing Near Infrared Spectroscopy (NIRS) data, offers an alternative to genomic prediction (GP) for breeding applications. In PP, a hyperspectral relationship matrix replaces the genomic relationship matrix, potentially capturing both additive and non-additive genetic effects. While PP boasts advantages in cost and throughput compared to GP, the factors influencing its accuracy remain unclear and need to be defined. This study investigated the impact of various factors, namely the training population size, the multi-environment information integration, and the incorporations of genotype x environment (GxE) effects, on PP compared to GP. We evaluated the prediction accuracies for several agronomically important traits (days to flowering, plant height, yield, harvest index, thousand-grain weight, and grain nitrogen content) in a rice diversity panel grown in four distinct environments. Training population size and GxE effects inclusion had minimal influence on PP accuracy. The key factor impacting the accuracy of PP was the number of environments included. Using data from a single environment, GP generally outperformed PP. However, with data from multiple environments, using genotypic random effect and relationship matrix per environment, PP achieved comparable accuracies to GP. Combining PP and GP information did not significantly improve predictions compared to the best model using a single source of information (e.g., average predictive ability of GP, PP, and combined GP and PP for grain yield were of 0.44, 0.42, and 0.44, respectively). Our findings suggest that PP can be as accurate as GP when all genotypes have at least one NIRS measurement, potentially offering significant advantages for rice breeding programs, reducing the breeding cycles and lowering program costs.