The energy efficiency of dry fine grinding process of powders dramatically declines with decreasing particle size, and represents a growing concern in various cutting-edge technical applications for pharmaceutical and cosmetic products, advanced-ceramics, and food industry. In this paper, we report on a detailed experimental analysis of the long-term grinding behavior of silica sand as a model material in an oscillatory ball mill for a broad range of the values of vibration frequency and amount of powder. We show that the re-agglomeration of fine particles unambiguously explains the anomalous increase of grinding power consumption. We also find that the agglomeration phenomena are correlated with the volume fraction of the class of finest particles. A nonlinear comminution model is shown to capture this effect for the prediction of the evolution of the specific surface area.