Applying ideas of statistical mechanics in ecology have recently received quite some attention. The entropy maximization (EM) formalism looks particularly attractive, as it provides a simple algorithm to infer detailed system variables from a limited number of constraints. However, we point out that a blind application of this formalism can easily lead to wrong conclusions. To illustrate this, we reanalyze an ecological data set that has been used to claim the good performance of EM in predicting species abundances from trait measurements. We show that these results are entirely due to the restrictive constraints, and do not provide any support for the applicability of EM in ecology. By comparing with a simple example from physics, we indicate which characteristic mechanism of EM, and of statistical mechanics in general, is missing for the ecological example. This analysis introduces a series of methods to evaluate future attempts to apply EM in ecology.