A common practice in managerial and public service contexts is to quantity, calculate, and use numbers and metrics which provide a presumption of scientificity, a sense of measurability, objectivity, reliability, and precision upon which smarter decisions can be made. Besides providing a theoretical background, we analyze counter-productive effects of over-relying on numbers and metrics, notably in public administration. We discuss the following traps: preferring what is measurable over what is important, replacing the strategy by a measure and dehumanizing the decision making. We suggest some practical ways to facilitate a more parsimonious, smarter, and adequate use of numbers.