Most food products are highly perishable as they constitute a rich nutrient source for microbial development. Among the microorganisms contaminating food, some present metabolic activities leading to spoilage. In addition to hygienic rules to reduce contamination, various treatments are applied during production and storage to avoid the growth of unwanted microbes. The nature and appearance of spoilage therefore depend on the physiological state of spoilers and on their ability to resist the processing/storage conditions and flourish on the food matrix. Spoilage also relies on the interactions between the microorganisms composing the ecosystems encountered in food. The recent rapid increase in publicly available bacterial genome sequences, as well as the access to high-throughput methods, should lead to a better understanding of spoiler behavior and to the possibility of decreasing food spoilage. This review lists the main bacterial species identified as food spoilers, their ability to develop during storage and/or processing, and the functions potentially involved in spoilage. We have also compiled an inventory of the available genome sequences of species encompassing spoilage strains. Combining in silico analysis of genome sequences with experimental data is proposed in order to understand and thus control the bacterial spoilage of food better.