Conceptualized by the Food and Agriculture Organization in 2010, climate-smart agriculture aims to simultaneously tackle three main objectives. These are increasing food security, building the resilience of agricultural systems for adaptation to climate change and mitigation of GHG. As much research focuses on one of these three objectives, our understanding of how agricultural systems address these three challenges simultaneously is limited by the lack of a comprehensive evaluation tool. In order to fill this gap, we have developed a generic evaluation framework that comprises 19 indicators that we measured in a sample of 12 representative farms of the North Basse-Terre region in Guadeloupe. The evaluation revealed clear differences in the performance of these farming systems. For example, nutritional performance varied from 0 to 13 people fed per hectare, the average potential impact of climatic conditions varied from 27% to 33% and the GHG emissions balance varied from +0.8 tCO2eq·ha−1 to +3.6 tCO2eq·ha−1. The results obtained can guide the design of innovative production systems that better meet the objectives of climate-smart agriculture for the study region. The evaluation framework is intended as a generic tool for a common evaluation basis across regions at a larger scale. Future prospects are its application and validation in different contexts.