The success of Integrated Assessment and Modeling of social-ecological systems (SESs) requires a framework allowing members of this process to share, organize and integrate their knowledge about the system under consideration. To meet this need and ease management of successful modeling processes, we present a conceptual framework for integrated agent-based modeling and simulation of SESs in the form of a formal “entity-process meta-model”, along with a distinction between three levels of models—conceptual, concrete and simulation—and characterization of the research question using indicators and scenarios. We then describe how to represent the structural and dynamic dimensions of SESs into conceptual and concrete models and to derive the simulation model from these two types of models. Finally, we discuss how our framework solves some of the challenges of integrated SES modeling: integration and sharing of heterogeneous knowledge, reliability of simulation results, expressiveness issues, and flexibility of the modeling process.