This chapter reviews the data reduction problem for instance and feature selection methods in the context of supervised classification. In the first part, instance and feature selections are studied separatively. As instance and feature selection are not independent, algorithms dealing with simultaneous selection are then presented. To provide a comprehensive and tractable view of this field, the strategy was to start from the fundamental and original contributions go towards state of the art algorithms, paying particular attention to large scale selections. Detailed pseudo codes of representative algorithms are given to consolidate the whole.