Purpose The control of body composition by genetics and dietary nutrients is of the upmost importance for both human and animal physiology. Adult stem cells (aSC) may represent a relevant level of tissue adaptation. The purpose of this study was to determine the impact of macronutrient composition on aSC populations isolated from adipose tissue or muscle in growing pigs. Methods Pigs from two lines divergently selected for feed efficiency were fed ad libitum either a high-fat/high-fiber (HF) diet or a low-fat/low-fiber (LF) diet (n = 6 per line and diet) from 74 to 132 days of age. Stroma vascular cells were isolated from adipose tissue and muscle and characterized with cell surface markers. Results In both lines, pigs fed the HF diet exhibited a reduced adiposity (P < 0.001) compared with pigs fed the LF diet. In the four groups, CD90 and PDGFRα markers were predominantly expressed in adipose cells, whereas CD90 and CD56 markers were highly expressed in muscle cells. In adipose tissue, the proportions of CD56+/PDGFRα + and of CD90+/PDGFRα + cells were lower (P < 0.05) in HF pigs than in LF pigs. On the opposite, in muscle, these proportions were higher (P < 0.001) in HF pigs. Conclusion This study indicates that dietary nutrients affected the relative proportions of CD56+/PDGFRα + cells with opposite effects between muscle and adipose tissue. These cell populations exhibiting adipogenic potential in adipose tissue and myogenic potential in muscle may be a target to modulate body composition.