Knowledge is lacking on the effects of associated crops on cocoa productivity in complex cocoa-based agroforestry systems. Through a generalized linear mixed model, we analyzed the influence of (i) the density and (ii) the height of associated species on cocoa tree productivity along a distance gradient of 0 to 5 m between cocoa trees and associated plants. We mapped 34 mature agroforestry systems in farmers’ plots to locate the position of different crop species in relation to each other. The production of each plant was monitored at fortnightly intervals over one year. The main results showed that cocoa tree productivity increased in the presence of Fabaceae, even when at a short distance (< 3 m), and to a lesser extent in the presence of timber trees. All the other species, i.e., food-producing trees, had a negative effect on cocoa productivity, which diminished with increasing distance up to a distance ranging from 3 to 5 m depending on the species. The height of associated plants also had a significantly negative effect on cocoa productivity at a distance of less than 3 m, beyond which the negative effect decreased. To our knowledge, it is the first study that finely analyzes the effect of associated crops on cocoa productivity according to farmers’ use and for short distances. These results are of particular interest for the structural redesign of agroforestry systems, in order to optimize their agro-economic performance, and can be used to recommend the distance to be respected between cocoa trees and associated plants as well as the height of the different crops or forest trees.