Weeds are a major threat in tropical regions where climate conditions favor their growth and development. This is particularly true in low-input rice-based cropping systems in the Malagasy highlands, where weed management is mainly done by manual removal. Crop rotation is often promoted as an efficient way to control weed infestations, while the role of fertilization is more controversial. In this study, we compared rice monoculture to three rainfed rice-based two-year rotations: rice followed by groundnut, rice followed by sorghum-cowpea mixture, and rice followed by a velvet-bean crotalaria mixture. Each rotation was tested with two levels of fertilization (5 t DM ha−1 organic manure, sole or in combination with mineral fertilizer - 400 kg ha−1 NPK + 200 kg ha−1 urea). We assessed the effect of rotation and fertilization on weed composition, diversity, biomass and rice yield. Additionally, the farmers’ perception of weed harmfulness and the relation between their assessment of weed harmfulness and rice production was tested. Our results showed that weed biomass significantly decreased rice yield but only under the low fertilization level. The rotation of rice with the velvet bean-crotalaria mixture was efficient in reducing weed biomass, modified weed community composition and allowed to achieve the highest rice yield. A significant negative relationship was found between weed community harmfulness index and weed species richness. Yet, the lowest rice yield was observed under rice monoculture despite a higher species richness over years and under high fertilization level. The lack of significant correlation between the harmfulness index and the actual rice yield is probably because our index is partly based on farmer’s perception, and only on major weeds. More studies on tropical weed harmfulness are needed to support the design of ecologically intensified cropping systems.