We use the bond-based peridynamics approach to analyze the strength and fracture of dense granular aggregates with variable amount of a solid binding matrix, distributed according to a simple protocol in the interstitial space between particles. We show the versatility of the peridynamics approach in application to crack propagation and its scaling behavior in a homogeneous medium (in the absence of particles and pores). Then we apply this method to simulate the deformation and failure of aggregates as a function of the amount of the binding matrix under tensile loading. We find that the tensile strength is a strongly nonlinear function of the matrix volume fraction. It first increases slowly and levels off as the gap space in-between touching particles is gradually filled by the binding matrix, up to nearly 90% of the total pore volume, and then a rapid increase occurs to the maximum strength as the remaining interstitial space, composed of isolated pores between four or more particles, is filled. By analyzing the probability density functions of stresses in the particle and matrix phases, we show that the adhesion of the matrix to the particles and the thickening of stress chains (i.e., stresses distributed over larger cross sections) control the strength in the first case whereas the homogenizing effect of the matrix by filling the pores (hence reducing stress concentration) is at the origin of further increase of the strength in the second case. Interestingly, these two mechanisms contribute almost equally to the total strength.