For prediction of rockfalls, the failure of rock joints is studied. Considering these failures as constitutive instabilities, a second‐order work criterion is used because it explains all divergence instabilities (flutter instabilities are excluded). The bifurcation domain and the loading directions of instabilities, which fulfill the criterion, are determined for any piecewise linear constitutive relation. The instability of rock joints appears to be ruled by coupling features of the behavior (e.g., dilatancy). Depending on the loading parameters, instabilities can lead to failure, even before the plastic limit criterion. Results for two given constitutive relations illustrate the approach. Some given loading paths are especially considered. Constant volume (undrained) shear and τ‐constant paths are stable or not depending on the link between the deviatoric stress and strain along undrained paths, as found for soils. Some unstable loading paths are illustrated. Along these paths, failure before the plastic limit criterion is possible. The corresponding failure rules are determined.