Peptide-based inhibitors hold promising potential in the development of antiviral therapy. Here, we investigated the antiviral potential of fragmented viral proteins derived from ribonucleoprotein (RNP) components of the human respiratory syncytial virus (HRSV). Based on a mimicking approach that targets the functional domains of viral proteins, we designed various fragments of nucleoprotein (N), matrix protein M2-1 and phosphoprotein (P) and tested the antiviral activity in an RSV mini-genome system. We found that the fragment comprising residues 130–180 and 212–241 in the C-terminal region of P (81 amino acid length), denoted as P Fr, significantly inhibited the polymerase activity through competitive binding to the full-length P. Further deletion analysis of P Fr suggested that three functional domains in P Fr (oligomerization, L-binding and nucleocapsid binding) are required for maximum inhibitory activity. More importantly, a purified recombinant P Fr displayed significant antiviral activity at low nanomolar range in RSV-infected HEp-2 cells. These results highlight P as an important target for the development of antiviral compounds against RSV and other paramyxoviruses.