Rapid response teams are intended to improve early diagnosis and intervention in ward patients who develop acute respiratory or circulatory failure. A management protocol including the use of a handheld ultrasound device for immediate point-of-care ultrasound (POCUS) examination at the bedside may improve team performance. The main objective of the study was to assess the impact of implementing such a POCUS-guided management on the proportion of adequate immediate diagnoses in two groups. Secondary endpoints included time to treatment and patient outcomes. Methods A prospective, observational, controlled study was conducted in a single university hospital. Two teams alternated every other day for managing in-hospital ward patients developing acute respiratory and/or circulatory failures. Only one of the team used an ultrasound device (POCUS group). Results We included 165 patients (POCUS group 83, control group 82). Proportion of adequate immediate diagnoses was 94% in the POCUS group and 80% in the control group ( p = 0.009). Time to first treatment/intervention was shorter in the POCUS group (15 [10–25] min vs. 34 [15–40] min, p < 0.001). In-hospital mortality rates were 17% in the POCUS group and 35% in the control group ( p = 0.007), but this difference was not confirmed in the propensity score sample (29% vs. 34%, p = 0.53). Conclusion Our study suggests that protocolized use of a handheld POCUS device at the bedside in the ward may improve the proportion of adequate diagnosis, the time to initial treatment and perhaps also survival of ward patients developing acute respiratory or circulatory failure. Clinical Trial Registration NCT02967809. Registered 18 November 2016, https://clinicaltrials.gov/ct2/show/NCT02967809 .