Amanda C Schondelmeyer, MD, MSc

Hospital discharge is a complex, multi-stakeholder event, and evidence suggests that the quality of that transition directly relates to mortality, readmissions, and postdischarge quality of life and functional status.¹ The Centers for Medicare & Medicaid Services call for team-based and patient-centered discharge planning,² yet the process for achieving this is poorly defined.

In this issue of the Journal of Hospital Medicine, Manges et al³ use shared mental models (SMM) as a conceptual framework to describe differences in how care team members and patients perceive hospital discharge readiness. While our understanding of factors associated with safe and patient-centered hospital discharges is still growing, the authors focus on one critical component: lack of agreement between patients and interprofessional teams regarding discharge readiness.

Manges et al³ measured whether interprofessional team members agree, or converge, on their assessment of a patient’s discharge readiness (team-SMM convergence) and whether that assessment converges with the patient’s self-assessment (team-patient SMM convergence). They found good team-SMM convergence regarding the patient’s discharge readiness, yet teams overestimated readiness compared with the patient’s self-assessment nearly half (48.4%) of the time. A clinical trial found that clinician assessments of discharge readiness were poorly predictive of readmissions unless they were combined with a patient’s self-assessment.⁴ Manges et al’s study findings, while of limited generalizability, enhance our understanding of a potential gap in achieving patient-centered care as outlined in the Institute of Medicine’s Crossing the Quality Chasm,⁵ which urges clinicians to see patients and families as partners in improving care.

The authors also found that higher team-patient convergence was associated with teams that reported high-quality teamwork and those having more baccalaureate degree−educated nurses (BSN). While Manges et al³ did not elucidate the mechanism by which this occurs, their findings align with existing literature showing that patients receiving care from a higher proportion of BSN-prepared nurses experience an 18.7% reduction in odds of readmission.⁶ Further research investigating the link between team communication, registered nurse education, and discharge outcomes may reveal additional opportunities for interventions to improve discharge quality.

The lack of patient outcomes and the limited diversity of the patient population are substantial limitations of the study. The authors did not assess the relationship between SMMs and important outcomes like readmission or adverse events. Furthermore, most of the patients were White and English-speaking, precluding assessment of factors that disproportionately impact patient populations that already experience disparities in a multitude of health outcomes.

In summary, Manges et al³ highlight challenges and opportunities in optimizing clinician communication and ensuring that the team’s and the patient’s self-assessments align and inform discharge planning. Their findings suggest the theoretical framework of SMM holds promise in identifying and evaluating some of the complex determinants involved in high-quality, patient-centered hospital discharges.

Hospital discharge is a complex, multi-stakeholder event, and evidence suggests that the quality of that transition directly relates to mortality, readmissions, and postdischarge quality of life and functional status.¹ The Centers for Medicare & Medicaid Services call for team-based and patient-centered discharge planning,² yet the process for achieving this is poorly defined.

In this issue of the Journal of Hospital Medicine, Manges et al³ use shared mental models (SMM) as a conceptual framework to describe differences in how care team members and patients perceive hospital discharge readiness. While our understanding of factors associated with safe and patient-centered hospital discharges is still growing, the authors focus on one critical component: lack of agreement between patients and interprofessional teams regarding discharge readiness.

Manges et al³ measured whether interprofessional team members agree, or converge, on their assessment of a patient’s discharge readiness (team-SMM convergence) and whether that assessment converges with the patient’s self-assessment (team-patient SMM convergence). They found good team-SMM convergence regarding the patient’s discharge readiness, yet teams overestimated readiness compared with the patient’s self-assessment nearly half (48.4%) of the time. A clinical trial found that clinician assessments of discharge readiness were poorly predictive of readmissions unless they were combined with a patient’s self-assessment.⁴ Manges et al’s study findings, while of limited generalizability, enhance our understanding of a potential gap in achieving patient-centered care as outlined in the Institute of Medicine’s Crossing the Quality Chasm,⁵ which urges clinicians to see patients and families as partners in improving care.

The authors also found that higher team-patient convergence was associated with teams that reported high-quality teamwork and those having more baccalaureate degree−educated nurses (BSN). While Manges et al³ did not elucidate the mechanism by which this occurs, their findings align with existing literature showing that patients receiving care from a higher proportion of BSN-prepared nurses experience an 18.7% reduction in odds of readmission.⁶ Further research investigating the link between team communication, registered nurse education, and discharge outcomes may reveal additional opportunities for interventions to improve discharge quality.

The lack of patient outcomes and the limited diversity of the patient population are substantial limitations of the study. The authors did not assess the relationship between SMMs and important outcomes like readmission or adverse events. Furthermore, most of the patients were White and English-speaking, precluding assessment of factors that disproportionately impact patient populations that already experience disparities in a multitude of health outcomes.

In summary, Manges et al³ highlight challenges and opportunities in optimizing clinician communication and ensuring that the team’s and the patient’s self-assessments align and inform discharge planning. Their findings suggest the theoretical framework of SMM holds promise in identifying and evaluating some of the complex determinants involved in high-quality, patient-centered hospital discharges.

Hospital discharge is a complex, multi-stakeholder event, and evidence suggests that the quality of that transition directly relates to mortality, readmissions, and postdischarge quality of life and functional status.¹ The Centers for Medicare & Medicaid Services call for team-based and patient-centered discharge planning,² yet the process for achieving this is poorly defined.

In this issue of the Journal of Hospital Medicine, Manges et al³ use shared mental models (SMM) as a conceptual framework to describe differences in how care team members and patients perceive hospital discharge readiness. While our understanding of factors associated with safe and patient-centered hospital discharges is still growing, the authors focus on one critical component: lack of agreement between patients and interprofessional teams regarding discharge readiness.

Manges et al³ measured whether interprofessional team members agree, or converge, on their assessment of a patient’s discharge readiness (team-SMM convergence) and whether that assessment converges with the patient’s self-assessment (team-patient SMM convergence). They found good team-SMM convergence regarding the patient’s discharge readiness, yet teams overestimated readiness compared with the patient’s self-assessment nearly half (48.4%) of the time. A clinical trial found that clinician assessments of discharge readiness were poorly predictive of readmissions unless they were combined with a patient’s self-assessment.⁴ Manges et al’s study findings, while of limited generalizability, enhance our understanding of a potential gap in achieving patient-centered care as outlined in the Institute of Medicine’s Crossing the Quality Chasm,⁵ which urges clinicians to see patients and families as partners in improving care.

The authors also found that higher team-patient convergence was associated with teams that reported high-quality teamwork and those having more baccalaureate degree−educated nurses (BSN). While Manges et al³ did not elucidate the mechanism by which this occurs, their findings align with existing literature showing that patients receiving care from a higher proportion of BSN-prepared nurses experience an 18.7% reduction in odds of readmission.⁶ Further research investigating the link between team communication, registered nurse education, and discharge outcomes may reveal additional opportunities for interventions to improve discharge quality.

The lack of patient outcomes and the limited diversity of the patient population are substantial limitations of the study. The authors did not assess the relationship between SMMs and important outcomes like readmission or adverse events. Furthermore, most of the patients were White and English-speaking, precluding assessment of factors that disproportionately impact patient populations that already experience disparities in a multitude of health outcomes.

In summary, Manges et al³ highlight challenges and opportunities in optimizing clinician communication and ensuring that the team’s and the patient’s self-assessments align and inform discharge planning. Their findings suggest the theoretical framework of SMM holds promise in identifying and evaluating some of the complex determinants involved in high-quality, patient-centered hospital discharges.

Continuous pulse oximetry monitoring (cSpO₂) in children with bronchiolitis is associated with increased rates of hospital admission, longer lengths of stay, more frequent treatment with supplemental oxygen, alarm fatigue, and higher hospital cost. There is no evidence that it improves clinical outcomes.^1-7 The safety of reducing cSpO₂ for stable bronchiolitis patients (ie, those who are clinically well and not requiring supplemental oxygen) has been assessed in quality improvement initiatives^8-10 and a randomized controlled trial.² These studies showed no increase in intensive care unit transfers, codes, or readmissions associated with reduced cSpO₂. Current national guidelines from the American Academy of Pediatrics⁵ and the Society of Hospital Medicine Choosing Wisely in Pediatric Hospital Medicine workgroup⁴ support limiting monitoring of children with bronchiolitis. Despite this, the practice of cSpO₂ in stable bronchiolitis patients off supplemental oxygen remains widespread.^11,12

Deimplementation, defined as reducing or stopping low-value or ineffective healthcare practices,^13,14 is a discrete focus area within implementation science. Deimplementation research involves the reduction of unnecessary and overused services for which there is potential for harm or no benefit.^15,16 In pediatrics, there are a number of potential targets for deimplementation,^4,17-20 including cSpO₂ for stable infants with bronchiolitis, but efforts to reduce low-value practices have met limited success to date. ^21,22

Implementation science offers rigorous methods for advancing the development and evaluation of strategies for deimplementation.²³ In particular, implementation science frameworks can facilitate our understanding of relevant contextual factors that may hinder or help efforts to deimplement low-value practices. To develop broadly applicable strategies to reduce monitoring overuse, it is important to understand the barriers, facilitators, and contextual factors (eg, clinical, political, interpersonal) that contribute to guideline-discordant cSpO₂ in hospitalized bronchiolitis patients. Further, the process by which one can develop a rigorous understanding of these factors and how they may impact deimplementation efforts could generalize to other scenarios in pediatrics where overuse remains an issue.

The goal of this study was to use semistructured interviews, informed by an established implementation science framework, specifically the Consolidated Framework for Implementation Research (CFIR),²⁴ to (1) identify barriers and facilitators to deimplementing unnecessary cSpO₂, and (2) develop strategies to deimplement cSpO₂ in a multicenter cohort of hospital-based clinician and administrative stakeholders.

Study Setting

This multicenter qualitative study using semistructured interviews took place within the Eliminating Monitor Overuse (EMO) SpO₂ study. The EMO SpO₂ study established rates of cSpO₂ in bronchiolitis patients not receiving supplemental oxygen or not receiving room air flow at 56 hospitals across the United States and in Canada from December 1, 2018, through March 31, 2019.¹² The study identified hospital-level risk-adjusted cSpO₂ rates ranging from 6% to 82%. A description of the EMO SpO₂ study methods²⁵ and its findings¹² have been published elsewhere.

Participants

We approached EMO study site principal investigators at 12 hospitals: the two highest- and two lowest-use hospitals within three hospital types (ie, freestanding children’s hospitals, children’s hospitals within large general hospitals, and community hospitals). We collaborated with the participating site principal investigators (n = 12), who were primarily hospitalist physicians in leadership roles, to recruit a purposive sample of additional stakeholders including bedside nurses (n = 12), hospitalist physicians (n = 15), respiratory therapists (n = 9), and hospital administrators (n = 8) to participate in semistructured interviews. Interviews were conducted until we achieved thematic saturation within each stakeholder group and within the high and low performing strata (total 56 interviews). Participants were asked to self-report basic demographic information (see Appendix, interview guide) as required by the study funder and to allow us to comment on the representativeness of the participant group.

Procedure

The interview guide was informed by the CFIR, a comprehensive framework detailing contextual factors that require consideration when planning for the implementation of a health service intervention. Table 1 details the CFIR domains with study-related examples. The interview guide (Appendix) provided limited clinical context apart from the age, diagnosis, and oxygen requirement for the population of interest to promote a broad array of responses and to avoid anchoring on specific clinical scenarios. Interviews were conducted by master’s degree or doctoral-level research coordinators with qualitative interviewing experience and supervised by a medical anthropologist and qualitative methods expert (F.K.B.). Prior to engaging in audio recorded phone interviews, the interviewer explained the risks and benefits of participating. Participants were compensated $50. Audio recordings were transcribed, deidentified, and uploaded to NVivo 12 Plus (QSR International) for data management.

The Institutional Review Boards of Children’s Hospital of Philadelphia, Pennsylvania, and the University of Pennsylvania in Philadelphia determined that the study met eligibility criteria for IRB exemption.

Data Analysis

Using an integrated approach to codebook development,²⁶ a priori codes were developed using constructs from the CFIR. Additional codes were added by the research team following a close reading of the first five transcripts.^27,28 Each code was defined, including decision rules for its application. Two research coordinators independently coded each transcript. Using the intercoder reliability function within NVivo, the coders established strong interrater reliability accordance scores (k > .8) by double coding 20% of the transcripts. Data were stratified by sites with low and high use of cSpO₂ to examine differences in barriers and facilitators to deimplementation. Each code was subcoded, summarized, and examined for patterns within and across participating disciplines, which yielded themes related to barriers and facilitators. We conducted member checking and reviewed our conclusions with a multidisciplinary group of clinical stakeholders (n = 13) to validate our analyses.

Barriers and facilitators to deimplementation were identified in multiple domains of the CFIR: outer setting, inner setting, characteristics of the individuals, and intervention characteristics (Table 1). Participants also suggested strategies to facilitate deimplementation in response to some identified barriers. See Table 2 for participant demographics and Table 3 for illustrative participant quotations.

Barriers

Outer Setting: Clinician Perceptions of Parental Discomfort With Discontinuing Monitoring

Participants mentioned parental preferences as a barrier to discontinuing cSpO₂, noting that parents seem to take comfort in watching the numbers on the monitor screen and are reluctant to have it withdrawn. Clinicians noted that parents sometimes put the monitor back on their child after a clinician removed it or have expressed concern that their unmonitored child was not receiving the same level of care as other patients who were being monitored. In these scenarios, clinicians reported they have found it helpful to educate caregivers about when cSpO₂ is and is not appropriate.

Inner Setting: Unclear or Nonexistent Guideline to Discontinue cSpO₂

Guidelines to discontinue cSpO₂ reportedly did not exist at all institutions. If a guideline did exist, lack of clarity or conflicting guidelines about when to use oxygen presented a barrier. Participants suggested that a clear guideline or additional oversight to ensure all clinicians are informed of the procedure for discontinuing cSpO₂ may help prevent miscommunication. Participants noted that their electronic health record (EHR) order sets commonly included cSpO₂ orders and that removing that option would facilitate deimplementation.

Inner Setting: Difficulty Educating All Staff

Participants noted difficulty with incorporating education about discontinuing cSpO₂ to all clinicians, particularly to those who are nightshift only or to rotating staff or trainees. This created barriers for frequent re-education because these staff are not familiar with the policies and procedures of the unit, which is crucial to developing a culture that supports the deimplementation of cSpO₂. Participants suggested that recurring education about procedures for discontinuing cSpO₂ should target trainees, new nurses, and overnight nurses. This would help to ensure that the guideline is uniformly followed.

Inner Setting: Culture of High cSpO₂ Use

Participants from high-use sites discussed a culture driven by readily available monitoring features or an expectation that monitoring indicates higher-quality care. Participants from low-use sites discussed increased cSpO₂ driven by clinicians who were accustomed to caring for higher-acuity patients, for whom continuous monitoring is likely appropriate, and were simultaneously caring for stable bronchiolitis patients.

Some suggested that visual cues would be useful to clinicians to sustain awareness about a cSpO₂ deimplementation guideline. It was also suggested that audit and feedback techniques like posting unit deimplementation statistics and creating a competition among units by posting unit performance could facilitate deimplementation. Additionally, some noted that visual aids in common spaces would be useful to remind clinicians and to engage caregivers about discontinuing cSpO₂.

Characteristics of Individuals: Clinician Discomfort Discontinuing cSpO₂

One frequently cited barrier across participants is that cSpO₂ provides “peace of mind” to alert clinicians to patients with low oxygen saturations that might otherwise be missed. Participants identified that clinician discomfort with reducing cSpO₂ may be driven by inexperienced clinicians less familiar with the bronchiolitis disease process, such as trainees, new nurses, or rotating clinicians unaccustomed to pediatric care. Trainees and new nurses were perceived as being more likely to work at night when there are fewer clinicians to provide patient care. Additionally, participants perceived that night shift clinicians favored cSpO₂ because they could measure vital signs without waking patients and families.

Clinicians discussed that discontinuing cSpO₂ would require alternative methods for assessing patient status, particularly for night shift nurses. Participants suggested strategies including changes to pulse oximetry assessment procedures to include more frequent “spot checks,” incorporation of assessments during sleep events (eg, naps) to ensure the patient does not experience desaturations during sleep, and training nurses to become more comfortable with suctioning patients. Suggestions also included education on the typical features of transient oxygen desaturations in otherwise stable patients with bronchiolitis² to bolster clinical confidence for clinicians unfamiliar with caring for bronchiolitis patients. Participants perceived that education about appropriate vs inappropriate use may help to empower clinicians to employ cSpO₂ appropriately.

Facilitators

Outer Setting: Standards and Evidence From Research, Professional Organizations, and Leaders in the Field

Many participants expressed the importance of consistent guidelines that are advocated by thought leaders in the field, supported by robust evidence, and consistent with approaches at peer hospitals. The more authoritative support a guideline has, the more comfortable people are adopting it and taking it seriously. Additionally, consistent education about guidelines was desired. Participants noted that all clinicians should be receiving education related to the American Academy of Pediatrics (AAP) Bronchiolitis and Choosing Wisely^® guidelines, ranging from a one-time update to annually. Continual updates and re-education sessions for clinicians who shared evidence about how cSpO₂ deimplementation could improve the quality of patient care by shortening hospital length of stay and lowering cost were suggested strategies.

Inner Setting: Leadership

Participants noted that successful deimplementation depends upon the presence of a champion or educator who will be able to lead the institutional charge in making practice change. This is typically an individual who is trusted at the institution, experienced in their field, or already doing implementation work. This could be either a single individual (champion) or a team. The most commonly noted clinician roles to engage in a leadership role or team were physicians and nurses.

Participants noted that a change in related clinical care pathways or EHR order sets would require cooperation from multiple clinical disciplines, administrators, and information technology leaders and explained that messaging and education about the value of the change would facilitate buy-in from those clinicians.

Inner Setting: EHR Support for Guidelines

Participants often endorsed the use of an order set within the EHR that supports guidelines and includes reminders to decrease cSpO₂. These reminders could come up when supplemental oxygen is discontinued or occur regularly throughout the patient’s stay to prompt the clinician to consider discontinuing cSpO₂.

Intervention Characteristics/Inner Setting: Clear Bronchiolitis Guidelines

The presence of a well-articulated hospital policy that delineates the appropriate and inappropriate use of cSpO₂ in bronchiolitis was mentioned as another facilitator of deimplementation.

Results of this qualitative study of stakeholders across hospitals with high and low cSpO₂ use illustrated the complexities involved with deimplementation of cSpO₂ in pediatric patients hospitalized with bronchiolitis. We identified numerous barriers spanning the CFIR constructs, including unclear or absent guidelines for stopping cSpO₂, clinician knowledge and comfort with bronchiolitis disease features, and unit culture. This suggests that multicomponent strategies that target various domains and a variety of stakeholders are needed to deimplement cSpO₂ use for stable bronchiolitis patients. Participants also identified facilitators, including clear cSpO₂ guidelines, supportive leaders and champions, and EHR modifications, that provide insight into strategies that may help sites reduce their use of cSpO₂. Additionally, participants also provided concrete, actionable suggestions for ways to reduce unnecessary monitoring that will be useful in informing promising deimplementation strategies for subsequent trials.

The importance of having specific and well-known guidelines from trusted sources, such as the AAP, about cSpO₂ and bronchiolitis treatment that are thoughtfully integrated in the EHR came through in multiple themes of our analysis. Prior studies on the effect of guidelines on clinical practice have suggested that rigorously designed guidelines can positively impact practice.²⁹ Participants also noted that cSpO₂ guidelines should be authoritative and that knowledge of guideline adoption by peer institutions was a facilitator of adoption. Usability issues negatively impact clinicians’ ability to follow guidelines.³⁰ Further, prior studies have demonstrated that EHR integration of guidelines can change practice.^31-33 Based on our findings, incorporating clear guidelines into commonly used formats, such as EHR order sets, could be an important deimplementation tool for cSpO₂ in stable bronchiolitis patients.

Education about and awareness of cSpO₂ guidelines was described as an important facilitator for appropriate cSpO₂ use and was suggested as a potential deimplementation strategy. Participants noted that educational need may vary by stakeholder group. For example, education may facilitate obtaining buy-in from hospital leaders, which is necessary to support changes to the EHR. Education incorporating information on the typical features of bronchiolitis and examples of appropriate and inappropriate cSpO₂ use was suggested for clinical team members. The limitations of education as a stand-alone deimplementation strategy were also noted, and participants highlighted challenges such as time needed for education and the need for ongoing education for rotating trainees. Inner and outer setting barriers, such as a perceived “culture of high pulse oximetryuse” and patient and family expectations, could also make education less effective as a stand-alone strategy. That—coupled with evidence that education and training alone are generally insufficient for producing reliable, sustained behavior change^34,35—suggests that a multifaceted approach will be important.

Our respondents consider parental perceptions and preference in their practice, which provides nuance to recent studies suggesting that parents prefer continuous monitors when their child is hospitalized with bronchiolitis. Chi et al described the impact of a brief educational intervention on parental preferences for monitoring children hospitalized for bronchiolitis.³⁶ This work suggests that educational interventions aimed at families should be considered in future (de)implementation studies because they may indirectly impact clinician behavior. Future studies should directly assess parental discomfort with discontinuing monitoring. Participants highlighted the link between knowledge and confidence in caring for typical bronchiolitis patients and monitoring practice, perceiving that less experienced clinicians are more likely to rely on cSpO₂. Participants at high-use sites emphasized the expectation that monitoring should occur during hospitalizations. This reflection is particularly pertinent for bronchiolitis, a disease characterized by frequent, self-resolving desaturations even after hospital discharge.³ This may reinforce a perceived need to capture and react to these desaturation events even though they are expected in bronchiolitis and can occur in healthy infants.³⁷ Some participants suggested that continuous monitoring be replaced with “nap tests” (ie, assessment for desaturations during a nap prior to discharge); however, like cSpO₂ in stable infants with bronchiolitis, this is another low-value practice. Otherwise healthy infants with mild to moderate disease are unlikely to subsequently worsen after showing signs of clinical improvement.³⁸ Nap tests are likely to lead to infants who are clinically improving being placed unnecessarily back on oxygen in reaction to the transient desaturations. Participants’ perception about the importance of cSpO₂ in bronchiolitis management, despite evidence suggesting it is a low-value practice, underscores the importance of not simply telling clinicians to stop cSpO₂. Employing strategies that replace continuous monitoring with another acceptable and feasible alternative (eg, regular clinician assessments including intermittent pulse oximetry checks) should be considered when planning for deimplementation.³⁹

Previous studies indicate that continuous monitoring can affect clinician decision-making, independent of other factors,^6,40 despite limited evidence that continuous monitors improve patient outcomes.^1-7 Studies have demonstrated noticeable increase in admissions based purely on pulse oximetry values,⁴⁰ with no evidence that this type of admission changes outcomes for bronchiolitis patients.⁶ One previous, single-center study identified inexperience as a potential driver for monitor use,⁴¹ and studies in adult populations have suggested that clinicians overestimate the value that continuous monitoring contributes to patient care,^42,43 which promotes guideline-discordant use. Our study provides novel insight into the issue of monitoring in bronchiolitis. Our results suggest that there is a need to shift organizational cultures around monitoring (which likely vary based on a range of factors) and that educational strategies addressing typical disease course, especially desaturations, in bronchiolitis will be an essential component in any deimplementation effort.

This study is strengthened by its sample of diverse stakeholder groups from multiple US health systems. Additionally, we interviewed individuals at sites with high cSpO₂ rates and at sites with low rates, as well as from community hospitals, children’s hospitals within general hospitals, and freestanding children’s hospitals, which allows us to understand barriers high-use sites encounter and facilitators of lower cSpO₂ rates at low-use sites. We also employed an interview approach informed by an established implementation science framework. Nonetheless, several limitations exist. First, participants at low-use sites did not necessarily have direct experience with a previous deimplementation effort to reduce cSpO₂. Additionally, participants were predominantly White and female; more diverse perspectives would strengthen confidence in the generalizability of our findings. While thematic saturation was achieved within each stakeholder group and within the high- and low-use strata, we interviewed fewer administrators and respiratory therapists relative to other stakeholder groups. Nevertheless, our conclusions were validated by our interdisciplinary stakeholder panel. As noted by participants, family preferences may influence clinician practice, and parents were not interviewed for this study. The information gleaned from the present study will inform the development of strategies to deimplement unnecessary cSpO₂ in pediatric hospitals, which we aim to rigorously evaluate in a future trial.

We identified barriers and facilitators to deimplementation of cSpO₂ for stable patients with bronchiolitis across children’s hospitals with high and low utilization of cSpO₂. These themes map to multiple CFIR domains and, along with participant-suggested strategies, can directly inform an approach to cSpO₂ deimplementation in a range of inpatient settings. Based on these data, future deimplementation efforts should focus on clear protocols for use and discontinuation of cSpO₂, EHR changes, and regular bronchiolitis education for hospital staff that emphasizes reducing unnecessary cSpO₂ utilization.

We acknowledge the NHLBI scientists who contributed their expertise to this project as part of the U01 Cooperative Agreement funding mechanism as federal employees conducting their official job duties: Lora Reineck, MD, MS, Karen Bienstock, MS, and Cheryl Boyce, PhD. We thank the Executive Council of the Pediatric Research in Inpatient Settings (PRIS) Network for their contributions to the early scientific development of this project. The Network assessed a Collaborative Support Fee for access to the hospitals and support of this project. We thank the PRIS Network collaborators for their major contributions to data collection measuring utilization to identify the hospitals we subsequently chose for this project. We thank Claire Bocage and the Mixed Methods Research Lab for major help in data management and data analysis.

Continuous pulse oximetry monitoring (cSpO₂) in children with bronchiolitis is associated with increased rates of hospital admission, longer lengths of stay, more frequent treatment with supplemental oxygen, alarm fatigue, and higher hospital cost. There is no evidence that it improves clinical outcomes.^1-7 The safety of reducing cSpO₂ for stable bronchiolitis patients (ie, those who are clinically well and not requiring supplemental oxygen) has been assessed in quality improvement initiatives^8-10 and a randomized controlled trial.² These studies showed no increase in intensive care unit transfers, codes, or readmissions associated with reduced cSpO₂. Current national guidelines from the American Academy of Pediatrics⁵ and the Society of Hospital Medicine Choosing Wisely in Pediatric Hospital Medicine workgroup⁴ support limiting monitoring of children with bronchiolitis. Despite this, the practice of cSpO₂ in stable bronchiolitis patients off supplemental oxygen remains widespread.^11,12

Deimplementation, defined as reducing or stopping low-value or ineffective healthcare practices,^13,14 is a discrete focus area within implementation science. Deimplementation research involves the reduction of unnecessary and overused services for which there is potential for harm or no benefit.^15,16 In pediatrics, there are a number of potential targets for deimplementation,^4,17-20 including cSpO₂ for stable infants with bronchiolitis, but efforts to reduce low-value practices have met limited success to date. ^21,22

Implementation science offers rigorous methods for advancing the development and evaluation of strategies for deimplementation.²³ In particular, implementation science frameworks can facilitate our understanding of relevant contextual factors that may hinder or help efforts to deimplement low-value practices. To develop broadly applicable strategies to reduce monitoring overuse, it is important to understand the barriers, facilitators, and contextual factors (eg, clinical, political, interpersonal) that contribute to guideline-discordant cSpO₂ in hospitalized bronchiolitis patients. Further, the process by which one can develop a rigorous understanding of these factors and how they may impact deimplementation efforts could generalize to other scenarios in pediatrics where overuse remains an issue.

The goal of this study was to use semistructured interviews, informed by an established implementation science framework, specifically the Consolidated Framework for Implementation Research (CFIR),²⁴ to (1) identify barriers and facilitators to deimplementing unnecessary cSpO₂, and (2) develop strategies to deimplement cSpO₂ in a multicenter cohort of hospital-based clinician and administrative stakeholders.

Study Setting

This multicenter qualitative study using semistructured interviews took place within the Eliminating Monitor Overuse (EMO) SpO₂ study. The EMO SpO₂ study established rates of cSpO₂ in bronchiolitis patients not receiving supplemental oxygen or not receiving room air flow at 56 hospitals across the United States and in Canada from December 1, 2018, through March 31, 2019.¹² The study identified hospital-level risk-adjusted cSpO₂ rates ranging from 6% to 82%. A description of the EMO SpO₂ study methods²⁵ and its findings¹² have been published elsewhere.

Participants

We approached EMO study site principal investigators at 12 hospitals: the two highest- and two lowest-use hospitals within three hospital types (ie, freestanding children’s hospitals, children’s hospitals within large general hospitals, and community hospitals). We collaborated with the participating site principal investigators (n = 12), who were primarily hospitalist physicians in leadership roles, to recruit a purposive sample of additional stakeholders including bedside nurses (n = 12), hospitalist physicians (n = 15), respiratory therapists (n = 9), and hospital administrators (n = 8) to participate in semistructured interviews. Interviews were conducted until we achieved thematic saturation within each stakeholder group and within the high and low performing strata (total 56 interviews). Participants were asked to self-report basic demographic information (see Appendix, interview guide) as required by the study funder and to allow us to comment on the representativeness of the participant group.

Procedure

The interview guide was informed by the CFIR, a comprehensive framework detailing contextual factors that require consideration when planning for the implementation of a health service intervention. Table 1 details the CFIR domains with study-related examples. The interview guide (Appendix) provided limited clinical context apart from the age, diagnosis, and oxygen requirement for the population of interest to promote a broad array of responses and to avoid anchoring on specific clinical scenarios. Interviews were conducted by master’s degree or doctoral-level research coordinators with qualitative interviewing experience and supervised by a medical anthropologist and qualitative methods expert (F.K.B.). Prior to engaging in audio recorded phone interviews, the interviewer explained the risks and benefits of participating. Participants were compensated $50. Audio recordings were transcribed, deidentified, and uploaded to NVivo 12 Plus (QSR International) for data management.

The Institutional Review Boards of Children’s Hospital of Philadelphia, Pennsylvania, and the University of Pennsylvania in Philadelphia determined that the study met eligibility criteria for IRB exemption.

Data Analysis

Using an integrated approach to codebook development,²⁶ a priori codes were developed using constructs from the CFIR. Additional codes were added by the research team following a close reading of the first five transcripts.^27,28 Each code was defined, including decision rules for its application. Two research coordinators independently coded each transcript. Using the intercoder reliability function within NVivo, the coders established strong interrater reliability accordance scores (k > .8) by double coding 20% of the transcripts. Data were stratified by sites with low and high use of cSpO₂ to examine differences in barriers and facilitators to deimplementation. Each code was subcoded, summarized, and examined for patterns within and across participating disciplines, which yielded themes related to barriers and facilitators. We conducted member checking and reviewed our conclusions with a multidisciplinary group of clinical stakeholders (n = 13) to validate our analyses.

Barriers and facilitators to deimplementation were identified in multiple domains of the CFIR: outer setting, inner setting, characteristics of the individuals, and intervention characteristics (Table 1). Participants also suggested strategies to facilitate deimplementation in response to some identified barriers. See Table 2 for participant demographics and Table 3 for illustrative participant quotations.

Barriers

Outer Setting: Clinician Perceptions of Parental Discomfort With Discontinuing Monitoring

Participants mentioned parental preferences as a barrier to discontinuing cSpO₂, noting that parents seem to take comfort in watching the numbers on the monitor screen and are reluctant to have it withdrawn. Clinicians noted that parents sometimes put the monitor back on their child after a clinician removed it or have expressed concern that their unmonitored child was not receiving the same level of care as other patients who were being monitored. In these scenarios, clinicians reported they have found it helpful to educate caregivers about when cSpO₂ is and is not appropriate.

Inner Setting: Unclear or Nonexistent Guideline to Discontinue cSpO₂

Guidelines to discontinue cSpO₂ reportedly did not exist at all institutions. If a guideline did exist, lack of clarity or conflicting guidelines about when to use oxygen presented a barrier. Participants suggested that a clear guideline or additional oversight to ensure all clinicians are informed of the procedure for discontinuing cSpO₂ may help prevent miscommunication. Participants noted that their electronic health record (EHR) order sets commonly included cSpO₂ orders and that removing that option would facilitate deimplementation.

Inner Setting: Difficulty Educating All Staff

Participants noted difficulty with incorporating education about discontinuing cSpO₂ to all clinicians, particularly to those who are nightshift only or to rotating staff or trainees. This created barriers for frequent re-education because these staff are not familiar with the policies and procedures of the unit, which is crucial to developing a culture that supports the deimplementation of cSpO₂. Participants suggested that recurring education about procedures for discontinuing cSpO₂ should target trainees, new nurses, and overnight nurses. This would help to ensure that the guideline is uniformly followed.

Inner Setting: Culture of High cSpO₂ Use

Participants from high-use sites discussed a culture driven by readily available monitoring features or an expectation that monitoring indicates higher-quality care. Participants from low-use sites discussed increased cSpO₂ driven by clinicians who were accustomed to caring for higher-acuity patients, for whom continuous monitoring is likely appropriate, and were simultaneously caring for stable bronchiolitis patients.

Some suggested that visual cues would be useful to clinicians to sustain awareness about a cSpO₂ deimplementation guideline. It was also suggested that audit and feedback techniques like posting unit deimplementation statistics and creating a competition among units by posting unit performance could facilitate deimplementation. Additionally, some noted that visual aids in common spaces would be useful to remind clinicians and to engage caregivers about discontinuing cSpO₂.

Characteristics of Individuals: Clinician Discomfort Discontinuing cSpO₂

One frequently cited barrier across participants is that cSpO₂ provides “peace of mind” to alert clinicians to patients with low oxygen saturations that might otherwise be missed. Participants identified that clinician discomfort with reducing cSpO₂ may be driven by inexperienced clinicians less familiar with the bronchiolitis disease process, such as trainees, new nurses, or rotating clinicians unaccustomed to pediatric care. Trainees and new nurses were perceived as being more likely to work at night when there are fewer clinicians to provide patient care. Additionally, participants perceived that night shift clinicians favored cSpO₂ because they could measure vital signs without waking patients and families.

Clinicians discussed that discontinuing cSpO₂ would require alternative methods for assessing patient status, particularly for night shift nurses. Participants suggested strategies including changes to pulse oximetry assessment procedures to include more frequent “spot checks,” incorporation of assessments during sleep events (eg, naps) to ensure the patient does not experience desaturations during sleep, and training nurses to become more comfortable with suctioning patients. Suggestions also included education on the typical features of transient oxygen desaturations in otherwise stable patients with bronchiolitis² to bolster clinical confidence for clinicians unfamiliar with caring for bronchiolitis patients. Participants perceived that education about appropriate vs inappropriate use may help to empower clinicians to employ cSpO₂ appropriately.

Facilitators

Outer Setting: Standards and Evidence From Research, Professional Organizations, and Leaders in the Field

Many participants expressed the importance of consistent guidelines that are advocated by thought leaders in the field, supported by robust evidence, and consistent with approaches at peer hospitals. The more authoritative support a guideline has, the more comfortable people are adopting it and taking it seriously. Additionally, consistent education about guidelines was desired. Participants noted that all clinicians should be receiving education related to the American Academy of Pediatrics (AAP) Bronchiolitis and Choosing Wisely^® guidelines, ranging from a one-time update to annually. Continual updates and re-education sessions for clinicians who shared evidence about how cSpO₂ deimplementation could improve the quality of patient care by shortening hospital length of stay and lowering cost were suggested strategies.

Inner Setting: Leadership

Participants noted that successful deimplementation depends upon the presence of a champion or educator who will be able to lead the institutional charge in making practice change. This is typically an individual who is trusted at the institution, experienced in their field, or already doing implementation work. This could be either a single individual (champion) or a team. The most commonly noted clinician roles to engage in a leadership role or team were physicians and nurses.

Participants noted that a change in related clinical care pathways or EHR order sets would require cooperation from multiple clinical disciplines, administrators, and information technology leaders and explained that messaging and education about the value of the change would facilitate buy-in from those clinicians.

Inner Setting: EHR Support for Guidelines

Participants often endorsed the use of an order set within the EHR that supports guidelines and includes reminders to decrease cSpO₂. These reminders could come up when supplemental oxygen is discontinued or occur regularly throughout the patient’s stay to prompt the clinician to consider discontinuing cSpO₂.

Intervention Characteristics/Inner Setting: Clear Bronchiolitis Guidelines

The presence of a well-articulated hospital policy that delineates the appropriate and inappropriate use of cSpO₂ in bronchiolitis was mentioned as another facilitator of deimplementation.

Results of this qualitative study of stakeholders across hospitals with high and low cSpO₂ use illustrated the complexities involved with deimplementation of cSpO₂ in pediatric patients hospitalized with bronchiolitis. We identified numerous barriers spanning the CFIR constructs, including unclear or absent guidelines for stopping cSpO₂, clinician knowledge and comfort with bronchiolitis disease features, and unit culture. This suggests that multicomponent strategies that target various domains and a variety of stakeholders are needed to deimplement cSpO₂ use for stable bronchiolitis patients. Participants also identified facilitators, including clear cSpO₂ guidelines, supportive leaders and champions, and EHR modifications, that provide insight into strategies that may help sites reduce their use of cSpO₂. Additionally, participants also provided concrete, actionable suggestions for ways to reduce unnecessary monitoring that will be useful in informing promising deimplementation strategies for subsequent trials.

The importance of having specific and well-known guidelines from trusted sources, such as the AAP, about cSpO₂ and bronchiolitis treatment that are thoughtfully integrated in the EHR came through in multiple themes of our analysis. Prior studies on the effect of guidelines on clinical practice have suggested that rigorously designed guidelines can positively impact practice.²⁹ Participants also noted that cSpO₂ guidelines should be authoritative and that knowledge of guideline adoption by peer institutions was a facilitator of adoption. Usability issues negatively impact clinicians’ ability to follow guidelines.³⁰ Further, prior studies have demonstrated that EHR integration of guidelines can change practice.^31-33 Based on our findings, incorporating clear guidelines into commonly used formats, such as EHR order sets, could be an important deimplementation tool for cSpO₂ in stable bronchiolitis patients.

Education about and awareness of cSpO₂ guidelines was described as an important facilitator for appropriate cSpO₂ use and was suggested as a potential deimplementation strategy. Participants noted that educational need may vary by stakeholder group. For example, education may facilitate obtaining buy-in from hospital leaders, which is necessary to support changes to the EHR. Education incorporating information on the typical features of bronchiolitis and examples of appropriate and inappropriate cSpO₂ use was suggested for clinical team members. The limitations of education as a stand-alone deimplementation strategy were also noted, and participants highlighted challenges such as time needed for education and the need for ongoing education for rotating trainees. Inner and outer setting barriers, such as a perceived “culture of high pulse oximetryuse” and patient and family expectations, could also make education less effective as a stand-alone strategy. That—coupled with evidence that education and training alone are generally insufficient for producing reliable, sustained behavior change^34,35—suggests that a multifaceted approach will be important.

Our respondents consider parental perceptions and preference in their practice, which provides nuance to recent studies suggesting that parents prefer continuous monitors when their child is hospitalized with bronchiolitis. Chi et al described the impact of a brief educational intervention on parental preferences for monitoring children hospitalized for bronchiolitis.³⁶ This work suggests that educational interventions aimed at families should be considered in future (de)implementation studies because they may indirectly impact clinician behavior. Future studies should directly assess parental discomfort with discontinuing monitoring. Participants highlighted the link between knowledge and confidence in caring for typical bronchiolitis patients and monitoring practice, perceiving that less experienced clinicians are more likely to rely on cSpO₂. Participants at high-use sites emphasized the expectation that monitoring should occur during hospitalizations. This reflection is particularly pertinent for bronchiolitis, a disease characterized by frequent, self-resolving desaturations even after hospital discharge.³ This may reinforce a perceived need to capture and react to these desaturation events even though they are expected in bronchiolitis and can occur in healthy infants.³⁷ Some participants suggested that continuous monitoring be replaced with “nap tests” (ie, assessment for desaturations during a nap prior to discharge); however, like cSpO₂ in stable infants with bronchiolitis, this is another low-value practice. Otherwise healthy infants with mild to moderate disease are unlikely to subsequently worsen after showing signs of clinical improvement.³⁸ Nap tests are likely to lead to infants who are clinically improving being placed unnecessarily back on oxygen in reaction to the transient desaturations. Participants’ perception about the importance of cSpO₂ in bronchiolitis management, despite evidence suggesting it is a low-value practice, underscores the importance of not simply telling clinicians to stop cSpO₂. Employing strategies that replace continuous monitoring with another acceptable and feasible alternative (eg, regular clinician assessments including intermittent pulse oximetry checks) should be considered when planning for deimplementation.³⁹

Previous studies indicate that continuous monitoring can affect clinician decision-making, independent of other factors,^6,40 despite limited evidence that continuous monitors improve patient outcomes.^1-7 Studies have demonstrated noticeable increase in admissions based purely on pulse oximetry values,⁴⁰ with no evidence that this type of admission changes outcomes for bronchiolitis patients.⁶ One previous, single-center study identified inexperience as a potential driver for monitor use,⁴¹ and studies in adult populations have suggested that clinicians overestimate the value that continuous monitoring contributes to patient care,^42,43 which promotes guideline-discordant use. Our study provides novel insight into the issue of monitoring in bronchiolitis. Our results suggest that there is a need to shift organizational cultures around monitoring (which likely vary based on a range of factors) and that educational strategies addressing typical disease course, especially desaturations, in bronchiolitis will be an essential component in any deimplementation effort.

This study is strengthened by its sample of diverse stakeholder groups from multiple US health systems. Additionally, we interviewed individuals at sites with high cSpO₂ rates and at sites with low rates, as well as from community hospitals, children’s hospitals within general hospitals, and freestanding children’s hospitals, which allows us to understand barriers high-use sites encounter and facilitators of lower cSpO₂ rates at low-use sites. We also employed an interview approach informed by an established implementation science framework. Nonetheless, several limitations exist. First, participants at low-use sites did not necessarily have direct experience with a previous deimplementation effort to reduce cSpO₂. Additionally, participants were predominantly White and female; more diverse perspectives would strengthen confidence in the generalizability of our findings. While thematic saturation was achieved within each stakeholder group and within the high- and low-use strata, we interviewed fewer administrators and respiratory therapists relative to other stakeholder groups. Nevertheless, our conclusions were validated by our interdisciplinary stakeholder panel. As noted by participants, family preferences may influence clinician practice, and parents were not interviewed for this study. The information gleaned from the present study will inform the development of strategies to deimplement unnecessary cSpO₂ in pediatric hospitals, which we aim to rigorously evaluate in a future trial.

We identified barriers and facilitators to deimplementation of cSpO₂ for stable patients with bronchiolitis across children’s hospitals with high and low utilization of cSpO₂. These themes map to multiple CFIR domains and, along with participant-suggested strategies, can directly inform an approach to cSpO₂ deimplementation in a range of inpatient settings. Based on these data, future deimplementation efforts should focus on clear protocols for use and discontinuation of cSpO₂, EHR changes, and regular bronchiolitis education for hospital staff that emphasizes reducing unnecessary cSpO₂ utilization.

We acknowledge the NHLBI scientists who contributed their expertise to this project as part of the U01 Cooperative Agreement funding mechanism as federal employees conducting their official job duties: Lora Reineck, MD, MS, Karen Bienstock, MS, and Cheryl Boyce, PhD. We thank the Executive Council of the Pediatric Research in Inpatient Settings (PRIS) Network for their contributions to the early scientific development of this project. The Network assessed a Collaborative Support Fee for access to the hospitals and support of this project. We thank the PRIS Network collaborators for their major contributions to data collection measuring utilization to identify the hospitals we subsequently chose for this project. We thank Claire Bocage and the Mixed Methods Research Lab for major help in data management and data analysis.

Continuous pulse oximetry monitoring (cSpO₂) in children with bronchiolitis is associated with increased rates of hospital admission, longer lengths of stay, more frequent treatment with supplemental oxygen, alarm fatigue, and higher hospital cost. There is no evidence that it improves clinical outcomes.^1-7 The safety of reducing cSpO₂ for stable bronchiolitis patients (ie, those who are clinically well and not requiring supplemental oxygen) has been assessed in quality improvement initiatives^8-10 and a randomized controlled trial.² These studies showed no increase in intensive care unit transfers, codes, or readmissions associated with reduced cSpO₂. Current national guidelines from the American Academy of Pediatrics⁵ and the Society of Hospital Medicine Choosing Wisely in Pediatric Hospital Medicine workgroup⁴ support limiting monitoring of children with bronchiolitis. Despite this, the practice of cSpO₂ in stable bronchiolitis patients off supplemental oxygen remains widespread.^11,12

Deimplementation, defined as reducing or stopping low-value or ineffective healthcare practices,^13,14 is a discrete focus area within implementation science. Deimplementation research involves the reduction of unnecessary and overused services for which there is potential for harm or no benefit.^15,16 In pediatrics, there are a number of potential targets for deimplementation,^4,17-20 including cSpO₂ for stable infants with bronchiolitis, but efforts to reduce low-value practices have met limited success to date. ^21,22

Implementation science offers rigorous methods for advancing the development and evaluation of strategies for deimplementation.²³ In particular, implementation science frameworks can facilitate our understanding of relevant contextual factors that may hinder or help efforts to deimplement low-value practices. To develop broadly applicable strategies to reduce monitoring overuse, it is important to understand the barriers, facilitators, and contextual factors (eg, clinical, political, interpersonal) that contribute to guideline-discordant cSpO₂ in hospitalized bronchiolitis patients. Further, the process by which one can develop a rigorous understanding of these factors and how they may impact deimplementation efforts could generalize to other scenarios in pediatrics where overuse remains an issue.

The goal of this study was to use semistructured interviews, informed by an established implementation science framework, specifically the Consolidated Framework for Implementation Research (CFIR),²⁴ to (1) identify barriers and facilitators to deimplementing unnecessary cSpO₂, and (2) develop strategies to deimplement cSpO₂ in a multicenter cohort of hospital-based clinician and administrative stakeholders.

Study Setting

This multicenter qualitative study using semistructured interviews took place within the Eliminating Monitor Overuse (EMO) SpO₂ study. The EMO SpO₂ study established rates of cSpO₂ in bronchiolitis patients not receiving supplemental oxygen or not receiving room air flow at 56 hospitals across the United States and in Canada from December 1, 2018, through March 31, 2019.¹² The study identified hospital-level risk-adjusted cSpO₂ rates ranging from 6% to 82%. A description of the EMO SpO₂ study methods²⁵ and its findings¹² have been published elsewhere.

Participants

We approached EMO study site principal investigators at 12 hospitals: the two highest- and two lowest-use hospitals within three hospital types (ie, freestanding children’s hospitals, children’s hospitals within large general hospitals, and community hospitals). We collaborated with the participating site principal investigators (n = 12), who were primarily hospitalist physicians in leadership roles, to recruit a purposive sample of additional stakeholders including bedside nurses (n = 12), hospitalist physicians (n = 15), respiratory therapists (n = 9), and hospital administrators (n = 8) to participate in semistructured interviews. Interviews were conducted until we achieved thematic saturation within each stakeholder group and within the high and low performing strata (total 56 interviews). Participants were asked to self-report basic demographic information (see Appendix, interview guide) as required by the study funder and to allow us to comment on the representativeness of the participant group.

Procedure

The interview guide was informed by the CFIR, a comprehensive framework detailing contextual factors that require consideration when planning for the implementation of a health service intervention. Table 1 details the CFIR domains with study-related examples. The interview guide (Appendix) provided limited clinical context apart from the age, diagnosis, and oxygen requirement for the population of interest to promote a broad array of responses and to avoid anchoring on specific clinical scenarios. Interviews were conducted by master’s degree or doctoral-level research coordinators with qualitative interviewing experience and supervised by a medical anthropologist and qualitative methods expert (F.K.B.). Prior to engaging in audio recorded phone interviews, the interviewer explained the risks and benefits of participating. Participants were compensated $50. Audio recordings were transcribed, deidentified, and uploaded to NVivo 12 Plus (QSR International) for data management.

The Institutional Review Boards of Children’s Hospital of Philadelphia, Pennsylvania, and the University of Pennsylvania in Philadelphia determined that the study met eligibility criteria for IRB exemption.

Data Analysis

Using an integrated approach to codebook development,²⁶ a priori codes were developed using constructs from the CFIR. Additional codes were added by the research team following a close reading of the first five transcripts.^27,28 Each code was defined, including decision rules for its application. Two research coordinators independently coded each transcript. Using the intercoder reliability function within NVivo, the coders established strong interrater reliability accordance scores (k > .8) by double coding 20% of the transcripts. Data were stratified by sites with low and high use of cSpO₂ to examine differences in barriers and facilitators to deimplementation. Each code was subcoded, summarized, and examined for patterns within and across participating disciplines, which yielded themes related to barriers and facilitators. We conducted member checking and reviewed our conclusions with a multidisciplinary group of clinical stakeholders (n = 13) to validate our analyses.

Barriers and facilitators to deimplementation were identified in multiple domains of the CFIR: outer setting, inner setting, characteristics of the individuals, and intervention characteristics (Table 1). Participants also suggested strategies to facilitate deimplementation in response to some identified barriers. See Table 2 for participant demographics and Table 3 for illustrative participant quotations.

Barriers

Outer Setting: Clinician Perceptions of Parental Discomfort With Discontinuing Monitoring

Participants mentioned parental preferences as a barrier to discontinuing cSpO₂, noting that parents seem to take comfort in watching the numbers on the monitor screen and are reluctant to have it withdrawn. Clinicians noted that parents sometimes put the monitor back on their child after a clinician removed it or have expressed concern that their unmonitored child was not receiving the same level of care as other patients who were being monitored. In these scenarios, clinicians reported they have found it helpful to educate caregivers about when cSpO₂ is and is not appropriate.

Inner Setting: Unclear or Nonexistent Guideline to Discontinue cSpO₂

Guidelines to discontinue cSpO₂ reportedly did not exist at all institutions. If a guideline did exist, lack of clarity or conflicting guidelines about when to use oxygen presented a barrier. Participants suggested that a clear guideline or additional oversight to ensure all clinicians are informed of the procedure for discontinuing cSpO₂ may help prevent miscommunication. Participants noted that their electronic health record (EHR) order sets commonly included cSpO₂ orders and that removing that option would facilitate deimplementation.

Inner Setting: Difficulty Educating All Staff

Participants noted difficulty with incorporating education about discontinuing cSpO₂ to all clinicians, particularly to those who are nightshift only or to rotating staff or trainees. This created barriers for frequent re-education because these staff are not familiar with the policies and procedures of the unit, which is crucial to developing a culture that supports the deimplementation of cSpO₂. Participants suggested that recurring education about procedures for discontinuing cSpO₂ should target trainees, new nurses, and overnight nurses. This would help to ensure that the guideline is uniformly followed.

Inner Setting: Culture of High cSpO₂ Use

Participants from high-use sites discussed a culture driven by readily available monitoring features or an expectation that monitoring indicates higher-quality care. Participants from low-use sites discussed increased cSpO₂ driven by clinicians who were accustomed to caring for higher-acuity patients, for whom continuous monitoring is likely appropriate, and were simultaneously caring for stable bronchiolitis patients.

Some suggested that visual cues would be useful to clinicians to sustain awareness about a cSpO₂ deimplementation guideline. It was also suggested that audit and feedback techniques like posting unit deimplementation statistics and creating a competition among units by posting unit performance could facilitate deimplementation. Additionally, some noted that visual aids in common spaces would be useful to remind clinicians and to engage caregivers about discontinuing cSpO₂.

Characteristics of Individuals: Clinician Discomfort Discontinuing cSpO₂

One frequently cited barrier across participants is that cSpO₂ provides “peace of mind” to alert clinicians to patients with low oxygen saturations that might otherwise be missed. Participants identified that clinician discomfort with reducing cSpO₂ may be driven by inexperienced clinicians less familiar with the bronchiolitis disease process, such as trainees, new nurses, or rotating clinicians unaccustomed to pediatric care. Trainees and new nurses were perceived as being more likely to work at night when there are fewer clinicians to provide patient care. Additionally, participants perceived that night shift clinicians favored cSpO₂ because they could measure vital signs without waking patients and families.

Clinicians discussed that discontinuing cSpO₂ would require alternative methods for assessing patient status, particularly for night shift nurses. Participants suggested strategies including changes to pulse oximetry assessment procedures to include more frequent “spot checks,” incorporation of assessments during sleep events (eg, naps) to ensure the patient does not experience desaturations during sleep, and training nurses to become more comfortable with suctioning patients. Suggestions also included education on the typical features of transient oxygen desaturations in otherwise stable patients with bronchiolitis² to bolster clinical confidence for clinicians unfamiliar with caring for bronchiolitis patients. Participants perceived that education about appropriate vs inappropriate use may help to empower clinicians to employ cSpO₂ appropriately.

Facilitators

Outer Setting: Standards and Evidence From Research, Professional Organizations, and Leaders in the Field

Many participants expressed the importance of consistent guidelines that are advocated by thought leaders in the field, supported by robust evidence, and consistent with approaches at peer hospitals. The more authoritative support a guideline has, the more comfortable people are adopting it and taking it seriously. Additionally, consistent education about guidelines was desired. Participants noted that all clinicians should be receiving education related to the American Academy of Pediatrics (AAP) Bronchiolitis and Choosing Wisely^® guidelines, ranging from a one-time update to annually. Continual updates and re-education sessions for clinicians who shared evidence about how cSpO₂ deimplementation could improve the quality of patient care by shortening hospital length of stay and lowering cost were suggested strategies.

Inner Setting: Leadership

Participants noted that successful deimplementation depends upon the presence of a champion or educator who will be able to lead the institutional charge in making practice change. This is typically an individual who is trusted at the institution, experienced in their field, or already doing implementation work. This could be either a single individual (champion) or a team. The most commonly noted clinician roles to engage in a leadership role or team were physicians and nurses.

Participants noted that a change in related clinical care pathways or EHR order sets would require cooperation from multiple clinical disciplines, administrators, and information technology leaders and explained that messaging and education about the value of the change would facilitate buy-in from those clinicians.

Inner Setting: EHR Support for Guidelines

Participants often endorsed the use of an order set within the EHR that supports guidelines and includes reminders to decrease cSpO₂. These reminders could come up when supplemental oxygen is discontinued or occur regularly throughout the patient’s stay to prompt the clinician to consider discontinuing cSpO₂.

Intervention Characteristics/Inner Setting: Clear Bronchiolitis Guidelines

The presence of a well-articulated hospital policy that delineates the appropriate and inappropriate use of cSpO₂ in bronchiolitis was mentioned as another facilitator of deimplementation.

Results of this qualitative study of stakeholders across hospitals with high and low cSpO₂ use illustrated the complexities involved with deimplementation of cSpO₂ in pediatric patients hospitalized with bronchiolitis. We identified numerous barriers spanning the CFIR constructs, including unclear or absent guidelines for stopping cSpO₂, clinician knowledge and comfort with bronchiolitis disease features, and unit culture. This suggests that multicomponent strategies that target various domains and a variety of stakeholders are needed to deimplement cSpO₂ use for stable bronchiolitis patients. Participants also identified facilitators, including clear cSpO₂ guidelines, supportive leaders and champions, and EHR modifications, that provide insight into strategies that may help sites reduce their use of cSpO₂. Additionally, participants also provided concrete, actionable suggestions for ways to reduce unnecessary monitoring that will be useful in informing promising deimplementation strategies for subsequent trials.

The importance of having specific and well-known guidelines from trusted sources, such as the AAP, about cSpO₂ and bronchiolitis treatment that are thoughtfully integrated in the EHR came through in multiple themes of our analysis. Prior studies on the effect of guidelines on clinical practice have suggested that rigorously designed guidelines can positively impact practice.²⁹ Participants also noted that cSpO₂ guidelines should be authoritative and that knowledge of guideline adoption by peer institutions was a facilitator of adoption. Usability issues negatively impact clinicians’ ability to follow guidelines.³⁰ Further, prior studies have demonstrated that EHR integration of guidelines can change practice.^31-33 Based on our findings, incorporating clear guidelines into commonly used formats, such as EHR order sets, could be an important deimplementation tool for cSpO₂ in stable bronchiolitis patients.

Education about and awareness of cSpO₂ guidelines was described as an important facilitator for appropriate cSpO₂ use and was suggested as a potential deimplementation strategy. Participants noted that educational need may vary by stakeholder group. For example, education may facilitate obtaining buy-in from hospital leaders, which is necessary to support changes to the EHR. Education incorporating information on the typical features of bronchiolitis and examples of appropriate and inappropriate cSpO₂ use was suggested for clinical team members. The limitations of education as a stand-alone deimplementation strategy were also noted, and participants highlighted challenges such as time needed for education and the need for ongoing education for rotating trainees. Inner and outer setting barriers, such as a perceived “culture of high pulse oximetryuse” and patient and family expectations, could also make education less effective as a stand-alone strategy. That—coupled with evidence that education and training alone are generally insufficient for producing reliable, sustained behavior change^34,35—suggests that a multifaceted approach will be important.

Our respondents consider parental perceptions and preference in their practice, which provides nuance to recent studies suggesting that parents prefer continuous monitors when their child is hospitalized with bronchiolitis. Chi et al described the impact of a brief educational intervention on parental preferences for monitoring children hospitalized for bronchiolitis.³⁶ This work suggests that educational interventions aimed at families should be considered in future (de)implementation studies because they may indirectly impact clinician behavior. Future studies should directly assess parental discomfort with discontinuing monitoring. Participants highlighted the link between knowledge and confidence in caring for typical bronchiolitis patients and monitoring practice, perceiving that less experienced clinicians are more likely to rely on cSpO₂. Participants at high-use sites emphasized the expectation that monitoring should occur during hospitalizations. This reflection is particularly pertinent for bronchiolitis, a disease characterized by frequent, self-resolving desaturations even after hospital discharge.³ This may reinforce a perceived need to capture and react to these desaturation events even though they are expected in bronchiolitis and can occur in healthy infants.³⁷ Some participants suggested that continuous monitoring be replaced with “nap tests” (ie, assessment for desaturations during a nap prior to discharge); however, like cSpO₂ in stable infants with bronchiolitis, this is another low-value practice. Otherwise healthy infants with mild to moderate disease are unlikely to subsequently worsen after showing signs of clinical improvement.³⁸ Nap tests are likely to lead to infants who are clinically improving being placed unnecessarily back on oxygen in reaction to the transient desaturations. Participants’ perception about the importance of cSpO₂ in bronchiolitis management, despite evidence suggesting it is a low-value practice, underscores the importance of not simply telling clinicians to stop cSpO₂. Employing strategies that replace continuous monitoring with another acceptable and feasible alternative (eg, regular clinician assessments including intermittent pulse oximetry checks) should be considered when planning for deimplementation.³⁹

Previous studies indicate that continuous monitoring can affect clinician decision-making, independent of other factors,^6,40 despite limited evidence that continuous monitors improve patient outcomes.^1-7 Studies have demonstrated noticeable increase in admissions based purely on pulse oximetry values,⁴⁰ with no evidence that this type of admission changes outcomes for bronchiolitis patients.⁶ One previous, single-center study identified inexperience as a potential driver for monitor use,⁴¹ and studies in adult populations have suggested that clinicians overestimate the value that continuous monitoring contributes to patient care,^42,43 which promotes guideline-discordant use. Our study provides novel insight into the issue of monitoring in bronchiolitis. Our results suggest that there is a need to shift organizational cultures around monitoring (which likely vary based on a range of factors) and that educational strategies addressing typical disease course, especially desaturations, in bronchiolitis will be an essential component in any deimplementation effort.

This study is strengthened by its sample of diverse stakeholder groups from multiple US health systems. Additionally, we interviewed individuals at sites with high cSpO₂ rates and at sites with low rates, as well as from community hospitals, children’s hospitals within general hospitals, and freestanding children’s hospitals, which allows us to understand barriers high-use sites encounter and facilitators of lower cSpO₂ rates at low-use sites. We also employed an interview approach informed by an established implementation science framework. Nonetheless, several limitations exist. First, participants at low-use sites did not necessarily have direct experience with a previous deimplementation effort to reduce cSpO₂. Additionally, participants were predominantly White and female; more diverse perspectives would strengthen confidence in the generalizability of our findings. While thematic saturation was achieved within each stakeholder group and within the high- and low-use strata, we interviewed fewer administrators and respiratory therapists relative to other stakeholder groups. Nevertheless, our conclusions were validated by our interdisciplinary stakeholder panel. As noted by participants, family preferences may influence clinician practice, and parents were not interviewed for this study. The information gleaned from the present study will inform the development of strategies to deimplement unnecessary cSpO₂ in pediatric hospitals, which we aim to rigorously evaluate in a future trial.

We identified barriers and facilitators to deimplementation of cSpO₂ for stable patients with bronchiolitis across children’s hospitals with high and low utilization of cSpO₂. These themes map to multiple CFIR domains and, along with participant-suggested strategies, can directly inform an approach to cSpO₂ deimplementation in a range of inpatient settings. Based on these data, future deimplementation efforts should focus on clear protocols for use and discontinuation of cSpO₂, EHR changes, and regular bronchiolitis education for hospital staff that emphasizes reducing unnecessary cSpO₂ utilization.

We acknowledge the NHLBI scientists who contributed their expertise to this project as part of the U01 Cooperative Agreement funding mechanism as federal employees conducting their official job duties: Lora Reineck, MD, MS, Karen Bienstock, MS, and Cheryl Boyce, PhD. We thank the Executive Council of the Pediatric Research in Inpatient Settings (PRIS) Network for their contributions to the early scientific development of this project. The Network assessed a Collaborative Support Fee for access to the hospitals and support of this project. We thank the PRIS Network collaborators for their major contributions to data collection measuring utilization to identify the hospitals we subsequently chose for this project. We thank Claire Bocage and the Mixed Methods Research Lab for major help in data management and data analysis.

As part of improvement collaboratives that aimed to reduce overuse of continuous pulse oximetry in children hospitalized with bronchiolitis, researchers used the presence of an active order for it as a proxy for the actual use of such monitoring.^1,2 With use of this proxy, investigators on a national study documented a high burden of continuous oximetry overuse (86.5% before quality improvement interventions and 45.5% after),¹ but the validity of orders in representing actual monitoring practice is unknown. If the presence of an active pulse oximetry order accurately identifies infants on monitors, electronic health record data could inform epidemiologic estimates of monitoring overuse and measure the success of quality improvement and deimplementation interventions. Alternatively, if nurses commonly begin and/or discontinue pulse oximetry without updated orders, a pulse oximetry order would not be an accurate proxy, and additional data capture methods (eg, bedside observation or data capture from bedside monitors) would be needed.

Understanding the validity of orders for detection of actual use is critical because continuous pulse oximetry monitoring is considered an overused practice in pediatric acute viral bronchiolitis,³ and national guidelines recommend against its use in low-risk hospitalized children.^4,5 Continuous monitoring may identify trivial, self-resolving oxygen desaturation and its use is not associated with improved outcomes.^6-9 When self-resolving desaturations are treated with additional supplemental oxygen, hospital stays may be unnecessarily prolonged.¹⁰ In order to reduce unnecessary continuous pulse oximetry use, measurement of the extent of the overused practice is necessary. In this 56-hospital study,¹¹ we aimed to determine the validity of using active continuous pulse oximetry orders instead of bedside observation of actual monitor use.

Design

In this multicenter, repeated cross-sectional study, investigators used direct bedside observation to determine continuous pulse oximetry monitor use and then assessed whether an active continuous monitoring order was present in the electronic health record. The study took place during one bronchiolitis season, December 1, 2018, through March 31, 2019.

Setting and Patients

Investigators at 56 freestanding children’s hospitals, children’s hospitals within general hospitals, and community hospitals in the Pediatric Research in Inpatient Settings (PRIS) Network collected data on infants aged 8 weeks to 23 months who were hospitalized with bronchiolitis. As this work was a substudy of the larger Eliminating Monitor Overuse study, only infants not currently receiving supplemental oxygen were included.¹¹ Investigators observed eligible infants outside of the intensive care unit on general hospital medicine units. We excluded infants born premature (documented prematurity of <28 weeks’ gestation or documented “premature” without a gestational age listed), as well as those with a home oxygen requirement, cyanotic congenital heart disease, pulmonary hypertension, tracheostomy, primary neuromuscular disease, immunodeficiency, or cancer.

Data Collection

Investigators used the electronic health record to identify eligible infants. Investigators entered patient rooms to confirm the infant was not on supplemental oxygen (hence confirming eligibility for the study) and determine if continuous pulse oximetry was actively in use by examining the monitor display for a pulse oximetry waveform. Investigators then confirmed if active orders for pulse oximetry were present in the patient’s chart. Per study design, site investigators aimed to observe approximately half of eligible infants during the day (10 am to 5 pm) and the other half during the night (11 pm to 7 am).

Analysis

We excluded patients with conditional orders (eg, monitored only when certain conditions exist, such as when asleep) because of the time-varying and wide range of conditions that could be specified. Furthermore, conditional orders would not be useful as proxies to measure oximetry use because investigators would still need additional data (eg, bedside observation) to determine current monitoring status.

We calculated the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of active orders using the reference standard of direct bedside observation, as well as corresponding 95% CIs that accounted for within-hospital clustering. We calculated these test characteristics overall and as stratified across four age groups: 8 weeks to 5 months, 6 months to 11 months, 12 months to 17 months, and 18 months to 23 months. We also calculated the test characteristics for each hospital. We decided a priori that a PPV and NPV of 80% would represent a reasonable threshold to use active orders as a proxy in multicenter research. For hospital-level analyses we included only hospitals with 60 or more total observations and more than 15 observations with active orders for PPV and more than 15 observations without active orders for NPV. We used Stata (StataCorp LLC, College Station, Texas) version 15.1 for analysis.

For US sites, the Institutional Review Board (IRB) at Children’s Hospital of Philadelphia approved the study as the single reviewing IRB, and the remaining US sites established reliance agreements with the reviewing IRB. Research Ethics Boards at the Canadian sites (University of Calgary and The Hospital for Sick Children) also reviewed and approved the study. All sites granted waivers of consent, assent, parental permission, and HIPAA authorization.

Investigators completed 3,612 observations in 56 hospitals. This included 33 freestanding children’s hospitals, 14 hospitals within large general hospitals, and 9 community hospitals. Of 3,612 completed observations, on 631 occasions (17%) patients had conditional orders (eg, continuous monitoring only when sleeping) and were excluded from further analysis.

Most pulse oximetry–monitored infants did not have an active monitoring order (670 out of 1,309; sensitivity of 49%). Test characteristics, stratified by age group, are presented in the Table. Across all observations, the overall PPV was 77% (95% CI, 72-82), and the overall NPV was 69% (95% CI, 61-77). Variation of all test characteristics across age group was small (eg, the sensitivity ranged from 43% to 51%).

With inclusion of only those hospitals with sufficient observations, hospital-level variation in the PPV and NPV of using active orders was substantial (PPV range of 48% to 96% and NPV range of 30% to 98%). Only two hospitals had both a PPV and NPV for using monitor orders that exceeded the 80% threshold.

Active continuous pulse oximetry orders did not accurately represent actual monitoring status in this study. Monitoring orders alone frequently misrepresent true monitoring status and, as such, should be interpreted with caution in research or quality improvement activities. If more valid estimates of monitoring use and overuse are needed, potential measurement options include direct observation, as used in our study, as well as the use of more complex data streams such as the output of monitoring devices or pulse oximetry data in the electronic health record. In only two of the hospitals, using active continuous monitoring orders was a reasonable proxy for detecting actual monitor use. Monitoring orders could potentially be validly used for deimplementation efforts at those centers; other hospitals could consider targeted improvement efforts (eg, morning huddles examining the discordance between monitoring orders and monitoring status) to improve the accuracy of using continuous pulse oximetry orders.

We acknowledge several limitations of this study. Site investigators employed a convenience sampling approach, so it is possible that some investigators observed sicker or less sick infants. Although the PRIS network includes a geographically diverse group of North American hospitals, community hospitals were underrepresented in this study. Our results hence generalize more precisely to freestanding children’s hospitals than to community hospitals. We did not observe infants currently on supplemental oxygen, so we do not know to what degree using orders is valid in that context. We did not collect data on why actual monitoring status differed from monitoring orders and hence cannot quantify to what extent different factors (eg, nurse belief that monitors are a safety net or infants inadvertently left on monitors after a spot check pulse oximetry reading) contributed to this discordance. Finally, our study only examined one electronic health record variable—the presence of an active order. It may be that other variables in the health record (eg, minute-by-minute pulse oximetry values in a vital sign flowsheet) are much better proxies of actual continuous monitor use.

Using an active order for continuous pulse oximetry has poor sensitivity, PPV, and NPV for detecting true monitoring status at the bedside. Teams intending to measure the actual use of pulse oximetry should be aware of the limitations of using active orders alone as an accurate measure of pulse oximetry monitoring.

We thank the NHLBI scientists who contributed to this project as part of the U01 Cooperative Agreement funding mechanism: Lora Reineck, MD, MS, Karen Bienstock, MS, and Cheryl Boyce, PhD.

We thank the Executive Council of the PRIS Network for their contributions to the early scientific development of this project. We thank the PRIS site investigators for their major contributions to the Eliminating Monitor Overuse (EMO) Study data collection. Each listed collaborator is a group author for the PRIS Network in this manuscript. Their names can be found in the online supplemental information.

As part of improvement collaboratives that aimed to reduce overuse of continuous pulse oximetry in children hospitalized with bronchiolitis, researchers used the presence of an active order for it as a proxy for the actual use of such monitoring.^1,2 With use of this proxy, investigators on a national study documented a high burden of continuous oximetry overuse (86.5% before quality improvement interventions and 45.5% after),¹ but the validity of orders in representing actual monitoring practice is unknown. If the presence of an active pulse oximetry order accurately identifies infants on monitors, electronic health record data could inform epidemiologic estimates of monitoring overuse and measure the success of quality improvement and deimplementation interventions. Alternatively, if nurses commonly begin and/or discontinue pulse oximetry without updated orders, a pulse oximetry order would not be an accurate proxy, and additional data capture methods (eg, bedside observation or data capture from bedside monitors) would be needed.

Understanding the validity of orders for detection of actual use is critical because continuous pulse oximetry monitoring is considered an overused practice in pediatric acute viral bronchiolitis,³ and national guidelines recommend against its use in low-risk hospitalized children.^4,5 Continuous monitoring may identify trivial, self-resolving oxygen desaturation and its use is not associated with improved outcomes.^6-9 When self-resolving desaturations are treated with additional supplemental oxygen, hospital stays may be unnecessarily prolonged.¹⁰ In order to reduce unnecessary continuous pulse oximetry use, measurement of the extent of the overused practice is necessary. In this 56-hospital study,¹¹ we aimed to determine the validity of using active continuous pulse oximetry orders instead of bedside observation of actual monitor use.

Design

In this multicenter, repeated cross-sectional study, investigators used direct bedside observation to determine continuous pulse oximetry monitor use and then assessed whether an active continuous monitoring order was present in the electronic health record. The study took place during one bronchiolitis season, December 1, 2018, through March 31, 2019.

Setting and Patients

Investigators at 56 freestanding children’s hospitals, children’s hospitals within general hospitals, and community hospitals in the Pediatric Research in Inpatient Settings (PRIS) Network collected data on infants aged 8 weeks to 23 months who were hospitalized with bronchiolitis. As this work was a substudy of the larger Eliminating Monitor Overuse study, only infants not currently receiving supplemental oxygen were included.¹¹ Investigators observed eligible infants outside of the intensive care unit on general hospital medicine units. We excluded infants born premature (documented prematurity of <28 weeks’ gestation or documented “premature” without a gestational age listed), as well as those with a home oxygen requirement, cyanotic congenital heart disease, pulmonary hypertension, tracheostomy, primary neuromuscular disease, immunodeficiency, or cancer.

Data Collection

Investigators used the electronic health record to identify eligible infants. Investigators entered patient rooms to confirm the infant was not on supplemental oxygen (hence confirming eligibility for the study) and determine if continuous pulse oximetry was actively in use by examining the monitor display for a pulse oximetry waveform. Investigators then confirmed if active orders for pulse oximetry were present in the patient’s chart. Per study design, site investigators aimed to observe approximately half of eligible infants during the day (10 am to 5 pm) and the other half during the night (11 pm to 7 am).

Analysis

We excluded patients with conditional orders (eg, monitored only when certain conditions exist, such as when asleep) because of the time-varying and wide range of conditions that could be specified. Furthermore, conditional orders would not be useful as proxies to measure oximetry use because investigators would still need additional data (eg, bedside observation) to determine current monitoring status.

We calculated the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of active orders using the reference standard of direct bedside observation, as well as corresponding 95% CIs that accounted for within-hospital clustering. We calculated these test characteristics overall and as stratified across four age groups: 8 weeks to 5 months, 6 months to 11 months, 12 months to 17 months, and 18 months to 23 months. We also calculated the test characteristics for each hospital. We decided a priori that a PPV and NPV of 80% would represent a reasonable threshold to use active orders as a proxy in multicenter research. For hospital-level analyses we included only hospitals with 60 or more total observations and more than 15 observations with active orders for PPV and more than 15 observations without active orders for NPV. We used Stata (StataCorp LLC, College Station, Texas) version 15.1 for analysis.

For US sites, the Institutional Review Board (IRB) at Children’s Hospital of Philadelphia approved the study as the single reviewing IRB, and the remaining US sites established reliance agreements with the reviewing IRB. Research Ethics Boards at the Canadian sites (University of Calgary and The Hospital for Sick Children) also reviewed and approved the study. All sites granted waivers of consent, assent, parental permission, and HIPAA authorization.

Investigators completed 3,612 observations in 56 hospitals. This included 33 freestanding children’s hospitals, 14 hospitals within large general hospitals, and 9 community hospitals. Of 3,612 completed observations, on 631 occasions (17%) patients had conditional orders (eg, continuous monitoring only when sleeping) and were excluded from further analysis.

Most pulse oximetry–monitored infants did not have an active monitoring order (670 out of 1,309; sensitivity of 49%). Test characteristics, stratified by age group, are presented in the Table. Across all observations, the overall PPV was 77% (95% CI, 72-82), and the overall NPV was 69% (95% CI, 61-77). Variation of all test characteristics across age group was small (eg, the sensitivity ranged from 43% to 51%).

With inclusion of only those hospitals with sufficient observations, hospital-level variation in the PPV and NPV of using active orders was substantial (PPV range of 48% to 96% and NPV range of 30% to 98%). Only two hospitals had both a PPV and NPV for using monitor orders that exceeded the 80% threshold.

Active continuous pulse oximetry orders did not accurately represent actual monitoring status in this study. Monitoring orders alone frequently misrepresent true monitoring status and, as such, should be interpreted with caution in research or quality improvement activities. If more valid estimates of monitoring use and overuse are needed, potential measurement options include direct observation, as used in our study, as well as the use of more complex data streams such as the output of monitoring devices or pulse oximetry data in the electronic health record. In only two of the hospitals, using active continuous monitoring orders was a reasonable proxy for detecting actual monitor use. Monitoring orders could potentially be validly used for deimplementation efforts at those centers; other hospitals could consider targeted improvement efforts (eg, morning huddles examining the discordance between monitoring orders and monitoring status) to improve the accuracy of using continuous pulse oximetry orders.

We acknowledge several limitations of this study. Site investigators employed a convenience sampling approach, so it is possible that some investigators observed sicker or less sick infants. Although the PRIS network includes a geographically diverse group of North American hospitals, community hospitals were underrepresented in this study. Our results hence generalize more precisely to freestanding children’s hospitals than to community hospitals. We did not observe infants currently on supplemental oxygen, so we do not know to what degree using orders is valid in that context. We did not collect data on why actual monitoring status differed from monitoring orders and hence cannot quantify to what extent different factors (eg, nurse belief that monitors are a safety net or infants inadvertently left on monitors after a spot check pulse oximetry reading) contributed to this discordance. Finally, our study only examined one electronic health record variable—the presence of an active order. It may be that other variables in the health record (eg, minute-by-minute pulse oximetry values in a vital sign flowsheet) are much better proxies of actual continuous monitor use.

Using an active order for continuous pulse oximetry has poor sensitivity, PPV, and NPV for detecting true monitoring status at the bedside. Teams intending to measure the actual use of pulse oximetry should be aware of the limitations of using active orders alone as an accurate measure of pulse oximetry monitoring.

We thank the NHLBI scientists who contributed to this project as part of the U01 Cooperative Agreement funding mechanism: Lora Reineck, MD, MS, Karen Bienstock, MS, and Cheryl Boyce, PhD.

We thank the Executive Council of the PRIS Network for their contributions to the early scientific development of this project. We thank the PRIS site investigators for their major contributions to the Eliminating Monitor Overuse (EMO) Study data collection. Each listed collaborator is a group author for the PRIS Network in this manuscript. Their names can be found in the online supplemental information.

As part of improvement collaboratives that aimed to reduce overuse of continuous pulse oximetry in children hospitalized with bronchiolitis, researchers used the presence of an active order for it as a proxy for the actual use of such monitoring.^1,2 With use of this proxy, investigators on a national study documented a high burden of continuous oximetry overuse (86.5% before quality improvement interventions and 45.5% after),¹ but the validity of orders in representing actual monitoring practice is unknown. If the presence of an active pulse oximetry order accurately identifies infants on monitors, electronic health record data could inform epidemiologic estimates of monitoring overuse and measure the success of quality improvement and deimplementation interventions. Alternatively, if nurses commonly begin and/or discontinue pulse oximetry without updated orders, a pulse oximetry order would not be an accurate proxy, and additional data capture methods (eg, bedside observation or data capture from bedside monitors) would be needed.

Understanding the validity of orders for detection of actual use is critical because continuous pulse oximetry monitoring is considered an overused practice in pediatric acute viral bronchiolitis,³ and national guidelines recommend against its use in low-risk hospitalized children.^4,5 Continuous monitoring may identify trivial, self-resolving oxygen desaturation and its use is not associated with improved outcomes.^6-9 When self-resolving desaturations are treated with additional supplemental oxygen, hospital stays may be unnecessarily prolonged.¹⁰ In order to reduce unnecessary continuous pulse oximetry use, measurement of the extent of the overused practice is necessary. In this 56-hospital study,¹¹ we aimed to determine the validity of using active continuous pulse oximetry orders instead of bedside observation of actual monitor use.

Design

In this multicenter, repeated cross-sectional study, investigators used direct bedside observation to determine continuous pulse oximetry monitor use and then assessed whether an active continuous monitoring order was present in the electronic health record. The study took place during one bronchiolitis season, December 1, 2018, through March 31, 2019.

Setting and Patients

Investigators at 56 freestanding children’s hospitals, children’s hospitals within general hospitals, and community hospitals in the Pediatric Research in Inpatient Settings (PRIS) Network collected data on infants aged 8 weeks to 23 months who were hospitalized with bronchiolitis. As this work was a substudy of the larger Eliminating Monitor Overuse study, only infants not currently receiving supplemental oxygen were included.¹¹ Investigators observed eligible infants outside of the intensive care unit on general hospital medicine units. We excluded infants born premature (documented prematurity of <28 weeks’ gestation or documented “premature” without a gestational age listed), as well as those with a home oxygen requirement, cyanotic congenital heart disease, pulmonary hypertension, tracheostomy, primary neuromuscular disease, immunodeficiency, or cancer.

Data Collection

Investigators used the electronic health record to identify eligible infants. Investigators entered patient rooms to confirm the infant was not on supplemental oxygen (hence confirming eligibility for the study) and determine if continuous pulse oximetry was actively in use by examining the monitor display for a pulse oximetry waveform. Investigators then confirmed if active orders for pulse oximetry were present in the patient’s chart. Per study design, site investigators aimed to observe approximately half of eligible infants during the day (10 am to 5 pm) and the other half during the night (11 pm to 7 am).

Analysis

We excluded patients with conditional orders (eg, monitored only when certain conditions exist, such as when asleep) because of the time-varying and wide range of conditions that could be specified. Furthermore, conditional orders would not be useful as proxies to measure oximetry use because investigators would still need additional data (eg, bedside observation) to determine current monitoring status.

We calculated the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of active orders using the reference standard of direct bedside observation, as well as corresponding 95% CIs that accounted for within-hospital clustering. We calculated these test characteristics overall and as stratified across four age groups: 8 weeks to 5 months, 6 months to 11 months, 12 months to 17 months, and 18 months to 23 months. We also calculated the test characteristics for each hospital. We decided a priori that a PPV and NPV of 80% would represent a reasonable threshold to use active orders as a proxy in multicenter research. For hospital-level analyses we included only hospitals with 60 or more total observations and more than 15 observations with active orders for PPV and more than 15 observations without active orders for NPV. We used Stata (StataCorp LLC, College Station, Texas) version 15.1 for analysis.

For US sites, the Institutional Review Board (IRB) at Children’s Hospital of Philadelphia approved the study as the single reviewing IRB, and the remaining US sites established reliance agreements with the reviewing IRB. Research Ethics Boards at the Canadian sites (University of Calgary and The Hospital for Sick Children) also reviewed and approved the study. All sites granted waivers of consent, assent, parental permission, and HIPAA authorization.

Investigators completed 3,612 observations in 56 hospitals. This included 33 freestanding children’s hospitals, 14 hospitals within large general hospitals, and 9 community hospitals. Of 3,612 completed observations, on 631 occasions (17%) patients had conditional orders (eg, continuous monitoring only when sleeping) and were excluded from further analysis.

Most pulse oximetry–monitored infants did not have an active monitoring order (670 out of 1,309; sensitivity of 49%). Test characteristics, stratified by age group, are presented in the Table. Across all observations, the overall PPV was 77% (95% CI, 72-82), and the overall NPV was 69% (95% CI, 61-77). Variation of all test characteristics across age group was small (eg, the sensitivity ranged from 43% to 51%).

With inclusion of only those hospitals with sufficient observations, hospital-level variation in the PPV and NPV of using active orders was substantial (PPV range of 48% to 96% and NPV range of 30% to 98%). Only two hospitals had both a PPV and NPV for using monitor orders that exceeded the 80% threshold.

Active continuous pulse oximetry orders did not accurately represent actual monitoring status in this study. Monitoring orders alone frequently misrepresent true monitoring status and, as such, should be interpreted with caution in research or quality improvement activities. If more valid estimates of monitoring use and overuse are needed, potential measurement options include direct observation, as used in our study, as well as the use of more complex data streams such as the output of monitoring devices or pulse oximetry data in the electronic health record. In only two of the hospitals, using active continuous monitoring orders was a reasonable proxy for detecting actual monitor use. Monitoring orders could potentially be validly used for deimplementation efforts at those centers; other hospitals could consider targeted improvement efforts (eg, morning huddles examining the discordance between monitoring orders and monitoring status) to improve the accuracy of using continuous pulse oximetry orders.

We acknowledge several limitations of this study. Site investigators employed a convenience sampling approach, so it is possible that some investigators observed sicker or less sick infants. Although the PRIS network includes a geographically diverse group of North American hospitals, community hospitals were underrepresented in this study. Our results hence generalize more precisely to freestanding children’s hospitals than to community hospitals. We did not observe infants currently on supplemental oxygen, so we do not know to what degree using orders is valid in that context. We did not collect data on why actual monitoring status differed from monitoring orders and hence cannot quantify to what extent different factors (eg, nurse belief that monitors are a safety net or infants inadvertently left on monitors after a spot check pulse oximetry reading) contributed to this discordance. Finally, our study only examined one electronic health record variable—the presence of an active order. It may be that other variables in the health record (eg, minute-by-minute pulse oximetry values in a vital sign flowsheet) are much better proxies of actual continuous monitor use.

Using an active order for continuous pulse oximetry has poor sensitivity, PPV, and NPV for detecting true monitoring status at the bedside. Teams intending to measure the actual use of pulse oximetry should be aware of the limitations of using active orders alone as an accurate measure of pulse oximetry monitoring.

We thank the NHLBI scientists who contributed to this project as part of the U01 Cooperative Agreement funding mechanism: Lora Reineck, MD, MS, Karen Bienstock, MS, and Cheryl Boyce, PhD.

We thank the Executive Council of the PRIS Network for their contributions to the early scientific development of this project. We thank the PRIS site investigators for their major contributions to the Eliminating Monitor Overuse (EMO) Study data collection. Each listed collaborator is a group author for the PRIS Network in this manuscript. Their names can be found in the online supplemental information.

Alarms from bedside continuous physiologic monitors (CPMs) occur frequently in children’s hospitals and can lead to harm. Recent studies conducted in children’s hospitals have identified alarm rates of up to 152 alarms per patient per day outside of the intensive care unit,^1-3 with as few as 1% of alarms being considered clinically important.⁴ Excessive alarms have been linked to alarm fatigue, when providers become desensitized to and may miss alarms indicating impending patient deterioration. Alarm fatigue has been identified by national patient safety organizations as a patient safety concern given the risk of patient harm.^5-7 Despite these concerns, CPMs are routinely used: up to 48% of pediatric patients in nonintensive care units at children’s hospitals are monitored.²

Although the low number of alarms that receive responses has been well-described,^8,9 the reasons why clinicians do or do not respond to alarms are unclear. A study conducted in an adult perioperative unit noted prolonged nurse response times for patients with high alarm rates.¹⁰ A second study conducted in the pediatric inpatient setting demonstrated a dose-response effect and noted progressively prolonged nurse response times with increased rates of nonactionable alarms.^4,11 Findings from another study suggested that underlying factors are highly complex and may be a result of excessive alarms, clinician characteristics, and working conditions (eg, workload and unit noise level).¹² Evidence also suggests that humans have difficulty distinguishing the importance of alarms in situations where multiple alarm tones are used, a common scenario in hospitals.^13,14 Understanding the factors that contribute to clinicians responding or not responding to CPM alarms will be crucial for addressing this serious patient safety issue.

An enhanced understanding of why nurses respond to alarms in daily practice will inform intervention development and improvement work. In the long term, this information could help improve systems for monitoring pediatric inpatients that are less prone to issues with alarm fatigue. The objective of this qualitative study, which employed structured observation, was to describe how bedside nurses think about and act upon bedside monitor alarms in a general pediatric inpatient unit.

Study Design and Setting

This prospective observational study took place on a 48-bed hospital medicine unit at a large, freestanding children’s hospital with >650 beds and >19,000 annual admissions. General Electric (Little Chalfont, United Kingdom) physiologic monitors (models Dash 3000, 4000, and 5000) were used at the time of the study, and nurses could be notified of monitor alarms in four ways: First, an in-room auditory alarm sounds. Second, a light positioned above the door outside of each patient room blinks for alarms that are at a “warning” or “critical level” (eg ventricular tachycardia or low oxygen saturation). Third, audible alarms occur at the unit’s central monitoring station. Lastly, another staff member can notify the patient’s nurse via in-person conversion or secure smart phone communication. On the study unit, CPMs are initiated and discontinued through a physician order.

This study was reviewed and approved by the hospital’s institutional review board.

Study Population

We used a purposive recruitment strategy to enroll bedside nurses working on general hospital medicine units, stratified to ensure varying levels of experience and primary shifts (eg, day vs night). We planned to conduct approximately two observations with each participating nurse and to continue collecting data until we could no longer identify new insights in terms of responses to alarms (ie, thematic saturation¹⁵). Observations were targeted to cover times of day that coincided with increased rates of distraction. These times included just prior to and after the morning and evening change of shifts (7:00 am and 7:00 pm), during morning rounds (8:00 am-12:00 pm), and heavy admission times (12:00 pm-10:00 pm). After written informed consent, a nurse was eligible for observation during his/her shift if he/she was caring for at least one monitored patient. Enrolled nurses were made aware of the general study topic but were blinded to the study team’s hypotheses.

Data Sources

Prior to data collection, the research team, which consisted of physicians, bedside nurses, research coordinators, and a human factors expert, created a system for categorizing alarm responses. Categories for observed responses were based on the location and corresponding action taken. Initial categories were developed a priori from existing literature and expanded through input from the multidisciplinary study team, then vetted with bedside staff, and finally pilot tested through >4 hours of observations, thus producing the final categories. These categories were entered into a work-sampling program (WorkStudy by Quetech Ltd., Waterloo, Ontario, Canada) to facilitate quick data recording during observations.

The hospital uses a central alarm collection software (BedMasterEx by Anandic Medical Systems, Feuerthalen, Switzerland), which permitted the collection of date, time, trigger (eg, high heart rate), and level (eg, crisis, warning) of the generated CPM alarms. Alarms collected are based on thresholds preset at the bedside monitor. The central collection software does not differentiate between accurate (eg, correctly representing the physiologic state of the patient) and inaccurate alarms.

Observation Procedure

At the time of observation, nurse demographic information (eg, primary shift worked and years working as a nurse) was obtained. A brief preobservation questionnaire was administered to collect patient information (eg, age and diagnosis) and the nurses’ perspectives on the necessity of monitors for each monitored patient in his/her care.

The observer shadowed the nurse for a two-hour block of his/her shift. During this time, nurses were instructed to “think aloud” as they responded to alarms (eg, “I notice the oxygen saturation monitor alarming off, but the probe has fallen off”). A trained observer (AML or KMT) recorded responses verbalized by the nurse and his/her reaction by selecting the appropriate category using the work-sampling software. Data were also collected on the vital sign associated with the alarm (eg, heart rate). Moreover, the observer kept written notes to provide context for electronically recorded data. Alarms that were not verbalized by the nurse were not counted. Similarly, alarms that were noted outside of the room by the nurse were not classified by vital sign unless the nurse confirmed with the bedside monitor. Observers did not adjudicate the accuracy of the alarms. The session was stopped if monitors were discontinued during the observation period. Alarm data generated by the bedside monitor were pulled for each patient room after observations were completed.

Analysis

Descriptive statistics were used to assess the percentage of each nurse response category and each alarm type (eg, heart rate and respiratory rate). The observed alarm rate was calculated by taking the total number of observed alarms (ie, alarms noted by the nurse) divided by the total number of patient-hours observed. The monitor-generated alarm rate was calculated by taking the total number of alarms from the bedside-alarm generated data divided by the number of patient-hours observed.

Electronically recorded observations using the work-sampling program were cross-referenced with hand-written field notes to assess for any discrepancies or identify relevant events not captured by the program. Three study team members (AML, KMT, and ACS) reviewed each observation independently and compared field notes to ensure accurate categorization. Discrepancies were referred to the larger study group in cases of uncertainty.

Nine nurses had monitored patients during the available observations and participated in 19 observation sessions, which included 35 monitored patients for a total of 61.3 patient-hours of observation. Nurses were observed for a median of two times each (range 1-4). The median number of monitored patients during a single observation session was two (range 1-3). Observed nurses were female with a median of eight years of experience (range 0.5-26 years). Patients represented a broad range of age categories and were hospitalized with a variety of diagnoses (Table). Nurses, when queried at the start of the observation, felt that monitors were necessary for 29 (82.9%) of the observed patients given either patient condition or unit policy.

A total of 207 observed nurse responses to alarms occurred during the study period for a rate of 3.4 responses per patient per hour. Of the total number of responses, 45 (21.7%) were noted outside of a patient room, and in 15 (33.3%) the nurse chose to go to the room. The other 162 were recorded when the nurse was present in the room when the alarm activated. Of the 177 in-person nurse responses, 50 were related to a pulse oximetry alarm, 66 were related to a heart rate alarm, and 61 were related to a respiratory rate alarm. The most common observed in-person response to an alarm involved the nurse judging that no intervention was necessary (n = 152, 73.1%). Only 14 (7% of total responses) observed in-person responses involved a clinical intervention, such as suctioning or titrating supplemental oxygen. Findings are summarized in the Figure and describe nurse-verbalized reasons to further assess (or not) and then whether the nurse chose to take action (or not) after an alarm.

We characterized responses to physiologic monitor alarms by a group of nurses with a range of experience levels. We found that most nurse responses to alarms in continuously monitored general pediatric patients involved no intervention, and further assessment was often not conducted for alarms that occurred outside of the room if the nurse noted otherwise reassuring clinical context. Observed responses occurred for 36% of alarms during the study period when compared with bedside monitor-alarm generated data. Overall, only 14 clinical interventions were noted among the observed responses. Nurses noted that they felt the monitors were necessary for 82.9% of monitored patients because of the clinical context or because of unit policy.

Our study findings highlight some potential contradictions in the current widespread use of CPMs in general pediatric units and how clinicians respond to them in practice.² First, while nurses reported that monitors were necessary for most of their patients, participating nurses deemed few alarms clinically actionable and often chose not to further assess when they noted alarms outside of the room. This is in line with findings from prior studies suggesting that clinicians overvalue the contribution of monitoring systems to patient safety.^16,17 Second, while this finding occurred in a minority of the observations, the presence of family members at the patient’s bedside was cited by nurses as a rationale for whether they responded to alarms. While family members are capable of identifying safety issues,¹⁸ formal systems to engage them in patient safety and physiologic monitoring are lacking. Finally, clinical interventions or responses to the alerts of deteriorating patients, which best represented the original intent of CPMs, were rare and accounted for just 7% of the responses. Further work elucidating why physicians and nurses choose to use CPMs may be helpful to identify interventions to reduce inappropriate monitor use and highlight gaps in frontline staff knowledge about the benefits and risks of CPM use.

Our findings provide a novel understanding of previously observed phenomena, such as long response times or nonresponses in settings with high alarm rates.^4,10 Similar to that in a prior study conducted in the pediatric setting,¹¹ alarms with an observed response constituted a minority of the total alarms that occurred in our study. This finding has previously been attributed to mental fatigue, caregiver apathy, and desensitization.⁸ However, even though a minority of observed responses in our study included an intervention, the nurse had a rationale for why the alarm did or did not need a response. This behavior and the verbalized rationale indicate that in his/her opinion, not responding to the alarm was clinically appropriate. Study participants also reflected on the difficulties of responding to alarms given the monitor system setup, in which they may not always be capable of hearing alarms for their patients. Without data from nurses regarding the alarms that had no observed response, we can only speculate; however, based on our findings, each of these factors could contribute to nonresponse. Finally, while high numbers of false alarms have been posited as an underlying cause of alarm fatigue, we noted that a majority of nonresponse was reported to be related to other clinical factors. This relationship suggests that from the nurse’s perspective, a more applicable framework for understanding alarms would be based on clinical actionability⁴ over physiologic accuracy.

In total, our findings suggest that a multifaceted approach will be necessary to improve alarm response rates. These interventions should include adjusting parameters such that alarms are highly likely to indicate a need for intervention coupled with educational interventions addressing clinician knowledge of the alarm system and bias about the actionability of alarms may improve response rates. Changes in the monitoring system setup such that nurses can easily be notified when alarms occur may also be indicated, in addition to formally engaging patients and families around response to alarms. Although secondary notification systems (eg, alarms transmitted to individual clinician’s devices) are one solution, the utilization of these systems needs to be balanced with the risks of contributing to existing alarm fatigue and the need to appropriately tailor monitoring thresholds and strategies to patients.

Our study has several limitations. First, nurses may have responded in a way they perceive to be socially desirable, and studies using in-person observers are also prone to a Hawthorne-like effect,^19-21 where the nurse may have tried to respond more frequently to alarms than usual during observations. However, given that the majority of bedside alarms did not receive a response and a substantial number of responses involved no action, these effects were likely weak. Second, we were unable to assess which alarms were accurately reflecting the patient’s physiologic status and which were not; we were also unable to link observed alarm response to monitor-recorded alarms. Third, despite the use of silent observers and an actual, rather than a simulated, clinical setting, by virtue of the data collection method we likely captured a more deliberate thought process (so-called System 2 thinking)²² rather than the subconscious processes that may predominate when nurses respond to alarms in the course of clinical care (System 1 thinking).²² Despite this limitation, our study findings, which reflect a nurse’s in-the-moment thinking, remain relevant to guiding the improvement of monitoring systems, and the development of nurse-facing interventions and education. Finally, we studied a small, purposive sample of nurses at a single hospital. Our study sample impacts the generalizability of our results and precluded a detailed analysis of the effect of nurse- and patient-level variables.

We found that nurses often deemed that no response was necessary for CPM alarms. Nurses cited contextual factors, including the duration of alarms and the presence of other providers or parents in their decision-making. Few (7%) of the alarm responses in our study included a clinical intervention. The number of observed alarm responses constituted roughly a third of the alarms recorded by bedside CPMs during the study. This result supports concerns about the nurse’s capacity to hear and process all CPM alarms given system limitations and a heavy clinical workload. Subsequent steps should include staff education, reducing overall alarm rates with appropriate monitor use and actionable alarm thresholds, and ensuring that patient alarms are easily recognizable for frontline staff.

Disclosures

The authors have no conflicts of interest to disclose.

Funding

This work was supported by the Place Outcomes Research Award from the Cincinnati Children’s Research Foundation. Dr. Brady is supported by the Agency for Healthcare Research and Quality under Award Number K08HS23827. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality.

Alarms from bedside continuous physiologic monitors (CPMs) occur frequently in children’s hospitals and can lead to harm. Recent studies conducted in children’s hospitals have identified alarm rates of up to 152 alarms per patient per day outside of the intensive care unit,^1-3 with as few as 1% of alarms being considered clinically important.⁴ Excessive alarms have been linked to alarm fatigue, when providers become desensitized to and may miss alarms indicating impending patient deterioration. Alarm fatigue has been identified by national patient safety organizations as a patient safety concern given the risk of patient harm.^5-7 Despite these concerns, CPMs are routinely used: up to 48% of pediatric patients in nonintensive care units at children’s hospitals are monitored.²

Although the low number of alarms that receive responses has been well-described,^8,9 the reasons why clinicians do or do not respond to alarms are unclear. A study conducted in an adult perioperative unit noted prolonged nurse response times for patients with high alarm rates.¹⁰ A second study conducted in the pediatric inpatient setting demonstrated a dose-response effect and noted progressively prolonged nurse response times with increased rates of nonactionable alarms.^4,11 Findings from another study suggested that underlying factors are highly complex and may be a result of excessive alarms, clinician characteristics, and working conditions (eg, workload and unit noise level).¹² Evidence also suggests that humans have difficulty distinguishing the importance of alarms in situations where multiple alarm tones are used, a common scenario in hospitals.^13,14 Understanding the factors that contribute to clinicians responding or not responding to CPM alarms will be crucial for addressing this serious patient safety issue.

An enhanced understanding of why nurses respond to alarms in daily practice will inform intervention development and improvement work. In the long term, this information could help improve systems for monitoring pediatric inpatients that are less prone to issues with alarm fatigue. The objective of this qualitative study, which employed structured observation, was to describe how bedside nurses think about and act upon bedside monitor alarms in a general pediatric inpatient unit.

Study Design and Setting

This prospective observational study took place on a 48-bed hospital medicine unit at a large, freestanding children’s hospital with >650 beds and >19,000 annual admissions. General Electric (Little Chalfont, United Kingdom) physiologic monitors (models Dash 3000, 4000, and 5000) were used at the time of the study, and nurses could be notified of monitor alarms in four ways: First, an in-room auditory alarm sounds. Second, a light positioned above the door outside of each patient room blinks for alarms that are at a “warning” or “critical level” (eg ventricular tachycardia or low oxygen saturation). Third, audible alarms occur at the unit’s central monitoring station. Lastly, another staff member can notify the patient’s nurse via in-person conversion or secure smart phone communication. On the study unit, CPMs are initiated and discontinued through a physician order.

This study was reviewed and approved by the hospital’s institutional review board.

Study Population

We used a purposive recruitment strategy to enroll bedside nurses working on general hospital medicine units, stratified to ensure varying levels of experience and primary shifts (eg, day vs night). We planned to conduct approximately two observations with each participating nurse and to continue collecting data until we could no longer identify new insights in terms of responses to alarms (ie, thematic saturation¹⁵). Observations were targeted to cover times of day that coincided with increased rates of distraction. These times included just prior to and after the morning and evening change of shifts (7:00 am and 7:00 pm), during morning rounds (8:00 am-12:00 pm), and heavy admission times (12:00 pm-10:00 pm). After written informed consent, a nurse was eligible for observation during his/her shift if he/she was caring for at least one monitored patient. Enrolled nurses were made aware of the general study topic but were blinded to the study team’s hypotheses.

Data Sources

Prior to data collection, the research team, which consisted of physicians, bedside nurses, research coordinators, and a human factors expert, created a system for categorizing alarm responses. Categories for observed responses were based on the location and corresponding action taken. Initial categories were developed a priori from existing literature and expanded through input from the multidisciplinary study team, then vetted with bedside staff, and finally pilot tested through >4 hours of observations, thus producing the final categories. These categories were entered into a work-sampling program (WorkStudy by Quetech Ltd., Waterloo, Ontario, Canada) to facilitate quick data recording during observations.

The hospital uses a central alarm collection software (BedMasterEx by Anandic Medical Systems, Feuerthalen, Switzerland), which permitted the collection of date, time, trigger (eg, high heart rate), and level (eg, crisis, warning) of the generated CPM alarms. Alarms collected are based on thresholds preset at the bedside monitor. The central collection software does not differentiate between accurate (eg, correctly representing the physiologic state of the patient) and inaccurate alarms.

Observation Procedure

At the time of observation, nurse demographic information (eg, primary shift worked and years working as a nurse) was obtained. A brief preobservation questionnaire was administered to collect patient information (eg, age and diagnosis) and the nurses’ perspectives on the necessity of monitors for each monitored patient in his/her care.

The observer shadowed the nurse for a two-hour block of his/her shift. During this time, nurses were instructed to “think aloud” as they responded to alarms (eg, “I notice the oxygen saturation monitor alarming off, but the probe has fallen off”). A trained observer (AML or KMT) recorded responses verbalized by the nurse and his/her reaction by selecting the appropriate category using the work-sampling software. Data were also collected on the vital sign associated with the alarm (eg, heart rate). Moreover, the observer kept written notes to provide context for electronically recorded data. Alarms that were not verbalized by the nurse were not counted. Similarly, alarms that were noted outside of the room by the nurse were not classified by vital sign unless the nurse confirmed with the bedside monitor. Observers did not adjudicate the accuracy of the alarms. The session was stopped if monitors were discontinued during the observation period. Alarm data generated by the bedside monitor were pulled for each patient room after observations were completed.

Analysis

Descriptive statistics were used to assess the percentage of each nurse response category and each alarm type (eg, heart rate and respiratory rate). The observed alarm rate was calculated by taking the total number of observed alarms (ie, alarms noted by the nurse) divided by the total number of patient-hours observed. The monitor-generated alarm rate was calculated by taking the total number of alarms from the bedside-alarm generated data divided by the number of patient-hours observed.

Electronically recorded observations using the work-sampling program were cross-referenced with hand-written field notes to assess for any discrepancies or identify relevant events not captured by the program. Three study team members (AML, KMT, and ACS) reviewed each observation independently and compared field notes to ensure accurate categorization. Discrepancies were referred to the larger study group in cases of uncertainty.

Nine nurses had monitored patients during the available observations and participated in 19 observation sessions, which included 35 monitored patients for a total of 61.3 patient-hours of observation. Nurses were observed for a median of two times each (range 1-4). The median number of monitored patients during a single observation session was two (range 1-3). Observed nurses were female with a median of eight years of experience (range 0.5-26 years). Patients represented a broad range of age categories and were hospitalized with a variety of diagnoses (Table). Nurses, when queried at the start of the observation, felt that monitors were necessary for 29 (82.9%) of the observed patients given either patient condition or unit policy.

A total of 207 observed nurse responses to alarms occurred during the study period for a rate of 3.4 responses per patient per hour. Of the total number of responses, 45 (21.7%) were noted outside of a patient room, and in 15 (33.3%) the nurse chose to go to the room. The other 162 were recorded when the nurse was present in the room when the alarm activated. Of the 177 in-person nurse responses, 50 were related to a pulse oximetry alarm, 66 were related to a heart rate alarm, and 61 were related to a respiratory rate alarm. The most common observed in-person response to an alarm involved the nurse judging that no intervention was necessary (n = 152, 73.1%). Only 14 (7% of total responses) observed in-person responses involved a clinical intervention, such as suctioning or titrating supplemental oxygen. Findings are summarized in the Figure and describe nurse-verbalized reasons to further assess (or not) and then whether the nurse chose to take action (or not) after an alarm.

We characterized responses to physiologic monitor alarms by a group of nurses with a range of experience levels. We found that most nurse responses to alarms in continuously monitored general pediatric patients involved no intervention, and further assessment was often not conducted for alarms that occurred outside of the room if the nurse noted otherwise reassuring clinical context. Observed responses occurred for 36% of alarms during the study period when compared with bedside monitor-alarm generated data. Overall, only 14 clinical interventions were noted among the observed responses. Nurses noted that they felt the monitors were necessary for 82.9% of monitored patients because of the clinical context or because of unit policy.

Our study findings highlight some potential contradictions in the current widespread use of CPMs in general pediatric units and how clinicians respond to them in practice.² First, while nurses reported that monitors were necessary for most of their patients, participating nurses deemed few alarms clinically actionable and often chose not to further assess when they noted alarms outside of the room. This is in line with findings from prior studies suggesting that clinicians overvalue the contribution of monitoring systems to patient safety.^16,17 Second, while this finding occurred in a minority of the observations, the presence of family members at the patient’s bedside was cited by nurses as a rationale for whether they responded to alarms. While family members are capable of identifying safety issues,¹⁸ formal systems to engage them in patient safety and physiologic monitoring are lacking. Finally, clinical interventions or responses to the alerts of deteriorating patients, which best represented the original intent of CPMs, were rare and accounted for just 7% of the responses. Further work elucidating why physicians and nurses choose to use CPMs may be helpful to identify interventions to reduce inappropriate monitor use and highlight gaps in frontline staff knowledge about the benefits and risks of CPM use.

Our findings provide a novel understanding of previously observed phenomena, such as long response times or nonresponses in settings with high alarm rates.^4,10 Similar to that in a prior study conducted in the pediatric setting,¹¹ alarms with an observed response constituted a minority of the total alarms that occurred in our study. This finding has previously been attributed to mental fatigue, caregiver apathy, and desensitization.⁸ However, even though a minority of observed responses in our study included an intervention, the nurse had a rationale for why the alarm did or did not need a response. This behavior and the verbalized rationale indicate that in his/her opinion, not responding to the alarm was clinically appropriate. Study participants also reflected on the difficulties of responding to alarms given the monitor system setup, in which they may not always be capable of hearing alarms for their patients. Without data from nurses regarding the alarms that had no observed response, we can only speculate; however, based on our findings, each of these factors could contribute to nonresponse. Finally, while high numbers of false alarms have been posited as an underlying cause of alarm fatigue, we noted that a majority of nonresponse was reported to be related to other clinical factors. This relationship suggests that from the nurse’s perspective, a more applicable framework for understanding alarms would be based on clinical actionability⁴ over physiologic accuracy.

In total, our findings suggest that a multifaceted approach will be necessary to improve alarm response rates. These interventions should include adjusting parameters such that alarms are highly likely to indicate a need for intervention coupled with educational interventions addressing clinician knowledge of the alarm system and bias about the actionability of alarms may improve response rates. Changes in the monitoring system setup such that nurses can easily be notified when alarms occur may also be indicated, in addition to formally engaging patients and families around response to alarms. Although secondary notification systems (eg, alarms transmitted to individual clinician’s devices) are one solution, the utilization of these systems needs to be balanced with the risks of contributing to existing alarm fatigue and the need to appropriately tailor monitoring thresholds and strategies to patients.

Our study has several limitations. First, nurses may have responded in a way they perceive to be socially desirable, and studies using in-person observers are also prone to a Hawthorne-like effect,^19-21 where the nurse may have tried to respond more frequently to alarms than usual during observations. However, given that the majority of bedside alarms did not receive a response and a substantial number of responses involved no action, these effects were likely weak. Second, we were unable to assess which alarms were accurately reflecting the patient’s physiologic status and which were not; we were also unable to link observed alarm response to monitor-recorded alarms. Third, despite the use of silent observers and an actual, rather than a simulated, clinical setting, by virtue of the data collection method we likely captured a more deliberate thought process (so-called System 2 thinking)²² rather than the subconscious processes that may predominate when nurses respond to alarms in the course of clinical care (System 1 thinking).²² Despite this limitation, our study findings, which reflect a nurse’s in-the-moment thinking, remain relevant to guiding the improvement of monitoring systems, and the development of nurse-facing interventions and education. Finally, we studied a small, purposive sample of nurses at a single hospital. Our study sample impacts the generalizability of our results and precluded a detailed analysis of the effect of nurse- and patient-level variables.

We found that nurses often deemed that no response was necessary for CPM alarms. Nurses cited contextual factors, including the duration of alarms and the presence of other providers or parents in their decision-making. Few (7%) of the alarm responses in our study included a clinical intervention. The number of observed alarm responses constituted roughly a third of the alarms recorded by bedside CPMs during the study. This result supports concerns about the nurse’s capacity to hear and process all CPM alarms given system limitations and a heavy clinical workload. Subsequent steps should include staff education, reducing overall alarm rates with appropriate monitor use and actionable alarm thresholds, and ensuring that patient alarms are easily recognizable for frontline staff.

Disclosures

The authors have no conflicts of interest to disclose.

Funding

This work was supported by the Place Outcomes Research Award from the Cincinnati Children’s Research Foundation. Dr. Brady is supported by the Agency for Healthcare Research and Quality under Award Number K08HS23827. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality.

Alarms from bedside continuous physiologic monitors (CPMs) occur frequently in children’s hospitals and can lead to harm. Recent studies conducted in children’s hospitals have identified alarm rates of up to 152 alarms per patient per day outside of the intensive care unit,^1-3 with as few as 1% of alarms being considered clinically important.⁴ Excessive alarms have been linked to alarm fatigue, when providers become desensitized to and may miss alarms indicating impending patient deterioration. Alarm fatigue has been identified by national patient safety organizations as a patient safety concern given the risk of patient harm.^5-7 Despite these concerns, CPMs are routinely used: up to 48% of pediatric patients in nonintensive care units at children’s hospitals are monitored.²

Although the low number of alarms that receive responses has been well-described,^8,9 the reasons why clinicians do or do not respond to alarms are unclear. A study conducted in an adult perioperative unit noted prolonged nurse response times for patients with high alarm rates.¹⁰ A second study conducted in the pediatric inpatient setting demonstrated a dose-response effect and noted progressively prolonged nurse response times with increased rates of nonactionable alarms.^4,11 Findings from another study suggested that underlying factors are highly complex and may be a result of excessive alarms, clinician characteristics, and working conditions (eg, workload and unit noise level).¹² Evidence also suggests that humans have difficulty distinguishing the importance of alarms in situations where multiple alarm tones are used, a common scenario in hospitals.^13,14 Understanding the factors that contribute to clinicians responding or not responding to CPM alarms will be crucial for addressing this serious patient safety issue.

An enhanced understanding of why nurses respond to alarms in daily practice will inform intervention development and improvement work. In the long term, this information could help improve systems for monitoring pediatric inpatients that are less prone to issues with alarm fatigue. The objective of this qualitative study, which employed structured observation, was to describe how bedside nurses think about and act upon bedside monitor alarms in a general pediatric inpatient unit.

Study Design and Setting

This prospective observational study took place on a 48-bed hospital medicine unit at a large, freestanding children’s hospital with >650 beds and >19,000 annual admissions. General Electric (Little Chalfont, United Kingdom) physiologic monitors (models Dash 3000, 4000, and 5000) were used at the time of the study, and nurses could be notified of monitor alarms in four ways: First, an in-room auditory alarm sounds. Second, a light positioned above the door outside of each patient room blinks for alarms that are at a “warning” or “critical level” (eg ventricular tachycardia or low oxygen saturation). Third, audible alarms occur at the unit’s central monitoring station. Lastly, another staff member can notify the patient’s nurse via in-person conversion or secure smart phone communication. On the study unit, CPMs are initiated and discontinued through a physician order.

This study was reviewed and approved by the hospital’s institutional review board.

Study Population

We used a purposive recruitment strategy to enroll bedside nurses working on general hospital medicine units, stratified to ensure varying levels of experience and primary shifts (eg, day vs night). We planned to conduct approximately two observations with each participating nurse and to continue collecting data until we could no longer identify new insights in terms of responses to alarms (ie, thematic saturation¹⁵). Observations were targeted to cover times of day that coincided with increased rates of distraction. These times included just prior to and after the morning and evening change of shifts (7:00 am and 7:00 pm), during morning rounds (8:00 am-12:00 pm), and heavy admission times (12:00 pm-10:00 pm). After written informed consent, a nurse was eligible for observation during his/her shift if he/she was caring for at least one monitored patient. Enrolled nurses were made aware of the general study topic but were blinded to the study team’s hypotheses.

Data Sources

Prior to data collection, the research team, which consisted of physicians, bedside nurses, research coordinators, and a human factors expert, created a system for categorizing alarm responses. Categories for observed responses were based on the location and corresponding action taken. Initial categories were developed a priori from existing literature and expanded through input from the multidisciplinary study team, then vetted with bedside staff, and finally pilot tested through >4 hours of observations, thus producing the final categories. These categories were entered into a work-sampling program (WorkStudy by Quetech Ltd., Waterloo, Ontario, Canada) to facilitate quick data recording during observations.

The hospital uses a central alarm collection software (BedMasterEx by Anandic Medical Systems, Feuerthalen, Switzerland), which permitted the collection of date, time, trigger (eg, high heart rate), and level (eg, crisis, warning) of the generated CPM alarms. Alarms collected are based on thresholds preset at the bedside monitor. The central collection software does not differentiate between accurate (eg, correctly representing the physiologic state of the patient) and inaccurate alarms.

Observation Procedure

At the time of observation, nurse demographic information (eg, primary shift worked and years working as a nurse) was obtained. A brief preobservation questionnaire was administered to collect patient information (eg, age and diagnosis) and the nurses’ perspectives on the necessity of monitors for each monitored patient in his/her care.

The observer shadowed the nurse for a two-hour block of his/her shift. During this time, nurses were instructed to “think aloud” as they responded to alarms (eg, “I notice the oxygen saturation monitor alarming off, but the probe has fallen off”). A trained observer (AML or KMT) recorded responses verbalized by the nurse and his/her reaction by selecting the appropriate category using the work-sampling software. Data were also collected on the vital sign associated with the alarm (eg, heart rate). Moreover, the observer kept written notes to provide context for electronically recorded data. Alarms that were not verbalized by the nurse were not counted. Similarly, alarms that were noted outside of the room by the nurse were not classified by vital sign unless the nurse confirmed with the bedside monitor. Observers did not adjudicate the accuracy of the alarms. The session was stopped if monitors were discontinued during the observation period. Alarm data generated by the bedside monitor were pulled for each patient room after observations were completed.

Analysis

Descriptive statistics were used to assess the percentage of each nurse response category and each alarm type (eg, heart rate and respiratory rate). The observed alarm rate was calculated by taking the total number of observed alarms (ie, alarms noted by the nurse) divided by the total number of patient-hours observed. The monitor-generated alarm rate was calculated by taking the total number of alarms from the bedside-alarm generated data divided by the number of patient-hours observed.

Electronically recorded observations using the work-sampling program were cross-referenced with hand-written field notes to assess for any discrepancies or identify relevant events not captured by the program. Three study team members (AML, KMT, and ACS) reviewed each observation independently and compared field notes to ensure accurate categorization. Discrepancies were referred to the larger study group in cases of uncertainty.

Nine nurses had monitored patients during the available observations and participated in 19 observation sessions, which included 35 monitored patients for a total of 61.3 patient-hours of observation. Nurses were observed for a median of two times each (range 1-4). The median number of monitored patients during a single observation session was two (range 1-3). Observed nurses were female with a median of eight years of experience (range 0.5-26 years). Patients represented a broad range of age categories and were hospitalized with a variety of diagnoses (Table). Nurses, when queried at the start of the observation, felt that monitors were necessary for 29 (82.9%) of the observed patients given either patient condition or unit policy.

A total of 207 observed nurse responses to alarms occurred during the study period for a rate of 3.4 responses per patient per hour. Of the total number of responses, 45 (21.7%) were noted outside of a patient room, and in 15 (33.3%) the nurse chose to go to the room. The other 162 were recorded when the nurse was present in the room when the alarm activated. Of the 177 in-person nurse responses, 50 were related to a pulse oximetry alarm, 66 were related to a heart rate alarm, and 61 were related to a respiratory rate alarm. The most common observed in-person response to an alarm involved the nurse judging that no intervention was necessary (n = 152, 73.1%). Only 14 (7% of total responses) observed in-person responses involved a clinical intervention, such as suctioning or titrating supplemental oxygen. Findings are summarized in the Figure and describe nurse-verbalized reasons to further assess (or not) and then whether the nurse chose to take action (or not) after an alarm.

We characterized responses to physiologic monitor alarms by a group of nurses with a range of experience levels. We found that most nurse responses to alarms in continuously monitored general pediatric patients involved no intervention, and further assessment was often not conducted for alarms that occurred outside of the room if the nurse noted otherwise reassuring clinical context. Observed responses occurred for 36% of alarms during the study period when compared with bedside monitor-alarm generated data. Overall, only 14 clinical interventions were noted among the observed responses. Nurses noted that they felt the monitors were necessary for 82.9% of monitored patients because of the clinical context or because of unit policy.

Our study findings highlight some potential contradictions in the current widespread use of CPMs in general pediatric units and how clinicians respond to them in practice.² First, while nurses reported that monitors were necessary for most of their patients, participating nurses deemed few alarms clinically actionable and often chose not to further assess when they noted alarms outside of the room. This is in line with findings from prior studies suggesting that clinicians overvalue the contribution of monitoring systems to patient safety.^16,17 Second, while this finding occurred in a minority of the observations, the presence of family members at the patient’s bedside was cited by nurses as a rationale for whether they responded to alarms. While family members are capable of identifying safety issues,¹⁸ formal systems to engage them in patient safety and physiologic monitoring are lacking. Finally, clinical interventions or responses to the alerts of deteriorating patients, which best represented the original intent of CPMs, were rare and accounted for just 7% of the responses. Further work elucidating why physicians and nurses choose to use CPMs may be helpful to identify interventions to reduce inappropriate monitor use and highlight gaps in frontline staff knowledge about the benefits and risks of CPM use.

Our findings provide a novel understanding of previously observed phenomena, such as long response times or nonresponses in settings with high alarm rates.^4,10 Similar to that in a prior study conducted in the pediatric setting,¹¹ alarms with an observed response constituted a minority of the total alarms that occurred in our study. This finding has previously been attributed to mental fatigue, caregiver apathy, and desensitization.⁸ However, even though a minority of observed responses in our study included an intervention, the nurse had a rationale for why the alarm did or did not need a response. This behavior and the verbalized rationale indicate that in his/her opinion, not responding to the alarm was clinically appropriate. Study participants also reflected on the difficulties of responding to alarms given the monitor system setup, in which they may not always be capable of hearing alarms for their patients. Without data from nurses regarding the alarms that had no observed response, we can only speculate; however, based on our findings, each of these factors could contribute to nonresponse. Finally, while high numbers of false alarms have been posited as an underlying cause of alarm fatigue, we noted that a majority of nonresponse was reported to be related to other clinical factors. This relationship suggests that from the nurse’s perspective, a more applicable framework for understanding alarms would be based on clinical actionability⁴ over physiologic accuracy.

In total, our findings suggest that a multifaceted approach will be necessary to improve alarm response rates. These interventions should include adjusting parameters such that alarms are highly likely to indicate a need for intervention coupled with educational interventions addressing clinician knowledge of the alarm system and bias about the actionability of alarms may improve response rates. Changes in the monitoring system setup such that nurses can easily be notified when alarms occur may also be indicated, in addition to formally engaging patients and families around response to alarms. Although secondary notification systems (eg, alarms transmitted to individual clinician’s devices) are one solution, the utilization of these systems needs to be balanced with the risks of contributing to existing alarm fatigue and the need to appropriately tailor monitoring thresholds and strategies to patients.

Our study has several limitations. First, nurses may have responded in a way they perceive to be socially desirable, and studies using in-person observers are also prone to a Hawthorne-like effect,^19-21 where the nurse may have tried to respond more frequently to alarms than usual during observations. However, given that the majority of bedside alarms did not receive a response and a substantial number of responses involved no action, these effects were likely weak. Second, we were unable to assess which alarms were accurately reflecting the patient’s physiologic status and which were not; we were also unable to link observed alarm response to monitor-recorded alarms. Third, despite the use of silent observers and an actual, rather than a simulated, clinical setting, by virtue of the data collection method we likely captured a more deliberate thought process (so-called System 2 thinking)²² rather than the subconscious processes that may predominate when nurses respond to alarms in the course of clinical care (System 1 thinking).²² Despite this limitation, our study findings, which reflect a nurse’s in-the-moment thinking, remain relevant to guiding the improvement of monitoring systems, and the development of nurse-facing interventions and education. Finally, we studied a small, purposive sample of nurses at a single hospital. Our study sample impacts the generalizability of our results and precluded a detailed analysis of the effect of nurse- and patient-level variables.

We found that nurses often deemed that no response was necessary for CPM alarms. Nurses cited contextual factors, including the duration of alarms and the presence of other providers or parents in their decision-making. Few (7%) of the alarm responses in our study included a clinical intervention. The number of observed alarm responses constituted roughly a third of the alarms recorded by bedside CPMs during the study. This result supports concerns about the nurse’s capacity to hear and process all CPM alarms given system limitations and a heavy clinical workload. Subsequent steps should include staff education, reducing overall alarm rates with appropriate monitor use and actionable alarm thresholds, and ensuring that patient alarms are easily recognizable for frontline staff.

Disclosures

The authors have no conflicts of interest to disclose.

Funding

This work was supported by the Place Outcomes Research Award from the Cincinnati Children’s Research Foundation. Dr. Brady is supported by the Agency for Healthcare Research and Quality under Award Number K08HS23827. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality.