Elizabeth Rice-Conboy, MS

BACKGROUND AND OBJECTIVES: Adherence to American Academy of Pediatrics (AAP) bronchiolitis clinical practice guideline recommendations improved significantly through the AAP’s multi-institutional collaborative the Bronchiolitis Quality Improvement Project (BQIP). We assessed sustainability of improvements at participating institutions for 1 year following completion of the collaborative.

METHODS: Twenty-one multidisciplinary hospital-based teams provided monthly data for key inpatient bronchiolitis measures during baseline and intervention bronchiolitis seasons. Nine sites provided data in the season following completion of the collaborative. Encounters included children younger than 24 months who were hospitalized for bronchiolitis without comorbid chronic illness, prematurity, or intensive care. Changes between baseline-, intervention-, and sustainability-season data were assessed using generalized linear mixed-effects models with site-specific random effects. Differences between hospital characteristics, baseline performance, and initial improvement among sites that did and did not participate in the sustainability season were compared.

RESULTS: A total of 2,275 discharges were reviewed, comprising 995 baseline, 877 intervention, and 403 sustainability-season encounters. Improvements in all key bronchiolitis quality measures achieved during the intervention season were maintained during the sustainability season, and orders for intermittent pulse oximetry increased from 40.6% (95% confidence interval, 22.8-61.1) to 79.2% (95% CI, 58.0-91.3). Sites that did and did not participate in the sustainability season had similar characteristics.

DISCUSSION: BQIP participating sites maintained improvements in key bronchiolitis quality measures for 1 year following the project’s completion. This approach, which provided an evidence-based best-practice toolkit while building the quality-improvement capacity of local interdisciplinary teams, may support performance gains that persist beyond the active phase of the collaborative.
 

Also in JHM this month

The effect of an inpatient smoking cessation treatment program on hospital readmissions and length of stayAUTHORS: Eline M. van den Broek-Altenburg, MS, MA, Adam J. Atherly, PhD

Treatment trends and outcomes in healthcare-associated pneumoniaAUTHORS: Sarah Haessler, MD; Tara Lagu, MD, MPH; Peter K. Lindenauer, MD, MSc; Daniel J. Skiest, MD; Aruna Priya, MA, MSc; Penelope S. Pekow, PhD; Marya D. Zilberberg, MD, MPH; Thomas L. Higgins, MD, MBA; Michael B. Rothberg, MD, MPH

What’s the purpose of rounds? A qualitative study examining the perceptions of faculty and studentsAUTHORS: Oliver Hulland; Jeanne Farnan, MD, MHPE; Raphael Rabinowitz; Lisa Kearns, MD, MS; Michele Long, MD; Bradley Monash, MD; Priti Bhansali, MD; H. Barrett Fromme, MD, MHPE

Association between anemia and fatigue in hospitalized patients: does the measure of anemia matter?AUTHORS: Micah T. Prochaska, MD, MS; Richard Newcomb, BA; Graham Block, BA; Brian Park, BA; David O. Meltzer MD, PhD

Helping seniors plan for posthospital discharge needs before a hospitalization occurs: Results from the randomized control trial of planyourlifespan.orgAUTHORS: Lee A. Lindquist, MD, MPH, MBA; Vanessa Ramirez-Zohfeld, MPH; Priya D. Sunkara, MA; Chris Forcucci, RN, BSN; Dianne S. Campbell, BS; Phyllis Mitzen, MA; Jody D. Ciolino, PhD; Gayle Kricke, MSW; Anne Seltzer, LSW; Ana V. Ramirez, BA; Kenzie A. Cameron, PhD, MPH

BACKGROUND AND OBJECTIVES: Adherence to American Academy of Pediatrics (AAP) bronchiolitis clinical practice guideline recommendations improved significantly through the AAP’s multi-institutional collaborative the Bronchiolitis Quality Improvement Project (BQIP). We assessed sustainability of improvements at participating institutions for 1 year following completion of the collaborative.

METHODS: Twenty-one multidisciplinary hospital-based teams provided monthly data for key inpatient bronchiolitis measures during baseline and intervention bronchiolitis seasons. Nine sites provided data in the season following completion of the collaborative. Encounters included children younger than 24 months who were hospitalized for bronchiolitis without comorbid chronic illness, prematurity, or intensive care. Changes between baseline-, intervention-, and sustainability-season data were assessed using generalized linear mixed-effects models with site-specific random effects. Differences between hospital characteristics, baseline performance, and initial improvement among sites that did and did not participate in the sustainability season were compared.

RESULTS: A total of 2,275 discharges were reviewed, comprising 995 baseline, 877 intervention, and 403 sustainability-season encounters. Improvements in all key bronchiolitis quality measures achieved during the intervention season were maintained during the sustainability season, and orders for intermittent pulse oximetry increased from 40.6% (95% confidence interval, 22.8-61.1) to 79.2% (95% CI, 58.0-91.3). Sites that did and did not participate in the sustainability season had similar characteristics.

DISCUSSION: BQIP participating sites maintained improvements in key bronchiolitis quality measures for 1 year following the project’s completion. This approach, which provided an evidence-based best-practice toolkit while building the quality-improvement capacity of local interdisciplinary teams, may support performance gains that persist beyond the active phase of the collaborative.
 

Also in JHM this month

The effect of an inpatient smoking cessation treatment program on hospital readmissions and length of stayAUTHORS: Eline M. van den Broek-Altenburg, MS, MA, Adam J. Atherly, PhD

Treatment trends and outcomes in healthcare-associated pneumoniaAUTHORS: Sarah Haessler, MD; Tara Lagu, MD, MPH; Peter K. Lindenauer, MD, MSc; Daniel J. Skiest, MD; Aruna Priya, MA, MSc; Penelope S. Pekow, PhD; Marya D. Zilberberg, MD, MPH; Thomas L. Higgins, MD, MBA; Michael B. Rothberg, MD, MPH

What’s the purpose of rounds? A qualitative study examining the perceptions of faculty and studentsAUTHORS: Oliver Hulland; Jeanne Farnan, MD, MHPE; Raphael Rabinowitz; Lisa Kearns, MD, MS; Michele Long, MD; Bradley Monash, MD; Priti Bhansali, MD; H. Barrett Fromme, MD, MHPE

Association between anemia and fatigue in hospitalized patients: does the measure of anemia matter?AUTHORS: Micah T. Prochaska, MD, MS; Richard Newcomb, BA; Graham Block, BA; Brian Park, BA; David O. Meltzer MD, PhD

Helping seniors plan for posthospital discharge needs before a hospitalization occurs: Results from the randomized control trial of planyourlifespan.orgAUTHORS: Lee A. Lindquist, MD, MPH, MBA; Vanessa Ramirez-Zohfeld, MPH; Priya D. Sunkara, MA; Chris Forcucci, RN, BSN; Dianne S. Campbell, BS; Phyllis Mitzen, MA; Jody D. Ciolino, PhD; Gayle Kricke, MSW; Anne Seltzer, LSW; Ana V. Ramirez, BA; Kenzie A. Cameron, PhD, MPH

BACKGROUND AND OBJECTIVES: Adherence to American Academy of Pediatrics (AAP) bronchiolitis clinical practice guideline recommendations improved significantly through the AAP’s multi-institutional collaborative the Bronchiolitis Quality Improvement Project (BQIP). We assessed sustainability of improvements at participating institutions for 1 year following completion of the collaborative.

METHODS: Twenty-one multidisciplinary hospital-based teams provided monthly data for key inpatient bronchiolitis measures during baseline and intervention bronchiolitis seasons. Nine sites provided data in the season following completion of the collaborative. Encounters included children younger than 24 months who were hospitalized for bronchiolitis without comorbid chronic illness, prematurity, or intensive care. Changes between baseline-, intervention-, and sustainability-season data were assessed using generalized linear mixed-effects models with site-specific random effects. Differences between hospital characteristics, baseline performance, and initial improvement among sites that did and did not participate in the sustainability season were compared.

RESULTS: A total of 2,275 discharges were reviewed, comprising 995 baseline, 877 intervention, and 403 sustainability-season encounters. Improvements in all key bronchiolitis quality measures achieved during the intervention season were maintained during the sustainability season, and orders for intermittent pulse oximetry increased from 40.6% (95% confidence interval, 22.8-61.1) to 79.2% (95% CI, 58.0-91.3). Sites that did and did not participate in the sustainability season had similar characteristics.

DISCUSSION: BQIP participating sites maintained improvements in key bronchiolitis quality measures for 1 year following the project’s completion. This approach, which provided an evidence-based best-practice toolkit while building the quality-improvement capacity of local interdisciplinary teams, may support performance gains that persist beyond the active phase of the collaborative.
 

Also in JHM this month

The effect of an inpatient smoking cessation treatment program on hospital readmissions and length of stayAUTHORS: Eline M. van den Broek-Altenburg, MS, MA, Adam J. Atherly, PhD

Treatment trends and outcomes in healthcare-associated pneumoniaAUTHORS: Sarah Haessler, MD; Tara Lagu, MD, MPH; Peter K. Lindenauer, MD, MSc; Daniel J. Skiest, MD; Aruna Priya, MA, MSc; Penelope S. Pekow, PhD; Marya D. Zilberberg, MD, MPH; Thomas L. Higgins, MD, MBA; Michael B. Rothberg, MD, MPH

What’s the purpose of rounds? A qualitative study examining the perceptions of faculty and studentsAUTHORS: Oliver Hulland; Jeanne Farnan, MD, MHPE; Raphael Rabinowitz; Lisa Kearns, MD, MS; Michele Long, MD; Bradley Monash, MD; Priti Bhansali, MD; H. Barrett Fromme, MD, MHPE

Association between anemia and fatigue in hospitalized patients: does the measure of anemia matter?AUTHORS: Micah T. Prochaska, MD, MS; Richard Newcomb, BA; Graham Block, BA; Brian Park, BA; David O. Meltzer MD, PhD

Helping seniors plan for posthospital discharge needs before a hospitalization occurs: Results from the randomized control trial of planyourlifespan.orgAUTHORS: Lee A. Lindquist, MD, MPH, MBA; Vanessa Ramirez-Zohfeld, MPH; Priya D. Sunkara, MA; Chris Forcucci, RN, BSN; Dianne S. Campbell, BS; Phyllis Mitzen, MA; Jody D. Ciolino, PhD; Gayle Kricke, MSW; Anne Seltzer, LSW; Ana V. Ramirez, BA; Kenzie A. Cameron, PhD, MPH

Acute viral bronchiolitis is the most common cause of hospitalization for children less than 1 year of age.¹ Overuse of ineffective therapies has persisted despite the existence of the evidence-based American Academy of Pediatrics (AAP) clinical practice guideline (CPG), which recommends primarily supportive care.^2-8 Adherence to the AAP CPG recommendations for management of bronchiolitis improved significantly through the AAP’s Bronchiolitis Quality Improvement Project (BQIP), a 12-month, multiinstitutional collaborative of community and free-standing children’s hospitals.⁹ This subsequent study investigates if these improvements were sustained after completion of the formal 12-month project.

Published multiinstitutional bronchiolitis quality improvement (QI) work is limited to 1 study⁵ that describes the results of a single intervention season at academic medical centers. Multiyear bronchiolitis QI projects are limited to single-center studies, and results have been mixed.^5,6,8,10-13 One study¹¹ observed continued improvement in bronchodilator use in subsequent seasons, whereas a second study¹⁰ observed a return to baseline bronchodilator use in the following season. Mittal⁶ observed inconsistent improvements in key bronchiolitis measures during postintervention seasons.

Our specific aim was to assess the sustainability of improvements in bronchiolitis management at participating institutions 1 year following completion of the AAP BQIP collaborative.⁹ Because no studies demonstrate the most effective way to support long-term improvement through a QI collaborative, we hypothesized that the initial collaborative activities, which were designed to build the capacity of local interdisciplinary teams while providing standardized evidence-based care pathways, would lead to performance in the subsequent season at levels similar to or better than those observed during the active phase of the collaborative, without additional project interventions.

Study Design and Setting

This was a follow-up study of the AAP Quality Improvement Innovation Networks project entitled “A Quality Collaborative for Improving Hospital Compliance with the AAP Bronchiolitis Guideline” (BQIP).⁹ The AAP Institutional Review Board approved this project.

Twenty-one multidisciplinary, hospital-based teams participated in the BQIP collaborative and provided monthly data during the January through March bronchiolitis season. Teams submitted 2013 baseline data and 2014 intervention data. Nine sites provided 2015 sustainability data following the completion of the collaborative.

Participants

Hospital encounters with a primary diagnosis of acute viral bronchiolitis were eligible for inclusion among patients from 1 month to 2 years of age. Encounters were excluded for prematurity (<35 weeks gestational age), congenital heart disease, bronchopulmonary dysplasia, genetic, congenital or neuromuscular abnormalities, and pediatric intensive-care admission.

Data Collection

Hospital characteristics were collected, including hospital type (academic, community), bed size, location (urban, rural), hospital distributions of race/ethnicity and public payer, cases of bronchiolitis per year, presence of an electronic medical record and a pediatric respiratory therapist, and self-rated QI knowledge of the multidisciplinary team (very knowledgeable, knowledgeable, and somewhat knowledgeable). A trained member at each site collected data through structured chart review in baseline, intervention, and sustainability bronchiolitis seasons for January, February, and March. Site members reviewed the first 20 charts per month that met the inclusion criteria or all charts if there were fewer than 20 eligible encounters. Sites input data about key quality measures into the AAP’s Quality Improvement Data Aggregator, a web-based data repository.

Intervention

The BQIP project was designed as a virtual collaborative consisting of monthly education webinars about QI methods and bronchiolitis management, opportunities for collaboration via teleconference and e-mail listserv, and individual site-coaching by e-mail or telephone.⁹ A change package was shared with sites that included examples of evidence-based pathways, ordersets, a respiratory scoring tool, communication tools for parents and referring physicians, and slide sets for individual site education efforts. Following completion of the collaborative, written resources remained available to participants, although virtual collaboration ceased and no additional project interventions to promote sustainability were introduced.

Bronchiolitis Process and Outcome Measures

Process measures following admission included the following: severity assessment using a respiratory score, respiratory score use to assess response to bronchodilators, bronchodilator use, bronchodilator doses, steroid doses per patient encounter, chest radiographs per encounter, and presence of an order to transition to intermittent pulse oximetry monitoring. Outcome measures included length of stay and readmissions within 72 hours.

Analysis

Changes among baseline-, intervention-, and sustainability-season data were assessed using generalized linear mixed-effects models with random effect for study sites. Negative binomial models were used for count variables to allow for overdispersion. Length of stay was log-transformed to achieve a normal distribution. We also analyzed each site individually to assess whether sustained improvements were the result of broad sustainability across all sites or whether they represented an aggregation of some sites that continued to improve while other sites actually worsened.

To address any bias introduced by the voluntary and incomplete participation of sites in the sustainability season, we planned a priori to conduct 3 additional analyses. First, we compared the characteristics of sites that did participate in the sustainability season with those that did not participate by using Chi-squared tests for differences in proportions and t tests for differences in means. Second, we determined whether the baseline-season process and outcome measures were different between sites that did and did not participate using descriptive statistics. Third, we assessed whether improvements between the baseline and intervention seasons were different between sites that did and did not participate using a linear mixed-effects model for normally distributed outcomes and generalized linear mixed-effects model with site-specific random effects for nonnormally distributed outcomes. All study outcomes were summarized in terms of model-adjusted means along with the corresponding 95% confidence intervals. All P values are 2-sided, and P < 0.05 was used to define statistical significance. Data analyses were conducted using SAS software (SAS Institute Inc., Cary, North Carolina) version 9.4.

To our knowledge, this is the first report of sustained improvements in care achieved through a multiinstitutional QI collaborative of community and academic hospitals focused on bronchiolitis care. We found that overall sites participating in a national bronchiolitis QI project sustained improvements in key bronchiolitis quality measures for 1 year following the project’s completion. For the aggregate group no measures worsened, and one measure, orders for intermittent pulse oximetry monitoring, continued to increase during the sustainability season. Furthermore, the sustained improvements were primarily the result of consistent sustained performance of each individual site, as opposed to averages wherein some sites worsened while others improved (Appendix 1a). These findings suggest that designing a collaborative approach, which provides an evidence-based best-practice toolkit while building the QI capacity of local interdisciplinary teams, can support performance gains that persist beyond the project’s active phase.

There are a number of possible reasons why improvements were sustained following the collaborative. The BQIP requirement for institutional leadership buy-in may have motivated accountability to local leaders in subsequent bronchiolitis seasons at each site. We suspect that culture change such as flattened hierarchies through multidisciplinary teams,¹⁴ which empowered nurse and respiratory therapy staff, may have facilitated consistent use of tools created locally. The synergy of interdisciplinary teams composed of physician, nurse, and respiratory therapy champions may have created accountability to perpetuate the previous year’s efforts.¹⁵ In addition, the sites adopted elements of the evidence-based toolkit, such as pathways,^16,17 forcing function tools^13,18 and order sets that limited management decision options and bronchodilator use contingent on respiratory scores,^9,19 which may have driven desired behaviors.

Moreover, the 2014 AAP CPG for the management of bronchiolitis,²⁰ released prior to the sustainability bronchiolitis season, may have underscored the key concepts of the collaborative. Similarly, national exposure of best practices for bronchiolitis management, including the 3 widespread Choosing Wisely recommendations related to bronchiolitis,²¹ might have been a compelling reason for sites to maintain their improvement efforts and contribute to secular trends toward decreasing interventions in bronchiolitis management nationally.³ Lastly, the mechanisms developed for local data collection may have created opportunities at each site to conduct ongoing evaluation of performance on key bronchiolitis quality measures through data-driven feedback systems.²² Our study highlights the need for additional research in order to understand why improvements are or are not sustained.

Even with substantial, sustained improvements in this initiative, further reduction in unnecessary care may be possible. Findings from previous studies suggest that even multifaceted QI interventions, including provider education, guidelines and use of respiratory scores, may only modestly reduce bronchodilators, steroids, and chest radiograph use.^8,13 To achieve continued improvements in bronchiolitis care, additional active efforts may be needed to develop new interventions that target root causes for areas of overuse at individual sites.

Future multiinstitutional collaboratives might benefit their participants if they include a focus on helping sites develop skills to ensure that local improvement activities continue after the collaborative phases are completed. Proactively scheduling intermittent check-ins with collaborative members to discuss experiences with both sustainability and ongoing improvement may be valuable and likely needs to be incorporated into the initial collaborative planning.

As these sustainability data represent a subset of 9 of the original 21 BQIP sites, there is concern for potential selection bias related to factors that could have motivated sites to participate in the sustainability season’s data collection and simultaneously influenced their performance. These concerns were mitigated to some extent through 3 specific analyses: finding limited differences in hospital characteristics, baseline performance in key bronchiolitis measures, and performance change from baseline to intervention seasons between sites that did and did not participate in the sustainability season.

Notably, sites that participated in the sustainability phase actually had lower baseline respiratory score use and fewer orders for intermittent pulse oximetry at baseline. Theoretically, if participation in the collaborative highlighted this disparity for these sites, it could have been a motivating factor for their continued participation and sustained performance across these measures. Similarly, sites that recognized their higher baseline performance through participation in the collaborative might have felt less motivation to participate in ongoing data collection during the sustainability season. Whether they might have also sustained, declined, or continued improving is not known. Additionally, the magnitude of improvement in the collaborative period might have also motivated ongoing participation during the sustainability phase. For example, although all sites improved in score use during the collaborative, sites participating in the sustainability season demonstrated significantly more improvement in these measures. Sites with a higher magnitude of improvement in collaborative measures might have more enthusiasm about the project, more commitment to the project activities, or feel a sense of obligation to respond to requests for additional data collection.

This work has several limitations. Selection bias may limit generalizability of the results, as sites that did not participate in the sustainability season may have had different results than those that did participate. It is unknown whether sites that regressed toward their baseline were deterred from participating in the sustainability season. The analyses that we were able to preform, however, suggest that the 2 groups were similar in their characteristics as well as in their baseline and improvement performance.

We have limited knowledge of the local improvement work that sites conducted between the completion of the collaborative and the sustainability season. Site-specific factors may have influenced improvement sustainability. For example, qualitative research with the original group found that team engagement had a quantitative association with better performance, but only for the bronchodilator use measure.²³ Sites were responsible for their own data collection, and despite attempts to centralize and standardize the process, data collection inconsistencies may have occurred. For instance, it is unknown how closely that orders for intermittent pulse oximetry correlate with intermittent use at the bedside. Lastly, the absence of a control group limits examination of the causal relationships of interventions and the influence of secular trends.

Improvements gained during the BQIP collaborative were sustained at 1 year following completion of the collaborative. These findings are encouraging, as national QI collaborative efforts are increasingly common. Our findings suggest that opportunities exist to even further reduce unnecessary care in the management of bronchiolitis. Such opportunities highlight the importance of integrating strategies to both measure sustainability and plan for ongoing independent local activities after completion of the collaborative. Future efforts should focus on supporting local sites to continue individual practice-improvement as they transition from collaborative to independent quality initiatives.

Acknowledgments

The authors thank the 21 hospitals that participated in the BQIP collaborative, and in particular the 9 hospital teams that contributed sustainability data for their ongoing dedication. There was no external funding for this manuscript.

Disclosure

The authors report no financial conflicts of interest.

Acute viral bronchiolitis is the most common cause of hospitalization for children less than 1 year of age.¹ Overuse of ineffective therapies has persisted despite the existence of the evidence-based American Academy of Pediatrics (AAP) clinical practice guideline (CPG), which recommends primarily supportive care.^2-8 Adherence to the AAP CPG recommendations for management of bronchiolitis improved significantly through the AAP’s Bronchiolitis Quality Improvement Project (BQIP), a 12-month, multiinstitutional collaborative of community and free-standing children’s hospitals.⁹ This subsequent study investigates if these improvements were sustained after completion of the formal 12-month project.

Published multiinstitutional bronchiolitis quality improvement (QI) work is limited to 1 study⁵ that describes the results of a single intervention season at academic medical centers. Multiyear bronchiolitis QI projects are limited to single-center studies, and results have been mixed.^5,6,8,10-13 One study¹¹ observed continued improvement in bronchodilator use in subsequent seasons, whereas a second study¹⁰ observed a return to baseline bronchodilator use in the following season. Mittal⁶ observed inconsistent improvements in key bronchiolitis measures during postintervention seasons.

Our specific aim was to assess the sustainability of improvements in bronchiolitis management at participating institutions 1 year following completion of the AAP BQIP collaborative.⁹ Because no studies demonstrate the most effective way to support long-term improvement through a QI collaborative, we hypothesized that the initial collaborative activities, which were designed to build the capacity of local interdisciplinary teams while providing standardized evidence-based care pathways, would lead to performance in the subsequent season at levels similar to or better than those observed during the active phase of the collaborative, without additional project interventions.

Study Design and Setting

This was a follow-up study of the AAP Quality Improvement Innovation Networks project entitled “A Quality Collaborative for Improving Hospital Compliance with the AAP Bronchiolitis Guideline” (BQIP).⁹ The AAP Institutional Review Board approved this project.

Twenty-one multidisciplinary, hospital-based teams participated in the BQIP collaborative and provided monthly data during the January through March bronchiolitis season. Teams submitted 2013 baseline data and 2014 intervention data. Nine sites provided 2015 sustainability data following the completion of the collaborative.

Participants

Hospital encounters with a primary diagnosis of acute viral bronchiolitis were eligible for inclusion among patients from 1 month to 2 years of age. Encounters were excluded for prematurity (<35 weeks gestational age), congenital heart disease, bronchopulmonary dysplasia, genetic, congenital or neuromuscular abnormalities, and pediatric intensive-care admission.

Data Collection

Hospital characteristics were collected, including hospital type (academic, community), bed size, location (urban, rural), hospital distributions of race/ethnicity and public payer, cases of bronchiolitis per year, presence of an electronic medical record and a pediatric respiratory therapist, and self-rated QI knowledge of the multidisciplinary team (very knowledgeable, knowledgeable, and somewhat knowledgeable). A trained member at each site collected data through structured chart review in baseline, intervention, and sustainability bronchiolitis seasons for January, February, and March. Site members reviewed the first 20 charts per month that met the inclusion criteria or all charts if there were fewer than 20 eligible encounters. Sites input data about key quality measures into the AAP’s Quality Improvement Data Aggregator, a web-based data repository.

Intervention

The BQIP project was designed as a virtual collaborative consisting of monthly education webinars about QI methods and bronchiolitis management, opportunities for collaboration via teleconference and e-mail listserv, and individual site-coaching by e-mail or telephone.⁹ A change package was shared with sites that included examples of evidence-based pathways, ordersets, a respiratory scoring tool, communication tools for parents and referring physicians, and slide sets for individual site education efforts. Following completion of the collaborative, written resources remained available to participants, although virtual collaboration ceased and no additional project interventions to promote sustainability were introduced.

Bronchiolitis Process and Outcome Measures

Process measures following admission included the following: severity assessment using a respiratory score, respiratory score use to assess response to bronchodilators, bronchodilator use, bronchodilator doses, steroid doses per patient encounter, chest radiographs per encounter, and presence of an order to transition to intermittent pulse oximetry monitoring. Outcome measures included length of stay and readmissions within 72 hours.

Analysis

Changes among baseline-, intervention-, and sustainability-season data were assessed using generalized linear mixed-effects models with random effect for study sites. Negative binomial models were used for count variables to allow for overdispersion. Length of stay was log-transformed to achieve a normal distribution. We also analyzed each site individually to assess whether sustained improvements were the result of broad sustainability across all sites or whether they represented an aggregation of some sites that continued to improve while other sites actually worsened.

To address any bias introduced by the voluntary and incomplete participation of sites in the sustainability season, we planned a priori to conduct 3 additional analyses. First, we compared the characteristics of sites that did participate in the sustainability season with those that did not participate by using Chi-squared tests for differences in proportions and t tests for differences in means. Second, we determined whether the baseline-season process and outcome measures were different between sites that did and did not participate using descriptive statistics. Third, we assessed whether improvements between the baseline and intervention seasons were different between sites that did and did not participate using a linear mixed-effects model for normally distributed outcomes and generalized linear mixed-effects model with site-specific random effects for nonnormally distributed outcomes. All study outcomes were summarized in terms of model-adjusted means along with the corresponding 95% confidence intervals. All P values are 2-sided, and P < 0.05 was used to define statistical significance. Data analyses were conducted using SAS software (SAS Institute Inc., Cary, North Carolina) version 9.4.

To our knowledge, this is the first report of sustained improvements in care achieved through a multiinstitutional QI collaborative of community and academic hospitals focused on bronchiolitis care. We found that overall sites participating in a national bronchiolitis QI project sustained improvements in key bronchiolitis quality measures for 1 year following the project’s completion. For the aggregate group no measures worsened, and one measure, orders for intermittent pulse oximetry monitoring, continued to increase during the sustainability season. Furthermore, the sustained improvements were primarily the result of consistent sustained performance of each individual site, as opposed to averages wherein some sites worsened while others improved (Appendix 1a). These findings suggest that designing a collaborative approach, which provides an evidence-based best-practice toolkit while building the QI capacity of local interdisciplinary teams, can support performance gains that persist beyond the project’s active phase.

There are a number of possible reasons why improvements were sustained following the collaborative. The BQIP requirement for institutional leadership buy-in may have motivated accountability to local leaders in subsequent bronchiolitis seasons at each site. We suspect that culture change such as flattened hierarchies through multidisciplinary teams,¹⁴ which empowered nurse and respiratory therapy staff, may have facilitated consistent use of tools created locally. The synergy of interdisciplinary teams composed of physician, nurse, and respiratory therapy champions may have created accountability to perpetuate the previous year’s efforts.¹⁵ In addition, the sites adopted elements of the evidence-based toolkit, such as pathways,^16,17 forcing function tools^13,18 and order sets that limited management decision options and bronchodilator use contingent on respiratory scores,^9,19 which may have driven desired behaviors.

Moreover, the 2014 AAP CPG for the management of bronchiolitis,²⁰ released prior to the sustainability bronchiolitis season, may have underscored the key concepts of the collaborative. Similarly, national exposure of best practices for bronchiolitis management, including the 3 widespread Choosing Wisely recommendations related to bronchiolitis,²¹ might have been a compelling reason for sites to maintain their improvement efforts and contribute to secular trends toward decreasing interventions in bronchiolitis management nationally.³ Lastly, the mechanisms developed for local data collection may have created opportunities at each site to conduct ongoing evaluation of performance on key bronchiolitis quality measures through data-driven feedback systems.²² Our study highlights the need for additional research in order to understand why improvements are or are not sustained.

Even with substantial, sustained improvements in this initiative, further reduction in unnecessary care may be possible. Findings from previous studies suggest that even multifaceted QI interventions, including provider education, guidelines and use of respiratory scores, may only modestly reduce bronchodilators, steroids, and chest radiograph use.^8,13 To achieve continued improvements in bronchiolitis care, additional active efforts may be needed to develop new interventions that target root causes for areas of overuse at individual sites.

Future multiinstitutional collaboratives might benefit their participants if they include a focus on helping sites develop skills to ensure that local improvement activities continue after the collaborative phases are completed. Proactively scheduling intermittent check-ins with collaborative members to discuss experiences with both sustainability and ongoing improvement may be valuable and likely needs to be incorporated into the initial collaborative planning.

As these sustainability data represent a subset of 9 of the original 21 BQIP sites, there is concern for potential selection bias related to factors that could have motivated sites to participate in the sustainability season’s data collection and simultaneously influenced their performance. These concerns were mitigated to some extent through 3 specific analyses: finding limited differences in hospital characteristics, baseline performance in key bronchiolitis measures, and performance change from baseline to intervention seasons between sites that did and did not participate in the sustainability season.

Notably, sites that participated in the sustainability phase actually had lower baseline respiratory score use and fewer orders for intermittent pulse oximetry at baseline. Theoretically, if participation in the collaborative highlighted this disparity for these sites, it could have been a motivating factor for their continued participation and sustained performance across these measures. Similarly, sites that recognized their higher baseline performance through participation in the collaborative might have felt less motivation to participate in ongoing data collection during the sustainability season. Whether they might have also sustained, declined, or continued improving is not known. Additionally, the magnitude of improvement in the collaborative period might have also motivated ongoing participation during the sustainability phase. For example, although all sites improved in score use during the collaborative, sites participating in the sustainability season demonstrated significantly more improvement in these measures. Sites with a higher magnitude of improvement in collaborative measures might have more enthusiasm about the project, more commitment to the project activities, or feel a sense of obligation to respond to requests for additional data collection.

This work has several limitations. Selection bias may limit generalizability of the results, as sites that did not participate in the sustainability season may have had different results than those that did participate. It is unknown whether sites that regressed toward their baseline were deterred from participating in the sustainability season. The analyses that we were able to preform, however, suggest that the 2 groups were similar in their characteristics as well as in their baseline and improvement performance.

We have limited knowledge of the local improvement work that sites conducted between the completion of the collaborative and the sustainability season. Site-specific factors may have influenced improvement sustainability. For example, qualitative research with the original group found that team engagement had a quantitative association with better performance, but only for the bronchodilator use measure.²³ Sites were responsible for their own data collection, and despite attempts to centralize and standardize the process, data collection inconsistencies may have occurred. For instance, it is unknown how closely that orders for intermittent pulse oximetry correlate with intermittent use at the bedside. Lastly, the absence of a control group limits examination of the causal relationships of interventions and the influence of secular trends.

Improvements gained during the BQIP collaborative were sustained at 1 year following completion of the collaborative. These findings are encouraging, as national QI collaborative efforts are increasingly common. Our findings suggest that opportunities exist to even further reduce unnecessary care in the management of bronchiolitis. Such opportunities highlight the importance of integrating strategies to both measure sustainability and plan for ongoing independent local activities after completion of the collaborative. Future efforts should focus on supporting local sites to continue individual practice-improvement as they transition from collaborative to independent quality initiatives.

Acknowledgments

The authors thank the 21 hospitals that participated in the BQIP collaborative, and in particular the 9 hospital teams that contributed sustainability data for their ongoing dedication. There was no external funding for this manuscript.

Disclosure

The authors report no financial conflicts of interest.

Acute viral bronchiolitis is the most common cause of hospitalization for children less than 1 year of age.¹ Overuse of ineffective therapies has persisted despite the existence of the evidence-based American Academy of Pediatrics (AAP) clinical practice guideline (CPG), which recommends primarily supportive care.^2-8 Adherence to the AAP CPG recommendations for management of bronchiolitis improved significantly through the AAP’s Bronchiolitis Quality Improvement Project (BQIP), a 12-month, multiinstitutional collaborative of community and free-standing children’s hospitals.⁹ This subsequent study investigates if these improvements were sustained after completion of the formal 12-month project.

Published multiinstitutional bronchiolitis quality improvement (QI) work is limited to 1 study⁵ that describes the results of a single intervention season at academic medical centers. Multiyear bronchiolitis QI projects are limited to single-center studies, and results have been mixed.^5,6,8,10-13 One study¹¹ observed continued improvement in bronchodilator use in subsequent seasons, whereas a second study¹⁰ observed a return to baseline bronchodilator use in the following season. Mittal⁶ observed inconsistent improvements in key bronchiolitis measures during postintervention seasons.

Our specific aim was to assess the sustainability of improvements in bronchiolitis management at participating institutions 1 year following completion of the AAP BQIP collaborative.⁹ Because no studies demonstrate the most effective way to support long-term improvement through a QI collaborative, we hypothesized that the initial collaborative activities, which were designed to build the capacity of local interdisciplinary teams while providing standardized evidence-based care pathways, would lead to performance in the subsequent season at levels similar to or better than those observed during the active phase of the collaborative, without additional project interventions.

Study Design and Setting

This was a follow-up study of the AAP Quality Improvement Innovation Networks project entitled “A Quality Collaborative for Improving Hospital Compliance with the AAP Bronchiolitis Guideline” (BQIP).⁹ The AAP Institutional Review Board approved this project.

Twenty-one multidisciplinary, hospital-based teams participated in the BQIP collaborative and provided monthly data during the January through March bronchiolitis season. Teams submitted 2013 baseline data and 2014 intervention data. Nine sites provided 2015 sustainability data following the completion of the collaborative.

Participants

Hospital encounters with a primary diagnosis of acute viral bronchiolitis were eligible for inclusion among patients from 1 month to 2 years of age. Encounters were excluded for prematurity (<35 weeks gestational age), congenital heart disease, bronchopulmonary dysplasia, genetic, congenital or neuromuscular abnormalities, and pediatric intensive-care admission.

Data Collection

Hospital characteristics were collected, including hospital type (academic, community), bed size, location (urban, rural), hospital distributions of race/ethnicity and public payer, cases of bronchiolitis per year, presence of an electronic medical record and a pediatric respiratory therapist, and self-rated QI knowledge of the multidisciplinary team (very knowledgeable, knowledgeable, and somewhat knowledgeable). A trained member at each site collected data through structured chart review in baseline, intervention, and sustainability bronchiolitis seasons for January, February, and March. Site members reviewed the first 20 charts per month that met the inclusion criteria or all charts if there were fewer than 20 eligible encounters. Sites input data about key quality measures into the AAP’s Quality Improvement Data Aggregator, a web-based data repository.

Intervention

The BQIP project was designed as a virtual collaborative consisting of monthly education webinars about QI methods and bronchiolitis management, opportunities for collaboration via teleconference and e-mail listserv, and individual site-coaching by e-mail or telephone.⁹ A change package was shared with sites that included examples of evidence-based pathways, ordersets, a respiratory scoring tool, communication tools for parents and referring physicians, and slide sets for individual site education efforts. Following completion of the collaborative, written resources remained available to participants, although virtual collaboration ceased and no additional project interventions to promote sustainability were introduced.

Bronchiolitis Process and Outcome Measures

Process measures following admission included the following: severity assessment using a respiratory score, respiratory score use to assess response to bronchodilators, bronchodilator use, bronchodilator doses, steroid doses per patient encounter, chest radiographs per encounter, and presence of an order to transition to intermittent pulse oximetry monitoring. Outcome measures included length of stay and readmissions within 72 hours.

Analysis

Changes among baseline-, intervention-, and sustainability-season data were assessed using generalized linear mixed-effects models with random effect for study sites. Negative binomial models were used for count variables to allow for overdispersion. Length of stay was log-transformed to achieve a normal distribution. We also analyzed each site individually to assess whether sustained improvements were the result of broad sustainability across all sites or whether they represented an aggregation of some sites that continued to improve while other sites actually worsened.

To address any bias introduced by the voluntary and incomplete participation of sites in the sustainability season, we planned a priori to conduct 3 additional analyses. First, we compared the characteristics of sites that did participate in the sustainability season with those that did not participate by using Chi-squared tests for differences in proportions and t tests for differences in means. Second, we determined whether the baseline-season process and outcome measures were different between sites that did and did not participate using descriptive statistics. Third, we assessed whether improvements between the baseline and intervention seasons were different between sites that did and did not participate using a linear mixed-effects model for normally distributed outcomes and generalized linear mixed-effects model with site-specific random effects for nonnormally distributed outcomes. All study outcomes were summarized in terms of model-adjusted means along with the corresponding 95% confidence intervals. All P values are 2-sided, and P < 0.05 was used to define statistical significance. Data analyses were conducted using SAS software (SAS Institute Inc., Cary, North Carolina) version 9.4.

To our knowledge, this is the first report of sustained improvements in care achieved through a multiinstitutional QI collaborative of community and academic hospitals focused on bronchiolitis care. We found that overall sites participating in a national bronchiolitis QI project sustained improvements in key bronchiolitis quality measures for 1 year following the project’s completion. For the aggregate group no measures worsened, and one measure, orders for intermittent pulse oximetry monitoring, continued to increase during the sustainability season. Furthermore, the sustained improvements were primarily the result of consistent sustained performance of each individual site, as opposed to averages wherein some sites worsened while others improved (Appendix 1a). These findings suggest that designing a collaborative approach, which provides an evidence-based best-practice toolkit while building the QI capacity of local interdisciplinary teams, can support performance gains that persist beyond the project’s active phase.

There are a number of possible reasons why improvements were sustained following the collaborative. The BQIP requirement for institutional leadership buy-in may have motivated accountability to local leaders in subsequent bronchiolitis seasons at each site. We suspect that culture change such as flattened hierarchies through multidisciplinary teams,¹⁴ which empowered nurse and respiratory therapy staff, may have facilitated consistent use of tools created locally. The synergy of interdisciplinary teams composed of physician, nurse, and respiratory therapy champions may have created accountability to perpetuate the previous year’s efforts.¹⁵ In addition, the sites adopted elements of the evidence-based toolkit, such as pathways,^16,17 forcing function tools^13,18 and order sets that limited management decision options and bronchodilator use contingent on respiratory scores,^9,19 which may have driven desired behaviors.

Moreover, the 2014 AAP CPG for the management of bronchiolitis,²⁰ released prior to the sustainability bronchiolitis season, may have underscored the key concepts of the collaborative. Similarly, national exposure of best practices for bronchiolitis management, including the 3 widespread Choosing Wisely recommendations related to bronchiolitis,²¹ might have been a compelling reason for sites to maintain their improvement efforts and contribute to secular trends toward decreasing interventions in bronchiolitis management nationally.³ Lastly, the mechanisms developed for local data collection may have created opportunities at each site to conduct ongoing evaluation of performance on key bronchiolitis quality measures through data-driven feedback systems.²² Our study highlights the need for additional research in order to understand why improvements are or are not sustained.

Even with substantial, sustained improvements in this initiative, further reduction in unnecessary care may be possible. Findings from previous studies suggest that even multifaceted QI interventions, including provider education, guidelines and use of respiratory scores, may only modestly reduce bronchodilators, steroids, and chest radiograph use.^8,13 To achieve continued improvements in bronchiolitis care, additional active efforts may be needed to develop new interventions that target root causes for areas of overuse at individual sites.

Future multiinstitutional collaboratives might benefit their participants if they include a focus on helping sites develop skills to ensure that local improvement activities continue after the collaborative phases are completed. Proactively scheduling intermittent check-ins with collaborative members to discuss experiences with both sustainability and ongoing improvement may be valuable and likely needs to be incorporated into the initial collaborative planning.

As these sustainability data represent a subset of 9 of the original 21 BQIP sites, there is concern for potential selection bias related to factors that could have motivated sites to participate in the sustainability season’s data collection and simultaneously influenced their performance. These concerns were mitigated to some extent through 3 specific analyses: finding limited differences in hospital characteristics, baseline performance in key bronchiolitis measures, and performance change from baseline to intervention seasons between sites that did and did not participate in the sustainability season.

Notably, sites that participated in the sustainability phase actually had lower baseline respiratory score use and fewer orders for intermittent pulse oximetry at baseline. Theoretically, if participation in the collaborative highlighted this disparity for these sites, it could have been a motivating factor for their continued participation and sustained performance across these measures. Similarly, sites that recognized their higher baseline performance through participation in the collaborative might have felt less motivation to participate in ongoing data collection during the sustainability season. Whether they might have also sustained, declined, or continued improving is not known. Additionally, the magnitude of improvement in the collaborative period might have also motivated ongoing participation during the sustainability phase. For example, although all sites improved in score use during the collaborative, sites participating in the sustainability season demonstrated significantly more improvement in these measures. Sites with a higher magnitude of improvement in collaborative measures might have more enthusiasm about the project, more commitment to the project activities, or feel a sense of obligation to respond to requests for additional data collection.

This work has several limitations. Selection bias may limit generalizability of the results, as sites that did not participate in the sustainability season may have had different results than those that did participate. It is unknown whether sites that regressed toward their baseline were deterred from participating in the sustainability season. The analyses that we were able to preform, however, suggest that the 2 groups were similar in their characteristics as well as in their baseline and improvement performance.

We have limited knowledge of the local improvement work that sites conducted between the completion of the collaborative and the sustainability season. Site-specific factors may have influenced improvement sustainability. For example, qualitative research with the original group found that team engagement had a quantitative association with better performance, but only for the bronchodilator use measure.²³ Sites were responsible for their own data collection, and despite attempts to centralize and standardize the process, data collection inconsistencies may have occurred. For instance, it is unknown how closely that orders for intermittent pulse oximetry correlate with intermittent use at the bedside. Lastly, the absence of a control group limits examination of the causal relationships of interventions and the influence of secular trends.

Improvements gained during the BQIP collaborative were sustained at 1 year following completion of the collaborative. These findings are encouraging, as national QI collaborative efforts are increasingly common. Our findings suggest that opportunities exist to even further reduce unnecessary care in the management of bronchiolitis. Such opportunities highlight the importance of integrating strategies to both measure sustainability and plan for ongoing independent local activities after completion of the collaborative. Future efforts should focus on supporting local sites to continue individual practice-improvement as they transition from collaborative to independent quality initiatives.

Acknowledgments

The authors thank the 21 hospitals that participated in the BQIP collaborative, and in particular the 9 hospital teams that contributed sustainability data for their ongoing dedication. There was no external funding for this manuscript.

Disclosure

The authors report no financial conflicts of interest.