Brief Reports

In recent years, mounting evidence has emerged questioning the practice of using prolonged intravenous antibiotic therapy to treat certain serious bacterial infections in children, including complicated appendicitis, osteomyelitis, and complicated pneumonia. Historically, treatment of these conditions was often completed intravenously after hospital discharge using peripherally inserted central catheters (PICCs). Line infections, clots, mechanical problems, and general discomfort complicate PICCs, which led to their removal in more than 20% of children in one study.¹ Oral antibiotics avoid these complications and are less burdensome to families.² Recently, a series of multicenter studies showed no difference in outcomes between oral and postdischarge intravenous antibiotic therapy (PD-IV) for complicated appendicitis, osteomyelitis, and complicated pneumonia.^3-5

Despite a growing body of evidence suggesting that oral therapy ought to be the default treatment strategy rather than PD-IV, the extent to which practices have changed is unknown. In this study, we measured national trends in PD-IV use and variation by hospital for complicated appendicitis, osteomyelitis, and complicated pneumonia.

We performed a retrospective cohort study of children discharged from hospitals that contributed data to the Pediatric Health Information System (PHIS) database from January 2000 through December 2018. PHIS is an administrative database of children’s hospitals managed by the Children’s Hospital Association (Lenexa, Kansas) and contains deidentified patient-­level demographic data, discharge diagnosis and procedure codes, and detailed billing information, including medical supply charges.

The cohorts were defined using International Classification of Diseases, 9th and 10th Revisions (ICD-9 and ICD-10) discharge diagnosis and procedure codes. Patients admitted through September 2015 were identified using ICD-9 codes and patients admitted from October 2015 through December 2018 were identified using ICD-10 codes. The Centers for Medicaid & Medicare Services crosswalk was used to align ICD-9 and ICD-10 codes.⁶ Inclusion and exclusion criteria identifying cohorts of children hospitalized for complicated appendicitis, osteomyelitis, or complicated pneumonia were based on prior studies using the PHIS database.^3-5 These studies augmented the PHIS administrative dataset with local chart review to identify patients from 2009-2012 with the following inclusion and exclusion criteria: Patients with complicated appendicitis were defined by a diagnosis code for acute appendicitis and a procedure code for appendectomy, with postoperative length of stay lasting between 3 and 7 days. Patients with osteomyelitis had a diagnosis code of acute or unspecified osteomyelitis with a hospital length of stay between 2 and 14 days. Patients with complicated pneumonia were defined by a diagnosis code for both pneumonia and pleural effusion with one of these as the primary diagnosis. Patients were excluded if they were older than 18 years or if they were younger than 2 months for osteomyelitis and complicated pneumonia or younger than 3 years for appendicitis. For all three conditions, children with a complex chronic condition⁷ were excluded. Only the index encounter meeting inclusion and exclusion criteria for each patient was included. PD-IV therapy was defined using procedure codes and hospital charges during the index hospitalization. This definition for PD-IV therapy has been validated among children with complicated pneumonia, demonstrating positive and negative predictive values for PICC exposure of 85% and 99%, respectively.⁸

Trends in the percentage of patients receiving PD-IV were adjusted for age, race, insurance type, intensive care unit days, and hospital-level case mix index with use of Poisson regression. Calculated risk ratios represent the change in PD-IV across the entire 19-year study period for each condition (as opposed to an annual rate of change). An inflection point for each condition was identified using piecewise linear regression in which the line slope has one value up to a point in time and a second value after that point. The transition point is determined by maximizing model fit.

Some hospitals were added to the database throughout the time period and therefore did not have data for all years of the study. To account for the possibility of a group of high– or low–PD-IV use hospitals entering the cohort and biasing the overall trend, we performed a sensitivity analysis restricted to hospitals continuously contributing data to PHIS every year between 2004 (when a majority of hospitals joined PHIS) and 2018. Significance testing for individual hospital trends was conducted among continuously contributing hospitals, with each hospital tested in the above Poisson model independently.

For the most recent year of 2018, we reported the distribution of adjusted percentages of PD-IV at the individual hospital level. Only hospitals with at least five patients for a given condition are included in the percent PD-IV calculations for 2018. To examine the extent to which an individual hospital might be a low– or high–PD-IV user across conditions, we divided hospitals into quartiles based on PD-IV use for each condition in 2017-2018 and calculated the percent of hospitals in the lowest- and highest-use quartiles for all three conditions. All statistics were performed using Stata 15 (StataCorp).

Among 52 hospitals over a 19-year study period, there were 60,575 hospitalizations for complicated appendicitis, 24,753 hospitalizations for osteomyelitis, and 13,700 hospitalizations for complicated pneumonia. From 2000 to 2018, PD-IV decreased from 13% to 2% (RR, 0.15; 95% CI, 0.14-0.16) for complicated appendicitis, from 61% to 22% (RR, 0.41; 95% CI, 0.39-0.43) for osteomyelitis, and from 29% to 19% (RR, 0.63; 95% CI, 0.58-0.69) for complicated pneumonia (Figure 1). The inflection points occurred in 2009 for complicated appendicitis, 2009 for complicated pneumonia, and 2010 for osteomyelitis. The sensitivity analysis included 31 hospitals that contributed data to PHIS for every year between 2004-2018 and revealed similar findings for all three conditions: Complicated appendicitis had an RR of 0.15 (95% CI, 0.14-0.17), osteomyelitis had an RR of 0.34 (95% CI, 0.32-0.36), and complicated pneumonia had an RR of 0.55 (95% CI, 0.49-0.61). Most individual hospitals decreased PD-IV use (complicated appendicitis: 21 decreased, 8 no change, 2 increased; osteomyelitis: 25 decreased, 6 no change; complicated pneumonia: 14 decreased, 16 no change, 1 increased). While overall decreases in PD-IV were observed for all three conditions, considerable variation remained in 2018 for use of PD-IV (Figure 2), particularly for osteomyelitis (median, 18%; interquartile range [IQR] 9%-40%) and complicated pneumonia (median, 13%; IQR, 3%-30%). In 2017-2018, 1 out of 52 hospitals was in the lowest PD-IV–use quartile for all three conditions, and three hospitals were in the highest-use quartile for all three conditions.

Over a 19-year period, we observed a national decline in use of PD-IV for three serious and common bacterial infections. The decline in PD-IV is notable given that it has occurred largely in the absence of nationally coordinated guidelines or improvement efforts. Despite the overall declines, substantial variation in the use of PD-IV for these conditions persists across children’s hospitals.

The observed decrease in PD-IV use is a natural example of deimplementation, the abandonment of medical practices found to be harmful or ineffective.⁹ What is most compelling about the deimplementation of PD-IV for these infectious conditions is the seemingly organic motivation that propelled it. Studies of physician practice patterns for interventions that have undergone evidence reversals demonstrate that physicians might readily implement new interventions with an early evidence base but be less willing to deimplement them when more definitive evidence later questions their efficacy.¹⁰ Therefore, concerted improvement efforts backed by national guidelines are often needed to reduce the use of a widely accepted medical practice. For example, as evidence questioning the efficacy of steroid use in bronchiolitis mounted,¹¹ bronchiolitis guidelines recommended against steroid use¹² and a national quality improvement effort led to reductions in exposure to steroids among patients hospitalized with bronchiolitis.¹³ Complicated intra-abdominal infection guidelines acknowledge oral antibiotic therapy as an option,¹⁴ but no such national guidelines or improvement projects exist for osteomyelitis or complicated pneumonia PD-IV.

What is it about PD-IV for complicated appendicitis, osteomyelitis, and complicated pneumonia that fostered the observed organic deimplementation? Our findings that few hospitals were in the top or bottom quartile of PD-IV across all three conditions suggest that the impetus to decrease PD-IV was not likely the product of a broad hospital-wide practice shift. Most deimplementation frameworks suggest that successful deimplementation must be supported by high-quality evidence that the intervention is not only ineffective, but also harmful.¹⁵ In this case, the inflection point for osteomyelitis occurred in 2009, the same year that the first large multicenter study suggesting efficacy and decreased complications of early oral therapy for osteomyelitis was published.¹⁶ A direct link between a publication and inflection points for complicated pneumonia and appendicitis is less clear. It is possible that growth of the field of pediatric hospital medicine,¹⁷ with a stated emphasis on healthcare value,¹⁸ played a role. Greater understanding of the drivers and barriers to deimplementation in this and similar contexts will be important.

Our study has some important limitations. While inclusion and exclusion criteria were consistent over the study period, practice patterns (ie, length of stay in uncomplicated patients) change and could alter the case-mix of patients over time. Additionally, the PHIS database largely comprises children’s hospitals, and the trends we observed in PD-IV may not generalize to community settings.

The degree of deimplementation of PD-IV observed across children’s hospitals is impressive, but opportunity for further improvement likely remains. We found that marked hospital-­level variation in use of PD-IV still exists, with some hospitals almost never using PD-IV and others using it for most patients. While the ideal amount of PD-IV is probably not zero, a portion of the observed variation likely represents overuse of PD-IV. To reduce costs and complications associated with antibiotic therapy, national guidelines and a targeted national improvement collaborative may be necessary to achieve further reductions in PD-IV.

In recent years, mounting evidence has emerged questioning the practice of using prolonged intravenous antibiotic therapy to treat certain serious bacterial infections in children, including complicated appendicitis, osteomyelitis, and complicated pneumonia. Historically, treatment of these conditions was often completed intravenously after hospital discharge using peripherally inserted central catheters (PICCs). Line infections, clots, mechanical problems, and general discomfort complicate PICCs, which led to their removal in more than 20% of children in one study.¹ Oral antibiotics avoid these complications and are less burdensome to families.² Recently, a series of multicenter studies showed no difference in outcomes between oral and postdischarge intravenous antibiotic therapy (PD-IV) for complicated appendicitis, osteomyelitis, and complicated pneumonia.^3-5

Despite a growing body of evidence suggesting that oral therapy ought to be the default treatment strategy rather than PD-IV, the extent to which practices have changed is unknown. In this study, we measured national trends in PD-IV use and variation by hospital for complicated appendicitis, osteomyelitis, and complicated pneumonia.

We performed a retrospective cohort study of children discharged from hospitals that contributed data to the Pediatric Health Information System (PHIS) database from January 2000 through December 2018. PHIS is an administrative database of children’s hospitals managed by the Children’s Hospital Association (Lenexa, Kansas) and contains deidentified patient-­level demographic data, discharge diagnosis and procedure codes, and detailed billing information, including medical supply charges.

The cohorts were defined using International Classification of Diseases, 9th and 10th Revisions (ICD-9 and ICD-10) discharge diagnosis and procedure codes. Patients admitted through September 2015 were identified using ICD-9 codes and patients admitted from October 2015 through December 2018 were identified using ICD-10 codes. The Centers for Medicaid & Medicare Services crosswalk was used to align ICD-9 and ICD-10 codes.⁶ Inclusion and exclusion criteria identifying cohorts of children hospitalized for complicated appendicitis, osteomyelitis, or complicated pneumonia were based on prior studies using the PHIS database.^3-5 These studies augmented the PHIS administrative dataset with local chart review to identify patients from 2009-2012 with the following inclusion and exclusion criteria: Patients with complicated appendicitis were defined by a diagnosis code for acute appendicitis and a procedure code for appendectomy, with postoperative length of stay lasting between 3 and 7 days. Patients with osteomyelitis had a diagnosis code of acute or unspecified osteomyelitis with a hospital length of stay between 2 and 14 days. Patients with complicated pneumonia were defined by a diagnosis code for both pneumonia and pleural effusion with one of these as the primary diagnosis. Patients were excluded if they were older than 18 years or if they were younger than 2 months for osteomyelitis and complicated pneumonia or younger than 3 years for appendicitis. For all three conditions, children with a complex chronic condition⁷ were excluded. Only the index encounter meeting inclusion and exclusion criteria for each patient was included. PD-IV therapy was defined using procedure codes and hospital charges during the index hospitalization. This definition for PD-IV therapy has been validated among children with complicated pneumonia, demonstrating positive and negative predictive values for PICC exposure of 85% and 99%, respectively.⁸

Trends in the percentage of patients receiving PD-IV were adjusted for age, race, insurance type, intensive care unit days, and hospital-level case mix index with use of Poisson regression. Calculated risk ratios represent the change in PD-IV across the entire 19-year study period for each condition (as opposed to an annual rate of change). An inflection point for each condition was identified using piecewise linear regression in which the line slope has one value up to a point in time and a second value after that point. The transition point is determined by maximizing model fit.

Some hospitals were added to the database throughout the time period and therefore did not have data for all years of the study. To account for the possibility of a group of high– or low–PD-IV use hospitals entering the cohort and biasing the overall trend, we performed a sensitivity analysis restricted to hospitals continuously contributing data to PHIS every year between 2004 (when a majority of hospitals joined PHIS) and 2018. Significance testing for individual hospital trends was conducted among continuously contributing hospitals, with each hospital tested in the above Poisson model independently.

For the most recent year of 2018, we reported the distribution of adjusted percentages of PD-IV at the individual hospital level. Only hospitals with at least five patients for a given condition are included in the percent PD-IV calculations for 2018. To examine the extent to which an individual hospital might be a low– or high–PD-IV user across conditions, we divided hospitals into quartiles based on PD-IV use for each condition in 2017-2018 and calculated the percent of hospitals in the lowest- and highest-use quartiles for all three conditions. All statistics were performed using Stata 15 (StataCorp).

Among 52 hospitals over a 19-year study period, there were 60,575 hospitalizations for complicated appendicitis, 24,753 hospitalizations for osteomyelitis, and 13,700 hospitalizations for complicated pneumonia. From 2000 to 2018, PD-IV decreased from 13% to 2% (RR, 0.15; 95% CI, 0.14-0.16) for complicated appendicitis, from 61% to 22% (RR, 0.41; 95% CI, 0.39-0.43) for osteomyelitis, and from 29% to 19% (RR, 0.63; 95% CI, 0.58-0.69) for complicated pneumonia (Figure 1). The inflection points occurred in 2009 for complicated appendicitis, 2009 for complicated pneumonia, and 2010 for osteomyelitis. The sensitivity analysis included 31 hospitals that contributed data to PHIS for every year between 2004-2018 and revealed similar findings for all three conditions: Complicated appendicitis had an RR of 0.15 (95% CI, 0.14-0.17), osteomyelitis had an RR of 0.34 (95% CI, 0.32-0.36), and complicated pneumonia had an RR of 0.55 (95% CI, 0.49-0.61). Most individual hospitals decreased PD-IV use (complicated appendicitis: 21 decreased, 8 no change, 2 increased; osteomyelitis: 25 decreased, 6 no change; complicated pneumonia: 14 decreased, 16 no change, 1 increased). While overall decreases in PD-IV were observed for all three conditions, considerable variation remained in 2018 for use of PD-IV (Figure 2), particularly for osteomyelitis (median, 18%; interquartile range [IQR] 9%-40%) and complicated pneumonia (median, 13%; IQR, 3%-30%). In 2017-2018, 1 out of 52 hospitals was in the lowest PD-IV–use quartile for all three conditions, and three hospitals were in the highest-use quartile for all three conditions.

Over a 19-year period, we observed a national decline in use of PD-IV for three serious and common bacterial infections. The decline in PD-IV is notable given that it has occurred largely in the absence of nationally coordinated guidelines or improvement efforts. Despite the overall declines, substantial variation in the use of PD-IV for these conditions persists across children’s hospitals.

The observed decrease in PD-IV use is a natural example of deimplementation, the abandonment of medical practices found to be harmful or ineffective.⁹ What is most compelling about the deimplementation of PD-IV for these infectious conditions is the seemingly organic motivation that propelled it. Studies of physician practice patterns for interventions that have undergone evidence reversals demonstrate that physicians might readily implement new interventions with an early evidence base but be less willing to deimplement them when more definitive evidence later questions their efficacy.¹⁰ Therefore, concerted improvement efforts backed by national guidelines are often needed to reduce the use of a widely accepted medical practice. For example, as evidence questioning the efficacy of steroid use in bronchiolitis mounted,¹¹ bronchiolitis guidelines recommended against steroid use¹² and a national quality improvement effort led to reductions in exposure to steroids among patients hospitalized with bronchiolitis.¹³ Complicated intra-abdominal infection guidelines acknowledge oral antibiotic therapy as an option,¹⁴ but no such national guidelines or improvement projects exist for osteomyelitis or complicated pneumonia PD-IV.

What is it about PD-IV for complicated appendicitis, osteomyelitis, and complicated pneumonia that fostered the observed organic deimplementation? Our findings that few hospitals were in the top or bottom quartile of PD-IV across all three conditions suggest that the impetus to decrease PD-IV was not likely the product of a broad hospital-wide practice shift. Most deimplementation frameworks suggest that successful deimplementation must be supported by high-quality evidence that the intervention is not only ineffective, but also harmful.¹⁵ In this case, the inflection point for osteomyelitis occurred in 2009, the same year that the first large multicenter study suggesting efficacy and decreased complications of early oral therapy for osteomyelitis was published.¹⁶ A direct link between a publication and inflection points for complicated pneumonia and appendicitis is less clear. It is possible that growth of the field of pediatric hospital medicine,¹⁷ with a stated emphasis on healthcare value,¹⁸ played a role. Greater understanding of the drivers and barriers to deimplementation in this and similar contexts will be important.

Our study has some important limitations. While inclusion and exclusion criteria were consistent over the study period, practice patterns (ie, length of stay in uncomplicated patients) change and could alter the case-mix of patients over time. Additionally, the PHIS database largely comprises children’s hospitals, and the trends we observed in PD-IV may not generalize to community settings.

The degree of deimplementation of PD-IV observed across children’s hospitals is impressive, but opportunity for further improvement likely remains. We found that marked hospital-­level variation in use of PD-IV still exists, with some hospitals almost never using PD-IV and others using it for most patients. While the ideal amount of PD-IV is probably not zero, a portion of the observed variation likely represents overuse of PD-IV. To reduce costs and complications associated with antibiotic therapy, national guidelines and a targeted national improvement collaborative may be necessary to achieve further reductions in PD-IV.

In recent years, mounting evidence has emerged questioning the practice of using prolonged intravenous antibiotic therapy to treat certain serious bacterial infections in children, including complicated appendicitis, osteomyelitis, and complicated pneumonia. Historically, treatment of these conditions was often completed intravenously after hospital discharge using peripherally inserted central catheters (PICCs). Line infections, clots, mechanical problems, and general discomfort complicate PICCs, which led to their removal in more than 20% of children in one study.¹ Oral antibiotics avoid these complications and are less burdensome to families.² Recently, a series of multicenter studies showed no difference in outcomes between oral and postdischarge intravenous antibiotic therapy (PD-IV) for complicated appendicitis, osteomyelitis, and complicated pneumonia.^3-5

Despite a growing body of evidence suggesting that oral therapy ought to be the default treatment strategy rather than PD-IV, the extent to which practices have changed is unknown. In this study, we measured national trends in PD-IV use and variation by hospital for complicated appendicitis, osteomyelitis, and complicated pneumonia.

We performed a retrospective cohort study of children discharged from hospitals that contributed data to the Pediatric Health Information System (PHIS) database from January 2000 through December 2018. PHIS is an administrative database of children’s hospitals managed by the Children’s Hospital Association (Lenexa, Kansas) and contains deidentified patient-­level demographic data, discharge diagnosis and procedure codes, and detailed billing information, including medical supply charges.

The cohorts were defined using International Classification of Diseases, 9th and 10th Revisions (ICD-9 and ICD-10) discharge diagnosis and procedure codes. Patients admitted through September 2015 were identified using ICD-9 codes and patients admitted from October 2015 through December 2018 were identified using ICD-10 codes. The Centers for Medicaid & Medicare Services crosswalk was used to align ICD-9 and ICD-10 codes.⁶ Inclusion and exclusion criteria identifying cohorts of children hospitalized for complicated appendicitis, osteomyelitis, or complicated pneumonia were based on prior studies using the PHIS database.^3-5 These studies augmented the PHIS administrative dataset with local chart review to identify patients from 2009-2012 with the following inclusion and exclusion criteria: Patients with complicated appendicitis were defined by a diagnosis code for acute appendicitis and a procedure code for appendectomy, with postoperative length of stay lasting between 3 and 7 days. Patients with osteomyelitis had a diagnosis code of acute or unspecified osteomyelitis with a hospital length of stay between 2 and 14 days. Patients with complicated pneumonia were defined by a diagnosis code for both pneumonia and pleural effusion with one of these as the primary diagnosis. Patients were excluded if they were older than 18 years or if they were younger than 2 months for osteomyelitis and complicated pneumonia or younger than 3 years for appendicitis. For all three conditions, children with a complex chronic condition⁷ were excluded. Only the index encounter meeting inclusion and exclusion criteria for each patient was included. PD-IV therapy was defined using procedure codes and hospital charges during the index hospitalization. This definition for PD-IV therapy has been validated among children with complicated pneumonia, demonstrating positive and negative predictive values for PICC exposure of 85% and 99%, respectively.⁸

Trends in the percentage of patients receiving PD-IV were adjusted for age, race, insurance type, intensive care unit days, and hospital-level case mix index with use of Poisson regression. Calculated risk ratios represent the change in PD-IV across the entire 19-year study period for each condition (as opposed to an annual rate of change). An inflection point for each condition was identified using piecewise linear regression in which the line slope has one value up to a point in time and a second value after that point. The transition point is determined by maximizing model fit.

Some hospitals were added to the database throughout the time period and therefore did not have data for all years of the study. To account for the possibility of a group of high– or low–PD-IV use hospitals entering the cohort and biasing the overall trend, we performed a sensitivity analysis restricted to hospitals continuously contributing data to PHIS every year between 2004 (when a majority of hospitals joined PHIS) and 2018. Significance testing for individual hospital trends was conducted among continuously contributing hospitals, with each hospital tested in the above Poisson model independently.

For the most recent year of 2018, we reported the distribution of adjusted percentages of PD-IV at the individual hospital level. Only hospitals with at least five patients for a given condition are included in the percent PD-IV calculations for 2018. To examine the extent to which an individual hospital might be a low– or high–PD-IV user across conditions, we divided hospitals into quartiles based on PD-IV use for each condition in 2017-2018 and calculated the percent of hospitals in the lowest- and highest-use quartiles for all three conditions. All statistics were performed using Stata 15 (StataCorp).

Among 52 hospitals over a 19-year study period, there were 60,575 hospitalizations for complicated appendicitis, 24,753 hospitalizations for osteomyelitis, and 13,700 hospitalizations for complicated pneumonia. From 2000 to 2018, PD-IV decreased from 13% to 2% (RR, 0.15; 95% CI, 0.14-0.16) for complicated appendicitis, from 61% to 22% (RR, 0.41; 95% CI, 0.39-0.43) for osteomyelitis, and from 29% to 19% (RR, 0.63; 95% CI, 0.58-0.69) for complicated pneumonia (Figure 1). The inflection points occurred in 2009 for complicated appendicitis, 2009 for complicated pneumonia, and 2010 for osteomyelitis. The sensitivity analysis included 31 hospitals that contributed data to PHIS for every year between 2004-2018 and revealed similar findings for all three conditions: Complicated appendicitis had an RR of 0.15 (95% CI, 0.14-0.17), osteomyelitis had an RR of 0.34 (95% CI, 0.32-0.36), and complicated pneumonia had an RR of 0.55 (95% CI, 0.49-0.61). Most individual hospitals decreased PD-IV use (complicated appendicitis: 21 decreased, 8 no change, 2 increased; osteomyelitis: 25 decreased, 6 no change; complicated pneumonia: 14 decreased, 16 no change, 1 increased). While overall decreases in PD-IV were observed for all three conditions, considerable variation remained in 2018 for use of PD-IV (Figure 2), particularly for osteomyelitis (median, 18%; interquartile range [IQR] 9%-40%) and complicated pneumonia (median, 13%; IQR, 3%-30%). In 2017-2018, 1 out of 52 hospitals was in the lowest PD-IV–use quartile for all three conditions, and three hospitals were in the highest-use quartile for all three conditions.

Over a 19-year period, we observed a national decline in use of PD-IV for three serious and common bacterial infections. The decline in PD-IV is notable given that it has occurred largely in the absence of nationally coordinated guidelines or improvement efforts. Despite the overall declines, substantial variation in the use of PD-IV for these conditions persists across children’s hospitals.

The observed decrease in PD-IV use is a natural example of deimplementation, the abandonment of medical practices found to be harmful or ineffective.⁹ What is most compelling about the deimplementation of PD-IV for these infectious conditions is the seemingly organic motivation that propelled it. Studies of physician practice patterns for interventions that have undergone evidence reversals demonstrate that physicians might readily implement new interventions with an early evidence base but be less willing to deimplement them when more definitive evidence later questions their efficacy.¹⁰ Therefore, concerted improvement efforts backed by national guidelines are often needed to reduce the use of a widely accepted medical practice. For example, as evidence questioning the efficacy of steroid use in bronchiolitis mounted,¹¹ bronchiolitis guidelines recommended against steroid use¹² and a national quality improvement effort led to reductions in exposure to steroids among patients hospitalized with bronchiolitis.¹³ Complicated intra-abdominal infection guidelines acknowledge oral antibiotic therapy as an option,¹⁴ but no such national guidelines or improvement projects exist for osteomyelitis or complicated pneumonia PD-IV.

What is it about PD-IV for complicated appendicitis, osteomyelitis, and complicated pneumonia that fostered the observed organic deimplementation? Our findings that few hospitals were in the top or bottom quartile of PD-IV across all three conditions suggest that the impetus to decrease PD-IV was not likely the product of a broad hospital-wide practice shift. Most deimplementation frameworks suggest that successful deimplementation must be supported by high-quality evidence that the intervention is not only ineffective, but also harmful.¹⁵ In this case, the inflection point for osteomyelitis occurred in 2009, the same year that the first large multicenter study suggesting efficacy and decreased complications of early oral therapy for osteomyelitis was published.¹⁶ A direct link between a publication and inflection points for complicated pneumonia and appendicitis is less clear. It is possible that growth of the field of pediatric hospital medicine,¹⁷ with a stated emphasis on healthcare value,¹⁸ played a role. Greater understanding of the drivers and barriers to deimplementation in this and similar contexts will be important.

Our study has some important limitations. While inclusion and exclusion criteria were consistent over the study period, practice patterns (ie, length of stay in uncomplicated patients) change and could alter the case-mix of patients over time. Additionally, the PHIS database largely comprises children’s hospitals, and the trends we observed in PD-IV may not generalize to community settings.

The degree of deimplementation of PD-IV observed across children’s hospitals is impressive, but opportunity for further improvement likely remains. We found that marked hospital-­level variation in use of PD-IV still exists, with some hospitals almost never using PD-IV and others using it for most patients. While the ideal amount of PD-IV is probably not zero, a portion of the observed variation likely represents overuse of PD-IV. To reduce costs and complications associated with antibiotic therapy, national guidelines and a targeted national improvement collaborative may be necessary to achieve further reductions in PD-IV.

Delirium is the most common postsurgical complication for older adults, with incidence of 15%-54%, depending on surgery type.¹ Increasing numbers of older adults are undergoing surgery²; and those who develop delirium experience negative consequences including longer lengths of stay, higher likelihood of institutional discharge, and increased morbidity and mortality.³ The American Geriatrics Society Expert Panel on Postoperative Delirium in Older Adults and the European Society of Anaesthesiology⁴ recommend routine screening for delirium in those at risk.

Ultrabrief screens are designed to rule out delirium quickly and identify a subset of patients who require further testing.⁵ Our group, and others, have previously published ultrabrief screens for the general medicine, nonsurgical population and for patients with dementia.^5,6 The UB-2 is an ultrabrief screen consisting of “Months of the year backward” (MOYB) and “What day of the week is it?”, which has a sensitivity of 93% and specificity of 64% in hospitalized older adults and takes less than 40 seconds to administer.⁵ However, no such screens for delirium have been developed for the group with relatively high cognitive and physical functioning undergoing scheduled major surgery in which delirium may present differently. Thus, the purpose of this study was to develop an ultrabrief screen for postoperative delirium using data from a large study of delirium in cognitively intact, older adults undergoing scheduled major noncardiac surgery.

We performed a secondary data analysis on 560 patients enrolled between June 18, 2010, and August 8, 2013, in the Successful Aging After Elective Surgery (SAGES) study,⁷ an ongoing prospective cohort study of older adults undergoing major elective surgeries (eg, total hip or knee replacement; lumbar, cervical, or sacral laminectomy; lower extremity arterial bypass surgery; open abdominal aortic aneurysm repair; and open or laparoscopic colectomy). Exclusion criteria included evidence of dementia, delirium, prior hospitalization within 3 months, legal blindness, severe deafness, terminal condition, history of schizophrenia or psychosis, and history of alcohol abuse or withdrawal. The Institutional Review Boards of Beth Israel Deaconess Medical Center, Brigham and Women’s Hospital, and Hebrew SeniorLife, all in Boston, Massachusetts, approved the study.

SAGES Delirium Assessment and Additional Variables

The presence or absence of delirium was based on daily in-hospital assessments by trained research staff using the Confusion Assessment Method (CAM)⁸ long form. The Delirium Symptom Interview (DSI)⁹ and information related to acute changes in mental status were also included as provided by nursing staff and/or family. Delirium severity was determined using the CAM-S.¹⁰ Participants in The SAGES Study had an initial baseline, presurgical assessment in their homes. Cognitive and physical functioning, depression, comorbidities, laboratory, and self-reported demographic data were collected.

Statistical Analyses

We included CAM delirium data from postoperative days (POD) 1 and 2 for each participant, if available; postoperative day 0 was not included because of potential residual anesthetic effects. We chose these days because most delirium began on POD1 or 2, and patients started being discharged on POD3. We considered all one-, two-, and three-item combinations of the 12 cognitive items of the 3D-CAM¹¹ because of their demonstrated high information content for CAM diagnostic features per Item Response Theory.¹² There were 12 possible one-item screens, 66 two-item screens, and 220 three-item screens. Sensitivity, specificity, and 95% confidence intervals for each were compared with CAM delirium determination. An ideal ultrabrief screen for delirium has high sensitivity with moderate specificity; general guidelines considered based on investigator consensus included screens with a sensitivity higher than 0.90 and specificity greater than 0.70. Because these screens are used to quickly rule out delirium, we also present the percent positive screen among the entire population (whether delirium is present or not). Screens with a positive screen rate of more than 50% are unlikely to be helpful in ruling out delirium quickly in a large enough fraction of the population. We also required that in multiple item screens, no two items should assess the same CAM feature. For instance, we would eliminate a two-item screen with MOYB and four-digit span since both items measure CAM Feature 2 (Inattention). Finally, we evaluated screen performance separately on POD1 and POD2. Switching screens by POD can be confusing, so we chose a single best screen that retained excellent performance over both days. Data analyses used SAS version 9.4 (SAS Institute, Cary, North Carolina).

The dataset included 560 adults who had an average age of 76.6 years (SD = 5.2), were 58% women, and were highly educated (15.0 years; SD = 2.9; Table). Postoperative delirium occurred during one or more days in 134 individuals (24%). A total of 1,100 delirium assessments were used, with 113 that were CAM positive (10.3%). For POD1, we used 551 assessments, 61 of which were positive (11.1%); for POD2, 549 assessments were used, with 51 positive (9.3%). Appendix Tables present the positive screen rates, sensitivities, specificities, and 95% confidence intervals of all 12 one-item screens and the 12 best performing two- and three-item screens in order of decreasing sensitivity.

The best ultrabrief screen from POD1 included the following three items: “Does the patient report feeling confused?”, MOYB, and “Does the patient appear sleepy?”, with a sensitivity of 0.95 (95% CI, 0.87-0.99) and specificity of 0.73 (95% CI, 0.69-0.77). The same combination of items has a sensitivity of 0.88 (95% CI, 0.77-0.96) and a specificity of 0.70 (95% CI, 0.66-0.74) on POD2. When POD1 and POD2 are combined, the sensitivity is 0.92 (95% CI, 0.85-0.96) and specificity is 0.72 (95% CI, 0.69-0.74). We consider this to be our best screen overall.

We identified a three-item screen for delirium after elective surgery consisting of “Does the patient report feeling confused?”, MOYB, and “Does the patient appear sleepy?” In our own prior work, we identified a two-item screen consisting of MOYB and “What is the day of the week?” as the best ultrabrief screen for delirium in general medicine populations (termed the “UB-2”)⁵ and a subsequent screen for patients with delirium superimposed on dementia (DSD) including “What type of place is this?”, Days of the Week Backward, and “Does the patient appear sleepy?”⁶ All three contain a test of attention (a cardinal feature of delirium) and a test of orientation, although the specific test for that varies. Both the surgical and DSD screens include “Does the patient appear sleepy?”, which addresses a reduced level of consciousness. This might be particularly important in the postoperative setting because of residual effects of anesthesia and/or postoperative analgesic medications contributing to delirium. Work done by others confirms our current findings, which is that MOYB is the best single item for most groups. Belleli et al¹³ and Han et al¹⁴ included MOYB as the single attentional item in the 4AT and B-CAM, respectively. The Nu-DESC has been used as a screen in surgical patients; however, it involves only nursing observations and no direct questioning of the patient.¹⁵

The Figure describes how our “best screen” could be integrated into clinical care. One or more “positive” or incorrect responses on these three items constitutes a positive screen that should be further evaluated with the CAM or 3D-CAM. If all three items are correct or negative, this effectively rules out delirium; however, continued periodic screening on a daily (or per shift) basis is indicated. On repeat testing, if any of the previously negative or correct items becomes positive or incorrect, this would be evidence for Acute Change, CAM Feature 1. Finally, it should be noted that, if all three items in our best screen are positive, full CAM criteria for delirium diagnosis are met within the screen itself, and no further testing is required. We envision this process being facilitated by use of an app-based program that generates optimal screening items based on patient and setting characteristics.

There are several limitations that must be noted. First, our three-item screen may not generalize to nonsurgical candidates or those undergoing emergent surgery and should be tested in these groups. Second, the SAGES sample is relatively homogenous with respect to racial and ethnic diversity and was highly educated with little functional impairment and no dementia. Therefore, results may not be generalizable to populations with lower educational attainment and/or preexisting mental and physical disabilities. A third limitation is that screen items were included in the reference standard delirium assessment, leading to a potential bias toward increased sensitivity. Finally, all screens were derived from secondary data analysis and further research will be needed to prospectively validate the results. Despite these limitations, this study has several strengths including the use of a well-characterized surgical population and a rigorous approach to delirium measurement. It is one of the first studies to identify a screening tool targeted to identifying delirium in postoperative older adults.

Future research should prospectively validate our screening tool and test its implementation in a real-world clinical environment. As part of this process, clinicians should document barriers and facilitators to widespread implementation. The goal of such screens is to facilitate early identification of postoperative delirium, which will allow timely intervention to address underlying causes and prevent adverse consequences, thereby improving the outcomes of vulnerable older surgical patients.

Delirium is the most common postsurgical complication for older adults, with incidence of 15%-54%, depending on surgery type.¹ Increasing numbers of older adults are undergoing surgery²; and those who develop delirium experience negative consequences including longer lengths of stay, higher likelihood of institutional discharge, and increased morbidity and mortality.³ The American Geriatrics Society Expert Panel on Postoperative Delirium in Older Adults and the European Society of Anaesthesiology⁴ recommend routine screening for delirium in those at risk.

Ultrabrief screens are designed to rule out delirium quickly and identify a subset of patients who require further testing.⁵ Our group, and others, have previously published ultrabrief screens for the general medicine, nonsurgical population and for patients with dementia.^5,6 The UB-2 is an ultrabrief screen consisting of “Months of the year backward” (MOYB) and “What day of the week is it?”, which has a sensitivity of 93% and specificity of 64% in hospitalized older adults and takes less than 40 seconds to administer.⁵ However, no such screens for delirium have been developed for the group with relatively high cognitive and physical functioning undergoing scheduled major surgery in which delirium may present differently. Thus, the purpose of this study was to develop an ultrabrief screen for postoperative delirium using data from a large study of delirium in cognitively intact, older adults undergoing scheduled major noncardiac surgery.

We performed a secondary data analysis on 560 patients enrolled between June 18, 2010, and August 8, 2013, in the Successful Aging After Elective Surgery (SAGES) study,⁷ an ongoing prospective cohort study of older adults undergoing major elective surgeries (eg, total hip or knee replacement; lumbar, cervical, or sacral laminectomy; lower extremity arterial bypass surgery; open abdominal aortic aneurysm repair; and open or laparoscopic colectomy). Exclusion criteria included evidence of dementia, delirium, prior hospitalization within 3 months, legal blindness, severe deafness, terminal condition, history of schizophrenia or psychosis, and history of alcohol abuse or withdrawal. The Institutional Review Boards of Beth Israel Deaconess Medical Center, Brigham and Women’s Hospital, and Hebrew SeniorLife, all in Boston, Massachusetts, approved the study.

SAGES Delirium Assessment and Additional Variables

The presence or absence of delirium was based on daily in-hospital assessments by trained research staff using the Confusion Assessment Method (CAM)⁸ long form. The Delirium Symptom Interview (DSI)⁹ and information related to acute changes in mental status were also included as provided by nursing staff and/or family. Delirium severity was determined using the CAM-S.¹⁰ Participants in The SAGES Study had an initial baseline, presurgical assessment in their homes. Cognitive and physical functioning, depression, comorbidities, laboratory, and self-reported demographic data were collected.

Statistical Analyses

We included CAM delirium data from postoperative days (POD) 1 and 2 for each participant, if available; postoperative day 0 was not included because of potential residual anesthetic effects. We chose these days because most delirium began on POD1 or 2, and patients started being discharged on POD3. We considered all one-, two-, and three-item combinations of the 12 cognitive items of the 3D-CAM¹¹ because of their demonstrated high information content for CAM diagnostic features per Item Response Theory.¹² There were 12 possible one-item screens, 66 two-item screens, and 220 three-item screens. Sensitivity, specificity, and 95% confidence intervals for each were compared with CAM delirium determination. An ideal ultrabrief screen for delirium has high sensitivity with moderate specificity; general guidelines considered based on investigator consensus included screens with a sensitivity higher than 0.90 and specificity greater than 0.70. Because these screens are used to quickly rule out delirium, we also present the percent positive screen among the entire population (whether delirium is present or not). Screens with a positive screen rate of more than 50% are unlikely to be helpful in ruling out delirium quickly in a large enough fraction of the population. We also required that in multiple item screens, no two items should assess the same CAM feature. For instance, we would eliminate a two-item screen with MOYB and four-digit span since both items measure CAM Feature 2 (Inattention). Finally, we evaluated screen performance separately on POD1 and POD2. Switching screens by POD can be confusing, so we chose a single best screen that retained excellent performance over both days. Data analyses used SAS version 9.4 (SAS Institute, Cary, North Carolina).

The dataset included 560 adults who had an average age of 76.6 years (SD = 5.2), were 58% women, and were highly educated (15.0 years; SD = 2.9; Table). Postoperative delirium occurred during one or more days in 134 individuals (24%). A total of 1,100 delirium assessments were used, with 113 that were CAM positive (10.3%). For POD1, we used 551 assessments, 61 of which were positive (11.1%); for POD2, 549 assessments were used, with 51 positive (9.3%). Appendix Tables present the positive screen rates, sensitivities, specificities, and 95% confidence intervals of all 12 one-item screens and the 12 best performing two- and three-item screens in order of decreasing sensitivity.

The best ultrabrief screen from POD1 included the following three items: “Does the patient report feeling confused?”, MOYB, and “Does the patient appear sleepy?”, with a sensitivity of 0.95 (95% CI, 0.87-0.99) and specificity of 0.73 (95% CI, 0.69-0.77). The same combination of items has a sensitivity of 0.88 (95% CI, 0.77-0.96) and a specificity of 0.70 (95% CI, 0.66-0.74) on POD2. When POD1 and POD2 are combined, the sensitivity is 0.92 (95% CI, 0.85-0.96) and specificity is 0.72 (95% CI, 0.69-0.74). We consider this to be our best screen overall.

We identified a three-item screen for delirium after elective surgery consisting of “Does the patient report feeling confused?”, MOYB, and “Does the patient appear sleepy?” In our own prior work, we identified a two-item screen consisting of MOYB and “What is the day of the week?” as the best ultrabrief screen for delirium in general medicine populations (termed the “UB-2”)⁵ and a subsequent screen for patients with delirium superimposed on dementia (DSD) including “What type of place is this?”, Days of the Week Backward, and “Does the patient appear sleepy?”⁶ All three contain a test of attention (a cardinal feature of delirium) and a test of orientation, although the specific test for that varies. Both the surgical and DSD screens include “Does the patient appear sleepy?”, which addresses a reduced level of consciousness. This might be particularly important in the postoperative setting because of residual effects of anesthesia and/or postoperative analgesic medications contributing to delirium. Work done by others confirms our current findings, which is that MOYB is the best single item for most groups. Belleli et al¹³ and Han et al¹⁴ included MOYB as the single attentional item in the 4AT and B-CAM, respectively. The Nu-DESC has been used as a screen in surgical patients; however, it involves only nursing observations and no direct questioning of the patient.¹⁵

The Figure describes how our “best screen” could be integrated into clinical care. One or more “positive” or incorrect responses on these three items constitutes a positive screen that should be further evaluated with the CAM or 3D-CAM. If all three items are correct or negative, this effectively rules out delirium; however, continued periodic screening on a daily (or per shift) basis is indicated. On repeat testing, if any of the previously negative or correct items becomes positive or incorrect, this would be evidence for Acute Change, CAM Feature 1. Finally, it should be noted that, if all three items in our best screen are positive, full CAM criteria for delirium diagnosis are met within the screen itself, and no further testing is required. We envision this process being facilitated by use of an app-based program that generates optimal screening items based on patient and setting characteristics.

There are several limitations that must be noted. First, our three-item screen may not generalize to nonsurgical candidates or those undergoing emergent surgery and should be tested in these groups. Second, the SAGES sample is relatively homogenous with respect to racial and ethnic diversity and was highly educated with little functional impairment and no dementia. Therefore, results may not be generalizable to populations with lower educational attainment and/or preexisting mental and physical disabilities. A third limitation is that screen items were included in the reference standard delirium assessment, leading to a potential bias toward increased sensitivity. Finally, all screens were derived from secondary data analysis and further research will be needed to prospectively validate the results. Despite these limitations, this study has several strengths including the use of a well-characterized surgical population and a rigorous approach to delirium measurement. It is one of the first studies to identify a screening tool targeted to identifying delirium in postoperative older adults.

Future research should prospectively validate our screening tool and test its implementation in a real-world clinical environment. As part of this process, clinicians should document barriers and facilitators to widespread implementation. The goal of such screens is to facilitate early identification of postoperative delirium, which will allow timely intervention to address underlying causes and prevent adverse consequences, thereby improving the outcomes of vulnerable older surgical patients.

Delirium is the most common postsurgical complication for older adults, with incidence of 15%-54%, depending on surgery type.¹ Increasing numbers of older adults are undergoing surgery²; and those who develop delirium experience negative consequences including longer lengths of stay, higher likelihood of institutional discharge, and increased morbidity and mortality.³ The American Geriatrics Society Expert Panel on Postoperative Delirium in Older Adults and the European Society of Anaesthesiology⁴ recommend routine screening for delirium in those at risk.

Ultrabrief screens are designed to rule out delirium quickly and identify a subset of patients who require further testing.⁵ Our group, and others, have previously published ultrabrief screens for the general medicine, nonsurgical population and for patients with dementia.^5,6 The UB-2 is an ultrabrief screen consisting of “Months of the year backward” (MOYB) and “What day of the week is it?”, which has a sensitivity of 93% and specificity of 64% in hospitalized older adults and takes less than 40 seconds to administer.⁵ However, no such screens for delirium have been developed for the group with relatively high cognitive and physical functioning undergoing scheduled major surgery in which delirium may present differently. Thus, the purpose of this study was to develop an ultrabrief screen for postoperative delirium using data from a large study of delirium in cognitively intact, older adults undergoing scheduled major noncardiac surgery.

We performed a secondary data analysis on 560 patients enrolled between June 18, 2010, and August 8, 2013, in the Successful Aging After Elective Surgery (SAGES) study,⁷ an ongoing prospective cohort study of older adults undergoing major elective surgeries (eg, total hip or knee replacement; lumbar, cervical, or sacral laminectomy; lower extremity arterial bypass surgery; open abdominal aortic aneurysm repair; and open or laparoscopic colectomy). Exclusion criteria included evidence of dementia, delirium, prior hospitalization within 3 months, legal blindness, severe deafness, terminal condition, history of schizophrenia or psychosis, and history of alcohol abuse or withdrawal. The Institutional Review Boards of Beth Israel Deaconess Medical Center, Brigham and Women’s Hospital, and Hebrew SeniorLife, all in Boston, Massachusetts, approved the study.

SAGES Delirium Assessment and Additional Variables

The presence or absence of delirium was based on daily in-hospital assessments by trained research staff using the Confusion Assessment Method (CAM)⁸ long form. The Delirium Symptom Interview (DSI)⁹ and information related to acute changes in mental status were also included as provided by nursing staff and/or family. Delirium severity was determined using the CAM-S.¹⁰ Participants in The SAGES Study had an initial baseline, presurgical assessment in their homes. Cognitive and physical functioning, depression, comorbidities, laboratory, and self-reported demographic data were collected.

Statistical Analyses

We included CAM delirium data from postoperative days (POD) 1 and 2 for each participant, if available; postoperative day 0 was not included because of potential residual anesthetic effects. We chose these days because most delirium began on POD1 or 2, and patients started being discharged on POD3. We considered all one-, two-, and three-item combinations of the 12 cognitive items of the 3D-CAM¹¹ because of their demonstrated high information content for CAM diagnostic features per Item Response Theory.¹² There were 12 possible one-item screens, 66 two-item screens, and 220 three-item screens. Sensitivity, specificity, and 95% confidence intervals for each were compared with CAM delirium determination. An ideal ultrabrief screen for delirium has high sensitivity with moderate specificity; general guidelines considered based on investigator consensus included screens with a sensitivity higher than 0.90 and specificity greater than 0.70. Because these screens are used to quickly rule out delirium, we also present the percent positive screen among the entire population (whether delirium is present or not). Screens with a positive screen rate of more than 50% are unlikely to be helpful in ruling out delirium quickly in a large enough fraction of the population. We also required that in multiple item screens, no two items should assess the same CAM feature. For instance, we would eliminate a two-item screen with MOYB and four-digit span since both items measure CAM Feature 2 (Inattention). Finally, we evaluated screen performance separately on POD1 and POD2. Switching screens by POD can be confusing, so we chose a single best screen that retained excellent performance over both days. Data analyses used SAS version 9.4 (SAS Institute, Cary, North Carolina).

The dataset included 560 adults who had an average age of 76.6 years (SD = 5.2), were 58% women, and were highly educated (15.0 years; SD = 2.9; Table). Postoperative delirium occurred during one or more days in 134 individuals (24%). A total of 1,100 delirium assessments were used, with 113 that were CAM positive (10.3%). For POD1, we used 551 assessments, 61 of which were positive (11.1%); for POD2, 549 assessments were used, with 51 positive (9.3%). Appendix Tables present the positive screen rates, sensitivities, specificities, and 95% confidence intervals of all 12 one-item screens and the 12 best performing two- and three-item screens in order of decreasing sensitivity.

The best ultrabrief screen from POD1 included the following three items: “Does the patient report feeling confused?”, MOYB, and “Does the patient appear sleepy?”, with a sensitivity of 0.95 (95% CI, 0.87-0.99) and specificity of 0.73 (95% CI, 0.69-0.77). The same combination of items has a sensitivity of 0.88 (95% CI, 0.77-0.96) and a specificity of 0.70 (95% CI, 0.66-0.74) on POD2. When POD1 and POD2 are combined, the sensitivity is 0.92 (95% CI, 0.85-0.96) and specificity is 0.72 (95% CI, 0.69-0.74). We consider this to be our best screen overall.

We identified a three-item screen for delirium after elective surgery consisting of “Does the patient report feeling confused?”, MOYB, and “Does the patient appear sleepy?” In our own prior work, we identified a two-item screen consisting of MOYB and “What is the day of the week?” as the best ultrabrief screen for delirium in general medicine populations (termed the “UB-2”)⁵ and a subsequent screen for patients with delirium superimposed on dementia (DSD) including “What type of place is this?”, Days of the Week Backward, and “Does the patient appear sleepy?”⁶ All three contain a test of attention (a cardinal feature of delirium) and a test of orientation, although the specific test for that varies. Both the surgical and DSD screens include “Does the patient appear sleepy?”, which addresses a reduced level of consciousness. This might be particularly important in the postoperative setting because of residual effects of anesthesia and/or postoperative analgesic medications contributing to delirium. Work done by others confirms our current findings, which is that MOYB is the best single item for most groups. Belleli et al¹³ and Han et al¹⁴ included MOYB as the single attentional item in the 4AT and B-CAM, respectively. The Nu-DESC has been used as a screen in surgical patients; however, it involves only nursing observations and no direct questioning of the patient.¹⁵

The Figure describes how our “best screen” could be integrated into clinical care. One or more “positive” or incorrect responses on these three items constitutes a positive screen that should be further evaluated with the CAM or 3D-CAM. If all three items are correct or negative, this effectively rules out delirium; however, continued periodic screening on a daily (or per shift) basis is indicated. On repeat testing, if any of the previously negative or correct items becomes positive or incorrect, this would be evidence for Acute Change, CAM Feature 1. Finally, it should be noted that, if all three items in our best screen are positive, full CAM criteria for delirium diagnosis are met within the screen itself, and no further testing is required. We envision this process being facilitated by use of an app-based program that generates optimal screening items based on patient and setting characteristics.

There are several limitations that must be noted. First, our three-item screen may not generalize to nonsurgical candidates or those undergoing emergent surgery and should be tested in these groups. Second, the SAGES sample is relatively homogenous with respect to racial and ethnic diversity and was highly educated with little functional impairment and no dementia. Therefore, results may not be generalizable to populations with lower educational attainment and/or preexisting mental and physical disabilities. A third limitation is that screen items were included in the reference standard delirium assessment, leading to a potential bias toward increased sensitivity. Finally, all screens were derived from secondary data analysis and further research will be needed to prospectively validate the results. Despite these limitations, this study has several strengths including the use of a well-characterized surgical population and a rigorous approach to delirium measurement. It is one of the first studies to identify a screening tool targeted to identifying delirium in postoperative older adults.

Future research should prospectively validate our screening tool and test its implementation in a real-world clinical environment. As part of this process, clinicians should document barriers and facilitators to widespread implementation. The goal of such screens is to facilitate early identification of postoperative delirium, which will allow timely intervention to address underlying causes and prevent adverse consequences, thereby improving the outcomes of vulnerable older surgical patients.

Reduction in serious pediatric medical errors has been achieved through sharing of best practices and structured collaboration.¹ However, limited progress has been made in reducing complex, multifactorial events such as unrecognized and undertreated patient deterioration events.² To address this critical gap, interventions to improve clinician situation awareness (SA) have increasingly been applied.³

SA is the ability to recognize and monitor cues regarding what is happening, create a comprehensive picture with available information, and extrapolate whether it indicates adverse developments either immediately or in the near future.⁴ Methods such as care team huddling^5-8 and using standardized patient acuity scoring instruments⁹ increase SA shared across care team roles. Shared SA is the degree to which each team member possesses a common understanding of what is going on. A team is considered to have shared SA when all the individuals agree on both what is happening (accurate perception and comprehension) and what is going to happen in the future (correct projection). Shared SA for high-risk patients in the pediatric intensive care unit (PICU) has not previously been described and may be an opportunity to improve interprofessional team communication for the sickest patients. Shared SA for high-risk patient status is only one aspect of SA, but it facilitates team-based mitigation planning and is an important starting place for understanding opportunities to improve SA. The primary objective of this study was to measure and compare SA among care team roles regarding patients with high-risk status in the PICU.

We conducted a prospective, cross-sectional study from March 2018 to July 2019 examining the individual and shared SA of patient care team trios: the nurse, respiratory therapist (RT), and pediatric resident. The Institutional Review Board at Cincinnati Children’s Hospital Medical Center (CCHMC) determined this study to be non–human-subjects research.

Setting

Research was conducted in the 35-bed PICU of CCHMC, a 500-bed academic free-standing quaternary care children’s hospital.

Participants

We conducted independent surveys of the nurse, RT, and pediatric resident (care team trio) caring for each patient regarding the patient’s clinical deterioration risk status. No patients or care team trios were excluded.

Reference Standard

In 2016, a local panel of experts derived clinical criteria to determine high-risk status for PICU patients, the definition of which, as well as other study terms, appears in Table 1. A PICU attending or fellow identifies a patient as “high risk” when these clinical criteria are met. A plan for prevention and mitigation is formulated and documented for high-risk patients by the PICU attending or fellow at two preexisting daily SA huddles. This plan includes prevention measures to take immediately, specific vital sign thresholds for early identification of deterioration, and guidance on which emergency medication order sets should be utilized to expedite treatment in the event of clinical decline. Dissemination of the care team’s plan is the responsibility of the PICU fellow with additional follow-up by the charge nurse to improve reliability. Identification of high-risk status and development of the prevention and mitigation plan, as completed by the PICU fellow or attending, served as the reference standard for this study.

Survey Instrument Development

The locally developed survey tool was modeled after a validated handoff communication instrument.¹⁰ The tool covered the patient’s risk status, which high-risk clinical criteria were met, the presence and content of a mitigation plan, and planned patient interventions (Appendix).

Data Collection

Care team trios were sampled weekly on weekdays during day and night shifts within 4 to 6 hours of the SA huddle by a core group of three research assistants. Care team trios for one group of five to nine patients within a small geographically isolated pod were surveyed each time. The care team trio was surveyed individually regarding the patient’s risk status, the high-risk clinical criteria met, the presence and content of a mitigation plan, and planned patient interventions. The responses were compared for accuracy against the reference standard, which was defined as identification of high-risk patient status and development of the prevention and mitigation plan as completed by the PICU fellow or attending.

Data Analysis

Rates of agreement between the reference standard and individual members of the care team trio were evaluated via a calculation of proportions by care team role. The agreement between each care team trio member and the reference standard was compared with the nurse role performance using chi-square tests. Rates of concordance within the members of the care team trio were calculated via Light’s kappa for determination of high-risk status.¹¹ Assuming a correct assessment of high-risk status of 62%,¹² with a difference between groups of 10%, a sample size of 400 bedside provider trios gives a power of 85% at the P < .05 significance level for a two-sided chi-square test.

Between March 1, 2018, and July 11, 2019, 400 care team trios were surveyed. Seventy-three trios cared for patients designated high risk (Table 2 for N and proportions). Among all surveyed trios, 94% of nurses (reference), 95% of RTs (P = .4), and 87% of residents (P = .002) identified patient’s risk status correctly. Care trio member concordance for high-risk status was moderate agreement as assessed by a kappa of 0.57 (95% CI, 0.25-0.90).

Of the 73 high-risk patients, nurses correctly identified risk status for 82% (reference), RTs 85% (P = .7), and residents 67% (P = .04). For high-risk patients, nurses identified the presence of a mitigation plan for 98% of patients (reference), RTs 90% (P = .06), and residents 88% (P = .03). Among the care team members who correctly identified the presence of a mitigation plan, nurses were able to specify the correct plan for 83% of patients (reference), RTs for 68% (P = .09), and residents for 70% (P = .11; Figure).

When shared SA for high-risk patients was examined more closely, all three care team roles correctly identified the clinical reason for high-risk status for 32% of patients, with only one or two clinicians being correct for 53%. All three care team clinicians were incorrect for 15% of high-risk patients. Among trios with partial accuracy in which two of three care team members correctly identified a patient as high risk, we examined which care-member was most likely to be incorrect. Nurses incorrectly identified risk for 17% of patients (reference), RTs 19% (P = .8), and residents 64% (P < .0001).

Examining 400 care team trios, we found lower individual SA for residents, compared with nurses, regarding high-risk status, the reason for this status, and the presence of a mitigation plan. In all reported measures except for the content of mitigation plans, residents were significantly less correct than the bedside nurses while RTs performed similarly to bedside nurses throughout. In addition, there was only moderate agreement between care team roles, which shows further opportunities for improvement in shared SA. The disparities between care team roles are consistent with studies that suggest certain factors grounded in institutional culture and interpersonal dynamics, such as poor communication, can lead to breakdowns in shared knowledge.^13,14 Communication issues demonstrate differences across care team roles¹⁴ and may provide insight into barriers to individual and shared SA throughout the care team.

In addition, the effects of patient load on SA needs further study. While our PICU nurses are commonly assigned to 1 to 2 patients, RTs care for 7 to 11 patients, and an on-call resident may be covering 15 to 20 patients during a high-census season. The increased patient load cannot serve as an excuse for the knowledge gap regarding high-risk status and mitigation plan, but may provide an opportunity to support residents and other medical providers through the use of clinical decision-­support tools that indicate high-risk status and represent mitigation plans.¹²

This study has multiple limitations. First, while we based our survey tool on a communication assessment tool with prior validity evidence,^10,12 our tool has not been used prior to this study. The adapted tool contained relevant categorizations of patient information, including explicit statement of patient status and planned treatment consistent with study definitions of SA, and has been used in the critical care setting previously.¹¹ The survey tool used to measure SA in this study was locally designed and implemented only within the study unit, which could lead to decreased reliability and generalizability of the results to other units and institutions at large. Second, while the sample size for the primary measure (N = 400) was adequately powered because our baseline SA was higher than estimated, we had insufficient power for some subgroup analyses that can lead to type II errors. Third, care team trios may have been surveyed repeatedly on the same patient without adjustment in the results for repeated measures. However, as we surveyed on average only once a week and alternated areas of the PICU surveyed, it is unlikely that it affected results given that the most lengths of stay within the PICU range from 3 to 4 days. Finally, individual characteristics of patients were not collected for this work, and therefore, no adjustments or further analysis can be made on the effect of the patient characteristic on the care team role SA.

This study is the first to assess differences in individual and shared SA within a PICU by care team role. Efforts to expand on these findings should include investigation into the causes for the disparities in SA among care team roles for individual patients and among the care teams of high-risk and normal-risk patients. Given the association between increased SA and improved patient outcomes,⁴ future efforts should be structured to address care team role–specific gaps in SA because these may advance the quality of care in the pediatric inpatient setting.

Reduction in serious pediatric medical errors has been achieved through sharing of best practices and structured collaboration.¹ However, limited progress has been made in reducing complex, multifactorial events such as unrecognized and undertreated patient deterioration events.² To address this critical gap, interventions to improve clinician situation awareness (SA) have increasingly been applied.³

SA is the ability to recognize and monitor cues regarding what is happening, create a comprehensive picture with available information, and extrapolate whether it indicates adverse developments either immediately or in the near future.⁴ Methods such as care team huddling^5-8 and using standardized patient acuity scoring instruments⁹ increase SA shared across care team roles. Shared SA is the degree to which each team member possesses a common understanding of what is going on. A team is considered to have shared SA when all the individuals agree on both what is happening (accurate perception and comprehension) and what is going to happen in the future (correct projection). Shared SA for high-risk patients in the pediatric intensive care unit (PICU) has not previously been described and may be an opportunity to improve interprofessional team communication for the sickest patients. Shared SA for high-risk patient status is only one aspect of SA, but it facilitates team-based mitigation planning and is an important starting place for understanding opportunities to improve SA. The primary objective of this study was to measure and compare SA among care team roles regarding patients with high-risk status in the PICU.

We conducted a prospective, cross-sectional study from March 2018 to July 2019 examining the individual and shared SA of patient care team trios: the nurse, respiratory therapist (RT), and pediatric resident. The Institutional Review Board at Cincinnati Children’s Hospital Medical Center (CCHMC) determined this study to be non–human-subjects research.

Setting

Research was conducted in the 35-bed PICU of CCHMC, a 500-bed academic free-standing quaternary care children’s hospital.

Participants

We conducted independent surveys of the nurse, RT, and pediatric resident (care team trio) caring for each patient regarding the patient’s clinical deterioration risk status. No patients or care team trios were excluded.

Reference Standard

In 2016, a local panel of experts derived clinical criteria to determine high-risk status for PICU patients, the definition of which, as well as other study terms, appears in Table 1. A PICU attending or fellow identifies a patient as “high risk” when these clinical criteria are met. A plan for prevention and mitigation is formulated and documented for high-risk patients by the PICU attending or fellow at two preexisting daily SA huddles. This plan includes prevention measures to take immediately, specific vital sign thresholds for early identification of deterioration, and guidance on which emergency medication order sets should be utilized to expedite treatment in the event of clinical decline. Dissemination of the care team’s plan is the responsibility of the PICU fellow with additional follow-up by the charge nurse to improve reliability. Identification of high-risk status and development of the prevention and mitigation plan, as completed by the PICU fellow or attending, served as the reference standard for this study.

Survey Instrument Development

The locally developed survey tool was modeled after a validated handoff communication instrument.¹⁰ The tool covered the patient’s risk status, which high-risk clinical criteria were met, the presence and content of a mitigation plan, and planned patient interventions (Appendix).

Data Collection

Care team trios were sampled weekly on weekdays during day and night shifts within 4 to 6 hours of the SA huddle by a core group of three research assistants. Care team trios for one group of five to nine patients within a small geographically isolated pod were surveyed each time. The care team trio was surveyed individually regarding the patient’s risk status, the high-risk clinical criteria met, the presence and content of a mitigation plan, and planned patient interventions. The responses were compared for accuracy against the reference standard, which was defined as identification of high-risk patient status and development of the prevention and mitigation plan as completed by the PICU fellow or attending.

Data Analysis

Rates of agreement between the reference standard and individual members of the care team trio were evaluated via a calculation of proportions by care team role. The agreement between each care team trio member and the reference standard was compared with the nurse role performance using chi-square tests. Rates of concordance within the members of the care team trio were calculated via Light’s kappa for determination of high-risk status.¹¹ Assuming a correct assessment of high-risk status of 62%,¹² with a difference between groups of 10%, a sample size of 400 bedside provider trios gives a power of 85% at the P < .05 significance level for a two-sided chi-square test.

Between March 1, 2018, and July 11, 2019, 400 care team trios were surveyed. Seventy-three trios cared for patients designated high risk (Table 2 for N and proportions). Among all surveyed trios, 94% of nurses (reference), 95% of RTs (P = .4), and 87% of residents (P = .002) identified patient’s risk status correctly. Care trio member concordance for high-risk status was moderate agreement as assessed by a kappa of 0.57 (95% CI, 0.25-0.90).

Of the 73 high-risk patients, nurses correctly identified risk status for 82% (reference), RTs 85% (P = .7), and residents 67% (P = .04). For high-risk patients, nurses identified the presence of a mitigation plan for 98% of patients (reference), RTs 90% (P = .06), and residents 88% (P = .03). Among the care team members who correctly identified the presence of a mitigation plan, nurses were able to specify the correct plan for 83% of patients (reference), RTs for 68% (P = .09), and residents for 70% (P = .11; Figure).

When shared SA for high-risk patients was examined more closely, all three care team roles correctly identified the clinical reason for high-risk status for 32% of patients, with only one or two clinicians being correct for 53%. All three care team clinicians were incorrect for 15% of high-risk patients. Among trios with partial accuracy in which two of three care team members correctly identified a patient as high risk, we examined which care-member was most likely to be incorrect. Nurses incorrectly identified risk for 17% of patients (reference), RTs 19% (P = .8), and residents 64% (P < .0001).

Examining 400 care team trios, we found lower individual SA for residents, compared with nurses, regarding high-risk status, the reason for this status, and the presence of a mitigation plan. In all reported measures except for the content of mitigation plans, residents were significantly less correct than the bedside nurses while RTs performed similarly to bedside nurses throughout. In addition, there was only moderate agreement between care team roles, which shows further opportunities for improvement in shared SA. The disparities between care team roles are consistent with studies that suggest certain factors grounded in institutional culture and interpersonal dynamics, such as poor communication, can lead to breakdowns in shared knowledge.^13,14 Communication issues demonstrate differences across care team roles¹⁴ and may provide insight into barriers to individual and shared SA throughout the care team.

In addition, the effects of patient load on SA needs further study. While our PICU nurses are commonly assigned to 1 to 2 patients, RTs care for 7 to 11 patients, and an on-call resident may be covering 15 to 20 patients during a high-census season. The increased patient load cannot serve as an excuse for the knowledge gap regarding high-risk status and mitigation plan, but may provide an opportunity to support residents and other medical providers through the use of clinical decision-­support tools that indicate high-risk status and represent mitigation plans.¹²

This study has multiple limitations. First, while we based our survey tool on a communication assessment tool with prior validity evidence,^10,12 our tool has not been used prior to this study. The adapted tool contained relevant categorizations of patient information, including explicit statement of patient status and planned treatment consistent with study definitions of SA, and has been used in the critical care setting previously.¹¹ The survey tool used to measure SA in this study was locally designed and implemented only within the study unit, which could lead to decreased reliability and generalizability of the results to other units and institutions at large. Second, while the sample size for the primary measure (N = 400) was adequately powered because our baseline SA was higher than estimated, we had insufficient power for some subgroup analyses that can lead to type II errors. Third, care team trios may have been surveyed repeatedly on the same patient without adjustment in the results for repeated measures. However, as we surveyed on average only once a week and alternated areas of the PICU surveyed, it is unlikely that it affected results given that the most lengths of stay within the PICU range from 3 to 4 days. Finally, individual characteristics of patients were not collected for this work, and therefore, no adjustments or further analysis can be made on the effect of the patient characteristic on the care team role SA.

This study is the first to assess differences in individual and shared SA within a PICU by care team role. Efforts to expand on these findings should include investigation into the causes for the disparities in SA among care team roles for individual patients and among the care teams of high-risk and normal-risk patients. Given the association between increased SA and improved patient outcomes,⁴ future efforts should be structured to address care team role–specific gaps in SA because these may advance the quality of care in the pediatric inpatient setting.

Reduction in serious pediatric medical errors has been achieved through sharing of best practices and structured collaboration.¹ However, limited progress has been made in reducing complex, multifactorial events such as unrecognized and undertreated patient deterioration events.² To address this critical gap, interventions to improve clinician situation awareness (SA) have increasingly been applied.³

SA is the ability to recognize and monitor cues regarding what is happening, create a comprehensive picture with available information, and extrapolate whether it indicates adverse developments either immediately or in the near future.⁴ Methods such as care team huddling^5-8 and using standardized patient acuity scoring instruments⁹ increase SA shared across care team roles. Shared SA is the degree to which each team member possesses a common understanding of what is going on. A team is considered to have shared SA when all the individuals agree on both what is happening (accurate perception and comprehension) and what is going to happen in the future (correct projection). Shared SA for high-risk patients in the pediatric intensive care unit (PICU) has not previously been described and may be an opportunity to improve interprofessional team communication for the sickest patients. Shared SA for high-risk patient status is only one aspect of SA, but it facilitates team-based mitigation planning and is an important starting place for understanding opportunities to improve SA. The primary objective of this study was to measure and compare SA among care team roles regarding patients with high-risk status in the PICU.

We conducted a prospective, cross-sectional study from March 2018 to July 2019 examining the individual and shared SA of patient care team trios: the nurse, respiratory therapist (RT), and pediatric resident. The Institutional Review Board at Cincinnati Children’s Hospital Medical Center (CCHMC) determined this study to be non–human-subjects research.

Setting

Research was conducted in the 35-bed PICU of CCHMC, a 500-bed academic free-standing quaternary care children’s hospital.

Participants

We conducted independent surveys of the nurse, RT, and pediatric resident (care team trio) caring for each patient regarding the patient’s clinical deterioration risk status. No patients or care team trios were excluded.

Reference Standard

In 2016, a local panel of experts derived clinical criteria to determine high-risk status for PICU patients, the definition of which, as well as other study terms, appears in Table 1. A PICU attending or fellow identifies a patient as “high risk” when these clinical criteria are met. A plan for prevention and mitigation is formulated and documented for high-risk patients by the PICU attending or fellow at two preexisting daily SA huddles. This plan includes prevention measures to take immediately, specific vital sign thresholds for early identification of deterioration, and guidance on which emergency medication order sets should be utilized to expedite treatment in the event of clinical decline. Dissemination of the care team’s plan is the responsibility of the PICU fellow with additional follow-up by the charge nurse to improve reliability. Identification of high-risk status and development of the prevention and mitigation plan, as completed by the PICU fellow or attending, served as the reference standard for this study.

Survey Instrument Development

The locally developed survey tool was modeled after a validated handoff communication instrument.¹⁰ The tool covered the patient’s risk status, which high-risk clinical criteria were met, the presence and content of a mitigation plan, and planned patient interventions (Appendix).

Data Collection

Care team trios were sampled weekly on weekdays during day and night shifts within 4 to 6 hours of the SA huddle by a core group of three research assistants. Care team trios for one group of five to nine patients within a small geographically isolated pod were surveyed each time. The care team trio was surveyed individually regarding the patient’s risk status, the high-risk clinical criteria met, the presence and content of a mitigation plan, and planned patient interventions. The responses were compared for accuracy against the reference standard, which was defined as identification of high-risk patient status and development of the prevention and mitigation plan as completed by the PICU fellow or attending.

Data Analysis

Rates of agreement between the reference standard and individual members of the care team trio were evaluated via a calculation of proportions by care team role. The agreement between each care team trio member and the reference standard was compared with the nurse role performance using chi-square tests. Rates of concordance within the members of the care team trio were calculated via Light’s kappa for determination of high-risk status.¹¹ Assuming a correct assessment of high-risk status of 62%,¹² with a difference between groups of 10%, a sample size of 400 bedside provider trios gives a power of 85% at the P < .05 significance level for a two-sided chi-square test.

Between March 1, 2018, and July 11, 2019, 400 care team trios were surveyed. Seventy-three trios cared for patients designated high risk (Table 2 for N and proportions). Among all surveyed trios, 94% of nurses (reference), 95% of RTs (P = .4), and 87% of residents (P = .002) identified patient’s risk status correctly. Care trio member concordance for high-risk status was moderate agreement as assessed by a kappa of 0.57 (95% CI, 0.25-0.90).

Of the 73 high-risk patients, nurses correctly identified risk status for 82% (reference), RTs 85% (P = .7), and residents 67% (P = .04). For high-risk patients, nurses identified the presence of a mitigation plan for 98% of patients (reference), RTs 90% (P = .06), and residents 88% (P = .03). Among the care team members who correctly identified the presence of a mitigation plan, nurses were able to specify the correct plan for 83% of patients (reference), RTs for 68% (P = .09), and residents for 70% (P = .11; Figure).

When shared SA for high-risk patients was examined more closely, all three care team roles correctly identified the clinical reason for high-risk status for 32% of patients, with only one or two clinicians being correct for 53%. All three care team clinicians were incorrect for 15% of high-risk patients. Among trios with partial accuracy in which two of three care team members correctly identified a patient as high risk, we examined which care-member was most likely to be incorrect. Nurses incorrectly identified risk for 17% of patients (reference), RTs 19% (P = .8), and residents 64% (P < .0001).

Examining 400 care team trios, we found lower individual SA for residents, compared with nurses, regarding high-risk status, the reason for this status, and the presence of a mitigation plan. In all reported measures except for the content of mitigation plans, residents were significantly less correct than the bedside nurses while RTs performed similarly to bedside nurses throughout. In addition, there was only moderate agreement between care team roles, which shows further opportunities for improvement in shared SA. The disparities between care team roles are consistent with studies that suggest certain factors grounded in institutional culture and interpersonal dynamics, such as poor communication, can lead to breakdowns in shared knowledge.^13,14 Communication issues demonstrate differences across care team roles¹⁴ and may provide insight into barriers to individual and shared SA throughout the care team.

In addition, the effects of patient load on SA needs further study. While our PICU nurses are commonly assigned to 1 to 2 patients, RTs care for 7 to 11 patients, and an on-call resident may be covering 15 to 20 patients during a high-census season. The increased patient load cannot serve as an excuse for the knowledge gap regarding high-risk status and mitigation plan, but may provide an opportunity to support residents and other medical providers through the use of clinical decision-­support tools that indicate high-risk status and represent mitigation plans.¹²

This study has multiple limitations. First, while we based our survey tool on a communication assessment tool with prior validity evidence,^10,12 our tool has not been used prior to this study. The adapted tool contained relevant categorizations of patient information, including explicit statement of patient status and planned treatment consistent with study definitions of SA, and has been used in the critical care setting previously.¹¹ The survey tool used to measure SA in this study was locally designed and implemented only within the study unit, which could lead to decreased reliability and generalizability of the results to other units and institutions at large. Second, while the sample size for the primary measure (N = 400) was adequately powered because our baseline SA was higher than estimated, we had insufficient power for some subgroup analyses that can lead to type II errors. Third, care team trios may have been surveyed repeatedly on the same patient without adjustment in the results for repeated measures. However, as we surveyed on average only once a week and alternated areas of the PICU surveyed, it is unlikely that it affected results given that the most lengths of stay within the PICU range from 3 to 4 days. Finally, individual characteristics of patients were not collected for this work, and therefore, no adjustments or further analysis can be made on the effect of the patient characteristic on the care team role SA.

This study is the first to assess differences in individual and shared SA within a PICU by care team role. Efforts to expand on these findings should include investigation into the causes for the disparities in SA among care team roles for individual patients and among the care teams of high-risk and normal-risk patients. Given the association between increased SA and improved patient outcomes,⁴ future efforts should be structured to address care team role–specific gaps in SA because these may advance the quality of care in the pediatric inpatient setting.

Coronavirus disease 2019 (COVID-19) is the most important public health emergency of the 21st century. The pandemic has devastated New York City, where over 17,000 confirmed deaths have occurred as of June 5, 2020.¹ The most common cause of death in COVID-19 patients is respiratory failure from acute respiratory distress syndrome (ARDS). A recent study reported high mortality rates among COVID-19 patients who received mechanical ventilation (MV).²

Glucocorticoids are useful as adjunctive treatment for some infections with inflammatory responses, but their efficacy in COVID-19 is unclear. Prior experience with influenza and other coronaviruses may be relevant. A recent meta-analysis of influenza pneumonia showed increased mortality and a higher rate of secondary infections in patients who were administered glucocorticoids.³ For Middle East respiratory syndrome, severe acute respiratory syndrome, and influenza, some studies have demonstrated an association between glucocorticoid use and delayed viral clearance.^4-7 However, a recent retrospective series of patients with COVID-19 and ARDS demonstrated a decrease in mortality with glucocorticoid use.⁸ Glucocorticoids are easily obtained and familiar to providers caring for COVID-19 patients. Hence their empiric use is widespread.^8,9

The primary goal of this study was to determine whether early glucocorticoid treatment is associated with reduced mortality or need for MV in COVID-19 patients.

Study Setting and Overview

Montefiore Medical Center comprises four hospitals totaling 1,536 beds in the Bronx borough of New York, New York. Based upon early experience, some clinicians began prescribing systemic glucocorticoids to patients with COVID-19 while others did not. We leveraged this variation in practice to examine the effectiveness of glucocorticoids in reducing mortality and the rate of MV in hospitalized COVID-19 patients.

Study Populations

There were 2,998 patients admitted with a positive COVID-19 test between March 11, 2020, and April 13, 2020. An a priori decision was made to include all hospitalized COVID-19 patients, including children. Because the outcomes of in-hospital mortality and in-hospital MV cannot be assessed in patients still hospitalized, we included only patients who either died or had been discharged from the hospital. Patients who died or were placed on MV within the first 48 hours of admission were excluded because outcome events occurred before having the opportunity for glucocorticoid treatment. To ensure treatment preceded outcome measurement, we included only patients treated with glucocorticoids within the first 48 hours of admission (treatment group) and compared them with patients never treated with glucocorticoids (control group).

Outcomes and Independent Variables

The primary outcome was a composite of in-hospital mortality or in-hospital MV. Secondary outcomes were the components of the primary. Timing of MV was determined using the first documentation of a ventilator respiratory rate or tidal volume. The independent variable of interest was treatment with glucocorticoids within the first 48 hours of admission. Formulations included are described in the Appendix.

To compare treatment and control groups and to perform adjusted analyses, we also examined the demographic and clinical characteristics, comorbidities, and laboratory values of each admission. For the comparison of study populations, missing values for each variable were ignored. In the primary (unstratified) multivariable analysis, continuous variables were categorized, with missing values assumed to be normal when used as an adjustment variable. All variables extracted, number of missing values, candidates for inclusion in the multivariable analysis, and those that fell out of the model are presented in the Appendix. Several subgroup analyses were predefined including age, diabetes, admission glucose, C-reactive protein (CRP), D-dimer, and troponin T levels.

Statistical Analysis

The treated and control groups were compared with respect to demographics, clinical characteristics, comorbidities, and laboratory values. Primary and secondary outcomes in the groups were compared in unadjusted and adjusted analyses using univariable and multivariable logistic regression models. All patient characteristics that were candidates for inclusion in the adjustment models are listed in the Appendix. Variables were included in the final model if they were associated with the primary outcome (Wald test P < .20) in univariable regression. A sensitivity analysis excluded all variables missing greater than 10% of data, including CRP. Interactions between treatment and six predefined subgroups were tested using logistic regression with interaction terms (eg, [steroids]*[age]). Stratified logistic regression was used to test the association between treatment and the primary outcome in each of the predefined subgroups. Patients who were missing CRP were excluded from the stratified analysis. Because a significant interaction between treatment and initial CRP level was discovered, we undertook a post hoc adjusted analysis within each of the 15 predefined subgroup variables. Because there were fewer outcome events in each subgroup, we constructed a parsimonious logistic regression model that included all variables independently associated with the exposure (P < .05). The same seven adjustment variables were used in each of the predefined subgroups. The study was approved by the Albert Einstein College of Medicine Institutional Review Board. Stata 15.1 software (StataCorp) was used for data analysis.

Of 2,998 patients examined, 1,806 met inclusion criteria and included 140 (7.7%) treated with glucocorticoids within 48 hours of admission and 1,666 who never received glucocorticoids. Reasons for exclusion of 1,192 patients are provided in the Appendix. Among patients who remained hospitalized and were excluded, 169 of 962 (17.6%) received glucocorticoids. Characteristics of the study population are presented in Table 1. Treatment and control groups were similar except that glucocorticoid-treated patients were more likely to have chronic obstructive pulmonary disease (COPD), asthma, rheumatoid arthritis or lupus, or to have received glucocorticoids in the year prior to admission.

There were 318 who met the primary outcome of death or mechanical ventilation, 270 of whom died and 135 of whom required mechanical ventilation. Overall, early use of glucocorticoids was not associated with in-hospital mortality or MV as a composite outcome or as separate outcomes in both unadjusted and adjusted models (Table 2A). However, there was significant heterogeneity of treatment effect in the subgroups defined by CRP levels (P for interaction = .008; Figure). Early glucocorticoid use and an initial CRP of 20 mg/dL or higher was associated with a significantly reduced risk of mortality or MV in unadjusted (odds ratio, 0.23; 95% CI, 0.08-0.70) and adjusted (aOR, 0.20; 95% CI, 0.06-0.67) analyses (Table 2B). Conversely, glucocorticoid treatment in patients with CRP levels less than 10 mg/dL was associated with a significantly increased risk of mortality or MV in unadjusted (OR_, 2.64; 95% CI, 1.39-5.03) and adjusted (aOR_, 3.14; 95% CI, 1.52-6.50) analyses.

The results of this study indicate that early treatment with glucocorticoids is not associated with mortality or need for MV in unselected patients with COVID-19. Subgroup analyses suggest that glucocorticoid-treated patients with markedly elevated CRP may benefit from glucocorticoid treatment, whereas those patients with lower CRP may be harmed. Our findings were consistent after adjustment for clinical characteristics. The public health implications of these findings are hard to overestimate. Given the global growth of the pandemic and that glucocorticoids are widely available and inexpensive, glucocorticoid therapy may save many thousands of lives. Equally important because we have been able to identify a group that may be harmed, some patients may be saved because glucocorticoids will not be given.

Our study reaffirms the finding of the as yet unpublished Randomised Evaluation of COVID-19 Therapy (RECOVERY) trial that there is a subset of patients with COVID-19 who benefit from treatment with glucocorticoids.¹⁰ Our study extends the findings of the RECOVERY trial in two important ways. First, in addition to finding some patients who may benefit, we also have identified patient groups that may experience harm from treatment with glucocorticoids. This finding suggests choosing the right patients for glucocorticoid treatment is critical to maximize the likelihood of benefit and minimize the risk of harm. Second, we have identified patient groups who are likely to benefit (or be harmed) on the basis of a widely available lab test (CRP).

Our results are also consistent with previous studies of patients with SARS-CoV and MERS-CoV, in which no associations between glucocorticoid treatment and mortality were found.⁷ However, the results of studies examining the effect of glucocorticoids in patients with COVID-19 are less consistent.^8,11,12

Few of the previous studies examined the effects of glucocorticoids in subgroups of patients. In our study, the improved outcomes associated with glucocorticoid use in patients with elevated CRPs is intriguing and may be clinically important. Proinflammatory cytokines, especially interleukin-6, acutely increase CRP levels. Cytokine storm syndrome (CSS) is a hyperinflammatory condition that occurs in a subset of COVID-19 patients, often resulting in multiorgan dysfunction.¹³ CRP is markedly elevated in CSS,¹⁴ and improved outcomes with glucocorticoid therapy in this subgroup may indicate benefit in this inflammatory phenotype. Patients with lower CRP are less likely to have CSS and may experience more harm than benefit associated with glucocorticoid treatment.

Several limitations are inherent to this study. Since it was done at a single center, the results may not be generalizable. As a retrospective analysis, it is subject to confounding and bias. In addition, because patients were included only if they had reached the outcome of death/MV or hospital discharge, the sample size was truncated. We believe glucocorticoid use in hospitalized patients excluded from the study reflects increased use with time because of a growing belief in their effectiveness.

Preliminary analysis from the RECOVERY study showed a reduced rate of mortality in patients randomized to dexamethasone, compared with those who received standard of care.¹⁰ These results led to the National Institutes for Health COVID-19 Treatment Guidelines Panel recommendation for dexamethasone treatment in patients with COVID-19 who require supplemental oxygen or MV.¹⁵ Our findings suggest a role for CRP to identify patients who may benefit from glucocorticoid therapy, as well as those in whom it may be harmful. Additional studies to further elucidate the role of CRP in guiding glucocorticoid therapy and to predict clinical response are needed.

Coronavirus disease 2019 (COVID-19) is the most important public health emergency of the 21st century. The pandemic has devastated New York City, where over 17,000 confirmed deaths have occurred as of June 5, 2020.¹ The most common cause of death in COVID-19 patients is respiratory failure from acute respiratory distress syndrome (ARDS). A recent study reported high mortality rates among COVID-19 patients who received mechanical ventilation (MV).²

Glucocorticoids are useful as adjunctive treatment for some infections with inflammatory responses, but their efficacy in COVID-19 is unclear. Prior experience with influenza and other coronaviruses may be relevant. A recent meta-analysis of influenza pneumonia showed increased mortality and a higher rate of secondary infections in patients who were administered glucocorticoids.³ For Middle East respiratory syndrome, severe acute respiratory syndrome, and influenza, some studies have demonstrated an association between glucocorticoid use and delayed viral clearance.^4-7 However, a recent retrospective series of patients with COVID-19 and ARDS demonstrated a decrease in mortality with glucocorticoid use.⁸ Glucocorticoids are easily obtained and familiar to providers caring for COVID-19 patients. Hence their empiric use is widespread.^8,9

The primary goal of this study was to determine whether early glucocorticoid treatment is associated with reduced mortality or need for MV in COVID-19 patients.

Study Setting and Overview

Montefiore Medical Center comprises four hospitals totaling 1,536 beds in the Bronx borough of New York, New York. Based upon early experience, some clinicians began prescribing systemic glucocorticoids to patients with COVID-19 while others did not. We leveraged this variation in practice to examine the effectiveness of glucocorticoids in reducing mortality and the rate of MV in hospitalized COVID-19 patients.

Study Populations

There were 2,998 patients admitted with a positive COVID-19 test between March 11, 2020, and April 13, 2020. An a priori decision was made to include all hospitalized COVID-19 patients, including children. Because the outcomes of in-hospital mortality and in-hospital MV cannot be assessed in patients still hospitalized, we included only patients who either died or had been discharged from the hospital. Patients who died or were placed on MV within the first 48 hours of admission were excluded because outcome events occurred before having the opportunity for glucocorticoid treatment. To ensure treatment preceded outcome measurement, we included only patients treated with glucocorticoids within the first 48 hours of admission (treatment group) and compared them with patients never treated with glucocorticoids (control group).

Outcomes and Independent Variables

The primary outcome was a composite of in-hospital mortality or in-hospital MV. Secondary outcomes were the components of the primary. Timing of MV was determined using the first documentation of a ventilator respiratory rate or tidal volume. The independent variable of interest was treatment with glucocorticoids within the first 48 hours of admission. Formulations included are described in the Appendix.

To compare treatment and control groups and to perform adjusted analyses, we also examined the demographic and clinical characteristics, comorbidities, and laboratory values of each admission. For the comparison of study populations, missing values for each variable were ignored. In the primary (unstratified) multivariable analysis, continuous variables were categorized, with missing values assumed to be normal when used as an adjustment variable. All variables extracted, number of missing values, candidates for inclusion in the multivariable analysis, and those that fell out of the model are presented in the Appendix. Several subgroup analyses were predefined including age, diabetes, admission glucose, C-reactive protein (CRP), D-dimer, and troponin T levels.

Statistical Analysis

The treated and control groups were compared with respect to demographics, clinical characteristics, comorbidities, and laboratory values. Primary and secondary outcomes in the groups were compared in unadjusted and adjusted analyses using univariable and multivariable logistic regression models. All patient characteristics that were candidates for inclusion in the adjustment models are listed in the Appendix. Variables were included in the final model if they were associated with the primary outcome (Wald test P < .20) in univariable regression. A sensitivity analysis excluded all variables missing greater than 10% of data, including CRP. Interactions between treatment and six predefined subgroups were tested using logistic regression with interaction terms (eg, [steroids]*[age]). Stratified logistic regression was used to test the association between treatment and the primary outcome in each of the predefined subgroups. Patients who were missing CRP were excluded from the stratified analysis. Because a significant interaction between treatment and initial CRP level was discovered, we undertook a post hoc adjusted analysis within each of the 15 predefined subgroup variables. Because there were fewer outcome events in each subgroup, we constructed a parsimonious logistic regression model that included all variables independently associated with the exposure (P < .05). The same seven adjustment variables were used in each of the predefined subgroups. The study was approved by the Albert Einstein College of Medicine Institutional Review Board. Stata 15.1 software (StataCorp) was used for data analysis.

Of 2,998 patients examined, 1,806 met inclusion criteria and included 140 (7.7%) treated with glucocorticoids within 48 hours of admission and 1,666 who never received glucocorticoids. Reasons for exclusion of 1,192 patients are provided in the Appendix. Among patients who remained hospitalized and were excluded, 169 of 962 (17.6%) received glucocorticoids. Characteristics of the study population are presented in Table 1. Treatment and control groups were similar except that glucocorticoid-treated patients were more likely to have chronic obstructive pulmonary disease (COPD), asthma, rheumatoid arthritis or lupus, or to have received glucocorticoids in the year prior to admission.

There were 318 who met the primary outcome of death or mechanical ventilation, 270 of whom died and 135 of whom required mechanical ventilation. Overall, early use of glucocorticoids was not associated with in-hospital mortality or MV as a composite outcome or as separate outcomes in both unadjusted and adjusted models (Table 2A). However, there was significant heterogeneity of treatment effect in the subgroups defined by CRP levels (P for interaction = .008; Figure). Early glucocorticoid use and an initial CRP of 20 mg/dL or higher was associated with a significantly reduced risk of mortality or MV in unadjusted (odds ratio, 0.23; 95% CI, 0.08-0.70) and adjusted (aOR, 0.20; 95% CI, 0.06-0.67) analyses (Table 2B). Conversely, glucocorticoid treatment in patients with CRP levels less than 10 mg/dL was associated with a significantly increased risk of mortality or MV in unadjusted (OR_, 2.64; 95% CI, 1.39-5.03) and adjusted (aOR_, 3.14; 95% CI, 1.52-6.50) analyses.

The results of this study indicate that early treatment with glucocorticoids is not associated with mortality or need for MV in unselected patients with COVID-19. Subgroup analyses suggest that glucocorticoid-treated patients with markedly elevated CRP may benefit from glucocorticoid treatment, whereas those patients with lower CRP may be harmed. Our findings were consistent after adjustment for clinical characteristics. The public health implications of these findings are hard to overestimate. Given the global growth of the pandemic and that glucocorticoids are widely available and inexpensive, glucocorticoid therapy may save many thousands of lives. Equally important because we have been able to identify a group that may be harmed, some patients may be saved because glucocorticoids will not be given.

Our study reaffirms the finding of the as yet unpublished Randomised Evaluation of COVID-19 Therapy (RECOVERY) trial that there is a subset of patients with COVID-19 who benefit from treatment with glucocorticoids.¹⁰ Our study extends the findings of the RECOVERY trial in two important ways. First, in addition to finding some patients who may benefit, we also have identified patient groups that may experience harm from treatment with glucocorticoids. This finding suggests choosing the right patients for glucocorticoid treatment is critical to maximize the likelihood of benefit and minimize the risk of harm. Second, we have identified patient groups who are likely to benefit (or be harmed) on the basis of a widely available lab test (CRP).

Our results are also consistent with previous studies of patients with SARS-CoV and MERS-CoV, in which no associations between glucocorticoid treatment and mortality were found.⁷ However, the results of studies examining the effect of glucocorticoids in patients with COVID-19 are less consistent.^8,11,12

Few of the previous studies examined the effects of glucocorticoids in subgroups of patients. In our study, the improved outcomes associated with glucocorticoid use in patients with elevated CRPs is intriguing and may be clinically important. Proinflammatory cytokines, especially interleukin-6, acutely increase CRP levels. Cytokine storm syndrome (CSS) is a hyperinflammatory condition that occurs in a subset of COVID-19 patients, often resulting in multiorgan dysfunction.¹³ CRP is markedly elevated in CSS,¹⁴ and improved outcomes with glucocorticoid therapy in this subgroup may indicate benefit in this inflammatory phenotype. Patients with lower CRP are less likely to have CSS and may experience more harm than benefit associated with glucocorticoid treatment.

Several limitations are inherent to this study. Since it was done at a single center, the results may not be generalizable. As a retrospective analysis, it is subject to confounding and bias. In addition, because patients were included only if they had reached the outcome of death/MV or hospital discharge, the sample size was truncated. We believe glucocorticoid use in hospitalized patients excluded from the study reflects increased use with time because of a growing belief in their effectiveness.

Preliminary analysis from the RECOVERY study showed a reduced rate of mortality in patients randomized to dexamethasone, compared with those who received standard of care.¹⁰ These results led to the National Institutes for Health COVID-19 Treatment Guidelines Panel recommendation for dexamethasone treatment in patients with COVID-19 who require supplemental oxygen or MV.¹⁵ Our findings suggest a role for CRP to identify patients who may benefit from glucocorticoid therapy, as well as those in whom it may be harmful. Additional studies to further elucidate the role of CRP in guiding glucocorticoid therapy and to predict clinical response are needed.

Coronavirus disease 2019 (COVID-19) is the most important public health emergency of the 21st century. The pandemic has devastated New York City, where over 17,000 confirmed deaths have occurred as of June 5, 2020.¹ The most common cause of death in COVID-19 patients is respiratory failure from acute respiratory distress syndrome (ARDS). A recent study reported high mortality rates among COVID-19 patients who received mechanical ventilation (MV).²

Glucocorticoids are useful as adjunctive treatment for some infections with inflammatory responses, but their efficacy in COVID-19 is unclear. Prior experience with influenza and other coronaviruses may be relevant. A recent meta-analysis of influenza pneumonia showed increased mortality and a higher rate of secondary infections in patients who were administered glucocorticoids.³ For Middle East respiratory syndrome, severe acute respiratory syndrome, and influenza, some studies have demonstrated an association between glucocorticoid use and delayed viral clearance.^4-7 However, a recent retrospective series of patients with COVID-19 and ARDS demonstrated a decrease in mortality with glucocorticoid use.⁸ Glucocorticoids are easily obtained and familiar to providers caring for COVID-19 patients. Hence their empiric use is widespread.^8,9

The primary goal of this study was to determine whether early glucocorticoid treatment is associated with reduced mortality or need for MV in COVID-19 patients.

Study Setting and Overview

Montefiore Medical Center comprises four hospitals totaling 1,536 beds in the Bronx borough of New York, New York. Based upon early experience, some clinicians began prescribing systemic glucocorticoids to patients with COVID-19 while others did not. We leveraged this variation in practice to examine the effectiveness of glucocorticoids in reducing mortality and the rate of MV in hospitalized COVID-19 patients.

Study Populations

There were 2,998 patients admitted with a positive COVID-19 test between March 11, 2020, and April 13, 2020. An a priori decision was made to include all hospitalized COVID-19 patients, including children. Because the outcomes of in-hospital mortality and in-hospital MV cannot be assessed in patients still hospitalized, we included only patients who either died or had been discharged from the hospital. Patients who died or were placed on MV within the first 48 hours of admission were excluded because outcome events occurred before having the opportunity for glucocorticoid treatment. To ensure treatment preceded outcome measurement, we included only patients treated with glucocorticoids within the first 48 hours of admission (treatment group) and compared them with patients never treated with glucocorticoids (control group).

Outcomes and Independent Variables

The primary outcome was a composite of in-hospital mortality or in-hospital MV. Secondary outcomes were the components of the primary. Timing of MV was determined using the first documentation of a ventilator respiratory rate or tidal volume. The independent variable of interest was treatment with glucocorticoids within the first 48 hours of admission. Formulations included are described in the Appendix.

To compare treatment and control groups and to perform adjusted analyses, we also examined the demographic and clinical characteristics, comorbidities, and laboratory values of each admission. For the comparison of study populations, missing values for each variable were ignored. In the primary (unstratified) multivariable analysis, continuous variables were categorized, with missing values assumed to be normal when used as an adjustment variable. All variables extracted, number of missing values, candidates for inclusion in the multivariable analysis, and those that fell out of the model are presented in the Appendix. Several subgroup analyses were predefined including age, diabetes, admission glucose, C-reactive protein (CRP), D-dimer, and troponin T levels.

Statistical Analysis

The treated and control groups were compared with respect to demographics, clinical characteristics, comorbidities, and laboratory values. Primary and secondary outcomes in the groups were compared in unadjusted and adjusted analyses using univariable and multivariable logistic regression models. All patient characteristics that were candidates for inclusion in the adjustment models are listed in the Appendix. Variables were included in the final model if they were associated with the primary outcome (Wald test P < .20) in univariable regression. A sensitivity analysis excluded all variables missing greater than 10% of data, including CRP. Interactions between treatment and six predefined subgroups were tested using logistic regression with interaction terms (eg, [steroids]*[age]). Stratified logistic regression was used to test the association between treatment and the primary outcome in each of the predefined subgroups. Patients who were missing CRP were excluded from the stratified analysis. Because a significant interaction between treatment and initial CRP level was discovered, we undertook a post hoc adjusted analysis within each of the 15 predefined subgroup variables. Because there were fewer outcome events in each subgroup, we constructed a parsimonious logistic regression model that included all variables independently associated with the exposure (P < .05). The same seven adjustment variables were used in each of the predefined subgroups. The study was approved by the Albert Einstein College of Medicine Institutional Review Board. Stata 15.1 software (StataCorp) was used for data analysis.

Of 2,998 patients examined, 1,806 met inclusion criteria and included 140 (7.7%) treated with glucocorticoids within 48 hours of admission and 1,666 who never received glucocorticoids. Reasons for exclusion of 1,192 patients are provided in the Appendix. Among patients who remained hospitalized and were excluded, 169 of 962 (17.6%) received glucocorticoids. Characteristics of the study population are presented in Table 1. Treatment and control groups were similar except that glucocorticoid-treated patients were more likely to have chronic obstructive pulmonary disease (COPD), asthma, rheumatoid arthritis or lupus, or to have received glucocorticoids in the year prior to admission.

There were 318 who met the primary outcome of death or mechanical ventilation, 270 of whom died and 135 of whom required mechanical ventilation. Overall, early use of glucocorticoids was not associated with in-hospital mortality or MV as a composite outcome or as separate outcomes in both unadjusted and adjusted models (Table 2A). However, there was significant heterogeneity of treatment effect in the subgroups defined by CRP levels (P for interaction = .008; Figure). Early glucocorticoid use and an initial CRP of 20 mg/dL or higher was associated with a significantly reduced risk of mortality or MV in unadjusted (odds ratio, 0.23; 95% CI, 0.08-0.70) and adjusted (aOR, 0.20; 95% CI, 0.06-0.67) analyses (Table 2B). Conversely, glucocorticoid treatment in patients with CRP levels less than 10 mg/dL was associated with a significantly increased risk of mortality or MV in unadjusted (OR_, 2.64; 95% CI, 1.39-5.03) and adjusted (aOR_, 3.14; 95% CI, 1.52-6.50) analyses.

The results of this study indicate that early treatment with glucocorticoids is not associated with mortality or need for MV in unselected patients with COVID-19. Subgroup analyses suggest that glucocorticoid-treated patients with markedly elevated CRP may benefit from glucocorticoid treatment, whereas those patients with lower CRP may be harmed. Our findings were consistent after adjustment for clinical characteristics. The public health implications of these findings are hard to overestimate. Given the global growth of the pandemic and that glucocorticoids are widely available and inexpensive, glucocorticoid therapy may save many thousands of lives. Equally important because we have been able to identify a group that may be harmed, some patients may be saved because glucocorticoids will not be given.

Our study reaffirms the finding of the as yet unpublished Randomised Evaluation of COVID-19 Therapy (RECOVERY) trial that there is a subset of patients with COVID-19 who benefit from treatment with glucocorticoids.¹⁰ Our study extends the findings of the RECOVERY trial in two important ways. First, in addition to finding some patients who may benefit, we also have identified patient groups that may experience harm from treatment with glucocorticoids. This finding suggests choosing the right patients for glucocorticoid treatment is critical to maximize the likelihood of benefit and minimize the risk of harm. Second, we have identified patient groups who are likely to benefit (or be harmed) on the basis of a widely available lab test (CRP).

Our results are also consistent with previous studies of patients with SARS-CoV and MERS-CoV, in which no associations between glucocorticoid treatment and mortality were found.⁷ However, the results of studies examining the effect of glucocorticoids in patients with COVID-19 are less consistent.^8,11,12

Few of the previous studies examined the effects of glucocorticoids in subgroups of patients. In our study, the improved outcomes associated with glucocorticoid use in patients with elevated CRPs is intriguing and may be clinically important. Proinflammatory cytokines, especially interleukin-6, acutely increase CRP levels. Cytokine storm syndrome (CSS) is a hyperinflammatory condition that occurs in a subset of COVID-19 patients, often resulting in multiorgan dysfunction.¹³ CRP is markedly elevated in CSS,¹⁴ and improved outcomes with glucocorticoid therapy in this subgroup may indicate benefit in this inflammatory phenotype. Patients with lower CRP are less likely to have CSS and may experience more harm than benefit associated with glucocorticoid treatment.

Several limitations are inherent to this study. Since it was done at a single center, the results may not be generalizable. As a retrospective analysis, it is subject to confounding and bias. In addition, because patients were included only if they had reached the outcome of death/MV or hospital discharge, the sample size was truncated. We believe glucocorticoid use in hospitalized patients excluded from the study reflects increased use with time because of a growing belief in their effectiveness.

Preliminary analysis from the RECOVERY study showed a reduced rate of mortality in patients randomized to dexamethasone, compared with those who received standard of care.¹⁰ These results led to the National Institutes for Health COVID-19 Treatment Guidelines Panel recommendation for dexamethasone treatment in patients with COVID-19 who require supplemental oxygen or MV.¹⁵ Our findings suggest a role for CRP to identify patients who may benefit from glucocorticoid therapy, as well as those in whom it may be harmful. Additional studies to further elucidate the role of CRP in guiding glucocorticoid therapy and to predict clinical response are needed.

The coronavirus disease of 2019 (COVID-19) pandemic has resulted in a surge in hospitalizations of patients with a novel, serious, and highly contagious infectious disease for which there is yet no proven treatment. Currently, much of the focus has been on intensive care unit (ICU) and ventilator capacity for the sickest of these patients who develop respiratory failure. However, most hospitalized patients are being cared for in general medical units.¹ Some evidence exists to describe adaptations to capacity needs outside of medical wards,^2-4 but few studies have specifically addressed the ward setting. Therefore, there is a pressing need for evidence to describe how to expand capacity and deliver medical ward–based care.

To better understand how inpatient care in the United States is adapting to the COVID-19 pandemic, we surveyed 72 sites participating in the Hospital Medicine Reengineering Network (HOMERuN), a national consortium of hospital medicine groups.⁵ We report results of this survey, carried out between April 3 and April 5, 2020.

Sites and Subjects

HOMERuN is a collaborative network of hospitalists from across the United States whose primary goal is to catalyze research and share best practices across hospital medicine groups. Using surveys of Hospital Medicine leaders, targeted medical record review, and other methods, HOMERuN’s funded research interests to date have included care transitions, workforce issues, patient and family engagement, and diagnostic errors. Sites participating in HOMERuN sites are relatively large urban academic medical centers (Appendix).

Survey Development and Deployment

We designed a focused survey that aimed to provide a snapshot of evolving operational and clinical aspects of COVID-19 care (Appendix). Domains included COVID-19 testing turnaround times, personal protective equipment (PPE) stewardship,⁶ features of respiratory isolation units (RIUs; ie, dedicated units for patients with known or suspected COVID-19), and observed effects on clinical care. We tested the instrument to ensure feasibility and clarity internally, performed brief cognitive testing with several hospital medicine leaders in HOMERuN, then disseminated the survey by email on April 3, with two follow-up emails on 2 subsequent days. Our study was deemed non–human subjects research by the University of California, San Francisco, Committee on Human Research. Descriptive statistics were used to characterize survey responses.

Of 72 hospitals surveyed, 51 (71%) responded. Mean hospital bed count was 940, three were safety-net hospitals, and one was a community-based teaching center; responding and nonresponding hospitals did not differ significantly in terms of bed count (Appendix).

Health System Adaptations, Testing, and PPE Status

Nearly all responding hospitals (46 of 51; 90%) had RIUs for patients with known or suspected COVID-19 (Table 1). Nearly all hospitals took steps to keep potentially sick healthcare providers from infecting others (eg, staying home if sick or exposed). Among respondents, 32% had rapid response teams, 24% had respiratory therapy teams, and 29% had case management teams that were dedicated to COVID-19 care. Thirty-two (63%) had developed models, such as ethics or palliative care consult services, to assist with difficult resource-allocation decisions (eg, how to prioritize ventilator use if demand exceeded supply). Twenty-three (45%) had developed post-acute care monitoring programs dedicated to COVID-19 patients.

At the time of our survey, only 2 sites (4%) reported COVID-19 test time turnaround under 1 hour, and 15 (30%) reported turnaround in less than 6 hours. Of the 29 sites able to provide estimates of PPE stockpile, 14 (48%) reported a supply of 2 weeks or less. The most common approaches to PPE stewardship focused on reuse of masks and face shields if not obviously soiled, centralizing PPE distribution, and disinfecting or sterilizing masks. Ten sites (20%) were utilizing 3-D printed masks, while 10% used homemade face shields or masks.

Characteristics of COVID-19 RIUs

Forty-six hospitals (90% of all respondents) in our cohort had developed RIUs at the time of survey administration. The earliest RIU implementation date was February 10, 2020, and the most recent was launched on the day of our survey. Admission to RIUs was primarily based on clinical factors associated with known or suspected COVID-19 infection (Table 2). The number of non–critical care RIU beds among locations at that time ranged from 10 or less to more than 50. The mean number of hospitalist attendings caring for patients in the RIUs was 10.2, with a mean 4.1 advanced practice providers, 5.5 residents, and 0 medical students. The number of planned patients per attending was typically 5 to 15. Nurses and physicians typically rounded separately. Medical distancing (eg, reducing patient room entry) was accomplished most commonly by grouped timing of medication administration (76% of sites), video links to room outside of rounding times (54% of sites), the use of video or telemedicine during rounds (17%), and clustering of activities such as medication administration or phlebotomy. The most common criteria prompting discharge from the RIU were a negative COVID-19 test (59%) and hospital discharge (57%), though comments from many respondents suggested that discharge criteria were changing rapidly.

Effects of Isolation Measures on In-Room Encounters and Diagnostic Processes

More than 90% of sites reported decreases in in-room encounter frequency across all provider types whether as a result of policies in place or not. Reductions were reported among hospitalists, advanced practice providers, residents, consultants, and therapists (Table 3). Reduced room entry most often resulted from an established or developing policy, but many noted reduced room entry without formal policies in place. Nearly all sites reported moving specialty consultations to phone or video evaluations. Diagnostic error was commonly reported, with missed non–COVID-19 medical diagnoses among COVID-19 infected patients being reported by 22 sites (46%) and missed COVID-19 diagnoses in patients admitted for other reasons by 22 sites (45%).

In this study of medical wards at academic medical centers, we found that, in response to the COVID-19 pandemic, hospitals made several changes in a short period of time to adapt to the crisis. These included implementation and rapid expansion of dedicated RIUs, greatly expanded use of inpatient telehealth for patient assessments and consultation, implementation of other approaches to minimize room entry (such as grouping in-room activities), and deployment of ethics consultation services to help manage issues around potential scarcity of life-saving measures such as ventilators. We also found that availability of PPE and timely testing was limited. Finally, a large proportion of sites reported potential diagnostic problems in the assessment of both patients suspected and those not suspected of having COVID-19.

RIUs are emerging as a primary modality for caring for non-ICU COVID-19 patients, though they never involved medical students; we hope the role of students in particular will increase as new models of training emerge in response to the pandemic.⁷ In contrast, telemedicine evolved rapidly to hold a substantial role in RIUs, with both ward and specialty teams using video visit technology to communicate with patients. COVID-19 has been viewed as a perfect use case for outpatient telemedicine,⁸ and a growing number of studies are examining its outpatient use^9,10; however, to date, somewhat less attention has been paid to inpatient deployment. Although our data suggest telemedicine has found a prominent place in RIUs, it remains to be seen whether it is associated with differences in patient or provider outcomes. For example, deficiencies in the physical examination, limited face-to-face contact, and lack of physical presence could all affect the patient–provider relationship, patient engagement, and the accuracy of the diagnostic process.

Our data suggest the possibility of missing non–COVID-19 diagnoses in patients suspected of COVID-19 and missing COVID-19 in those admitted for nonrespiratory reasons. The latter may be addressed as routine COVID-19 screening of admitted patients becomes commonplace. For the former, however, it is possible that physicians are “anchoring” their thinking on COVID-19 to the exclusion of other diagnoses, that physicians are not fully aware of complications unique to COVID-19 infection (such as thromboembolism), and/or that the above-mentioned limitations of telemedicine have decreased diagnostic performance.

Although PPE stockpile data were not easily available for some sites, a distressingly large number reported stockpiles of 2 weeks or less, with reuse being the most common approach to extending PPE supply. We also found it concerning that 43% of hospital leaders did not know their stockpile data; we believe this is an important question that hospital leaders need to be asking. Most sites in our study reported test turnaround times of longer than 6 hours; lack of rapid COVID-19 testing further stresses PPE stockpile and may slow patients’ transition out of the RIU or discharge to home.

Our study has several limitations, including the evolving nature of the pandemic and rapid adaptations of care systems in the pandemic’s surge phase. However, we attempted to frame our questions in ways that provided a focused snapshot of care. Furthermore, respondents may not have had exhaustive knowledge of their institution’s COVID-19 response strategies, but most were the directors of their hospitalist services, and we encouraged the respondents to confer with others to gather high-fidelity data. Finally, as a survey of large academic medical centers, our results may not apply to nonacademic centers.

Approaches to caring for non-ICU patients during the COVID-19 pandemic are rapidly evolving. Expansion of RIUs and developing the workforce to support them has been a primary focus, with rapid innovation in use of technology emerging as a critical adaptation while PPE limitations persist and needs for “medical distancing” continue to grow. Although rates of missed COVID-19 diagnoses will likely be reduced with testing and systems improvements, physicians and systems will also need to consider how to utilize emerging technology in ways that can improve clinical care and provider safety while aiding diagnostic thinking. This survey illustrates the rapid adaptations made by our hospitals in response to the pandemic; ongoing adaptation will likely be needed to optimally care for hospitalized patients with COVID-19 while the pandemic continues to evolve.

Thanks to members of the HOMERuN COVID-19 Collaborative Group: Baylor Scott & White Medical Center – Temple, Texas - Tresa McNeal MD; Beth Israel Deaconess Medical Center - Shani Herzig MD MPH, Joseph Li MD, Julius Yang MD PhD; Brigham and Women’s Hospital - Christopher Roy MD, Jeffrey Schnipper MD MPH; Cedars-Sinai Medical Center - Ed Seferian MD, ; ChristianaCare - Surekha Bhamidipati MD; Cleveland Clinic - Matthew Pappas MD MPH; Dartmouth-Hitchcock Medical Center - Jonathan Lurie MD MS; Dell Medical School at The University of Texas at Austin - Chris Moriates MD, Luci Leykum MD MBA MSc; Denver Health and Hospitals Authority - Diana Mancini MD; Emory University Hospital - Dan Hunt MD; Johns Hopkins Hospital - Daniel J Brotman MD, Zishan K Siddiqui MD, Shaker Eid MD MBA; Maine Medical Center - Daniel A Meyer MD, Robert Trowbridge MD; Massachusetts General Hospital - Melissa Mattison MD; Mayo Clinic Rochester – Caroline Burton MD, Sagar Dugani MD PhD; Medical College of Wisconsin - Sanjay Bhandari MD; Miriam Hospital - Kwame Dapaah-Afriyie MD MBA; Mount Sinai Hospital - Andrew Dunn MD; NorthShore - David Lovinger MD; Northwestern Memorial Hospital - Kevin O’Leary MD MS; Ohio State University Wexner Medical Center - Eric Schumacher DO; Oregon Health & Science University - Angela Alday MD; Penn Medicine - Ryan Greysen MD MHS MA; Rutgers- Robert Wood Johnson University Hospital - Michael Steinberg MD MPH; Stanford University School of Medicine - Neera Ahuja MD; Tulane Hospital and University Medical Center - Geraldine Ménard MD; UC San Diego Health - Ian Jenkins MD; UC Los Angeles Health - Michael Lazarus MD, Magdalena E. Ptaszny, MD; UC San Francisco Health - Bradley A Sharpe, MD, Margaret Fang MD MPH; UK HealthCare - Mark Williams MD MHM, John Romond MD; University of Chicago – David Meltzer MD PhD, Gregory Ruhnke MD; University of Colorado - Marisha Burden MD; University of Florida - Nila Radhakrishnan MD; University of Iowa Hospitals and Clinics - Kevin Glenn MD MS; University of Miami - Efren Manjarrez MD; University of Michigan - Vineet Chopra MD MSc, Valerie Vaughn MD MSc; University of Missouri-Columbia Hospital - Hasan Naqvi MD; University of Nebraska Medical Center - Chad Vokoun MD; University of North Carolina at Chapel Hill - David Hemsey MD; University of Pittsburgh Medical Center - Gena Marie Walker MD; University of Vermont Medical Center - Steven Grant MD; University of Washington Medical Center - Christopher Kim MD MBA, Andrew White MD; University of Washington-Harborview Medical Center - Maralyssa Bann MD; University of Wisconsin Hospital and Clinics - David Sterken MD, Farah Kaiksow MD MPP, Ann Sheehy MD MS, Jordan Kenik MD MPH; UW Northwest Campus - Ben Wolpaw MD; Vanderbilt University Medical Center - Sunil Kripalani MD MSc, Eduard E Vasilevskis MD, Kathleene T Wooldridge MD MPH; Wake Forest Baptist Health - Erik Summers MD; Washington University St. Louis - Michael Lin MD; Weill Cornell - Justin Choi MD; Yale New Haven Hospital - William Cushing MA, Chris Sankey MD; Zuckerberg San Francisco General Hospital - Sumant Ranji MD.

The coronavirus disease of 2019 (COVID-19) pandemic has resulted in a surge in hospitalizations of patients with a novel, serious, and highly contagious infectious disease for which there is yet no proven treatment. Currently, much of the focus has been on intensive care unit (ICU) and ventilator capacity for the sickest of these patients who develop respiratory failure. However, most hospitalized patients are being cared for in general medical units.¹ Some evidence exists to describe adaptations to capacity needs outside of medical wards,^2-4 but few studies have specifically addressed the ward setting. Therefore, there is a pressing need for evidence to describe how to expand capacity and deliver medical ward–based care.

To better understand how inpatient care in the United States is adapting to the COVID-19 pandemic, we surveyed 72 sites participating in the Hospital Medicine Reengineering Network (HOMERuN), a national consortium of hospital medicine groups.⁵ We report results of this survey, carried out between April 3 and April 5, 2020.

Sites and Subjects

HOMERuN is a collaborative network of hospitalists from across the United States whose primary goal is to catalyze research and share best practices across hospital medicine groups. Using surveys of Hospital Medicine leaders, targeted medical record review, and other methods, HOMERuN’s funded research interests to date have included care transitions, workforce issues, patient and family engagement, and diagnostic errors. Sites participating in HOMERuN sites are relatively large urban academic medical centers (Appendix).

Survey Development and Deployment

We designed a focused survey that aimed to provide a snapshot of evolving operational and clinical aspects of COVID-19 care (Appendix). Domains included COVID-19 testing turnaround times, personal protective equipment (PPE) stewardship,⁶ features of respiratory isolation units (RIUs; ie, dedicated units for patients with known or suspected COVID-19), and observed effects on clinical care. We tested the instrument to ensure feasibility and clarity internally, performed brief cognitive testing with several hospital medicine leaders in HOMERuN, then disseminated the survey by email on April 3, with two follow-up emails on 2 subsequent days. Our study was deemed non–human subjects research by the University of California, San Francisco, Committee on Human Research. Descriptive statistics were used to characterize survey responses.

Of 72 hospitals surveyed, 51 (71%) responded. Mean hospital bed count was 940, three were safety-net hospitals, and one was a community-based teaching center; responding and nonresponding hospitals did not differ significantly in terms of bed count (Appendix).

Health System Adaptations, Testing, and PPE Status

Nearly all responding hospitals (46 of 51; 90%) had RIUs for patients with known or suspected COVID-19 (Table 1). Nearly all hospitals took steps to keep potentially sick healthcare providers from infecting others (eg, staying home if sick or exposed). Among respondents, 32% had rapid response teams, 24% had respiratory therapy teams, and 29% had case management teams that were dedicated to COVID-19 care. Thirty-two (63%) had developed models, such as ethics or palliative care consult services, to assist with difficult resource-allocation decisions (eg, how to prioritize ventilator use if demand exceeded supply). Twenty-three (45%) had developed post-acute care monitoring programs dedicated to COVID-19 patients.

At the time of our survey, only 2 sites (4%) reported COVID-19 test time turnaround under 1 hour, and 15 (30%) reported turnaround in less than 6 hours. Of the 29 sites able to provide estimates of PPE stockpile, 14 (48%) reported a supply of 2 weeks or less. The most common approaches to PPE stewardship focused on reuse of masks and face shields if not obviously soiled, centralizing PPE distribution, and disinfecting or sterilizing masks. Ten sites (20%) were utilizing 3-D printed masks, while 10% used homemade face shields or masks.

Characteristics of COVID-19 RIUs

Forty-six hospitals (90% of all respondents) in our cohort had developed RIUs at the time of survey administration. The earliest RIU implementation date was February 10, 2020, and the most recent was launched on the day of our survey. Admission to RIUs was primarily based on clinical factors associated with known or suspected COVID-19 infection (Table 2). The number of non–critical care RIU beds among locations at that time ranged from 10 or less to more than 50. The mean number of hospitalist attendings caring for patients in the RIUs was 10.2, with a mean 4.1 advanced practice providers, 5.5 residents, and 0 medical students. The number of planned patients per attending was typically 5 to 15. Nurses and physicians typically rounded separately. Medical distancing (eg, reducing patient room entry) was accomplished most commonly by grouped timing of medication administration (76% of sites), video links to room outside of rounding times (54% of sites), the use of video or telemedicine during rounds (17%), and clustering of activities such as medication administration or phlebotomy. The most common criteria prompting discharge from the RIU were a negative COVID-19 test (59%) and hospital discharge (57%), though comments from many respondents suggested that discharge criteria were changing rapidly.

Effects of Isolation Measures on In-Room Encounters and Diagnostic Processes

More than 90% of sites reported decreases in in-room encounter frequency across all provider types whether as a result of policies in place or not. Reductions were reported among hospitalists, advanced practice providers, residents, consultants, and therapists (Table 3). Reduced room entry most often resulted from an established or developing policy, but many noted reduced room entry without formal policies in place. Nearly all sites reported moving specialty consultations to phone or video evaluations. Diagnostic error was commonly reported, with missed non–COVID-19 medical diagnoses among COVID-19 infected patients being reported by 22 sites (46%) and missed COVID-19 diagnoses in patients admitted for other reasons by 22 sites (45%).

In this study of medical wards at academic medical centers, we found that, in response to the COVID-19 pandemic, hospitals made several changes in a short period of time to adapt to the crisis. These included implementation and rapid expansion of dedicated RIUs, greatly expanded use of inpatient telehealth for patient assessments and consultation, implementation of other approaches to minimize room entry (such as grouping in-room activities), and deployment of ethics consultation services to help manage issues around potential scarcity of life-saving measures such as ventilators. We also found that availability of PPE and timely testing was limited. Finally, a large proportion of sites reported potential diagnostic problems in the assessment of both patients suspected and those not suspected of having COVID-19.

RIUs are emerging as a primary modality for caring for non-ICU COVID-19 patients, though they never involved medical students; we hope the role of students in particular will increase as new models of training emerge in response to the pandemic.⁷ In contrast, telemedicine evolved rapidly to hold a substantial role in RIUs, with both ward and specialty teams using video visit technology to communicate with patients. COVID-19 has been viewed as a perfect use case for outpatient telemedicine,⁸ and a growing number of studies are examining its outpatient use^9,10; however, to date, somewhat less attention has been paid to inpatient deployment. Although our data suggest telemedicine has found a prominent place in RIUs, it remains to be seen whether it is associated with differences in patient or provider outcomes. For example, deficiencies in the physical examination, limited face-to-face contact, and lack of physical presence could all affect the patient–provider relationship, patient engagement, and the accuracy of the diagnostic process.

Our data suggest the possibility of missing non–COVID-19 diagnoses in patients suspected of COVID-19 and missing COVID-19 in those admitted for nonrespiratory reasons. The latter may be addressed as routine COVID-19 screening of admitted patients becomes commonplace. For the former, however, it is possible that physicians are “anchoring” their thinking on COVID-19 to the exclusion of other diagnoses, that physicians are not fully aware of complications unique to COVID-19 infection (such as thromboembolism), and/or that the above-mentioned limitations of telemedicine have decreased diagnostic performance.

Although PPE stockpile data were not easily available for some sites, a distressingly large number reported stockpiles of 2 weeks or less, with reuse being the most common approach to extending PPE supply. We also found it concerning that 43% of hospital leaders did not know their stockpile data; we believe this is an important question that hospital leaders need to be asking. Most sites in our study reported test turnaround times of longer than 6 hours; lack of rapid COVID-19 testing further stresses PPE stockpile and may slow patients’ transition out of the RIU or discharge to home.

Our study has several limitations, including the evolving nature of the pandemic and rapid adaptations of care systems in the pandemic’s surge phase. However, we attempted to frame our questions in ways that provided a focused snapshot of care. Furthermore, respondents may not have had exhaustive knowledge of their institution’s COVID-19 response strategies, but most were the directors of their hospitalist services, and we encouraged the respondents to confer with others to gather high-fidelity data. Finally, as a survey of large academic medical centers, our results may not apply to nonacademic centers.

Approaches to caring for non-ICU patients during the COVID-19 pandemic are rapidly evolving. Expansion of RIUs and developing the workforce to support them has been a primary focus, with rapid innovation in use of technology emerging as a critical adaptation while PPE limitations persist and needs for “medical distancing” continue to grow. Although rates of missed COVID-19 diagnoses will likely be reduced with testing and systems improvements, physicians and systems will also need to consider how to utilize emerging technology in ways that can improve clinical care and provider safety while aiding diagnostic thinking. This survey illustrates the rapid adaptations made by our hospitals in response to the pandemic; ongoing adaptation will likely be needed to optimally care for hospitalized patients with COVID-19 while the pandemic continues to evolve.

Thanks to members of the HOMERuN COVID-19 Collaborative Group: Baylor Scott & White Medical Center – Temple, Texas - Tresa McNeal MD; Beth Israel Deaconess Medical Center - Shani Herzig MD MPH, Joseph Li MD, Julius Yang MD PhD; Brigham and Women’s Hospital - Christopher Roy MD, Jeffrey Schnipper MD MPH; Cedars-Sinai Medical Center - Ed Seferian MD, ; ChristianaCare - Surekha Bhamidipati MD; Cleveland Clinic - Matthew Pappas MD MPH; Dartmouth-Hitchcock Medical Center - Jonathan Lurie MD MS; Dell Medical School at The University of Texas at Austin - Chris Moriates MD, Luci Leykum MD MBA MSc; Denver Health and Hospitals Authority - Diana Mancini MD; Emory University Hospital - Dan Hunt MD; Johns Hopkins Hospital - Daniel J Brotman MD, Zishan K Siddiqui MD, Shaker Eid MD MBA; Maine Medical Center - Daniel A Meyer MD, Robert Trowbridge MD; Massachusetts General Hospital - Melissa Mattison MD; Mayo Clinic Rochester – Caroline Burton MD, Sagar Dugani MD PhD; Medical College of Wisconsin - Sanjay Bhandari MD; Miriam Hospital - Kwame Dapaah-Afriyie MD MBA; Mount Sinai Hospital - Andrew Dunn MD; NorthShore - David Lovinger MD; Northwestern Memorial Hospital - Kevin O’Leary MD MS; Ohio State University Wexner Medical Center - Eric Schumacher DO; Oregon Health & Science University - Angela Alday MD; Penn Medicine - Ryan Greysen MD MHS MA; Rutgers- Robert Wood Johnson University Hospital - Michael Steinberg MD MPH; Stanford University School of Medicine - Neera Ahuja MD; Tulane Hospital and University Medical Center - Geraldine Ménard MD; UC San Diego Health - Ian Jenkins MD; UC Los Angeles Health - Michael Lazarus MD, Magdalena E. Ptaszny, MD; UC San Francisco Health - Bradley A Sharpe, MD, Margaret Fang MD MPH; UK HealthCare - Mark Williams MD MHM, John Romond MD; University of Chicago – David Meltzer MD PhD, Gregory Ruhnke MD; University of Colorado - Marisha Burden MD; University of Florida - Nila Radhakrishnan MD; University of Iowa Hospitals and Clinics - Kevin Glenn MD MS; University of Miami - Efren Manjarrez MD; University of Michigan - Vineet Chopra MD MSc, Valerie Vaughn MD MSc; University of Missouri-Columbia Hospital - Hasan Naqvi MD; University of Nebraska Medical Center - Chad Vokoun MD; University of North Carolina at Chapel Hill - David Hemsey MD; University of Pittsburgh Medical Center - Gena Marie Walker MD; University of Vermont Medical Center - Steven Grant MD; University of Washington Medical Center - Christopher Kim MD MBA, Andrew White MD; University of Washington-Harborview Medical Center - Maralyssa Bann MD; University of Wisconsin Hospital and Clinics - David Sterken MD, Farah Kaiksow MD MPP, Ann Sheehy MD MS, Jordan Kenik MD MPH; UW Northwest Campus - Ben Wolpaw MD; Vanderbilt University Medical Center - Sunil Kripalani MD MSc, Eduard E Vasilevskis MD, Kathleene T Wooldridge MD MPH; Wake Forest Baptist Health - Erik Summers MD; Washington University St. Louis - Michael Lin MD; Weill Cornell - Justin Choi MD; Yale New Haven Hospital - William Cushing MA, Chris Sankey MD; Zuckerberg San Francisco General Hospital - Sumant Ranji MD.

The coronavirus disease of 2019 (COVID-19) pandemic has resulted in a surge in hospitalizations of patients with a novel, serious, and highly contagious infectious disease for which there is yet no proven treatment. Currently, much of the focus has been on intensive care unit (ICU) and ventilator capacity for the sickest of these patients who develop respiratory failure. However, most hospitalized patients are being cared for in general medical units.¹ Some evidence exists to describe adaptations to capacity needs outside of medical wards,^2-4 but few studies have specifically addressed the ward setting. Therefore, there is a pressing need for evidence to describe how to expand capacity and deliver medical ward–based care.

To better understand how inpatient care in the United States is adapting to the COVID-19 pandemic, we surveyed 72 sites participating in the Hospital Medicine Reengineering Network (HOMERuN), a national consortium of hospital medicine groups.⁵ We report results of this survey, carried out between April 3 and April 5, 2020.

Sites and Subjects

HOMERuN is a collaborative network of hospitalists from across the United States whose primary goal is to catalyze research and share best practices across hospital medicine groups. Using surveys of Hospital Medicine leaders, targeted medical record review, and other methods, HOMERuN’s funded research interests to date have included care transitions, workforce issues, patient and family engagement, and diagnostic errors. Sites participating in HOMERuN sites are relatively large urban academic medical centers (Appendix).

Survey Development and Deployment

We designed a focused survey that aimed to provide a snapshot of evolving operational and clinical aspects of COVID-19 care (Appendix). Domains included COVID-19 testing turnaround times, personal protective equipment (PPE) stewardship,⁶ features of respiratory isolation units (RIUs; ie, dedicated units for patients with known or suspected COVID-19), and observed effects on clinical care. We tested the instrument to ensure feasibility and clarity internally, performed brief cognitive testing with several hospital medicine leaders in HOMERuN, then disseminated the survey by email on April 3, with two follow-up emails on 2 subsequent days. Our study was deemed non–human subjects research by the University of California, San Francisco, Committee on Human Research. Descriptive statistics were used to characterize survey responses.

Of 72 hospitals surveyed, 51 (71%) responded. Mean hospital bed count was 940, three were safety-net hospitals, and one was a community-based teaching center; responding and nonresponding hospitals did not differ significantly in terms of bed count (Appendix).

Health System Adaptations, Testing, and PPE Status

Nearly all responding hospitals (46 of 51; 90%) had RIUs for patients with known or suspected COVID-19 (Table 1). Nearly all hospitals took steps to keep potentially sick healthcare providers from infecting others (eg, staying home if sick or exposed). Among respondents, 32% had rapid response teams, 24% had respiratory therapy teams, and 29% had case management teams that were dedicated to COVID-19 care. Thirty-two (63%) had developed models, such as ethics or palliative care consult services, to assist with difficult resource-allocation decisions (eg, how to prioritize ventilator use if demand exceeded supply). Twenty-three (45%) had developed post-acute care monitoring programs dedicated to COVID-19 patients.

At the time of our survey, only 2 sites (4%) reported COVID-19 test time turnaround under 1 hour, and 15 (30%) reported turnaround in less than 6 hours. Of the 29 sites able to provide estimates of PPE stockpile, 14 (48%) reported a supply of 2 weeks or less. The most common approaches to PPE stewardship focused on reuse of masks and face shields if not obviously soiled, centralizing PPE distribution, and disinfecting or sterilizing masks. Ten sites (20%) were utilizing 3-D printed masks, while 10% used homemade face shields or masks.

Characteristics of COVID-19 RIUs

Forty-six hospitals (90% of all respondents) in our cohort had developed RIUs at the time of survey administration. The earliest RIU implementation date was February 10, 2020, and the most recent was launched on the day of our survey. Admission to RIUs was primarily based on clinical factors associated with known or suspected COVID-19 infection (Table 2). The number of non–critical care RIU beds among locations at that time ranged from 10 or less to more than 50. The mean number of hospitalist attendings caring for patients in the RIUs was 10.2, with a mean 4.1 advanced practice providers, 5.5 residents, and 0 medical students. The number of planned patients per attending was typically 5 to 15. Nurses and physicians typically rounded separately. Medical distancing (eg, reducing patient room entry) was accomplished most commonly by grouped timing of medication administration (76% of sites), video links to room outside of rounding times (54% of sites), the use of video or telemedicine during rounds (17%), and clustering of activities such as medication administration or phlebotomy. The most common criteria prompting discharge from the RIU were a negative COVID-19 test (59%) and hospital discharge (57%), though comments from many respondents suggested that discharge criteria were changing rapidly.

Effects of Isolation Measures on In-Room Encounters and Diagnostic Processes

More than 90% of sites reported decreases in in-room encounter frequency across all provider types whether as a result of policies in place or not. Reductions were reported among hospitalists, advanced practice providers, residents, consultants, and therapists (Table 3). Reduced room entry most often resulted from an established or developing policy, but many noted reduced room entry without formal policies in place. Nearly all sites reported moving specialty consultations to phone or video evaluations. Diagnostic error was commonly reported, with missed non–COVID-19 medical diagnoses among COVID-19 infected patients being reported by 22 sites (46%) and missed COVID-19 diagnoses in patients admitted for other reasons by 22 sites (45%).

In this study of medical wards at academic medical centers, we found that, in response to the COVID-19 pandemic, hospitals made several changes in a short period of time to adapt to the crisis. These included implementation and rapid expansion of dedicated RIUs, greatly expanded use of inpatient telehealth for patient assessments and consultation, implementation of other approaches to minimize room entry (such as grouping in-room activities), and deployment of ethics consultation services to help manage issues around potential scarcity of life-saving measures such as ventilators. We also found that availability of PPE and timely testing was limited. Finally, a large proportion of sites reported potential diagnostic problems in the assessment of both patients suspected and those not suspected of having COVID-19.

RIUs are emerging as a primary modality for caring for non-ICU COVID-19 patients, though they never involved medical students; we hope the role of students in particular will increase as new models of training emerge in response to the pandemic.⁷ In contrast, telemedicine evolved rapidly to hold a substantial role in RIUs, with both ward and specialty teams using video visit technology to communicate with patients. COVID-19 has been viewed as a perfect use case for outpatient telemedicine,⁸ and a growing number of studies are examining its outpatient use^9,10; however, to date, somewhat less attention has been paid to inpatient deployment. Although our data suggest telemedicine has found a prominent place in RIUs, it remains to be seen whether it is associated with differences in patient or provider outcomes. For example, deficiencies in the physical examination, limited face-to-face contact, and lack of physical presence could all affect the patient–provider relationship, patient engagement, and the accuracy of the diagnostic process.

Our data suggest the possibility of missing non–COVID-19 diagnoses in patients suspected of COVID-19 and missing COVID-19 in those admitted for nonrespiratory reasons. The latter may be addressed as routine COVID-19 screening of admitted patients becomes commonplace. For the former, however, it is possible that physicians are “anchoring” their thinking on COVID-19 to the exclusion of other diagnoses, that physicians are not fully aware of complications unique to COVID-19 infection (such as thromboembolism), and/or that the above-mentioned limitations of telemedicine have decreased diagnostic performance.

Although PPE stockpile data were not easily available for some sites, a distressingly large number reported stockpiles of 2 weeks or less, with reuse being the most common approach to extending PPE supply. We also found it concerning that 43% of hospital leaders did not know their stockpile data; we believe this is an important question that hospital leaders need to be asking. Most sites in our study reported test turnaround times of longer than 6 hours; lack of rapid COVID-19 testing further stresses PPE stockpile and may slow patients’ transition out of the RIU or discharge to home.

Our study has several limitations, including the evolving nature of the pandemic and rapid adaptations of care systems in the pandemic’s surge phase. However, we attempted to frame our questions in ways that provided a focused snapshot of care. Furthermore, respondents may not have had exhaustive knowledge of their institution’s COVID-19 response strategies, but most were the directors of their hospitalist services, and we encouraged the respondents to confer with others to gather high-fidelity data. Finally, as a survey of large academic medical centers, our results may not apply to nonacademic centers.

Approaches to caring for non-ICU patients during the COVID-19 pandemic are rapidly evolving. Expansion of RIUs and developing the workforce to support them has been a primary focus, with rapid innovation in use of technology emerging as a critical adaptation while PPE limitations persist and needs for “medical distancing” continue to grow. Although rates of missed COVID-19 diagnoses will likely be reduced with testing and systems improvements, physicians and systems will also need to consider how to utilize emerging technology in ways that can improve clinical care and provider safety while aiding diagnostic thinking. This survey illustrates the rapid adaptations made by our hospitals in response to the pandemic; ongoing adaptation will likely be needed to optimally care for hospitalized patients with COVID-19 while the pandemic continues to evolve.

Thanks to members of the HOMERuN COVID-19 Collaborative Group: Baylor Scott & White Medical Center – Temple, Texas - Tresa McNeal MD; Beth Israel Deaconess Medical Center - Shani Herzig MD MPH, Joseph Li MD, Julius Yang MD PhD; Brigham and Women’s Hospital - Christopher Roy MD, Jeffrey Schnipper MD MPH; Cedars-Sinai Medical Center - Ed Seferian MD, ; ChristianaCare - Surekha Bhamidipati MD; Cleveland Clinic - Matthew Pappas MD MPH; Dartmouth-Hitchcock Medical Center - Jonathan Lurie MD MS; Dell Medical School at The University of Texas at Austin - Chris Moriates MD, Luci Leykum MD MBA MSc; Denver Health and Hospitals Authority - Diana Mancini MD; Emory University Hospital - Dan Hunt MD; Johns Hopkins Hospital - Daniel J Brotman MD, Zishan K Siddiqui MD, Shaker Eid MD MBA; Maine Medical Center - Daniel A Meyer MD, Robert Trowbridge MD; Massachusetts General Hospital - Melissa Mattison MD; Mayo Clinic Rochester – Caroline Burton MD, Sagar Dugani MD PhD; Medical College of Wisconsin - Sanjay Bhandari MD; Miriam Hospital - Kwame Dapaah-Afriyie MD MBA; Mount Sinai Hospital - Andrew Dunn MD; NorthShore - David Lovinger MD; Northwestern Memorial Hospital - Kevin O’Leary MD MS; Ohio State University Wexner Medical Center - Eric Schumacher DO; Oregon Health & Science University - Angela Alday MD; Penn Medicine - Ryan Greysen MD MHS MA; Rutgers- Robert Wood Johnson University Hospital - Michael Steinberg MD MPH; Stanford University School of Medicine - Neera Ahuja MD; Tulane Hospital and University Medical Center - Geraldine Ménard MD; UC San Diego Health - Ian Jenkins MD; UC Los Angeles Health - Michael Lazarus MD, Magdalena E. Ptaszny, MD; UC San Francisco Health - Bradley A Sharpe, MD, Margaret Fang MD MPH; UK HealthCare - Mark Williams MD MHM, John Romond MD; University of Chicago – David Meltzer MD PhD, Gregory Ruhnke MD; University of Colorado - Marisha Burden MD; University of Florida - Nila Radhakrishnan MD; University of Iowa Hospitals and Clinics - Kevin Glenn MD MS; University of Miami - Efren Manjarrez MD; University of Michigan - Vineet Chopra MD MSc, Valerie Vaughn MD MSc; University of Missouri-Columbia Hospital - Hasan Naqvi MD; University of Nebraska Medical Center - Chad Vokoun MD; University of North Carolina at Chapel Hill - David Hemsey MD; University of Pittsburgh Medical Center - Gena Marie Walker MD; University of Vermont Medical Center - Steven Grant MD; University of Washington Medical Center - Christopher Kim MD MBA, Andrew White MD; University of Washington-Harborview Medical Center - Maralyssa Bann MD; University of Wisconsin Hospital and Clinics - David Sterken MD, Farah Kaiksow MD MPP, Ann Sheehy MD MS, Jordan Kenik MD MPH; UW Northwest Campus - Ben Wolpaw MD; Vanderbilt University Medical Center - Sunil Kripalani MD MSc, Eduard E Vasilevskis MD, Kathleene T Wooldridge MD MPH; Wake Forest Baptist Health - Erik Summers MD; Washington University St. Louis - Michael Lin MD; Weill Cornell - Justin Choi MD; Yale New Haven Hospital - William Cushing MA, Chris Sankey MD; Zuckerberg San Francisco General Hospital - Sumant Ranji MD.

Frequent daily laboratory testing for inpatients contributes to excessive costs,¹ anemia,² and unnecessary testing.³ The ABIM Foundation’s Choosing Wisely^® campaign recommends avoiding routine labs, like complete blood counts (CBCs) and basic metabolic panels (BMP), in the face of clinical and laboratory stability.^4,5 Prior interventions have reduced unnecessary labs without adverse outcomes.^6-8

In addition to lab frequency, hospitalized patients face suboptimal lab timing. Labs are often ordered as early as 4 am at many institutions.^9,10 This practice disrupts sleep, undermining patient health.^11-13 While prior interventions have reduced daily phlebotomy, few have optimized lab timing for patient sleep.¹⁰ No study has harnessed the electronic health record (EHR) to optimize frequency and timing of labs simultaneously.¹⁴ We aimed to determine the effectiveness of a multicomponent intervention, called Order SMARTT (Sleep: Making Appropriate Reductions in Testing and Timing), to reduce frequency and optimize timing of daily routine labs for medical inpatients.

Setting

This study was conducted on the University of Chicago Medicine (UCM) general medicine services, which consisted of a resident-covered service supervised by general medicine, subspecialist, or hospitalist attendings and a hospitalist service staffed by hospitalists and advanced practice providers.

Development of Order SMARTT

To inform intervention development, we surveyed providers about lab-ordering preferences with use of questions from a prior survey to provide a benchmark (Appendix Table 2).¹⁵ While reducing lab frequency was supported, the modal response for how frequently a stable patient should receive routine labs was every 48 hours (Appendix Table 2). Therefore, we hypothesized that labs ordered every 48 hours may be popular. Taking labs every 48 hours would not require an urgent 4 am draw, so we created a 48-hour 6 am phlebotomy option to “step down” from daily labs. To promote these options, we created two EHR tools: First, an “Order Sleep” shortcut was launched in March 2018 by which physicians could type “sleep” in routine lab orders and three sleep-friendly options would become available (a 48-hour 6 am draw, a daily 6 am draw, or a daily 10 pm draw), and second, a “4 am Labs” column and icon on the electronic patient list to signal who had 4 am labs ordered was launched May 2018 (Appendix Table 1).

Physician Education

We created a 20-minute presentation on the harms of excessive labs and the benefits of sleep-friendly ordering. Instructional Order SMARTT posters were posted in clinician workrooms that emphasized forgoing labs on stable patients and using the “Order Sleep” shortcut when nonurgent labs were needed.

Labs Utilization Data

We used Epic Systems software (Verona, Wisconsin) and our institutional Tableau scorecard to obtain data on CBC and BMP ordering, patient census, and demographics for medical inpatients between July 1, 2017, and November 1, 2018.

Cost Analysis

Costs of lab tests (actual cost to our institution) were obtained from our institutional phlebotomy services’ estimates of direct variable labor and benefits costs and direct variable supplies cost.

Statistical Analysis

Data analysis was performed with SAS version 9.4 statistical software (Cary, North Carolina, USA) and R version 3.6.2 (Vienna, Austria). Descriptive statistics were used to summarize data. Surveys were analyzed using chi-square tests for categorical variables and two-sample t tests for continuous variables. For lab ordering data, interrupted time series analyses (ITSA) were used to determine the changes in ordering practices with the implementation of the two interventions controlling for service lines (resident vs hospitalist service). ITSA enables examination of changes in lab ordering while controlling for time. The AUTOREG function in SAS was used to build the model and estimate final parameters. This function automatically tests for autocorrelation, heteroscedasticity, and estimates any autoregressive parameters required in the model. Our main model tested the association between our two separate interventions on ordering practices, controlling for service (hospitalist or resident).¹⁶

Of 125 residents, 82 (65.6%) attended the session and completed the survey. Attendance and response rate for hospitalists was 80% (16 of 20). Similar to a prior study, many residents (73.1%) reported they would be comfortable if patients received less daily laboratory testing (Appendix Table 2).

We reviewed data from 7,045 total patients over 50,951 total patient days between July1, 2017, and November 1, 2018 (Appendix Table 3).

Total Lab Draws

After accounting for total patient days, we saw 26.3% reduction on average in total lab draws per patient-day per week postintervention (4.68 before vs 3.45 after; difference, 1.23; 95% CI, 0.82-1.63; P < .05; Appendix Table 3). When total lab draws were stratified by service, we saw 28% reduction on average in total lab draws per patient-day per week on resident services (4.67 before vs 3.36 after; difference, 1.31; 95% CI, 0.88-1.74; P < .05) and 23.9% reduction on average in lab draws/patient-day per week on the hospitalist service (4.73 before vs 3.60 after; difference, 1.13; 95% CI, 0.61-1.64; P < .05; Appendix Table 3).

Sleep-Friendly Labs by Intervention

For patients with routine labs, the proportion of sleep-friendly labs drawn per patient-day increased from 6% preintervention to 21% postintervention (P < .001). ITSA demonstrated both interventions were associated with improving lab timing. There was a statistically significant increase in sleep-friendly labs ordered per patient encounter per week immediately after the launch of “Order Sleep” (intercept, 0.49; standard error (SE), 0.14; P = .001) and the “4 am Labs” column (intercept, 0.32; SE, 0.13; P = .02; Table, Figure A).

Sleep-Friendly Lab Orders by Service

Over the study period, there was no significant difference in total sleep-friendly labs ordered/month between resident and hospitalist services (84.88 vs 86.19; P = .95).

In ITSA, “Order Sleep” was associated with a statistically significant immediate increase in sleep-friendly lab orders per patient encounter per week on resident services (intercept, 1.03; SE, 0.29; P < .001). However, this initial increase was followed by a decrease over time in sleep-friendly lab orders per week (slope change, –0.1; SE, 0.04; P = .02; Table, Figure B). There was no statistically significant change observed on the hospitalist service with “Order Sleep.”

In contrast, the “4 am Labs” column was associated with a statistically significant immediate increase in sleep-friendly lab orders per patient encounter per week on hospitalist service (intercept, 1.17; SE, 0.50; P = .02; Table, Figure B). While there was no immediate change on resident service, we observed a significant increase over time in sleep-friendly orders per encounter per week on resident services with the introduction of the “4 am Labs” column (slope change, 0.11; SE, 0.04; P = .01; Table, Figure B).

Cost Savings

Using an estimated cost of $7.70 for CBCs and $8.01 for BMPs from our laboratory, our intervention saved an estimated $60,278 in lab costs alone over the 16-month study period (Appendix Table 4).

To our knowledge, this is the first study showing a multicomponent intervention using EHR tools can both reduce frequency and optimize timing of routine lab ordering. Our project had two interventions implemented at two different times: First, an “Order Sleep” shortcut was introduced to select sleep-friendly lab timing, including a 6 am draw every 48 hours, and later, a “4 am Labs” column was added to electronic patient lists to passively nudge physicians to consider sleep-friendly labs. The “Order Sleep” tool was associated with a significant immediate increase in sleep-friendly lab ordering on resident services, while the “4 am Labs” column was associated with a significant immediate increase in sleep-friendly lab ordering on the hospitalist service. An overall reduction in total lab draws was seen on both services.

While the “Order Sleep” tool was initially associated with significant increases in sleep-friendly orders on resident services, this change was not sustained. This could have been caused by the short-lived effect of education more than sustained adoption of the tool. In contrast, the “4 am Labs” column on the patient list resulted in a significant sustained increase in sleep-friendly labs on resident services. While residents responded to both tools, both interventions were associated with lasting changes in practice.

The “4 am Labs” column on patient lists was associated with increased adoption of sleep-friendly labs for hospitalist services. Hospitalists care for a larger census with more frequent handoffs and greater reliance on the patient list, which makes patient lists in general an important tool to target value improvement.

While other institutions have attempted to shift lab-timing by altering phlebotomy workflows¹⁰ or via conscious decision-making on rounds,⁹ our study differs in several ways. We avoided default options and allowed clinicians to select sleep-friendly labs to promote buy-in. It is sometimes necessary to order 4 am labs for sick patients who need urgent decision-making, which highlights the need to preserve this option for clinicians. Similarly, our intervention did not aim to eliminate lab draws entirely but offer a more judicious frequency of every 48 hours, consistent with the survey preferences noted. This intervention encouraged reappraisal of patients’ overall needs for labs and created variability in ordering times to reduce the volume of labs ordered at 4 am.

Our study had several limitations. First, this was a single center study on adult medicine services, which limits generalizability. Although we considered surgical services, their early rounds made deviations from 4 am undesirable. Given the observational study design, we cannot assume causal relationships or rule out secular trends. There were large swings in sleep-friendly lab ordering during our study that could be attributed to different physicians rotating on the services monthly. We did not obtain objective data on patient sleep or patient satisfaction because of the low response rate to the HCAHPS (Hospital Consumer Assessment of Healthcare Providers and Systems) survey.

In conclusion, a multicomponent intervention using EHR tools can reduce inpatient daily lab frequency and optimize lab timing to help promote patient sleep.

The authors would like to thank The University of Chicago Center for Healthcare Delivery Science and Innovation for sponsoring their annual Choosing Wisely Challenge, which allowed for access to institutional support and resources for this study. We would also like to thank Mary Kate Springman, MHA, and John Fahrenbach, PhD, for their assistance with this project. Dr Tapaskar also received mentorship through the Future Leader Program for the High Value Practice Academic Alliance.

Frequent daily laboratory testing for inpatients contributes to excessive costs,¹ anemia,² and unnecessary testing.³ The ABIM Foundation’s Choosing Wisely^® campaign recommends avoiding routine labs, like complete blood counts (CBCs) and basic metabolic panels (BMP), in the face of clinical and laboratory stability.^4,5 Prior interventions have reduced unnecessary labs without adverse outcomes.^6-8

In addition to lab frequency, hospitalized patients face suboptimal lab timing. Labs are often ordered as early as 4 am at many institutions.^9,10 This practice disrupts sleep, undermining patient health.^11-13 While prior interventions have reduced daily phlebotomy, few have optimized lab timing for patient sleep.¹⁰ No study has harnessed the electronic health record (EHR) to optimize frequency and timing of labs simultaneously.¹⁴ We aimed to determine the effectiveness of a multicomponent intervention, called Order SMARTT (Sleep: Making Appropriate Reductions in Testing and Timing), to reduce frequency and optimize timing of daily routine labs for medical inpatients.

Setting

This study was conducted on the University of Chicago Medicine (UCM) general medicine services, which consisted of a resident-covered service supervised by general medicine, subspecialist, or hospitalist attendings and a hospitalist service staffed by hospitalists and advanced practice providers.

Development of Order SMARTT

To inform intervention development, we surveyed providers about lab-ordering preferences with use of questions from a prior survey to provide a benchmark (Appendix Table 2).¹⁵ While reducing lab frequency was supported, the modal response for how frequently a stable patient should receive routine labs was every 48 hours (Appendix Table 2). Therefore, we hypothesized that labs ordered every 48 hours may be popular. Taking labs every 48 hours would not require an urgent 4 am draw, so we created a 48-hour 6 am phlebotomy option to “step down” from daily labs. To promote these options, we created two EHR tools: First, an “Order Sleep” shortcut was launched in March 2018 by which physicians could type “sleep” in routine lab orders and three sleep-friendly options would become available (a 48-hour 6 am draw, a daily 6 am draw, or a daily 10 pm draw), and second, a “4 am Labs” column and icon on the electronic patient list to signal who had 4 am labs ordered was launched May 2018 (Appendix Table 1).

Physician Education

We created a 20-minute presentation on the harms of excessive labs and the benefits of sleep-friendly ordering. Instructional Order SMARTT posters were posted in clinician workrooms that emphasized forgoing labs on stable patients and using the “Order Sleep” shortcut when nonurgent labs were needed.

Labs Utilization Data

We used Epic Systems software (Verona, Wisconsin) and our institutional Tableau scorecard to obtain data on CBC and BMP ordering, patient census, and demographics for medical inpatients between July 1, 2017, and November 1, 2018.

Cost Analysis

Costs of lab tests (actual cost to our institution) were obtained from our institutional phlebotomy services’ estimates of direct variable labor and benefits costs and direct variable supplies cost.

Statistical Analysis

Data analysis was performed with SAS version 9.4 statistical software (Cary, North Carolina, USA) and R version 3.6.2 (Vienna, Austria). Descriptive statistics were used to summarize data. Surveys were analyzed using chi-square tests for categorical variables and two-sample t tests for continuous variables. For lab ordering data, interrupted time series analyses (ITSA) were used to determine the changes in ordering practices with the implementation of the two interventions controlling for service lines (resident vs hospitalist service). ITSA enables examination of changes in lab ordering while controlling for time. The AUTOREG function in SAS was used to build the model and estimate final parameters. This function automatically tests for autocorrelation, heteroscedasticity, and estimates any autoregressive parameters required in the model. Our main model tested the association between our two separate interventions on ordering practices, controlling for service (hospitalist or resident).¹⁶

Of 125 residents, 82 (65.6%) attended the session and completed the survey. Attendance and response rate for hospitalists was 80% (16 of 20). Similar to a prior study, many residents (73.1%) reported they would be comfortable if patients received less daily laboratory testing (Appendix Table 2).

We reviewed data from 7,045 total patients over 50,951 total patient days between July1, 2017, and November 1, 2018 (Appendix Table 3).

Total Lab Draws

After accounting for total patient days, we saw 26.3% reduction on average in total lab draws per patient-day per week postintervention (4.68 before vs 3.45 after; difference, 1.23; 95% CI, 0.82-1.63; P < .05; Appendix Table 3). When total lab draws were stratified by service, we saw 28% reduction on average in total lab draws per patient-day per week on resident services (4.67 before vs 3.36 after; difference, 1.31; 95% CI, 0.88-1.74; P < .05) and 23.9% reduction on average in lab draws/patient-day per week on the hospitalist service (4.73 before vs 3.60 after; difference, 1.13; 95% CI, 0.61-1.64; P < .05; Appendix Table 3).

Sleep-Friendly Labs by Intervention

For patients with routine labs, the proportion of sleep-friendly labs drawn per patient-day increased from 6% preintervention to 21% postintervention (P < .001). ITSA demonstrated both interventions were associated with improving lab timing. There was a statistically significant increase in sleep-friendly labs ordered per patient encounter per week immediately after the launch of “Order Sleep” (intercept, 0.49; standard error (SE), 0.14; P = .001) and the “4 am Labs” column (intercept, 0.32; SE, 0.13; P = .02; Table, Figure A).

Sleep-Friendly Lab Orders by Service

Over the study period, there was no significant difference in total sleep-friendly labs ordered/month between resident and hospitalist services (84.88 vs 86.19; P = .95).

In ITSA, “Order Sleep” was associated with a statistically significant immediate increase in sleep-friendly lab orders per patient encounter per week on resident services (intercept, 1.03; SE, 0.29; P < .001). However, this initial increase was followed by a decrease over time in sleep-friendly lab orders per week (slope change, –0.1; SE, 0.04; P = .02; Table, Figure B). There was no statistically significant change observed on the hospitalist service with “Order Sleep.”

In contrast, the “4 am Labs” column was associated with a statistically significant immediate increase in sleep-friendly lab orders per patient encounter per week on hospitalist service (intercept, 1.17; SE, 0.50; P = .02; Table, Figure B). While there was no immediate change on resident service, we observed a significant increase over time in sleep-friendly orders per encounter per week on resident services with the introduction of the “4 am Labs” column (slope change, 0.11; SE, 0.04; P = .01; Table, Figure B).

Cost Savings

Using an estimated cost of $7.70 for CBCs and $8.01 for BMPs from our laboratory, our intervention saved an estimated $60,278 in lab costs alone over the 16-month study period (Appendix Table 4).

To our knowledge, this is the first study showing a multicomponent intervention using EHR tools can both reduce frequency and optimize timing of routine lab ordering. Our project had two interventions implemented at two different times: First, an “Order Sleep” shortcut was introduced to select sleep-friendly lab timing, including a 6 am draw every 48 hours, and later, a “4 am Labs” column was added to electronic patient lists to passively nudge physicians to consider sleep-friendly labs. The “Order Sleep” tool was associated with a significant immediate increase in sleep-friendly lab ordering on resident services, while the “4 am Labs” column was associated with a significant immediate increase in sleep-friendly lab ordering on the hospitalist service. An overall reduction in total lab draws was seen on both services.

While the “Order Sleep” tool was initially associated with significant increases in sleep-friendly orders on resident services, this change was not sustained. This could have been caused by the short-lived effect of education more than sustained adoption of the tool. In contrast, the “4 am Labs” column on the patient list resulted in a significant sustained increase in sleep-friendly labs on resident services. While residents responded to both tools, both interventions were associated with lasting changes in practice.

The “4 am Labs” column on patient lists was associated with increased adoption of sleep-friendly labs for hospitalist services. Hospitalists care for a larger census with more frequent handoffs and greater reliance on the patient list, which makes patient lists in general an important tool to target value improvement.

While other institutions have attempted to shift lab-timing by altering phlebotomy workflows¹⁰ or via conscious decision-making on rounds,⁹ our study differs in several ways. We avoided default options and allowed clinicians to select sleep-friendly labs to promote buy-in. It is sometimes necessary to order 4 am labs for sick patients who need urgent decision-making, which highlights the need to preserve this option for clinicians. Similarly, our intervention did not aim to eliminate lab draws entirely but offer a more judicious frequency of every 48 hours, consistent with the survey preferences noted. This intervention encouraged reappraisal of patients’ overall needs for labs and created variability in ordering times to reduce the volume of labs ordered at 4 am.

Our study had several limitations. First, this was a single center study on adult medicine services, which limits generalizability. Although we considered surgical services, their early rounds made deviations from 4 am undesirable. Given the observational study design, we cannot assume causal relationships or rule out secular trends. There were large swings in sleep-friendly lab ordering during our study that could be attributed to different physicians rotating on the services monthly. We did not obtain objective data on patient sleep or patient satisfaction because of the low response rate to the HCAHPS (Hospital Consumer Assessment of Healthcare Providers and Systems) survey.

In conclusion, a multicomponent intervention using EHR tools can reduce inpatient daily lab frequency and optimize lab timing to help promote patient sleep.

The authors would like to thank The University of Chicago Center for Healthcare Delivery Science and Innovation for sponsoring their annual Choosing Wisely Challenge, which allowed for access to institutional support and resources for this study. We would also like to thank Mary Kate Springman, MHA, and John Fahrenbach, PhD, for their assistance with this project. Dr Tapaskar also received mentorship through the Future Leader Program for the High Value Practice Academic Alliance.

Frequent daily laboratory testing for inpatients contributes to excessive costs,¹ anemia,² and unnecessary testing.³ The ABIM Foundation’s Choosing Wisely^® campaign recommends avoiding routine labs, like complete blood counts (CBCs) and basic metabolic panels (BMP), in the face of clinical and laboratory stability.^4,5 Prior interventions have reduced unnecessary labs without adverse outcomes.^6-8

In addition to lab frequency, hospitalized patients face suboptimal lab timing. Labs are often ordered as early as 4 am at many institutions.^9,10 This practice disrupts sleep, undermining patient health.^11-13 While prior interventions have reduced daily phlebotomy, few have optimized lab timing for patient sleep.¹⁰ No study has harnessed the electronic health record (EHR) to optimize frequency and timing of labs simultaneously.¹⁴ We aimed to determine the effectiveness of a multicomponent intervention, called Order SMARTT (Sleep: Making Appropriate Reductions in Testing and Timing), to reduce frequency and optimize timing of daily routine labs for medical inpatients.

Setting

This study was conducted on the University of Chicago Medicine (UCM) general medicine services, which consisted of a resident-covered service supervised by general medicine, subspecialist, or hospitalist attendings and a hospitalist service staffed by hospitalists and advanced practice providers.

Development of Order SMARTT

To inform intervention development, we surveyed providers about lab-ordering preferences with use of questions from a prior survey to provide a benchmark (Appendix Table 2).¹⁵ While reducing lab frequency was supported, the modal response for how frequently a stable patient should receive routine labs was every 48 hours (Appendix Table 2). Therefore, we hypothesized that labs ordered every 48 hours may be popular. Taking labs every 48 hours would not require an urgent 4 am draw, so we created a 48-hour 6 am phlebotomy option to “step down” from daily labs. To promote these options, we created two EHR tools: First, an “Order Sleep” shortcut was launched in March 2018 by which physicians could type “sleep” in routine lab orders and three sleep-friendly options would become available (a 48-hour 6 am draw, a daily 6 am draw, or a daily 10 pm draw), and second, a “4 am Labs” column and icon on the electronic patient list to signal who had 4 am labs ordered was launched May 2018 (Appendix Table 1).

Physician Education

We created a 20-minute presentation on the harms of excessive labs and the benefits of sleep-friendly ordering. Instructional Order SMARTT posters were posted in clinician workrooms that emphasized forgoing labs on stable patients and using the “Order Sleep” shortcut when nonurgent labs were needed.

Labs Utilization Data

We used Epic Systems software (Verona, Wisconsin) and our institutional Tableau scorecard to obtain data on CBC and BMP ordering, patient census, and demographics for medical inpatients between July 1, 2017, and November 1, 2018.

Cost Analysis

Costs of lab tests (actual cost to our institution) were obtained from our institutional phlebotomy services’ estimates of direct variable labor and benefits costs and direct variable supplies cost.

Statistical Analysis

Data analysis was performed with SAS version 9.4 statistical software (Cary, North Carolina, USA) and R version 3.6.2 (Vienna, Austria). Descriptive statistics were used to summarize data. Surveys were analyzed using chi-square tests for categorical variables and two-sample t tests for continuous variables. For lab ordering data, interrupted time series analyses (ITSA) were used to determine the changes in ordering practices with the implementation of the two interventions controlling for service lines (resident vs hospitalist service). ITSA enables examination of changes in lab ordering while controlling for time. The AUTOREG function in SAS was used to build the model and estimate final parameters. This function automatically tests for autocorrelation, heteroscedasticity, and estimates any autoregressive parameters required in the model. Our main model tested the association between our two separate interventions on ordering practices, controlling for service (hospitalist or resident).¹⁶

Of 125 residents, 82 (65.6%) attended the session and completed the survey. Attendance and response rate for hospitalists was 80% (16 of 20). Similar to a prior study, many residents (73.1%) reported they would be comfortable if patients received less daily laboratory testing (Appendix Table 2).

We reviewed data from 7,045 total patients over 50,951 total patient days between July1, 2017, and November 1, 2018 (Appendix Table 3).

Total Lab Draws

After accounting for total patient days, we saw 26.3% reduction on average in total lab draws per patient-day per week postintervention (4.68 before vs 3.45 after; difference, 1.23; 95% CI, 0.82-1.63; P < .05; Appendix Table 3). When total lab draws were stratified by service, we saw 28% reduction on average in total lab draws per patient-day per week on resident services (4.67 before vs 3.36 after; difference, 1.31; 95% CI, 0.88-1.74; P < .05) and 23.9% reduction on average in lab draws/patient-day per week on the hospitalist service (4.73 before vs 3.60 after; difference, 1.13; 95% CI, 0.61-1.64; P < .05; Appendix Table 3).

Sleep-Friendly Labs by Intervention

For patients with routine labs, the proportion of sleep-friendly labs drawn per patient-day increased from 6% preintervention to 21% postintervention (P < .001). ITSA demonstrated both interventions were associated with improving lab timing. There was a statistically significant increase in sleep-friendly labs ordered per patient encounter per week immediately after the launch of “Order Sleep” (intercept, 0.49; standard error (SE), 0.14; P = .001) and the “4 am Labs” column (intercept, 0.32; SE, 0.13; P = .02; Table, Figure A).

Sleep-Friendly Lab Orders by Service

Over the study period, there was no significant difference in total sleep-friendly labs ordered/month between resident and hospitalist services (84.88 vs 86.19; P = .95).

In ITSA, “Order Sleep” was associated with a statistically significant immediate increase in sleep-friendly lab orders per patient encounter per week on resident services (intercept, 1.03; SE, 0.29; P < .001). However, this initial increase was followed by a decrease over time in sleep-friendly lab orders per week (slope change, –0.1; SE, 0.04; P = .02; Table, Figure B). There was no statistically significant change observed on the hospitalist service with “Order Sleep.”

In contrast, the “4 am Labs” column was associated with a statistically significant immediate increase in sleep-friendly lab orders per patient encounter per week on hospitalist service (intercept, 1.17; SE, 0.50; P = .02; Table, Figure B). While there was no immediate change on resident service, we observed a significant increase over time in sleep-friendly orders per encounter per week on resident services with the introduction of the “4 am Labs” column (slope change, 0.11; SE, 0.04; P = .01; Table, Figure B).

Cost Savings

Using an estimated cost of $7.70 for CBCs and $8.01 for BMPs from our laboratory, our intervention saved an estimated $60,278 in lab costs alone over the 16-month study period (Appendix Table 4).

To our knowledge, this is the first study showing a multicomponent intervention using EHR tools can both reduce frequency and optimize timing of routine lab ordering. Our project had two interventions implemented at two different times: First, an “Order Sleep” shortcut was introduced to select sleep-friendly lab timing, including a 6 am draw every 48 hours, and later, a “4 am Labs” column was added to electronic patient lists to passively nudge physicians to consider sleep-friendly labs. The “Order Sleep” tool was associated with a significant immediate increase in sleep-friendly lab ordering on resident services, while the “4 am Labs” column was associated with a significant immediate increase in sleep-friendly lab ordering on the hospitalist service. An overall reduction in total lab draws was seen on both services.

While the “Order Sleep” tool was initially associated with significant increases in sleep-friendly orders on resident services, this change was not sustained. This could have been caused by the short-lived effect of education more than sustained adoption of the tool. In contrast, the “4 am Labs” column on the patient list resulted in a significant sustained increase in sleep-friendly labs on resident services. While residents responded to both tools, both interventions were associated with lasting changes in practice.

The “4 am Labs” column on patient lists was associated with increased adoption of sleep-friendly labs for hospitalist services. Hospitalists care for a larger census with more frequent handoffs and greater reliance on the patient list, which makes patient lists in general an important tool to target value improvement.

While other institutions have attempted to shift lab-timing by altering phlebotomy workflows¹⁰ or via conscious decision-making on rounds,⁹ our study differs in several ways. We avoided default options and allowed clinicians to select sleep-friendly labs to promote buy-in. It is sometimes necessary to order 4 am labs for sick patients who need urgent decision-making, which highlights the need to preserve this option for clinicians. Similarly, our intervention did not aim to eliminate lab draws entirely but offer a more judicious frequency of every 48 hours, consistent with the survey preferences noted. This intervention encouraged reappraisal of patients’ overall needs for labs and created variability in ordering times to reduce the volume of labs ordered at 4 am.

Our study had several limitations. First, this was a single center study on adult medicine services, which limits generalizability. Although we considered surgical services, their early rounds made deviations from 4 am undesirable. Given the observational study design, we cannot assume causal relationships or rule out secular trends. There were large swings in sleep-friendly lab ordering during our study that could be attributed to different physicians rotating on the services monthly. We did not obtain objective data on patient sleep or patient satisfaction because of the low response rate to the HCAHPS (Hospital Consumer Assessment of Healthcare Providers and Systems) survey.

In conclusion, a multicomponent intervention using EHR tools can reduce inpatient daily lab frequency and optimize lab timing to help promote patient sleep.

The authors would like to thank The University of Chicago Center for Healthcare Delivery Science and Innovation for sponsoring their annual Choosing Wisely Challenge, which allowed for access to institutional support and resources for this study. We would also like to thank Mary Kate Springman, MHA, and John Fahrenbach, PhD, for their assistance with this project. Dr Tapaskar also received mentorship through the Future Leader Program for the High Value Practice Academic Alliance.

A large body of evidence has demonstrated significant gender disparities in academic medicine. Women are less likely than men to reach the rank of full professor, be speakers at Grand Rounds, and author studies in medical journals.^1-4 Gender-based differences in these achievements reduce the visibility of women role models in all academic medicine domains, including research, education, health systems leadership, and clinical excellence. Clinical problem-solving exercises are an opportunity to highlight the skills of women physicians as master clinicians and to establish women as clinician role models.

Clinical problem-solving exercises are highly visible demonstrations of clinical excellence in the medical literature. These exercises follow a specific format in which a clinician analyzes a diagnostic dilemma in a step-by-step manner in response to sequential segments of clinical data. The clinical problem-­solving format was introduced in 1992 in the New England Journal of Medicine and has been adopted by other journals.⁵ (The clinical problem-solving format differs from the clinical pathologic conference format, in which an entire case is presented followed by an extended analysis). Clinical problem-­solving publications are forums for learners of all levels to witness an expert clinician reason through a case.

Authorship teams on clinical reasoning exercises typically include the patient’s physician(s), specialists relevant to the final diagnosis, and the invited discussant who analyzes the clinical dilemma. Journals stipulate in the author instructions, series introductions, or standardized manuscript text of the series that the discussant be a skilled and experienced clinician.^5,6 The patient’s physicians who initiate the clinical reasoning manuscript typically select the discussant; in some journals, the series editors may provide input on discussant choice. To our knowledge, this is the only author role in the medical literature in which authors are invited specifically for their diagnostic reasoning ability.

While women have been authors on fewer original research articles and guest editorials than men have,³ the proportion of women among authors of published clinical reasoning exercises is unknown. This represents a gap in our understanding of the landscape of gender inequity in academic medicine. We sought to determine the proportion of women authors in major clinical problem-solving series and examine the change in women authorship over time.

We selected published clinical problem-solving series targeting a general medicine audience. We excluded general medicine journals in which authors were restricted to one institution or those in which the clinical problem-solving format was not a regular series. Series which met these criteria were the Clinical Problem-Solving series in the New England Journal of Medicine (NEJM), the Clinical Care Conundrums series in the Journal of Hospital Medicine (JHM), and the Exercises in Clinical Reasoning series in the Journal of General Internal Medicine (JGIM). We analyzed the proportion of women authors in each clinical reasoning series from the inaugural articles (1992 for NEJM, 2006 for JHM, and 2010 for JGIM) until July 2019. We also analyzed the change in proportion of women authors from year to year by using data up to 2018 to avoid including a partial year.

We used the gender-guesser python library⁷ to categorize the gender of first, last, and all authors based on their first names. The library uses a database of approximately 40,000 names⁸ and maps first names to the genders they are associated with across languages, classifying each name as “man,” “woman,” “mostly man,” ”mostly woman,” “androgynous,” or “unknown.” When a name is commonly associated with multiple genders, or is associated with different genders in different languages, it is classified either as mostly man, mostly woman, or androgynous. When a name is not found in the database, it is classified as unknown. For all names classified by the database as unknown, androgynous, or mostly man/mostly woman, we determined gender identities by finding the authors’ institutional webpages and consulting their listed gender pronouns. We used gender based on first name to best approximate what a reader would interpret as the author’s gender. We used gender rather than biological sex because authors may have changed their names to better express their gender identity, which may differ from sex assigned at birth.

To test for the statistical significance of changes in the proportion of women authors over time, we performed the Cochran-­Armitage trend test. A P value less than .05 was considered significant.

We analyzed 402 articles: 280 from NEJM, 83 from JHM, and 39 from JGIM. There were 1,026 authors of clinical reasoning articles from NEJM, 362 from JHM, and 168 from JGIM. The Table shows the number of total articles, total authors, and women among first, last, and all authors by journal and by year (inaugural year and 2018). Data for all years are shown in the Appendix Table.

Over the entire time period studied, the percentage of women across the three journals was lowest for last authors (28/280 [10.0%] for NEJM, 6/83 [7.2%] for JHM, and 9/39 [23.1%] for JGIM) and highest for first authors (80/280 [28.6%] for NEJM, 36/83 [43.4%] for JHM, and 13/39 [33.3%] for JGIM). The percentage of women among all authors was similar for all three journals: 224/1,026 (21.8%) for NEJM, 83/362 (22.9%) for JHM, and 36/168 (21.4%) for JGIM.

The Figure shows the change in percentage of women authors from year to year through 2018. There was a significant increase in the proportion of women first authors in NEJM (from 0/12 [0.0%] in 1992 to 4/12 [33.3%] in 2018; P < .0001) and JHM (from 2/5 [40.0%] in 2006 to 7/9 [77.8%] in 2018 P = .01). There was also a significant increase in the proportion of women among all authors in NEJM (from 0/17 [0.0%] in 1992 to 17/59 [28.8%] in 2018; P < .0001) and JHM (from 3/19 [15.8%] in 2006 to 14/37 [37.8%] in 2018; P = .005). There was no significant change in the proportion of women last authors in any of the three journals. There were no statistically significant changes in JGIM authorship over time.

Clinical problem-solving exercises provide a forum for physicians to demonstrate diagnostic reasoning skills and clinical acumen. In this study, we focused on three prominent clinical problem-solving series in general medicine journals. We found that women authors were underrepresented in each series. The percentage of women authors has increased over time, especially among first and all authors; however, there was no change in the last author position. In all three series women still constituted less than 40% of all authors and less than 25% of last authors. In comparison, women currently constitute about 40% of general internal medicine physicians, and this proportion has been rapidly growing over time; women now represent over half of all medical school graduates as opposed to 6% in 1960.^9,10 Our findings are consistent with the large body of evidence that describes gender-based differences in opportunities within academic medicine.

Prior studies have shown that gender inequities in academic medicine stem from a longstanding culture of sexism; these inequities are perpetuated in part by having too few visible women role models and mentors.¹¹ These factors may lead to editorial practices that favor articles written by men. In addition, women may be less likely to be invited as expert discussants if other authors have a bias of associating clinical expertise with men physicians. This is consistent with data showing that women are less likely to be invited to write commentaries in peer-reviewed journals.¹²

Gender-based differences in authorship of clinical problem-solving publications also have important implications for women in medicine. In order to address the gender gap in academic achievement, women need visible role models and mentors.¹³ Including more women authors of clinical reasoning publications has the potential to establish more women as master clinicians and role models.

There are a number of actions that can help establish more women clinical problem-solving authors. Editorial boards and editors in chief should track their review and publication practices to hold themselves accountable to author diversity. For example, JHM has announced plans to analyze author representation of women and racial and ethnic minorities, including those among first and senior authors.¹⁴ Clinicians who are assembling author teams for clinical problem-solving manuscripts should also strongly consider if an equal number of men and women have been invited to serve as specialty consultants and case discussants.

Our study has limitations. We used a python library to classify author gender based on first name (supplemented by internet searches), which may have misclassified authors and did not take into account nonbinary gender identities. Because there is no convention for assigning the expert discussant to a specific author position, we could not determine the gender distribution of the discussants. However, given that women were underrepresented among first, last, and all authors in all three journals, they are likely a minority of discussants as well.

A preponderance of male voices in clinical reasoning exercises, in which learners see clinical role models, may perpetuate a culture in which women are not seen—and do not see themselves—as having the potential to be master clinicians. Including more women in clinical reasoning exercises is an opportunity to amplify the voices of women as master clinicians and combat gender discrimination in medicine.

A large body of evidence has demonstrated significant gender disparities in academic medicine. Women are less likely than men to reach the rank of full professor, be speakers at Grand Rounds, and author studies in medical journals.^1-4 Gender-based differences in these achievements reduce the visibility of women role models in all academic medicine domains, including research, education, health systems leadership, and clinical excellence. Clinical problem-solving exercises are an opportunity to highlight the skills of women physicians as master clinicians and to establish women as clinician role models.

Clinical problem-solving exercises are highly visible demonstrations of clinical excellence in the medical literature. These exercises follow a specific format in which a clinician analyzes a diagnostic dilemma in a step-by-step manner in response to sequential segments of clinical data. The clinical problem-­solving format was introduced in 1992 in the New England Journal of Medicine and has been adopted by other journals.⁵ (The clinical problem-solving format differs from the clinical pathologic conference format, in which an entire case is presented followed by an extended analysis). Clinical problem-­solving publications are forums for learners of all levels to witness an expert clinician reason through a case.

Authorship teams on clinical reasoning exercises typically include the patient’s physician(s), specialists relevant to the final diagnosis, and the invited discussant who analyzes the clinical dilemma. Journals stipulate in the author instructions, series introductions, or standardized manuscript text of the series that the discussant be a skilled and experienced clinician.^5,6 The patient’s physicians who initiate the clinical reasoning manuscript typically select the discussant; in some journals, the series editors may provide input on discussant choice. To our knowledge, this is the only author role in the medical literature in which authors are invited specifically for their diagnostic reasoning ability.

While women have been authors on fewer original research articles and guest editorials than men have,³ the proportion of women among authors of published clinical reasoning exercises is unknown. This represents a gap in our understanding of the landscape of gender inequity in academic medicine. We sought to determine the proportion of women authors in major clinical problem-solving series and examine the change in women authorship over time.

We selected published clinical problem-solving series targeting a general medicine audience. We excluded general medicine journals in which authors were restricted to one institution or those in which the clinical problem-solving format was not a regular series. Series which met these criteria were the Clinical Problem-Solving series in the New England Journal of Medicine (NEJM), the Clinical Care Conundrums series in the Journal of Hospital Medicine (JHM), and the Exercises in Clinical Reasoning series in the Journal of General Internal Medicine (JGIM). We analyzed the proportion of women authors in each clinical reasoning series from the inaugural articles (1992 for NEJM, 2006 for JHM, and 2010 for JGIM) until July 2019. We also analyzed the change in proportion of women authors from year to year by using data up to 2018 to avoid including a partial year.

We used the gender-guesser python library⁷ to categorize the gender of first, last, and all authors based on their first names. The library uses a database of approximately 40,000 names⁸ and maps first names to the genders they are associated with across languages, classifying each name as “man,” “woman,” “mostly man,” ”mostly woman,” “androgynous,” or “unknown.” When a name is commonly associated with multiple genders, or is associated with different genders in different languages, it is classified either as mostly man, mostly woman, or androgynous. When a name is not found in the database, it is classified as unknown. For all names classified by the database as unknown, androgynous, or mostly man/mostly woman, we determined gender identities by finding the authors’ institutional webpages and consulting their listed gender pronouns. We used gender based on first name to best approximate what a reader would interpret as the author’s gender. We used gender rather than biological sex because authors may have changed their names to better express their gender identity, which may differ from sex assigned at birth.

To test for the statistical significance of changes in the proportion of women authors over time, we performed the Cochran-­Armitage trend test. A P value less than .05 was considered significant.

We analyzed 402 articles: 280 from NEJM, 83 from JHM, and 39 from JGIM. There were 1,026 authors of clinical reasoning articles from NEJM, 362 from JHM, and 168 from JGIM. The Table shows the number of total articles, total authors, and women among first, last, and all authors by journal and by year (inaugural year and 2018). Data for all years are shown in the Appendix Table.

Over the entire time period studied, the percentage of women across the three journals was lowest for last authors (28/280 [10.0%] for NEJM, 6/83 [7.2%] for JHM, and 9/39 [23.1%] for JGIM) and highest for first authors (80/280 [28.6%] for NEJM, 36/83 [43.4%] for JHM, and 13/39 [33.3%] for JGIM). The percentage of women among all authors was similar for all three journals: 224/1,026 (21.8%) for NEJM, 83/362 (22.9%) for JHM, and 36/168 (21.4%) for JGIM.

The Figure shows the change in percentage of women authors from year to year through 2018. There was a significant increase in the proportion of women first authors in NEJM (from 0/12 [0.0%] in 1992 to 4/12 [33.3%] in 2018; P < .0001) and JHM (from 2/5 [40.0%] in 2006 to 7/9 [77.8%] in 2018 P = .01). There was also a significant increase in the proportion of women among all authors in NEJM (from 0/17 [0.0%] in 1992 to 17/59 [28.8%] in 2018; P < .0001) and JHM (from 3/19 [15.8%] in 2006 to 14/37 [37.8%] in 2018; P = .005). There was no significant change in the proportion of women last authors in any of the three journals. There were no statistically significant changes in JGIM authorship over time.

Clinical problem-solving exercises provide a forum for physicians to demonstrate diagnostic reasoning skills and clinical acumen. In this study, we focused on three prominent clinical problem-solving series in general medicine journals. We found that women authors were underrepresented in each series. The percentage of women authors has increased over time, especially among first and all authors; however, there was no change in the last author position. In all three series women still constituted less than 40% of all authors and less than 25% of last authors. In comparison, women currently constitute about 40% of general internal medicine physicians, and this proportion has been rapidly growing over time; women now represent over half of all medical school graduates as opposed to 6% in 1960.^9,10 Our findings are consistent with the large body of evidence that describes gender-based differences in opportunities within academic medicine.

Prior studies have shown that gender inequities in academic medicine stem from a longstanding culture of sexism; these inequities are perpetuated in part by having too few visible women role models and mentors.¹¹ These factors may lead to editorial practices that favor articles written by men. In addition, women may be less likely to be invited as expert discussants if other authors have a bias of associating clinical expertise with men physicians. This is consistent with data showing that women are less likely to be invited to write commentaries in peer-reviewed journals.¹²

Gender-based differences in authorship of clinical problem-solving publications also have important implications for women in medicine. In order to address the gender gap in academic achievement, women need visible role models and mentors.¹³ Including more women authors of clinical reasoning publications has the potential to establish more women as master clinicians and role models.

There are a number of actions that can help establish more women clinical problem-solving authors. Editorial boards and editors in chief should track their review and publication practices to hold themselves accountable to author diversity. For example, JHM has announced plans to analyze author representation of women and racial and ethnic minorities, including those among first and senior authors.¹⁴ Clinicians who are assembling author teams for clinical problem-solving manuscripts should also strongly consider if an equal number of men and women have been invited to serve as specialty consultants and case discussants.

Our study has limitations. We used a python library to classify author gender based on first name (supplemented by internet searches), which may have misclassified authors and did not take into account nonbinary gender identities. Because there is no convention for assigning the expert discussant to a specific author position, we could not determine the gender distribution of the discussants. However, given that women were underrepresented among first, last, and all authors in all three journals, they are likely a minority of discussants as well.

A preponderance of male voices in clinical reasoning exercises, in which learners see clinical role models, may perpetuate a culture in which women are not seen—and do not see themselves—as having the potential to be master clinicians. Including more women in clinical reasoning exercises is an opportunity to amplify the voices of women as master clinicians and combat gender discrimination in medicine.

A large body of evidence has demonstrated significant gender disparities in academic medicine. Women are less likely than men to reach the rank of full professor, be speakers at Grand Rounds, and author studies in medical journals.^1-4 Gender-based differences in these achievements reduce the visibility of women role models in all academic medicine domains, including research, education, health systems leadership, and clinical excellence. Clinical problem-solving exercises are an opportunity to highlight the skills of women physicians as master clinicians and to establish women as clinician role models.

Clinical problem-solving exercises are highly visible demonstrations of clinical excellence in the medical literature. These exercises follow a specific format in which a clinician analyzes a diagnostic dilemma in a step-by-step manner in response to sequential segments of clinical data. The clinical problem-­solving format was introduced in 1992 in the New England Journal of Medicine and has been adopted by other journals.⁵ (The clinical problem-solving format differs from the clinical pathologic conference format, in which an entire case is presented followed by an extended analysis). Clinical problem-­solving publications are forums for learners of all levels to witness an expert clinician reason through a case.

Authorship teams on clinical reasoning exercises typically include the patient’s physician(s), specialists relevant to the final diagnosis, and the invited discussant who analyzes the clinical dilemma. Journals stipulate in the author instructions, series introductions, or standardized manuscript text of the series that the discussant be a skilled and experienced clinician.^5,6 The patient’s physicians who initiate the clinical reasoning manuscript typically select the discussant; in some journals, the series editors may provide input on discussant choice. To our knowledge, this is the only author role in the medical literature in which authors are invited specifically for their diagnostic reasoning ability.

While women have been authors on fewer original research articles and guest editorials than men have,³ the proportion of women among authors of published clinical reasoning exercises is unknown. This represents a gap in our understanding of the landscape of gender inequity in academic medicine. We sought to determine the proportion of women authors in major clinical problem-solving series and examine the change in women authorship over time.

We selected published clinical problem-solving series targeting a general medicine audience. We excluded general medicine journals in which authors were restricted to one institution or those in which the clinical problem-solving format was not a regular series. Series which met these criteria were the Clinical Problem-Solving series in the New England Journal of Medicine (NEJM), the Clinical Care Conundrums series in the Journal of Hospital Medicine (JHM), and the Exercises in Clinical Reasoning series in the Journal of General Internal Medicine (JGIM). We analyzed the proportion of women authors in each clinical reasoning series from the inaugural articles (1992 for NEJM, 2006 for JHM, and 2010 for JGIM) until July 2019. We also analyzed the change in proportion of women authors from year to year by using data up to 2018 to avoid including a partial year.

We used the gender-guesser python library⁷ to categorize the gender of first, last, and all authors based on their first names. The library uses a database of approximately 40,000 names⁸ and maps first names to the genders they are associated with across languages, classifying each name as “man,” “woman,” “mostly man,” ”mostly woman,” “androgynous,” or “unknown.” When a name is commonly associated with multiple genders, or is associated with different genders in different languages, it is classified either as mostly man, mostly woman, or androgynous. When a name is not found in the database, it is classified as unknown. For all names classified by the database as unknown, androgynous, or mostly man/mostly woman, we determined gender identities by finding the authors’ institutional webpages and consulting their listed gender pronouns. We used gender based on first name to best approximate what a reader would interpret as the author’s gender. We used gender rather than biological sex because authors may have changed their names to better express their gender identity, which may differ from sex assigned at birth.

To test for the statistical significance of changes in the proportion of women authors over time, we performed the Cochran-­Armitage trend test. A P value less than .05 was considered significant.

We analyzed 402 articles: 280 from NEJM, 83 from JHM, and 39 from JGIM. There were 1,026 authors of clinical reasoning articles from NEJM, 362 from JHM, and 168 from JGIM. The Table shows the number of total articles, total authors, and women among first, last, and all authors by journal and by year (inaugural year and 2018). Data for all years are shown in the Appendix Table.

Over the entire time period studied, the percentage of women across the three journals was lowest for last authors (28/280 [10.0%] for NEJM, 6/83 [7.2%] for JHM, and 9/39 [23.1%] for JGIM) and highest for first authors (80/280 [28.6%] for NEJM, 36/83 [43.4%] for JHM, and 13/39 [33.3%] for JGIM). The percentage of women among all authors was similar for all three journals: 224/1,026 (21.8%) for NEJM, 83/362 (22.9%) for JHM, and 36/168 (21.4%) for JGIM.

The Figure shows the change in percentage of women authors from year to year through 2018. There was a significant increase in the proportion of women first authors in NEJM (from 0/12 [0.0%] in 1992 to 4/12 [33.3%] in 2018; P < .0001) and JHM (from 2/5 [40.0%] in 2006 to 7/9 [77.8%] in 2018 P = .01). There was also a significant increase in the proportion of women among all authors in NEJM (from 0/17 [0.0%] in 1992 to 17/59 [28.8%] in 2018; P < .0001) and JHM (from 3/19 [15.8%] in 2006 to 14/37 [37.8%] in 2018; P = .005). There was no significant change in the proportion of women last authors in any of the three journals. There were no statistically significant changes in JGIM authorship over time.

Clinical problem-solving exercises provide a forum for physicians to demonstrate diagnostic reasoning skills and clinical acumen. In this study, we focused on three prominent clinical problem-solving series in general medicine journals. We found that women authors were underrepresented in each series. The percentage of women authors has increased over time, especially among first and all authors; however, there was no change in the last author position. In all three series women still constituted less than 40% of all authors and less than 25% of last authors. In comparison, women currently constitute about 40% of general internal medicine physicians, and this proportion has been rapidly growing over time; women now represent over half of all medical school graduates as opposed to 6% in 1960.^9,10 Our findings are consistent with the large body of evidence that describes gender-based differences in opportunities within academic medicine.

Prior studies have shown that gender inequities in academic medicine stem from a longstanding culture of sexism; these inequities are perpetuated in part by having too few visible women role models and mentors.¹¹ These factors may lead to editorial practices that favor articles written by men. In addition, women may be less likely to be invited as expert discussants if other authors have a bias of associating clinical expertise with men physicians. This is consistent with data showing that women are less likely to be invited to write commentaries in peer-reviewed journals.¹²

Gender-based differences in authorship of clinical problem-solving publications also have important implications for women in medicine. In order to address the gender gap in academic achievement, women need visible role models and mentors.¹³ Including more women authors of clinical reasoning publications has the potential to establish more women as master clinicians and role models.

There are a number of actions that can help establish more women clinical problem-solving authors. Editorial boards and editors in chief should track their review and publication practices to hold themselves accountable to author diversity. For example, JHM has announced plans to analyze author representation of women and racial and ethnic minorities, including those among first and senior authors.¹⁴ Clinicians who are assembling author teams for clinical problem-solving manuscripts should also strongly consider if an equal number of men and women have been invited to serve as specialty consultants and case discussants.

Our study has limitations. We used a python library to classify author gender based on first name (supplemented by internet searches), which may have misclassified authors and did not take into account nonbinary gender identities. Because there is no convention for assigning the expert discussant to a specific author position, we could not determine the gender distribution of the discussants. However, given that women were underrepresented among first, last, and all authors in all three journals, they are likely a minority of discussants as well.

A preponderance of male voices in clinical reasoning exercises, in which learners see clinical role models, may perpetuate a culture in which women are not seen—and do not see themselves—as having the potential to be master clinicians. Including more women in clinical reasoning exercises is an opportunity to amplify the voices of women as master clinicians and combat gender discrimination in medicine.