Hospital Medicine

Unrecognized in-hospital clinical deterioration can lead to substantial morbidity and mortality.¹ As a result, hospitals have implemented systems to identify and mitigate this form of potentially preventable harm.^2-4 Cardiopulmonary arrest rates are useful metrics to evaluate the effectiveness of systems designed to identify and respond to deteriorating adult patients.⁵ Pediatric arrests outside of the intensive care unit (ICU) are rare; therefore, the identification of valid and more frequent proximal measures of deterioration is critical to the assessment of current systems and to guide future improvement efforts.⁶

Bonafide et al developed and validated the critical deterioration event (CDE) metric, demonstrating that children who were transferred to the ICU and who received noninvasive ventilation, intubation, or vasopressor initiation within 12 hours of transfer had an over 13-fold increased risk of in-hospital mortality.⁷ Implementation of a rapid response system was subsequently associated with a decrease in the trajectory of CDEs.² At Cincinnati Children’s Hospital Medical Center (CCHMC), an additional proximal outcome measure was developed for unrecognized clinical deterioration: emergency transfers (ETs).^8,9 An event meets criteria for an ET when the patient undergoes intubation, inotropic support, or three or more fluid boluses in the first hour after arrival or prior to ICU transfer.⁹ Recently, ETs were associated with an increased in-hospital mortality, ICU length of stay, and post-transfer hospital length of stay when compared with nonemergent transfers.^10,11

While both CDEs and ETs were associated with adverse outcomes in children and may be modifiable through better rapid response systems, researchers have not previously compared the extent to which CDEs and ETs capture similar versus distinct events. Furthermore, the ability of focused situation awareness interventions to identify high-risk patients has not previously been assessed. Situation awareness is defined as the perception of elements in the environment, the comprehension of their meaning, and the projection of their status in the near future.¹² Clinically, improved situation awareness can lead to earlier recognition of deterioration and a reduction in failure to rescue events.⁹ The objectives of this study were to (1) describe CDEs and ETs and assess for similarities, differences, and trends, and (2) evaluate the utility of situation awareness interventions to detect patients who experience these events.

Setting and Inclusion Criteria

We conducted a retrospective cross-sectional study at CCHMC, a free-standing tertiary care children’s hospital. We included all patients cared for outside of the ICU during their hospitalization from January 2016 to July 2018. Transfer to the ICU included the pediatric and the cardiac ICUs.

Study Definitions

CDEs were events in which a patient received noninvasive ventilation, intubation, or vasopressor initiation within 12 hours of ICU transfer (Figure).⁷ ETs were events in which a patient underwent intubation, inotropes, or three or more fluid boluses in the first hour after arrival or before transfer (Figure).⁹ We examined two distinct situation awareness interventions: watcher identification and the pediatric early warning score (PEWS). A watcher is a situation awareness concern based on clinician perception, or “gut feeling,” that the patient is at high risk for deterioration.^9,13 When clinicians designate a patient as a watcher in the electronic medical record, they establish an action plan, reassessment timeline, and objective criteria for activation of the rapid response team to assess the patient. Watcher patients are discussed at institution-wide safety huddles three times daily. The PEWS is a reproducible assessment of the patient’s status based on physiologic parameters, including behavior, cardiovascular, and respiratory assessments.^3,4 At CCHMC, a Monaghan PEWS score is calculated with each assessment of vital signs.¹⁴ The bedside nurse calls the physician or advanced practice provider to assess the patient for a score of 4 or greater.

Event Identification and Classification

Two trained research nurses (C.F. and D.H.) manually reviewed all ICU transfers during the study period to determine if CDE criteria were met. Events meeting CDE criteria were classified as respiratory (requiring noninvasive or invasive ventilation), cardiac (requiring inotropes), or cardiopulmonary resuscitation (CPR) in which cardiac and respiratory interventions were initiated simultaneously. Additional information obtained included the time the patient met CDE criteria relative to the time of ICU transfer, watcher identification prior to the event, and the highest PEWS documented within 12 hours of the event. A physician (T.S.) performed manual chart review of each CDE as an additional validation step. ETs during the study period were obtained from an existing institutional database. ICU transfers meeting ET criteria are entered into this database in nearly real time by the inpatient nurse manager; this nurse attends all rapid response team calls and is aware of the disposition for each event. A physician (T.S.) performed manual chart review of each ET to determine event classification by intervention type, watcher identification, and the highest PEWS documented within 12 hours of the event. All CDEs and ETs were cross-referenced to determine overlap.

Outcome Measures and Statistical Analysis

The primary outcomes were CDEs and ETs, calculated as absolute counts and number of events per 10,000 non-ICU patient days. Events were classified by (1) category of intervention, (2) watcher identification prior to the event, and (3) PEWS of 4 or greater documented in the 12 hours prior to the event.

Incidence and Overlap of CDEs and ETs

There were 1,828 ICU transfers during the study period, of which 365 (20%) met criteria for a CDE, ET, or both. Among events captured, 359 (98.4%) met criteria for a CDE, occurring at a rate of 16.7 per 10,000 non-ICU patient days, and 88 (24.1%) met criteria for an ET, occurring at a rate of 4.1 per 10,000 non-ICU patient days (Table). Of the 88 ETs, 82 also met criteria for a CDE.

Timing and Categorization of CDEs and ETs

Despite the 12-hour time horizon, most CDEs (62.1%) met criteria within 1 hour of ICU transfer, and 79.9% met criteria within 3 hours (Figure). Respiratory events were most common for both CDEs (80.5%) and ETs (47.7%) (Table). Of respiratory CDEs, 67.4% required noninvasive ventilation, and 32.5% required invasive ventilation. Fluid or inotrope support were responsible for 11.7% of CDEs and nearly one-third of ETs; of note, the CDE definition does not include fluid boluses. Less than 10% of CDEs were characterized by CPR, whereas this accounted for 22.7% of ETs.

Identification of Events by Situation Awareness Interventions

The Table depicts the identification of events by watcher status and PEWS. All events were included for watcher identification, and events with a documented score in the 12 hours prior to transfer were included for PEWS. While half or less of the events were captured by watcher or PEWS separately, over 85% of events were captured by either one or both of the situation awareness interventions. The situation awareness interventions identified CDEs and ETs similarly.

This study is the first to classify and compare two proximal measures of clinical deterioration in children. Given that children with escalating respiratory symptoms are often treated successfully outside of the ICU, the findings that most events are respiratory in nature and occur within 1 hour of transfer are not unexpected. The analysis of situation awareness interventions suggests that neither watcher identification nor PEWS is independently sufficient to predict future deterioration. These findings support the necessity of both a clinician “gut feeling” and objective vital sign and physical exam findings to indicate a patient’s clinical status.⁹ Initiatives to improve the early recognition and mitigation of patient deterioration should focus on both tools to initiate an escalation of care, and work to understand gaps in these identification systems, which currently miss approximately 15% of acutely deteriorating patients. Although most patients had watcher identification or elevated PEWS prior to the event, they still required emergent life-sustaining care, which suggests that opportunities exist to improve mitigation and escalation pathways as a critical prevention effort.^7,10

It is likely that CDEs and ETs are important outcome metrics in the evaluation of pediatric escalation systems, including rapid response systems.¹⁵ ETs are less common and more specific for unrecognized deterioration, which makes them a more feasible early metric for assessment. CDEs, which are likely more sensitive, may be useful in settings in which deterioration is rare or a more common outcome enhances power to detect the effect of interventions.¹⁰

This study has limitations and lends itself to future work. While CDEs and ETs are more common than cardiopulmonary arrest, they remain relatively uncommon. This was a single-site study at a large, tertiary care, free-standing children’s hospital, so generalizability to centers with different characteristics and patient populations may be limited. Future work should focus on comparing patient-level outcomes of CDEs and ETs, including length of stay and mortality. The determination of specific diagnoses and conditions associated with CDEs and ETs may inform targeted preventive improvement science interventions.

CDEs were roughly fourfold more common than ETs, with most CDEs occurring within 1 hour of ICU transfer. Most patients were identified by either watcher status or elevated PEWS, suggesting that these tools, when utilized as complementary situation awareness interventions, are important for identifying patients at risk for deterioration. Opportunities exist for improved escalation plans for patients identified as high-risk to prevent the need for emergent life-sustaining intervention.

Unrecognized in-hospital clinical deterioration can lead to substantial morbidity and mortality.¹ As a result, hospitals have implemented systems to identify and mitigate this form of potentially preventable harm.^2-4 Cardiopulmonary arrest rates are useful metrics to evaluate the effectiveness of systems designed to identify and respond to deteriorating adult patients.⁵ Pediatric arrests outside of the intensive care unit (ICU) are rare; therefore, the identification of valid and more frequent proximal measures of deterioration is critical to the assessment of current systems and to guide future improvement efforts.⁶

Bonafide et al developed and validated the critical deterioration event (CDE) metric, demonstrating that children who were transferred to the ICU and who received noninvasive ventilation, intubation, or vasopressor initiation within 12 hours of transfer had an over 13-fold increased risk of in-hospital mortality.⁷ Implementation of a rapid response system was subsequently associated with a decrease in the trajectory of CDEs.² At Cincinnati Children’s Hospital Medical Center (CCHMC), an additional proximal outcome measure was developed for unrecognized clinical deterioration: emergency transfers (ETs).^8,9 An event meets criteria for an ET when the patient undergoes intubation, inotropic support, or three or more fluid boluses in the first hour after arrival or prior to ICU transfer.⁹ Recently, ETs were associated with an increased in-hospital mortality, ICU length of stay, and post-transfer hospital length of stay when compared with nonemergent transfers.^10,11

While both CDEs and ETs were associated with adverse outcomes in children and may be modifiable through better rapid response systems, researchers have not previously compared the extent to which CDEs and ETs capture similar versus distinct events. Furthermore, the ability of focused situation awareness interventions to identify high-risk patients has not previously been assessed. Situation awareness is defined as the perception of elements in the environment, the comprehension of their meaning, and the projection of their status in the near future.¹² Clinically, improved situation awareness can lead to earlier recognition of deterioration and a reduction in failure to rescue events.⁹ The objectives of this study were to (1) describe CDEs and ETs and assess for similarities, differences, and trends, and (2) evaluate the utility of situation awareness interventions to detect patients who experience these events.

Setting and Inclusion Criteria

We conducted a retrospective cross-sectional study at CCHMC, a free-standing tertiary care children’s hospital. We included all patients cared for outside of the ICU during their hospitalization from January 2016 to July 2018. Transfer to the ICU included the pediatric and the cardiac ICUs.

Study Definitions

CDEs were events in which a patient received noninvasive ventilation, intubation, or vasopressor initiation within 12 hours of ICU transfer (Figure).⁷ ETs were events in which a patient underwent intubation, inotropes, or three or more fluid boluses in the first hour after arrival or before transfer (Figure).⁹ We examined two distinct situation awareness interventions: watcher identification and the pediatric early warning score (PEWS). A watcher is a situation awareness concern based on clinician perception, or “gut feeling,” that the patient is at high risk for deterioration.^9,13 When clinicians designate a patient as a watcher in the electronic medical record, they establish an action plan, reassessment timeline, and objective criteria for activation of the rapid response team to assess the patient. Watcher patients are discussed at institution-wide safety huddles three times daily. The PEWS is a reproducible assessment of the patient’s status based on physiologic parameters, including behavior, cardiovascular, and respiratory assessments.^3,4 At CCHMC, a Monaghan PEWS score is calculated with each assessment of vital signs.¹⁴ The bedside nurse calls the physician or advanced practice provider to assess the patient for a score of 4 or greater.

Event Identification and Classification

Two trained research nurses (C.F. and D.H.) manually reviewed all ICU transfers during the study period to determine if CDE criteria were met. Events meeting CDE criteria were classified as respiratory (requiring noninvasive or invasive ventilation), cardiac (requiring inotropes), or cardiopulmonary resuscitation (CPR) in which cardiac and respiratory interventions were initiated simultaneously. Additional information obtained included the time the patient met CDE criteria relative to the time of ICU transfer, watcher identification prior to the event, and the highest PEWS documented within 12 hours of the event. A physician (T.S.) performed manual chart review of each CDE as an additional validation step. ETs during the study period were obtained from an existing institutional database. ICU transfers meeting ET criteria are entered into this database in nearly real time by the inpatient nurse manager; this nurse attends all rapid response team calls and is aware of the disposition for each event. A physician (T.S.) performed manual chart review of each ET to determine event classification by intervention type, watcher identification, and the highest PEWS documented within 12 hours of the event. All CDEs and ETs were cross-referenced to determine overlap.

Outcome Measures and Statistical Analysis

The primary outcomes were CDEs and ETs, calculated as absolute counts and number of events per 10,000 non-ICU patient days. Events were classified by (1) category of intervention, (2) watcher identification prior to the event, and (3) PEWS of 4 or greater documented in the 12 hours prior to the event.

Incidence and Overlap of CDEs and ETs

There were 1,828 ICU transfers during the study period, of which 365 (20%) met criteria for a CDE, ET, or both. Among events captured, 359 (98.4%) met criteria for a CDE, occurring at a rate of 16.7 per 10,000 non-ICU patient days, and 88 (24.1%) met criteria for an ET, occurring at a rate of 4.1 per 10,000 non-ICU patient days (Table). Of the 88 ETs, 82 also met criteria for a CDE.

Timing and Categorization of CDEs and ETs

Despite the 12-hour time horizon, most CDEs (62.1%) met criteria within 1 hour of ICU transfer, and 79.9% met criteria within 3 hours (Figure). Respiratory events were most common for both CDEs (80.5%) and ETs (47.7%) (Table). Of respiratory CDEs, 67.4% required noninvasive ventilation, and 32.5% required invasive ventilation. Fluid or inotrope support were responsible for 11.7% of CDEs and nearly one-third of ETs; of note, the CDE definition does not include fluid boluses. Less than 10% of CDEs were characterized by CPR, whereas this accounted for 22.7% of ETs.

Identification of Events by Situation Awareness Interventions

The Table depicts the identification of events by watcher status and PEWS. All events were included for watcher identification, and events with a documented score in the 12 hours prior to transfer were included for PEWS. While half or less of the events were captured by watcher or PEWS separately, over 85% of events were captured by either one or both of the situation awareness interventions. The situation awareness interventions identified CDEs and ETs similarly.

This study is the first to classify and compare two proximal measures of clinical deterioration in children. Given that children with escalating respiratory symptoms are often treated successfully outside of the ICU, the findings that most events are respiratory in nature and occur within 1 hour of transfer are not unexpected. The analysis of situation awareness interventions suggests that neither watcher identification nor PEWS is independently sufficient to predict future deterioration. These findings support the necessity of both a clinician “gut feeling” and objective vital sign and physical exam findings to indicate a patient’s clinical status.⁹ Initiatives to improve the early recognition and mitigation of patient deterioration should focus on both tools to initiate an escalation of care, and work to understand gaps in these identification systems, which currently miss approximately 15% of acutely deteriorating patients. Although most patients had watcher identification or elevated PEWS prior to the event, they still required emergent life-sustaining care, which suggests that opportunities exist to improve mitigation and escalation pathways as a critical prevention effort.^7,10

It is likely that CDEs and ETs are important outcome metrics in the evaluation of pediatric escalation systems, including rapid response systems.¹⁵ ETs are less common and more specific for unrecognized deterioration, which makes them a more feasible early metric for assessment. CDEs, which are likely more sensitive, may be useful in settings in which deterioration is rare or a more common outcome enhances power to detect the effect of interventions.¹⁰

This study has limitations and lends itself to future work. While CDEs and ETs are more common than cardiopulmonary arrest, they remain relatively uncommon. This was a single-site study at a large, tertiary care, free-standing children’s hospital, so generalizability to centers with different characteristics and patient populations may be limited. Future work should focus on comparing patient-level outcomes of CDEs and ETs, including length of stay and mortality. The determination of specific diagnoses and conditions associated with CDEs and ETs may inform targeted preventive improvement science interventions.

CDEs were roughly fourfold more common than ETs, with most CDEs occurring within 1 hour of ICU transfer. Most patients were identified by either watcher status or elevated PEWS, suggesting that these tools, when utilized as complementary situation awareness interventions, are important for identifying patients at risk for deterioration. Opportunities exist for improved escalation plans for patients identified as high-risk to prevent the need for emergent life-sustaining intervention.

Unrecognized in-hospital clinical deterioration can lead to substantial morbidity and mortality.¹ As a result, hospitals have implemented systems to identify and mitigate this form of potentially preventable harm.^2-4 Cardiopulmonary arrest rates are useful metrics to evaluate the effectiveness of systems designed to identify and respond to deteriorating adult patients.⁵ Pediatric arrests outside of the intensive care unit (ICU) are rare; therefore, the identification of valid and more frequent proximal measures of deterioration is critical to the assessment of current systems and to guide future improvement efforts.⁶

Bonafide et al developed and validated the critical deterioration event (CDE) metric, demonstrating that children who were transferred to the ICU and who received noninvasive ventilation, intubation, or vasopressor initiation within 12 hours of transfer had an over 13-fold increased risk of in-hospital mortality.⁷ Implementation of a rapid response system was subsequently associated with a decrease in the trajectory of CDEs.² At Cincinnati Children’s Hospital Medical Center (CCHMC), an additional proximal outcome measure was developed for unrecognized clinical deterioration: emergency transfers (ETs).^8,9 An event meets criteria for an ET when the patient undergoes intubation, inotropic support, or three or more fluid boluses in the first hour after arrival or prior to ICU transfer.⁹ Recently, ETs were associated with an increased in-hospital mortality, ICU length of stay, and post-transfer hospital length of stay when compared with nonemergent transfers.^10,11

While both CDEs and ETs were associated with adverse outcomes in children and may be modifiable through better rapid response systems, researchers have not previously compared the extent to which CDEs and ETs capture similar versus distinct events. Furthermore, the ability of focused situation awareness interventions to identify high-risk patients has not previously been assessed. Situation awareness is defined as the perception of elements in the environment, the comprehension of their meaning, and the projection of their status in the near future.¹² Clinically, improved situation awareness can lead to earlier recognition of deterioration and a reduction in failure to rescue events.⁹ The objectives of this study were to (1) describe CDEs and ETs and assess for similarities, differences, and trends, and (2) evaluate the utility of situation awareness interventions to detect patients who experience these events.

Setting and Inclusion Criteria

We conducted a retrospective cross-sectional study at CCHMC, a free-standing tertiary care children’s hospital. We included all patients cared for outside of the ICU during their hospitalization from January 2016 to July 2018. Transfer to the ICU included the pediatric and the cardiac ICUs.

Study Definitions

CDEs were events in which a patient received noninvasive ventilation, intubation, or vasopressor initiation within 12 hours of ICU transfer (Figure).⁷ ETs were events in which a patient underwent intubation, inotropes, or three or more fluid boluses in the first hour after arrival or before transfer (Figure).⁹ We examined two distinct situation awareness interventions: watcher identification and the pediatric early warning score (PEWS). A watcher is a situation awareness concern based on clinician perception, or “gut feeling,” that the patient is at high risk for deterioration.^9,13 When clinicians designate a patient as a watcher in the electronic medical record, they establish an action plan, reassessment timeline, and objective criteria for activation of the rapid response team to assess the patient. Watcher patients are discussed at institution-wide safety huddles three times daily. The PEWS is a reproducible assessment of the patient’s status based on physiologic parameters, including behavior, cardiovascular, and respiratory assessments.^3,4 At CCHMC, a Monaghan PEWS score is calculated with each assessment of vital signs.¹⁴ The bedside nurse calls the physician or advanced practice provider to assess the patient for a score of 4 or greater.

Event Identification and Classification

Two trained research nurses (C.F. and D.H.) manually reviewed all ICU transfers during the study period to determine if CDE criteria were met. Events meeting CDE criteria were classified as respiratory (requiring noninvasive or invasive ventilation), cardiac (requiring inotropes), or cardiopulmonary resuscitation (CPR) in which cardiac and respiratory interventions were initiated simultaneously. Additional information obtained included the time the patient met CDE criteria relative to the time of ICU transfer, watcher identification prior to the event, and the highest PEWS documented within 12 hours of the event. A physician (T.S.) performed manual chart review of each CDE as an additional validation step. ETs during the study period were obtained from an existing institutional database. ICU transfers meeting ET criteria are entered into this database in nearly real time by the inpatient nurse manager; this nurse attends all rapid response team calls and is aware of the disposition for each event. A physician (T.S.) performed manual chart review of each ET to determine event classification by intervention type, watcher identification, and the highest PEWS documented within 12 hours of the event. All CDEs and ETs were cross-referenced to determine overlap.

Outcome Measures and Statistical Analysis

The primary outcomes were CDEs and ETs, calculated as absolute counts and number of events per 10,000 non-ICU patient days. Events were classified by (1) category of intervention, (2) watcher identification prior to the event, and (3) PEWS of 4 or greater documented in the 12 hours prior to the event.

Incidence and Overlap of CDEs and ETs

There were 1,828 ICU transfers during the study period, of which 365 (20%) met criteria for a CDE, ET, or both. Among events captured, 359 (98.4%) met criteria for a CDE, occurring at a rate of 16.7 per 10,000 non-ICU patient days, and 88 (24.1%) met criteria for an ET, occurring at a rate of 4.1 per 10,000 non-ICU patient days (Table). Of the 88 ETs, 82 also met criteria for a CDE.

Timing and Categorization of CDEs and ETs

Despite the 12-hour time horizon, most CDEs (62.1%) met criteria within 1 hour of ICU transfer, and 79.9% met criteria within 3 hours (Figure). Respiratory events were most common for both CDEs (80.5%) and ETs (47.7%) (Table). Of respiratory CDEs, 67.4% required noninvasive ventilation, and 32.5% required invasive ventilation. Fluid or inotrope support were responsible for 11.7% of CDEs and nearly one-third of ETs; of note, the CDE definition does not include fluid boluses. Less than 10% of CDEs were characterized by CPR, whereas this accounted for 22.7% of ETs.

Identification of Events by Situation Awareness Interventions

The Table depicts the identification of events by watcher status and PEWS. All events were included for watcher identification, and events with a documented score in the 12 hours prior to transfer were included for PEWS. While half or less of the events were captured by watcher or PEWS separately, over 85% of events were captured by either one or both of the situation awareness interventions. The situation awareness interventions identified CDEs and ETs similarly.

This study is the first to classify and compare two proximal measures of clinical deterioration in children. Given that children with escalating respiratory symptoms are often treated successfully outside of the ICU, the findings that most events are respiratory in nature and occur within 1 hour of transfer are not unexpected. The analysis of situation awareness interventions suggests that neither watcher identification nor PEWS is independently sufficient to predict future deterioration. These findings support the necessity of both a clinician “gut feeling” and objective vital sign and physical exam findings to indicate a patient’s clinical status.⁹ Initiatives to improve the early recognition and mitigation of patient deterioration should focus on both tools to initiate an escalation of care, and work to understand gaps in these identification systems, which currently miss approximately 15% of acutely deteriorating patients. Although most patients had watcher identification or elevated PEWS prior to the event, they still required emergent life-sustaining care, which suggests that opportunities exist to improve mitigation and escalation pathways as a critical prevention effort.^7,10

It is likely that CDEs and ETs are important outcome metrics in the evaluation of pediatric escalation systems, including rapid response systems.¹⁵ ETs are less common and more specific for unrecognized deterioration, which makes them a more feasible early metric for assessment. CDEs, which are likely more sensitive, may be useful in settings in which deterioration is rare or a more common outcome enhances power to detect the effect of interventions.¹⁰

This study has limitations and lends itself to future work. While CDEs and ETs are more common than cardiopulmonary arrest, they remain relatively uncommon. This was a single-site study at a large, tertiary care, free-standing children’s hospital, so generalizability to centers with different characteristics and patient populations may be limited. Future work should focus on comparing patient-level outcomes of CDEs and ETs, including length of stay and mortality. The determination of specific diagnoses and conditions associated with CDEs and ETs may inform targeted preventive improvement science interventions.

CDEs were roughly fourfold more common than ETs, with most CDEs occurring within 1 hour of ICU transfer. Most patients were identified by either watcher status or elevated PEWS, suggesting that these tools, when utilized as complementary situation awareness interventions, are important for identifying patients at risk for deterioration. Opportunities exist for improved escalation plans for patients identified as high-risk to prevent the need for emergent life-sustaining intervention.

As part of improvement collaboratives that aimed to reduce overuse of continuous pulse oximetry in children hospitalized with bronchiolitis, researchers used the presence of an active order for it as a proxy for the actual use of such monitoring.^1,2 With use of this proxy, investigators on a national study documented a high burden of continuous oximetry overuse (86.5% before quality improvement interventions and 45.5% after),¹ but the validity of orders in representing actual monitoring practice is unknown. If the presence of an active pulse oximetry order accurately identifies infants on monitors, electronic health record data could inform epidemiologic estimates of monitoring overuse and measure the success of quality improvement and deimplementation interventions. Alternatively, if nurses commonly begin and/or discontinue pulse oximetry without updated orders, a pulse oximetry order would not be an accurate proxy, and additional data capture methods (eg, bedside observation or data capture from bedside monitors) would be needed.

Understanding the validity of orders for detection of actual use is critical because continuous pulse oximetry monitoring is considered an overused practice in pediatric acute viral bronchiolitis,³ and national guidelines recommend against its use in low-risk hospitalized children.^4,5 Continuous monitoring may identify trivial, self-resolving oxygen desaturation and its use is not associated with improved outcomes.^6-9 When self-resolving desaturations are treated with additional supplemental oxygen, hospital stays may be unnecessarily prolonged.¹⁰ In order to reduce unnecessary continuous pulse oximetry use, measurement of the extent of the overused practice is necessary. In this 56-hospital study,¹¹ we aimed to determine the validity of using active continuous pulse oximetry orders instead of bedside observation of actual monitor use.

Design

In this multicenter, repeated cross-sectional study, investigators used direct bedside observation to determine continuous pulse oximetry monitor use and then assessed whether an active continuous monitoring order was present in the electronic health record. The study took place during one bronchiolitis season, December 1, 2018, through March 31, 2019.

Setting and Patients

Investigators at 56 freestanding children’s hospitals, children’s hospitals within general hospitals, and community hospitals in the Pediatric Research in Inpatient Settings (PRIS) Network collected data on infants aged 8 weeks to 23 months who were hospitalized with bronchiolitis. As this work was a substudy of the larger Eliminating Monitor Overuse study, only infants not currently receiving supplemental oxygen were included.¹¹ Investigators observed eligible infants outside of the intensive care unit on general hospital medicine units. We excluded infants born premature (documented prematurity of <28 weeks’ gestation or documented “premature” without a gestational age listed), as well as those with a home oxygen requirement, cyanotic congenital heart disease, pulmonary hypertension, tracheostomy, primary neuromuscular disease, immunodeficiency, or cancer.

Data Collection

Investigators used the electronic health record to identify eligible infants. Investigators entered patient rooms to confirm the infant was not on supplemental oxygen (hence confirming eligibility for the study) and determine if continuous pulse oximetry was actively in use by examining the monitor display for a pulse oximetry waveform. Investigators then confirmed if active orders for pulse oximetry were present in the patient’s chart. Per study design, site investigators aimed to observe approximately half of eligible infants during the day (10 am to 5 pm) and the other half during the night (11 pm to 7 am).

Analysis

We excluded patients with conditional orders (eg, monitored only when certain conditions exist, such as when asleep) because of the time-varying and wide range of conditions that could be specified. Furthermore, conditional orders would not be useful as proxies to measure oximetry use because investigators would still need additional data (eg, bedside observation) to determine current monitoring status.

We calculated the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of active orders using the reference standard of direct bedside observation, as well as corresponding 95% CIs that accounted for within-hospital clustering. We calculated these test characteristics overall and as stratified across four age groups: 8 weeks to 5 months, 6 months to 11 months, 12 months to 17 months, and 18 months to 23 months. We also calculated the test characteristics for each hospital. We decided a priori that a PPV and NPV of 80% would represent a reasonable threshold to use active orders as a proxy in multicenter research. For hospital-level analyses we included only hospitals with 60 or more total observations and more than 15 observations with active orders for PPV and more than 15 observations without active orders for NPV. We used Stata (StataCorp LLC, College Station, Texas) version 15.1 for analysis.

For US sites, the Institutional Review Board (IRB) at Children’s Hospital of Philadelphia approved the study as the single reviewing IRB, and the remaining US sites established reliance agreements with the reviewing IRB. Research Ethics Boards at the Canadian sites (University of Calgary and The Hospital for Sick Children) also reviewed and approved the study. All sites granted waivers of consent, assent, parental permission, and HIPAA authorization.

Investigators completed 3,612 observations in 56 hospitals. This included 33 freestanding children’s hospitals, 14 hospitals within large general hospitals, and 9 community hospitals. Of 3,612 completed observations, on 631 occasions (17%) patients had conditional orders (eg, continuous monitoring only when sleeping) and were excluded from further analysis.

Most pulse oximetry–monitored infants did not have an active monitoring order (670 out of 1,309; sensitivity of 49%). Test characteristics, stratified by age group, are presented in the Table. Across all observations, the overall PPV was 77% (95% CI, 72-82), and the overall NPV was 69% (95% CI, 61-77). Variation of all test characteristics across age group was small (eg, the sensitivity ranged from 43% to 51%).

With inclusion of only those hospitals with sufficient observations, hospital-level variation in the PPV and NPV of using active orders was substantial (PPV range of 48% to 96% and NPV range of 30% to 98%). Only two hospitals had both a PPV and NPV for using monitor orders that exceeded the 80% threshold.

Active continuous pulse oximetry orders did not accurately represent actual monitoring status in this study. Monitoring orders alone frequently misrepresent true monitoring status and, as such, should be interpreted with caution in research or quality improvement activities. If more valid estimates of monitoring use and overuse are needed, potential measurement options include direct observation, as used in our study, as well as the use of more complex data streams such as the output of monitoring devices or pulse oximetry data in the electronic health record. In only two of the hospitals, using active continuous monitoring orders was a reasonable proxy for detecting actual monitor use. Monitoring orders could potentially be validly used for deimplementation efforts at those centers; other hospitals could consider targeted improvement efforts (eg, morning huddles examining the discordance between monitoring orders and monitoring status) to improve the accuracy of using continuous pulse oximetry orders.

We acknowledge several limitations of this study. Site investigators employed a convenience sampling approach, so it is possible that some investigators observed sicker or less sick infants. Although the PRIS network includes a geographically diverse group of North American hospitals, community hospitals were underrepresented in this study. Our results hence generalize more precisely to freestanding children’s hospitals than to community hospitals. We did not observe infants currently on supplemental oxygen, so we do not know to what degree using orders is valid in that context. We did not collect data on why actual monitoring status differed from monitoring orders and hence cannot quantify to what extent different factors (eg, nurse belief that monitors are a safety net or infants inadvertently left on monitors after a spot check pulse oximetry reading) contributed to this discordance. Finally, our study only examined one electronic health record variable—the presence of an active order. It may be that other variables in the health record (eg, minute-by-minute pulse oximetry values in a vital sign flowsheet) are much better proxies of actual continuous monitor use.

Using an active order for continuous pulse oximetry has poor sensitivity, PPV, and NPV for detecting true monitoring status at the bedside. Teams intending to measure the actual use of pulse oximetry should be aware of the limitations of using active orders alone as an accurate measure of pulse oximetry monitoring.

We thank the NHLBI scientists who contributed to this project as part of the U01 Cooperative Agreement funding mechanism: Lora Reineck, MD, MS, Karen Bienstock, MS, and Cheryl Boyce, PhD.

We thank the Executive Council of the PRIS Network for their contributions to the early scientific development of this project. We thank the PRIS site investigators for their major contributions to the Eliminating Monitor Overuse (EMO) Study data collection. Each listed collaborator is a group author for the PRIS Network in this manuscript. Their names can be found in the online supplemental information.

As part of improvement collaboratives that aimed to reduce overuse of continuous pulse oximetry in children hospitalized with bronchiolitis, researchers used the presence of an active order for it as a proxy for the actual use of such monitoring.^1,2 With use of this proxy, investigators on a national study documented a high burden of continuous oximetry overuse (86.5% before quality improvement interventions and 45.5% after),¹ but the validity of orders in representing actual monitoring practice is unknown. If the presence of an active pulse oximetry order accurately identifies infants on monitors, electronic health record data could inform epidemiologic estimates of monitoring overuse and measure the success of quality improvement and deimplementation interventions. Alternatively, if nurses commonly begin and/or discontinue pulse oximetry without updated orders, a pulse oximetry order would not be an accurate proxy, and additional data capture methods (eg, bedside observation or data capture from bedside monitors) would be needed.

Understanding the validity of orders for detection of actual use is critical because continuous pulse oximetry monitoring is considered an overused practice in pediatric acute viral bronchiolitis,³ and national guidelines recommend against its use in low-risk hospitalized children.^4,5 Continuous monitoring may identify trivial, self-resolving oxygen desaturation and its use is not associated with improved outcomes.^6-9 When self-resolving desaturations are treated with additional supplemental oxygen, hospital stays may be unnecessarily prolonged.¹⁰ In order to reduce unnecessary continuous pulse oximetry use, measurement of the extent of the overused practice is necessary. In this 56-hospital study,¹¹ we aimed to determine the validity of using active continuous pulse oximetry orders instead of bedside observation of actual monitor use.

Design

In this multicenter, repeated cross-sectional study, investigators used direct bedside observation to determine continuous pulse oximetry monitor use and then assessed whether an active continuous monitoring order was present in the electronic health record. The study took place during one bronchiolitis season, December 1, 2018, through March 31, 2019.

Setting and Patients

Investigators at 56 freestanding children’s hospitals, children’s hospitals within general hospitals, and community hospitals in the Pediatric Research in Inpatient Settings (PRIS) Network collected data on infants aged 8 weeks to 23 months who were hospitalized with bronchiolitis. As this work was a substudy of the larger Eliminating Monitor Overuse study, only infants not currently receiving supplemental oxygen were included.¹¹ Investigators observed eligible infants outside of the intensive care unit on general hospital medicine units. We excluded infants born premature (documented prematurity of <28 weeks’ gestation or documented “premature” without a gestational age listed), as well as those with a home oxygen requirement, cyanotic congenital heart disease, pulmonary hypertension, tracheostomy, primary neuromuscular disease, immunodeficiency, or cancer.

Data Collection

Investigators used the electronic health record to identify eligible infants. Investigators entered patient rooms to confirm the infant was not on supplemental oxygen (hence confirming eligibility for the study) and determine if continuous pulse oximetry was actively in use by examining the monitor display for a pulse oximetry waveform. Investigators then confirmed if active orders for pulse oximetry were present in the patient’s chart. Per study design, site investigators aimed to observe approximately half of eligible infants during the day (10 am to 5 pm) and the other half during the night (11 pm to 7 am).

Analysis

We excluded patients with conditional orders (eg, monitored only when certain conditions exist, such as when asleep) because of the time-varying and wide range of conditions that could be specified. Furthermore, conditional orders would not be useful as proxies to measure oximetry use because investigators would still need additional data (eg, bedside observation) to determine current monitoring status.

We calculated the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of active orders using the reference standard of direct bedside observation, as well as corresponding 95% CIs that accounted for within-hospital clustering. We calculated these test characteristics overall and as stratified across four age groups: 8 weeks to 5 months, 6 months to 11 months, 12 months to 17 months, and 18 months to 23 months. We also calculated the test characteristics for each hospital. We decided a priori that a PPV and NPV of 80% would represent a reasonable threshold to use active orders as a proxy in multicenter research. For hospital-level analyses we included only hospitals with 60 or more total observations and more than 15 observations with active orders for PPV and more than 15 observations without active orders for NPV. We used Stata (StataCorp LLC, College Station, Texas) version 15.1 for analysis.

For US sites, the Institutional Review Board (IRB) at Children’s Hospital of Philadelphia approved the study as the single reviewing IRB, and the remaining US sites established reliance agreements with the reviewing IRB. Research Ethics Boards at the Canadian sites (University of Calgary and The Hospital for Sick Children) also reviewed and approved the study. All sites granted waivers of consent, assent, parental permission, and HIPAA authorization.

Investigators completed 3,612 observations in 56 hospitals. This included 33 freestanding children’s hospitals, 14 hospitals within large general hospitals, and 9 community hospitals. Of 3,612 completed observations, on 631 occasions (17%) patients had conditional orders (eg, continuous monitoring only when sleeping) and were excluded from further analysis.

Most pulse oximetry–monitored infants did not have an active monitoring order (670 out of 1,309; sensitivity of 49%). Test characteristics, stratified by age group, are presented in the Table. Across all observations, the overall PPV was 77% (95% CI, 72-82), and the overall NPV was 69% (95% CI, 61-77). Variation of all test characteristics across age group was small (eg, the sensitivity ranged from 43% to 51%).

With inclusion of only those hospitals with sufficient observations, hospital-level variation in the PPV and NPV of using active orders was substantial (PPV range of 48% to 96% and NPV range of 30% to 98%). Only two hospitals had both a PPV and NPV for using monitor orders that exceeded the 80% threshold.

Active continuous pulse oximetry orders did not accurately represent actual monitoring status in this study. Monitoring orders alone frequently misrepresent true monitoring status and, as such, should be interpreted with caution in research or quality improvement activities. If more valid estimates of monitoring use and overuse are needed, potential measurement options include direct observation, as used in our study, as well as the use of more complex data streams such as the output of monitoring devices or pulse oximetry data in the electronic health record. In only two of the hospitals, using active continuous monitoring orders was a reasonable proxy for detecting actual monitor use. Monitoring orders could potentially be validly used for deimplementation efforts at those centers; other hospitals could consider targeted improvement efforts (eg, morning huddles examining the discordance between monitoring orders and monitoring status) to improve the accuracy of using continuous pulse oximetry orders.

We acknowledge several limitations of this study. Site investigators employed a convenience sampling approach, so it is possible that some investigators observed sicker or less sick infants. Although the PRIS network includes a geographically diverse group of North American hospitals, community hospitals were underrepresented in this study. Our results hence generalize more precisely to freestanding children’s hospitals than to community hospitals. We did not observe infants currently on supplemental oxygen, so we do not know to what degree using orders is valid in that context. We did not collect data on why actual monitoring status differed from monitoring orders and hence cannot quantify to what extent different factors (eg, nurse belief that monitors are a safety net or infants inadvertently left on monitors after a spot check pulse oximetry reading) contributed to this discordance. Finally, our study only examined one electronic health record variable—the presence of an active order. It may be that other variables in the health record (eg, minute-by-minute pulse oximetry values in a vital sign flowsheet) are much better proxies of actual continuous monitor use.

Using an active order for continuous pulse oximetry has poor sensitivity, PPV, and NPV for detecting true monitoring status at the bedside. Teams intending to measure the actual use of pulse oximetry should be aware of the limitations of using active orders alone as an accurate measure of pulse oximetry monitoring.

We thank the NHLBI scientists who contributed to this project as part of the U01 Cooperative Agreement funding mechanism: Lora Reineck, MD, MS, Karen Bienstock, MS, and Cheryl Boyce, PhD.

We thank the Executive Council of the PRIS Network for their contributions to the early scientific development of this project. We thank the PRIS site investigators for their major contributions to the Eliminating Monitor Overuse (EMO) Study data collection. Each listed collaborator is a group author for the PRIS Network in this manuscript. Their names can be found in the online supplemental information.

As part of improvement collaboratives that aimed to reduce overuse of continuous pulse oximetry in children hospitalized with bronchiolitis, researchers used the presence of an active order for it as a proxy for the actual use of such monitoring.^1,2 With use of this proxy, investigators on a national study documented a high burden of continuous oximetry overuse (86.5% before quality improvement interventions and 45.5% after),¹ but the validity of orders in representing actual monitoring practice is unknown. If the presence of an active pulse oximetry order accurately identifies infants on monitors, electronic health record data could inform epidemiologic estimates of monitoring overuse and measure the success of quality improvement and deimplementation interventions. Alternatively, if nurses commonly begin and/or discontinue pulse oximetry without updated orders, a pulse oximetry order would not be an accurate proxy, and additional data capture methods (eg, bedside observation or data capture from bedside monitors) would be needed.

Understanding the validity of orders for detection of actual use is critical because continuous pulse oximetry monitoring is considered an overused practice in pediatric acute viral bronchiolitis,³ and national guidelines recommend against its use in low-risk hospitalized children.^4,5 Continuous monitoring may identify trivial, self-resolving oxygen desaturation and its use is not associated with improved outcomes.^6-9 When self-resolving desaturations are treated with additional supplemental oxygen, hospital stays may be unnecessarily prolonged.¹⁰ In order to reduce unnecessary continuous pulse oximetry use, measurement of the extent of the overused practice is necessary. In this 56-hospital study,¹¹ we aimed to determine the validity of using active continuous pulse oximetry orders instead of bedside observation of actual monitor use.

Design

In this multicenter, repeated cross-sectional study, investigators used direct bedside observation to determine continuous pulse oximetry monitor use and then assessed whether an active continuous monitoring order was present in the electronic health record. The study took place during one bronchiolitis season, December 1, 2018, through March 31, 2019.

Setting and Patients

Investigators at 56 freestanding children’s hospitals, children’s hospitals within general hospitals, and community hospitals in the Pediatric Research in Inpatient Settings (PRIS) Network collected data on infants aged 8 weeks to 23 months who were hospitalized with bronchiolitis. As this work was a substudy of the larger Eliminating Monitor Overuse study, only infants not currently receiving supplemental oxygen were included.¹¹ Investigators observed eligible infants outside of the intensive care unit on general hospital medicine units. We excluded infants born premature (documented prematurity of <28 weeks’ gestation or documented “premature” without a gestational age listed), as well as those with a home oxygen requirement, cyanotic congenital heart disease, pulmonary hypertension, tracheostomy, primary neuromuscular disease, immunodeficiency, or cancer.

Data Collection

Investigators used the electronic health record to identify eligible infants. Investigators entered patient rooms to confirm the infant was not on supplemental oxygen (hence confirming eligibility for the study) and determine if continuous pulse oximetry was actively in use by examining the monitor display for a pulse oximetry waveform. Investigators then confirmed if active orders for pulse oximetry were present in the patient’s chart. Per study design, site investigators aimed to observe approximately half of eligible infants during the day (10 am to 5 pm) and the other half during the night (11 pm to 7 am).

Analysis

We excluded patients with conditional orders (eg, monitored only when certain conditions exist, such as when asleep) because of the time-varying and wide range of conditions that could be specified. Furthermore, conditional orders would not be useful as proxies to measure oximetry use because investigators would still need additional data (eg, bedside observation) to determine current monitoring status.

We calculated the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of active orders using the reference standard of direct bedside observation, as well as corresponding 95% CIs that accounted for within-hospital clustering. We calculated these test characteristics overall and as stratified across four age groups: 8 weeks to 5 months, 6 months to 11 months, 12 months to 17 months, and 18 months to 23 months. We also calculated the test characteristics for each hospital. We decided a priori that a PPV and NPV of 80% would represent a reasonable threshold to use active orders as a proxy in multicenter research. For hospital-level analyses we included only hospitals with 60 or more total observations and more than 15 observations with active orders for PPV and more than 15 observations without active orders for NPV. We used Stata (StataCorp LLC, College Station, Texas) version 15.1 for analysis.

For US sites, the Institutional Review Board (IRB) at Children’s Hospital of Philadelphia approved the study as the single reviewing IRB, and the remaining US sites established reliance agreements with the reviewing IRB. Research Ethics Boards at the Canadian sites (University of Calgary and The Hospital for Sick Children) also reviewed and approved the study. All sites granted waivers of consent, assent, parental permission, and HIPAA authorization.

Investigators completed 3,612 observations in 56 hospitals. This included 33 freestanding children’s hospitals, 14 hospitals within large general hospitals, and 9 community hospitals. Of 3,612 completed observations, on 631 occasions (17%) patients had conditional orders (eg, continuous monitoring only when sleeping) and were excluded from further analysis.

Most pulse oximetry–monitored infants did not have an active monitoring order (670 out of 1,309; sensitivity of 49%). Test characteristics, stratified by age group, are presented in the Table. Across all observations, the overall PPV was 77% (95% CI, 72-82), and the overall NPV was 69% (95% CI, 61-77). Variation of all test characteristics across age group was small (eg, the sensitivity ranged from 43% to 51%).

With inclusion of only those hospitals with sufficient observations, hospital-level variation in the PPV and NPV of using active orders was substantial (PPV range of 48% to 96% and NPV range of 30% to 98%). Only two hospitals had both a PPV and NPV for using monitor orders that exceeded the 80% threshold.

Active continuous pulse oximetry orders did not accurately represent actual monitoring status in this study. Monitoring orders alone frequently misrepresent true monitoring status and, as such, should be interpreted with caution in research or quality improvement activities. If more valid estimates of monitoring use and overuse are needed, potential measurement options include direct observation, as used in our study, as well as the use of more complex data streams such as the output of monitoring devices or pulse oximetry data in the electronic health record. In only two of the hospitals, using active continuous monitoring orders was a reasonable proxy for detecting actual monitor use. Monitoring orders could potentially be validly used for deimplementation efforts at those centers; other hospitals could consider targeted improvement efforts (eg, morning huddles examining the discordance between monitoring orders and monitoring status) to improve the accuracy of using continuous pulse oximetry orders.

We acknowledge several limitations of this study. Site investigators employed a convenience sampling approach, so it is possible that some investigators observed sicker or less sick infants. Although the PRIS network includes a geographically diverse group of North American hospitals, community hospitals were underrepresented in this study. Our results hence generalize more precisely to freestanding children’s hospitals than to community hospitals. We did not observe infants currently on supplemental oxygen, so we do not know to what degree using orders is valid in that context. We did not collect data on why actual monitoring status differed from monitoring orders and hence cannot quantify to what extent different factors (eg, nurse belief that monitors are a safety net or infants inadvertently left on monitors after a spot check pulse oximetry reading) contributed to this discordance. Finally, our study only examined one electronic health record variable—the presence of an active order. It may be that other variables in the health record (eg, minute-by-minute pulse oximetry values in a vital sign flowsheet) are much better proxies of actual continuous monitor use.

Using an active order for continuous pulse oximetry has poor sensitivity, PPV, and NPV for detecting true monitoring status at the bedside. Teams intending to measure the actual use of pulse oximetry should be aware of the limitations of using active orders alone as an accurate measure of pulse oximetry monitoring.

We thank the NHLBI scientists who contributed to this project as part of the U01 Cooperative Agreement funding mechanism: Lora Reineck, MD, MS, Karen Bienstock, MS, and Cheryl Boyce, PhD.

We thank the Executive Council of the PRIS Network for their contributions to the early scientific development of this project. We thank the PRIS site investigators for their major contributions to the Eliminating Monitor Overuse (EMO) Study data collection. Each listed collaborator is a group author for the PRIS Network in this manuscript. Their names can be found in the online supplemental information.

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), the cause of coronavirus disease 2019 (COVID-19), is associated with a wide range of illness severity and community prevalence, with an estimated 20% to 30% of patients requiring hospitalization.^1,2 Outcome studies of hospitalized patients to date have focused on epicenter healthcare systems operating at surge-level bed capacity in resource-limited settings with mortality exceeding 20% among patients with a discharge disposition^3,4 and have had a publication bias toward those suffering critical illness.^5-7 Generalizability of these results to nonepicenter hospital systems is unclear given potential differences in triage practices and resource availability according to disease prevalence, with nonepicenter systems that are operating below capacity potentially able to accommodate the needs of most, if not all patients, requiring inpatient level care. Clinical outcomes associated with non–critically ill patients in nonepicenter regions remain poorly characterized yet highly relevant because these will ultimately apply to most US and global healthcare environments.

Nonepicenter healthcare systems must anticipate disease requirements for noncritically ill patients hospitalized with COVID-19 in order to appropriately allocate resources, including monitoring services like continuous pulse oximetry and cardiac telemetry. Data regarding the incidence of in-hospital respiratory and cardiovascular complications, including arrhythmias, among non–intensive care unit (non-ICU) hospitalized patients with COVID-19 are limited, with little granularity in terms of associated variables.^7-11 Further data are needed to guide prioritization of valuable non-ICU continuous monitoring resources to the highest-risk patients in order to minimize consumption of personal protective equipment, reduce healthcare worker exposure, and ensure adequate availability for the expansion of prepandemic services.

Projections indicate that COVID-19 incidence may persist in the coming months.^11-13 As nonessential hospital operations simultaneously resume, planning for resource allocation for patients with COVID-19 must be incorporated into broader systems of care. Further data are needed to help hospitals anticipate resource needs during this transition, especially by most systems that are caring for COVID-19 patients in nonepicenter environments. Therefore, we conducted a retrospective study of a large, multihospital, nonepicenter health system equipped with centralized continuous monitoring services in order to describe the detailed clinical course, resource utilization, and risk factors for adverse events in patients with COVID-19 initially admitted to the non-ICU setting.

Central Monitoring Unit

The central monitoring unit (CMU) provides standardized and continuous off-site secondary monitoring of cardiac telemetry and pulse oximetry for non-ICU patients within Cleveland Clinic hospitals (Ohio, Florida), with direct communication to bedside nursing and inpatient emergency response teams for clinically significant cardiac arrhythmias, respiratory events, and vital sign changes according to standardized indications, as previously reported.¹⁴ Clinical variables of interest, including electrocardiographic and vital sign data, are collected and periodically analyzed within a central registry for quality assurance, risk stratification, and resource allocation. The data registry carries Institutional Review Board approval for retrospective analysis and deidentified outcomes reporting with consent form waiver.

Study Design and Data Collection

All patients positive for SARS-CoV-2 infection by nasopharyngeal polymerase chain reaction assay (Applied Biosystems) admitted from the emergency department to a non-ICU bed at a CMU hospital on or after March 13, 2020, and subsequently discharged on or before May 1, 2020, were identified. Retrospective review of the electronic medical record was performed, with follow-up continued through hospital discharge. Data were collected on patient demographics, clinical characteristics including admission laboratories and chest x-ray findings (abnormal defined as presence of an infiltrate/opacity consistent with airspace disease), continuous monitoring utilization, respiratory support, medication treatment, ICU transfer, and final hospital disposition. In addition, prospective recordings of cardiac arrhythmias that prompted CMU notification of bedside nursing were reviewed.

The primary outcome was a composite of death, ICU transfer, or increased oxygen requirement defined as escalation from simple nasal cannula to either high-flow nasal cannula (HFNC), noninvasive ventilation (NIV) consisting of continuous positive airway pressure (CPAP) or bilevel positive airway pressure (BiPAP), or mechanical ventilation. In accordance with published guidelines, patients were treated with supplemental oxygen to maintain peripheral oxygen saturation between 92% and 96%.¹⁵

Of note, based on the validated performance of high sensitivity troponin primarily for the diagnosis of acute myocardial infarction in patients presenting to the emergency department with chest pain, our system reserves its use for this context and prefers conventional (fourth generation) troponin T testing for inpatients. Therefore, conventional troponin T values are reported in this study.

Statistical Analyses

Continuous variables are expressed as mean ± standard deviation or median (interquartile range), and categorical variables are expressed as absolute numbers with percentages. Independent samples t and Mann-Whitney U tests were used to compare continuous variables, as appropriate, and chi-square testing was used to compare categorical variables. Clinical variables satisfying an a priori two-tailed threshold of P < .05 were retained for multivariable logistic regression analysis. Variables retaining P < .05 in multivariable modeling were considered statistically significant. Analyses were performed using SPSS software, Version 23 (SPSS Inc).

Baseline Characteristics

Between March 13, 2020, and May 1, 2020, a total of 350 patients admitted from the emergency department to a non-ICU inpatient bed had a final hospital disposition. Baseline characteristics, medication treatments, and continuous monitoring utilization are shown in Table 1 and Table 2. The average age was 64 ± 16 years, more than half of patients were male (n = 194; 55%), and most patients had at least one underlying comorbidity (n = 297; 85%), the most common being hypertension (n = 230; 66%), diabetes mellitus (n = 113; 32%), and current or prior tobacco use (n = 99; 28%). The presenting syndrome most frequently included subjective fever (n = 191; 55%), cough (n = 191; 55%), or dyspnea (n = 180; 51%).

Continuous Monitoring Use

Continuous monitoring was used in most patients (n = 289; 83%), including telemetry with intermittent pulse oximetry (n = 197; 56%), telemetry with continuous pulse oximetry (n = 81; 23%), or continuous pulse oximetry alone (n = 11; 3%). Among telemetry-monitored patients (n = 278; 79%), the most frequent indication was for a noncardiac disease state (n = 187; 67%), while indications for known cardiac arrhythmia (n = 74; 27%), heart failure (n = 10; 4%), or coronary artery disease (n = 7; 2%) were less common.

Oxygen Requirements and Cardiac Arrhythmias

The maximum level of respiratory support required by each patient is shown in Appendix Figure 1A. A total of 256 patients (73%) required 3 L/min or less of supplemental oxygen by nasal cannula, 45 (13%) required more than 3 L/min of supplemental oxygen by nasal cannula, 19 (5%) required HFNC, 8 (2%) required NIV, and 22 patients (6%) required mechanical ventilation. Among patients requiring HFNC or NIV, there were 13 (48%) who remained in a non-ICU bed, while the remaining 14 patients (52%) were transferred to the ICU.

Cardiac arrhythmias were detected in 39 (14%) of the 278 telemetry-monitored patients (Appendix Figure 1B). Clinical arrhythmias consisted of supraventricular tachycardia (SVT) in 17 patients (6%), nonsustained monomorphic ventricular tachycardia (VT) in 15 patients (5%), and a prolonged pause or severe bradyarrhythmia in 12 patients (4%). There were no cases of sustained monomorphic VT, polymorphic VT (including torsades de pointes), or ventricular fibrillation. All supraventricular tachycardias, nonsustained monomorphic VTs, and bradyarrhythmias/pauses were managed medically in the non-ICU setting, with the exception of one patient who was transferred to the ICU for a primary indication of atrial fibrillation with rapid ventricular response, which was treated with amiodarone. No patient with supraventricular tachycardia required emergent cardioversion, and no patient with a bradyarrhythmia or pause required temporary or permanent pacemaker implantation.

The detection of any arrhythmia was more common in patients with a history of cardiac arrhythmia (n = 18/41 vs 21/237; 44% vs 9%; P < .001), congestive heart failure (n = 11/36 vs 28/242; 31% vs 12%; P = .002), coronary artery disease (n = 12/49 vs 27/229; 24% vs 12%; P = .02), hypertension (n = 33/190 vs 6/88; 17% vs 7%; P = .02), and an abnormal admission troponin level (n = 13/40 vs 19/142; 33% vs 13%; P = .005). Notably, of the 39 patients with cardiac arrhythmias, 35 (90%) had either an abnormal admission troponin level or a history of cardiac arrhythmia, congestive heart failure, coronary artery disease, or hypertension. Of the 17 patients with SVT episodes, 13 (76%) had a known history of atrial fibrillation. Among patients who had a cardiac arrhythmia vs those who did not, there were no differences in levels of C-reactive protein (CRP; 7.3 ± 6.2 mg/dL vs. 7.8 ± 6.8 mg/dL, P = .63) or lactate dehydrogenase (LDH; 281 ± 89 U/L vs. 318 ± 142 U/L; P = .17). Approximately half of patients were treated with hydroxychloroquine (n = 185; 53%) or azithromycin (n = 182; 52%); 41% were treated with both (n = 142), with no observed association between any arrhythmia type and treatment with one or both medications (P > .05 for all comparisons).

Discharge Disposition and Adverse Outcomes

After an average length of stay of 6.1 ± 5.9 days, final hospital disposition included discharge to home (n = 278; 79%), discharge to subacute facility (n = 40; 11%), discharge to hospice (n = 8; 2%), death (n = 22, 6%), or release against medical advice (n = 2; 1%) (Figure). The primary composite outcome occurred in 62 patients (18%), including 22 deaths (6%), 48 ICU transfers (14%), and 49 patients with increased oxygen requirements (14%). Only two deaths occurred in the absence of an increased oxygen requirement or ICU transfer.

Increased oxygen requirement was the indication for ICU transfer in 37 of 48 patients (77%), with 22 patients (46%) requiring mechanical ventilation. Of the 48 patients requiring ICU transfer, 14 (29%) died, including 10 of the 22 patients (45%) treated with mechanical ventilation. Of the 302 patients who remained in the non-ICU setting, 8 (3%) died and 8 (3%) were discharged to hospice.

In univariable analyses, the primary composite outcome was more common among older patients (event vs event free, 72 ± 13 years vs 63 ± 16 years; P < .001); it was also more common in patients with congestive heart failure (n = 14/62 vs 28/288; 23% vs 10%; P = .005), chronic obstructive pulmonary disease (n = 9/62 vs 19/288; 15% vs 7%; P = .04), lower body mass index (29 ± 5 kg/m² vs 31 ± 7 kg/m²; P = .006), lower peripheral oxygen saturation on room air (93% ± 5% vs 95% ± 3%; P = .005), higher CRP level (12.0 ± 7.8 mg/dL vs 6.9 ± 6.1 mg/dL; P < .001), higher LDH level (358 ± 140 U/L vs 302 ± 133 U/L; P = .009), higher troponin level (0.05 ± 0.13 ng/dL vs 0.02 ± 0.06 ng/dL; P = .01), abnormal D-dimer level (n = 39/42 vs 102/145; 93% vs 70%; P = .003), and abnormal chest x-ray findings (n = 48/62 vs 166/285; 77% vs 58%; P = .005) (Table 1 and Table 2). After multivariable adjustment, CRP level (odds ratio [OR], 1.09 per 1 mg/dL increase; 95% CI, 1.01-1.18; P = .04) and LDH level (OR, 1.006 per 1 U/L increase; 95% CI, 1.001-1.012; P = .03) remained significantly associated with the composite adverse outcome (Table 3). The rate of death, ICU transfer, or increased oxygen requirement was sixfold higher in patients with a CRP level in the fourth quartile (≥11.0 mg/dL) than it was among those in the first quartile (≤ 2.6 mg/dL) (P < .001 for trend), and it was fivefold higher in patients with an LDH level in the fourth quartile (≥ 354 U/L) than it was among those in the first quartile (≤ 232 U/L) (P = .001 for trend) (Appendix Figure 2). No patient with a CRP level in the reference range (≤ 0.9 mg/dL) experienced the composite adverse event, compared to three patients (n = 3/49, 6.1%) within the reference range for LDH level (≤ 225 U/L), all of whom had an elevated CRP.

DISCUSSION

In this study of 350 patients initially admitted to a non-ICU hospital bed within a large, nonepicenter healthcare system, the primary outcome of death, ICU transfer, or increased oxygen requirement occurred in 18% of patients and was independently associated with higher admission CRP and LDH levels on multivariable analysis. Most patients (73%) required 3 L/min or less of supplemental oxygen, while 14% of patients required escalation to HFNC, NIV, or mechanical ventilation. Despite frequent telemetry use (79%), cardiac arrhythmias were uncommon (14%), including no life-threatening ventricular arrhythmias. Clinical deterioration requiring ICU transfer occurred in 14% of patients, most often for an indication of increased oxygen requirement (77%). In-hospital mortality was 6% for the entire cohort, 29% for patients requiring ICU transfer, and 3% for patients who remained in the non-ICU setting.

Nonepicenter, Non-ICU Mortality

This study offers an assessment of clinical outcomes in patients with COVID-19 hospitalized in a non-ICU, nonepicenter healthcare system operating below capacity. Although such systems account for most institutions caring for patients with COVID-19, this population has been underrepresented in the literature, which has focused on epicenter hospitals and critically ill patients.^3-7 Existing epicenter estimates of in-hospital mortality for patients not requiring ICU-level care range from 6% in Northern California² to at least 10% in New York, New York,³ and 11% in Wuhan, China.⁴ The corresponding non-ICU in-hospital mortality in our study was only 3%, supporting the vital role of social distancing in reducing COVID-19 mortality by facilitating care delivery in a non–resource limited hospital setting.

Oxygen Requirements and Cardiac Arrhythmias in Non-ICU Patients

Beyond nonepicenter mortality estimates, this study is the first to provide a detailed characterization of the clinical course and resource usage among patients with COVID-19 admitted to the non-ICU setting. Given the predicted persistence of SARS-CoV-2 spread,^11-13 this information is crucial to healthcare systems that must anticipate resource requirements, such as respiratory support and continuous monitoring equipment, for the care of hospitalized patients with COVID-19. Such informed planning takes on even greater importance as prepandemic hospital services resume.

While most patients (73%) with COVID-19 admitted to a non-ICU bed required peak supplemental oxygen of 3 L/min or less, a relevant proportion (14%) developed a need for HFNC, NIV, or mechanical ventilation. Furthermore, among telemetry-monitored patients (79%), cardiac arrhythmias were uncommon (14%), and nearly all (90%) occurred in patients with either a positive troponin or known history of cardiac disease. There were no life-threatening ventricular arrhythmias associated with frequent use of hydroxychloroquine (53%) and azithromycin (52%).

These telemetry findings expand upon a smaller study of non-ICU patients receiving either hydroxychloroquine or azithromycin, in which no life-threatening ventricular tachyarrhythmias were detected.⁸ A separate study reported a 5.9% incidence of malignant ventricular tachyarrhythmias in hospitalized patients with COVID-19,¹⁰ but this study did not stratify arrhythmias by illness severity, and a high frequency of critical illness is suggested by the mechanical ventilation rate of 24%, thereby limiting comparison with our non-ICU telemetry findings.

CRP and LDH Levels as Predictors of Adverse Outcomes

This study supports the utility of obtaining CRP and LDH levels for risk stratification at the time of non-ICU hospital admission. In multivariable analysis, higher CRP and LDH levels were significantly associated with the composite adverse outcome. The adverse event rates was increased sixfold between patients with a CRP in the fourth quartile (≥ 11.0 mg/dL, 36%) and those in the first quartile (≤ 2.6 mg/dL, 5.3%), and it was fivefold higher in patients with an LDH level in the fourth quartile (≥ 354 U/L, 34%) compared with those in the first quartile (≤ 232 U/L, 7%).

These findings are consistent with prior studies that have associated elevated inflammatory markers with poor prognosis and death.^7,9,16 In some cases, COVID-19 may manifest similar to a cytokine storm syndrome, which highlights the importance of inflammation-associated tissue injury and leads to widespread interest in the use of immunosuppressive medications.^17,18 Several studies also have demonstrated an association between LDH level and severe illness,^4,7,19 although this is the first to specifically demonstrate its association with clinical decompensation in the non-ICU hospitalized population. Given that SARS-CoV-2 can infect multiple organs,^20,21 there is biological plausibility for the use of LDH levels as a nonspecific marker of tissue injury for early identification of more severe infection.

Notably, while elevated troponin levels have been strongly associated with the need for mechanical ventilation and with death, this has primarily been established using either high-sensitivity troponin assays at the time of admission²² or using peak conventional troponin levels during hospitalization.¹⁰ In this study, while abnormal conventional troponin levels at the time of non-ICU admission were not significantly associated with the primary outcome in multivariable analysis, absolute troponin values were significantly higher in univariable analysis. Incomplete troponin sampling and the lack of routine high-sensitivity troponin assay use may explain the lack of more robust troponin significance in this study.

Implications for Non-ICU Continuous Monitoring Resource Allocation

Prioritization of non-ICU continuous monitoring resources among patients with COVID-19 has numerous benefits, including reduced consumption of personal protective equipment, fewer healthcare worker exposures, and adequate availability of continuous monitoring for the expansion of prepandemic hospital services. While individualized clinical discretion is still required, the results of this study can be used as a guide for the allocation of continuous pulse oximetry and cardiac telemetry. Patients with a normal presenting CRP level and/or LDH level had a low incidence of clinical decompensation, which suggests that such patients could be monitored with intermittent rather than continuous pulse oximetry. Furthermore, cardiac telemetry could be reserved for patients with a history of cardiac comorbidities or abnormal troponin levels because such patients accounted for 90% of cardiac arrhythmias in this study.

Limitations

This study was limited to a single health system, and it lacks a direct comparison to nonhospitalized patients and those directly admitted to the ICU. Triage practices and thresholds for hospitalization may differ across institutions and regions, thereby limiting the generalizability of our study. Additional limitations include the lack of selected admission laboratories for all patients, as well as the lack of telemetry monitoring in all patients. However, any resulting selection bias may be more likely to attenuate the magnitude of observed effects given that additional testing and increased telemetry use may be expected in patients who are felt to be higher risk by routine clinical assessment.

In this study of non–critically ill patients hospitalized within a nonepicenter health system, the development of more severe illness or death was significantly associated with higher levels of CRP and LDH on admission. Clinical decompensation was driven largely by respiratory complications, while cardiac arrhythmias were rare. Overall, the non-ICU mortality rate was at least half of that reported in epicenter regions. Altogether, these findings provide valuable information for resource allocation planning while nonepicenter health systems continue caring for patients with COVID-19 as they also resume prepandemic operations.

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), the cause of coronavirus disease 2019 (COVID-19), is associated with a wide range of illness severity and community prevalence, with an estimated 20% to 30% of patients requiring hospitalization.^1,2 Outcome studies of hospitalized patients to date have focused on epicenter healthcare systems operating at surge-level bed capacity in resource-limited settings with mortality exceeding 20% among patients with a discharge disposition^3,4 and have had a publication bias toward those suffering critical illness.^5-7 Generalizability of these results to nonepicenter hospital systems is unclear given potential differences in triage practices and resource availability according to disease prevalence, with nonepicenter systems that are operating below capacity potentially able to accommodate the needs of most, if not all patients, requiring inpatient level care. Clinical outcomes associated with non–critically ill patients in nonepicenter regions remain poorly characterized yet highly relevant because these will ultimately apply to most US and global healthcare environments.

Nonepicenter healthcare systems must anticipate disease requirements for noncritically ill patients hospitalized with COVID-19 in order to appropriately allocate resources, including monitoring services like continuous pulse oximetry and cardiac telemetry. Data regarding the incidence of in-hospital respiratory and cardiovascular complications, including arrhythmias, among non–intensive care unit (non-ICU) hospitalized patients with COVID-19 are limited, with little granularity in terms of associated variables.^7-11 Further data are needed to guide prioritization of valuable non-ICU continuous monitoring resources to the highest-risk patients in order to minimize consumption of personal protective equipment, reduce healthcare worker exposure, and ensure adequate availability for the expansion of prepandemic services.

Projections indicate that COVID-19 incidence may persist in the coming months.^11-13 As nonessential hospital operations simultaneously resume, planning for resource allocation for patients with COVID-19 must be incorporated into broader systems of care. Further data are needed to help hospitals anticipate resource needs during this transition, especially by most systems that are caring for COVID-19 patients in nonepicenter environments. Therefore, we conducted a retrospective study of a large, multihospital, nonepicenter health system equipped with centralized continuous monitoring services in order to describe the detailed clinical course, resource utilization, and risk factors for adverse events in patients with COVID-19 initially admitted to the non-ICU setting.

Central Monitoring Unit

The central monitoring unit (CMU) provides standardized and continuous off-site secondary monitoring of cardiac telemetry and pulse oximetry for non-ICU patients within Cleveland Clinic hospitals (Ohio, Florida), with direct communication to bedside nursing and inpatient emergency response teams for clinically significant cardiac arrhythmias, respiratory events, and vital sign changes according to standardized indications, as previously reported.¹⁴ Clinical variables of interest, including electrocardiographic and vital sign data, are collected and periodically analyzed within a central registry for quality assurance, risk stratification, and resource allocation. The data registry carries Institutional Review Board approval for retrospective analysis and deidentified outcomes reporting with consent form waiver.

Study Design and Data Collection

All patients positive for SARS-CoV-2 infection by nasopharyngeal polymerase chain reaction assay (Applied Biosystems) admitted from the emergency department to a non-ICU bed at a CMU hospital on or after March 13, 2020, and subsequently discharged on or before May 1, 2020, were identified. Retrospective review of the electronic medical record was performed, with follow-up continued through hospital discharge. Data were collected on patient demographics, clinical characteristics including admission laboratories and chest x-ray findings (abnormal defined as presence of an infiltrate/opacity consistent with airspace disease), continuous monitoring utilization, respiratory support, medication treatment, ICU transfer, and final hospital disposition. In addition, prospective recordings of cardiac arrhythmias that prompted CMU notification of bedside nursing were reviewed.

The primary outcome was a composite of death, ICU transfer, or increased oxygen requirement defined as escalation from simple nasal cannula to either high-flow nasal cannula (HFNC), noninvasive ventilation (NIV) consisting of continuous positive airway pressure (CPAP) or bilevel positive airway pressure (BiPAP), or mechanical ventilation. In accordance with published guidelines, patients were treated with supplemental oxygen to maintain peripheral oxygen saturation between 92% and 96%.¹⁵

Of note, based on the validated performance of high sensitivity troponin primarily for the diagnosis of acute myocardial infarction in patients presenting to the emergency department with chest pain, our system reserves its use for this context and prefers conventional (fourth generation) troponin T testing for inpatients. Therefore, conventional troponin T values are reported in this study.

Statistical Analyses

Continuous variables are expressed as mean ± standard deviation or median (interquartile range), and categorical variables are expressed as absolute numbers with percentages. Independent samples t and Mann-Whitney U tests were used to compare continuous variables, as appropriate, and chi-square testing was used to compare categorical variables. Clinical variables satisfying an a priori two-tailed threshold of P < .05 were retained for multivariable logistic regression analysis. Variables retaining P < .05 in multivariable modeling were considered statistically significant. Analyses were performed using SPSS software, Version 23 (SPSS Inc).

Baseline Characteristics

Between March 13, 2020, and May 1, 2020, a total of 350 patients admitted from the emergency department to a non-ICU inpatient bed had a final hospital disposition. Baseline characteristics, medication treatments, and continuous monitoring utilization are shown in Table 1 and Table 2. The average age was 64 ± 16 years, more than half of patients were male (n = 194; 55%), and most patients had at least one underlying comorbidity (n = 297; 85%), the most common being hypertension (n = 230; 66%), diabetes mellitus (n = 113; 32%), and current or prior tobacco use (n = 99; 28%). The presenting syndrome most frequently included subjective fever (n = 191; 55%), cough (n = 191; 55%), or dyspnea (n = 180; 51%).

Continuous Monitoring Use

Continuous monitoring was used in most patients (n = 289; 83%), including telemetry with intermittent pulse oximetry (n = 197; 56%), telemetry with continuous pulse oximetry (n = 81; 23%), or continuous pulse oximetry alone (n = 11; 3%). Among telemetry-monitored patients (n = 278; 79%), the most frequent indication was for a noncardiac disease state (n = 187; 67%), while indications for known cardiac arrhythmia (n = 74; 27%), heart failure (n = 10; 4%), or coronary artery disease (n = 7; 2%) were less common.

Oxygen Requirements and Cardiac Arrhythmias

The maximum level of respiratory support required by each patient is shown in Appendix Figure 1A. A total of 256 patients (73%) required 3 L/min or less of supplemental oxygen by nasal cannula, 45 (13%) required more than 3 L/min of supplemental oxygen by nasal cannula, 19 (5%) required HFNC, 8 (2%) required NIV, and 22 patients (6%) required mechanical ventilation. Among patients requiring HFNC or NIV, there were 13 (48%) who remained in a non-ICU bed, while the remaining 14 patients (52%) were transferred to the ICU.

Cardiac arrhythmias were detected in 39 (14%) of the 278 telemetry-monitored patients (Appendix Figure 1B). Clinical arrhythmias consisted of supraventricular tachycardia (SVT) in 17 patients (6%), nonsustained monomorphic ventricular tachycardia (VT) in 15 patients (5%), and a prolonged pause or severe bradyarrhythmia in 12 patients (4%). There were no cases of sustained monomorphic VT, polymorphic VT (including torsades de pointes), or ventricular fibrillation. All supraventricular tachycardias, nonsustained monomorphic VTs, and bradyarrhythmias/pauses were managed medically in the non-ICU setting, with the exception of one patient who was transferred to the ICU for a primary indication of atrial fibrillation with rapid ventricular response, which was treated with amiodarone. No patient with supraventricular tachycardia required emergent cardioversion, and no patient with a bradyarrhythmia or pause required temporary or permanent pacemaker implantation.

The detection of any arrhythmia was more common in patients with a history of cardiac arrhythmia (n = 18/41 vs 21/237; 44% vs 9%; P < .001), congestive heart failure (n = 11/36 vs 28/242; 31% vs 12%; P = .002), coronary artery disease (n = 12/49 vs 27/229; 24% vs 12%; P = .02), hypertension (n = 33/190 vs 6/88; 17% vs 7%; P = .02), and an abnormal admission troponin level (n = 13/40 vs 19/142; 33% vs 13%; P = .005). Notably, of the 39 patients with cardiac arrhythmias, 35 (90%) had either an abnormal admission troponin level or a history of cardiac arrhythmia, congestive heart failure, coronary artery disease, or hypertension. Of the 17 patients with SVT episodes, 13 (76%) had a known history of atrial fibrillation. Among patients who had a cardiac arrhythmia vs those who did not, there were no differences in levels of C-reactive protein (CRP; 7.3 ± 6.2 mg/dL vs. 7.8 ± 6.8 mg/dL, P = .63) or lactate dehydrogenase (LDH; 281 ± 89 U/L vs. 318 ± 142 U/L; P = .17). Approximately half of patients were treated with hydroxychloroquine (n = 185; 53%) or azithromycin (n = 182; 52%); 41% were treated with both (n = 142), with no observed association between any arrhythmia type and treatment with one or both medications (P > .05 for all comparisons).

Discharge Disposition and Adverse Outcomes

After an average length of stay of 6.1 ± 5.9 days, final hospital disposition included discharge to home (n = 278; 79%), discharge to subacute facility (n = 40; 11%), discharge to hospice (n = 8; 2%), death (n = 22, 6%), or release against medical advice (n = 2; 1%) (Figure). The primary composite outcome occurred in 62 patients (18%), including 22 deaths (6%), 48 ICU transfers (14%), and 49 patients with increased oxygen requirements (14%). Only two deaths occurred in the absence of an increased oxygen requirement or ICU transfer.

Increased oxygen requirement was the indication for ICU transfer in 37 of 48 patients (77%), with 22 patients (46%) requiring mechanical ventilation. Of the 48 patients requiring ICU transfer, 14 (29%) died, including 10 of the 22 patients (45%) treated with mechanical ventilation. Of the 302 patients who remained in the non-ICU setting, 8 (3%) died and 8 (3%) were discharged to hospice.

In univariable analyses, the primary composite outcome was more common among older patients (event vs event free, 72 ± 13 years vs 63 ± 16 years; P < .001); it was also more common in patients with congestive heart failure (n = 14/62 vs 28/288; 23% vs 10%; P = .005), chronic obstructive pulmonary disease (n = 9/62 vs 19/288; 15% vs 7%; P = .04), lower body mass index (29 ± 5 kg/m² vs 31 ± 7 kg/m²; P = .006), lower peripheral oxygen saturation on room air (93% ± 5% vs 95% ± 3%; P = .005), higher CRP level (12.0 ± 7.8 mg/dL vs 6.9 ± 6.1 mg/dL; P < .001), higher LDH level (358 ± 140 U/L vs 302 ± 133 U/L; P = .009), higher troponin level (0.05 ± 0.13 ng/dL vs 0.02 ± 0.06 ng/dL; P = .01), abnormal D-dimer level (n = 39/42 vs 102/145; 93% vs 70%; P = .003), and abnormal chest x-ray findings (n = 48/62 vs 166/285; 77% vs 58%; P = .005) (Table 1 and Table 2). After multivariable adjustment, CRP level (odds ratio [OR], 1.09 per 1 mg/dL increase; 95% CI, 1.01-1.18; P = .04) and LDH level (OR, 1.006 per 1 U/L increase; 95% CI, 1.001-1.012; P = .03) remained significantly associated with the composite adverse outcome (Table 3). The rate of death, ICU transfer, or increased oxygen requirement was sixfold higher in patients with a CRP level in the fourth quartile (≥11.0 mg/dL) than it was among those in the first quartile (≤ 2.6 mg/dL) (P < .001 for trend), and it was fivefold higher in patients with an LDH level in the fourth quartile (≥ 354 U/L) than it was among those in the first quartile (≤ 232 U/L) (P = .001 for trend) (Appendix Figure 2). No patient with a CRP level in the reference range (≤ 0.9 mg/dL) experienced the composite adverse event, compared to three patients (n = 3/49, 6.1%) within the reference range for LDH level (≤ 225 U/L), all of whom had an elevated CRP.

DISCUSSION

In this study of 350 patients initially admitted to a non-ICU hospital bed within a large, nonepicenter healthcare system, the primary outcome of death, ICU transfer, or increased oxygen requirement occurred in 18% of patients and was independently associated with higher admission CRP and LDH levels on multivariable analysis. Most patients (73%) required 3 L/min or less of supplemental oxygen, while 14% of patients required escalation to HFNC, NIV, or mechanical ventilation. Despite frequent telemetry use (79%), cardiac arrhythmias were uncommon (14%), including no life-threatening ventricular arrhythmias. Clinical deterioration requiring ICU transfer occurred in 14% of patients, most often for an indication of increased oxygen requirement (77%). In-hospital mortality was 6% for the entire cohort, 29% for patients requiring ICU transfer, and 3% for patients who remained in the non-ICU setting.

Nonepicenter, Non-ICU Mortality

This study offers an assessment of clinical outcomes in patients with COVID-19 hospitalized in a non-ICU, nonepicenter healthcare system operating below capacity. Although such systems account for most institutions caring for patients with COVID-19, this population has been underrepresented in the literature, which has focused on epicenter hospitals and critically ill patients.^3-7 Existing epicenter estimates of in-hospital mortality for patients not requiring ICU-level care range from 6% in Northern California² to at least 10% in New York, New York,³ and 11% in Wuhan, China.⁴ The corresponding non-ICU in-hospital mortality in our study was only 3%, supporting the vital role of social distancing in reducing COVID-19 mortality by facilitating care delivery in a non–resource limited hospital setting.

Oxygen Requirements and Cardiac Arrhythmias in Non-ICU Patients

Beyond nonepicenter mortality estimates, this study is the first to provide a detailed characterization of the clinical course and resource usage among patients with COVID-19 admitted to the non-ICU setting. Given the predicted persistence of SARS-CoV-2 spread,^11-13 this information is crucial to healthcare systems that must anticipate resource requirements, such as respiratory support and continuous monitoring equipment, for the care of hospitalized patients with COVID-19. Such informed planning takes on even greater importance as prepandemic hospital services resume.

While most patients (73%) with COVID-19 admitted to a non-ICU bed required peak supplemental oxygen of 3 L/min or less, a relevant proportion (14%) developed a need for HFNC, NIV, or mechanical ventilation. Furthermore, among telemetry-monitored patients (79%), cardiac arrhythmias were uncommon (14%), and nearly all (90%) occurred in patients with either a positive troponin or known history of cardiac disease. There were no life-threatening ventricular arrhythmias associated with frequent use of hydroxychloroquine (53%) and azithromycin (52%).

These telemetry findings expand upon a smaller study of non-ICU patients receiving either hydroxychloroquine or azithromycin, in which no life-threatening ventricular tachyarrhythmias were detected.⁸ A separate study reported a 5.9% incidence of malignant ventricular tachyarrhythmias in hospitalized patients with COVID-19,¹⁰ but this study did not stratify arrhythmias by illness severity, and a high frequency of critical illness is suggested by the mechanical ventilation rate of 24%, thereby limiting comparison with our non-ICU telemetry findings.

CRP and LDH Levels as Predictors of Adverse Outcomes

This study supports the utility of obtaining CRP and LDH levels for risk stratification at the time of non-ICU hospital admission. In multivariable analysis, higher CRP and LDH levels were significantly associated with the composite adverse outcome. The adverse event rates was increased sixfold between patients with a CRP in the fourth quartile (≥ 11.0 mg/dL, 36%) and those in the first quartile (≤ 2.6 mg/dL, 5.3%), and it was fivefold higher in patients with an LDH level in the fourth quartile (≥ 354 U/L, 34%) compared with those in the first quartile (≤ 232 U/L, 7%).

These findings are consistent with prior studies that have associated elevated inflammatory markers with poor prognosis and death.^7,9,16 In some cases, COVID-19 may manifest similar to a cytokine storm syndrome, which highlights the importance of inflammation-associated tissue injury and leads to widespread interest in the use of immunosuppressive medications.^17,18 Several studies also have demonstrated an association between LDH level and severe illness,^4,7,19 although this is the first to specifically demonstrate its association with clinical decompensation in the non-ICU hospitalized population. Given that SARS-CoV-2 can infect multiple organs,^20,21 there is biological plausibility for the use of LDH levels as a nonspecific marker of tissue injury for early identification of more severe infection.

Notably, while elevated troponin levels have been strongly associated with the need for mechanical ventilation and with death, this has primarily been established using either high-sensitivity troponin assays at the time of admission²² or using peak conventional troponin levels during hospitalization.¹⁰ In this study, while abnormal conventional troponin levels at the time of non-ICU admission were not significantly associated with the primary outcome in multivariable analysis, absolute troponin values were significantly higher in univariable analysis. Incomplete troponin sampling and the lack of routine high-sensitivity troponin assay use may explain the lack of more robust troponin significance in this study.

Implications for Non-ICU Continuous Monitoring Resource Allocation

Prioritization of non-ICU continuous monitoring resources among patients with COVID-19 has numerous benefits, including reduced consumption of personal protective equipment, fewer healthcare worker exposures, and adequate availability of continuous monitoring for the expansion of prepandemic hospital services. While individualized clinical discretion is still required, the results of this study can be used as a guide for the allocation of continuous pulse oximetry and cardiac telemetry. Patients with a normal presenting CRP level and/or LDH level had a low incidence of clinical decompensation, which suggests that such patients could be monitored with intermittent rather than continuous pulse oximetry. Furthermore, cardiac telemetry could be reserved for patients with a history of cardiac comorbidities or abnormal troponin levels because such patients accounted for 90% of cardiac arrhythmias in this study.

Limitations

This study was limited to a single health system, and it lacks a direct comparison to nonhospitalized patients and those directly admitted to the ICU. Triage practices and thresholds for hospitalization may differ across institutions and regions, thereby limiting the generalizability of our study. Additional limitations include the lack of selected admission laboratories for all patients, as well as the lack of telemetry monitoring in all patients. However, any resulting selection bias may be more likely to attenuate the magnitude of observed effects given that additional testing and increased telemetry use may be expected in patients who are felt to be higher risk by routine clinical assessment.

In this study of non–critically ill patients hospitalized within a nonepicenter health system, the development of more severe illness or death was significantly associated with higher levels of CRP and LDH on admission. Clinical decompensation was driven largely by respiratory complications, while cardiac arrhythmias were rare. Overall, the non-ICU mortality rate was at least half of that reported in epicenter regions. Altogether, these findings provide valuable information for resource allocation planning while nonepicenter health systems continue caring for patients with COVID-19 as they also resume prepandemic operations.

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), the cause of coronavirus disease 2019 (COVID-19), is associated with a wide range of illness severity and community prevalence, with an estimated 20% to 30% of patients requiring hospitalization.^1,2 Outcome studies of hospitalized patients to date have focused on epicenter healthcare systems operating at surge-level bed capacity in resource-limited settings with mortality exceeding 20% among patients with a discharge disposition^3,4 and have had a publication bias toward those suffering critical illness.^5-7 Generalizability of these results to nonepicenter hospital systems is unclear given potential differences in triage practices and resource availability according to disease prevalence, with nonepicenter systems that are operating below capacity potentially able to accommodate the needs of most, if not all patients, requiring inpatient level care. Clinical outcomes associated with non–critically ill patients in nonepicenter regions remain poorly characterized yet highly relevant because these will ultimately apply to most US and global healthcare environments.

Nonepicenter healthcare systems must anticipate disease requirements for noncritically ill patients hospitalized with COVID-19 in order to appropriately allocate resources, including monitoring services like continuous pulse oximetry and cardiac telemetry. Data regarding the incidence of in-hospital respiratory and cardiovascular complications, including arrhythmias, among non–intensive care unit (non-ICU) hospitalized patients with COVID-19 are limited, with little granularity in terms of associated variables.^7-11 Further data are needed to guide prioritization of valuable non-ICU continuous monitoring resources to the highest-risk patients in order to minimize consumption of personal protective equipment, reduce healthcare worker exposure, and ensure adequate availability for the expansion of prepandemic services.

Projections indicate that COVID-19 incidence may persist in the coming months.^11-13 As nonessential hospital operations simultaneously resume, planning for resource allocation for patients with COVID-19 must be incorporated into broader systems of care. Further data are needed to help hospitals anticipate resource needs during this transition, especially by most systems that are caring for COVID-19 patients in nonepicenter environments. Therefore, we conducted a retrospective study of a large, multihospital, nonepicenter health system equipped with centralized continuous monitoring services in order to describe the detailed clinical course, resource utilization, and risk factors for adverse events in patients with COVID-19 initially admitted to the non-ICU setting.

Central Monitoring Unit

The central monitoring unit (CMU) provides standardized and continuous off-site secondary monitoring of cardiac telemetry and pulse oximetry for non-ICU patients within Cleveland Clinic hospitals (Ohio, Florida), with direct communication to bedside nursing and inpatient emergency response teams for clinically significant cardiac arrhythmias, respiratory events, and vital sign changes according to standardized indications, as previously reported.¹⁴ Clinical variables of interest, including electrocardiographic and vital sign data, are collected and periodically analyzed within a central registry for quality assurance, risk stratification, and resource allocation. The data registry carries Institutional Review Board approval for retrospective analysis and deidentified outcomes reporting with consent form waiver.

Study Design and Data Collection

All patients positive for SARS-CoV-2 infection by nasopharyngeal polymerase chain reaction assay (Applied Biosystems) admitted from the emergency department to a non-ICU bed at a CMU hospital on or after March 13, 2020, and subsequently discharged on or before May 1, 2020, were identified. Retrospective review of the electronic medical record was performed, with follow-up continued through hospital discharge. Data were collected on patient demographics, clinical characteristics including admission laboratories and chest x-ray findings (abnormal defined as presence of an infiltrate/opacity consistent with airspace disease), continuous monitoring utilization, respiratory support, medication treatment, ICU transfer, and final hospital disposition. In addition, prospective recordings of cardiac arrhythmias that prompted CMU notification of bedside nursing were reviewed.

The primary outcome was a composite of death, ICU transfer, or increased oxygen requirement defined as escalation from simple nasal cannula to either high-flow nasal cannula (HFNC), noninvasive ventilation (NIV) consisting of continuous positive airway pressure (CPAP) or bilevel positive airway pressure (BiPAP), or mechanical ventilation. In accordance with published guidelines, patients were treated with supplemental oxygen to maintain peripheral oxygen saturation between 92% and 96%.¹⁵

Of note, based on the validated performance of high sensitivity troponin primarily for the diagnosis of acute myocardial infarction in patients presenting to the emergency department with chest pain, our system reserves its use for this context and prefers conventional (fourth generation) troponin T testing for inpatients. Therefore, conventional troponin T values are reported in this study.

Statistical Analyses

Continuous variables are expressed as mean ± standard deviation or median (interquartile range), and categorical variables are expressed as absolute numbers with percentages. Independent samples t and Mann-Whitney U tests were used to compare continuous variables, as appropriate, and chi-square testing was used to compare categorical variables. Clinical variables satisfying an a priori two-tailed threshold of P < .05 were retained for multivariable logistic regression analysis. Variables retaining P < .05 in multivariable modeling were considered statistically significant. Analyses were performed using SPSS software, Version 23 (SPSS Inc).

Baseline Characteristics

Between March 13, 2020, and May 1, 2020, a total of 350 patients admitted from the emergency department to a non-ICU inpatient bed had a final hospital disposition. Baseline characteristics, medication treatments, and continuous monitoring utilization are shown in Table 1 and Table 2. The average age was 64 ± 16 years, more than half of patients were male (n = 194; 55%), and most patients had at least one underlying comorbidity (n = 297; 85%), the most common being hypertension (n = 230; 66%), diabetes mellitus (n = 113; 32%), and current or prior tobacco use (n = 99; 28%). The presenting syndrome most frequently included subjective fever (n = 191; 55%), cough (n = 191; 55%), or dyspnea (n = 180; 51%).

Continuous Monitoring Use

Continuous monitoring was used in most patients (n = 289; 83%), including telemetry with intermittent pulse oximetry (n = 197; 56%), telemetry with continuous pulse oximetry (n = 81; 23%), or continuous pulse oximetry alone (n = 11; 3%). Among telemetry-monitored patients (n = 278; 79%), the most frequent indication was for a noncardiac disease state (n = 187; 67%), while indications for known cardiac arrhythmia (n = 74; 27%), heart failure (n = 10; 4%), or coronary artery disease (n = 7; 2%) were less common.

Oxygen Requirements and Cardiac Arrhythmias

The maximum level of respiratory support required by each patient is shown in Appendix Figure 1A. A total of 256 patients (73%) required 3 L/min or less of supplemental oxygen by nasal cannula, 45 (13%) required more than 3 L/min of supplemental oxygen by nasal cannula, 19 (5%) required HFNC, 8 (2%) required NIV, and 22 patients (6%) required mechanical ventilation. Among patients requiring HFNC or NIV, there were 13 (48%) who remained in a non-ICU bed, while the remaining 14 patients (52%) were transferred to the ICU.

Cardiac arrhythmias were detected in 39 (14%) of the 278 telemetry-monitored patients (Appendix Figure 1B). Clinical arrhythmias consisted of supraventricular tachycardia (SVT) in 17 patients (6%), nonsustained monomorphic ventricular tachycardia (VT) in 15 patients (5%), and a prolonged pause or severe bradyarrhythmia in 12 patients (4%). There were no cases of sustained monomorphic VT, polymorphic VT (including torsades de pointes), or ventricular fibrillation. All supraventricular tachycardias, nonsustained monomorphic VTs, and bradyarrhythmias/pauses were managed medically in the non-ICU setting, with the exception of one patient who was transferred to the ICU for a primary indication of atrial fibrillation with rapid ventricular response, which was treated with amiodarone. No patient with supraventricular tachycardia required emergent cardioversion, and no patient with a bradyarrhythmia or pause required temporary or permanent pacemaker implantation.

The detection of any arrhythmia was more common in patients with a history of cardiac arrhythmia (n = 18/41 vs 21/237; 44% vs 9%; P < .001), congestive heart failure (n = 11/36 vs 28/242; 31% vs 12%; P = .002), coronary artery disease (n = 12/49 vs 27/229; 24% vs 12%; P = .02), hypertension (n = 33/190 vs 6/88; 17% vs 7%; P = .02), and an abnormal admission troponin level (n = 13/40 vs 19/142; 33% vs 13%; P = .005). Notably, of the 39 patients with cardiac arrhythmias, 35 (90%) had either an abnormal admission troponin level or a history of cardiac arrhythmia, congestive heart failure, coronary artery disease, or hypertension. Of the 17 patients with SVT episodes, 13 (76%) had a known history of atrial fibrillation. Among patients who had a cardiac arrhythmia vs those who did not, there were no differences in levels of C-reactive protein (CRP; 7.3 ± 6.2 mg/dL vs. 7.8 ± 6.8 mg/dL, P = .63) or lactate dehydrogenase (LDH; 281 ± 89 U/L vs. 318 ± 142 U/L; P = .17). Approximately half of patients were treated with hydroxychloroquine (n = 185; 53%) or azithromycin (n = 182; 52%); 41% were treated with both (n = 142), with no observed association between any arrhythmia type and treatment with one or both medications (P > .05 for all comparisons).

Discharge Disposition and Adverse Outcomes

After an average length of stay of 6.1 ± 5.9 days, final hospital disposition included discharge to home (n = 278; 79%), discharge to subacute facility (n = 40; 11%), discharge to hospice (n = 8; 2%), death (n = 22, 6%), or release against medical advice (n = 2; 1%) (Figure). The primary composite outcome occurred in 62 patients (18%), including 22 deaths (6%), 48 ICU transfers (14%), and 49 patients with increased oxygen requirements (14%). Only two deaths occurred in the absence of an increased oxygen requirement or ICU transfer.

Increased oxygen requirement was the indication for ICU transfer in 37 of 48 patients (77%), with 22 patients (46%) requiring mechanical ventilation. Of the 48 patients requiring ICU transfer, 14 (29%) died, including 10 of the 22 patients (45%) treated with mechanical ventilation. Of the 302 patients who remained in the non-ICU setting, 8 (3%) died and 8 (3%) were discharged to hospice.

In univariable analyses, the primary composite outcome was more common among older patients (event vs event free, 72 ± 13 years vs 63 ± 16 years; P < .001); it was also more common in patients with congestive heart failure (n = 14/62 vs 28/288; 23% vs 10%; P = .005), chronic obstructive pulmonary disease (n = 9/62 vs 19/288; 15% vs 7%; P = .04), lower body mass index (29 ± 5 kg/m² vs 31 ± 7 kg/m²; P = .006), lower peripheral oxygen saturation on room air (93% ± 5% vs 95% ± 3%; P = .005), higher CRP level (12.0 ± 7.8 mg/dL vs 6.9 ± 6.1 mg/dL; P < .001), higher LDH level (358 ± 140 U/L vs 302 ± 133 U/L; P = .009), higher troponin level (0.05 ± 0.13 ng/dL vs 0.02 ± 0.06 ng/dL; P = .01), abnormal D-dimer level (n = 39/42 vs 102/145; 93% vs 70%; P = .003), and abnormal chest x-ray findings (n = 48/62 vs 166/285; 77% vs 58%; P = .005) (Table 1 and Table 2). After multivariable adjustment, CRP level (odds ratio [OR], 1.09 per 1 mg/dL increase; 95% CI, 1.01-1.18; P = .04) and LDH level (OR, 1.006 per 1 U/L increase; 95% CI, 1.001-1.012; P = .03) remained significantly associated with the composite adverse outcome (Table 3). The rate of death, ICU transfer, or increased oxygen requirement was sixfold higher in patients with a CRP level in the fourth quartile (≥11.0 mg/dL) than it was among those in the first quartile (≤ 2.6 mg/dL) (P < .001 for trend), and it was fivefold higher in patients with an LDH level in the fourth quartile (≥ 354 U/L) than it was among those in the first quartile (≤ 232 U/L) (P = .001 for trend) (Appendix Figure 2). No patient with a CRP level in the reference range (≤ 0.9 mg/dL) experienced the composite adverse event, compared to three patients (n = 3/49, 6.1%) within the reference range for LDH level (≤ 225 U/L), all of whom had an elevated CRP.

DISCUSSION

In this study of 350 patients initially admitted to a non-ICU hospital bed within a large, nonepicenter healthcare system, the primary outcome of death, ICU transfer, or increased oxygen requirement occurred in 18% of patients and was independently associated with higher admission CRP and LDH levels on multivariable analysis. Most patients (73%) required 3 L/min or less of supplemental oxygen, while 14% of patients required escalation to HFNC, NIV, or mechanical ventilation. Despite frequent telemetry use (79%), cardiac arrhythmias were uncommon (14%), including no life-threatening ventricular arrhythmias. Clinical deterioration requiring ICU transfer occurred in 14% of patients, most often for an indication of increased oxygen requirement (77%). In-hospital mortality was 6% for the entire cohort, 29% for patients requiring ICU transfer, and 3% for patients who remained in the non-ICU setting.

Nonepicenter, Non-ICU Mortality

This study offers an assessment of clinical outcomes in patients with COVID-19 hospitalized in a non-ICU, nonepicenter healthcare system operating below capacity. Although such systems account for most institutions caring for patients with COVID-19, this population has been underrepresented in the literature, which has focused on epicenter hospitals and critically ill patients.^3-7 Existing epicenter estimates of in-hospital mortality for patients not requiring ICU-level care range from 6% in Northern California² to at least 10% in New York, New York,³ and 11% in Wuhan, China.⁴ The corresponding non-ICU in-hospital mortality in our study was only 3%, supporting the vital role of social distancing in reducing COVID-19 mortality by facilitating care delivery in a non–resource limited hospital setting.

Oxygen Requirements and Cardiac Arrhythmias in Non-ICU Patients

Beyond nonepicenter mortality estimates, this study is the first to provide a detailed characterization of the clinical course and resource usage among patients with COVID-19 admitted to the non-ICU setting. Given the predicted persistence of SARS-CoV-2 spread,^11-13 this information is crucial to healthcare systems that must anticipate resource requirements, such as respiratory support and continuous monitoring equipment, for the care of hospitalized patients with COVID-19. Such informed planning takes on even greater importance as prepandemic hospital services resume.

While most patients (73%) with COVID-19 admitted to a non-ICU bed required peak supplemental oxygen of 3 L/min or less, a relevant proportion (14%) developed a need for HFNC, NIV, or mechanical ventilation. Furthermore, among telemetry-monitored patients (79%), cardiac arrhythmias were uncommon (14%), and nearly all (90%) occurred in patients with either a positive troponin or known history of cardiac disease. There were no life-threatening ventricular arrhythmias associated with frequent use of hydroxychloroquine (53%) and azithromycin (52%).

These telemetry findings expand upon a smaller study of non-ICU patients receiving either hydroxychloroquine or azithromycin, in which no life-threatening ventricular tachyarrhythmias were detected.⁸ A separate study reported a 5.9% incidence of malignant ventricular tachyarrhythmias in hospitalized patients with COVID-19,¹⁰ but this study did not stratify arrhythmias by illness severity, and a high frequency of critical illness is suggested by the mechanical ventilation rate of 24%, thereby limiting comparison with our non-ICU telemetry findings.

CRP and LDH Levels as Predictors of Adverse Outcomes

This study supports the utility of obtaining CRP and LDH levels for risk stratification at the time of non-ICU hospital admission. In multivariable analysis, higher CRP and LDH levels were significantly associated with the composite adverse outcome. The adverse event rates was increased sixfold between patients with a CRP in the fourth quartile (≥ 11.0 mg/dL, 36%) and those in the first quartile (≤ 2.6 mg/dL, 5.3%), and it was fivefold higher in patients with an LDH level in the fourth quartile (≥ 354 U/L, 34%) compared with those in the first quartile (≤ 232 U/L, 7%).

These findings are consistent with prior studies that have associated elevated inflammatory markers with poor prognosis and death.^7,9,16 In some cases, COVID-19 may manifest similar to a cytokine storm syndrome, which highlights the importance of inflammation-associated tissue injury and leads to widespread interest in the use of immunosuppressive medications.^17,18 Several studies also have demonstrated an association between LDH level and severe illness,^4,7,19 although this is the first to specifically demonstrate its association with clinical decompensation in the non-ICU hospitalized population. Given that SARS-CoV-2 can infect multiple organs,^20,21 there is biological plausibility for the use of LDH levels as a nonspecific marker of tissue injury for early identification of more severe infection.

Notably, while elevated troponin levels have been strongly associated with the need for mechanical ventilation and with death, this has primarily been established using either high-sensitivity troponin assays at the time of admission²² or using peak conventional troponin levels during hospitalization.¹⁰ In this study, while abnormal conventional troponin levels at the time of non-ICU admission were not significantly associated with the primary outcome in multivariable analysis, absolute troponin values were significantly higher in univariable analysis. Incomplete troponin sampling and the lack of routine high-sensitivity troponin assay use may explain the lack of more robust troponin significance in this study.

Implications for Non-ICU Continuous Monitoring Resource Allocation

Prioritization of non-ICU continuous monitoring resources among patients with COVID-19 has numerous benefits, including reduced consumption of personal protective equipment, fewer healthcare worker exposures, and adequate availability of continuous monitoring for the expansion of prepandemic hospital services. While individualized clinical discretion is still required, the results of this study can be used as a guide for the allocation of continuous pulse oximetry and cardiac telemetry. Patients with a normal presenting CRP level and/or LDH level had a low incidence of clinical decompensation, which suggests that such patients could be monitored with intermittent rather than continuous pulse oximetry. Furthermore, cardiac telemetry could be reserved for patients with a history of cardiac comorbidities or abnormal troponin levels because such patients accounted for 90% of cardiac arrhythmias in this study.

Limitations

This study was limited to a single health system, and it lacks a direct comparison to nonhospitalized patients and those directly admitted to the ICU. Triage practices and thresholds for hospitalization may differ across institutions and regions, thereby limiting the generalizability of our study. Additional limitations include the lack of selected admission laboratories for all patients, as well as the lack of telemetry monitoring in all patients. However, any resulting selection bias may be more likely to attenuate the magnitude of observed effects given that additional testing and increased telemetry use may be expected in patients who are felt to be higher risk by routine clinical assessment.

In this study of non–critically ill patients hospitalized within a nonepicenter health system, the development of more severe illness or death was significantly associated with higher levels of CRP and LDH on admission. Clinical decompensation was driven largely by respiratory complications, while cardiac arrhythmias were rare. Overall, the non-ICU mortality rate was at least half of that reported in epicenter regions. Altogether, these findings provide valuable information for resource allocation planning while nonepicenter health systems continue caring for patients with COVID-19 as they also resume prepandemic operations.

Rising US healthcare costs coupled with high cost-sharing insurance plans have led to increased out-of-pocket healthcare expenditures, especially for those who are low income or in poorer health.^1-7 Increased out-of-pocket expenditures can lead to “financial distress” (defined as the subjective level of stress felt toward one’s personal financial situation) and to “medical financial burden” (defined as the subjective assessment of financial problems relating specifically to medical costs). Financial distress and medical financial burden (defined together as “financial difficulty”) lead to impaired access and delayed presentation to care and treatment nonadherence in hopes of alleviating costs.^8-12

Between 20% and 50% of families with children requiring frequent medical care report that their child’s healthcare has caused a financial difficulty.^13,14 In addition to direct medical costs, these parents can also suffer from indirect costs of their child’s care, such as unemployment or missed work.^15-17 Along with these families, families who are low income (generally defined as living below 200% of the Federal Poverty Level) also have higher absolute and relative out-of-pocket healthcare costs, and both groups are more likely to have unmet medical needs or to delay or forgo care.^18-20 Medically complex children also represent an increasing percentage of patients admitted to children’s hospitals^21,22 where their families may be more vulnerable to worsening financial difficulties caused by direct costs and income depletion—due to lost wages, transportation, and meals—associated with hospitalization.²³

The hospitalized population can be readily screened and provided interventions. Although evidence on effective inpatient financial interventions is lacking, financial navigation programs piloted in the ambulatory setting that standardize financial screening and support trained financial navigators could prove a promising model for inpatient care.^24-26 Therefore, understanding the prevalence of financial difficulties in this population and potential high-yield screening characteristics is critical in laying the groundwork for more robust in-hospital financial screening and support systems.

Our primary objective was to assess the prevalence of financial distress and medical financial burden in families of hospitalized children. Our secondary objective was to examine measurable factors during hospitalization that could identify families at risk for these financial difficulties to better understand how to target and implement hospital-based interventions.

We conducted a cross-sectional survey at six university-affiliated children’s hospitals (Table 1). Each site’s institutional review board approved the study. All participants were verbally informed of the research goals of the study and provided with a research information document. Need for written informed consent was determined by each institutional review board.

Study enrollment occurred between October 2017 and November 2018, with individual sites having shorter active enrollment periods (ranging from 25 to 100 days) until sample size goals were met as explained below. Participants represented a convenience sample of parents or guardians (hereafter referred to only as “parents”), who were eligible for enrollment if their child was admitted to one of the six hospitals during the active enrollment period at that site. To avoid sampling bias, each site made an effort to enroll a consecutive sample of parents, but this was limited by resources and investigator availability. Parents were excluded if their child was admitted to a neonatal unit because of difficulty in complexity categorization and the confounding issue of mothers often being admitted simultaneously. There were no other unit-, diagnosis-, or service-based exclusions to participation. Parents were also excluded if their child was 18 years or older or if they themselves were younger than 18 years. Parents were approached once their child was identified for discharge from the hospital within 48 hours. Surveys were self-administered at the time of enrollment on provided electronic tablets. Participants at some sites were offered a $5 gift card as an incentive for survey completion.

The survey included a previously published financial distress scale (InCharge Financial Distress/Financial Wellbeing Scale [IFDFW])(Appendix).²⁷ A question in addition to the IFDFW assessed whether families were currently experiencing financial burden from medical care^28,29 and whether that burden was caused by their child (Appendix) because the IFDFW does not address the source of financial distress. The survey also included questions assessing perspectives on healthcare costs (data not presented here). The survey was refined through review by psychometric experts and members of the Family Advisory Council at the primary research site, which led to minor modifications. The final survey consisted of 40 items and was professionally translated into Spanish by a third-party company (Idem Translations). It was pilot tested by 10 parents of hospitalized children to assess for adequate comprehension and clarity; these parents were not included in the final data analysis.

Variables

The primary outcome variables were level of financial distress as defined by the IFDFW scale²⁷ and the presence of medical financial burden. The IFDFW scale has eight questions answered on a scale of 1-10, and the final score is calculated by averaging these answers. The scale defines three categories of financial distress (high, 1-3.9; average, 4-6.9; low, 7-10); however, we dichotomized our outcome as high (<4) or not high (≥4). The outcome was analyzed as both continuous and dichotomous variables because small differences in continuous scores, if detected, may be less clinically relevant. Medical financial burden was categorized as child related, child unrelated, and none.

Our secondary aim was to identify predictors of financial distress and medical financial burden. The primary predictor variable of interest was the hospitalized child’s level of chronic disease (complex chronic disease, C-CD; noncomplex chronic disease, NC-CD; no chronic disease, no-CD) as categorized by the consensus definitions from the Center of Excellence on Quality of Care Measures for Children with Complex Needs (Appendix).³⁰ We assigned level of chronic disease based on manual review of problem lists and diagnoses in the electronic health record (EHR) from up to 3 years prior. At sites with multiple researchers, the first five to ten charts were reviewed together to ensure consistency in categorization, but no formal assessment of interrater reliability was conducted. Other predictor variables are listed in Tables 2 and 3. Insurance payer was defined as “public” or “private” based on the documented insurance plan in the EHR. Patients with dual public and private insurance were categorized as public.

Statistical Analysis

We estimated sample size requirements using an expected mean IFDFW score with standard deviation of 5.7 ± 2 based on preliminary data from the primary study site and previously published data.²⁷ We used a significance level of P = .05, power of 0.80, and an effect size of 0.5 points difference on the IFDFW scale between the families of children with C-CD and those with either NC-CD or no-CD. We assumed there would be unequal representation of chronic disease states, with an expectation that children with C-CD would make up approximately 40% of the total population.^21,22,31 Under these assumptions, we calculated a desired total sample size of 519. This would also allow us to detect a 12% absolute difference in the rate of high financial distress between families with and without C-CD, assuming a baseline level of high financial distress of 30%.²⁷ Our goal enrollment was 150 parents at the primary site and 75 parents at each of the other 5 sites.

We fit mixed effects logistic regression models to evaluate the odds of high financial distress and polytomous logistic regression models (for our three-level outcome) to evaluate the odds of having child-related medical financial burden vs having child-unrelated burden vs having no burden. We fit linear mixed effects models to evaluate the effect of chronic disease level and medical financial burden on mean IFDFW scores. Respondents who answered “I don’t know” to the medical financial burden question were aggregated with those who reported no medical financial burden. Models were fit as a function of chronic disease level, race, ethnicity, percentage of Federal Poverty Level (FPL), insurance payer, and having a deductible less than $1,000 per year. These models included a random intercept for facility. We also fit logistic regression models that used an interaction term between chronic disease level and percentage of FPL, as well as insurance payer and percentage of FPL, to explore potential effect modification between poverty and both chronic disease level and insurance payer on financial distress. For our models, we used the MICE package for multiple imputation to fill in missing data. We imputed 25 data sets with 25 iterations each and pooled model results using Rubin’s Rules.³² All analyses were performed in R 3.5.³³

Of 644 parents who were invited to participate, 526 (82%) were enrolled. Participants and their hospitalized children were mostly White/Caucasian (69%) and not Hispanic/Latino (76%), with 34% of families living below 200% FPL and 274 (52%) having private insurance (Table 1). Of the hospitalized children, 225 (43%) were categorized as C-CD, 143 (27%) as NC-CD, and 157 (30%) as no-CD. All participants completed the IFDFW; however, there were five missing responses to the medical financial burden question. Table 1 lists missing demographic and financial difficulty data.

Financial Distress

The mean IFDFW score of all participants was 5.6 ± 2.1, with 125 having high financial distress (24%; 95% CI, 20-28) (Table 1). There was no difference in mean IFDFW scores among families of children with different chronic disease levels (Figure). On unadjusted and adjusted analyses, there was no association between level of chronic disease and high financial distress when C-CD and NC-CD groups were each compared with no-CD (Table 2). However, families living below 400% FPL (annual income of $100,400 for a family of four) were significantly more likely than families living at 400% FPL and above to have high financial distress. Families tended to have lower financial distress (as indicated by mean IFDFW scores) with increasing percentage of FPL; however, there were families in every FPL bracket who experienced high financial distress (Appendix Figure 1a). A secondary analysis of families below and those at or above 200% FPL did not find any significant interactions between percentage of FPL and either chronic disease level (P = .86) or insurance payer (P = .83) on financial distress.

Overall, 160 parents (30%; 95% CI, 27-35) reported having medical financial burden, with 86 of those parents (54%) indicating their financial burden was related to their child’s medical care (Table 1). Compared with families with no such medical financial burden, respondents with medical financial burden, either child related or child unrelated, had significantly lower mean IFDFW scores (Figure), which indicates overall higher financial distress in these families. However, some families with low financial distress also reported medical financial burden.

Adjusted analyses demonstrated that, compared with families of children with no-CD, families of children with C-CD (adjusted odds ratio [AOR], 4.98; 95% CI, 2.41-10.29) or NC-CD (AOR, 2.57; 95% CI, 1.11-5.93) had significantly higher odds of having child-related medical financial burden (Table 3). Families of children with NC-CD were also more likely than families of children with no-CD to have child-unrelated medical burden (Table 3). Percentage of FPL was the only other significant predictor of child-related and child-unrelated medical financial burden (Table 3), but as with the distribution of financial distress, medical financial burden was seen across family income brackets (Appendix Figure 1b).

In this multicenter study of parents of hospitalized children, almost one in four families experienced high financial distress and almost one in three families reported having medical financial burden, with both measures of financial difficulty affecting families across all income levels. While these percentages are not substantially higher than those seen in the general population,^27,34 70% of our population was composed of children with chronic disease who are more likely to have short-term and long-term healthcare needs, which places them at risk for significant ongoing medical costs.

We hypothesized that families of children with complex chronic disease would have higher levels of financial difficulties,^13,35,36 but we found that level of chronic disease was associated only with medical financial burden and not with high financial distress. Financial distress is likely multifactorial and dynamic, with different drivers across various income levels. Therefore, while medical financial burden likely contributes to high financial distress, there may be other contributing factors not captured by the IFDFW. However, subjective medical financial burden has still been associated with impaired access to care.^10,34 Therefore, our results suggest that families of children with chronic diseases might be at higher risk for barriers to consistent healthcare because of the financial burden their frequent healthcare utilization incurs.

Household poverty level was also associated with financial distress and medical financial burden, although surprisingly both measures of financial difficulty were present in all FPL brackets. This highlights an important reality that financial vulnerability extends beyond income and federally defined “poverty.” Non-income factors, such as high local costs of living and the growing problem of underinsurance, may significantly contribute to financial difficulty, which may render static financial metrics such as percentage of FPL insufficient screeners. Furthermore, as evidenced by the nearly 10% of our respondents who declined to provide their income information, this is a sensitive topic for some families, so gathering income data during admission could likely be a nonstarter.

In the absence of other consistent predictors of financial difficulty that could trigger interventions such as an automatic financial counselor consult, hospitals and healthcare providers could consider implementing routine non-income based financial screening questions on admission, such as one assessing medical financial burden, as a nondiscriminatory way of identifying at-risk families and provide further education and assistance regarding their financial needs. Systematically gathering this data may also further demonstrate the need for broad financial navigation programs as a mainstay in comprehensive inpatient care.

We acknowledge several limitations of this study. Primarily, we surveyed families prior to discharge and receipt of hospitalization-related bills, and these bills could contribute significantly to financial difficulties. While the families of children with chronic disease, who likely have recurrent medical bills, did not demonstrate higher financial distress, it is possible that the overall rate of financial difficulties would have been higher had we surveyed families several weeks after discharge. Our measures of financial difficulty were also subjective and, therefore, at risk for response biases (such as recall bias) that could have misestimated the prevalence of these problems in our population. However, published literature on the IFDFW scale demonstrates concordance between the subjective score and tangible outcomes of financial distress (eg, contacting a credit agency). The IFDFW scale was validated in the general population, and although it has been used in studies of medical populations,^37-41 none have been in hospitalized populations, which may affect the scale’s applicability in our study. The study was also conducted only at university-affiliated children’s hospitals, and although these hospitals are geographically diverse, most children in the United States are admitted to general or community hospitals.³¹ Our population was also largely White, non-Hispanic/Latino, and English speaking. Therefore, our sample may not reflect the general population of hospitalized children and their families. We also assigned levels of chronic disease based on manual EHR review. While the EHR should capture each patient’s breadth of medical issues, inaccurate or missing documentation could have led to misclassification of complexity in some cases. Additionally, our sample size was calculated to detect fairly large differences in our primary outcome, and some of our unexpected results may have resulted from this study being underpowered for detection of smaller, but perhaps still clinically relevant, differences. Finally, we do not have data for several possible confounders in our study, such as employment status, health insurance concordance among family members, or sources of supplemental income, that may impact a family’s overall financial health, along with some potential important hospital-based screening characteristics, such as admitting service team or primary diagnosis.

Financial difficulties are common in families of hospitalized pediatric patients. Low-income families and those who have children with chronic conditions are at particular risk; however, all subsets of families can be affected. Given the potential negative health outcomes financial difficulties impose on families and children, the ability to identify and support vulnerable families is a crucial component of care. Hospitalization may be a prime opportunity to identify and support our at-risk families.

The authors would like to thank the parents at each of the study sites for their participation, as well as the multiple research coordinators across the study sites for assisting in recruitment of families, survey administration, and data collection. KT Park, MD, MS (Stanford University School of Medicine) served as an adviser for the study’s design.

All authors have no financial relationships or conflicts of interest relevant to this article to disclose.

Rising US healthcare costs coupled with high cost-sharing insurance plans have led to increased out-of-pocket healthcare expenditures, especially for those who are low income or in poorer health.^1-7 Increased out-of-pocket expenditures can lead to “financial distress” (defined as the subjective level of stress felt toward one’s personal financial situation) and to “medical financial burden” (defined as the subjective assessment of financial problems relating specifically to medical costs). Financial distress and medical financial burden (defined together as “financial difficulty”) lead to impaired access and delayed presentation to care and treatment nonadherence in hopes of alleviating costs.^8-12

Between 20% and 50% of families with children requiring frequent medical care report that their child’s healthcare has caused a financial difficulty.^13,14 In addition to direct medical costs, these parents can also suffer from indirect costs of their child’s care, such as unemployment or missed work.^15-17 Along with these families, families who are low income (generally defined as living below 200% of the Federal Poverty Level) also have higher absolute and relative out-of-pocket healthcare costs, and both groups are more likely to have unmet medical needs or to delay or forgo care.^18-20 Medically complex children also represent an increasing percentage of patients admitted to children’s hospitals^21,22 where their families may be more vulnerable to worsening financial difficulties caused by direct costs and income depletion—due to lost wages, transportation, and meals—associated with hospitalization.²³

The hospitalized population can be readily screened and provided interventions. Although evidence on effective inpatient financial interventions is lacking, financial navigation programs piloted in the ambulatory setting that standardize financial screening and support trained financial navigators could prove a promising model for inpatient care.^24-26 Therefore, understanding the prevalence of financial difficulties in this population and potential high-yield screening characteristics is critical in laying the groundwork for more robust in-hospital financial screening and support systems.

Our primary objective was to assess the prevalence of financial distress and medical financial burden in families of hospitalized children. Our secondary objective was to examine measurable factors during hospitalization that could identify families at risk for these financial difficulties to better understand how to target and implement hospital-based interventions.

We conducted a cross-sectional survey at six university-affiliated children’s hospitals (Table 1). Each site’s institutional review board approved the study. All participants were verbally informed of the research goals of the study and provided with a research information document. Need for written informed consent was determined by each institutional review board.

Study enrollment occurred between October 2017 and November 2018, with individual sites having shorter active enrollment periods (ranging from 25 to 100 days) until sample size goals were met as explained below. Participants represented a convenience sample of parents or guardians (hereafter referred to only as “parents”), who were eligible for enrollment if their child was admitted to one of the six hospitals during the active enrollment period at that site. To avoid sampling bias, each site made an effort to enroll a consecutive sample of parents, but this was limited by resources and investigator availability. Parents were excluded if their child was admitted to a neonatal unit because of difficulty in complexity categorization and the confounding issue of mothers often being admitted simultaneously. There were no other unit-, diagnosis-, or service-based exclusions to participation. Parents were also excluded if their child was 18 years or older or if they themselves were younger than 18 years. Parents were approached once their child was identified for discharge from the hospital within 48 hours. Surveys were self-administered at the time of enrollment on provided electronic tablets. Participants at some sites were offered a $5 gift card as an incentive for survey completion.

The survey included a previously published financial distress scale (InCharge Financial Distress/Financial Wellbeing Scale [IFDFW])(Appendix).²⁷ A question in addition to the IFDFW assessed whether families were currently experiencing financial burden from medical care^28,29 and whether that burden was caused by their child (Appendix) because the IFDFW does not address the source of financial distress. The survey also included questions assessing perspectives on healthcare costs (data not presented here). The survey was refined through review by psychometric experts and members of the Family Advisory Council at the primary research site, which led to minor modifications. The final survey consisted of 40 items and was professionally translated into Spanish by a third-party company (Idem Translations). It was pilot tested by 10 parents of hospitalized children to assess for adequate comprehension and clarity; these parents were not included in the final data analysis.

Variables

The primary outcome variables were level of financial distress as defined by the IFDFW scale²⁷ and the presence of medical financial burden. The IFDFW scale has eight questions answered on a scale of 1-10, and the final score is calculated by averaging these answers. The scale defines three categories of financial distress (high, 1-3.9; average, 4-6.9; low, 7-10); however, we dichotomized our outcome as high (<4) or not high (≥4). The outcome was analyzed as both continuous and dichotomous variables because small differences in continuous scores, if detected, may be less clinically relevant. Medical financial burden was categorized as child related, child unrelated, and none.

Our secondary aim was to identify predictors of financial distress and medical financial burden. The primary predictor variable of interest was the hospitalized child’s level of chronic disease (complex chronic disease, C-CD; noncomplex chronic disease, NC-CD; no chronic disease, no-CD) as categorized by the consensus definitions from the Center of Excellence on Quality of Care Measures for Children with Complex Needs (Appendix).³⁰ We assigned level of chronic disease based on manual review of problem lists and diagnoses in the electronic health record (EHR) from up to 3 years prior. At sites with multiple researchers, the first five to ten charts were reviewed together to ensure consistency in categorization, but no formal assessment of interrater reliability was conducted. Other predictor variables are listed in Tables 2 and 3. Insurance payer was defined as “public” or “private” based on the documented insurance plan in the EHR. Patients with dual public and private insurance were categorized as public.

Statistical Analysis

We estimated sample size requirements using an expected mean IFDFW score with standard deviation of 5.7 ± 2 based on preliminary data from the primary study site and previously published data.²⁷ We used a significance level of P = .05, power of 0.80, and an effect size of 0.5 points difference on the IFDFW scale between the families of children with C-CD and those with either NC-CD or no-CD. We assumed there would be unequal representation of chronic disease states, with an expectation that children with C-CD would make up approximately 40% of the total population.^21,22,31 Under these assumptions, we calculated a desired total sample size of 519. This would also allow us to detect a 12% absolute difference in the rate of high financial distress between families with and without C-CD, assuming a baseline level of high financial distress of 30%.²⁷ Our goal enrollment was 150 parents at the primary site and 75 parents at each of the other 5 sites.

We fit mixed effects logistic regression models to evaluate the odds of high financial distress and polytomous logistic regression models (for our three-level outcome) to evaluate the odds of having child-related medical financial burden vs having child-unrelated burden vs having no burden. We fit linear mixed effects models to evaluate the effect of chronic disease level and medical financial burden on mean IFDFW scores. Respondents who answered “I don’t know” to the medical financial burden question were aggregated with those who reported no medical financial burden. Models were fit as a function of chronic disease level, race, ethnicity, percentage of Federal Poverty Level (FPL), insurance payer, and having a deductible less than $1,000 per year. These models included a random intercept for facility. We also fit logistic regression models that used an interaction term between chronic disease level and percentage of FPL, as well as insurance payer and percentage of FPL, to explore potential effect modification between poverty and both chronic disease level and insurance payer on financial distress. For our models, we used the MICE package for multiple imputation to fill in missing data. We imputed 25 data sets with 25 iterations each and pooled model results using Rubin’s Rules.³² All analyses were performed in R 3.5.³³

Of 644 parents who were invited to participate, 526 (82%) were enrolled. Participants and their hospitalized children were mostly White/Caucasian (69%) and not Hispanic/Latino (76%), with 34% of families living below 200% FPL and 274 (52%) having private insurance (Table 1). Of the hospitalized children, 225 (43%) were categorized as C-CD, 143 (27%) as NC-CD, and 157 (30%) as no-CD. All participants completed the IFDFW; however, there were five missing responses to the medical financial burden question. Table 1 lists missing demographic and financial difficulty data.

Financial Distress

The mean IFDFW score of all participants was 5.6 ± 2.1, with 125 having high financial distress (24%; 95% CI, 20-28) (Table 1). There was no difference in mean IFDFW scores among families of children with different chronic disease levels (Figure). On unadjusted and adjusted analyses, there was no association between level of chronic disease and high financial distress when C-CD and NC-CD groups were each compared with no-CD (Table 2). However, families living below 400% FPL (annual income of $100,400 for a family of four) were significantly more likely than families living at 400% FPL and above to have high financial distress. Families tended to have lower financial distress (as indicated by mean IFDFW scores) with increasing percentage of FPL; however, there were families in every FPL bracket who experienced high financial distress (Appendix Figure 1a). A secondary analysis of families below and those at or above 200% FPL did not find any significant interactions between percentage of FPL and either chronic disease level (P = .86) or insurance payer (P = .83) on financial distress.

Overall, 160 parents (30%; 95% CI, 27-35) reported having medical financial burden, with 86 of those parents (54%) indicating their financial burden was related to their child’s medical care (Table 1). Compared with families with no such medical financial burden, respondents with medical financial burden, either child related or child unrelated, had significantly lower mean IFDFW scores (Figure), which indicates overall higher financial distress in these families. However, some families with low financial distress also reported medical financial burden.

Adjusted analyses demonstrated that, compared with families of children with no-CD, families of children with C-CD (adjusted odds ratio [AOR], 4.98; 95% CI, 2.41-10.29) or NC-CD (AOR, 2.57; 95% CI, 1.11-5.93) had significantly higher odds of having child-related medical financial burden (Table 3). Families of children with NC-CD were also more likely than families of children with no-CD to have child-unrelated medical burden (Table 3). Percentage of FPL was the only other significant predictor of child-related and child-unrelated medical financial burden (Table 3), but as with the distribution of financial distress, medical financial burden was seen across family income brackets (Appendix Figure 1b).

In this multicenter study of parents of hospitalized children, almost one in four families experienced high financial distress and almost one in three families reported having medical financial burden, with both measures of financial difficulty affecting families across all income levels. While these percentages are not substantially higher than those seen in the general population,^27,34 70% of our population was composed of children with chronic disease who are more likely to have short-term and long-term healthcare needs, which places them at risk for significant ongoing medical costs.

We hypothesized that families of children with complex chronic disease would have higher levels of financial difficulties,^13,35,36 but we found that level of chronic disease was associated only with medical financial burden and not with high financial distress. Financial distress is likely multifactorial and dynamic, with different drivers across various income levels. Therefore, while medical financial burden likely contributes to high financial distress, there may be other contributing factors not captured by the IFDFW. However, subjective medical financial burden has still been associated with impaired access to care.^10,34 Therefore, our results suggest that families of children with chronic diseases might be at higher risk for barriers to consistent healthcare because of the financial burden their frequent healthcare utilization incurs.

Household poverty level was also associated with financial distress and medical financial burden, although surprisingly both measures of financial difficulty were present in all FPL brackets. This highlights an important reality that financial vulnerability extends beyond income and federally defined “poverty.” Non-income factors, such as high local costs of living and the growing problem of underinsurance, may significantly contribute to financial difficulty, which may render static financial metrics such as percentage of FPL insufficient screeners. Furthermore, as evidenced by the nearly 10% of our respondents who declined to provide their income information, this is a sensitive topic for some families, so gathering income data during admission could likely be a nonstarter.

In the absence of other consistent predictors of financial difficulty that could trigger interventions such as an automatic financial counselor consult, hospitals and healthcare providers could consider implementing routine non-income based financial screening questions on admission, such as one assessing medical financial burden, as a nondiscriminatory way of identifying at-risk families and provide further education and assistance regarding their financial needs. Systematically gathering this data may also further demonstrate the need for broad financial navigation programs as a mainstay in comprehensive inpatient care.

We acknowledge several limitations of this study. Primarily, we surveyed families prior to discharge and receipt of hospitalization-related bills, and these bills could contribute significantly to financial difficulties. While the families of children with chronic disease, who likely have recurrent medical bills, did not demonstrate higher financial distress, it is possible that the overall rate of financial difficulties would have been higher had we surveyed families several weeks after discharge. Our measures of financial difficulty were also subjective and, therefore, at risk for response biases (such as recall bias) that could have misestimated the prevalence of these problems in our population. However, published literature on the IFDFW scale demonstrates concordance between the subjective score and tangible outcomes of financial distress (eg, contacting a credit agency). The IFDFW scale was validated in the general population, and although it has been used in studies of medical populations,^37-41 none have been in hospitalized populations, which may affect the scale’s applicability in our study. The study was also conducted only at university-affiliated children’s hospitals, and although these hospitals are geographically diverse, most children in the United States are admitted to general or community hospitals.³¹ Our population was also largely White, non-Hispanic/Latino, and English speaking. Therefore, our sample may not reflect the general population of hospitalized children and their families. We also assigned levels of chronic disease based on manual EHR review. While the EHR should capture each patient’s breadth of medical issues, inaccurate or missing documentation could have led to misclassification of complexity in some cases. Additionally, our sample size was calculated to detect fairly large differences in our primary outcome, and some of our unexpected results may have resulted from this study being underpowered for detection of smaller, but perhaps still clinically relevant, differences. Finally, we do not have data for several possible confounders in our study, such as employment status, health insurance concordance among family members, or sources of supplemental income, that may impact a family’s overall financial health, along with some potential important hospital-based screening characteristics, such as admitting service team or primary diagnosis.

Financial difficulties are common in families of hospitalized pediatric patients. Low-income families and those who have children with chronic conditions are at particular risk; however, all subsets of families can be affected. Given the potential negative health outcomes financial difficulties impose on families and children, the ability to identify and support vulnerable families is a crucial component of care. Hospitalization may be a prime opportunity to identify and support our at-risk families.

The authors would like to thank the parents at each of the study sites for their participation, as well as the multiple research coordinators across the study sites for assisting in recruitment of families, survey administration, and data collection. KT Park, MD, MS (Stanford University School of Medicine) served as an adviser for the study’s design.

All authors have no financial relationships or conflicts of interest relevant to this article to disclose.

Rising US healthcare costs coupled with high cost-sharing insurance plans have led to increased out-of-pocket healthcare expenditures, especially for those who are low income or in poorer health.^1-7 Increased out-of-pocket expenditures can lead to “financial distress” (defined as the subjective level of stress felt toward one’s personal financial situation) and to “medical financial burden” (defined as the subjective assessment of financial problems relating specifically to medical costs). Financial distress and medical financial burden (defined together as “financial difficulty”) lead to impaired access and delayed presentation to care and treatment nonadherence in hopes of alleviating costs.^8-12

Between 20% and 50% of families with children requiring frequent medical care report that their child’s healthcare has caused a financial difficulty.^13,14 In addition to direct medical costs, these parents can also suffer from indirect costs of their child’s care, such as unemployment or missed work.^15-17 Along with these families, families who are low income (generally defined as living below 200% of the Federal Poverty Level) also have higher absolute and relative out-of-pocket healthcare costs, and both groups are more likely to have unmet medical needs or to delay or forgo care.^18-20 Medically complex children also represent an increasing percentage of patients admitted to children’s hospitals^21,22 where their families may be more vulnerable to worsening financial difficulties caused by direct costs and income depletion—due to lost wages, transportation, and meals—associated with hospitalization.²³

The hospitalized population can be readily screened and provided interventions. Although evidence on effective inpatient financial interventions is lacking, financial navigation programs piloted in the ambulatory setting that standardize financial screening and support trained financial navigators could prove a promising model for inpatient care.^24-26 Therefore, understanding the prevalence of financial difficulties in this population and potential high-yield screening characteristics is critical in laying the groundwork for more robust in-hospital financial screening and support systems.

Our primary objective was to assess the prevalence of financial distress and medical financial burden in families of hospitalized children. Our secondary objective was to examine measurable factors during hospitalization that could identify families at risk for these financial difficulties to better understand how to target and implement hospital-based interventions.

We conducted a cross-sectional survey at six university-affiliated children’s hospitals (Table 1). Each site’s institutional review board approved the study. All participants were verbally informed of the research goals of the study and provided with a research information document. Need for written informed consent was determined by each institutional review board.

Study enrollment occurred between October 2017 and November 2018, with individual sites having shorter active enrollment periods (ranging from 25 to 100 days) until sample size goals were met as explained below. Participants represented a convenience sample of parents or guardians (hereafter referred to only as “parents”), who were eligible for enrollment if their child was admitted to one of the six hospitals during the active enrollment period at that site. To avoid sampling bias, each site made an effort to enroll a consecutive sample of parents, but this was limited by resources and investigator availability. Parents were excluded if their child was admitted to a neonatal unit because of difficulty in complexity categorization and the confounding issue of mothers often being admitted simultaneously. There were no other unit-, diagnosis-, or service-based exclusions to participation. Parents were also excluded if their child was 18 years or older or if they themselves were younger than 18 years. Parents were approached once their child was identified for discharge from the hospital within 48 hours. Surveys were self-administered at the time of enrollment on provided electronic tablets. Participants at some sites were offered a $5 gift card as an incentive for survey completion.

The survey included a previously published financial distress scale (InCharge Financial Distress/Financial Wellbeing Scale [IFDFW])(Appendix).²⁷ A question in addition to the IFDFW assessed whether families were currently experiencing financial burden from medical care^28,29 and whether that burden was caused by their child (Appendix) because the IFDFW does not address the source of financial distress. The survey also included questions assessing perspectives on healthcare costs (data not presented here). The survey was refined through review by psychometric experts and members of the Family Advisory Council at the primary research site, which led to minor modifications. The final survey consisted of 40 items and was professionally translated into Spanish by a third-party company (Idem Translations). It was pilot tested by 10 parents of hospitalized children to assess for adequate comprehension and clarity; these parents were not included in the final data analysis.

Variables

The primary outcome variables were level of financial distress as defined by the IFDFW scale²⁷ and the presence of medical financial burden. The IFDFW scale has eight questions answered on a scale of 1-10, and the final score is calculated by averaging these answers. The scale defines three categories of financial distress (high, 1-3.9; average, 4-6.9; low, 7-10); however, we dichotomized our outcome as high (<4) or not high (≥4). The outcome was analyzed as both continuous and dichotomous variables because small differences in continuous scores, if detected, may be less clinically relevant. Medical financial burden was categorized as child related, child unrelated, and none.

Our secondary aim was to identify predictors of financial distress and medical financial burden. The primary predictor variable of interest was the hospitalized child’s level of chronic disease (complex chronic disease, C-CD; noncomplex chronic disease, NC-CD; no chronic disease, no-CD) as categorized by the consensus definitions from the Center of Excellence on Quality of Care Measures for Children with Complex Needs (Appendix).³⁰ We assigned level of chronic disease based on manual review of problem lists and diagnoses in the electronic health record (EHR) from up to 3 years prior. At sites with multiple researchers, the first five to ten charts were reviewed together to ensure consistency in categorization, but no formal assessment of interrater reliability was conducted. Other predictor variables are listed in Tables 2 and 3. Insurance payer was defined as “public” or “private” based on the documented insurance plan in the EHR. Patients with dual public and private insurance were categorized as public.

Statistical Analysis

We estimated sample size requirements using an expected mean IFDFW score with standard deviation of 5.7 ± 2 based on preliminary data from the primary study site and previously published data.²⁷ We used a significance level of P = .05, power of 0.80, and an effect size of 0.5 points difference on the IFDFW scale between the families of children with C-CD and those with either NC-CD or no-CD. We assumed there would be unequal representation of chronic disease states, with an expectation that children with C-CD would make up approximately 40% of the total population.^21,22,31 Under these assumptions, we calculated a desired total sample size of 519. This would also allow us to detect a 12% absolute difference in the rate of high financial distress between families with and without C-CD, assuming a baseline level of high financial distress of 30%.²⁷ Our goal enrollment was 150 parents at the primary site and 75 parents at each of the other 5 sites.

We fit mixed effects logistic regression models to evaluate the odds of high financial distress and polytomous logistic regression models (for our three-level outcome) to evaluate the odds of having child-related medical financial burden vs having child-unrelated burden vs having no burden. We fit linear mixed effects models to evaluate the effect of chronic disease level and medical financial burden on mean IFDFW scores. Respondents who answered “I don’t know” to the medical financial burden question were aggregated with those who reported no medical financial burden. Models were fit as a function of chronic disease level, race, ethnicity, percentage of Federal Poverty Level (FPL), insurance payer, and having a deductible less than $1,000 per year. These models included a random intercept for facility. We also fit logistic regression models that used an interaction term between chronic disease level and percentage of FPL, as well as insurance payer and percentage of FPL, to explore potential effect modification between poverty and both chronic disease level and insurance payer on financial distress. For our models, we used the MICE package for multiple imputation to fill in missing data. We imputed 25 data sets with 25 iterations each and pooled model results using Rubin’s Rules.³² All analyses were performed in R 3.5.³³

Of 644 parents who were invited to participate, 526 (82%) were enrolled. Participants and their hospitalized children were mostly White/Caucasian (69%) and not Hispanic/Latino (76%), with 34% of families living below 200% FPL and 274 (52%) having private insurance (Table 1). Of the hospitalized children, 225 (43%) were categorized as C-CD, 143 (27%) as NC-CD, and 157 (30%) as no-CD. All participants completed the IFDFW; however, there were five missing responses to the medical financial burden question. Table 1 lists missing demographic and financial difficulty data.

Financial Distress

The mean IFDFW score of all participants was 5.6 ± 2.1, with 125 having high financial distress (24%; 95% CI, 20-28) (Table 1). There was no difference in mean IFDFW scores among families of children with different chronic disease levels (Figure). On unadjusted and adjusted analyses, there was no association between level of chronic disease and high financial distress when C-CD and NC-CD groups were each compared with no-CD (Table 2). However, families living below 400% FPL (annual income of $100,400 for a family of four) were significantly more likely than families living at 400% FPL and above to have high financial distress. Families tended to have lower financial distress (as indicated by mean IFDFW scores) with increasing percentage of FPL; however, there were families in every FPL bracket who experienced high financial distress (Appendix Figure 1a). A secondary analysis of families below and those at or above 200% FPL did not find any significant interactions between percentage of FPL and either chronic disease level (P = .86) or insurance payer (P = .83) on financial distress.

Overall, 160 parents (30%; 95% CI, 27-35) reported having medical financial burden, with 86 of those parents (54%) indicating their financial burden was related to their child’s medical care (Table 1). Compared with families with no such medical financial burden, respondents with medical financial burden, either child related or child unrelated, had significantly lower mean IFDFW scores (Figure), which indicates overall higher financial distress in these families. However, some families with low financial distress also reported medical financial burden.

Adjusted analyses demonstrated that, compared with families of children with no-CD, families of children with C-CD (adjusted odds ratio [AOR], 4.98; 95% CI, 2.41-10.29) or NC-CD (AOR, 2.57; 95% CI, 1.11-5.93) had significantly higher odds of having child-related medical financial burden (Table 3). Families of children with NC-CD were also more likely than families of children with no-CD to have child-unrelated medical burden (Table 3). Percentage of FPL was the only other significant predictor of child-related and child-unrelated medical financial burden (Table 3), but as with the distribution of financial distress, medical financial burden was seen across family income brackets (Appendix Figure 1b).

In this multicenter study of parents of hospitalized children, almost one in four families experienced high financial distress and almost one in three families reported having medical financial burden, with both measures of financial difficulty affecting families across all income levels. While these percentages are not substantially higher than those seen in the general population,^27,34 70% of our population was composed of children with chronic disease who are more likely to have short-term and long-term healthcare needs, which places them at risk for significant ongoing medical costs.

We hypothesized that families of children with complex chronic disease would have higher levels of financial difficulties,^13,35,36 but we found that level of chronic disease was associated only with medical financial burden and not with high financial distress. Financial distress is likely multifactorial and dynamic, with different drivers across various income levels. Therefore, while medical financial burden likely contributes to high financial distress, there may be other contributing factors not captured by the IFDFW. However, subjective medical financial burden has still been associated with impaired access to care.^10,34 Therefore, our results suggest that families of children with chronic diseases might be at higher risk for barriers to consistent healthcare because of the financial burden their frequent healthcare utilization incurs.

Household poverty level was also associated with financial distress and medical financial burden, although surprisingly both measures of financial difficulty were present in all FPL brackets. This highlights an important reality that financial vulnerability extends beyond income and federally defined “poverty.” Non-income factors, such as high local costs of living and the growing problem of underinsurance, may significantly contribute to financial difficulty, which may render static financial metrics such as percentage of FPL insufficient screeners. Furthermore, as evidenced by the nearly 10% of our respondents who declined to provide their income information, this is a sensitive topic for some families, so gathering income data during admission could likely be a nonstarter.

In the absence of other consistent predictors of financial difficulty that could trigger interventions such as an automatic financial counselor consult, hospitals and healthcare providers could consider implementing routine non-income based financial screening questions on admission, such as one assessing medical financial burden, as a nondiscriminatory way of identifying at-risk families and provide further education and assistance regarding their financial needs. Systematically gathering this data may also further demonstrate the need for broad financial navigation programs as a mainstay in comprehensive inpatient care.

We acknowledge several limitations of this study. Primarily, we surveyed families prior to discharge and receipt of hospitalization-related bills, and these bills could contribute significantly to financial difficulties. While the families of children with chronic disease, who likely have recurrent medical bills, did not demonstrate higher financial distress, it is possible that the overall rate of financial difficulties would have been higher had we surveyed families several weeks after discharge. Our measures of financial difficulty were also subjective and, therefore, at risk for response biases (such as recall bias) that could have misestimated the prevalence of these problems in our population. However, published literature on the IFDFW scale demonstrates concordance between the subjective score and tangible outcomes of financial distress (eg, contacting a credit agency). The IFDFW scale was validated in the general population, and although it has been used in studies of medical populations,^37-41 none have been in hospitalized populations, which may affect the scale’s applicability in our study. The study was also conducted only at university-affiliated children’s hospitals, and although these hospitals are geographically diverse, most children in the United States are admitted to general or community hospitals.³¹ Our population was also largely White, non-Hispanic/Latino, and English speaking. Therefore, our sample may not reflect the general population of hospitalized children and their families. We also assigned levels of chronic disease based on manual EHR review. While the EHR should capture each patient’s breadth of medical issues, inaccurate or missing documentation could have led to misclassification of complexity in some cases. Additionally, our sample size was calculated to detect fairly large differences in our primary outcome, and some of our unexpected results may have resulted from this study being underpowered for detection of smaller, but perhaps still clinically relevant, differences. Finally, we do not have data for several possible confounders in our study, such as employment status, health insurance concordance among family members, or sources of supplemental income, that may impact a family’s overall financial health, along with some potential important hospital-based screening characteristics, such as admitting service team or primary diagnosis.

Financial difficulties are common in families of hospitalized pediatric patients. Low-income families and those who have children with chronic conditions are at particular risk; however, all subsets of families can be affected. Given the potential negative health outcomes financial difficulties impose on families and children, the ability to identify and support vulnerable families is a crucial component of care. Hospitalization may be a prime opportunity to identify and support our at-risk families.

The authors would like to thank the parents at each of the study sites for their participation, as well as the multiple research coordinators across the study sites for assisting in recruitment of families, survey administration, and data collection. KT Park, MD, MS (Stanford University School of Medicine) served as an adviser for the study’s design.

All authors have no financial relationships or conflicts of interest relevant to this article to disclose.

Throughout history, physicians have formed communities to aid in the dissemination of knowledge, skills, and professional norms. From local physician groups to international societies and conferences, this drive to connect with members of our profession across the globe is timeless. We do so to learn from each other and continue to move the field of medicine forward. 

Yet, these communities are being strained by necessary physical distancing required during the COVID-19 pandemic. Many physicians accustomed to a sense of community are now finding themselves surprisingly isolated and alone. Into this distanced landscape, however, new digital groups—specifically social media (SoMe), online learning communities, and virtual conferences—have emerged. We are all active members in virtual communities; all of the authors are team members of The Clinical Problem Solvers podcast and one author of this paper, A.P., has previously served as the medical education lead for the Human Diagnosis Project. Both entities are described later in this article. Here, we provide an overview of these virtual communities and discuss how they have the potential to more equitably and effectively disseminate medical knowledge and education both during and after the COVID-19 pandemic (Table).

Even prior to the COVID-19 pandemic, SoMe—especially Twitter—had become a virtual gathering place where digital colleagues exchange Twitter handles like business cards.^1,2 They celebrate each other’s achievements and provide support during difficult times.

Importantly, the format of Twitter tends toward a flattened hierarchy. It is this egalitarian nature that has served SoMe well in its position as a modern learning community. Users from across the experience spectrum engage with and create novel educational content. This often occurs in the form of Tweetorials, or short lessons conveyed over a series of linked tweets. These have gained immense popularity on the platform and are becoming increasingly recognized forms of scholarship.³ Further, case-based lessons have become ubiquitous and are valuable opportunities for users to learn from other members of their digital communities. During the current pandemic, SoMe has become extremely important in the early dissemination and critique of the slew of research on the COVID-19 crisis.⁴

Beyond its role as an educational platform, SoMe functions as a virtual gathering place for members of the medical community to discuss topics relevant to the field. Subspecialists and researchers have gathered in digital journal clubs (eg, #NephJC, #IDJClub, #BloodandBone) and a number of journals have hosted live Twitter chats covering topics like controversies in clinical practice or professional development (eg, #JHMChat). More recently, social issues affecting the medical field, such as gender equity and the growing antiracism movement, have led to robust discussion on this medium.

Beyond Twitter, many medical professionals gather and exchange ideas on other platforms. Virtual networking and educational groups have arisen using Slack and Facebook.^5-7 Trainees and faculty members alike consume and produce content on YouTube, which often serve to teach technical skills.⁸ Given widespread use of SoMe, we anticipate that the range of platforms utilized by medical professionals will continue to expand in the future.

There have long existed multiple print and online forums dedicated to the development of clinical skills. These include clinical challenges in medical journals, interactive online cases, and more formal diagnostic education curricula at academic centers.^9-11 With the COVID-19 pandemic, it has become more difficult to ensure that trainees have an in-person learning community to discuss and receive feedback. This has led to a wider adoption of application-based clinical exercises, educational podcasts, and curricular innovations to support these virtual efforts.

The Human Diagnosis Project (Human Dx) is a smart-phone application that provides a platform for individuals to submit clinical cases that can be rapidly peer-reviewed and disseminated to the larger user pool. Human Dx is notable for fostering a strong sense of community amongst its users.^12,13 Case consumers and case creators are able to engage in further discussion after solving a case, and opportunities for feedback and growth are ample.

Medical education podcasts have taken on greater importance during the pandemic.^14,15 Many educators have begun referring their learners towards certain podcasts as in-person learning communities have been put on hold. Medical professionals may appreciate the up-to-date and candid conversations held on many podcasts, which can provide both educationally useful and emotionally sympathetic connections to their distanced peers. Similarly, while academic clinicians previously benefitted from invited grand rounds speakers, they may now find that such expert discussants are most easily accessible through their appearances on podcasts.

As institutions suspended clerkships during the pandemic, many created virtual communities for trainees to engage in diagnostic reasoning and education. They built novel curricula that meld asynchronous learning with online community-based learning.¹⁴ Gamified learning tools and quizzes have also been incorporated into these hybrid curricula to help ensure participation of learners within their virtual communities.^16,17 

Perhaps the most notable advance in digital communities catalyzed by the COVID-19 pandemic has been the increasing reliance on and comfort with video-based software. While many of our clinical, administrative, and social activities have migrated toward these virtual environments, they have also been used for a variety of activities related to education and professional development. 

As institutions struggled to adapt to physical distancing, many medical schools and residency programs have moved their regular meetings and conferences to virtual platforms. Similar free and open-access conferences have also emerged, including the “Virtual Morning Report” (VMR) series from The Clinical Problem Solvers podcast, wherein a few individuals are invited to discuss a case on the video conference, with the remainder of the audience contributing via the chat feature.

Beyond the growing popularity of video conferencing for education, these virtual sessions have become their own community. On The Clinical Problem Solvers VMR, many participants, ranging from preclinical students to seasoned attendings, show up on a daily basis and interact with each other as close friends, as do members of more insular institutional sessions (eg, residency run reports). In these strangely isolating times, many of us have experienced comfort in seeing the faces of our friends and colleagues joining us to listen and discuss cases. 

Separately, many professional societies have struggled with how to replace their large yearly in-person conferences, which would pose substantial infectious risks were they to be held in person. While many of those scheduled to occur during the early days of the pandemic were canceled or held limited online sessions, the trend towards virtual conference platforms seems to be accelerating. Organizers of the 2020 Conference on Retroviruses and Opportunistic Infections (March 8-11, 2020) decided to convert from an in-person to entirely virtual conference 48 hours before it started. With the benefit of more forewarning, other conferences are planning and exploring best practices to promote networking and advancement of research goals at future academic meetings.^18,19

The growing importance of these new digital communities could be viewed as a necessary evolution in the way that we gather and learn from each other. Traditional physician communities were inherently restricted by location, specialty, and hierarchy, thereby limiting the dissemination of knowledge and changes to professional norms. These restrictions could conceivably insulate and promote elite institutions in a fashion that perpetuates the inequalities within global medical systems. Unrestricted and open-access virtual communities, in contrast, have the potential to remove historical barriers and connect first-class mentors with trainees they would never have met otherwise.

Beyond promoting a more equitable distribution of knowledge and resources, these virtual communities are well suited to harness the benefits of group learning. The concept of communities of practice (CoP) refers to groupings of individuals involved in a personal or professional endeavor, with the community facilitating advancement of their own knowledge and skill set. Members of the CoP learn from each other, with more established members passing down essential knowledge and cultural norms. The three main components of CoP are maintaining a social network, a mutual enterprise (eg, a common goal), and a shared repertoire (eg, experiences, languages, etc).

Designing virtual learning spaces with these aspects in mind may allow these communities to function as CoPs. Some strategies include use of chat functions in videoconferences (to promote further dialogue) and development of dedicated sessions for specific subgroups or aims (eg, professional mentorship). The anticipated benefits of integrating virtual CoPs into medical education are notable, as a number of studies have already suggested that they are effective for disseminating knowledge, enhancing social learning, and aiding with professional development.^7,20-23 These virtual CoPs continue to evolve, however, and further research is warranted to clarify how best to utilize them in medical education and professional societies.

Amidst the tragic loss of lives and financial calamity, the COVID-19 pandemic has also spurred innovation and change in the way health professionals learn and communicate. Going forward, the medical establishment should capitalize on these recent innovations and work to further build, recognize, and foster such digital gathering spaces in order to more equitably and effectively disseminate knowledge and educational resources.

Despite physical distancing, health professionals have grown closer during these past few months. Innovations spurred by the pandemic have made us stronger and more united. Our experience with social media, online learning communities, and virtual conferences suggests the opportunity to grow and evolve from this experience. As Anthony Fauci, MD, director of the National Institute of Allergy and Infectious Diseases, said in March 2020, “...life is not going to be how it used to be [after the pandemic]…” Let’s hope he’s right.

We thank Reza Manesh, MD, Rabih Geha, MD, and Jack Penner, MD, for their careful review of the manuscript.

Throughout history, physicians have formed communities to aid in the dissemination of knowledge, skills, and professional norms. From local physician groups to international societies and conferences, this drive to connect with members of our profession across the globe is timeless. We do so to learn from each other and continue to move the field of medicine forward. 

Yet, these communities are being strained by necessary physical distancing required during the COVID-19 pandemic. Many physicians accustomed to a sense of community are now finding themselves surprisingly isolated and alone. Into this distanced landscape, however, new digital groups—specifically social media (SoMe), online learning communities, and virtual conferences—have emerged. We are all active members in virtual communities; all of the authors are team members of The Clinical Problem Solvers podcast and one author of this paper, A.P., has previously served as the medical education lead for the Human Diagnosis Project. Both entities are described later in this article. Here, we provide an overview of these virtual communities and discuss how they have the potential to more equitably and effectively disseminate medical knowledge and education both during and after the COVID-19 pandemic (Table).

Even prior to the COVID-19 pandemic, SoMe—especially Twitter—had become a virtual gathering place where digital colleagues exchange Twitter handles like business cards.^1,2 They celebrate each other’s achievements and provide support during difficult times.

Importantly, the format of Twitter tends toward a flattened hierarchy. It is this egalitarian nature that has served SoMe well in its position as a modern learning community. Users from across the experience spectrum engage with and create novel educational content. This often occurs in the form of Tweetorials, or short lessons conveyed over a series of linked tweets. These have gained immense popularity on the platform and are becoming increasingly recognized forms of scholarship.³ Further, case-based lessons have become ubiquitous and are valuable opportunities for users to learn from other members of their digital communities. During the current pandemic, SoMe has become extremely important in the early dissemination and critique of the slew of research on the COVID-19 crisis.⁴

Beyond its role as an educational platform, SoMe functions as a virtual gathering place for members of the medical community to discuss topics relevant to the field. Subspecialists and researchers have gathered in digital journal clubs (eg, #NephJC, #IDJClub, #BloodandBone) and a number of journals have hosted live Twitter chats covering topics like controversies in clinical practice or professional development (eg, #JHMChat). More recently, social issues affecting the medical field, such as gender equity and the growing antiracism movement, have led to robust discussion on this medium.

Beyond Twitter, many medical professionals gather and exchange ideas on other platforms. Virtual networking and educational groups have arisen using Slack and Facebook.^5-7 Trainees and faculty members alike consume and produce content on YouTube, which often serve to teach technical skills.⁸ Given widespread use of SoMe, we anticipate that the range of platforms utilized by medical professionals will continue to expand in the future.

There have long existed multiple print and online forums dedicated to the development of clinical skills. These include clinical challenges in medical journals, interactive online cases, and more formal diagnostic education curricula at academic centers.^9-11 With the COVID-19 pandemic, it has become more difficult to ensure that trainees have an in-person learning community to discuss and receive feedback. This has led to a wider adoption of application-based clinical exercises, educational podcasts, and curricular innovations to support these virtual efforts.

The Human Diagnosis Project (Human Dx) is a smart-phone application that provides a platform for individuals to submit clinical cases that can be rapidly peer-reviewed and disseminated to the larger user pool. Human Dx is notable for fostering a strong sense of community amongst its users.^12,13 Case consumers and case creators are able to engage in further discussion after solving a case, and opportunities for feedback and growth are ample.

Medical education podcasts have taken on greater importance during the pandemic.^14,15 Many educators have begun referring their learners towards certain podcasts as in-person learning communities have been put on hold. Medical professionals may appreciate the up-to-date and candid conversations held on many podcasts, which can provide both educationally useful and emotionally sympathetic connections to their distanced peers. Similarly, while academic clinicians previously benefitted from invited grand rounds speakers, they may now find that such expert discussants are most easily accessible through their appearances on podcasts.

As institutions suspended clerkships during the pandemic, many created virtual communities for trainees to engage in diagnostic reasoning and education. They built novel curricula that meld asynchronous learning with online community-based learning.¹⁴ Gamified learning tools and quizzes have also been incorporated into these hybrid curricula to help ensure participation of learners within their virtual communities.^16,17 

Perhaps the most notable advance in digital communities catalyzed by the COVID-19 pandemic has been the increasing reliance on and comfort with video-based software. While many of our clinical, administrative, and social activities have migrated toward these virtual environments, they have also been used for a variety of activities related to education and professional development. 

As institutions struggled to adapt to physical distancing, many medical schools and residency programs have moved their regular meetings and conferences to virtual platforms. Similar free and open-access conferences have also emerged, including the “Virtual Morning Report” (VMR) series from The Clinical Problem Solvers podcast, wherein a few individuals are invited to discuss a case on the video conference, with the remainder of the audience contributing via the chat feature.

Beyond the growing popularity of video conferencing for education, these virtual sessions have become their own community. On The Clinical Problem Solvers VMR, many participants, ranging from preclinical students to seasoned attendings, show up on a daily basis and interact with each other as close friends, as do members of more insular institutional sessions (eg, residency run reports). In these strangely isolating times, many of us have experienced comfort in seeing the faces of our friends and colleagues joining us to listen and discuss cases. 

Separately, many professional societies have struggled with how to replace their large yearly in-person conferences, which would pose substantial infectious risks were they to be held in person. While many of those scheduled to occur during the early days of the pandemic were canceled or held limited online sessions, the trend towards virtual conference platforms seems to be accelerating. Organizers of the 2020 Conference on Retroviruses and Opportunistic Infections (March 8-11, 2020) decided to convert from an in-person to entirely virtual conference 48 hours before it started. With the benefit of more forewarning, other conferences are planning and exploring best practices to promote networking and advancement of research goals at future academic meetings.^18,19

The growing importance of these new digital communities could be viewed as a necessary evolution in the way that we gather and learn from each other. Traditional physician communities were inherently restricted by location, specialty, and hierarchy, thereby limiting the dissemination of knowledge and changes to professional norms. These restrictions could conceivably insulate and promote elite institutions in a fashion that perpetuates the inequalities within global medical systems. Unrestricted and open-access virtual communities, in contrast, have the potential to remove historical barriers and connect first-class mentors with trainees they would never have met otherwise.

Beyond promoting a more equitable distribution of knowledge and resources, these virtual communities are well suited to harness the benefits of group learning. The concept of communities of practice (CoP) refers to groupings of individuals involved in a personal or professional endeavor, with the community facilitating advancement of their own knowledge and skill set. Members of the CoP learn from each other, with more established members passing down essential knowledge and cultural norms. The three main components of CoP are maintaining a social network, a mutual enterprise (eg, a common goal), and a shared repertoire (eg, experiences, languages, etc).

Designing virtual learning spaces with these aspects in mind may allow these communities to function as CoPs. Some strategies include use of chat functions in videoconferences (to promote further dialogue) and development of dedicated sessions for specific subgroups or aims (eg, professional mentorship). The anticipated benefits of integrating virtual CoPs into medical education are notable, as a number of studies have already suggested that they are effective for disseminating knowledge, enhancing social learning, and aiding with professional development.^7,20-23 These virtual CoPs continue to evolve, however, and further research is warranted to clarify how best to utilize them in medical education and professional societies.

Amidst the tragic loss of lives and financial calamity, the COVID-19 pandemic has also spurred innovation and change in the way health professionals learn and communicate. Going forward, the medical establishment should capitalize on these recent innovations and work to further build, recognize, and foster such digital gathering spaces in order to more equitably and effectively disseminate knowledge and educational resources.

Despite physical distancing, health professionals have grown closer during these past few months. Innovations spurred by the pandemic have made us stronger and more united. Our experience with social media, online learning communities, and virtual conferences suggests the opportunity to grow and evolve from this experience. As Anthony Fauci, MD, director of the National Institute of Allergy and Infectious Diseases, said in March 2020, “...life is not going to be how it used to be [after the pandemic]…” Let’s hope he’s right.

We thank Reza Manesh, MD, Rabih Geha, MD, and Jack Penner, MD, for their careful review of the manuscript.

Throughout history, physicians have formed communities to aid in the dissemination of knowledge, skills, and professional norms. From local physician groups to international societies and conferences, this drive to connect with members of our profession across the globe is timeless. We do so to learn from each other and continue to move the field of medicine forward. 

Yet, these communities are being strained by necessary physical distancing required during the COVID-19 pandemic. Many physicians accustomed to a sense of community are now finding themselves surprisingly isolated and alone. Into this distanced landscape, however, new digital groups—specifically social media (SoMe), online learning communities, and virtual conferences—have emerged. We are all active members in virtual communities; all of the authors are team members of The Clinical Problem Solvers podcast and one author of this paper, A.P., has previously served as the medical education lead for the Human Diagnosis Project. Both entities are described later in this article. Here, we provide an overview of these virtual communities and discuss how they have the potential to more equitably and effectively disseminate medical knowledge and education both during and after the COVID-19 pandemic (Table).

Even prior to the COVID-19 pandemic, SoMe—especially Twitter—had become a virtual gathering place where digital colleagues exchange Twitter handles like business cards.^1,2 They celebrate each other’s achievements and provide support during difficult times.

Importantly, the format of Twitter tends toward a flattened hierarchy. It is this egalitarian nature that has served SoMe well in its position as a modern learning community. Users from across the experience spectrum engage with and create novel educational content. This often occurs in the form of Tweetorials, or short lessons conveyed over a series of linked tweets. These have gained immense popularity on the platform and are becoming increasingly recognized forms of scholarship.³ Further, case-based lessons have become ubiquitous and are valuable opportunities for users to learn from other members of their digital communities. During the current pandemic, SoMe has become extremely important in the early dissemination and critique of the slew of research on the COVID-19 crisis.⁴

Beyond its role as an educational platform, SoMe functions as a virtual gathering place for members of the medical community to discuss topics relevant to the field. Subspecialists and researchers have gathered in digital journal clubs (eg, #NephJC, #IDJClub, #BloodandBone) and a number of journals have hosted live Twitter chats covering topics like controversies in clinical practice or professional development (eg, #JHMChat). More recently, social issues affecting the medical field, such as gender equity and the growing antiracism movement, have led to robust discussion on this medium.

Beyond Twitter, many medical professionals gather and exchange ideas on other platforms. Virtual networking and educational groups have arisen using Slack and Facebook.^5-7 Trainees and faculty members alike consume and produce content on YouTube, which often serve to teach technical skills.⁸ Given widespread use of SoMe, we anticipate that the range of platforms utilized by medical professionals will continue to expand in the future.

There have long existed multiple print and online forums dedicated to the development of clinical skills. These include clinical challenges in medical journals, interactive online cases, and more formal diagnostic education curricula at academic centers.^9-11 With the COVID-19 pandemic, it has become more difficult to ensure that trainees have an in-person learning community to discuss and receive feedback. This has led to a wider adoption of application-based clinical exercises, educational podcasts, and curricular innovations to support these virtual efforts.

The Human Diagnosis Project (Human Dx) is a smart-phone application that provides a platform for individuals to submit clinical cases that can be rapidly peer-reviewed and disseminated to the larger user pool. Human Dx is notable for fostering a strong sense of community amongst its users.^12,13 Case consumers and case creators are able to engage in further discussion after solving a case, and opportunities for feedback and growth are ample.

Medical education podcasts have taken on greater importance during the pandemic.^14,15 Many educators have begun referring their learners towards certain podcasts as in-person learning communities have been put on hold. Medical professionals may appreciate the up-to-date and candid conversations held on many podcasts, which can provide both educationally useful and emotionally sympathetic connections to their distanced peers. Similarly, while academic clinicians previously benefitted from invited grand rounds speakers, they may now find that such expert discussants are most easily accessible through their appearances on podcasts.

As institutions suspended clerkships during the pandemic, many created virtual communities for trainees to engage in diagnostic reasoning and education. They built novel curricula that meld asynchronous learning with online community-based learning.¹⁴ Gamified learning tools and quizzes have also been incorporated into these hybrid curricula to help ensure participation of learners within their virtual communities.^16,17 

Perhaps the most notable advance in digital communities catalyzed by the COVID-19 pandemic has been the increasing reliance on and comfort with video-based software. While many of our clinical, administrative, and social activities have migrated toward these virtual environments, they have also been used for a variety of activities related to education and professional development. 

As institutions struggled to adapt to physical distancing, many medical schools and residency programs have moved their regular meetings and conferences to virtual platforms. Similar free and open-access conferences have also emerged, including the “Virtual Morning Report” (VMR) series from The Clinical Problem Solvers podcast, wherein a few individuals are invited to discuss a case on the video conference, with the remainder of the audience contributing via the chat feature.

Beyond the growing popularity of video conferencing for education, these virtual sessions have become their own community. On The Clinical Problem Solvers VMR, many participants, ranging from preclinical students to seasoned attendings, show up on a daily basis and interact with each other as close friends, as do members of more insular institutional sessions (eg, residency run reports). In these strangely isolating times, many of us have experienced comfort in seeing the faces of our friends and colleagues joining us to listen and discuss cases. 

Separately, many professional societies have struggled with how to replace their large yearly in-person conferences, which would pose substantial infectious risks were they to be held in person. While many of those scheduled to occur during the early days of the pandemic were canceled or held limited online sessions, the trend towards virtual conference platforms seems to be accelerating. Organizers of the 2020 Conference on Retroviruses and Opportunistic Infections (March 8-11, 2020) decided to convert from an in-person to entirely virtual conference 48 hours before it started. With the benefit of more forewarning, other conferences are planning and exploring best practices to promote networking and advancement of research goals at future academic meetings.^18,19

The growing importance of these new digital communities could be viewed as a necessary evolution in the way that we gather and learn from each other. Traditional physician communities were inherently restricted by location, specialty, and hierarchy, thereby limiting the dissemination of knowledge and changes to professional norms. These restrictions could conceivably insulate and promote elite institutions in a fashion that perpetuates the inequalities within global medical systems. Unrestricted and open-access virtual communities, in contrast, have the potential to remove historical barriers and connect first-class mentors with trainees they would never have met otherwise.

Beyond promoting a more equitable distribution of knowledge and resources, these virtual communities are well suited to harness the benefits of group learning. The concept of communities of practice (CoP) refers to groupings of individuals involved in a personal or professional endeavor, with the community facilitating advancement of their own knowledge and skill set. Members of the CoP learn from each other, with more established members passing down essential knowledge and cultural norms. The three main components of CoP are maintaining a social network, a mutual enterprise (eg, a common goal), and a shared repertoire (eg, experiences, languages, etc).

Designing virtual learning spaces with these aspects in mind may allow these communities to function as CoPs. Some strategies include use of chat functions in videoconferences (to promote further dialogue) and development of dedicated sessions for specific subgroups or aims (eg, professional mentorship). The anticipated benefits of integrating virtual CoPs into medical education are notable, as a number of studies have already suggested that they are effective for disseminating knowledge, enhancing social learning, and aiding with professional development.^7,20-23 These virtual CoPs continue to evolve, however, and further research is warranted to clarify how best to utilize them in medical education and professional societies.

Amidst the tragic loss of lives and financial calamity, the COVID-19 pandemic has also spurred innovation and change in the way health professionals learn and communicate. Going forward, the medical establishment should capitalize on these recent innovations and work to further build, recognize, and foster such digital gathering spaces in order to more equitably and effectively disseminate knowledge and educational resources.

Despite physical distancing, health professionals have grown closer during these past few months. Innovations spurred by the pandemic have made us stronger and more united. Our experience with social media, online learning communities, and virtual conferences suggests the opportunity to grow and evolve from this experience. As Anthony Fauci, MD, director of the National Institute of Allergy and Infectious Diseases, said in March 2020, “...life is not going to be how it used to be [after the pandemic]…” Let’s hope he’s right.

We thank Reza Manesh, MD, Rabih Geha, MD, and Jack Penner, MD, for their careful review of the manuscript.

The author would like to make the following correction to the Editorial, originally published in the July issue of the Journal of Hospital Medicine 2020;15(7):447-448. DOI 10.12788/jhm.3468. In the third paragraph, MINS was described as an “umbrella term that can indicate either a myocardial infarction (MI) or nonischemic myocardial injury (NIMI).” This is not fully accurate: MINS is an umbrella term that can indicate either an MI or other myocardial injury due to ischemia. The correction to the paragraph is as follows, indicated in bold type:

In this journal issue, Cohn and colleagues summarize the current information around this phenomenon of myocardial injury after noncardiac surgery, or MINS.¹ Consistent with the literature, they define MINS as an acute rise and/or fall in troponin (above the assay’s upper limit of normal) at any point in the 30 days following noncardiac surgery. Importantly, MINS is an umbrella term that can indicate either an MI or other myocardial injury due to ischemia. An MI exists if there are clinical signs of ischemia and/or objective evidence of infarction on imaging.

The author would like to make the following correction to the Editorial, originally published in the July issue of the Journal of Hospital Medicine 2020;15(7):447-448. DOI 10.12788/jhm.3468. In the third paragraph, MINS was described as an “umbrella term that can indicate either a myocardial infarction (MI) or nonischemic myocardial injury (NIMI).” This is not fully accurate: MINS is an umbrella term that can indicate either an MI or other myocardial injury due to ischemia. The correction to the paragraph is as follows, indicated in bold type:

In this journal issue, Cohn and colleagues summarize the current information around this phenomenon of myocardial injury after noncardiac surgery, or MINS.¹ Consistent with the literature, they define MINS as an acute rise and/or fall in troponin (above the assay’s upper limit of normal) at any point in the 30 days following noncardiac surgery. Importantly, MINS is an umbrella term that can indicate either an MI or other myocardial injury due to ischemia. An MI exists if there are clinical signs of ischemia and/or objective evidence of infarction on imaging.

The author would like to make the following correction to the Editorial, originally published in the July issue of the Journal of Hospital Medicine 2020;15(7):447-448. DOI 10.12788/jhm.3468. In the third paragraph, MINS was described as an “umbrella term that can indicate either a myocardial infarction (MI) or nonischemic myocardial injury (NIMI).” This is not fully accurate: MINS is an umbrella term that can indicate either an MI or other myocardial injury due to ischemia. The correction to the paragraph is as follows, indicated in bold type:

In this journal issue, Cohn and colleagues summarize the current information around this phenomenon of myocardial injury after noncardiac surgery, or MINS.¹ Consistent with the literature, they define MINS as an acute rise and/or fall in troponin (above the assay’s upper limit of normal) at any point in the 30 days following noncardiac surgery. Importantly, MINS is an umbrella term that can indicate either an MI or other myocardial injury due to ischemia. An MI exists if there are clinical signs of ischemia and/or objective evidence of infarction on imaging.

When asked by friend or family “Which hospital did you go to?” or “Which doctor did you see?” most are likely to answer with a single institution or clinician. Yet for hospital stays the patient’s experience and outcomes are a product of many individuals and an entire system of care, so measuring performance at the group, or “team,” level is appropriate.

Assessing and managing performance of individuals in healthcare is also important. In this regard, though, healthcare may be more like assessing individual baseball players prior to the widespread adoption of detailed statistics, a transition to what is often referred to as sabermetrics (and popularized by the 2004 book Moneyball).¹ An individual player’s performance and future potential went from being assessed largely by the opinion of expert talent scouts to including, or even principally relying on, a wide array of measurements and statistics.

It sometimes seems healthcare has arrived at its “sabermetrics moment.” There is a rapidly growing set of measures for individual clinicians, and nearly every week, hospitalists will open a new report of their performance sent by a payer, a government agency, their own hospitals, or other organizations. But most of these metrics suffer from problems with attributing performance to a single clinician; for example, many or most metrics attribute performance to the attending at the time of a patient’s discharge according to the clinical record. Yet while clinical metrics (eg, administer beta-blocker when indicated, length of stay (LOS), readmissions), patient experience, financial metrics (eg, cost per case), and others are vital to understanding performance at an aggregate level such as a hospital or physician group, they are potentially confusing or even misleading when attributed entirely to the discharging provider. So healthcare leaders still tend to rely meaningfully on expert opinion—“talent scouts”—to identify high performers.

In this issue of the Journal of Hospital Medicine, Dow and colleagues have advanced our understanding of the current state of individual- rather than group-level hospitalist performance measurement.² This scoping review identified 43 studies published over the last 25 years reporting individual adult or pediatric hospitalist performance across one or more of the STEEEP framework domains of performance: Safe, Timely, Effective, Efficient, Equitable, Patient Centered.³

The most common domain assessed in the studies was Patient Centered (20 studies), and in descending order from there were Safe (16), Efficient (13), Timely (10), Effective (9). No studies reported individual hospitalist performance on Equitable care. This distribution of studied domains is likely a function of readily available data and processes for study more than level of interest or importance attached to each domain. Their research was not designed to assess the quality of each study, and some—or even many—might have weaknesses in both determining which clinicians met the definition of hospitalist and how performance was attributed to individuals. The authors appropriately conclude that “further defining and refining approaches to assess individual performance is necessary to ensure the highest quality.”

Their findings should help guide research priorities regarding measurement of individual hospitalist performance. Yet each hospitalist group and individual hospitalist still faces decisions about managing their own group and personal performance and must navigate without the benefit of research providing clear direction. Many hospitalist metrics are tracked and reported to meet regulatory requirements such as those from Centers for Medicare & Medicaid Services, financial metrics for the local hospital and hospitalist group, and for use as components of hospitalist compensation. (The biennial State of Hospital Medicine Report captures extensive data regarding the latter.⁴)

Many people and processes across an entire healthcare system influence performance on every metric, but it is useful and practical to attribute some metrics entirely to a single hospitalist provider, such as timely documentation and the time of day the discharge order is entered. And arguably, it is useful to attribute readmission rate entirely to the discharging provider—the last hospital provider who can influence readmission risk. But for most other metrics individual attribution is problematic or misleading and collective experience and expert opinion are helpful here. Two examples come to mind of relatively simple approaches that have gained some popularity in teasing out individual contribution to hospitalist performance.

One can estimate individual hospitalist contribution to patient LOS by calculating the ratio of current procedural terminology (CPT) codes for all follow-up services to all discharge codes. For each hospitalist in the group who cares for a similar population, those with the highest ratios likely manage patients in ways associated with longer LOS. It is relatively simple to use billing data to calculate the ratio, and some groups report it for all providers monthly.

Many metrics that aggregate performance across an entire hospital stay, such as patient experience surveys, can be apportioned to each hospitalist who had a billed encounter with the patient. For example, if a hospitalist has 4 of a patient’s 10 billed encounters within the same group, then 40% of the patient’s survey score could be attributed to that hospitalist. It’s still imperfect, but it’s likely more meaningful than attributing the entire survey result to only the discharging provider.

These approaches have value but still leave us unsatisfied and unable to assess performance as effectively as we would like. Advancements in measurement have been slow and incremental, but they are likely to accelerate with maturation of electronic health records paired with machine learning or artificial intelligence, wearable devices, and sensors in patient rooms, which collectively may make capturing a robust set of metrics trivially easy (and raise questions regarding privacy and so forth). For example, it is already possible to capture via a smart speaker all conversations between patient, loved ones, and clinician.⁵ Imagine you are presented with a word cloud summary of all conversations you had with all patients over a year. Did you use empathy words often enough? How reliably did you address all appropriate discharge-related topics?

As performance metrics become more numerous and ubiquitous, the challenge will be to ensure they accurately capture what they appear to measure, are appropriately attributed to individuals or groups, and provide insights into important domains of performance. Significant opportunity for improvement remains.

Dr Nelson has no conflict of interest to disclose.

When asked by friend or family “Which hospital did you go to?” or “Which doctor did you see?” most are likely to answer with a single institution or clinician. Yet for hospital stays the patient’s experience and outcomes are a product of many individuals and an entire system of care, so measuring performance at the group, or “team,” level is appropriate.

Assessing and managing performance of individuals in healthcare is also important. In this regard, though, healthcare may be more like assessing individual baseball players prior to the widespread adoption of detailed statistics, a transition to what is often referred to as sabermetrics (and popularized by the 2004 book Moneyball).¹ An individual player’s performance and future potential went from being assessed largely by the opinion of expert talent scouts to including, or even principally relying on, a wide array of measurements and statistics.

It sometimes seems healthcare has arrived at its “sabermetrics moment.” There is a rapidly growing set of measures for individual clinicians, and nearly every week, hospitalists will open a new report of their performance sent by a payer, a government agency, their own hospitals, or other organizations. But most of these metrics suffer from problems with attributing performance to a single clinician; for example, many or most metrics attribute performance to the attending at the time of a patient’s discharge according to the clinical record. Yet while clinical metrics (eg, administer beta-blocker when indicated, length of stay (LOS), readmissions), patient experience, financial metrics (eg, cost per case), and others are vital to understanding performance at an aggregate level such as a hospital or physician group, they are potentially confusing or even misleading when attributed entirely to the discharging provider. So healthcare leaders still tend to rely meaningfully on expert opinion—“talent scouts”—to identify high performers.

In this issue of the Journal of Hospital Medicine, Dow and colleagues have advanced our understanding of the current state of individual- rather than group-level hospitalist performance measurement.² This scoping review identified 43 studies published over the last 25 years reporting individual adult or pediatric hospitalist performance across one or more of the STEEEP framework domains of performance: Safe, Timely, Effective, Efficient, Equitable, Patient Centered.³

The most common domain assessed in the studies was Patient Centered (20 studies), and in descending order from there were Safe (16), Efficient (13), Timely (10), Effective (9). No studies reported individual hospitalist performance on Equitable care. This distribution of studied domains is likely a function of readily available data and processes for study more than level of interest or importance attached to each domain. Their research was not designed to assess the quality of each study, and some—or even many—might have weaknesses in both determining which clinicians met the definition of hospitalist and how performance was attributed to individuals. The authors appropriately conclude that “further defining and refining approaches to assess individual performance is necessary to ensure the highest quality.”

Their findings should help guide research priorities regarding measurement of individual hospitalist performance. Yet each hospitalist group and individual hospitalist still faces decisions about managing their own group and personal performance and must navigate without the benefit of research providing clear direction. Many hospitalist metrics are tracked and reported to meet regulatory requirements such as those from Centers for Medicare & Medicaid Services, financial metrics for the local hospital and hospitalist group, and for use as components of hospitalist compensation. (The biennial State of Hospital Medicine Report captures extensive data regarding the latter.⁴)

Many people and processes across an entire healthcare system influence performance on every metric, but it is useful and practical to attribute some metrics entirely to a single hospitalist provider, such as timely documentation and the time of day the discharge order is entered. And arguably, it is useful to attribute readmission rate entirely to the discharging provider—the last hospital provider who can influence readmission risk. But for most other metrics individual attribution is problematic or misleading and collective experience and expert opinion are helpful here. Two examples come to mind of relatively simple approaches that have gained some popularity in teasing out individual contribution to hospitalist performance.

One can estimate individual hospitalist contribution to patient LOS by calculating the ratio of current procedural terminology (CPT) codes for all follow-up services to all discharge codes. For each hospitalist in the group who cares for a similar population, those with the highest ratios likely manage patients in ways associated with longer LOS. It is relatively simple to use billing data to calculate the ratio, and some groups report it for all providers monthly.

Many metrics that aggregate performance across an entire hospital stay, such as patient experience surveys, can be apportioned to each hospitalist who had a billed encounter with the patient. For example, if a hospitalist has 4 of a patient’s 10 billed encounters within the same group, then 40% of the patient’s survey score could be attributed to that hospitalist. It’s still imperfect, but it’s likely more meaningful than attributing the entire survey result to only the discharging provider.

These approaches have value but still leave us unsatisfied and unable to assess performance as effectively as we would like. Advancements in measurement have been slow and incremental, but they are likely to accelerate with maturation of electronic health records paired with machine learning or artificial intelligence, wearable devices, and sensors in patient rooms, which collectively may make capturing a robust set of metrics trivially easy (and raise questions regarding privacy and so forth). For example, it is already possible to capture via a smart speaker all conversations between patient, loved ones, and clinician.⁵ Imagine you are presented with a word cloud summary of all conversations you had with all patients over a year. Did you use empathy words often enough? How reliably did you address all appropriate discharge-related topics?

As performance metrics become more numerous and ubiquitous, the challenge will be to ensure they accurately capture what they appear to measure, are appropriately attributed to individuals or groups, and provide insights into important domains of performance. Significant opportunity for improvement remains.

Dr Nelson has no conflict of interest to disclose.

When asked by friend or family “Which hospital did you go to?” or “Which doctor did you see?” most are likely to answer with a single institution or clinician. Yet for hospital stays the patient’s experience and outcomes are a product of many individuals and an entire system of care, so measuring performance at the group, or “team,” level is appropriate.

Assessing and managing performance of individuals in healthcare is also important. In this regard, though, healthcare may be more like assessing individual baseball players prior to the widespread adoption of detailed statistics, a transition to what is often referred to as sabermetrics (and popularized by the 2004 book Moneyball).¹ An individual player’s performance and future potential went from being assessed largely by the opinion of expert talent scouts to including, or even principally relying on, a wide array of measurements and statistics.

It sometimes seems healthcare has arrived at its “sabermetrics moment.” There is a rapidly growing set of measures for individual clinicians, and nearly every week, hospitalists will open a new report of their performance sent by a payer, a government agency, their own hospitals, or other organizations. But most of these metrics suffer from problems with attributing performance to a single clinician; for example, many or most metrics attribute performance to the attending at the time of a patient’s discharge according to the clinical record. Yet while clinical metrics (eg, administer beta-blocker when indicated, length of stay (LOS), readmissions), patient experience, financial metrics (eg, cost per case), and others are vital to understanding performance at an aggregate level such as a hospital or physician group, they are potentially confusing or even misleading when attributed entirely to the discharging provider. So healthcare leaders still tend to rely meaningfully on expert opinion—“talent scouts”—to identify high performers.

In this issue of the Journal of Hospital Medicine, Dow and colleagues have advanced our understanding of the current state of individual- rather than group-level hospitalist performance measurement.² This scoping review identified 43 studies published over the last 25 years reporting individual adult or pediatric hospitalist performance across one or more of the STEEEP framework domains of performance: Safe, Timely, Effective, Efficient, Equitable, Patient Centered.³

The most common domain assessed in the studies was Patient Centered (20 studies), and in descending order from there were Safe (16), Efficient (13), Timely (10), Effective (9). No studies reported individual hospitalist performance on Equitable care. This distribution of studied domains is likely a function of readily available data and processes for study more than level of interest or importance attached to each domain. Their research was not designed to assess the quality of each study, and some—or even many—might have weaknesses in both determining which clinicians met the definition of hospitalist and how performance was attributed to individuals. The authors appropriately conclude that “further defining and refining approaches to assess individual performance is necessary to ensure the highest quality.”

Their findings should help guide research priorities regarding measurement of individual hospitalist performance. Yet each hospitalist group and individual hospitalist still faces decisions about managing their own group and personal performance and must navigate without the benefit of research providing clear direction. Many hospitalist metrics are tracked and reported to meet regulatory requirements such as those from Centers for Medicare & Medicaid Services, financial metrics for the local hospital and hospitalist group, and for use as components of hospitalist compensation. (The biennial State of Hospital Medicine Report captures extensive data regarding the latter.⁴)

Many people and processes across an entire healthcare system influence performance on every metric, but it is useful and practical to attribute some metrics entirely to a single hospitalist provider, such as timely documentation and the time of day the discharge order is entered. And arguably, it is useful to attribute readmission rate entirely to the discharging provider—the last hospital provider who can influence readmission risk. But for most other metrics individual attribution is problematic or misleading and collective experience and expert opinion are helpful here. Two examples come to mind of relatively simple approaches that have gained some popularity in teasing out individual contribution to hospitalist performance.

One can estimate individual hospitalist contribution to patient LOS by calculating the ratio of current procedural terminology (CPT) codes for all follow-up services to all discharge codes. For each hospitalist in the group who cares for a similar population, those with the highest ratios likely manage patients in ways associated with longer LOS. It is relatively simple to use billing data to calculate the ratio, and some groups report it for all providers monthly.

Many metrics that aggregate performance across an entire hospital stay, such as patient experience surveys, can be apportioned to each hospitalist who had a billed encounter with the patient. For example, if a hospitalist has 4 of a patient’s 10 billed encounters within the same group, then 40% of the patient’s survey score could be attributed to that hospitalist. It’s still imperfect, but it’s likely more meaningful than attributing the entire survey result to only the discharging provider.

These approaches have value but still leave us unsatisfied and unable to assess performance as effectively as we would like. Advancements in measurement have been slow and incremental, but they are likely to accelerate with maturation of electronic health records paired with machine learning or artificial intelligence, wearable devices, and sensors in patient rooms, which collectively may make capturing a robust set of metrics trivially easy (and raise questions regarding privacy and so forth). For example, it is already possible to capture via a smart speaker all conversations between patient, loved ones, and clinician.⁵ Imagine you are presented with a word cloud summary of all conversations you had with all patients over a year. Did you use empathy words often enough? How reliably did you address all appropriate discharge-related topics?

As performance metrics become more numerous and ubiquitous, the challenge will be to ensure they accurately capture what they appear to measure, are appropriately attributed to individuals or groups, and provide insights into important domains of performance. Significant opportunity for improvement remains.

Dr Nelson has no conflict of interest to disclose.

Still in its infancy, the Hospital Compare overall hospital quality star rating program introduced by the Centers for Medicare & Medicaid Services (CMS) has generated intense industry debate. Individual health systems are microcosms of the challenges of ratings and measurement design. Sibley Memorial Hospital, a member of Johns Hopkins Medicine, is a well-run, 288-bed, community hospital located in a wealthy section of northwest District of Columbia with a five-star rating. In contrast, its academic partner, the Johns Hopkins Hospital, a 1,162-bed hospital with a century-long history of innovation situated in an impoverished Baltimore, Maryland, neighborhood, received a three-star rating.

Hospital ratings are the product of an industry in transition: As care delivery has shifted from an individual provider-driven industry to an increasingly scaled systems enterprise, policymakers implemented regulatory standards targeting quality measurement. Subsequent to the National Academy of Medicine’s 1999 report To Err is Human, policy efforts brought public reporting of quality ratings to multiple market segments, including dialysis facilities (2001), nursing homes (2003), Medicare Advantage plans (2007), and physicians (2015). The hospital industry was no exception, and in 2016—with much controversy¹—CMS launched the hospital star ratings program.

CMS Star Ratings for hospitals are based on seven measure groups: mortality, safety, readmission, patient experience, effectiveness, timeliness, and efficient use of medical imaging. Both industry and researchers have decried the challenges of star ratings, noting that hospitals with a narrower scope of services are more likely to receive higher ratings.² Measure groupings may be further flawed as shown by recent work demonstrating that larger, safety net, or academic hospitals, as well as hospitals offering transplant services, have higher readmission rates,³ which may be caused by differences in patient complexity. Other research has demonstrated that overall quality ratings inappropriately pool all hospitals together, when it may be fairer to initially categorize hospitals and then score them.⁴

It is within this maelstrom of debate that, in this month’s issue of the Journal of Hospital Medicine, Shi and colleagues explore the relationship between hospital star ratings and the socioeconomic features of the surrounding communities.⁵ Conducting their analysis by linking multiple reputable government and industry sources, Shi and colleagues found that counties with higher education attainment and a lower proportion of dual Medicare-Medicaid–eligible populations had higher hospital star ratings. Furthermore, a county’s minority population percentage negatively correlated with hospital ratings. Validating the experience of many rural hospital executives—who frequently experience financial challenges—Shi and colleagues noted that rural hospitals were less likely to receive five-star ratings.

Do these findings reflect a true disparity and lack of access to high-quality hospitals, or are they artifactual—secondary to a flawed construct of hospital quality measurement? Many lower-ranking hospitals are urban academic centers frequently providing services not offered at their five-star community counterparts, such as neurosurgery, comprehensive cancer care, and organ transplants, while simultaneously serving as safety net hospitals, research institutions, trauma centers, and national referral centers.

Sociodemographics factor significantly in self-care management for hospital aftercare. Health literacy, access to primary and behavioral healthcare, and transportation all affect star indicators. Recent work⁶ demonstrated that comprehensive investments in transitional care strategies and the social determinants of health were ineffective at reducing readmissions, which suggests that high readmission rates for hospitals in impoverished areas are not only common, but also may not accurately reflect hospital quality and local investment.

Patient experience is also complicating, with research demonstrating that patient perceptions vary significantly by education, age, primary language, ethnicity, and overall health. For example, one-third of average-ranked hospitals would have rankings vary by at least 18 percentile points when evaluated by Spanish-speaking patients. Star ratings fail to capture and communicate this granularity.⁷

More concerning is that star ratings inherently assume that hospital performance is being compared across the same tasks, regardless of patient characteristics, local resources, or the scope of services provided, the latter of which may vary between hospitals. For example, communication may differ in both complexity and time intensity: Explaining an antibiotic to the uncomplicated patient with pneumonia differs from prescribing an antibiotic to a patient who is legally blind from optic neuritis, walks with a cane because of multiple sclerosis, and has 24 other prescription medications. Similar challenges exist for differences in local neighborhood resources and for facilities with differing service scope.

Although one strategy to handle these “disparities” in star ratings might be to risk-adjust for social determinants of health, patients may be better served by first rethinking how star ratings are constructed. Clustering hospitals by scope of services provided and geographic region prior to determining star ratings would provide consumers with meaningful information by helping patients compare and make choices among either local or regional hospitals; national quality rankings are unhelpful for patients.

Arguably one of the most complex and person-dependent service enterprises, care delivery presents unique challenges for evaluation of customer experience and medical quality. Hospital star ratings are no exception: We must rethink their construction so they can be more meaningful for both patients and physicians.

The authors would like to acknowledge Daniel J Brotman, MD, for his editorial advice and input.

Dr Miller reported consulting for the Federal Trade Commission and serving as a member of the Centers for Medicare & Medicaid Services Medicare Evidence Development Coverage Advisory Committee. Drs Siddiqui and Deutschendorf have nothing to disclose.

Still in its infancy, the Hospital Compare overall hospital quality star rating program introduced by the Centers for Medicare & Medicaid Services (CMS) has generated intense industry debate. Individual health systems are microcosms of the challenges of ratings and measurement design. Sibley Memorial Hospital, a member of Johns Hopkins Medicine, is a well-run, 288-bed, community hospital located in a wealthy section of northwest District of Columbia with a five-star rating. In contrast, its academic partner, the Johns Hopkins Hospital, a 1,162-bed hospital with a century-long history of innovation situated in an impoverished Baltimore, Maryland, neighborhood, received a three-star rating.

Hospital ratings are the product of an industry in transition: As care delivery has shifted from an individual provider-driven industry to an increasingly scaled systems enterprise, policymakers implemented regulatory standards targeting quality measurement. Subsequent to the National Academy of Medicine’s 1999 report To Err is Human, policy efforts brought public reporting of quality ratings to multiple market segments, including dialysis facilities (2001), nursing homes (2003), Medicare Advantage plans (2007), and physicians (2015). The hospital industry was no exception, and in 2016—with much controversy¹—CMS launched the hospital star ratings program.

CMS Star Ratings for hospitals are based on seven measure groups: mortality, safety, readmission, patient experience, effectiveness, timeliness, and efficient use of medical imaging. Both industry and researchers have decried the challenges of star ratings, noting that hospitals with a narrower scope of services are more likely to receive higher ratings.² Measure groupings may be further flawed as shown by recent work demonstrating that larger, safety net, or academic hospitals, as well as hospitals offering transplant services, have higher readmission rates,³ which may be caused by differences in patient complexity. Other research has demonstrated that overall quality ratings inappropriately pool all hospitals together, when it may be fairer to initially categorize hospitals and then score them.⁴

It is within this maelstrom of debate that, in this month’s issue of the Journal of Hospital Medicine, Shi and colleagues explore the relationship between hospital star ratings and the socioeconomic features of the surrounding communities.⁵ Conducting their analysis by linking multiple reputable government and industry sources, Shi and colleagues found that counties with higher education attainment and a lower proportion of dual Medicare-Medicaid–eligible populations had higher hospital star ratings. Furthermore, a county’s minority population percentage negatively correlated with hospital ratings. Validating the experience of many rural hospital executives—who frequently experience financial challenges—Shi and colleagues noted that rural hospitals were less likely to receive five-star ratings.

Do these findings reflect a true disparity and lack of access to high-quality hospitals, or are they artifactual—secondary to a flawed construct of hospital quality measurement? Many lower-ranking hospitals are urban academic centers frequently providing services not offered at their five-star community counterparts, such as neurosurgery, comprehensive cancer care, and organ transplants, while simultaneously serving as safety net hospitals, research institutions, trauma centers, and national referral centers.

Sociodemographics factor significantly in self-care management for hospital aftercare. Health literacy, access to primary and behavioral healthcare, and transportation all affect star indicators. Recent work⁶ demonstrated that comprehensive investments in transitional care strategies and the social determinants of health were ineffective at reducing readmissions, which suggests that high readmission rates for hospitals in impoverished areas are not only common, but also may not accurately reflect hospital quality and local investment.

Patient experience is also complicating, with research demonstrating that patient perceptions vary significantly by education, age, primary language, ethnicity, and overall health. For example, one-third of average-ranked hospitals would have rankings vary by at least 18 percentile points when evaluated by Spanish-speaking patients. Star ratings fail to capture and communicate this granularity.⁷

More concerning is that star ratings inherently assume that hospital performance is being compared across the same tasks, regardless of patient characteristics, local resources, or the scope of services provided, the latter of which may vary between hospitals. For example, communication may differ in both complexity and time intensity: Explaining an antibiotic to the uncomplicated patient with pneumonia differs from prescribing an antibiotic to a patient who is legally blind from optic neuritis, walks with a cane because of multiple sclerosis, and has 24 other prescription medications. Similar challenges exist for differences in local neighborhood resources and for facilities with differing service scope.

Although one strategy to handle these “disparities” in star ratings might be to risk-adjust for social determinants of health, patients may be better served by first rethinking how star ratings are constructed. Clustering hospitals by scope of services provided and geographic region prior to determining star ratings would provide consumers with meaningful information by helping patients compare and make choices among either local or regional hospitals; national quality rankings are unhelpful for patients.

Arguably one of the most complex and person-dependent service enterprises, care delivery presents unique challenges for evaluation of customer experience and medical quality. Hospital star ratings are no exception: We must rethink their construction so they can be more meaningful for both patients and physicians.

The authors would like to acknowledge Daniel J Brotman, MD, for his editorial advice and input.

Dr Miller reported consulting for the Federal Trade Commission and serving as a member of the Centers for Medicare & Medicaid Services Medicare Evidence Development Coverage Advisory Committee. Drs Siddiqui and Deutschendorf have nothing to disclose.

Still in its infancy, the Hospital Compare overall hospital quality star rating program introduced by the Centers for Medicare & Medicaid Services (CMS) has generated intense industry debate. Individual health systems are microcosms of the challenges of ratings and measurement design. Sibley Memorial Hospital, a member of Johns Hopkins Medicine, is a well-run, 288-bed, community hospital located in a wealthy section of northwest District of Columbia with a five-star rating. In contrast, its academic partner, the Johns Hopkins Hospital, a 1,162-bed hospital with a century-long history of innovation situated in an impoverished Baltimore, Maryland, neighborhood, received a three-star rating.

Hospital ratings are the product of an industry in transition: As care delivery has shifted from an individual provider-driven industry to an increasingly scaled systems enterprise, policymakers implemented regulatory standards targeting quality measurement. Subsequent to the National Academy of Medicine’s 1999 report To Err is Human, policy efforts brought public reporting of quality ratings to multiple market segments, including dialysis facilities (2001), nursing homes (2003), Medicare Advantage plans (2007), and physicians (2015). The hospital industry was no exception, and in 2016—with much controversy¹—CMS launched the hospital star ratings program.

CMS Star Ratings for hospitals are based on seven measure groups: mortality, safety, readmission, patient experience, effectiveness, timeliness, and efficient use of medical imaging. Both industry and researchers have decried the challenges of star ratings, noting that hospitals with a narrower scope of services are more likely to receive higher ratings.² Measure groupings may be further flawed as shown by recent work demonstrating that larger, safety net, or academic hospitals, as well as hospitals offering transplant services, have higher readmission rates,³ which may be caused by differences in patient complexity. Other research has demonstrated that overall quality ratings inappropriately pool all hospitals together, when it may be fairer to initially categorize hospitals and then score them.⁴

It is within this maelstrom of debate that, in this month’s issue of the Journal of Hospital Medicine, Shi and colleagues explore the relationship between hospital star ratings and the socioeconomic features of the surrounding communities.⁵ Conducting their analysis by linking multiple reputable government and industry sources, Shi and colleagues found that counties with higher education attainment and a lower proportion of dual Medicare-Medicaid–eligible populations had higher hospital star ratings. Furthermore, a county’s minority population percentage negatively correlated with hospital ratings. Validating the experience of many rural hospital executives—who frequently experience financial challenges—Shi and colleagues noted that rural hospitals were less likely to receive five-star ratings.

Do these findings reflect a true disparity and lack of access to high-quality hospitals, or are they artifactual—secondary to a flawed construct of hospital quality measurement? Many lower-ranking hospitals are urban academic centers frequently providing services not offered at their five-star community counterparts, such as neurosurgery, comprehensive cancer care, and organ transplants, while simultaneously serving as safety net hospitals, research institutions, trauma centers, and national referral centers.

Sociodemographics factor significantly in self-care management for hospital aftercare. Health literacy, access to primary and behavioral healthcare, and transportation all affect star indicators. Recent work⁶ demonstrated that comprehensive investments in transitional care strategies and the social determinants of health were ineffective at reducing readmissions, which suggests that high readmission rates for hospitals in impoverished areas are not only common, but also may not accurately reflect hospital quality and local investment.

Patient experience is also complicating, with research demonstrating that patient perceptions vary significantly by education, age, primary language, ethnicity, and overall health. For example, one-third of average-ranked hospitals would have rankings vary by at least 18 percentile points when evaluated by Spanish-speaking patients. Star ratings fail to capture and communicate this granularity.⁷

More concerning is that star ratings inherently assume that hospital performance is being compared across the same tasks, regardless of patient characteristics, local resources, or the scope of services provided, the latter of which may vary between hospitals. For example, communication may differ in both complexity and time intensity: Explaining an antibiotic to the uncomplicated patient with pneumonia differs from prescribing an antibiotic to a patient who is legally blind from optic neuritis, walks with a cane because of multiple sclerosis, and has 24 other prescription medications. Similar challenges exist for differences in local neighborhood resources and for facilities with differing service scope.

Although one strategy to handle these “disparities” in star ratings might be to risk-adjust for social determinants of health, patients may be better served by first rethinking how star ratings are constructed. Clustering hospitals by scope of services provided and geographic region prior to determining star ratings would provide consumers with meaningful information by helping patients compare and make choices among either local or regional hospitals; national quality rankings are unhelpful for patients.

Arguably one of the most complex and person-dependent service enterprises, care delivery presents unique challenges for evaluation of customer experience and medical quality. Hospital star ratings are no exception: We must rethink their construction so they can be more meaningful for both patients and physicians.

The authors would like to acknowledge Daniel J Brotman, MD, for his editorial advice and input.

Dr Miller reported consulting for the Federal Trade Commission and serving as a member of the Centers for Medicare & Medicaid Services Medicare Evidence Development Coverage Advisory Committee. Drs Siddiqui and Deutschendorf have nothing to disclose.

“In the end, we will remember not the words of our enemies, but the silence of our friends.”

—Martin Luther King, Jr.

"Code Blue, Emergency Department Code Team to PACU.” A female senior resident dons her personal protective equipment and assembles her team. An enthusiastic male junior resident asks if he can accompany her, and off they go. They encounter a frantic scene in the post-anesthesia care unit (PACU). Before the senior resident can lead the rapid response, a PACU nurse addresses the junior resident: “You are leading the code, correct? What medications would you like?”

“Microaggressions” are subtle, commonplace exchanges that—whether intentional or unintentional—communicate disparaging messages to members of marginalized groups.¹ These groups often include women, members of racial/ethnic groups that are underrepresented in medicine, and lesbian, gay, bisexual, transgender, and queer/questioning (LGBTQ) individuals. Although an individual may not intend to cause harm, their words may still negatively impact the receiving party, who regularly experiences differential treatment based on sex, race, ethnicity, or other social identities. The effects of microaggressions extend beyond personal offense to include anxiety, depression, and even hypertension.^1,2

Addressing microaggressions can be challenging. Given that the recipients of microaggressions are often burdened with responding to them, it is important for bystanders to be empowered to respond as well. A bystander witnesses and recognizes the microaggression and can address it. Based on the work of Sue et al,³ we suggest that bystanders adopt the following strategies:

• Make the “invisible” visible. Many people do not perceive their actions as biased or prejudiced. It is therefore important to bring the implicit bias to the forefront by asking for clarification, naming the implication, or challenging the stereotype.
• Disarm the microaggression. Don’t be afraid to stop, deflect, disagree, or challenge what was said or done, thereby highlighting its potentially harmful impact. Another option is to interrupt the comment as it’s being said and redirect the conversation.
• Educate the speaker. Create a nonpunitive discussion by appealing to common values, promoting empathy, and increasing awareness of societal benefits. The speaker may become defensive and emphasize that their intent was not to cause harm. You must emphasize that, regardless of intent, the impact was hurtful. You may refocus the discussion with a simple statement such as, “I know you meant well, and…”
• Seek external support when needed. Addressing microaggressions can be emotionally taxing. Don’t be afraid to utilize community services, find a support group, or seek advice from professionals.

By virtue of being a neutral third party, bystanders who intervene may have greater success at explaining the impact of the microaggression. In doing so, the bystander also relieves the recipient of the microaggression of a burdensome response. In the above example, another provider in the PACU might pull the nurse aside later and say, “When you asked the junior resident if he was leading the code, you unintentionally indicated that he was the most experienced, which made it more challenging for the female senior resident to lead the response.” In this way, the “invisible” implication of the nurse’s words—that the male resident was the most knowledgeable physician in the room—is made visible, and the female resident is relieved of responding.

Microaggressions do not occur in a vacuum; context matters. Before employing these strategies, consider when, where, and how you address microaggressions. These strategies validate and support those on the receiving end of microaggressions, and thus counteract their deleterious effects. The onus is on us: we must not be silent.

The authors have nothing to disclose.

“In the end, we will remember not the words of our enemies, but the silence of our friends.”

—Martin Luther King, Jr.

"Code Blue, Emergency Department Code Team to PACU.” A female senior resident dons her personal protective equipment and assembles her team. An enthusiastic male junior resident asks if he can accompany her, and off they go. They encounter a frantic scene in the post-anesthesia care unit (PACU). Before the senior resident can lead the rapid response, a PACU nurse addresses the junior resident: “You are leading the code, correct? What medications would you like?”

“Microaggressions” are subtle, commonplace exchanges that—whether intentional or unintentional—communicate disparaging messages to members of marginalized groups.¹ These groups often include women, members of racial/ethnic groups that are underrepresented in medicine, and lesbian, gay, bisexual, transgender, and queer/questioning (LGBTQ) individuals. Although an individual may not intend to cause harm, their words may still negatively impact the receiving party, who regularly experiences differential treatment based on sex, race, ethnicity, or other social identities. The effects of microaggressions extend beyond personal offense to include anxiety, depression, and even hypertension.^1,2

Addressing microaggressions can be challenging. Given that the recipients of microaggressions are often burdened with responding to them, it is important for bystanders to be empowered to respond as well. A bystander witnesses and recognizes the microaggression and can address it. Based on the work of Sue et al,³ we suggest that bystanders adopt the following strategies:

• Make the “invisible” visible. Many people do not perceive their actions as biased or prejudiced. It is therefore important to bring the implicit bias to the forefront by asking for clarification, naming the implication, or challenging the stereotype.
• Disarm the microaggression. Don’t be afraid to stop, deflect, disagree, or challenge what was said or done, thereby highlighting its potentially harmful impact. Another option is to interrupt the comment as it’s being said and redirect the conversation.
• Educate the speaker. Create a nonpunitive discussion by appealing to common values, promoting empathy, and increasing awareness of societal benefits. The speaker may become defensive and emphasize that their intent was not to cause harm. You must emphasize that, regardless of intent, the impact was hurtful. You may refocus the discussion with a simple statement such as, “I know you meant well, and…”
• Seek external support when needed. Addressing microaggressions can be emotionally taxing. Don’t be afraid to utilize community services, find a support group, or seek advice from professionals.

By virtue of being a neutral third party, bystanders who intervene may have greater success at explaining the impact of the microaggression. In doing so, the bystander also relieves the recipient of the microaggression of a burdensome response. In the above example, another provider in the PACU might pull the nurse aside later and say, “When you asked the junior resident if he was leading the code, you unintentionally indicated that he was the most experienced, which made it more challenging for the female senior resident to lead the response.” In this way, the “invisible” implication of the nurse’s words—that the male resident was the most knowledgeable physician in the room—is made visible, and the female resident is relieved of responding.

Microaggressions do not occur in a vacuum; context matters. Before employing these strategies, consider when, where, and how you address microaggressions. These strategies validate and support those on the receiving end of microaggressions, and thus counteract their deleterious effects. The onus is on us: we must not be silent.

The authors have nothing to disclose.

“In the end, we will remember not the words of our enemies, but the silence of our friends.”

—Martin Luther King, Jr.

"Code Blue, Emergency Department Code Team to PACU.” A female senior resident dons her personal protective equipment and assembles her team. An enthusiastic male junior resident asks if he can accompany her, and off they go. They encounter a frantic scene in the post-anesthesia care unit (PACU). Before the senior resident can lead the rapid response, a PACU nurse addresses the junior resident: “You are leading the code, correct? What medications would you like?”

“Microaggressions” are subtle, commonplace exchanges that—whether intentional or unintentional—communicate disparaging messages to members of marginalized groups.¹ These groups often include women, members of racial/ethnic groups that are underrepresented in medicine, and lesbian, gay, bisexual, transgender, and queer/questioning (LGBTQ) individuals. Although an individual may not intend to cause harm, their words may still negatively impact the receiving party, who regularly experiences differential treatment based on sex, race, ethnicity, or other social identities. The effects of microaggressions extend beyond personal offense to include anxiety, depression, and even hypertension.^1,2

Addressing microaggressions can be challenging. Given that the recipients of microaggressions are often burdened with responding to them, it is important for bystanders to be empowered to respond as well. A bystander witnesses and recognizes the microaggression and can address it. Based on the work of Sue et al,³ we suggest that bystanders adopt the following strategies:

• Make the “invisible” visible. Many people do not perceive their actions as biased or prejudiced. It is therefore important to bring the implicit bias to the forefront by asking for clarification, naming the implication, or challenging the stereotype.
• Disarm the microaggression. Don’t be afraid to stop, deflect, disagree, or challenge what was said or done, thereby highlighting its potentially harmful impact. Another option is to interrupt the comment as it’s being said and redirect the conversation.
• Educate the speaker. Create a nonpunitive discussion by appealing to common values, promoting empathy, and increasing awareness of societal benefits. The speaker may become defensive and emphasize that their intent was not to cause harm. You must emphasize that, regardless of intent, the impact was hurtful. You may refocus the discussion with a simple statement such as, “I know you meant well, and…”
• Seek external support when needed. Addressing microaggressions can be emotionally taxing. Don’t be afraid to utilize community services, find a support group, or seek advice from professionals.

By virtue of being a neutral third party, bystanders who intervene may have greater success at explaining the impact of the microaggression. In doing so, the bystander also relieves the recipient of the microaggression of a burdensome response. In the above example, another provider in the PACU might pull the nurse aside later and say, “When you asked the junior resident if he was leading the code, you unintentionally indicated that he was the most experienced, which made it more challenging for the female senior resident to lead the response.” In this way, the “invisible” implication of the nurse’s words—that the male resident was the most knowledgeable physician in the room—is made visible, and the female resident is relieved of responding.

Microaggressions do not occur in a vacuum; context matters. Before employing these strategies, consider when, where, and how you address microaggressions. These strategies validate and support those on the receiving end of microaggressions, and thus counteract their deleterious effects. The onus is on us: we must not be silent.

The authors have nothing to disclose.

From the University of California San Francisco (Dr. Treadwell, Dr. Hessler, Yumei Chen, Swapandeep Mushiana, Dr. Potter, and Dr. Vichinsky), the University of California Los Angeles (Dr. Jacob), and the University of California Berkeley (Alex Chen).

Abstract

• Objective: Adolescents and adults with sickle cell disease (SCD) face pervasive disparities in health resources and outcomes. We explored barriers to and facilitators of care to identify opportunities to support implementation of evidence-based interventions aimed at improving care quality for patients with SCD.
• Methods: We engaged a representative sample of adolescents and adults with SCD (n = 58), health care providers (n = 51), and community stakeholders (health care administrators and community-based organization leads (n = 5) in Northern California in a community-based needs assessment. We conducted group interviews separately with participant groups to obtain in-depth perspectives. Adolescents and adults with SCD completed validated measures of pain interference, quality of care, self-efficacy, and barriers to care. Providers and community stakeholders completed surveys about barriers to SCD care.
• Results: We triangulated qualitative and quantitative data and found that participants with SCD (mean age, 31 ± 8.6 years), providers, and community stakeholders emphasized the social and emotional burden of SCD as barriers. Concrete barriers agreed upon included insurance and lack of resources for addressing pain impact. Adolescents and adults with SCD identified provider issues (lack of knowledge, implicit bias), transportation, and limited social support as barriers. Negative encounters with the health care system contributed to 84% of adolescents and adults with SCD reporting they chose to manage severe pain at home. Providers focused on structural barriers: lack of access to care guidelines, comfort level with and knowledge of SCD management, and poor care coordination.
• Conclusion: Strategies for improving access to compassionate, evidence-based quality care, as well as strategies for minimizing the burden of having SCD, are warranted for this medically complex population.

Keywords: barriers to care; quality of care; care access; care coordination.

Sickle cell disease (SCD), an inherited chronic medical condition, affects about 100,000 individuals in the United States, a population that is predominantly African American.¹ These individuals experience multiple serious and life-threatening complications, most frequently recurrent vaso-occlusive pain episodes,² and they require interactions with multidisciplinary specialists from childhood. Because of advances in treatments, the majority are reaching adulthood; however, there is a dearth of adult health care providers with the training and expertise to manage their complex medical needs.³ Other concrete barriers to adequate SCD care include insurance and distance to comprehensive SCD centers.^4,5

Social, behavioral, and emotional factors may also contribute to challenges with SCD management. SCD may limit daily functional abilities and lead to diminished overall quality of life.^6,7 Some adolescents and adults may require high doses of opioids, which contributes to health care providers’ perceptions that there is a high prevalence of drug addiction in the population.^8,9 These providers express negative attitudes towards adults with SCD, and, consequently, delay medication administration when it is acutely needed and provide otherwise suboptimal treatment.^8,10,11 Adult care providers may also be uncomfortable with prescribing and managing disease-modifying therapies (blood transfusion, hydroxyurea) that have established efficacy.^12-17

As 1 of 8 programs funded by the National Heart, Lung, and Blood Institute’s (NHLBI) Sickle Cell Disease Implementation Consortium (SCDIC), we are using implementation science to reduce barriers to care and improve quality of care and health care outcomes in SCD.^18,19 Given that adolescents and adults with SCD experience high mortality, severe pain, and progressive decline in their ability to function day to day, and also face lack of access to knowledgeable, compassionate providers in primary and emergency settings, the SCDIC focuses on individuals aged 15 to 45 years.^6,8,9,11,12

Our regional SCDIC program, the Sickle Cell Care Coordination Initiative (SCCCI), brings together researchers, clinicians, adolescents, and adults with SCD and their families, dedicated community members, policy makers, and administrators to identify and address barriers to health care within 5 counties in Northern California. One of our first steps was to conduct a community-based needs assessment, designed to inform implementation of evidence-based interventions, accounting for unique contextual factors in our region.

Conceptual Framework for Improving Medical Practice

Our needs assessment is guided by Solberg’s Conceptual Framework for Improving Medical Practice (Figure 1).²⁰ Consistent with the overarching principles of the SCDIC, this conceptual framework focuses on the inadequate implementation of evidence-based guidelines, and on the need to first understand multifactorial facilitators and barriers to guideline implementation in order to effect change. The framework identifies 3 main elements that must be present to ensure improvements in quality-of-care processes and patient outcomes: priority, change process capability, and care process content. Priority refers to ample resource allocation for the specific change, as well as freedom from competing priorities for those implementing the change. Change process capability includes strong, effective leadership, adequate infrastructure for managing change (including resources and time), change management skills at all levels, and an established clinical information system. Care process content refers to context and systems-level changes, such as delivery system redesign as needed, support for self-management to lessen the impact of the disease, and decision support.^21-23

The purpose of our community-based needs assessment was to evaluate barriers to care and quality of care in SCD, within Solberg’s conceptual model for improving medical practice. The specific aims were to evaluate access and barriers to care (eg, lack of provider expertise and training, health care system barriers such as poor care coordination and provider communication); evaluate quality of care; and assess patient needs related to pain, pain interference, self-efficacy, and self-management for adolescents and adults with SCD. We gathered the perspectives of a representative community of adolescents and adults with SCD, their providers, and community stakeholders in order to examine barriers, quality of life and care, and patient experiences in our region.

Methods

Design

In this cross-sectional study, adolescents and adults with SCD, their providers, and community stakeholders participated in group or individual qualitative interviews and completed surveys between October 2017 and March 2018.

Setting and Sample

Recruitment flyers were posted on a regional SCD-focused website, and clinical providers or a study coordinator introduced information about the needs assessment to potential participants with SCD during clinic visits at the participating centers. Participants with SCD were eligible if they had any diagnosis of SCD, were aged 15 to 48 years, and received health services within 5 Northern California counties (Alameda, Contra Costa, Sacramento, San Francisco, and Solano). They were excluded if they did not have a SCD diagnosis or had not received health services within the catchment area. As the project proceeded, participants were asked to refer other adolescents and adults with SCD for the interviews and surveys (snowball sampling). Our goal was to recruit 50 adolescents and adults with SCD into the study, aiming for 10 representatives from each county.

Providers and community stakeholders were recruited via emails, letters and informational flyers. We engaged our partner, the Sickle Cell Data Collection Program,² to generate a list of providers and institutions that had seen patients with SCD in primary, emergency, or inpatient settings in the region. We contacted these institutions to describe the SCCCI and invite participation in the needs assessment. We also invited community-based organization leads and health care administrators who worked with SCD to participate. Providers accessed confidential surveys via a secure link on the study website or completed paper versions. Common data collected across providers included demographics and descriptions of practice settings.

Participants were eligible to be part of the study if they were health care providers (physicians and nurses) representing hematology, primary care, family medicine, internal medicine, or emergency medicine; ancillary staff (social work, psychology, child life); or leaders or administrators of clinical or sickle cell community-based organizations in Northern California (recruitment goal of n = 50). Providers were excluded if they practiced in specialties other than those noted or did not practice within the region.

Data Collection Procedures

After providing assent/consent, participating adolescents and adults with SCD took part in individual and group interviews and completed survey questionnaires. All procedures were conducted in a private space in the sickle cell center or community. Adolescents and adults with SCD completed the survey questionnaire on a tablet, with responses recorded directly in a REDCap (Research Electronic Data Capture) database,²⁴ or on a paper version. Interviews lasted 60 (individual) to 90 (group) minutes, while survey completion time was 20 to 25 minutes. Each participant received a gift card upon completion as an expression of appreciation. All procedures were approved by the institutional review boards of the participating health care facilities.

Group and Individual Interviews

Participants with SCD and providers were invited to participate in a semi-structured qualitative interview prior to being presented with the surveys. Adolescents and adults with SCD were interviewed about barriers to care, quality of care, and pain-related experiences. Providers were asked about barriers to care and treatments. Interview guides were modified for community-based organization leaders and health care administrators who did not provide clinical services. Interview guides can be found in the Appendix. Interviews were conducted by research coordinators trained in qualitative research methods by the first author (MT). As appropriate with semi-structured interviews, the interviewers could word questions spontaneously, change the order of questions for ease of flow of conversation, and inform simultaneous coding of interviews with new themes as those might arise, as long as they touched on all topics within the interview guide.²⁵ The interview guides were written, per qualitative research standards, based on the aims and purpose of the research,²⁶ and were informed by existing literature on access and barriers to care in SCD, quality of care, and the needs of individuals with SCD, including in relation to impact of the disease, self-efficacy, and self-management.

Interviewees participated in either individual or group interviews, but not both. The decision for which type of interview an individual participated in was based on 2 factors: if there were not comparable participants for group interviews (eg, health care administrator and community-based organization lead), these interviews were done individually; and given that we were drawing participants from a 5-county area in Northern California, scheduling was challenging for individuals with SCD with regard to aligning schedules and traveling to a central location where the group interviews were conducted. Provider group interviews were easier to arrange because we could schedule them at the same time as regularly scheduled meetings at the participants’ health care institutions.

Interview Data Gathering and Analysis

Digital recordings of the interviews were cleaned of any participant identifying data and sent for transcription to an outside service. Transcripts were reviewed for completeness and imported into NVivo (www.qsrinternational.com), a qualitative data management program.

A thematic content analysis and deductive and inductive approaches were used to analyze the verbatim transcripts generated from the interviews. The research team was trained in the use of NVivo software to facilitate the coding process. A deductive coding scheme was initially used based on existing concepts in the literature regarding challenges to optimal SCD care, with new codes added as the thematic content analyses progressed. The initial coding, pattern coding, and use of displays to examine the relationships between different categories were conducted simultaneously.^27,28 Using the constant comparative method, new concepts from participants with SCD and providers could be incorporated into subsequent interviews with other participants. For this study, the only additional concepts added were in relation to participant recruitment and retention in the SCDIC Registry. Research team members coded transcripts separately and came together weekly, constantly comparing codes and developing the consensus coding scheme. Where differences between coders existed, code meanings were discussed and clarified until consensus was reached.²⁹

Quantitative data were analyzed using SPSS (v. 25, Chicago, IL). Descriptive statistics (means, standard deviations, frequencies, percentages) were used to summarize demographics (eg, age, gender, and race), economic status, and type of SCD. No systematic differences were detected from cases with missing values. Scale reliabilities (ie, Cronbach α) were evaluated for self-report measures.

Measurement

Adolescents and adults with SCD completed items from the PhenX Toolkit (consensus measures for Phenotypes and eXposures), assessing sociodemographics (age, sex, race, ethnicity, educational attainment, occupation, marital status, annual income, insurance), and clinical characteristics (sickle cell diagnosis and emergency department [ED] and hospital utilization for pain).³⁰

Pain Interference Short Form (Patient-Reported Outcomes Measurement Information System [PROMIS]). The Pain Interference Form consists of 8 items that assess the degree to which pain interfered with day-to-day activities in the previous 7 days at home, including impacts on social, cognitive, emotional, and physical functioning; household chores and recreational activities; sleep; and enjoyment in life. Reliability and validity of the PROMIS Pain Interference Scale has been demonstrated, with strong negative correlations with Physical Function Scales (r = 0.717, P < 0.01), indicating that higher scores are associated with lower function (β = 0.707, P < 0.001).³¹ The Cronbach α estimate for the other items on the pain interference scale was 0.99. Validity analysis indicated strong correlations with pain-related domains: BPI Interference Subscale (rho = 0.90), SF-36 Bodily Pain Subscale (rho = –0.84), and 0–10 Numerical Rating of Pain Intensity (rho = 0.48).³²

Adult Sickle Cell Quality of Life Measurement Information System (ASCQ-Me) Quality of Care (QOC). ASCQ-Me QOC consists of 27 items that measure the quality of care that adults with SCD have received from health care providers.³³ There are 3 composites: provider communication (quality of patient and provider communication), ED care (quality of care in the ED), and access (to routine and emergency care). Internal consistency reliability for all 3 composites is greater than 0.70. Strong correlations of the provider communication composite with overall ratings of routine care (r = 0.65) and overall provider ratings (r = 0.83) provided evidence of construct validity. Similarly, the ED care composite was strongly correlated with overall ratings of QOC in the ED, and the access composite was highly correlated with overall evaluations of ED care (r = 0.70). Access, provider interaction, and ED care composites were reliable (Cronbach α, 0.70–0.83) and correlated with ratings of global care (r = 0.32–0.83), further indicating construct validity.³³

Sickle Cell Self-Efficacy Scale (SCSES). The SCSES is a 9-item, self-administered questionnaire measuring perceptions of the ability to manage day-to-day issues resulting from SCD. SCSES items are scored on a 5-point scale ranging from Not sure at all (1) to Very sure (5). Individual item responses are summed to give an overall score, with higher scores indicating greater self-efficacy. The SCSES has acceptable reliability (r = 0.45, P < 0.001) and validity (α = 0.89).^34,35

Sickle Cell Disease Barriers Checklist. This checklist consists of 53 items organized into 8 categories: insurance, transportation, accommodations and accessibility, provider knowledge and attitudes, social support, individual barriers such as forgetting or difficulties understanding instructions, emotional barriers (fear, anger), and disease-related barriers. Participants check applicable barriers, with a total score range of 0 to 53 and higher scores indicating more barriers to care. The SCD Barriers Checklist has demonstrated face validity and test-retest reliability (Pearson r = 0.74, P < 0.05).⁵

ED Provider Checklist. The ED provider survey is a checklist of 14 statements pertaining to issues regarding patient care, with which the provider rates level of agreement. Items representing the attitudes and beliefs of providers towards patients with SCD are rated on a Likert-type scale, with level of agreement indicated as 1 (strongly disagree) to 6 (strongly agree). The positive attitudes subscale consists of 4 items (Cronbach α= 0.85), and the negative attitudes subscale consists of 6 items (Cronbach α = 0.89). The Red-Flag Behaviors subscale includes 4 items that indicate behavior concerns about drug-seeking, such as requesting specific narcotics and changing behavior when the provider walks in.^8,36,37

Sickle cell and primary care providers also completed a survey consisting of sets of items compiled from existing provider surveys; this survey consisted of a list of 16 barriers to using opioids, which the providers rated on a 5-point Likert-type scale (1, not a barrier; 5, complete barrier).^13,16,38 Providers indicated their level of experience with caring for patients with SCD; care provided, such as routine health screenings; and comfort level with providing preventive care, managing comorbidities, and managing acute and chronic pain. Providers were asked what potential facilitators might improve care for patients with SCD, including higher reimbursement, case management services, access to pain management specialists, and access to clinical decision-support tools. Providers responded to specific questions about management with hydroxyurea (eg, criteria for, barriers to, and comfort level with prescribing).³⁹ The surveys are included in the Appendix.

Triangulation

Data from the interviews and surveys were triangulated to enhance understanding of results generated from the different data sources.⁴⁰ Convergence of findings, different facets of the same phenomenon, or new perspectives were examined.

 

 

Results

Qualitative Data

Adolescents and adults with SCD (n = 55) and health care providers and community stakeholders (n = 56) participated in group or individual interviews to help us gain an in-depth understanding of the needs and barriers related to SCD care in our 5-county region. Participants with SCD described their experiences, which included stigma, racism, labeling, and, consequently, stress. They also identified barriers such as lack of transportation, challenges with insurance, and lack of access to providers who were competent with pain management. They reported that having SCD in a health care system that was unable to meet their needs was burdensome.

Barriers to Care and Treatments. Adolescents and adults indicated that SCD and its sequelae posed significant barriers to health care. Feelings of tiredness and pain make it more difficult for them to seek care. The emotional burden of SCD (fear and anger) was a frequently cited barrier, which was fueled by previous negative encounters with the health care system. All adolescents and adults with SCD reported that they knew of stigma in relation to seeking pain management that was pervasive and long-standing, and the majority reported they had directly experienced stigma. They reported that being labeled as “drug-seekers” was typical when in the ED for pain management. Participants articulated unconscious bias or overt racism among providers: “people with sickle cell are Black ... and Black pain is never as valuable as White pain” (25-year-old male). Respondents with SCD described challenges to the credibility of their pain reports in the ED. They reported that ED providers expressed doubts regarding the existence and/or severity of their pain, consequently creating a feeling of disrespect for patients seeking pain relief. The issue of stigma was mentioned by only 2 of 56 providers during their interviews.

Lack of Access to Knowledgeable, Compassionate Providers. Lack of access to knowledgeable care providers was another prevalent theme expressed by adolescents and adults with SCD. Frustration occurred when providers did not have knowledge of SCD and its management, particularly pain assessment. Adolescents and adults with SCD noted the lack of compassion among providers: “I’ve been kicked out of the hospital because they felt like okay, well we gave you enough medication, you should be all right” (29-year-old female). Providers specifically mentioned lack of compassion and knowledge as barriers to SCD care much less often during their interviews compared with the adolescents and adults with SCD.

Health Care System Barriers. Patient participants often expressed concerns about concrete and structural aspects of care. Getting to their appointments was a challenge for half of the interviewees, as they either did not have access to a vehicle or could not afford to travel the needed distance to obtain quality care. Even when hospitals were accessible by public transportation, those with excruciating pain understandably preferred a more comfortable and private way to travel: “I would like to change that, something that will be much easier, convenient for sickle cell patients that do suffer with pain, that they don’t have to travel always to see the doctor” (30-year-old male).

Insurance and other financial barriers also played an important role in influencing decisions to seek health care services. Medical expenses were not covered, or co-pays were too high. The Medicaid managed care system could prevent access to knowledgeable providers who were not within network. Such a lack of access discouraged some adolescents and adults with SCD from seeking acute and preventive care.

Transition From Pediatric to Adult Care. Interviewees with SCD expressed distress about the gap between pediatric and adult care. They described how they had a long-standing relationship with their medical providers, who were familiar with their medical background and history from childhood. Adolescent interviewees reported an understanding of their own pain management as well as adherence to and satisfaction with their individualized pain plans. However, adults noted that satisfaction plummeted with increasing age due to the limited number of experienced adult SCD providers, which was compounded by negative experiences (stigma, racism, drug-seeking label).

One interviewee emphasized the difficulty of finding knowledgeable providers after transition: “When you’re a pediatric sickle cell [patient], you have the doctors there every step of the way, but not with adult sickle cell… I know when I first transitioned I never felt more alone in my life… you look at that ER doctor kind of with the same mindset as you would your hematologist who just hand walked you through everything. And adult care providers were a lot more blunt and cold and they’re like… ‘I don’t know; I’m not really educated in sickle cell.’” A sickle cell provider shared his insight about the problem of transitioning: “I think it’s particularly challenging because we, as a community, don’t really set them up for success. It’s different from other chronic conditions [in that] it’s much harder to find an adult sickle cell provider. There’s not a lot of adult hematologists that will take care of our adult patients, and so I know statistically, there’s like a drop-down in the overall outcomes of our kids after they age out of our pediatric program.”

Self-Management, Supporting Hydroxyurea Use. Interview participants with SCD reported using a variety of methods to manage pain at home and chose to go to the ED only when the pain became intolerable. Patients and providers expressed awareness of different resources for managing pain at home, yet they also indicated that these resources have not been consolidated in an accessible way for patients and families. Some resources cited included heat therapy, acupuncture, meditation, medical marijuana, virtual reality devices, and pain medications other than opioids.

Patients and providers expressed the need for increasing awareness and education about hydroxyurea. Many interview participants with SCD were concerned about side effects, multiple visits with a provider during dose titration, and ongoing laboratory monitoring. They also expressed difficulties with scheduling multiple appointments, depending on access to transportation and limited provider clinic hours. They were aware of strategies for improving adherence with hydroxyurea, including setting phone alarms, educating family members about hydroxyurea, and eliciting family support, but expressed needing help to consistently implement these strategies.

Safe Opioid Prescribing. Adult care providers expressed concerns about safe opioid prescribing for patients with SCD. They were reluctant to prescribe opioid doses needed to adequately control SCD pain. Providers expressed uncertainty and fear or concern about medical/legal liability or about their judgment about what’s safe and not safe for patients with chronic use/very high doses of opioids. “I know we’re in like this opiate epidemic here in this country but I feel like these patients don’t really fit under that umbrella that the problem is coming from so [I am] just trying to learn more about how to take care of them.”

Care Coordination and Provider Communication. Adolescents and adults with SCD reported having positive experiences—good communication, established trust, and compassionate care—with their usual providers. However, they perceived that ED physicians and nurses did not really care about them. Both interviewees with SCD and providers recognized the importance of good communication in all settings as the key to overcoming barriers to receiving quality care. All agreed on the importance of using individual pain plans so that all providers, especially ED providers, can be more at ease with treating adolescents and adults with SCD.

Quantitative Data: Adolescents and Adults With SCD

Fifty-eight adolescents and adults with SCD (aged 15 to 48 years) completed the survey. Three additional individuals who did not complete the interview completed the survey. Reasons for not completing the interview included scheduling challenges (n = 2) or a sickle cell pain episode (n = 1). The average age of participants was 31 years ± 8.6, more than half (57%) were female, and the majority (93%) were African American (Table 1). Most (71%) had never been married. Half (50%) had some college or an associate degree, and 40% were employed and reported an annual household income of less than $30,000. Insurance coverage was predominantly Medi-Cal (Medicaid, 69%). The majority of participants resided in Alameda (34.5%) or Contra Costa (21%) counties. The majority of sickle cell care was received in Alameda County, whether outpatient (52%), inpatient (40%), or ED care (41%). The majority (71%) had a diagnosis of SCD hemoglobin SS.

Pain. More than one-third of individuals with SCD reported 1 or 2 ED visits for pain in the previous 6 months (34%), and more than 3 hospitalizations (36%) related to pain in the previous year (Table 2). The majority (85%) reported having severe pain at home in the previous 6 months that they did not seek health care for, consistent with their reports in the qualitative interviews. More than half (59%) reported 4 or more of these severe pain episodes that led to inability to perform daily activities for 1 week or more. While pain interference on the PROMIS Pain Interference Short Form on average (T-score, 59.6 ± 8.6) was similar to that of the general population (T-score, 50 ± 10), a higher proportion of patients with SCD reported pain interference compared with the general population. The mean self-efficacy (confidence in ability to manage complications of SCD) score on the SCSES of 30.0 ± 7.3 (range, 9–45) was similar to that of other adults with SCD (mean, 32.2 ± 7.0). Twenty-five percent of the present sample had a low self-efficacy score (< 25).

Barriers to Care and Treatments. Consistent with the qualitative data, SCD-related symptoms such as tiredness (64%) and pain (62%) were reported most often as barriers to care (Table 3). Emotions (> 25%) such as worry/fear, frustration/anger, and lack of confidence were other important barriers to care. Provider knowledge and attitudes were cited next most often, with 38% of the sample indicating “Providers accuse me of drug-seeking” and “It is hard for me to find a provider who has enough experiences with or knowledge about SCD.” Participants expressed that they were not believed when in pain and “I am treated differently from other patients.” Almost half of respondents cited “I am not seen quickly enough when I am in pain” as a barrier to their care.

Consistent with the qualitative data, transportation barriers (not having a vehicle, costs of transportation, public transit not easy to get to) were cited by 55% of participants. About half of participants reported that insurance was an important barrier, with high co-pays and medications and other services not covered. In addition, gathering approvals was a long and fragmented process, particularly for consultations among providers (hematology, primary care provider, pain specialist). Furthermore, insurance provided limited choices about location for services.

Participants reported social support system burnout (22%), help needed with daily activities (21%), and social isolation or generally not having enough support (33%) as ongoing barriers. Difficulties were encountered with self-management (eg, taking medications on time or making follow-up appointments, 19%), with 22% of participants finding the health care system confusing or hard to understand. Thirty percent reported “Places for me to go to learn how to stay well are not close by or easy to get to.” ”Worry about side effects” (33%) was a common barrier to hydroxyurea use. Participants described “forgetting to take the medicine,” “tried before but it did not work,” “heard scary things” about hydroxyurea, and “not interested in taking another medicine” as barriers.

Quality of Care. More than half (51%) of the 53 participants who had accessed health care in the previous year rated their overall health care as poor on the ASCQ-Me QOC measure. This was significantly higher compared to the reports from more than 47,000 adults with Medicaid in 2017 (16%),⁴¹ and to the 2008-2009 report from 556 adults with SCD from across the United States (37%, Figure 2).³³ The major contributor to these poor ratings for participants in our sample was low satisfaction with ED care.

Sixty percent of the 42 participants who had accessed ED care in the past year indicated “never” or “sometimes” to the question “When you went to the ED for care, how often did you get it as soon as you wanted?” compared with only 16% of the 2017 adult Medicaid population responding (n = 25,789) (Figure 3). Forty-seven percent of those with an ED visit indicated that, in the previous 12 months, they had been made to wait “more than 2 hours before receiving treatment for acute pain in the ED.” However, in the previous 12 months, 39% reported that their wait time in the ED had been only “between five minutes and one hour.”

On the ASCQ-Me QOC Access to Care composite measure, 33% of 42 participants responding reported they were seen at a routine appointment as soon as they would have liked. This is significantly lower compared to 56% of the adult Medicaid population responding to the same question. Reports of provider communication (Provider Communication composite) for adolescents and adults with SCD were comparable to reports of adults with SCD from the ASCQ-Me field test,³³ but adults with Medicaid reported higher ratings of quality communication behaviors (Figure 4).^33,41 Nearly 60% of both groups with SCD reported that providers “always” performed quality communication behaviors—listened carefully, spent enough time, treated them with respect, and explained things well—compared with more than 70% of adults with Medicaid.

Participants from all counties reported the same number of barriers to care on average (3.3 ± 2.1). Adolescents and adults who reported more barriers to care also reported lower satisfaction with care (r = –0.47, P < 0.01) and less confidence in their ability to manage their SCD (self-efficacy, r = – 0.36, P < 0.05). Female participants reported more barriers to care on average compared with male participants (2.6 ± 2.4 vs 1.4 ± 2.0, P = 0.05). Participants with higher self-efficacy reported lower pain ratings (r = –0.47, P < 0.001).

Quantitative Data: Health Care Providers

Providers (n = 56) and community stakeholders (2 leaders of community-based organizations and 3 health care administrators) were interviewed, with 29 also completing the survey. The reason for not completing (n = 22) was not having the time once the interview was complete. A link to the survey was sent to any provider not completing at the time of the interview, with 2 follow-up reminders. The majority of providers were between the ages of 31 and 50 years (46.4%), female (71.4%), and white (66.1%) (Table 4). None were of Hispanic, Latinx, or Spanish origin. Thirty-six were physicians (64.3%), and 16 were allied health professionals (28.6%). Of the 56 providers, 32 indicated they had expertise caring for patients with SCD (57.1%), 14 were ED providers (25%), and 5 were primary care providers. Most of the providers practiced in an urban setting (91.1%).

Barriers to Care: ED Provider Perspectives. Nine of 14 ED providers interviewed completed the survey on their perspectives regarding barriers to care in the ED, difficulty with follow-ups, ED training resources, and pain control for patients with SCD. ED providers (n = 8) indicated that “provider attitudes” were a barrier to care delivery in the ED for patients with SCD. Some providers (n = 7) indicated that “implicit bias,” “opioid epidemic,” “concern about addiction,” and “patient behavior” were barriers. Respondents indicated that “overcrowding” (n = 6) and “lack of care pathway/protocol” (n = 5) were barriers. When asked to express their level of agreement with statements about SCD care in the ED, respondents disagreed/strongly disagreed (n = 5) that they were “able to make a follow-up appointment” with a sickle cell specialist or primary care provider upon discharge from the ED, and others disagreed/strongly disagreed (n = 4) that they were able to make a “referral to a case management program.”

ED training and resources. Providers agreed/strongly agreed (n = 8) that they had the knowledge and training to care for patients with SCD, that they had access to needed medications, and that they had access to knowledgeable nursing staff with expertise in SCD care. All 9 ED providers indicated that they had sufficient physician/provider staffing to provide good pain management to persons with SCD in the ED.

Pain control in the ED. Seven ED providers indicated that their ED used individualized dosing protocols to treat sickle cell pain, and 5 respondents indicated their ED had a protocol for treating sickle cell pain. Surprisingly, only 3 indicated that they were aware of the NHLBI recommendations for the treatment of vaso-occlusive pain.

Barriers to Care: Primary Care Provider Perspectives. Twenty providers completed the SCD provider section of the survey, including 17 multidisciplinary SCD providers from 4 sickle cell special care centers and 3 community primary care providers. Of the 20, 12 were primary care providers for patients with SCD (Table 4).

Patient needs. Six primary care providers indicated that the medical needs of patients with SCD were being met, but none indicated that the behavioral health or mental health needs were being met.

Managing SCD comorbidities. Five primary care providers indicated they were very comfortable providing preventive ambulatory care to patients with SCD. Six indicated they were very comfortable managing acute pain episodes, but none were very comfortable managing comorbidities such as pulmonary hypertension, diabetes, or chronic pain.

Barriers to opioid use. Only 3 of 12 providers reviewing a list of 15 potential barriers to the use of opioids for SCD pain management indicated a perceived lack of efficacy of opioids, development of tolerance and dependence, and concerns about community perceptions as barriers. Two providers selected potential for diversion as a moderate barrier to opioid use.

Barriers to hydroxyurea use. Eight of 12 providers indicated that the common reasons that patients/families refuse hydroxyurea were “worry about side effects”; 7 chose “don’t want to take another medicine,” and 6 chose “worry about carcinogenic potential.” Others (n = 10) indicated that “patient/family adherence with hydroxyurea” and “patient/family adherence with required blood tests” were important barriers to hydroxyurea use. Eight of the 12 providers indicated that they were comfortable with managing hydroxyurea in patients with SCD.

Care redesign. Twenty SCD and primary care providers completed the Care Redesign section of the survey. Respondents (n = 11) indicated that they would see more patients with SCD if they had accessible case management services available without charge or if patient access to transportation to clinic was also available. Ten indicated that they would see more patients with SCD if they had an accessible community health worker (who understands patient’s/family’s social situation) and access to a pain management specialist on call to answer questions and who would manage chronic pain. All (n = 20) were willing to see more patients with SCD in their practices. Most reported that a clinical decision-support tool for SCD treatment (n = 13) and avoidance of complications (n = 12) would be useful.

Discussion

We evaluated access and barriers to care, quality of care, care coordination, and provider communication from the perspectives of adolescents and adults with SCD, their care providers, and community stakeholders, within the Solberg conceptual model for quality improvement. We found that barriers within the care process content domain (context and systems) were most salient for this population of adolescents and adults with SCD, with lack of provider knowledge and poor attitudes toward adolescents and adults with SCD, particularly in the ED, cited consistently by participant groups. Stigmatization and lack of provider compassion that affected the quality of care were particularly problematic. These findings are consistent with previous reports.^42,43 Adult health care (particularly ED) provider biases and negative attitudes have been recognized as major barriers to optimal pain management in SCD.^8,11,44,45 Interestingly, ED providers in our needs assessment indicated that they felt they had the training and resources to manage patients with SCD. However, only a few actually reported knowing about the NHLBI recommendations for the treatment of vaso-occlusive pain.

Within the care process content domain, we also found that SCD-related complications and associated emotions (fear, worry, anxiety), compounded by lack of access to knowledgeable and compassionate providers, pose a significant burden. Negative encounters with the health care system contributed to a striking 84% of patient participants choosing to manage severe pain at home, with pain seriously interfering with their ability to function on a daily basis. ED providers agreed that provider attitudes and implicit bias pose important barriers to care for adolescents and adults with SCD. Adolescents and adults with SCD wanted, and understood the need, to enhance self-management skills. Both they and their providers agreed that barriers to hydroxyurea uptake included worries about potential side effects, challenges with adherence to repeated laboratory testing, and support with remembering to take the medicine. However, providers uniformly expressed that access to behavioral and mental health services were, if not nonexistent, impossible to access.

Participants with SCD and their providers reported infrastructural challenges (change process capability), as manifested in limitations with accessing acute and preventive care due to transportation- and insurance- related issues. There were health system barriers that were particularly encountered during the transition from pediatric to adult care. These findings are consistent with previous reports that have found fewer interdisciplinary services available in the adult care settings compared with pediatrics.^46,47 Furthermore, adult care providers were less willing to accept adults with SCD because of the complexity of their management, for which the providers did not have the necessary expertise.^3,48-50 In addition, both adolescents and adults with SCD and primary care providers highlighted the inadequacies of the current system in addressing the chronic pain needs of this population. Linking back to the Solberg conceptual framework, our needs assessment results confirm the important role of establishing SCD care as a priority within a health care system—this requires leadership and vision. The vision and priorities must be implemented by effective health care teams. Multilevel approaches or interventions, when implemented, will lead to the desired outcomes.

Findings from our needs assessment within our 5-county region mirror needs assessment results from the broader consortium.⁵¹ The SCDIC has prioritized developing an intervention that addresses the challenges identified within the care process domain by directly enhancing provider access to patient individualized care plans in the electronic health record in the ED. Importantly, ED providers will be asked to view a short video that directly challenges bias and stigma in the ED. Previous studies have indeed found that attitudes can be improved by providers viewing short video segments of adults with SCD discussing their experiences.^36,52 This ED protocol will be one of the interventions that we will roll out in Northern California, given the significance of negative ED encounters reported by needs assessment participants. An additional feature of the intervention is a script for adults with SCD that guides them through introducing their individualized pain plan to their ED providers, thereby enhancing their self-efficacy in a situation that has been so overwhelmingly challenging.

We will implement a second SCDIC intervention that utilizes a mobile app to support self-management on the part of the patient, by supporting motivation and adherence with hydroxyurea.⁵³ A companion app supports hydroxyurea guideline adherence on the part of the provider, in keeping with one of our findings that providers are in need of decision-support tools. Elements of the intervention also align with our findings related to the importance of a support system in managing SCD, in that participants will identify a supportive partner who will play a specific role in supporting their adherence with hydroxyurea.

On our local level, we have, by necessity, partnered with leaders and community stakeholders throughout the region to ensure that these interventions to improve SCD care are prioritized. Grant funds provide initial resources for the SCDIC interventions, but our partnering health care administrators and medical directors must ensure that participating ED and hematology providers are free from competing priorities in order to implement the changes. We have partnered with a SCD community-based organization that is designing additional educational presentations for local emergency medicine providers, with the goal to bring to life very personal stories of bias and stigma within the EDs that directly contribute to decisions to avoid ED care despite severe symptoms.

Although we attempted to obtain samples of adolescents and adults with SCD and their providers that were representative across the 5-county region, the larger proportion of respondents were from 1 county. We did not assess concerns of age- and race-matched adults in our catchment area, so we cannot definitively say that our findings are unique to SCD. However, our results are consistent with findings from the national sample of adults with SCD who participated in the ASCQ-Me field test, and with results from the SCDIC needs assessment.^33,51 Interviews and surveys are subject to self-report bias and, therefore, may or may not reflect the actual behaviors or thoughts of participants. Confidence is increased in our results given the triangulation of expressed concerns across participant groups and across data collection strategies. The majority of adolescents and adults with SCD (95%) completed both the interview and survey, while 64% of ED providers interviewed completed the survey, compared with 54% of SCD specialists and primary care providers. These response rates are more than acceptable within the realm of survey response rates.^54,55

Although we encourage examining issues with care delivery within the conceptual framework for quality improvement presented, we recognize that grant funding allowed us to conduct an in-depth needs assessment that might not be feasible in other settings. Still, we would like readers to understand the importance of gathering data for improvement in a systematic manner across a range of participant groups, to ultimately inform the development of interventions and provide for evaluation of outcomes as a result of the interventions. This is particularly important for a disease, such as SCD, that is both medically and sociopolitically complex.

Conclusion

Our needs assessment brought into focus the multiple factors contributing to the disparities in health care experienced by adolescents and adults with SCD on our local level, and within the context of inequities in health resources and outcomes on the national level. We propose solutions that include specific interventions developed by a consortium of SCD and implementation science experts. We utilize a quality improvement framework to ensure that the elements of the interventions also address the barriers identified by our local providers and patients that are unique to our community. The pervasive challenges in SCD care, coupled with its medical complexities, may seem insurmountable, but our survey and qualitative results provide us with a road map for the way forward.

Acknowledgments: The authors thank the adolescents and adults with sickle cell disease, the providers, and the community stakeholders who completed the interviews and surveys. The authors also acknowledge the SCCCI co-investigators for their contributions to this project, including Michael Bell, MD, Ward Hagar, MD, Christine Hoehner, FNP, Kimberly Major, MSW, Anne Marsh, MD, Lynne Neumayr, MD, and Ted Wun, MD. We also thank Kamilah Bailey, Jameelah Hodge, Jennifer Kim, Michael Rowland, Adria Stauber, Amber Fearon, and Shanda Robertson, and the Sickle Cell Data Collection Program for their contributions.

Corresponding author: Marsha J. Treadwell, PhD, University of California San Francisco Benioff Children’s Hospital Oakland, 747 52nd St., Oakland, CA 94609; marsha.treadwell@ucsf.edu.

Financial disclosures: None.

Funding/support: This work was supported by grant # 1U01HL134007 from the National Heart, Lung, and Blood Institute to the University of California San Francisco Benioff Children’s Hospital Oakland.

From the University of California San Francisco (Dr. Treadwell, Dr. Hessler, Yumei Chen, Swapandeep Mushiana, Dr. Potter, and Dr. Vichinsky), the University of California Los Angeles (Dr. Jacob), and the University of California Berkeley (Alex Chen).

Abstract

• Objective: Adolescents and adults with sickle cell disease (SCD) face pervasive disparities in health resources and outcomes. We explored barriers to and facilitators of care to identify opportunities to support implementation of evidence-based interventions aimed at improving care quality for patients with SCD.
• Methods: We engaged a representative sample of adolescents and adults with SCD (n = 58), health care providers (n = 51), and community stakeholders (health care administrators and community-based organization leads (n = 5) in Northern California in a community-based needs assessment. We conducted group interviews separately with participant groups to obtain in-depth perspectives. Adolescents and adults with SCD completed validated measures of pain interference, quality of care, self-efficacy, and barriers to care. Providers and community stakeholders completed surveys about barriers to SCD care.
• Results: We triangulated qualitative and quantitative data and found that participants with SCD (mean age, 31 ± 8.6 years), providers, and community stakeholders emphasized the social and emotional burden of SCD as barriers. Concrete barriers agreed upon included insurance and lack of resources for addressing pain impact. Adolescents and adults with SCD identified provider issues (lack of knowledge, implicit bias), transportation, and limited social support as barriers. Negative encounters with the health care system contributed to 84% of adolescents and adults with SCD reporting they chose to manage severe pain at home. Providers focused on structural barriers: lack of access to care guidelines, comfort level with and knowledge of SCD management, and poor care coordination.
• Conclusion: Strategies for improving access to compassionate, evidence-based quality care, as well as strategies for minimizing the burden of having SCD, are warranted for this medically complex population.

Keywords: barriers to care; quality of care; care access; care coordination.

Sickle cell disease (SCD), an inherited chronic medical condition, affects about 100,000 individuals in the United States, a population that is predominantly African American.¹ These individuals experience multiple serious and life-threatening complications, most frequently recurrent vaso-occlusive pain episodes,² and they require interactions with multidisciplinary specialists from childhood. Because of advances in treatments, the majority are reaching adulthood; however, there is a dearth of adult health care providers with the training and expertise to manage their complex medical needs.³ Other concrete barriers to adequate SCD care include insurance and distance to comprehensive SCD centers.^4,5

Social, behavioral, and emotional factors may also contribute to challenges with SCD management. SCD may limit daily functional abilities and lead to diminished overall quality of life.^6,7 Some adolescents and adults may require high doses of opioids, which contributes to health care providers’ perceptions that there is a high prevalence of drug addiction in the population.^8,9 These providers express negative attitudes towards adults with SCD, and, consequently, delay medication administration when it is acutely needed and provide otherwise suboptimal treatment.^8,10,11 Adult care providers may also be uncomfortable with prescribing and managing disease-modifying therapies (blood transfusion, hydroxyurea) that have established efficacy.^12-17

As 1 of 8 programs funded by the National Heart, Lung, and Blood Institute’s (NHLBI) Sickle Cell Disease Implementation Consortium (SCDIC), we are using implementation science to reduce barriers to care and improve quality of care and health care outcomes in SCD.^18,19 Given that adolescents and adults with SCD experience high mortality, severe pain, and progressive decline in their ability to function day to day, and also face lack of access to knowledgeable, compassionate providers in primary and emergency settings, the SCDIC focuses on individuals aged 15 to 45 years.^6,8,9,11,12

Our regional SCDIC program, the Sickle Cell Care Coordination Initiative (SCCCI), brings together researchers, clinicians, adolescents, and adults with SCD and their families, dedicated community members, policy makers, and administrators to identify and address barriers to health care within 5 counties in Northern California. One of our first steps was to conduct a community-based needs assessment, designed to inform implementation of evidence-based interventions, accounting for unique contextual factors in our region.

Conceptual Framework for Improving Medical Practice

Our needs assessment is guided by Solberg’s Conceptual Framework for Improving Medical Practice (Figure 1).²⁰ Consistent with the overarching principles of the SCDIC, this conceptual framework focuses on the inadequate implementation of evidence-based guidelines, and on the need to first understand multifactorial facilitators and barriers to guideline implementation in order to effect change. The framework identifies 3 main elements that must be present to ensure improvements in quality-of-care processes and patient outcomes: priority, change process capability, and care process content. Priority refers to ample resource allocation for the specific change, as well as freedom from competing priorities for those implementing the change. Change process capability includes strong, effective leadership, adequate infrastructure for managing change (including resources and time), change management skills at all levels, and an established clinical information system. Care process content refers to context and systems-level changes, such as delivery system redesign as needed, support for self-management to lessen the impact of the disease, and decision support.^21-23

The purpose of our community-based needs assessment was to evaluate barriers to care and quality of care in SCD, within Solberg’s conceptual model for improving medical practice. The specific aims were to evaluate access and barriers to care (eg, lack of provider expertise and training, health care system barriers such as poor care coordination and provider communication); evaluate quality of care; and assess patient needs related to pain, pain interference, self-efficacy, and self-management for adolescents and adults with SCD. We gathered the perspectives of a representative community of adolescents and adults with SCD, their providers, and community stakeholders in order to examine barriers, quality of life and care, and patient experiences in our region.

Methods

Design

In this cross-sectional study, adolescents and adults with SCD, their providers, and community stakeholders participated in group or individual qualitative interviews and completed surveys between October 2017 and March 2018.

Setting and Sample

Recruitment flyers were posted on a regional SCD-focused website, and clinical providers or a study coordinator introduced information about the needs assessment to potential participants with SCD during clinic visits at the participating centers. Participants with SCD were eligible if they had any diagnosis of SCD, were aged 15 to 48 years, and received health services within 5 Northern California counties (Alameda, Contra Costa, Sacramento, San Francisco, and Solano). They were excluded if they did not have a SCD diagnosis or had not received health services within the catchment area. As the project proceeded, participants were asked to refer other adolescents and adults with SCD for the interviews and surveys (snowball sampling). Our goal was to recruit 50 adolescents and adults with SCD into the study, aiming for 10 representatives from each county.

Providers and community stakeholders were recruited via emails, letters and informational flyers. We engaged our partner, the Sickle Cell Data Collection Program,² to generate a list of providers and institutions that had seen patients with SCD in primary, emergency, or inpatient settings in the region. We contacted these institutions to describe the SCCCI and invite participation in the needs assessment. We also invited community-based organization leads and health care administrators who worked with SCD to participate. Providers accessed confidential surveys via a secure link on the study website or completed paper versions. Common data collected across providers included demographics and descriptions of practice settings.

Participants were eligible to be part of the study if they were health care providers (physicians and nurses) representing hematology, primary care, family medicine, internal medicine, or emergency medicine; ancillary staff (social work, psychology, child life); or leaders or administrators of clinical or sickle cell community-based organizations in Northern California (recruitment goal of n = 50). Providers were excluded if they practiced in specialties other than those noted or did not practice within the region.

Data Collection Procedures

After providing assent/consent, participating adolescents and adults with SCD took part in individual and group interviews and completed survey questionnaires. All procedures were conducted in a private space in the sickle cell center or community. Adolescents and adults with SCD completed the survey questionnaire on a tablet, with responses recorded directly in a REDCap (Research Electronic Data Capture) database,²⁴ or on a paper version. Interviews lasted 60 (individual) to 90 (group) minutes, while survey completion time was 20 to 25 minutes. Each participant received a gift card upon completion as an expression of appreciation. All procedures were approved by the institutional review boards of the participating health care facilities.

Group and Individual Interviews

Participants with SCD and providers were invited to participate in a semi-structured qualitative interview prior to being presented with the surveys. Adolescents and adults with SCD were interviewed about barriers to care, quality of care, and pain-related experiences. Providers were asked about barriers to care and treatments. Interview guides were modified for community-based organization leaders and health care administrators who did not provide clinical services. Interview guides can be found in the Appendix. Interviews were conducted by research coordinators trained in qualitative research methods by the first author (MT). As appropriate with semi-structured interviews, the interviewers could word questions spontaneously, change the order of questions for ease of flow of conversation, and inform simultaneous coding of interviews with new themes as those might arise, as long as they touched on all topics within the interview guide.²⁵ The interview guides were written, per qualitative research standards, based on the aims and purpose of the research,²⁶ and were informed by existing literature on access and barriers to care in SCD, quality of care, and the needs of individuals with SCD, including in relation to impact of the disease, self-efficacy, and self-management.

Interviewees participated in either individual or group interviews, but not both. The decision for which type of interview an individual participated in was based on 2 factors: if there were not comparable participants for group interviews (eg, health care administrator and community-based organization lead), these interviews were done individually; and given that we were drawing participants from a 5-county area in Northern California, scheduling was challenging for individuals with SCD with regard to aligning schedules and traveling to a central location where the group interviews were conducted. Provider group interviews were easier to arrange because we could schedule them at the same time as regularly scheduled meetings at the participants’ health care institutions.

Interview Data Gathering and Analysis

Digital recordings of the interviews were cleaned of any participant identifying data and sent for transcription to an outside service. Transcripts were reviewed for completeness and imported into NVivo (www.qsrinternational.com), a qualitative data management program.

A thematic content analysis and deductive and inductive approaches were used to analyze the verbatim transcripts generated from the interviews. The research team was trained in the use of NVivo software to facilitate the coding process. A deductive coding scheme was initially used based on existing concepts in the literature regarding challenges to optimal SCD care, with new codes added as the thematic content analyses progressed. The initial coding, pattern coding, and use of displays to examine the relationships between different categories were conducted simultaneously.^27,28 Using the constant comparative method, new concepts from participants with SCD and providers could be incorporated into subsequent interviews with other participants. For this study, the only additional concepts added were in relation to participant recruitment and retention in the SCDIC Registry. Research team members coded transcripts separately and came together weekly, constantly comparing codes and developing the consensus coding scheme. Where differences between coders existed, code meanings were discussed and clarified until consensus was reached.²⁹

Quantitative data were analyzed using SPSS (v. 25, Chicago, IL). Descriptive statistics (means, standard deviations, frequencies, percentages) were used to summarize demographics (eg, age, gender, and race), economic status, and type of SCD. No systematic differences were detected from cases with missing values. Scale reliabilities (ie, Cronbach α) were evaluated for self-report measures.

Measurement

Adolescents and adults with SCD completed items from the PhenX Toolkit (consensus measures for Phenotypes and eXposures), assessing sociodemographics (age, sex, race, ethnicity, educational attainment, occupation, marital status, annual income, insurance), and clinical characteristics (sickle cell diagnosis and emergency department [ED] and hospital utilization for pain).³⁰

Pain Interference Short Form (Patient-Reported Outcomes Measurement Information System [PROMIS]). The Pain Interference Form consists of 8 items that assess the degree to which pain interfered with day-to-day activities in the previous 7 days at home, including impacts on social, cognitive, emotional, and physical functioning; household chores and recreational activities; sleep; and enjoyment in life. Reliability and validity of the PROMIS Pain Interference Scale has been demonstrated, with strong negative correlations with Physical Function Scales (r = 0.717, P < 0.01), indicating that higher scores are associated with lower function (β = 0.707, P < 0.001).³¹ The Cronbach α estimate for the other items on the pain interference scale was 0.99. Validity analysis indicated strong correlations with pain-related domains: BPI Interference Subscale (rho = 0.90), SF-36 Bodily Pain Subscale (rho = –0.84), and 0–10 Numerical Rating of Pain Intensity (rho = 0.48).³²

Adult Sickle Cell Quality of Life Measurement Information System (ASCQ-Me) Quality of Care (QOC). ASCQ-Me QOC consists of 27 items that measure the quality of care that adults with SCD have received from health care providers.³³ There are 3 composites: provider communication (quality of patient and provider communication), ED care (quality of care in the ED), and access (to routine and emergency care). Internal consistency reliability for all 3 composites is greater than 0.70. Strong correlations of the provider communication composite with overall ratings of routine care (r = 0.65) and overall provider ratings (r = 0.83) provided evidence of construct validity. Similarly, the ED care composite was strongly correlated with overall ratings of QOC in the ED, and the access composite was highly correlated with overall evaluations of ED care (r = 0.70). Access, provider interaction, and ED care composites were reliable (Cronbach α, 0.70–0.83) and correlated with ratings of global care (r = 0.32–0.83), further indicating construct validity.³³

Sickle Cell Self-Efficacy Scale (SCSES). The SCSES is a 9-item, self-administered questionnaire measuring perceptions of the ability to manage day-to-day issues resulting from SCD. SCSES items are scored on a 5-point scale ranging from Not sure at all (1) to Very sure (5). Individual item responses are summed to give an overall score, with higher scores indicating greater self-efficacy. The SCSES has acceptable reliability (r = 0.45, P < 0.001) and validity (α = 0.89).^34,35

Sickle Cell Disease Barriers Checklist. This checklist consists of 53 items organized into 8 categories: insurance, transportation, accommodations and accessibility, provider knowledge and attitudes, social support, individual barriers such as forgetting or difficulties understanding instructions, emotional barriers (fear, anger), and disease-related barriers. Participants check applicable barriers, with a total score range of 0 to 53 and higher scores indicating more barriers to care. The SCD Barriers Checklist has demonstrated face validity and test-retest reliability (Pearson r = 0.74, P < 0.05).⁵

ED Provider Checklist. The ED provider survey is a checklist of 14 statements pertaining to issues regarding patient care, with which the provider rates level of agreement. Items representing the attitudes and beliefs of providers towards patients with SCD are rated on a Likert-type scale, with level of agreement indicated as 1 (strongly disagree) to 6 (strongly agree). The positive attitudes subscale consists of 4 items (Cronbach α= 0.85), and the negative attitudes subscale consists of 6 items (Cronbach α = 0.89). The Red-Flag Behaviors subscale includes 4 items that indicate behavior concerns about drug-seeking, such as requesting specific narcotics and changing behavior when the provider walks in.^8,36,37

Sickle cell and primary care providers also completed a survey consisting of sets of items compiled from existing provider surveys; this survey consisted of a list of 16 barriers to using opioids, which the providers rated on a 5-point Likert-type scale (1, not a barrier; 5, complete barrier).^13,16,38 Providers indicated their level of experience with caring for patients with SCD; care provided, such as routine health screenings; and comfort level with providing preventive care, managing comorbidities, and managing acute and chronic pain. Providers were asked what potential facilitators might improve care for patients with SCD, including higher reimbursement, case management services, access to pain management specialists, and access to clinical decision-support tools. Providers responded to specific questions about management with hydroxyurea (eg, criteria for, barriers to, and comfort level with prescribing).³⁹ The surveys are included in the Appendix.

Triangulation

Data from the interviews and surveys were triangulated to enhance understanding of results generated from the different data sources.⁴⁰ Convergence of findings, different facets of the same phenomenon, or new perspectives were examined.

 

 

Results

Qualitative Data

Adolescents and adults with SCD (n = 55) and health care providers and community stakeholders (n = 56) participated in group or individual interviews to help us gain an in-depth understanding of the needs and barriers related to SCD care in our 5-county region. Participants with SCD described their experiences, which included stigma, racism, labeling, and, consequently, stress. They also identified barriers such as lack of transportation, challenges with insurance, and lack of access to providers who were competent with pain management. They reported that having SCD in a health care system that was unable to meet their needs was burdensome.

Barriers to Care and Treatments. Adolescents and adults indicated that SCD and its sequelae posed significant barriers to health care. Feelings of tiredness and pain make it more difficult for them to seek care. The emotional burden of SCD (fear and anger) was a frequently cited barrier, which was fueled by previous negative encounters with the health care system. All adolescents and adults with SCD reported that they knew of stigma in relation to seeking pain management that was pervasive and long-standing, and the majority reported they had directly experienced stigma. They reported that being labeled as “drug-seekers” was typical when in the ED for pain management. Participants articulated unconscious bias or overt racism among providers: “people with sickle cell are Black ... and Black pain is never as valuable as White pain” (25-year-old male). Respondents with SCD described challenges to the credibility of their pain reports in the ED. They reported that ED providers expressed doubts regarding the existence and/or severity of their pain, consequently creating a feeling of disrespect for patients seeking pain relief. The issue of stigma was mentioned by only 2 of 56 providers during their interviews.

Lack of Access to Knowledgeable, Compassionate Providers. Lack of access to knowledgeable care providers was another prevalent theme expressed by adolescents and adults with SCD. Frustration occurred when providers did not have knowledge of SCD and its management, particularly pain assessment. Adolescents and adults with SCD noted the lack of compassion among providers: “I’ve been kicked out of the hospital because they felt like okay, well we gave you enough medication, you should be all right” (29-year-old female). Providers specifically mentioned lack of compassion and knowledge as barriers to SCD care much less often during their interviews compared with the adolescents and adults with SCD.

Health Care System Barriers. Patient participants often expressed concerns about concrete and structural aspects of care. Getting to their appointments was a challenge for half of the interviewees, as they either did not have access to a vehicle or could not afford to travel the needed distance to obtain quality care. Even when hospitals were accessible by public transportation, those with excruciating pain understandably preferred a more comfortable and private way to travel: “I would like to change that, something that will be much easier, convenient for sickle cell patients that do suffer with pain, that they don’t have to travel always to see the doctor” (30-year-old male).

Insurance and other financial barriers also played an important role in influencing decisions to seek health care services. Medical expenses were not covered, or co-pays were too high. The Medicaid managed care system could prevent access to knowledgeable providers who were not within network. Such a lack of access discouraged some adolescents and adults with SCD from seeking acute and preventive care.

Transition From Pediatric to Adult Care. Interviewees with SCD expressed distress about the gap between pediatric and adult care. They described how they had a long-standing relationship with their medical providers, who were familiar with their medical background and history from childhood. Adolescent interviewees reported an understanding of their own pain management as well as adherence to and satisfaction with their individualized pain plans. However, adults noted that satisfaction plummeted with increasing age due to the limited number of experienced adult SCD providers, which was compounded by negative experiences (stigma, racism, drug-seeking label).

One interviewee emphasized the difficulty of finding knowledgeable providers after transition: “When you’re a pediatric sickle cell [patient], you have the doctors there every step of the way, but not with adult sickle cell… I know when I first transitioned I never felt more alone in my life… you look at that ER doctor kind of with the same mindset as you would your hematologist who just hand walked you through everything. And adult care providers were a lot more blunt and cold and they’re like… ‘I don’t know; I’m not really educated in sickle cell.’” A sickle cell provider shared his insight about the problem of transitioning: “I think it’s particularly challenging because we, as a community, don’t really set them up for success. It’s different from other chronic conditions [in that] it’s much harder to find an adult sickle cell provider. There’s not a lot of adult hematologists that will take care of our adult patients, and so I know statistically, there’s like a drop-down in the overall outcomes of our kids after they age out of our pediatric program.”

Self-Management, Supporting Hydroxyurea Use. Interview participants with SCD reported using a variety of methods to manage pain at home and chose to go to the ED only when the pain became intolerable. Patients and providers expressed awareness of different resources for managing pain at home, yet they also indicated that these resources have not been consolidated in an accessible way for patients and families. Some resources cited included heat therapy, acupuncture, meditation, medical marijuana, virtual reality devices, and pain medications other than opioids.

Patients and providers expressed the need for increasing awareness and education about hydroxyurea. Many interview participants with SCD were concerned about side effects, multiple visits with a provider during dose titration, and ongoing laboratory monitoring. They also expressed difficulties with scheduling multiple appointments, depending on access to transportation and limited provider clinic hours. They were aware of strategies for improving adherence with hydroxyurea, including setting phone alarms, educating family members about hydroxyurea, and eliciting family support, but expressed needing help to consistently implement these strategies.

Safe Opioid Prescribing. Adult care providers expressed concerns about safe opioid prescribing for patients with SCD. They were reluctant to prescribe opioid doses needed to adequately control SCD pain. Providers expressed uncertainty and fear or concern about medical/legal liability or about their judgment about what’s safe and not safe for patients with chronic use/very high doses of opioids. “I know we’re in like this opiate epidemic here in this country but I feel like these patients don’t really fit under that umbrella that the problem is coming from so [I am] just trying to learn more about how to take care of them.”

Care Coordination and Provider Communication. Adolescents and adults with SCD reported having positive experiences—good communication, established trust, and compassionate care—with their usual providers. However, they perceived that ED physicians and nurses did not really care about them. Both interviewees with SCD and providers recognized the importance of good communication in all settings as the key to overcoming barriers to receiving quality care. All agreed on the importance of using individual pain plans so that all providers, especially ED providers, can be more at ease with treating adolescents and adults with SCD.

Quantitative Data: Adolescents and Adults With SCD

Fifty-eight adolescents and adults with SCD (aged 15 to 48 years) completed the survey. Three additional individuals who did not complete the interview completed the survey. Reasons for not completing the interview included scheduling challenges (n = 2) or a sickle cell pain episode (n = 1). The average age of participants was 31 years ± 8.6, more than half (57%) were female, and the majority (93%) were African American (Table 1). Most (71%) had never been married. Half (50%) had some college or an associate degree, and 40% were employed and reported an annual household income of less than $30,000. Insurance coverage was predominantly Medi-Cal (Medicaid, 69%). The majority of participants resided in Alameda (34.5%) or Contra Costa (21%) counties. The majority of sickle cell care was received in Alameda County, whether outpatient (52%), inpatient (40%), or ED care (41%). The majority (71%) had a diagnosis of SCD hemoglobin SS.

Pain. More than one-third of individuals with SCD reported 1 or 2 ED visits for pain in the previous 6 months (34%), and more than 3 hospitalizations (36%) related to pain in the previous year (Table 2). The majority (85%) reported having severe pain at home in the previous 6 months that they did not seek health care for, consistent with their reports in the qualitative interviews. More than half (59%) reported 4 or more of these severe pain episodes that led to inability to perform daily activities for 1 week or more. While pain interference on the PROMIS Pain Interference Short Form on average (T-score, 59.6 ± 8.6) was similar to that of the general population (T-score, 50 ± 10), a higher proportion of patients with SCD reported pain interference compared with the general population. The mean self-efficacy (confidence in ability to manage complications of SCD) score on the SCSES of 30.0 ± 7.3 (range, 9–45) was similar to that of other adults with SCD (mean, 32.2 ± 7.0). Twenty-five percent of the present sample had a low self-efficacy score (< 25).

Barriers to Care and Treatments. Consistent with the qualitative data, SCD-related symptoms such as tiredness (64%) and pain (62%) were reported most often as barriers to care (Table 3). Emotions (> 25%) such as worry/fear, frustration/anger, and lack of confidence were other important barriers to care. Provider knowledge and attitudes were cited next most often, with 38% of the sample indicating “Providers accuse me of drug-seeking” and “It is hard for me to find a provider who has enough experiences with or knowledge about SCD.” Participants expressed that they were not believed when in pain and “I am treated differently from other patients.” Almost half of respondents cited “I am not seen quickly enough when I am in pain” as a barrier to their care.

Consistent with the qualitative data, transportation barriers (not having a vehicle, costs of transportation, public transit not easy to get to) were cited by 55% of participants. About half of participants reported that insurance was an important barrier, with high co-pays and medications and other services not covered. In addition, gathering approvals was a long and fragmented process, particularly for consultations among providers (hematology, primary care provider, pain specialist). Furthermore, insurance provided limited choices about location for services.

Participants reported social support system burnout (22%), help needed with daily activities (21%), and social isolation or generally not having enough support (33%) as ongoing barriers. Difficulties were encountered with self-management (eg, taking medications on time or making follow-up appointments, 19%), with 22% of participants finding the health care system confusing or hard to understand. Thirty percent reported “Places for me to go to learn how to stay well are not close by or easy to get to.” ”Worry about side effects” (33%) was a common barrier to hydroxyurea use. Participants described “forgetting to take the medicine,” “tried before but it did not work,” “heard scary things” about hydroxyurea, and “not interested in taking another medicine” as barriers.

Quality of Care. More than half (51%) of the 53 participants who had accessed health care in the previous year rated their overall health care as poor on the ASCQ-Me QOC measure. This was significantly higher compared to the reports from more than 47,000 adults with Medicaid in 2017 (16%),⁴¹ and to the 2008-2009 report from 556 adults with SCD from across the United States (37%, Figure 2).³³ The major contributor to these poor ratings for participants in our sample was low satisfaction with ED care.

Sixty percent of the 42 participants who had accessed ED care in the past year indicated “never” or “sometimes” to the question “When you went to the ED for care, how often did you get it as soon as you wanted?” compared with only 16% of the 2017 adult Medicaid population responding (n = 25,789) (Figure 3). Forty-seven percent of those with an ED visit indicated that, in the previous 12 months, they had been made to wait “more than 2 hours before receiving treatment for acute pain in the ED.” However, in the previous 12 months, 39% reported that their wait time in the ED had been only “between five minutes and one hour.”

On the ASCQ-Me QOC Access to Care composite measure, 33% of 42 participants responding reported they were seen at a routine appointment as soon as they would have liked. This is significantly lower compared to 56% of the adult Medicaid population responding to the same question. Reports of provider communication (Provider Communication composite) for adolescents and adults with SCD were comparable to reports of adults with SCD from the ASCQ-Me field test,³³ but adults with Medicaid reported higher ratings of quality communication behaviors (Figure 4).^33,41 Nearly 60% of both groups with SCD reported that providers “always” performed quality communication behaviors—listened carefully, spent enough time, treated them with respect, and explained things well—compared with more than 70% of adults with Medicaid.

Participants from all counties reported the same number of barriers to care on average (3.3 ± 2.1). Adolescents and adults who reported more barriers to care also reported lower satisfaction with care (r = –0.47, P < 0.01) and less confidence in their ability to manage their SCD (self-efficacy, r = – 0.36, P < 0.05). Female participants reported more barriers to care on average compared with male participants (2.6 ± 2.4 vs 1.4 ± 2.0, P = 0.05). Participants with higher self-efficacy reported lower pain ratings (r = –0.47, P < 0.001).

Quantitative Data: Health Care Providers

Providers (n = 56) and community stakeholders (2 leaders of community-based organizations and 3 health care administrators) were interviewed, with 29 also completing the survey. The reason for not completing (n = 22) was not having the time once the interview was complete. A link to the survey was sent to any provider not completing at the time of the interview, with 2 follow-up reminders. The majority of providers were between the ages of 31 and 50 years (46.4%), female (71.4%), and white (66.1%) (Table 4). None were of Hispanic, Latinx, or Spanish origin. Thirty-six were physicians (64.3%), and 16 were allied health professionals (28.6%). Of the 56 providers, 32 indicated they had expertise caring for patients with SCD (57.1%), 14 were ED providers (25%), and 5 were primary care providers. Most of the providers practiced in an urban setting (91.1%).

Barriers to Care: ED Provider Perspectives. Nine of 14 ED providers interviewed completed the survey on their perspectives regarding barriers to care in the ED, difficulty with follow-ups, ED training resources, and pain control for patients with SCD. ED providers (n = 8) indicated that “provider attitudes” were a barrier to care delivery in the ED for patients with SCD. Some providers (n = 7) indicated that “implicit bias,” “opioid epidemic,” “concern about addiction,” and “patient behavior” were barriers. Respondents indicated that “overcrowding” (n = 6) and “lack of care pathway/protocol” (n = 5) were barriers. When asked to express their level of agreement with statements about SCD care in the ED, respondents disagreed/strongly disagreed (n = 5) that they were “able to make a follow-up appointment” with a sickle cell specialist or primary care provider upon discharge from the ED, and others disagreed/strongly disagreed (n = 4) that they were able to make a “referral to a case management program.”

ED training and resources. Providers agreed/strongly agreed (n = 8) that they had the knowledge and training to care for patients with SCD, that they had access to needed medications, and that they had access to knowledgeable nursing staff with expertise in SCD care. All 9 ED providers indicated that they had sufficient physician/provider staffing to provide good pain management to persons with SCD in the ED.

Pain control in the ED. Seven ED providers indicated that their ED used individualized dosing protocols to treat sickle cell pain, and 5 respondents indicated their ED had a protocol for treating sickle cell pain. Surprisingly, only 3 indicated that they were aware of the NHLBI recommendations for the treatment of vaso-occlusive pain.

Barriers to Care: Primary Care Provider Perspectives. Twenty providers completed the SCD provider section of the survey, including 17 multidisciplinary SCD providers from 4 sickle cell special care centers and 3 community primary care providers. Of the 20, 12 were primary care providers for patients with SCD (Table 4).

Patient needs. Six primary care providers indicated that the medical needs of patients with SCD were being met, but none indicated that the behavioral health or mental health needs were being met.

Managing SCD comorbidities. Five primary care providers indicated they were very comfortable providing preventive ambulatory care to patients with SCD. Six indicated they were very comfortable managing acute pain episodes, but none were very comfortable managing comorbidities such as pulmonary hypertension, diabetes, or chronic pain.

Barriers to opioid use. Only 3 of 12 providers reviewing a list of 15 potential barriers to the use of opioids for SCD pain management indicated a perceived lack of efficacy of opioids, development of tolerance and dependence, and concerns about community perceptions as barriers. Two providers selected potential for diversion as a moderate barrier to opioid use.

Barriers to hydroxyurea use. Eight of 12 providers indicated that the common reasons that patients/families refuse hydroxyurea were “worry about side effects”; 7 chose “don’t want to take another medicine,” and 6 chose “worry about carcinogenic potential.” Others (n = 10) indicated that “patient/family adherence with hydroxyurea” and “patient/family adherence with required blood tests” were important barriers to hydroxyurea use. Eight of the 12 providers indicated that they were comfortable with managing hydroxyurea in patients with SCD.

Care redesign. Twenty SCD and primary care providers completed the Care Redesign section of the survey. Respondents (n = 11) indicated that they would see more patients with SCD if they had accessible case management services available without charge or if patient access to transportation to clinic was also available. Ten indicated that they would see more patients with SCD if they had an accessible community health worker (who understands patient’s/family’s social situation) and access to a pain management specialist on call to answer questions and who would manage chronic pain. All (n = 20) were willing to see more patients with SCD in their practices. Most reported that a clinical decision-support tool for SCD treatment (n = 13) and avoidance of complications (n = 12) would be useful.

Discussion

We evaluated access and barriers to care, quality of care, care coordination, and provider communication from the perspectives of adolescents and adults with SCD, their care providers, and community stakeholders, within the Solberg conceptual model for quality improvement. We found that barriers within the care process content domain (context and systems) were most salient for this population of adolescents and adults with SCD, with lack of provider knowledge and poor attitudes toward adolescents and adults with SCD, particularly in the ED, cited consistently by participant groups. Stigmatization and lack of provider compassion that affected the quality of care were particularly problematic. These findings are consistent with previous reports.^42,43 Adult health care (particularly ED) provider biases and negative attitudes have been recognized as major barriers to optimal pain management in SCD.^8,11,44,45 Interestingly, ED providers in our needs assessment indicated that they felt they had the training and resources to manage patients with SCD. However, only a few actually reported knowing about the NHLBI recommendations for the treatment of vaso-occlusive pain.

Within the care process content domain, we also found that SCD-related complications and associated emotions (fear, worry, anxiety), compounded by lack of access to knowledgeable and compassionate providers, pose a significant burden. Negative encounters with the health care system contributed to a striking 84% of patient participants choosing to manage severe pain at home, with pain seriously interfering with their ability to function on a daily basis. ED providers agreed that provider attitudes and implicit bias pose important barriers to care for adolescents and adults with SCD. Adolescents and adults with SCD wanted, and understood the need, to enhance self-management skills. Both they and their providers agreed that barriers to hydroxyurea uptake included worries about potential side effects, challenges with adherence to repeated laboratory testing, and support with remembering to take the medicine. However, providers uniformly expressed that access to behavioral and mental health services were, if not nonexistent, impossible to access.

Participants with SCD and their providers reported infrastructural challenges (change process capability), as manifested in limitations with accessing acute and preventive care due to transportation- and insurance- related issues. There were health system barriers that were particularly encountered during the transition from pediatric to adult care. These findings are consistent with previous reports that have found fewer interdisciplinary services available in the adult care settings compared with pediatrics.^46,47 Furthermore, adult care providers were less willing to accept adults with SCD because of the complexity of their management, for which the providers did not have the necessary expertise.^3,48-50 In addition, both adolescents and adults with SCD and primary care providers highlighted the inadequacies of the current system in addressing the chronic pain needs of this population. Linking back to the Solberg conceptual framework, our needs assessment results confirm the important role of establishing SCD care as a priority within a health care system—this requires leadership and vision. The vision and priorities must be implemented by effective health care teams. Multilevel approaches or interventions, when implemented, will lead to the desired outcomes.

Findings from our needs assessment within our 5-county region mirror needs assessment results from the broader consortium.⁵¹ The SCDIC has prioritized developing an intervention that addresses the challenges identified within the care process domain by directly enhancing provider access to patient individualized care plans in the electronic health record in the ED. Importantly, ED providers will be asked to view a short video that directly challenges bias and stigma in the ED. Previous studies have indeed found that attitudes can be improved by providers viewing short video segments of adults with SCD discussing their experiences.^36,52 This ED protocol will be one of the interventions that we will roll out in Northern California, given the significance of negative ED encounters reported by needs assessment participants. An additional feature of the intervention is a script for adults with SCD that guides them through introducing their individualized pain plan to their ED providers, thereby enhancing their self-efficacy in a situation that has been so overwhelmingly challenging.

We will implement a second SCDIC intervention that utilizes a mobile app to support self-management on the part of the patient, by supporting motivation and adherence with hydroxyurea.⁵³ A companion app supports hydroxyurea guideline adherence on the part of the provider, in keeping with one of our findings that providers are in need of decision-support tools. Elements of the intervention also align with our findings related to the importance of a support system in managing SCD, in that participants will identify a supportive partner who will play a specific role in supporting their adherence with hydroxyurea.

On our local level, we have, by necessity, partnered with leaders and community stakeholders throughout the region to ensure that these interventions to improve SCD care are prioritized. Grant funds provide initial resources for the SCDIC interventions, but our partnering health care administrators and medical directors must ensure that participating ED and hematology providers are free from competing priorities in order to implement the changes. We have partnered with a SCD community-based organization that is designing additional educational presentations for local emergency medicine providers, with the goal to bring to life very personal stories of bias and stigma within the EDs that directly contribute to decisions to avoid ED care despite severe symptoms.

Although we attempted to obtain samples of adolescents and adults with SCD and their providers that were representative across the 5-county region, the larger proportion of respondents were from 1 county. We did not assess concerns of age- and race-matched adults in our catchment area, so we cannot definitively say that our findings are unique to SCD. However, our results are consistent with findings from the national sample of adults with SCD who participated in the ASCQ-Me field test, and with results from the SCDIC needs assessment.^33,51 Interviews and surveys are subject to self-report bias and, therefore, may or may not reflect the actual behaviors or thoughts of participants. Confidence is increased in our results given the triangulation of expressed concerns across participant groups and across data collection strategies. The majority of adolescents and adults with SCD (95%) completed both the interview and survey, while 64% of ED providers interviewed completed the survey, compared with 54% of SCD specialists and primary care providers. These response rates are more than acceptable within the realm of survey response rates.^54,55

Although we encourage examining issues with care delivery within the conceptual framework for quality improvement presented, we recognize that grant funding allowed us to conduct an in-depth needs assessment that might not be feasible in other settings. Still, we would like readers to understand the importance of gathering data for improvement in a systematic manner across a range of participant groups, to ultimately inform the development of interventions and provide for evaluation of outcomes as a result of the interventions. This is particularly important for a disease, such as SCD, that is both medically and sociopolitically complex.

Conclusion

Our needs assessment brought into focus the multiple factors contributing to the disparities in health care experienced by adolescents and adults with SCD on our local level, and within the context of inequities in health resources and outcomes on the national level. We propose solutions that include specific interventions developed by a consortium of SCD and implementation science experts. We utilize a quality improvement framework to ensure that the elements of the interventions also address the barriers identified by our local providers and patients that are unique to our community. The pervasive challenges in SCD care, coupled with its medical complexities, may seem insurmountable, but our survey and qualitative results provide us with a road map for the way forward.

Acknowledgments: The authors thank the adolescents and adults with sickle cell disease, the providers, and the community stakeholders who completed the interviews and surveys. The authors also acknowledge the SCCCI co-investigators for their contributions to this project, including Michael Bell, MD, Ward Hagar, MD, Christine Hoehner, FNP, Kimberly Major, MSW, Anne Marsh, MD, Lynne Neumayr, MD, and Ted Wun, MD. We also thank Kamilah Bailey, Jameelah Hodge, Jennifer Kim, Michael Rowland, Adria Stauber, Amber Fearon, and Shanda Robertson, and the Sickle Cell Data Collection Program for their contributions.

Corresponding author: Marsha J. Treadwell, PhD, University of California San Francisco Benioff Children’s Hospital Oakland, 747 52nd St., Oakland, CA 94609; marsha.treadwell@ucsf.edu.

Financial disclosures: None.

Funding/support: This work was supported by grant # 1U01HL134007 from the National Heart, Lung, and Blood Institute to the University of California San Francisco Benioff Children’s Hospital Oakland.

From the University of California San Francisco (Dr. Treadwell, Dr. Hessler, Yumei Chen, Swapandeep Mushiana, Dr. Potter, and Dr. Vichinsky), the University of California Los Angeles (Dr. Jacob), and the University of California Berkeley (Alex Chen).

Abstract

• Objective: Adolescents and adults with sickle cell disease (SCD) face pervasive disparities in health resources and outcomes. We explored barriers to and facilitators of care to identify opportunities to support implementation of evidence-based interventions aimed at improving care quality for patients with SCD.
• Methods: We engaged a representative sample of adolescents and adults with SCD (n = 58), health care providers (n = 51), and community stakeholders (health care administrators and community-based organization leads (n = 5) in Northern California in a community-based needs assessment. We conducted group interviews separately with participant groups to obtain in-depth perspectives. Adolescents and adults with SCD completed validated measures of pain interference, quality of care, self-efficacy, and barriers to care. Providers and community stakeholders completed surveys about barriers to SCD care.
• Results: We triangulated qualitative and quantitative data and found that participants with SCD (mean age, 31 ± 8.6 years), providers, and community stakeholders emphasized the social and emotional burden of SCD as barriers. Concrete barriers agreed upon included insurance and lack of resources for addressing pain impact. Adolescents and adults with SCD identified provider issues (lack of knowledge, implicit bias), transportation, and limited social support as barriers. Negative encounters with the health care system contributed to 84% of adolescents and adults with SCD reporting they chose to manage severe pain at home. Providers focused on structural barriers: lack of access to care guidelines, comfort level with and knowledge of SCD management, and poor care coordination.
• Conclusion: Strategies for improving access to compassionate, evidence-based quality care, as well as strategies for minimizing the burden of having SCD, are warranted for this medically complex population.

Keywords: barriers to care; quality of care; care access; care coordination.

Sickle cell disease (SCD), an inherited chronic medical condition, affects about 100,000 individuals in the United States, a population that is predominantly African American.¹ These individuals experience multiple serious and life-threatening complications, most frequently recurrent vaso-occlusive pain episodes,² and they require interactions with multidisciplinary specialists from childhood. Because of advances in treatments, the majority are reaching adulthood; however, there is a dearth of adult health care providers with the training and expertise to manage their complex medical needs.³ Other concrete barriers to adequate SCD care include insurance and distance to comprehensive SCD centers.^4,5

Social, behavioral, and emotional factors may also contribute to challenges with SCD management. SCD may limit daily functional abilities and lead to diminished overall quality of life.^6,7 Some adolescents and adults may require high doses of opioids, which contributes to health care providers’ perceptions that there is a high prevalence of drug addiction in the population.^8,9 These providers express negative attitudes towards adults with SCD, and, consequently, delay medication administration when it is acutely needed and provide otherwise suboptimal treatment.^8,10,11 Adult care providers may also be uncomfortable with prescribing and managing disease-modifying therapies (blood transfusion, hydroxyurea) that have established efficacy.^12-17

As 1 of 8 programs funded by the National Heart, Lung, and Blood Institute’s (NHLBI) Sickle Cell Disease Implementation Consortium (SCDIC), we are using implementation science to reduce barriers to care and improve quality of care and health care outcomes in SCD.^18,19 Given that adolescents and adults with SCD experience high mortality, severe pain, and progressive decline in their ability to function day to day, and also face lack of access to knowledgeable, compassionate providers in primary and emergency settings, the SCDIC focuses on individuals aged 15 to 45 years.^6,8,9,11,12

Our regional SCDIC program, the Sickle Cell Care Coordination Initiative (SCCCI), brings together researchers, clinicians, adolescents, and adults with SCD and their families, dedicated community members, policy makers, and administrators to identify and address barriers to health care within 5 counties in Northern California. One of our first steps was to conduct a community-based needs assessment, designed to inform implementation of evidence-based interventions, accounting for unique contextual factors in our region.

Conceptual Framework for Improving Medical Practice

Our needs assessment is guided by Solberg’s Conceptual Framework for Improving Medical Practice (Figure 1).²⁰ Consistent with the overarching principles of the SCDIC, this conceptual framework focuses on the inadequate implementation of evidence-based guidelines, and on the need to first understand multifactorial facilitators and barriers to guideline implementation in order to effect change. The framework identifies 3 main elements that must be present to ensure improvements in quality-of-care processes and patient outcomes: priority, change process capability, and care process content. Priority refers to ample resource allocation for the specific change, as well as freedom from competing priorities for those implementing the change. Change process capability includes strong, effective leadership, adequate infrastructure for managing change (including resources and time), change management skills at all levels, and an established clinical information system. Care process content refers to context and systems-level changes, such as delivery system redesign as needed, support for self-management to lessen the impact of the disease, and decision support.^21-23

The purpose of our community-based needs assessment was to evaluate barriers to care and quality of care in SCD, within Solberg’s conceptual model for improving medical practice. The specific aims were to evaluate access and barriers to care (eg, lack of provider expertise and training, health care system barriers such as poor care coordination and provider communication); evaluate quality of care; and assess patient needs related to pain, pain interference, self-efficacy, and self-management for adolescents and adults with SCD. We gathered the perspectives of a representative community of adolescents and adults with SCD, their providers, and community stakeholders in order to examine barriers, quality of life and care, and patient experiences in our region.

Methods

Design

In this cross-sectional study, adolescents and adults with SCD, their providers, and community stakeholders participated in group or individual qualitative interviews and completed surveys between October 2017 and March 2018.

Setting and Sample

Recruitment flyers were posted on a regional SCD-focused website, and clinical providers or a study coordinator introduced information about the needs assessment to potential participants with SCD during clinic visits at the participating centers. Participants with SCD were eligible if they had any diagnosis of SCD, were aged 15 to 48 years, and received health services within 5 Northern California counties (Alameda, Contra Costa, Sacramento, San Francisco, and Solano). They were excluded if they did not have a SCD diagnosis or had not received health services within the catchment area. As the project proceeded, participants were asked to refer other adolescents and adults with SCD for the interviews and surveys (snowball sampling). Our goal was to recruit 50 adolescents and adults with SCD into the study, aiming for 10 representatives from each county.

Providers and community stakeholders were recruited via emails, letters and informational flyers. We engaged our partner, the Sickle Cell Data Collection Program,² to generate a list of providers and institutions that had seen patients with SCD in primary, emergency, or inpatient settings in the region. We contacted these institutions to describe the SCCCI and invite participation in the needs assessment. We also invited community-based organization leads and health care administrators who worked with SCD to participate. Providers accessed confidential surveys via a secure link on the study website or completed paper versions. Common data collected across providers included demographics and descriptions of practice settings.

Participants were eligible to be part of the study if they were health care providers (physicians and nurses) representing hematology, primary care, family medicine, internal medicine, or emergency medicine; ancillary staff (social work, psychology, child life); or leaders or administrators of clinical or sickle cell community-based organizations in Northern California (recruitment goal of n = 50). Providers were excluded if they practiced in specialties other than those noted or did not practice within the region.

Data Collection Procedures

After providing assent/consent, participating adolescents and adults with SCD took part in individual and group interviews and completed survey questionnaires. All procedures were conducted in a private space in the sickle cell center or community. Adolescents and adults with SCD completed the survey questionnaire on a tablet, with responses recorded directly in a REDCap (Research Electronic Data Capture) database,²⁴ or on a paper version. Interviews lasted 60 (individual) to 90 (group) minutes, while survey completion time was 20 to 25 minutes. Each participant received a gift card upon completion as an expression of appreciation. All procedures were approved by the institutional review boards of the participating health care facilities.

Group and Individual Interviews

Participants with SCD and providers were invited to participate in a semi-structured qualitative interview prior to being presented with the surveys. Adolescents and adults with SCD were interviewed about barriers to care, quality of care, and pain-related experiences. Providers were asked about barriers to care and treatments. Interview guides were modified for community-based organization leaders and health care administrators who did not provide clinical services. Interview guides can be found in the Appendix. Interviews were conducted by research coordinators trained in qualitative research methods by the first author (MT). As appropriate with semi-structured interviews, the interviewers could word questions spontaneously, change the order of questions for ease of flow of conversation, and inform simultaneous coding of interviews with new themes as those might arise, as long as they touched on all topics within the interview guide.²⁵ The interview guides were written, per qualitative research standards, based on the aims and purpose of the research,²⁶ and were informed by existing literature on access and barriers to care in SCD, quality of care, and the needs of individuals with SCD, including in relation to impact of the disease, self-efficacy, and self-management.

Interviewees participated in either individual or group interviews, but not both. The decision for which type of interview an individual participated in was based on 2 factors: if there were not comparable participants for group interviews (eg, health care administrator and community-based organization lead), these interviews were done individually; and given that we were drawing participants from a 5-county area in Northern California, scheduling was challenging for individuals with SCD with regard to aligning schedules and traveling to a central location where the group interviews were conducted. Provider group interviews were easier to arrange because we could schedule them at the same time as regularly scheduled meetings at the participants’ health care institutions.

Interview Data Gathering and Analysis

Digital recordings of the interviews were cleaned of any participant identifying data and sent for transcription to an outside service. Transcripts were reviewed for completeness and imported into NVivo (www.qsrinternational.com), a qualitative data management program.

A thematic content analysis and deductive and inductive approaches were used to analyze the verbatim transcripts generated from the interviews. The research team was trained in the use of NVivo software to facilitate the coding process. A deductive coding scheme was initially used based on existing concepts in the literature regarding challenges to optimal SCD care, with new codes added as the thematic content analyses progressed. The initial coding, pattern coding, and use of displays to examine the relationships between different categories were conducted simultaneously.^27,28 Using the constant comparative method, new concepts from participants with SCD and providers could be incorporated into subsequent interviews with other participants. For this study, the only additional concepts added were in relation to participant recruitment and retention in the SCDIC Registry. Research team members coded transcripts separately and came together weekly, constantly comparing codes and developing the consensus coding scheme. Where differences between coders existed, code meanings were discussed and clarified until consensus was reached.²⁹

Quantitative data were analyzed using SPSS (v. 25, Chicago, IL). Descriptive statistics (means, standard deviations, frequencies, percentages) were used to summarize demographics (eg, age, gender, and race), economic status, and type of SCD. No systematic differences were detected from cases with missing values. Scale reliabilities (ie, Cronbach α) were evaluated for self-report measures.

Measurement

Adolescents and adults with SCD completed items from the PhenX Toolkit (consensus measures for Phenotypes and eXposures), assessing sociodemographics (age, sex, race, ethnicity, educational attainment, occupation, marital status, annual income, insurance), and clinical characteristics (sickle cell diagnosis and emergency department [ED] and hospital utilization for pain).³⁰

Pain Interference Short Form (Patient-Reported Outcomes Measurement Information System [PROMIS]). The Pain Interference Form consists of 8 items that assess the degree to which pain interfered with day-to-day activities in the previous 7 days at home, including impacts on social, cognitive, emotional, and physical functioning; household chores and recreational activities; sleep; and enjoyment in life. Reliability and validity of the PROMIS Pain Interference Scale has been demonstrated, with strong negative correlations with Physical Function Scales (r = 0.717, P < 0.01), indicating that higher scores are associated with lower function (β = 0.707, P < 0.001).³¹ The Cronbach α estimate for the other items on the pain interference scale was 0.99. Validity analysis indicated strong correlations with pain-related domains: BPI Interference Subscale (rho = 0.90), SF-36 Bodily Pain Subscale (rho = –0.84), and 0–10 Numerical Rating of Pain Intensity (rho = 0.48).³²

Adult Sickle Cell Quality of Life Measurement Information System (ASCQ-Me) Quality of Care (QOC). ASCQ-Me QOC consists of 27 items that measure the quality of care that adults with SCD have received from health care providers.³³ There are 3 composites: provider communication (quality of patient and provider communication), ED care (quality of care in the ED), and access (to routine and emergency care). Internal consistency reliability for all 3 composites is greater than 0.70. Strong correlations of the provider communication composite with overall ratings of routine care (r = 0.65) and overall provider ratings (r = 0.83) provided evidence of construct validity. Similarly, the ED care composite was strongly correlated with overall ratings of QOC in the ED, and the access composite was highly correlated with overall evaluations of ED care (r = 0.70). Access, provider interaction, and ED care composites were reliable (Cronbach α, 0.70–0.83) and correlated with ratings of global care (r = 0.32–0.83), further indicating construct validity.³³

Sickle Cell Self-Efficacy Scale (SCSES). The SCSES is a 9-item, self-administered questionnaire measuring perceptions of the ability to manage day-to-day issues resulting from SCD. SCSES items are scored on a 5-point scale ranging from Not sure at all (1) to Very sure (5). Individual item responses are summed to give an overall score, with higher scores indicating greater self-efficacy. The SCSES has acceptable reliability (r = 0.45, P < 0.001) and validity (α = 0.89).^34,35

Sickle Cell Disease Barriers Checklist. This checklist consists of 53 items organized into 8 categories: insurance, transportation, accommodations and accessibility, provider knowledge and attitudes, social support, individual barriers such as forgetting or difficulties understanding instructions, emotional barriers (fear, anger), and disease-related barriers. Participants check applicable barriers, with a total score range of 0 to 53 and higher scores indicating more barriers to care. The SCD Barriers Checklist has demonstrated face validity and test-retest reliability (Pearson r = 0.74, P < 0.05).⁵

ED Provider Checklist. The ED provider survey is a checklist of 14 statements pertaining to issues regarding patient care, with which the provider rates level of agreement. Items representing the attitudes and beliefs of providers towards patients with SCD are rated on a Likert-type scale, with level of agreement indicated as 1 (strongly disagree) to 6 (strongly agree). The positive attitudes subscale consists of 4 items (Cronbach α= 0.85), and the negative attitudes subscale consists of 6 items (Cronbach α = 0.89). The Red-Flag Behaviors subscale includes 4 items that indicate behavior concerns about drug-seeking, such as requesting specific narcotics and changing behavior when the provider walks in.^8,36,37

Sickle cell and primary care providers also completed a survey consisting of sets of items compiled from existing provider surveys; this survey consisted of a list of 16 barriers to using opioids, which the providers rated on a 5-point Likert-type scale (1, not a barrier; 5, complete barrier).^13,16,38 Providers indicated their level of experience with caring for patients with SCD; care provided, such as routine health screenings; and comfort level with providing preventive care, managing comorbidities, and managing acute and chronic pain. Providers were asked what potential facilitators might improve care for patients with SCD, including higher reimbursement, case management services, access to pain management specialists, and access to clinical decision-support tools. Providers responded to specific questions about management with hydroxyurea (eg, criteria for, barriers to, and comfort level with prescribing).³⁹ The surveys are included in the Appendix.

Triangulation

Data from the interviews and surveys were triangulated to enhance understanding of results generated from the different data sources.⁴⁰ Convergence of findings, different facets of the same phenomenon, or new perspectives were examined.

 

 

Results

Qualitative Data

Adolescents and adults with SCD (n = 55) and health care providers and community stakeholders (n = 56) participated in group or individual interviews to help us gain an in-depth understanding of the needs and barriers related to SCD care in our 5-county region. Participants with SCD described their experiences, which included stigma, racism, labeling, and, consequently, stress. They also identified barriers such as lack of transportation, challenges with insurance, and lack of access to providers who were competent with pain management. They reported that having SCD in a health care system that was unable to meet their needs was burdensome.

Barriers to Care and Treatments. Adolescents and adults indicated that SCD and its sequelae posed significant barriers to health care. Feelings of tiredness and pain make it more difficult for them to seek care. The emotional burden of SCD (fear and anger) was a frequently cited barrier, which was fueled by previous negative encounters with the health care system. All adolescents and adults with SCD reported that they knew of stigma in relation to seeking pain management that was pervasive and long-standing, and the majority reported they had directly experienced stigma. They reported that being labeled as “drug-seekers” was typical when in the ED for pain management. Participants articulated unconscious bias or overt racism among providers: “people with sickle cell are Black ... and Black pain is never as valuable as White pain” (25-year-old male). Respondents with SCD described challenges to the credibility of their pain reports in the ED. They reported that ED providers expressed doubts regarding the existence and/or severity of their pain, consequently creating a feeling of disrespect for patients seeking pain relief. The issue of stigma was mentioned by only 2 of 56 providers during their interviews.

Lack of Access to Knowledgeable, Compassionate Providers. Lack of access to knowledgeable care providers was another prevalent theme expressed by adolescents and adults with SCD. Frustration occurred when providers did not have knowledge of SCD and its management, particularly pain assessment. Adolescents and adults with SCD noted the lack of compassion among providers: “I’ve been kicked out of the hospital because they felt like okay, well we gave you enough medication, you should be all right” (29-year-old female). Providers specifically mentioned lack of compassion and knowledge as barriers to SCD care much less often during their interviews compared with the adolescents and adults with SCD.

Health Care System Barriers. Patient participants often expressed concerns about concrete and structural aspects of care. Getting to their appointments was a challenge for half of the interviewees, as they either did not have access to a vehicle or could not afford to travel the needed distance to obtain quality care. Even when hospitals were accessible by public transportation, those with excruciating pain understandably preferred a more comfortable and private way to travel: “I would like to change that, something that will be much easier, convenient for sickle cell patients that do suffer with pain, that they don’t have to travel always to see the doctor” (30-year-old male).

Insurance and other financial barriers also played an important role in influencing decisions to seek health care services. Medical expenses were not covered, or co-pays were too high. The Medicaid managed care system could prevent access to knowledgeable providers who were not within network. Such a lack of access discouraged some adolescents and adults with SCD from seeking acute and preventive care.

Transition From Pediatric to Adult Care. Interviewees with SCD expressed distress about the gap between pediatric and adult care. They described how they had a long-standing relationship with their medical providers, who were familiar with their medical background and history from childhood. Adolescent interviewees reported an understanding of their own pain management as well as adherence to and satisfaction with their individualized pain plans. However, adults noted that satisfaction plummeted with increasing age due to the limited number of experienced adult SCD providers, which was compounded by negative experiences (stigma, racism, drug-seeking label).

One interviewee emphasized the difficulty of finding knowledgeable providers after transition: “When you’re a pediatric sickle cell [patient], you have the doctors there every step of the way, but not with adult sickle cell… I know when I first transitioned I never felt more alone in my life… you look at that ER doctor kind of with the same mindset as you would your hematologist who just hand walked you through everything. And adult care providers were a lot more blunt and cold and they’re like… ‘I don’t know; I’m not really educated in sickle cell.’” A sickle cell provider shared his insight about the problem of transitioning: “I think it’s particularly challenging because we, as a community, don’t really set them up for success. It’s different from other chronic conditions [in that] it’s much harder to find an adult sickle cell provider. There’s not a lot of adult hematologists that will take care of our adult patients, and so I know statistically, there’s like a drop-down in the overall outcomes of our kids after they age out of our pediatric program.”

Self-Management, Supporting Hydroxyurea Use. Interview participants with SCD reported using a variety of methods to manage pain at home and chose to go to the ED only when the pain became intolerable. Patients and providers expressed awareness of different resources for managing pain at home, yet they also indicated that these resources have not been consolidated in an accessible way for patients and families. Some resources cited included heat therapy, acupuncture, meditation, medical marijuana, virtual reality devices, and pain medications other than opioids.

Patients and providers expressed the need for increasing awareness and education about hydroxyurea. Many interview participants with SCD were concerned about side effects, multiple visits with a provider during dose titration, and ongoing laboratory monitoring. They also expressed difficulties with scheduling multiple appointments, depending on access to transportation and limited provider clinic hours. They were aware of strategies for improving adherence with hydroxyurea, including setting phone alarms, educating family members about hydroxyurea, and eliciting family support, but expressed needing help to consistently implement these strategies.

Safe Opioid Prescribing. Adult care providers expressed concerns about safe opioid prescribing for patients with SCD. They were reluctant to prescribe opioid doses needed to adequately control SCD pain. Providers expressed uncertainty and fear or concern about medical/legal liability or about their judgment about what’s safe and not safe for patients with chronic use/very high doses of opioids. “I know we’re in like this opiate epidemic here in this country but I feel like these patients don’t really fit under that umbrella that the problem is coming from so [I am] just trying to learn more about how to take care of them.”

Care Coordination and Provider Communication. Adolescents and adults with SCD reported having positive experiences—good communication, established trust, and compassionate care—with their usual providers. However, they perceived that ED physicians and nurses did not really care about them. Both interviewees with SCD and providers recognized the importance of good communication in all settings as the key to overcoming barriers to receiving quality care. All agreed on the importance of using individual pain plans so that all providers, especially ED providers, can be more at ease with treating adolescents and adults with SCD.

Quantitative Data: Adolescents and Adults With SCD

Fifty-eight adolescents and adults with SCD (aged 15 to 48 years) completed the survey. Three additional individuals who did not complete the interview completed the survey. Reasons for not completing the interview included scheduling challenges (n = 2) or a sickle cell pain episode (n = 1). The average age of participants was 31 years ± 8.6, more than half (57%) were female, and the majority (93%) were African American (Table 1). Most (71%) had never been married. Half (50%) had some college or an associate degree, and 40% were employed and reported an annual household income of less than $30,000. Insurance coverage was predominantly Medi-Cal (Medicaid, 69%). The majority of participants resided in Alameda (34.5%) or Contra Costa (21%) counties. The majority of sickle cell care was received in Alameda County, whether outpatient (52%), inpatient (40%), or ED care (41%). The majority (71%) had a diagnosis of SCD hemoglobin SS.

Pain. More than one-third of individuals with SCD reported 1 or 2 ED visits for pain in the previous 6 months (34%), and more than 3 hospitalizations (36%) related to pain in the previous year (Table 2). The majority (85%) reported having severe pain at home in the previous 6 months that they did not seek health care for, consistent with their reports in the qualitative interviews. More than half (59%) reported 4 or more of these severe pain episodes that led to inability to perform daily activities for 1 week or more. While pain interference on the PROMIS Pain Interference Short Form on average (T-score, 59.6 ± 8.6) was similar to that of the general population (T-score, 50 ± 10), a higher proportion of patients with SCD reported pain interference compared with the general population. The mean self-efficacy (confidence in ability to manage complications of SCD) score on the SCSES of 30.0 ± 7.3 (range, 9–45) was similar to that of other adults with SCD (mean, 32.2 ± 7.0). Twenty-five percent of the present sample had a low self-efficacy score (< 25).

Barriers to Care and Treatments. Consistent with the qualitative data, SCD-related symptoms such as tiredness (64%) and pain (62%) were reported most often as barriers to care (Table 3). Emotions (> 25%) such as worry/fear, frustration/anger, and lack of confidence were other important barriers to care. Provider knowledge and attitudes were cited next most often, with 38% of the sample indicating “Providers accuse me of drug-seeking” and “It is hard for me to find a provider who has enough experiences with or knowledge about SCD.” Participants expressed that they were not believed when in pain and “I am treated differently from other patients.” Almost half of respondents cited “I am not seen quickly enough when I am in pain” as a barrier to their care.

Consistent with the qualitative data, transportation barriers (not having a vehicle, costs of transportation, public transit not easy to get to) were cited by 55% of participants. About half of participants reported that insurance was an important barrier, with high co-pays and medications and other services not covered. In addition, gathering approvals was a long and fragmented process, particularly for consultations among providers (hematology, primary care provider, pain specialist). Furthermore, insurance provided limited choices about location for services.

Participants reported social support system burnout (22%), help needed with daily activities (21%), and social isolation or generally not having enough support (33%) as ongoing barriers. Difficulties were encountered with self-management (eg, taking medications on time or making follow-up appointments, 19%), with 22% of participants finding the health care system confusing or hard to understand. Thirty percent reported “Places for me to go to learn how to stay well are not close by or easy to get to.” ”Worry about side effects” (33%) was a common barrier to hydroxyurea use. Participants described “forgetting to take the medicine,” “tried before but it did not work,” “heard scary things” about hydroxyurea, and “not interested in taking another medicine” as barriers.

Quality of Care. More than half (51%) of the 53 participants who had accessed health care in the previous year rated their overall health care as poor on the ASCQ-Me QOC measure. This was significantly higher compared to the reports from more than 47,000 adults with Medicaid in 2017 (16%),⁴¹ and to the 2008-2009 report from 556 adults with SCD from across the United States (37%, Figure 2).³³ The major contributor to these poor ratings for participants in our sample was low satisfaction with ED care.

Sixty percent of the 42 participants who had accessed ED care in the past year indicated “never” or “sometimes” to the question “When you went to the ED for care, how often did you get it as soon as you wanted?” compared with only 16% of the 2017 adult Medicaid population responding (n = 25,789) (Figure 3). Forty-seven percent of those with an ED visit indicated that, in the previous 12 months, they had been made to wait “more than 2 hours before receiving treatment for acute pain in the ED.” However, in the previous 12 months, 39% reported that their wait time in the ED had been only “between five minutes and one hour.”

On the ASCQ-Me QOC Access to Care composite measure, 33% of 42 participants responding reported they were seen at a routine appointment as soon as they would have liked. This is significantly lower compared to 56% of the adult Medicaid population responding to the same question. Reports of provider communication (Provider Communication composite) for adolescents and adults with SCD were comparable to reports of adults with SCD from the ASCQ-Me field test,³³ but adults with Medicaid reported higher ratings of quality communication behaviors (Figure 4).^33,41 Nearly 60% of both groups with SCD reported that providers “always” performed quality communication behaviors—listened carefully, spent enough time, treated them with respect, and explained things well—compared with more than 70% of adults with Medicaid.

Participants from all counties reported the same number of barriers to care on average (3.3 ± 2.1). Adolescents and adults who reported more barriers to care also reported lower satisfaction with care (r = –0.47, P < 0.01) and less confidence in their ability to manage their SCD (self-efficacy, r = – 0.36, P < 0.05). Female participants reported more barriers to care on average compared with male participants (2.6 ± 2.4 vs 1.4 ± 2.0, P = 0.05). Participants with higher self-efficacy reported lower pain ratings (r = –0.47, P < 0.001).

Quantitative Data: Health Care Providers

Providers (n = 56) and community stakeholders (2 leaders of community-based organizations and 3 health care administrators) were interviewed, with 29 also completing the survey. The reason for not completing (n = 22) was not having the time once the interview was complete. A link to the survey was sent to any provider not completing at the time of the interview, with 2 follow-up reminders. The majority of providers were between the ages of 31 and 50 years (46.4%), female (71.4%), and white (66.1%) (Table 4). None were of Hispanic, Latinx, or Spanish origin. Thirty-six were physicians (64.3%), and 16 were allied health professionals (28.6%). Of the 56 providers, 32 indicated they had expertise caring for patients with SCD (57.1%), 14 were ED providers (25%), and 5 were primary care providers. Most of the providers practiced in an urban setting (91.1%).

Barriers to Care: ED Provider Perspectives. Nine of 14 ED providers interviewed completed the survey on their perspectives regarding barriers to care in the ED, difficulty with follow-ups, ED training resources, and pain control for patients with SCD. ED providers (n = 8) indicated that “provider attitudes” were a barrier to care delivery in the ED for patients with SCD. Some providers (n = 7) indicated that “implicit bias,” “opioid epidemic,” “concern about addiction,” and “patient behavior” were barriers. Respondents indicated that “overcrowding” (n = 6) and “lack of care pathway/protocol” (n = 5) were barriers. When asked to express their level of agreement with statements about SCD care in the ED, respondents disagreed/strongly disagreed (n = 5) that they were “able to make a follow-up appointment” with a sickle cell specialist or primary care provider upon discharge from the ED, and others disagreed/strongly disagreed (n = 4) that they were able to make a “referral to a case management program.”

ED training and resources. Providers agreed/strongly agreed (n = 8) that they had the knowledge and training to care for patients with SCD, that they had access to needed medications, and that they had access to knowledgeable nursing staff with expertise in SCD care. All 9 ED providers indicated that they had sufficient physician/provider staffing to provide good pain management to persons with SCD in the ED.

Pain control in the ED. Seven ED providers indicated that their ED used individualized dosing protocols to treat sickle cell pain, and 5 respondents indicated their ED had a protocol for treating sickle cell pain. Surprisingly, only 3 indicated that they were aware of the NHLBI recommendations for the treatment of vaso-occlusive pain.

Barriers to Care: Primary Care Provider Perspectives. Twenty providers completed the SCD provider section of the survey, including 17 multidisciplinary SCD providers from 4 sickle cell special care centers and 3 community primary care providers. Of the 20, 12 were primary care providers for patients with SCD (Table 4).

Patient needs. Six primary care providers indicated that the medical needs of patients with SCD were being met, but none indicated that the behavioral health or mental health needs were being met.

Managing SCD comorbidities. Five primary care providers indicated they were very comfortable providing preventive ambulatory care to patients with SCD. Six indicated they were very comfortable managing acute pain episodes, but none were very comfortable managing comorbidities such as pulmonary hypertension, diabetes, or chronic pain.

Barriers to opioid use. Only 3 of 12 providers reviewing a list of 15 potential barriers to the use of opioids for SCD pain management indicated a perceived lack of efficacy of opioids, development of tolerance and dependence, and concerns about community perceptions as barriers. Two providers selected potential for diversion as a moderate barrier to opioid use.

Barriers to hydroxyurea use. Eight of 12 providers indicated that the common reasons that patients/families refuse hydroxyurea were “worry about side effects”; 7 chose “don’t want to take another medicine,” and 6 chose “worry about carcinogenic potential.” Others (n = 10) indicated that “patient/family adherence with hydroxyurea” and “patient/family adherence with required blood tests” were important barriers to hydroxyurea use. Eight of the 12 providers indicated that they were comfortable with managing hydroxyurea in patients with SCD.

Care redesign. Twenty SCD and primary care providers completed the Care Redesign section of the survey. Respondents (n = 11) indicated that they would see more patients with SCD if they had accessible case management services available without charge or if patient access to transportation to clinic was also available. Ten indicated that they would see more patients with SCD if they had an accessible community health worker (who understands patient’s/family’s social situation) and access to a pain management specialist on call to answer questions and who would manage chronic pain. All (n = 20) were willing to see more patients with SCD in their practices. Most reported that a clinical decision-support tool for SCD treatment (n = 13) and avoidance of complications (n = 12) would be useful.

Discussion

We evaluated access and barriers to care, quality of care, care coordination, and provider communication from the perspectives of adolescents and adults with SCD, their care providers, and community stakeholders, within the Solberg conceptual model for quality improvement. We found that barriers within the care process content domain (context and systems) were most salient for this population of adolescents and adults with SCD, with lack of provider knowledge and poor attitudes toward adolescents and adults with SCD, particularly in the ED, cited consistently by participant groups. Stigmatization and lack of provider compassion that affected the quality of care were particularly problematic. These findings are consistent with previous reports.^42,43 Adult health care (particularly ED) provider biases and negative attitudes have been recognized as major barriers to optimal pain management in SCD.^8,11,44,45 Interestingly, ED providers in our needs assessment indicated that they felt they had the training and resources to manage patients with SCD. However, only a few actually reported knowing about the NHLBI recommendations for the treatment of vaso-occlusive pain.

Within the care process content domain, we also found that SCD-related complications and associated emotions (fear, worry, anxiety), compounded by lack of access to knowledgeable and compassionate providers, pose a significant burden. Negative encounters with the health care system contributed to a striking 84% of patient participants choosing to manage severe pain at home, with pain seriously interfering with their ability to function on a daily basis. ED providers agreed that provider attitudes and implicit bias pose important barriers to care for adolescents and adults with SCD. Adolescents and adults with SCD wanted, and understood the need, to enhance self-management skills. Both they and their providers agreed that barriers to hydroxyurea uptake included worries about potential side effects, challenges with adherence to repeated laboratory testing, and support with remembering to take the medicine. However, providers uniformly expressed that access to behavioral and mental health services were, if not nonexistent, impossible to access.

Participants with SCD and their providers reported infrastructural challenges (change process capability), as manifested in limitations with accessing acute and preventive care due to transportation- and insurance- related issues. There were health system barriers that were particularly encountered during the transition from pediatric to adult care. These findings are consistent with previous reports that have found fewer interdisciplinary services available in the adult care settings compared with pediatrics.^46,47 Furthermore, adult care providers were less willing to accept adults with SCD because of the complexity of their management, for which the providers did not have the necessary expertise.^3,48-50 In addition, both adolescents and adults with SCD and primary care providers highlighted the inadequacies of the current system in addressing the chronic pain needs of this population. Linking back to the Solberg conceptual framework, our needs assessment results confirm the important role of establishing SCD care as a priority within a health care system—this requires leadership and vision. The vision and priorities must be implemented by effective health care teams. Multilevel approaches or interventions, when implemented, will lead to the desired outcomes.

Findings from our needs assessment within our 5-county region mirror needs assessment results from the broader consortium.⁵¹ The SCDIC has prioritized developing an intervention that addresses the challenges identified within the care process domain by directly enhancing provider access to patient individualized care plans in the electronic health record in the ED. Importantly, ED providers will be asked to view a short video that directly challenges bias and stigma in the ED. Previous studies have indeed found that attitudes can be improved by providers viewing short video segments of adults with SCD discussing their experiences.^36,52 This ED protocol will be one of the interventions that we will roll out in Northern California, given the significance of negative ED encounters reported by needs assessment participants. An additional feature of the intervention is a script for adults with SCD that guides them through introducing their individualized pain plan to their ED providers, thereby enhancing their self-efficacy in a situation that has been so overwhelmingly challenging.

We will implement a second SCDIC intervention that utilizes a mobile app to support self-management on the part of the patient, by supporting motivation and adherence with hydroxyurea.⁵³ A companion app supports hydroxyurea guideline adherence on the part of the provider, in keeping with one of our findings that providers are in need of decision-support tools. Elements of the intervention also align with our findings related to the importance of a support system in managing SCD, in that participants will identify a supportive partner who will play a specific role in supporting their adherence with hydroxyurea.

On our local level, we have, by necessity, partnered with leaders and community stakeholders throughout the region to ensure that these interventions to improve SCD care are prioritized. Grant funds provide initial resources for the SCDIC interventions, but our partnering health care administrators and medical directors must ensure that participating ED and hematology providers are free from competing priorities in order to implement the changes. We have partnered with a SCD community-based organization that is designing additional educational presentations for local emergency medicine providers, with the goal to bring to life very personal stories of bias and stigma within the EDs that directly contribute to decisions to avoid ED care despite severe symptoms.

Although we attempted to obtain samples of adolescents and adults with SCD and their providers that were representative across the 5-county region, the larger proportion of respondents were from 1 county. We did not assess concerns of age- and race-matched adults in our catchment area, so we cannot definitively say that our findings are unique to SCD. However, our results are consistent with findings from the national sample of adults with SCD who participated in the ASCQ-Me field test, and with results from the SCDIC needs assessment.^33,51 Interviews and surveys are subject to self-report bias and, therefore, may or may not reflect the actual behaviors or thoughts of participants. Confidence is increased in our results given the triangulation of expressed concerns across participant groups and across data collection strategies. The majority of adolescents and adults with SCD (95%) completed both the interview and survey, while 64% of ED providers interviewed completed the survey, compared with 54% of SCD specialists and primary care providers. These response rates are more than acceptable within the realm of survey response rates.^54,55

Although we encourage examining issues with care delivery within the conceptual framework for quality improvement presented, we recognize that grant funding allowed us to conduct an in-depth needs assessment that might not be feasible in other settings. Still, we would like readers to understand the importance of gathering data for improvement in a systematic manner across a range of participant groups, to ultimately inform the development of interventions and provide for evaluation of outcomes as a result of the interventions. This is particularly important for a disease, such as SCD, that is both medically and sociopolitically complex.

Conclusion

Our needs assessment brought into focus the multiple factors contributing to the disparities in health care experienced by adolescents and adults with SCD on our local level, and within the context of inequities in health resources and outcomes on the national level. We propose solutions that include specific interventions developed by a consortium of SCD and implementation science experts. We utilize a quality improvement framework to ensure that the elements of the interventions also address the barriers identified by our local providers and patients that are unique to our community. The pervasive challenges in SCD care, coupled with its medical complexities, may seem insurmountable, but our survey and qualitative results provide us with a road map for the way forward.

Acknowledgments: The authors thank the adolescents and adults with sickle cell disease, the providers, and the community stakeholders who completed the interviews and surveys. The authors also acknowledge the SCCCI co-investigators for their contributions to this project, including Michael Bell, MD, Ward Hagar, MD, Christine Hoehner, FNP, Kimberly Major, MSW, Anne Marsh, MD, Lynne Neumayr, MD, and Ted Wun, MD. We also thank Kamilah Bailey, Jameelah Hodge, Jennifer Kim, Michael Rowland, Adria Stauber, Amber Fearon, and Shanda Robertson, and the Sickle Cell Data Collection Program for their contributions.

Corresponding author: Marsha J. Treadwell, PhD, University of California San Francisco Benioff Children’s Hospital Oakland, 747 52nd St., Oakland, CA 94609; marsha.treadwell@ucsf.edu.

Financial disclosures: None.

Funding/support: This work was supported by grant # 1U01HL134007 from the National Heart, Lung, and Blood Institute to the University of California San Francisco Benioff Children’s Hospital Oakland.