Original Research

The use of mobile devices in health care settings has enhanced clinical practice through real-time communication and direct patient monitoring.¹ With advancements in technology, improving the accessibility and quality of patient care using mobile devices is a hot topic. In 2018, 261.34 million people worldwide used smartphones compared to 280.54 million in 2021—a 7.3% increase.² Revenue from sales of mobile applications (apps) is projected to reach $693 billion in 2021.³

A range of apps targeted to patients is available for acne, melanoma, and teledermatology.^4-6 Nail disorders are a common concern, representing 21.1 million outpatient visits in 2007 to 2016,⁷ but, to date, the availability of apps related to nail disorders has not been explored. In this study, we investigated iOS (Apple’s iPhone Operating System) and Android apps to determine the types of nail health apps that are available, using psoriasis and hair loss apps as comparator groups.

Methods

A standard app analytics and market data tool (App Annie; https://www.appannie.com/en/) was utilized to search for iOS and Android nail mobile apps.^4,5 The analysis was performed on a single day (March 23, 2020), given that app searches can change on a daily basis. Our search included the following keywords: nail, nail health, toenail fungus, nail tumor, brittle nails, onychomycosis, onycholysis, subungual melanoma, nail melanoma, paronychia, and nail squamous cell carcinoma. App Annie app descriptions were assessed to determine the type of each app (Lifestyle, Medical, Health & Fitness) and target audience (patient, physician, or both). Psoriasis and hair loss topics were chosen as controls for comparison, based on a prior study.⁸ For psoriasis, the keywords psoriasis and chronic skin disease were searched. Hair loss was searched using the keywords alopecia, hair loss, hair health, and scalp.

Results

Nail-Related Apps
Using keywords for nail-related terms on iOS and Android platforms, our search returned few specific and informative apps related to nail disorders (Table 1). When the terms brittle nails, nail, nail health, nail squamous cell carcinoma, and nail tumor were searched, all available nail apps were either nail games or virtual nail salons for entertainment purposes. For the terms nail melanoma and subungual melanoma, there were no specific nail apps that appeared in the search results; rather, the App Annie search yielded only general dermatology and melanoma apps. The terms onycholysis and paronychia both yielded 0 hits for iOS and Android.

The only search terms that returned specific nail apps were onychomycosis and toenail fungus. Initially, when onychomycosis was searched, only 1 Google Play Medical category app was found: “Nail fungal infection (model onychomycosis).” Although this app recently was removed from the app store, it previously allowed the user to upload a nail photograph, with which a computing algorithm assessed whether the presentation was a fungal nail infection. Toenail fungus returned 1 iOS Medical category app and 5 Android Health & Fitness category apps with reference material for patients. Neither of the 2 medical apps for onychomycosis and toenail fungus referenced a physician involved in the app development.

Psoriasis Comparator
On the contrary, a search for psoriasis yielded 22 hits for iOS and 34 hits for Android within the Health & Fitness, Medical, and Social Networking categories (Table 2). The search term chronic skin disease returned 18 apps for iOS and 60 apps for Android related to psoriasis; 100% were classified as Medical apps.

Hair Loss Comparator
Search terms related to hair conditions—specifically, alopecia—yielded 0 apps for iOS and 10 for Android platforms (Table 2). Using the search term hair loss, 12 apps for iOS and 50 apps for Android were found within the Health & Fitness, Medical, and Beauty categories. The search terms hair health and hair loss resulted in 2 and 12 apps in both iOS and Android, respectively. In addition, the search term scalp was associated with 6 related apps in iOS and 7 in Android, both in the Health & Fitness and Medical categories.

Other Findings
Most apps for psoriasis and hair health were identified as patient focused. Although iOS and Android are different operating systems, some health apps overlapped: subungual melanoma and toenail fungus had a 20% overlap; psoriasis, 19%; chronic skin disease, 2%; alopecia, 0%; hair loss and hair health, 10%; and scalp, 18%. iOS and Android nail entertainment games had approximately a 30% overlap. Tables 1 and 2 also compare the number of free and paid apps; most available apps were free.

 

Comment

With continued growth in mobile device ownership and app development has been parallel growth in the creation and use of apps to enhance medical care.¹ In a study analyzing the most popular dermatology apps, 62% (18/29) and 38% (11/29) of apps targeted patients and physicians, respectively.⁶ Our study showed that (1) there are few nail disorder apps available for patient education and (2) there is no evidence that a physician was consulted for content input. Because patients who can effectively communicate their health concerns before and after seeing a physician have better self-reported clinical outcomes,⁹ it is important to have nail disorder apps available to patients for referencing. The nail health app options differ notably from psoriasis and hair loss apps, with apps for the latter 2 topics found in Medical and Health & Fitness categories—targeting patients who seek immediate access to health care and education.

Although there are several general dermatology apps that contain reference information for patients pertaining to nail conditions,⁶ using any of those apps would require a patient to have prior knowledge that dermatologists specialize in nail disorders and necessitate several steps to find nail-relevant information. For example, the patient would have to search dermatology in the iOS and Android app stores, select the available app (eg, Dermatology Database), and then search within that app for nail disorders. Therefore, a patient who is concerned about a possible subungual melanoma would not be able to easily find clinical images and explanations using an app.

Study Limitations
This study was subject to several limitations. Android and iOS app stores have undisclosed computing algorithms that might have filtered apps based on specific word inquiry. Also, our queries were based on specific relevant keywords for nail conditions, psoriasis, and hair loss; use of different keywords might have yielded different results. Additionally, app options change on a daily basis, so a search today (ie, any given day) might yield slightly different results than it did on March 23, 2020.

Conclusion

Specific nail disorder apps available for patient reference are limited. App developers should consider accessibility (ie, clear language, ease of use, cost-effectiveness, usability on iOS- and Android-operated devices) and content (accurate medical information from experts) when considering new apps. A solution to this problem is for established medical organizations to create nail disorder apps specifically for patients.¹⁰ For example, the American Academy of Dermatology has iOS and Android apps that are relevant to physicians (MyDermPath+, Dialogues in Dermatology, Mohs Surgery Appropriate Use Criteria) but no comparable apps for patients; patient-appropriate nail apps are necessary.¹¹ In addition, it would be beneficial to patients if established app companies consulted with dermatologists on pertinent nail content.

In sum, we found few available nail health apps on the iOS or Android platforms that provided accessible and timely information to patients regarding nail disorders. There is an immediate need to produce apps related to nail health for appropriate patient education.

The use of mobile devices in health care settings has enhanced clinical practice through real-time communication and direct patient monitoring.¹ With advancements in technology, improving the accessibility and quality of patient care using mobile devices is a hot topic. In 2018, 261.34 million people worldwide used smartphones compared to 280.54 million in 2021—a 7.3% increase.² Revenue from sales of mobile applications (apps) is projected to reach $693 billion in 2021.³

A range of apps targeted to patients is available for acne, melanoma, and teledermatology.^4-6 Nail disorders are a common concern, representing 21.1 million outpatient visits in 2007 to 2016,⁷ but, to date, the availability of apps related to nail disorders has not been explored. In this study, we investigated iOS (Apple’s iPhone Operating System) and Android apps to determine the types of nail health apps that are available, using psoriasis and hair loss apps as comparator groups.

Methods

A standard app analytics and market data tool (App Annie; https://www.appannie.com/en/) was utilized to search for iOS and Android nail mobile apps.^4,5 The analysis was performed on a single day (March 23, 2020), given that app searches can change on a daily basis. Our search included the following keywords: nail, nail health, toenail fungus, nail tumor, brittle nails, onychomycosis, onycholysis, subungual melanoma, nail melanoma, paronychia, and nail squamous cell carcinoma. App Annie app descriptions were assessed to determine the type of each app (Lifestyle, Medical, Health & Fitness) and target audience (patient, physician, or both). Psoriasis and hair loss topics were chosen as controls for comparison, based on a prior study.⁸ For psoriasis, the keywords psoriasis and chronic skin disease were searched. Hair loss was searched using the keywords alopecia, hair loss, hair health, and scalp.

Results

Nail-Related Apps
Using keywords for nail-related terms on iOS and Android platforms, our search returned few specific and informative apps related to nail disorders (Table 1). When the terms brittle nails, nail, nail health, nail squamous cell carcinoma, and nail tumor were searched, all available nail apps were either nail games or virtual nail salons for entertainment purposes. For the terms nail melanoma and subungual melanoma, there were no specific nail apps that appeared in the search results; rather, the App Annie search yielded only general dermatology and melanoma apps. The terms onycholysis and paronychia both yielded 0 hits for iOS and Android.

The only search terms that returned specific nail apps were onychomycosis and toenail fungus. Initially, when onychomycosis was searched, only 1 Google Play Medical category app was found: “Nail fungal infection (model onychomycosis).” Although this app recently was removed from the app store, it previously allowed the user to upload a nail photograph, with which a computing algorithm assessed whether the presentation was a fungal nail infection. Toenail fungus returned 1 iOS Medical category app and 5 Android Health & Fitness category apps with reference material for patients. Neither of the 2 medical apps for onychomycosis and toenail fungus referenced a physician involved in the app development.

Psoriasis Comparator
On the contrary, a search for psoriasis yielded 22 hits for iOS and 34 hits for Android within the Health & Fitness, Medical, and Social Networking categories (Table 2). The search term chronic skin disease returned 18 apps for iOS and 60 apps for Android related to psoriasis; 100% were classified as Medical apps.

Hair Loss Comparator
Search terms related to hair conditions—specifically, alopecia—yielded 0 apps for iOS and 10 for Android platforms (Table 2). Using the search term hair loss, 12 apps for iOS and 50 apps for Android were found within the Health & Fitness, Medical, and Beauty categories. The search terms hair health and hair loss resulted in 2 and 12 apps in both iOS and Android, respectively. In addition, the search term scalp was associated with 6 related apps in iOS and 7 in Android, both in the Health & Fitness and Medical categories.

Other Findings
Most apps for psoriasis and hair health were identified as patient focused. Although iOS and Android are different operating systems, some health apps overlapped: subungual melanoma and toenail fungus had a 20% overlap; psoriasis, 19%; chronic skin disease, 2%; alopecia, 0%; hair loss and hair health, 10%; and scalp, 18%. iOS and Android nail entertainment games had approximately a 30% overlap. Tables 1 and 2 also compare the number of free and paid apps; most available apps were free.

 

Comment

With continued growth in mobile device ownership and app development has been parallel growth in the creation and use of apps to enhance medical care.¹ In a study analyzing the most popular dermatology apps, 62% (18/29) and 38% (11/29) of apps targeted patients and physicians, respectively.⁶ Our study showed that (1) there are few nail disorder apps available for patient education and (2) there is no evidence that a physician was consulted for content input. Because patients who can effectively communicate their health concerns before and after seeing a physician have better self-reported clinical outcomes,⁹ it is important to have nail disorder apps available to patients for referencing. The nail health app options differ notably from psoriasis and hair loss apps, with apps for the latter 2 topics found in Medical and Health & Fitness categories—targeting patients who seek immediate access to health care and education.

Although there are several general dermatology apps that contain reference information for patients pertaining to nail conditions,⁶ using any of those apps would require a patient to have prior knowledge that dermatologists specialize in nail disorders and necessitate several steps to find nail-relevant information. For example, the patient would have to search dermatology in the iOS and Android app stores, select the available app (eg, Dermatology Database), and then search within that app for nail disorders. Therefore, a patient who is concerned about a possible subungual melanoma would not be able to easily find clinical images and explanations using an app.

Study Limitations
This study was subject to several limitations. Android and iOS app stores have undisclosed computing algorithms that might have filtered apps based on specific word inquiry. Also, our queries were based on specific relevant keywords for nail conditions, psoriasis, and hair loss; use of different keywords might have yielded different results. Additionally, app options change on a daily basis, so a search today (ie, any given day) might yield slightly different results than it did on March 23, 2020.

Conclusion

Specific nail disorder apps available for patient reference are limited. App developers should consider accessibility (ie, clear language, ease of use, cost-effectiveness, usability on iOS- and Android-operated devices) and content (accurate medical information from experts) when considering new apps. A solution to this problem is for established medical organizations to create nail disorder apps specifically for patients.¹⁰ For example, the American Academy of Dermatology has iOS and Android apps that are relevant to physicians (MyDermPath+, Dialogues in Dermatology, Mohs Surgery Appropriate Use Criteria) but no comparable apps for patients; patient-appropriate nail apps are necessary.¹¹ In addition, it would be beneficial to patients if established app companies consulted with dermatologists on pertinent nail content.

In sum, we found few available nail health apps on the iOS or Android platforms that provided accessible and timely information to patients regarding nail disorders. There is an immediate need to produce apps related to nail health for appropriate patient education.

The use of mobile devices in health care settings has enhanced clinical practice through real-time communication and direct patient monitoring.¹ With advancements in technology, improving the accessibility and quality of patient care using mobile devices is a hot topic. In 2018, 261.34 million people worldwide used smartphones compared to 280.54 million in 2021—a 7.3% increase.² Revenue from sales of mobile applications (apps) is projected to reach $693 billion in 2021.³

A range of apps targeted to patients is available for acne, melanoma, and teledermatology.^4-6 Nail disorders are a common concern, representing 21.1 million outpatient visits in 2007 to 2016,⁷ but, to date, the availability of apps related to nail disorders has not been explored. In this study, we investigated iOS (Apple’s iPhone Operating System) and Android apps to determine the types of nail health apps that are available, using psoriasis and hair loss apps as comparator groups.

Methods

A standard app analytics and market data tool (App Annie; https://www.appannie.com/en/) was utilized to search for iOS and Android nail mobile apps.^4,5 The analysis was performed on a single day (March 23, 2020), given that app searches can change on a daily basis. Our search included the following keywords: nail, nail health, toenail fungus, nail tumor, brittle nails, onychomycosis, onycholysis, subungual melanoma, nail melanoma, paronychia, and nail squamous cell carcinoma. App Annie app descriptions were assessed to determine the type of each app (Lifestyle, Medical, Health & Fitness) and target audience (patient, physician, or both). Psoriasis and hair loss topics were chosen as controls for comparison, based on a prior study.⁸ For psoriasis, the keywords psoriasis and chronic skin disease were searched. Hair loss was searched using the keywords alopecia, hair loss, hair health, and scalp.

Results

Nail-Related Apps
Using keywords for nail-related terms on iOS and Android platforms, our search returned few specific and informative apps related to nail disorders (Table 1). When the terms brittle nails, nail, nail health, nail squamous cell carcinoma, and nail tumor were searched, all available nail apps were either nail games or virtual nail salons for entertainment purposes. For the terms nail melanoma and subungual melanoma, there were no specific nail apps that appeared in the search results; rather, the App Annie search yielded only general dermatology and melanoma apps. The terms onycholysis and paronychia both yielded 0 hits for iOS and Android.

The only search terms that returned specific nail apps were onychomycosis and toenail fungus. Initially, when onychomycosis was searched, only 1 Google Play Medical category app was found: “Nail fungal infection (model onychomycosis).” Although this app recently was removed from the app store, it previously allowed the user to upload a nail photograph, with which a computing algorithm assessed whether the presentation was a fungal nail infection. Toenail fungus returned 1 iOS Medical category app and 5 Android Health & Fitness category apps with reference material for patients. Neither of the 2 medical apps for onychomycosis and toenail fungus referenced a physician involved in the app development.

Psoriasis Comparator
On the contrary, a search for psoriasis yielded 22 hits for iOS and 34 hits for Android within the Health & Fitness, Medical, and Social Networking categories (Table 2). The search term chronic skin disease returned 18 apps for iOS and 60 apps for Android related to psoriasis; 100% were classified as Medical apps.

Hair Loss Comparator
Search terms related to hair conditions—specifically, alopecia—yielded 0 apps for iOS and 10 for Android platforms (Table 2). Using the search term hair loss, 12 apps for iOS and 50 apps for Android were found within the Health & Fitness, Medical, and Beauty categories. The search terms hair health and hair loss resulted in 2 and 12 apps in both iOS and Android, respectively. In addition, the search term scalp was associated with 6 related apps in iOS and 7 in Android, both in the Health & Fitness and Medical categories.

Other Findings
Most apps for psoriasis and hair health were identified as patient focused. Although iOS and Android are different operating systems, some health apps overlapped: subungual melanoma and toenail fungus had a 20% overlap; psoriasis, 19%; chronic skin disease, 2%; alopecia, 0%; hair loss and hair health, 10%; and scalp, 18%. iOS and Android nail entertainment games had approximately a 30% overlap. Tables 1 and 2 also compare the number of free and paid apps; most available apps were free.

 

Comment

With continued growth in mobile device ownership and app development has been parallel growth in the creation and use of apps to enhance medical care.¹ In a study analyzing the most popular dermatology apps, 62% (18/29) and 38% (11/29) of apps targeted patients and physicians, respectively.⁶ Our study showed that (1) there are few nail disorder apps available for patient education and (2) there is no evidence that a physician was consulted for content input. Because patients who can effectively communicate their health concerns before and after seeing a physician have better self-reported clinical outcomes,⁹ it is important to have nail disorder apps available to patients for referencing. The nail health app options differ notably from psoriasis and hair loss apps, with apps for the latter 2 topics found in Medical and Health & Fitness categories—targeting patients who seek immediate access to health care and education.

Although there are several general dermatology apps that contain reference information for patients pertaining to nail conditions,⁶ using any of those apps would require a patient to have prior knowledge that dermatologists specialize in nail disorders and necessitate several steps to find nail-relevant information. For example, the patient would have to search dermatology in the iOS and Android app stores, select the available app (eg, Dermatology Database), and then search within that app for nail disorders. Therefore, a patient who is concerned about a possible subungual melanoma would not be able to easily find clinical images and explanations using an app.

Study Limitations
This study was subject to several limitations. Android and iOS app stores have undisclosed computing algorithms that might have filtered apps based on specific word inquiry. Also, our queries were based on specific relevant keywords for nail conditions, psoriasis, and hair loss; use of different keywords might have yielded different results. Additionally, app options change on a daily basis, so a search today (ie, any given day) might yield slightly different results than it did on March 23, 2020.

Conclusion

Specific nail disorder apps available for patient reference are limited. App developers should consider accessibility (ie, clear language, ease of use, cost-effectiveness, usability on iOS- and Android-operated devices) and content (accurate medical information from experts) when considering new apps. A solution to this problem is for established medical organizations to create nail disorder apps specifically for patients.¹⁰ For example, the American Academy of Dermatology has iOS and Android apps that are relevant to physicians (MyDermPath+, Dialogues in Dermatology, Mohs Surgery Appropriate Use Criteria) but no comparable apps for patients; patient-appropriate nail apps are necessary.¹¹ In addition, it would be beneficial to patients if established app companies consulted with dermatologists on pertinent nail content.

In sum, we found few available nail health apps on the iOS or Android platforms that provided accessible and timely information to patients regarding nail disorders. There is an immediate need to produce apps related to nail health for appropriate patient education.

Hospitalizations for infections are common in children, with respiratory illnesses, including pneumonia and bronchiolitis, among the most prevalent indications for hospitalization.^1,2 Infections are also among the most frequent indications for all-cause readmissions and for potentially preventable readmissions in children.^3-5 Beyond hospital resource use, infection hospitalizations and readmissions represent a considerable cause of life disruption for patients and their families.^6,7 While emerging evidence supports shortened durations of parenteral antibiotics before transitioning to oral therapy for some infections (eg, pyelonephritis, osteomyelitis),^8-10 other infections may require extended treatment courses for weeks. The risk of adverse outcomes (eg, complications of medical treatment, readmission risk) and burdens placed on patients and their families may therefore differ across infection types and extend well beyond the immediate hospitalization.

Although infections are common and pediatric providers are expected to have proficiency in managing infections, substantial variation in the management of common pediatric infections exists and is associated with adverse hospitalization outcomes, including increased readmission risk and healthcare costs.^11-18 Potentially avoidable resource use associated with hospital readmission from infection has led to adoption of hospital-level readmission metrics as indicators of the quality of healthcare delivery. For example, the Pediatric Quality Measures Program, established by the Children’s Health Insurance Program Reauthorization Act of 2009, has prioritized measurement of readmissions following hospitalization for lower respiratory tract infection.¹⁹ With government agencies increasingly using readmission metrics to assess quality of healthcare delivery, developing metrics that focus on these resource-intensive conditions is essential.

Because infections are a common and costly indication for hospital resource use and because substantial variation in management has been observed, promoting a broader understanding of infection-specific readmission rates is important for prioritizing readmission-reduction opportunities in children. This study’s objectives were the following: (1) to describe the prevalence and characteristics of infection hospitalizations in children and their associated readmissions and (2) to estimate the number of readmissions avoided and costs saved if all hospitals achieved the 10th percentile of the hospitals’ risk-adjusted readmission rate (ie, readmission benchmark).

METHODS

Study Design and Data Source

We performed a retrospective cohort analysis using the 2014 Agency for Healthcare Research and Quality (AHRQ) Nationwide Readmissions Database (NRD).²⁰ The 2014 NRD is an administrative database that contains information on inpatient stays from January 1, 2014, to December 31, 2014, for all payers and allows for weighted national estimates of readmissions for all US individuals. Data within NRD are aggregated from 22 geographically diverse states representing approximately one-half of the US population. NRD contains deidentified patient-level data with unique verified patient identifiers to track individuals within and across hospitals in a state. However, AHRQ guidelines specify that NRD cannot be used for reporting hospital-specific readmission rates. Thus, for the current study, the Inpatient Essentials (Children’s Hospital Association), or IE, database was used to measure hospital-level readmission rates and to distinguish benchmark readmission rates for individual infection diagnoses.²¹ The IE database is composed of 90 children’s hospitals distributed throughout all regions of the United States. The inclusion of free-standing children’s hospitals and children’s hospitals within adult hospitals allows for comparisons and benchmarking across hospitals on multiple metrics, including readmissions.

Study Population

Children 0 to 17 years of age with a primary diagnosis at the index admission for infection between January 1, 2014, and November 30, 2014, were included. The end date of November 30, 2014, allowed for a full 30-day readmission window for all index admissions. We excluded index admissions that resulted in transfer to another acute care hospital or in-hospital mortality. Additionally, we excluded index admissions of children who had hematologic or immunologic conditions, malignancy, or history of bone marrow and solid-organ transplant, using the classification system for complex chronic conditions (CCCs) from Feudtner et al.²² Due to the high likelihood of immunosuppression in patients with these conditions, children may have nuanced experiences with illness severity, trajectory, and treatment associated with infection that place them at increased risk for nonpreventable readmission.

Main Exposure

The main exposure was infection type during the index admission. Condition-specific index admissions were identified using AHRQ’s Clinical Classifications Software (CCS) categories.²³ CCS is a classification schema that categorizes the greater than 14,000 International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) diagnosis codes and 3,900 ICD-9-CM procedure codes into clinically meaningful categories of 295 diagnosis (including mental health codes and E-codes) and 231 procedural groupings. Twenty-two groupings indicative of infection were distinguished and used for the current study. Examples of infections included aspiration pneumonia, pneumonia, bronchiolitis, and sexually transmitted infection. We combined related CCS categories when possible for ease of interpretation and presentation of data (Appendix Table 1).

Main Outcome Measure

The main outcome measure was 30-day hospital readmission. Readmission was defined as all-cause, unplanned admission within 30 days following discharge from a preceding hospitalization. Planned hospital readmissions were identified and excluded using methods from AHRQ’s Pediatric All-Condition Readmission Measure.²⁴ We defined a same-cause return as a return with the same CCS infection category as the index admission. Costs associated with readmissions were estimated from charges using hospital-specific cost-to-charge ratios provided with NRD.

Patient Demographic and Clinical Characteristics

Patient demographic characteristics included age at index admission (<1 year, 1-5 years, 6-9 years, 10-14 years, and 15-18 years), sex, payer (ie, government, private, other), and discharge disposition (ie, routine, home health, other). We assessed all patients for medical complexity, as defined by the presence of at least one CCC, and we reported the categories of CCCs by organ system involved.²² Otherwise, patients were identified as without medical complexity.

Statistical Analysis

We summarized continuous variables with medians and interquartile ranges and categorical variables with frequencies and percentages. To develop benchmark readmission rates for each infection type, we used generalized linear mixed models with random intercepts for each hospital in the IE database. For each infection type, the benchmark readmission rate was defined as the 10th percentile of hospitals’ risk-adjusted readmission rates. The 10th percentile was chosen to identify the best performing 10% of hospitals (ie, hospitals with the lowest readmission rates). Because children with medical complexity account for a large proportion of hospital resource use and are at high risk for readmission,^4,25 we developed benchmarks stratified by presence/absence of a CCC (ie, with complexity vs without complexity). Models were adjusted for severity of illness using the Hospitalization Resource Intensity Score for Kids (H-RISK),²⁶ a scoring system that assigns relative weights for each All Patient Refined Diagnosis-Related Group (3M Corporation) and severity of illness level, and each hospital’s risk-adjusted readmission rate was determined.

With use of weights to achieve national estimates of index admissions and readmissions, we determined the number of potentially avoidable readmissions by calculating the number of readmissions observed in the NRD that would not occur if all hospitals achieved readmission rates equal to the 10th percentile. Avoidable costs were estimated by multiplying the number of potentially avoidable readmissions by the mean cost of a readmission for infections of that type. Estimates of avoidable readmissions and costs were stratified by medical complexity. In addition to describing estimates at the 10th percentile benchmark, we similarly developed estimates of potentially avoidable readmissions and avoidable costs for the 5th and 25th percentiles, which are presented within Appendix Table 2 (children without complexity) and Appendix Table 3 (children with complexity).

All statistical analyses were performed using SAS version 9.4 (SAS Institute), and P values <.001 were considered statistically significant due to the large sample size. The Office of Research Integrity at Children’s Mercy Hospital deemed this study exempt from institutional board review.

RESULTS

Characteristics of the Study Population

The study included 380,067 index admissions for infection and an accompanying 18,469 unplanned all-cause readmissions over the 30 days following discharge (readmission rate, 4.9%; Table 1). Children ages 1 to 5 years accounted for the largest percentage (32.9%) of index hospitalizations, followed by infants younger than 1 year (30.3%). The readmission rate by age group was highest for infants younger than 1 year, compared with rates among all other age groups (5.6% among infants < 1 year vs 4.4%-4.7% for other age groups; P < .001). In the overall cohort, 16.2% of admissions included patients with a CCC. Children with medical complexity had higher readmission rates than those without medical complexity (no CCC, 3.2%; 1 CCC, 9.2%; 2+ CCCs, 18.9%). A greater percentage of children experiencing a readmission had government insurance (63.0% vs 59.2%; P < .001) and received home health nursing (10.1% vs 2.7%; P < .001) following the index encounter.

Children Without Complexity

Index Admissions and 30-day Readmissions

Among patients without medical complexity, index admissions occurred most frequently for pneumonia (n = 54,717), bronchiolitis (n = 53,959), and appendicitis (n = 45,036) (Figure 1). The median length of stay (LOS) for index admissions ranged from 1 to 5 days (Table 2), with septic arthritis and osteomyelitis having the longest median LOS at 5 (IQR, 3-7) days.

Thirty-day readmission rates varied substantially by infection at the index admission (range, 1.5% for sexually transmitted infection to 8.6% for peritonitis) (Figure 1). The median LOS for 30-day readmissions varied from 2 to 7 days (Table 2), while the median number of days to readmission varied substantially by infection type (range, 4 days for bacterial infection [site unspecified] to 24 days for sexually transmitted infections). Among the top five indications for admission for children without complexity, 14.9% to 52.8% of readmissions were for the same cause as the index admission; however, many additional returns were likely related to the index admission (Appendix Table 4). For example, among other return reasons, an additional 992 (61.7%) readmissions following appendicitis hospitalizations were for complications of surgical procedures or medical care, peritonitis, intestinal obstruction, and abdominal pain.

Impact of Achieving Readmission Benchmarks

Among children without complexity, readmission benchmarks (ie, the 10th percentile of readmission rates across hospitals) ranged from 0% to 26.7% (Figure 2). An estimated 54.7% of readmissions (n = 5,507) could potentially be reduced if hospitals achieved infection-specific benchmark readmission rates, which could result in an estimated $44.5 million in savings. Pneumonia, bronchiolitis, gastroenteritis, and upper respiratory tract infections were among conditions with the greatest potential reductions in readmissions and costs if a 10th percentile benchmark was achieved.

Children With Medical Complexity

Index Admissions and 30-day Readmissions

Among patients with complexity, index admissions occurred most frequently for pneumonia (n = 14,344), bronchiolitis (n = 8,618), and upper respiratory tract infection (n = 6,407) (Figure 1). The median LOS for index admissions ranged from 1 to 9 days (Table 2), with septicemia and CNS infections having the longest median LOS at 9 days.

Thirty-day readmission rates varied substantially by the type of infection at the index admission (range, 0% for sexually transmitted infection to 21.6% for aspiration pneumonia) (Figure 1). The median LOS for 30-day readmissions varied from 2 to 14 days (Table 2), and the median number of days to readmission varied substantially by infection type (range, 6 days for tonsillitis to 23 days for other infection). Among the top five indications for admission for medically complex children, 8% to 40.4% of readmissions were for the same cause as the index admission (Appendix Table 4). As with the children without complexity, additional returns were likely related to the index admission.

Impact of Achieving Readmission Benchmarks

Among children with medical complexity, readmission benchmarks ranged from 0% to 30.3% (Figure 2). An estimated 42.6% of readmissions (n = 3,576) could potentially be reduced if hospitals achieved infection-specific benchmark readmission rates, which could result in an estimated $70.8 million in savings. Pneumonia, bronchiolitis, septicemia, and upper respiratory tract infections were among conditions with the greatest potential reductions in readmissions and costs if the benchmarks were achieved.

DISCUSSION

The current study uncovered new findings regarding unplanned readmissions following index infection hospitalizations for children. In particular, readmission rates and time to readmission varied substantially by infection subtype. The study also identified priority infections and unique considerations for children with CCCs, all of which may help maximize the value of readmission metrics aimed at advancing hospital quality and reducing costs for infection hospitalizations in children. If all hospitals achieved the readmission rates of the best performing hospitals, 42.6% to 54.7% fewer readmissions could be realized with associated cost savings.

Nationally, studies have reported 30-day, all-cause unplanned readmission rates of 6.2% to 10.3%.^5,27 In our current study we observed an overall readmission rate of 4.9% across all infectious conditions; however, readmission rates varied substantially by condition, with upper and lower respiratory tract infections, septicemia, and gastroenteritis among infections with the greatest number of potentially avoidable readmissions based on achievement of readmission benchmarks. Currently, pediatric-specific all-cause and lower respiratory tract infection readmission metrics have been developed with the aim of improving quality of care and reducing healthcare expenditures.²⁸ Future readmission studies and metrics may prioritize conditions such as septicemia, gastroenteritis, and other respiratory tract infections. Our current study demonstrated that many readmissions following infection hospitalizations were associated with the same CCS category or within a related CCS category (eg, complications of surgical procedures or medical care, appendicitis, peritonitis, intestinal obstruction, and abdominal pain constituted the top five indications for readmission for appendicitis, whereas respiratory illnesses constituted the top five indications for readmissions for pneumonia). While this current study cannot clarify this relationship further, and the “avoidability” of unplanned readmissions is debated,^29-31 our findings suggest that future investigations might focus on identifying whether condition-specific interventions during the index admission could mitigate some readmissions.

While the LOS of the index admission and the readmission were similar for most infection subtypes, we observed substantial variability in the temporal risk for readmission by infection subtype. Our observations complement studies exploring the timing of readmissions for other pediatric conditions.^32-34 In particular, our findings further highlight that the composition and interaction of factors influencing infection readmissions may vary by condition. Infections represent a diverse group of conditions, with treatment courses that vary in need for parenteral antibiotics, ability to tailor treatment based on organism and susceptibilities, and length of treatment. While treatment for some infections may be accomplished, or nearly accomplished, prior to discharge, other infections (eg, osteomyelitis) may require prolonged treatment, shifting the burden of management and monitoring to patients and their families, which along with the timeliness and adequacy of outpatient follow-up, may modify an individual’s readmission risk. Consequently, a “one-size fits all” approach to discharge counseling may not be successful across all conditions. Our study lays the groundwork for how these temporal relationships may be used by clinicians to counsel families regarding the likely readmission timeframe, based on infection, and to establish follow-up appointments ahead of the infection-specific “readmission window,” which may allow outpatient providers to intervene when readmission risk is greatest.

Consistent with prior literature, children with medical complexity in our study had increased frequency of 30-day, all-cause unplanned readmissions and LOS, compared with peers who did not have complexity.⁵ Readmission rates following hospitalizations for aspiration pneumonia were comparable to those reported by Thompson et al in their study examining rates of pneumonia in children with neurologic impairment.³⁵ In our current study, nearly 96% of readmissions following aspiration pneumonia hospitalizations were for children with medical complexity, and more than 58% of these readmissions were for aspiration pneumonia or respiratory illness. Future investigations should seek to explore factors contributing to readmissions in children with medical complexity and to evaluate whether interventions such as postdischarge coaching or discharge bundles could assist with reductions in healthcare resource use.^36,37

Despite a lack of clear association between readmissions and quality of care for children,³⁸ readmissions rates continue to be used as a quality measure for hospitalized patients. Within our present study, we found that achieving benchmark readmission rates for infection hospitalizations could lead to substantial reductions in readmissions; however, these reductions were seen across this relatively similar group of infection diagnoses, and hospitals may face greater challenges when attempting to achieve a 10th percentile readmission benchmark across a more expansive set of diagnoses. Despite these challenges, understanding the intricacies of readmissions may lead to improved readmission metrics and the systematic identification of avoidable readmissions, the goal of which is to enhance the quality of healthcare for hospitalized children.

Our findings should be interpreted in the context of several limitations. We defined our readmission benchmark at the 10th percentile using the IE database. Because an estimated 70% of hospitalizations for children occur within general hospitals,³⁹ we believe that our use of the IE database allowed for increased generalizability, though the broadening of our findings to nonacademic hospital settings may be less reliable. While we describe the 10th percentile readmission benchmark here, alternative benchmarks would result in different estimates of avoidable readmissions and associated readmission costs (Appendix Tables 2 and 3). The IE and NRD databases do not distinguish intensive care use. We tried to address this limitation through model adjustments using H-RISK, which is particularly helpful for adjusting for NICU admissions through use of the 27 All Patient Refined Diagnosis-Related Groups for neonatal conditions. Additionally, the NRD uses state-level data to derive national estimates and is not equipped to measure readmissions to hospitals in a different state, distinguish patient deaths occurring after discharge, or to examine the specific indication for readmission (ie, unable to assess if the readmission is related to a complication of the treatment plan, progression of the illness course, or for an unrelated reason). While sociodemographic and socioeconomic factors may affect readmissions, the NRD does not contain information on patients’ race/ethnicity, family/social attributes, or postdischarge follow-up health services, which are known to influence the need for readmission.

Despite these limitations, this study highlights future areas for research and potential opportunities for reducing readmission following infection hospitalizations. First, identifying hospital- and systems-based factors that contribute to readmission reductions at the best-performing hospitals may identify opportunities for hospitals with the highest readmission rates to achieve the rates of the best-performing hospitals. Second, conditions such as upper and lower respiratory tract infections, along with septicemia and gastroenteritis, may serve as prime targets for future investigation based on the estimated number of avoidable readmissions and potential cost savings associated with these conditions. Finally, future investigations that explore discharge counseling and follow-up tailored to the infection-specific temporal risk and patient complexity may identify opportunities for further readmission reductions.

CONCLUSION

Our study provides a broad look at readmissions following infection hospitalizations and highlights substantial variation in readmissions based on infection type. To improve hospital resource use for infections, future preventive measures could prioritize children with complex chronic conditions and those with specific diagnoses (eg, upper and lower respiratory tract infections).

Disclaimer

This information or content and conclusions are those of the authors and should not be construed as the official position or policy of, nor should any endorsements be inferred by, NIH or the US government.

Hospitalizations for infections are common in children, with respiratory illnesses, including pneumonia and bronchiolitis, among the most prevalent indications for hospitalization.^1,2 Infections are also among the most frequent indications for all-cause readmissions and for potentially preventable readmissions in children.^3-5 Beyond hospital resource use, infection hospitalizations and readmissions represent a considerable cause of life disruption for patients and their families.^6,7 While emerging evidence supports shortened durations of parenteral antibiotics before transitioning to oral therapy for some infections (eg, pyelonephritis, osteomyelitis),^8-10 other infections may require extended treatment courses for weeks. The risk of adverse outcomes (eg, complications of medical treatment, readmission risk) and burdens placed on patients and their families may therefore differ across infection types and extend well beyond the immediate hospitalization.

Although infections are common and pediatric providers are expected to have proficiency in managing infections, substantial variation in the management of common pediatric infections exists and is associated with adverse hospitalization outcomes, including increased readmission risk and healthcare costs.^11-18 Potentially avoidable resource use associated with hospital readmission from infection has led to adoption of hospital-level readmission metrics as indicators of the quality of healthcare delivery. For example, the Pediatric Quality Measures Program, established by the Children’s Health Insurance Program Reauthorization Act of 2009, has prioritized measurement of readmissions following hospitalization for lower respiratory tract infection.¹⁹ With government agencies increasingly using readmission metrics to assess quality of healthcare delivery, developing metrics that focus on these resource-intensive conditions is essential.

Because infections are a common and costly indication for hospital resource use and because substantial variation in management has been observed, promoting a broader understanding of infection-specific readmission rates is important for prioritizing readmission-reduction opportunities in children. This study’s objectives were the following: (1) to describe the prevalence and characteristics of infection hospitalizations in children and their associated readmissions and (2) to estimate the number of readmissions avoided and costs saved if all hospitals achieved the 10th percentile of the hospitals’ risk-adjusted readmission rate (ie, readmission benchmark).

METHODS

Study Design and Data Source

We performed a retrospective cohort analysis using the 2014 Agency for Healthcare Research and Quality (AHRQ) Nationwide Readmissions Database (NRD).²⁰ The 2014 NRD is an administrative database that contains information on inpatient stays from January 1, 2014, to December 31, 2014, for all payers and allows for weighted national estimates of readmissions for all US individuals. Data within NRD are aggregated from 22 geographically diverse states representing approximately one-half of the US population. NRD contains deidentified patient-level data with unique verified patient identifiers to track individuals within and across hospitals in a state. However, AHRQ guidelines specify that NRD cannot be used for reporting hospital-specific readmission rates. Thus, for the current study, the Inpatient Essentials (Children’s Hospital Association), or IE, database was used to measure hospital-level readmission rates and to distinguish benchmark readmission rates for individual infection diagnoses.²¹ The IE database is composed of 90 children’s hospitals distributed throughout all regions of the United States. The inclusion of free-standing children’s hospitals and children’s hospitals within adult hospitals allows for comparisons and benchmarking across hospitals on multiple metrics, including readmissions.

Study Population

Children 0 to 17 years of age with a primary diagnosis at the index admission for infection between January 1, 2014, and November 30, 2014, were included. The end date of November 30, 2014, allowed for a full 30-day readmission window for all index admissions. We excluded index admissions that resulted in transfer to another acute care hospital or in-hospital mortality. Additionally, we excluded index admissions of children who had hematologic or immunologic conditions, malignancy, or history of bone marrow and solid-organ transplant, using the classification system for complex chronic conditions (CCCs) from Feudtner et al.²² Due to the high likelihood of immunosuppression in patients with these conditions, children may have nuanced experiences with illness severity, trajectory, and treatment associated with infection that place them at increased risk for nonpreventable readmission.

Main Exposure

The main exposure was infection type during the index admission. Condition-specific index admissions were identified using AHRQ’s Clinical Classifications Software (CCS) categories.²³ CCS is a classification schema that categorizes the greater than 14,000 International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) diagnosis codes and 3,900 ICD-9-CM procedure codes into clinically meaningful categories of 295 diagnosis (including mental health codes and E-codes) and 231 procedural groupings. Twenty-two groupings indicative of infection were distinguished and used for the current study. Examples of infections included aspiration pneumonia, pneumonia, bronchiolitis, and sexually transmitted infection. We combined related CCS categories when possible for ease of interpretation and presentation of data (Appendix Table 1).

Main Outcome Measure

The main outcome measure was 30-day hospital readmission. Readmission was defined as all-cause, unplanned admission within 30 days following discharge from a preceding hospitalization. Planned hospital readmissions were identified and excluded using methods from AHRQ’s Pediatric All-Condition Readmission Measure.²⁴ We defined a same-cause return as a return with the same CCS infection category as the index admission. Costs associated with readmissions were estimated from charges using hospital-specific cost-to-charge ratios provided with NRD.

Patient Demographic and Clinical Characteristics

Patient demographic characteristics included age at index admission (<1 year, 1-5 years, 6-9 years, 10-14 years, and 15-18 years), sex, payer (ie, government, private, other), and discharge disposition (ie, routine, home health, other). We assessed all patients for medical complexity, as defined by the presence of at least one CCC, and we reported the categories of CCCs by organ system involved.²² Otherwise, patients were identified as without medical complexity.

Statistical Analysis

We summarized continuous variables with medians and interquartile ranges and categorical variables with frequencies and percentages. To develop benchmark readmission rates for each infection type, we used generalized linear mixed models with random intercepts for each hospital in the IE database. For each infection type, the benchmark readmission rate was defined as the 10th percentile of hospitals’ risk-adjusted readmission rates. The 10th percentile was chosen to identify the best performing 10% of hospitals (ie, hospitals with the lowest readmission rates). Because children with medical complexity account for a large proportion of hospital resource use and are at high risk for readmission,^4,25 we developed benchmarks stratified by presence/absence of a CCC (ie, with complexity vs without complexity). Models were adjusted for severity of illness using the Hospitalization Resource Intensity Score for Kids (H-RISK),²⁶ a scoring system that assigns relative weights for each All Patient Refined Diagnosis-Related Group (3M Corporation) and severity of illness level, and each hospital’s risk-adjusted readmission rate was determined.

With use of weights to achieve national estimates of index admissions and readmissions, we determined the number of potentially avoidable readmissions by calculating the number of readmissions observed in the NRD that would not occur if all hospitals achieved readmission rates equal to the 10th percentile. Avoidable costs were estimated by multiplying the number of potentially avoidable readmissions by the mean cost of a readmission for infections of that type. Estimates of avoidable readmissions and costs were stratified by medical complexity. In addition to describing estimates at the 10th percentile benchmark, we similarly developed estimates of potentially avoidable readmissions and avoidable costs for the 5th and 25th percentiles, which are presented within Appendix Table 2 (children without complexity) and Appendix Table 3 (children with complexity).

All statistical analyses were performed using SAS version 9.4 (SAS Institute), and P values <.001 were considered statistically significant due to the large sample size. The Office of Research Integrity at Children’s Mercy Hospital deemed this study exempt from institutional board review.

RESULTS

Characteristics of the Study Population

The study included 380,067 index admissions for infection and an accompanying 18,469 unplanned all-cause readmissions over the 30 days following discharge (readmission rate, 4.9%; Table 1). Children ages 1 to 5 years accounted for the largest percentage (32.9%) of index hospitalizations, followed by infants younger than 1 year (30.3%). The readmission rate by age group was highest for infants younger than 1 year, compared with rates among all other age groups (5.6% among infants < 1 year vs 4.4%-4.7% for other age groups; P < .001). In the overall cohort, 16.2% of admissions included patients with a CCC. Children with medical complexity had higher readmission rates than those without medical complexity (no CCC, 3.2%; 1 CCC, 9.2%; 2+ CCCs, 18.9%). A greater percentage of children experiencing a readmission had government insurance (63.0% vs 59.2%; P < .001) and received home health nursing (10.1% vs 2.7%; P < .001) following the index encounter.

Children Without Complexity

Index Admissions and 30-day Readmissions

Among patients without medical complexity, index admissions occurred most frequently for pneumonia (n = 54,717), bronchiolitis (n = 53,959), and appendicitis (n = 45,036) (Figure 1). The median length of stay (LOS) for index admissions ranged from 1 to 5 days (Table 2), with septic arthritis and osteomyelitis having the longest median LOS at 5 (IQR, 3-7) days.

Thirty-day readmission rates varied substantially by infection at the index admission (range, 1.5% for sexually transmitted infection to 8.6% for peritonitis) (Figure 1). The median LOS for 30-day readmissions varied from 2 to 7 days (Table 2), while the median number of days to readmission varied substantially by infection type (range, 4 days for bacterial infection [site unspecified] to 24 days for sexually transmitted infections). Among the top five indications for admission for children without complexity, 14.9% to 52.8% of readmissions were for the same cause as the index admission; however, many additional returns were likely related to the index admission (Appendix Table 4). For example, among other return reasons, an additional 992 (61.7%) readmissions following appendicitis hospitalizations were for complications of surgical procedures or medical care, peritonitis, intestinal obstruction, and abdominal pain.

Impact of Achieving Readmission Benchmarks

Among children without complexity, readmission benchmarks (ie, the 10th percentile of readmission rates across hospitals) ranged from 0% to 26.7% (Figure 2). An estimated 54.7% of readmissions (n = 5,507) could potentially be reduced if hospitals achieved infection-specific benchmark readmission rates, which could result in an estimated $44.5 million in savings. Pneumonia, bronchiolitis, gastroenteritis, and upper respiratory tract infections were among conditions with the greatest potential reductions in readmissions and costs if a 10th percentile benchmark was achieved.

Children With Medical Complexity

Index Admissions and 30-day Readmissions

Among patients with complexity, index admissions occurred most frequently for pneumonia (n = 14,344), bronchiolitis (n = 8,618), and upper respiratory tract infection (n = 6,407) (Figure 1). The median LOS for index admissions ranged from 1 to 9 days (Table 2), with septicemia and CNS infections having the longest median LOS at 9 days.

Thirty-day readmission rates varied substantially by the type of infection at the index admission (range, 0% for sexually transmitted infection to 21.6% for aspiration pneumonia) (Figure 1). The median LOS for 30-day readmissions varied from 2 to 14 days (Table 2), and the median number of days to readmission varied substantially by infection type (range, 6 days for tonsillitis to 23 days for other infection). Among the top five indications for admission for medically complex children, 8% to 40.4% of readmissions were for the same cause as the index admission (Appendix Table 4). As with the children without complexity, additional returns were likely related to the index admission.

Impact of Achieving Readmission Benchmarks

Among children with medical complexity, readmission benchmarks ranged from 0% to 30.3% (Figure 2). An estimated 42.6% of readmissions (n = 3,576) could potentially be reduced if hospitals achieved infection-specific benchmark readmission rates, which could result in an estimated $70.8 million in savings. Pneumonia, bronchiolitis, septicemia, and upper respiratory tract infections were among conditions with the greatest potential reductions in readmissions and costs if the benchmarks were achieved.

DISCUSSION

The current study uncovered new findings regarding unplanned readmissions following index infection hospitalizations for children. In particular, readmission rates and time to readmission varied substantially by infection subtype. The study also identified priority infections and unique considerations for children with CCCs, all of which may help maximize the value of readmission metrics aimed at advancing hospital quality and reducing costs for infection hospitalizations in children. If all hospitals achieved the readmission rates of the best performing hospitals, 42.6% to 54.7% fewer readmissions could be realized with associated cost savings.

Nationally, studies have reported 30-day, all-cause unplanned readmission rates of 6.2% to 10.3%.^5,27 In our current study we observed an overall readmission rate of 4.9% across all infectious conditions; however, readmission rates varied substantially by condition, with upper and lower respiratory tract infections, septicemia, and gastroenteritis among infections with the greatest number of potentially avoidable readmissions based on achievement of readmission benchmarks. Currently, pediatric-specific all-cause and lower respiratory tract infection readmission metrics have been developed with the aim of improving quality of care and reducing healthcare expenditures.²⁸ Future readmission studies and metrics may prioritize conditions such as septicemia, gastroenteritis, and other respiratory tract infections. Our current study demonstrated that many readmissions following infection hospitalizations were associated with the same CCS category or within a related CCS category (eg, complications of surgical procedures or medical care, appendicitis, peritonitis, intestinal obstruction, and abdominal pain constituted the top five indications for readmission for appendicitis, whereas respiratory illnesses constituted the top five indications for readmissions for pneumonia). While this current study cannot clarify this relationship further, and the “avoidability” of unplanned readmissions is debated,^29-31 our findings suggest that future investigations might focus on identifying whether condition-specific interventions during the index admission could mitigate some readmissions.

While the LOS of the index admission and the readmission were similar for most infection subtypes, we observed substantial variability in the temporal risk for readmission by infection subtype. Our observations complement studies exploring the timing of readmissions for other pediatric conditions.^32-34 In particular, our findings further highlight that the composition and interaction of factors influencing infection readmissions may vary by condition. Infections represent a diverse group of conditions, with treatment courses that vary in need for parenteral antibiotics, ability to tailor treatment based on organism and susceptibilities, and length of treatment. While treatment for some infections may be accomplished, or nearly accomplished, prior to discharge, other infections (eg, osteomyelitis) may require prolonged treatment, shifting the burden of management and monitoring to patients and their families, which along with the timeliness and adequacy of outpatient follow-up, may modify an individual’s readmission risk. Consequently, a “one-size fits all” approach to discharge counseling may not be successful across all conditions. Our study lays the groundwork for how these temporal relationships may be used by clinicians to counsel families regarding the likely readmission timeframe, based on infection, and to establish follow-up appointments ahead of the infection-specific “readmission window,” which may allow outpatient providers to intervene when readmission risk is greatest.

Consistent with prior literature, children with medical complexity in our study had increased frequency of 30-day, all-cause unplanned readmissions and LOS, compared with peers who did not have complexity.⁵ Readmission rates following hospitalizations for aspiration pneumonia were comparable to those reported by Thompson et al in their study examining rates of pneumonia in children with neurologic impairment.³⁵ In our current study, nearly 96% of readmissions following aspiration pneumonia hospitalizations were for children with medical complexity, and more than 58% of these readmissions were for aspiration pneumonia or respiratory illness. Future investigations should seek to explore factors contributing to readmissions in children with medical complexity and to evaluate whether interventions such as postdischarge coaching or discharge bundles could assist with reductions in healthcare resource use.^36,37

Despite a lack of clear association between readmissions and quality of care for children,³⁸ readmissions rates continue to be used as a quality measure for hospitalized patients. Within our present study, we found that achieving benchmark readmission rates for infection hospitalizations could lead to substantial reductions in readmissions; however, these reductions were seen across this relatively similar group of infection diagnoses, and hospitals may face greater challenges when attempting to achieve a 10th percentile readmission benchmark across a more expansive set of diagnoses. Despite these challenges, understanding the intricacies of readmissions may lead to improved readmission metrics and the systematic identification of avoidable readmissions, the goal of which is to enhance the quality of healthcare for hospitalized children.

Our findings should be interpreted in the context of several limitations. We defined our readmission benchmark at the 10th percentile using the IE database. Because an estimated 70% of hospitalizations for children occur within general hospitals,³⁹ we believe that our use of the IE database allowed for increased generalizability, though the broadening of our findings to nonacademic hospital settings may be less reliable. While we describe the 10th percentile readmission benchmark here, alternative benchmarks would result in different estimates of avoidable readmissions and associated readmission costs (Appendix Tables 2 and 3). The IE and NRD databases do not distinguish intensive care use. We tried to address this limitation through model adjustments using H-RISK, which is particularly helpful for adjusting for NICU admissions through use of the 27 All Patient Refined Diagnosis-Related Groups for neonatal conditions. Additionally, the NRD uses state-level data to derive national estimates and is not equipped to measure readmissions to hospitals in a different state, distinguish patient deaths occurring after discharge, or to examine the specific indication for readmission (ie, unable to assess if the readmission is related to a complication of the treatment plan, progression of the illness course, or for an unrelated reason). While sociodemographic and socioeconomic factors may affect readmissions, the NRD does not contain information on patients’ race/ethnicity, family/social attributes, or postdischarge follow-up health services, which are known to influence the need for readmission.

Despite these limitations, this study highlights future areas for research and potential opportunities for reducing readmission following infection hospitalizations. First, identifying hospital- and systems-based factors that contribute to readmission reductions at the best-performing hospitals may identify opportunities for hospitals with the highest readmission rates to achieve the rates of the best-performing hospitals. Second, conditions such as upper and lower respiratory tract infections, along with septicemia and gastroenteritis, may serve as prime targets for future investigation based on the estimated number of avoidable readmissions and potential cost savings associated with these conditions. Finally, future investigations that explore discharge counseling and follow-up tailored to the infection-specific temporal risk and patient complexity may identify opportunities for further readmission reductions.

CONCLUSION

Our study provides a broad look at readmissions following infection hospitalizations and highlights substantial variation in readmissions based on infection type. To improve hospital resource use for infections, future preventive measures could prioritize children with complex chronic conditions and those with specific diagnoses (eg, upper and lower respiratory tract infections).

Disclaimer

This information or content and conclusions are those of the authors and should not be construed as the official position or policy of, nor should any endorsements be inferred by, NIH or the US government.

Hospitalizations for infections are common in children, with respiratory illnesses, including pneumonia and bronchiolitis, among the most prevalent indications for hospitalization.^1,2 Infections are also among the most frequent indications for all-cause readmissions and for potentially preventable readmissions in children.^3-5 Beyond hospital resource use, infection hospitalizations and readmissions represent a considerable cause of life disruption for patients and their families.^6,7 While emerging evidence supports shortened durations of parenteral antibiotics before transitioning to oral therapy for some infections (eg, pyelonephritis, osteomyelitis),^8-10 other infections may require extended treatment courses for weeks. The risk of adverse outcomes (eg, complications of medical treatment, readmission risk) and burdens placed on patients and their families may therefore differ across infection types and extend well beyond the immediate hospitalization.

Although infections are common and pediatric providers are expected to have proficiency in managing infections, substantial variation in the management of common pediatric infections exists and is associated with adverse hospitalization outcomes, including increased readmission risk and healthcare costs.^11-18 Potentially avoidable resource use associated with hospital readmission from infection has led to adoption of hospital-level readmission metrics as indicators of the quality of healthcare delivery. For example, the Pediatric Quality Measures Program, established by the Children’s Health Insurance Program Reauthorization Act of 2009, has prioritized measurement of readmissions following hospitalization for lower respiratory tract infection.¹⁹ With government agencies increasingly using readmission metrics to assess quality of healthcare delivery, developing metrics that focus on these resource-intensive conditions is essential.

Because infections are a common and costly indication for hospital resource use and because substantial variation in management has been observed, promoting a broader understanding of infection-specific readmission rates is important for prioritizing readmission-reduction opportunities in children. This study’s objectives were the following: (1) to describe the prevalence and characteristics of infection hospitalizations in children and their associated readmissions and (2) to estimate the number of readmissions avoided and costs saved if all hospitals achieved the 10th percentile of the hospitals’ risk-adjusted readmission rate (ie, readmission benchmark).

METHODS

Study Design and Data Source

We performed a retrospective cohort analysis using the 2014 Agency for Healthcare Research and Quality (AHRQ) Nationwide Readmissions Database (NRD).²⁰ The 2014 NRD is an administrative database that contains information on inpatient stays from January 1, 2014, to December 31, 2014, for all payers and allows for weighted national estimates of readmissions for all US individuals. Data within NRD are aggregated from 22 geographically diverse states representing approximately one-half of the US population. NRD contains deidentified patient-level data with unique verified patient identifiers to track individuals within and across hospitals in a state. However, AHRQ guidelines specify that NRD cannot be used for reporting hospital-specific readmission rates. Thus, for the current study, the Inpatient Essentials (Children’s Hospital Association), or IE, database was used to measure hospital-level readmission rates and to distinguish benchmark readmission rates for individual infection diagnoses.²¹ The IE database is composed of 90 children’s hospitals distributed throughout all regions of the United States. The inclusion of free-standing children’s hospitals and children’s hospitals within adult hospitals allows for comparisons and benchmarking across hospitals on multiple metrics, including readmissions.

Study Population

Children 0 to 17 years of age with a primary diagnosis at the index admission for infection between January 1, 2014, and November 30, 2014, were included. The end date of November 30, 2014, allowed for a full 30-day readmission window for all index admissions. We excluded index admissions that resulted in transfer to another acute care hospital or in-hospital mortality. Additionally, we excluded index admissions of children who had hematologic or immunologic conditions, malignancy, or history of bone marrow and solid-organ transplant, using the classification system for complex chronic conditions (CCCs) from Feudtner et al.²² Due to the high likelihood of immunosuppression in patients with these conditions, children may have nuanced experiences with illness severity, trajectory, and treatment associated with infection that place them at increased risk for nonpreventable readmission.

Main Exposure

The main exposure was infection type during the index admission. Condition-specific index admissions were identified using AHRQ’s Clinical Classifications Software (CCS) categories.²³ CCS is a classification schema that categorizes the greater than 14,000 International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) diagnosis codes and 3,900 ICD-9-CM procedure codes into clinically meaningful categories of 295 diagnosis (including mental health codes and E-codes) and 231 procedural groupings. Twenty-two groupings indicative of infection were distinguished and used for the current study. Examples of infections included aspiration pneumonia, pneumonia, bronchiolitis, and sexually transmitted infection. We combined related CCS categories when possible for ease of interpretation and presentation of data (Appendix Table 1).

Main Outcome Measure

The main outcome measure was 30-day hospital readmission. Readmission was defined as all-cause, unplanned admission within 30 days following discharge from a preceding hospitalization. Planned hospital readmissions were identified and excluded using methods from AHRQ’s Pediatric All-Condition Readmission Measure.²⁴ We defined a same-cause return as a return with the same CCS infection category as the index admission. Costs associated with readmissions were estimated from charges using hospital-specific cost-to-charge ratios provided with NRD.

Patient Demographic and Clinical Characteristics

Patient demographic characteristics included age at index admission (<1 year, 1-5 years, 6-9 years, 10-14 years, and 15-18 years), sex, payer (ie, government, private, other), and discharge disposition (ie, routine, home health, other). We assessed all patients for medical complexity, as defined by the presence of at least one CCC, and we reported the categories of CCCs by organ system involved.²² Otherwise, patients were identified as without medical complexity.

Statistical Analysis

We summarized continuous variables with medians and interquartile ranges and categorical variables with frequencies and percentages. To develop benchmark readmission rates for each infection type, we used generalized linear mixed models with random intercepts for each hospital in the IE database. For each infection type, the benchmark readmission rate was defined as the 10th percentile of hospitals’ risk-adjusted readmission rates. The 10th percentile was chosen to identify the best performing 10% of hospitals (ie, hospitals with the lowest readmission rates). Because children with medical complexity account for a large proportion of hospital resource use and are at high risk for readmission,^4,25 we developed benchmarks stratified by presence/absence of a CCC (ie, with complexity vs without complexity). Models were adjusted for severity of illness using the Hospitalization Resource Intensity Score for Kids (H-RISK),²⁶ a scoring system that assigns relative weights for each All Patient Refined Diagnosis-Related Group (3M Corporation) and severity of illness level, and each hospital’s risk-adjusted readmission rate was determined.

With use of weights to achieve national estimates of index admissions and readmissions, we determined the number of potentially avoidable readmissions by calculating the number of readmissions observed in the NRD that would not occur if all hospitals achieved readmission rates equal to the 10th percentile. Avoidable costs were estimated by multiplying the number of potentially avoidable readmissions by the mean cost of a readmission for infections of that type. Estimates of avoidable readmissions and costs were stratified by medical complexity. In addition to describing estimates at the 10th percentile benchmark, we similarly developed estimates of potentially avoidable readmissions and avoidable costs for the 5th and 25th percentiles, which are presented within Appendix Table 2 (children without complexity) and Appendix Table 3 (children with complexity).

All statistical analyses were performed using SAS version 9.4 (SAS Institute), and P values <.001 were considered statistically significant due to the large sample size. The Office of Research Integrity at Children’s Mercy Hospital deemed this study exempt from institutional board review.

RESULTS

Characteristics of the Study Population

The study included 380,067 index admissions for infection and an accompanying 18,469 unplanned all-cause readmissions over the 30 days following discharge (readmission rate, 4.9%; Table 1). Children ages 1 to 5 years accounted for the largest percentage (32.9%) of index hospitalizations, followed by infants younger than 1 year (30.3%). The readmission rate by age group was highest for infants younger than 1 year, compared with rates among all other age groups (5.6% among infants < 1 year vs 4.4%-4.7% for other age groups; P < .001). In the overall cohort, 16.2% of admissions included patients with a CCC. Children with medical complexity had higher readmission rates than those without medical complexity (no CCC, 3.2%; 1 CCC, 9.2%; 2+ CCCs, 18.9%). A greater percentage of children experiencing a readmission had government insurance (63.0% vs 59.2%; P < .001) and received home health nursing (10.1% vs 2.7%; P < .001) following the index encounter.

Children Without Complexity

Index Admissions and 30-day Readmissions

Among patients without medical complexity, index admissions occurred most frequently for pneumonia (n = 54,717), bronchiolitis (n = 53,959), and appendicitis (n = 45,036) (Figure 1). The median length of stay (LOS) for index admissions ranged from 1 to 5 days (Table 2), with septic arthritis and osteomyelitis having the longest median LOS at 5 (IQR, 3-7) days.

Thirty-day readmission rates varied substantially by infection at the index admission (range, 1.5% for sexually transmitted infection to 8.6% for peritonitis) (Figure 1). The median LOS for 30-day readmissions varied from 2 to 7 days (Table 2), while the median number of days to readmission varied substantially by infection type (range, 4 days for bacterial infection [site unspecified] to 24 days for sexually transmitted infections). Among the top five indications for admission for children without complexity, 14.9% to 52.8% of readmissions were for the same cause as the index admission; however, many additional returns were likely related to the index admission (Appendix Table 4). For example, among other return reasons, an additional 992 (61.7%) readmissions following appendicitis hospitalizations were for complications of surgical procedures or medical care, peritonitis, intestinal obstruction, and abdominal pain.

Impact of Achieving Readmission Benchmarks

Among children without complexity, readmission benchmarks (ie, the 10th percentile of readmission rates across hospitals) ranged from 0% to 26.7% (Figure 2). An estimated 54.7% of readmissions (n = 5,507) could potentially be reduced if hospitals achieved infection-specific benchmark readmission rates, which could result in an estimated $44.5 million in savings. Pneumonia, bronchiolitis, gastroenteritis, and upper respiratory tract infections were among conditions with the greatest potential reductions in readmissions and costs if a 10th percentile benchmark was achieved.

Children With Medical Complexity

Index Admissions and 30-day Readmissions

Among patients with complexity, index admissions occurred most frequently for pneumonia (n = 14,344), bronchiolitis (n = 8,618), and upper respiratory tract infection (n = 6,407) (Figure 1). The median LOS for index admissions ranged from 1 to 9 days (Table 2), with septicemia and CNS infections having the longest median LOS at 9 days.

Thirty-day readmission rates varied substantially by the type of infection at the index admission (range, 0% for sexually transmitted infection to 21.6% for aspiration pneumonia) (Figure 1). The median LOS for 30-day readmissions varied from 2 to 14 days (Table 2), and the median number of days to readmission varied substantially by infection type (range, 6 days for tonsillitis to 23 days for other infection). Among the top five indications for admission for medically complex children, 8% to 40.4% of readmissions were for the same cause as the index admission (Appendix Table 4). As with the children without complexity, additional returns were likely related to the index admission.

Impact of Achieving Readmission Benchmarks

Among children with medical complexity, readmission benchmarks ranged from 0% to 30.3% (Figure 2). An estimated 42.6% of readmissions (n = 3,576) could potentially be reduced if hospitals achieved infection-specific benchmark readmission rates, which could result in an estimated $70.8 million in savings. Pneumonia, bronchiolitis, septicemia, and upper respiratory tract infections were among conditions with the greatest potential reductions in readmissions and costs if the benchmarks were achieved.

DISCUSSION

The current study uncovered new findings regarding unplanned readmissions following index infection hospitalizations for children. In particular, readmission rates and time to readmission varied substantially by infection subtype. The study also identified priority infections and unique considerations for children with CCCs, all of which may help maximize the value of readmission metrics aimed at advancing hospital quality and reducing costs for infection hospitalizations in children. If all hospitals achieved the readmission rates of the best performing hospitals, 42.6% to 54.7% fewer readmissions could be realized with associated cost savings.

Nationally, studies have reported 30-day, all-cause unplanned readmission rates of 6.2% to 10.3%.^5,27 In our current study we observed an overall readmission rate of 4.9% across all infectious conditions; however, readmission rates varied substantially by condition, with upper and lower respiratory tract infections, septicemia, and gastroenteritis among infections with the greatest number of potentially avoidable readmissions based on achievement of readmission benchmarks. Currently, pediatric-specific all-cause and lower respiratory tract infection readmission metrics have been developed with the aim of improving quality of care and reducing healthcare expenditures.²⁸ Future readmission studies and metrics may prioritize conditions such as septicemia, gastroenteritis, and other respiratory tract infections. Our current study demonstrated that many readmissions following infection hospitalizations were associated with the same CCS category or within a related CCS category (eg, complications of surgical procedures or medical care, appendicitis, peritonitis, intestinal obstruction, and abdominal pain constituted the top five indications for readmission for appendicitis, whereas respiratory illnesses constituted the top five indications for readmissions for pneumonia). While this current study cannot clarify this relationship further, and the “avoidability” of unplanned readmissions is debated,^29-31 our findings suggest that future investigations might focus on identifying whether condition-specific interventions during the index admission could mitigate some readmissions.

While the LOS of the index admission and the readmission were similar for most infection subtypes, we observed substantial variability in the temporal risk for readmission by infection subtype. Our observations complement studies exploring the timing of readmissions for other pediatric conditions.^32-34 In particular, our findings further highlight that the composition and interaction of factors influencing infection readmissions may vary by condition. Infections represent a diverse group of conditions, with treatment courses that vary in need for parenteral antibiotics, ability to tailor treatment based on organism and susceptibilities, and length of treatment. While treatment for some infections may be accomplished, or nearly accomplished, prior to discharge, other infections (eg, osteomyelitis) may require prolonged treatment, shifting the burden of management and monitoring to patients and their families, which along with the timeliness and adequacy of outpatient follow-up, may modify an individual’s readmission risk. Consequently, a “one-size fits all” approach to discharge counseling may not be successful across all conditions. Our study lays the groundwork for how these temporal relationships may be used by clinicians to counsel families regarding the likely readmission timeframe, based on infection, and to establish follow-up appointments ahead of the infection-specific “readmission window,” which may allow outpatient providers to intervene when readmission risk is greatest.

Consistent with prior literature, children with medical complexity in our study had increased frequency of 30-day, all-cause unplanned readmissions and LOS, compared with peers who did not have complexity.⁵ Readmission rates following hospitalizations for aspiration pneumonia were comparable to those reported by Thompson et al in their study examining rates of pneumonia in children with neurologic impairment.³⁵ In our current study, nearly 96% of readmissions following aspiration pneumonia hospitalizations were for children with medical complexity, and more than 58% of these readmissions were for aspiration pneumonia or respiratory illness. Future investigations should seek to explore factors contributing to readmissions in children with medical complexity and to evaluate whether interventions such as postdischarge coaching or discharge bundles could assist with reductions in healthcare resource use.^36,37

Despite a lack of clear association between readmissions and quality of care for children,³⁸ readmissions rates continue to be used as a quality measure for hospitalized patients. Within our present study, we found that achieving benchmark readmission rates for infection hospitalizations could lead to substantial reductions in readmissions; however, these reductions were seen across this relatively similar group of infection diagnoses, and hospitals may face greater challenges when attempting to achieve a 10th percentile readmission benchmark across a more expansive set of diagnoses. Despite these challenges, understanding the intricacies of readmissions may lead to improved readmission metrics and the systematic identification of avoidable readmissions, the goal of which is to enhance the quality of healthcare for hospitalized children.

Our findings should be interpreted in the context of several limitations. We defined our readmission benchmark at the 10th percentile using the IE database. Because an estimated 70% of hospitalizations for children occur within general hospitals,³⁹ we believe that our use of the IE database allowed for increased generalizability, though the broadening of our findings to nonacademic hospital settings may be less reliable. While we describe the 10th percentile readmission benchmark here, alternative benchmarks would result in different estimates of avoidable readmissions and associated readmission costs (Appendix Tables 2 and 3). The IE and NRD databases do not distinguish intensive care use. We tried to address this limitation through model adjustments using H-RISK, which is particularly helpful for adjusting for NICU admissions through use of the 27 All Patient Refined Diagnosis-Related Groups for neonatal conditions. Additionally, the NRD uses state-level data to derive national estimates and is not equipped to measure readmissions to hospitals in a different state, distinguish patient deaths occurring after discharge, or to examine the specific indication for readmission (ie, unable to assess if the readmission is related to a complication of the treatment plan, progression of the illness course, or for an unrelated reason). While sociodemographic and socioeconomic factors may affect readmissions, the NRD does not contain information on patients’ race/ethnicity, family/social attributes, or postdischarge follow-up health services, which are known to influence the need for readmission.

Despite these limitations, this study highlights future areas for research and potential opportunities for reducing readmission following infection hospitalizations. First, identifying hospital- and systems-based factors that contribute to readmission reductions at the best-performing hospitals may identify opportunities for hospitals with the highest readmission rates to achieve the rates of the best-performing hospitals. Second, conditions such as upper and lower respiratory tract infections, along with septicemia and gastroenteritis, may serve as prime targets for future investigation based on the estimated number of avoidable readmissions and potential cost savings associated with these conditions. Finally, future investigations that explore discharge counseling and follow-up tailored to the infection-specific temporal risk and patient complexity may identify opportunities for further readmission reductions.

CONCLUSION

Our study provides a broad look at readmissions following infection hospitalizations and highlights substantial variation in readmissions based on infection type. To improve hospital resource use for infections, future preventive measures could prioritize children with complex chronic conditions and those with specific diagnoses (eg, upper and lower respiratory tract infections).

Disclaimer

This information or content and conclusions are those of the authors and should not be construed as the official position or policy of, nor should any endorsements be inferred by, NIH or the US government.

Confirmed cases of coronavirus disease 2019 (COVID-19) exceed 111 million, and the disease is responsible for approximately 2.4 million deaths worldwide.¹ In the United States, 28 million cases of COVID-19 have been reported, and the disease has caused more than 497,000 deaths.² The clinical presentation of COVID-19 varies widely, with the most severe presentation characterized by acute respiratory distress syndrome and a marked systemic inflammatory response. Corticosteroids have emerged as a potential therapeutic option in a subset of patients. Results from the recently published RECOVERY trial suggest a substantial mortality benefit of dexamethasone in patients who require mechanical ventilation, with a risk reduction of approximately 33%.³ In addition, a recent large retrospective study demonstrated a reduction in the risk of mechanical ventilation or mortality with corticosteroids in a prespecified subset of patients with C-reactive protein (CRP) ≥20 mg/dL, which indicates a high burden of inflammation.⁴

Some patients with severe COVID-19 experience a positive feedback cascade of proinflammatory cytokines, called the cytokine storm, which can worsen lung injury and, in some cases, progress to vasodilatory shock and multiorgan failure.⁵ This complication’s cytokine cascade includes interleukin (IL) 6, IL-1β, and CC chemokine ligand 3 (CCL3), which are released by airway macrophages and all of which are heavily implicated in the maladaptive forms of immune response to COVID-19.^6,7 The cytokine IL-6 is the primary signal for the production of CRP, and corticosteroids have been shown, both in vitro and in vivo, to reduce the production of IL-6 and other cytokines by airway macrophages.⁶ Levels of CRP have been shown to correlate with outcomes in COVID-19 and bacterial pneumonias.^7,8 Reduction in CRP levels following the institution of therapy, known as CRP response, has been shown to predict outcomes in other inflammatory conditions, such as osteomyelitis, hidradenitis suppurativa, and some cases of bacterial pneumonia.^8-10 Similar CRP response in hemophagocytic lymphohistiocytosis, an entity which closely resembles cytokine storm syndrome, has been shown to correlate with disease activity in patients following treatment with an IL-1 antagonist.¹¹ Whether the CRP response as a response to therapeutics in COVID-19 is associated with improved outcomes remains unknown.

Laboratory measurement of CRP levels offers several advantages over the measurement of interleukins. Notably, the half-life of CRP is approximately 19 hours, which is comparable across different age groups and inflammatory conditions because its concentration depends primarily on synthesis in the liver, and a decreased level suggests decreased stimulus for synthesis.⁸ This makes CRP a useful biomarker to assess response to therapy, in contrast to interleukins, which have short half-lives, are variable in heterogeneous populations, and can be difficult to measure. In addition, CRP measurement is rapid and relatively inexpensive.

We hypothesized that reduction in CRP levels by 50% or more within 72 hours after the initiation of corticosteroids in patients with COVID-19 is associated with reduced inpatient mortality and may be an early indicator of therapeutic response.

METHODS

Study Participants

In this retrospective cohort study, we reviewed all adult patients admitted to Montefiore Medical Center (Bronx, New York) for COVID-19 between March 10, 2020, and May 2, 2020. Patients must have been discharged (alive or deceased) by the administrative censor date (May 2, 2020) to be included. Patients who died within the first 48 hours of admission were excluded to allow sufficient time for corticosteroid treatment to take effect. For inclusion in the corticosteroid group, patients needed to have received at least 2 consecutive days of corticosteroid treatment beginning within the first 48 hours of admission with a total daily dose of 0.5 mg/kg prednisone equivalent or greater. Patients who received treatment-dose corticosteroids later in the hospital course were excluded (Appendix Figure).

Comparison Group and Outcome

We examined trends in CRP levels for patients who received corticosteroids vs trends among patients who did not receive corticosteroids. In addition, among patients who were treated with corticosteroids, we compared the inpatient mortality of those who did have a reduction in CRP level after treatment with inpatient mortality of those who did not have a reduction in CRP level after treatment. First, CRP level trends over time were examined in all patients, and compared between those who received corticosteroid treatment and those who did not. Then, patients who received corticosteroids were categorized based on changes in CRP levels after beginning corticosteroids. The first CRP level obtained during the first 48 hours of admission was used as the initial CRP level. For each patient, the last CRP level within the 72 hours after initiation of treatment was used to calculate the change in CRP level from admission. A patient was considered to be a “CRP responder” if their CRP level decreased by 50% or more within 72 hours after treatment and a “CRP nonresponder” if their CRP level did not drop by at least 50% within 72 hours of treatment. Patients who did not have a CRP level within the initial 48 hours of admission or a subsequent CRP measured in the 72 hours after treatment were considered to have an “undetermined CRP response” and excluded from the mortality analysis.

We observed a rise in CRP starting around day 6 among patients treated with corticosteroids and performed a post hoc analysis to determine if this was due to a selection effect whereby patients staying in the hospital longer had higher CRP levels or represented actual rise. In order to address this, we performed a stratified analysis comparing the trends in CRP levels among patients with a length of stay (LOS) of 7 or more days with trends among those with an LOS less than 7 days.

Statistical Analysis

To characterize differences in patients who received corticosteroids and those who did not, we examined their demographic, clinical characteristics, and admission laboratory values, using chi-square test for categorical variables and Kruskal-Wallis test for continuous variables (Table 1). The change in CRP levels from day 0 (presentation to the hospital) in both groups was plotted in a time-series analysis. For each day in the time series, the 95% CIs for the changes in CRP were computed using the t statistic for the corresponding distribution. The Kruskal-Wallis test was used to assess the significance of differences between groups at 72 hours after initiation of treatment.

After categorizing patients by CRP response, we compared demographic, clinical, and laboratory characteristics of patients who were CRP responsive with those of patients who were not, using the same tests of statistical inference mentioned above. To compare time to inpatient mortality differences between CRP response groups, Kaplan-Meier survival curves were generated and statistical significance determined via log-rank test. Univariable logistic regression was used to estimate the odds ratio of inpatient mortality between comparison groups in an unadjusted analysis. Last, to examine the independent association between CRP response and mortality, we constructed a multivariate model that included variables that were significantly associated with mortality in univariable analysis and considered to be important potential confounders by the authors. Details on variable selection for the model are listed in Appendix Table 1.

Data Collection

Data were directly extracted from our center’s electronic health record system. Data processing and recoding was performed using the Python programming language (version 2.7.17) and data analysis was done using Stata 12 (StataCorp LLC; 2011). This study was approved by the institutional review board of the Albert Einstein College of Medicine.

RESULTS

Corticosteroids vs No Corticosteroids

Between March 10, 2020, and May 2, 2020, a total of 3,382 adult patients were admitted for COVID-19 at Montefiore Medical Center. Of these, 2,707 patients met the study inclusion criteria, and 324 of those received corticosteroid treatment. Their demographic characteristics, comorbidities, and admission lab values are shown in Table 1. Patients who received corticosteroids were older, had higher comorbidity scores, were more likely to have asthma or chronic obstructive pulmonary disease, and were less likely to be full code status, compared with patients who did not receive corticosteroids. Patients who received corticosteroids also had higher initial white blood cell (WBC) and neutrophil counts but lower lymphocyte count. The two groups were comparable in initial creatinine level. Additional patient characteristics and addmission lab values are shown in Appendix Table 2.

Average change in CRP levels by hospital day for those who received corticosteroids and those who did not are shown in Figure 1A. Among patients who received corticosteroid treatment, there was a significant decrease in CRP level at 72 hours of treatment (P < .001). In the post hoc analysis of trends in CRP levels, we found that CRP levels among those treated with corticosteroids started to rise around day 6 after the initial drop. This trend was observed even after removing patients with shorter LOS (<7 days) (Figure 1B). The median durations of corticosteroid therapy were 3 days among patients whose LOS was less than 7 days and 6 days among those whose LOS was 7 days or greater. The rise in CRP level was seen at day 5 and day 7 within each group, respectively. Crude death rate was 41.7% among patients with LOS of less than 7 days and 40.6% in those with LOS of 7 days or greater.

CRP Responders vs Nonresponders

Among the 324 patients who received corticosteroids, 131 (40.4%) were classified as responders, 92 (28.4%) were classified as nonresponders, and 101 (31.2%) were undetermined. Characteristics of CRP responders and CRP nonresponders are shown in Table 2 and Appendix Table 3. CRP responders were more likely to have dementia, higher median admission platelet count, and fibrinogen level compared with CRP nonresponders. Patients whose CRP response was undetermined were excluded from the analysis. Their characteristics are shown in Appendix Table 4.

The observed inpatient mortality rate was 25.2% among CRP responders and 47.8% among CRP nonresponders. This was also demonstrated in the Kaplan-Meier survival curve (Figure 2). The odds of inpatient mortality among CRP responders was strongly and significantly reduced compared with those among nonresponders in an unadjusted analysis (odds ratio [OR], 0.37; 95% CI, 0.21-0.65; P = .001) and after adjustment for demographic and clinical characteristics including age, Charlson Comorbidity Index, initial WBC count, initial CRP level, and initial fibrinogen level (OR, 0.27; 95% CI, 0.14-0.54; P < .001). Details on how variables were operationalized and information on missing data are included in Appendix Table 1.

To explore whether this observed effect differed depending on severity of the respiratory illness, we examined the association between CRP response and mortality in subgroups stratified by intubation status. Within our cohort of 223 patients (92 CRP responders and 131 CRP nonresponders), 166 patients were never intubated, 50 patients were intubated in the first 48 hours, and 7 patients were intubated later on during the admission. The odds ratios for death among CRP responders vs nonresponders were 0.50 (P = .07) among patients never intubated and 0.46 (P = .2) among patients intubated within the initial 48 hours of admission.

DISCUSSION

In this retrospective study, we found that, on average, patients treated with corticosteroids had a swift and marked reduction in serum CRP. In addition, among patients treated with corticosteroids, those whose CRP was reduced by 50% or more within 72 hours after treatment had a dramatically reduced risk of inpatient mortality compared with the risk among nonresponders. This study contributes to a growing body of evidence that suggests that corticosteroids may be an efficacious treatment to reduce adverse events in patients with COVID-19 who have evidence of high levels of inflammation as measured by CRP level.^3,4,12,13

It remains unclear whether CRP is simply a biomarker of disease activity or if it plays a role in mediating inflammation. While CRP is commonly understood to be an acute phase reactant, it has been suggested that, after undergoing proteolysis, it functions as a chemoattractant for monocytes.¹⁴ In addition, it is now known that the inflammatory CD14+/CD16+ monocytes that express high levels of IL-6 are key drivers of the cytokine storm in COVID-19.¹⁵ Therefore, it may be possible that the high levels of circulating CRP in patients with cytokine storm recruits monocytes to the lungs, which leads to further lung injury.

Other mechanisms of immune dysregulation that may contribute to lung injury and respiratory failure in COVID-19, such as cytokine-induced T-cell suppression, have been proposed.^7,16 The related markers, such as levels of T-cells or specific cytokines, may therefore represent different but related underlying immune mechanisms affecting the clinical course of COVID-19 that may respond to different therapeutic modalities such as direct IL-6 blockade or chemokine receptor blockade, among others that are currently under investigation.^17,18

Regardless of the underlying mechanism of immune regulation, our study shows that serial measurement of CRP may serve as an early indicator of response to corticosteroids that correlates with decreased mortality. The association between CRP response and reduced risk of mortality was present in both subgroups, those requiring mechanical ventilation and those who did not. The risk reduction was similar in magnitude to the overall effect but was not statistically significant in either group. Interestingly, our time series analysis demonstrated a rise in CRP around day 6 among patients treated with corticosteroids (notably, most patients were treated for 5 to 7 days). Our post hoc analysis suggests that this may represent a “rebound” in inflammation after discontinuation of corticosteroids. However, the clinical significance of this rebound and whether a longer course of steroids would improve outcomes is not known. Because corticosteroid therapy may be associated with adverse effects in some patients,⁴ it is possible that CRP nonresponders represent a subset of patients in whom corticosteroids are not effective and for whom alternative therapies should be considered. In one study looking at the usefulness of IL-1 inhibition for severe COVID-19 infection, patients who received IL-1 inhibitor therapy had improved mortality and a significant decrease in CRP concentration as compared with the historical group.¹⁹ Finally, it is worth noting that, in one large retrospective study, there was harm associated with corticosteroid therapy in patients with low levels of CRP, and in the RECOVERY trial there was a trend toward harm for patients with no oxygen requirement.^3,4 Serial measurement of CRP may further identify the subset of patients in whom corticosteroid therapy might be harmful.

This study has several limitations. First, the retrospective nature of this study is inherently prone to selection bias, and despite the large number of clinical variables accounted for, unmeasured confounders may still exist. This study was also conducted at a single clinical center operating under emergency circumstances at a time during which healthcare resources were limited. Overall in-hospital mortality was high but similar to mortality rates reported at other hospitals in the New York City area during the same months.²⁰ The strengths of this study include a large cohort of COVID-19 patients from New York City, an epicenter of COVID-19, who received corticosteroids.

CONCLUSION

We found that therapy with corticosteroids in patients with COVID-19 is associated with a substantial reduction in CRP levels within 72 hours of therapy, and for those patients in whom CRP levels decrease by 50% or more, there is a significantly lower risk of inpatient mortality. Future studies are needed to validate these findings in other cohorts and to determine if markers other than CRP levels may be predictors of a therapeutic response or if CRP nonresponders would benefit from other targeted therapies.

Confirmed cases of coronavirus disease 2019 (COVID-19) exceed 111 million, and the disease is responsible for approximately 2.4 million deaths worldwide.¹ In the United States, 28 million cases of COVID-19 have been reported, and the disease has caused more than 497,000 deaths.² The clinical presentation of COVID-19 varies widely, with the most severe presentation characterized by acute respiratory distress syndrome and a marked systemic inflammatory response. Corticosteroids have emerged as a potential therapeutic option in a subset of patients. Results from the recently published RECOVERY trial suggest a substantial mortality benefit of dexamethasone in patients who require mechanical ventilation, with a risk reduction of approximately 33%.³ In addition, a recent large retrospective study demonstrated a reduction in the risk of mechanical ventilation or mortality with corticosteroids in a prespecified subset of patients with C-reactive protein (CRP) ≥20 mg/dL, which indicates a high burden of inflammation.⁴

Some patients with severe COVID-19 experience a positive feedback cascade of proinflammatory cytokines, called the cytokine storm, which can worsen lung injury and, in some cases, progress to vasodilatory shock and multiorgan failure.⁵ This complication’s cytokine cascade includes interleukin (IL) 6, IL-1β, and CC chemokine ligand 3 (CCL3), which are released by airway macrophages and all of which are heavily implicated in the maladaptive forms of immune response to COVID-19.^6,7 The cytokine IL-6 is the primary signal for the production of CRP, and corticosteroids have been shown, both in vitro and in vivo, to reduce the production of IL-6 and other cytokines by airway macrophages.⁶ Levels of CRP have been shown to correlate with outcomes in COVID-19 and bacterial pneumonias.^7,8 Reduction in CRP levels following the institution of therapy, known as CRP response, has been shown to predict outcomes in other inflammatory conditions, such as osteomyelitis, hidradenitis suppurativa, and some cases of bacterial pneumonia.^8-10 Similar CRP response in hemophagocytic lymphohistiocytosis, an entity which closely resembles cytokine storm syndrome, has been shown to correlate with disease activity in patients following treatment with an IL-1 antagonist.¹¹ Whether the CRP response as a response to therapeutics in COVID-19 is associated with improved outcomes remains unknown.

Laboratory measurement of CRP levels offers several advantages over the measurement of interleukins. Notably, the half-life of CRP is approximately 19 hours, which is comparable across different age groups and inflammatory conditions because its concentration depends primarily on synthesis in the liver, and a decreased level suggests decreased stimulus for synthesis.⁸ This makes CRP a useful biomarker to assess response to therapy, in contrast to interleukins, which have short half-lives, are variable in heterogeneous populations, and can be difficult to measure. In addition, CRP measurement is rapid and relatively inexpensive.

We hypothesized that reduction in CRP levels by 50% or more within 72 hours after the initiation of corticosteroids in patients with COVID-19 is associated with reduced inpatient mortality and may be an early indicator of therapeutic response.

METHODS

Study Participants

In this retrospective cohort study, we reviewed all adult patients admitted to Montefiore Medical Center (Bronx, New York) for COVID-19 between March 10, 2020, and May 2, 2020. Patients must have been discharged (alive or deceased) by the administrative censor date (May 2, 2020) to be included. Patients who died within the first 48 hours of admission were excluded to allow sufficient time for corticosteroid treatment to take effect. For inclusion in the corticosteroid group, patients needed to have received at least 2 consecutive days of corticosteroid treatment beginning within the first 48 hours of admission with a total daily dose of 0.5 mg/kg prednisone equivalent or greater. Patients who received treatment-dose corticosteroids later in the hospital course were excluded (Appendix Figure).

Comparison Group and Outcome

We examined trends in CRP levels for patients who received corticosteroids vs trends among patients who did not receive corticosteroids. In addition, among patients who were treated with corticosteroids, we compared the inpatient mortality of those who did have a reduction in CRP level after treatment with inpatient mortality of those who did not have a reduction in CRP level after treatment. First, CRP level trends over time were examined in all patients, and compared between those who received corticosteroid treatment and those who did not. Then, patients who received corticosteroids were categorized based on changes in CRP levels after beginning corticosteroids. The first CRP level obtained during the first 48 hours of admission was used as the initial CRP level. For each patient, the last CRP level within the 72 hours after initiation of treatment was used to calculate the change in CRP level from admission. A patient was considered to be a “CRP responder” if their CRP level decreased by 50% or more within 72 hours after treatment and a “CRP nonresponder” if their CRP level did not drop by at least 50% within 72 hours of treatment. Patients who did not have a CRP level within the initial 48 hours of admission or a subsequent CRP measured in the 72 hours after treatment were considered to have an “undetermined CRP response” and excluded from the mortality analysis.

We observed a rise in CRP starting around day 6 among patients treated with corticosteroids and performed a post hoc analysis to determine if this was due to a selection effect whereby patients staying in the hospital longer had higher CRP levels or represented actual rise. In order to address this, we performed a stratified analysis comparing the trends in CRP levels among patients with a length of stay (LOS) of 7 or more days with trends among those with an LOS less than 7 days.

Statistical Analysis

To characterize differences in patients who received corticosteroids and those who did not, we examined their demographic, clinical characteristics, and admission laboratory values, using chi-square test for categorical variables and Kruskal-Wallis test for continuous variables (Table 1). The change in CRP levels from day 0 (presentation to the hospital) in both groups was plotted in a time-series analysis. For each day in the time series, the 95% CIs for the changes in CRP were computed using the t statistic for the corresponding distribution. The Kruskal-Wallis test was used to assess the significance of differences between groups at 72 hours after initiation of treatment.

After categorizing patients by CRP response, we compared demographic, clinical, and laboratory characteristics of patients who were CRP responsive with those of patients who were not, using the same tests of statistical inference mentioned above. To compare time to inpatient mortality differences between CRP response groups, Kaplan-Meier survival curves were generated and statistical significance determined via log-rank test. Univariable logistic regression was used to estimate the odds ratio of inpatient mortality between comparison groups in an unadjusted analysis. Last, to examine the independent association between CRP response and mortality, we constructed a multivariate model that included variables that were significantly associated with mortality in univariable analysis and considered to be important potential confounders by the authors. Details on variable selection for the model are listed in Appendix Table 1.

Data Collection

Data were directly extracted from our center’s electronic health record system. Data processing and recoding was performed using the Python programming language (version 2.7.17) and data analysis was done using Stata 12 (StataCorp LLC; 2011). This study was approved by the institutional review board of the Albert Einstein College of Medicine.

RESULTS

Corticosteroids vs No Corticosteroids

Between March 10, 2020, and May 2, 2020, a total of 3,382 adult patients were admitted for COVID-19 at Montefiore Medical Center. Of these, 2,707 patients met the study inclusion criteria, and 324 of those received corticosteroid treatment. Their demographic characteristics, comorbidities, and admission lab values are shown in Table 1. Patients who received corticosteroids were older, had higher comorbidity scores, were more likely to have asthma or chronic obstructive pulmonary disease, and were less likely to be full code status, compared with patients who did not receive corticosteroids. Patients who received corticosteroids also had higher initial white blood cell (WBC) and neutrophil counts but lower lymphocyte count. The two groups were comparable in initial creatinine level. Additional patient characteristics and addmission lab values are shown in Appendix Table 2.

Average change in CRP levels by hospital day for those who received corticosteroids and those who did not are shown in Figure 1A. Among patients who received corticosteroid treatment, there was a significant decrease in CRP level at 72 hours of treatment (P < .001). In the post hoc analysis of trends in CRP levels, we found that CRP levels among those treated with corticosteroids started to rise around day 6 after the initial drop. This trend was observed even after removing patients with shorter LOS (<7 days) (Figure 1B). The median durations of corticosteroid therapy were 3 days among patients whose LOS was less than 7 days and 6 days among those whose LOS was 7 days or greater. The rise in CRP level was seen at day 5 and day 7 within each group, respectively. Crude death rate was 41.7% among patients with LOS of less than 7 days and 40.6% in those with LOS of 7 days or greater.

CRP Responders vs Nonresponders

Among the 324 patients who received corticosteroids, 131 (40.4%) were classified as responders, 92 (28.4%) were classified as nonresponders, and 101 (31.2%) were undetermined. Characteristics of CRP responders and CRP nonresponders are shown in Table 2 and Appendix Table 3. CRP responders were more likely to have dementia, higher median admission platelet count, and fibrinogen level compared with CRP nonresponders. Patients whose CRP response was undetermined were excluded from the analysis. Their characteristics are shown in Appendix Table 4.

The observed inpatient mortality rate was 25.2% among CRP responders and 47.8% among CRP nonresponders. This was also demonstrated in the Kaplan-Meier survival curve (Figure 2). The odds of inpatient mortality among CRP responders was strongly and significantly reduced compared with those among nonresponders in an unadjusted analysis (odds ratio [OR], 0.37; 95% CI, 0.21-0.65; P = .001) and after adjustment for demographic and clinical characteristics including age, Charlson Comorbidity Index, initial WBC count, initial CRP level, and initial fibrinogen level (OR, 0.27; 95% CI, 0.14-0.54; P < .001). Details on how variables were operationalized and information on missing data are included in Appendix Table 1.

To explore whether this observed effect differed depending on severity of the respiratory illness, we examined the association between CRP response and mortality in subgroups stratified by intubation status. Within our cohort of 223 patients (92 CRP responders and 131 CRP nonresponders), 166 patients were never intubated, 50 patients were intubated in the first 48 hours, and 7 patients were intubated later on during the admission. The odds ratios for death among CRP responders vs nonresponders were 0.50 (P = .07) among patients never intubated and 0.46 (P = .2) among patients intubated within the initial 48 hours of admission.

DISCUSSION

In this retrospective study, we found that, on average, patients treated with corticosteroids had a swift and marked reduction in serum CRP. In addition, among patients treated with corticosteroids, those whose CRP was reduced by 50% or more within 72 hours after treatment had a dramatically reduced risk of inpatient mortality compared with the risk among nonresponders. This study contributes to a growing body of evidence that suggests that corticosteroids may be an efficacious treatment to reduce adverse events in patients with COVID-19 who have evidence of high levels of inflammation as measured by CRP level.^3,4,12,13

It remains unclear whether CRP is simply a biomarker of disease activity or if it plays a role in mediating inflammation. While CRP is commonly understood to be an acute phase reactant, it has been suggested that, after undergoing proteolysis, it functions as a chemoattractant for monocytes.¹⁴ In addition, it is now known that the inflammatory CD14+/CD16+ monocytes that express high levels of IL-6 are key drivers of the cytokine storm in COVID-19.¹⁵ Therefore, it may be possible that the high levels of circulating CRP in patients with cytokine storm recruits monocytes to the lungs, which leads to further lung injury.

Other mechanisms of immune dysregulation that may contribute to lung injury and respiratory failure in COVID-19, such as cytokine-induced T-cell suppression, have been proposed.^7,16 The related markers, such as levels of T-cells or specific cytokines, may therefore represent different but related underlying immune mechanisms affecting the clinical course of COVID-19 that may respond to different therapeutic modalities such as direct IL-6 blockade or chemokine receptor blockade, among others that are currently under investigation.^17,18

Regardless of the underlying mechanism of immune regulation, our study shows that serial measurement of CRP may serve as an early indicator of response to corticosteroids that correlates with decreased mortality. The association between CRP response and reduced risk of mortality was present in both subgroups, those requiring mechanical ventilation and those who did not. The risk reduction was similar in magnitude to the overall effect but was not statistically significant in either group. Interestingly, our time series analysis demonstrated a rise in CRP around day 6 among patients treated with corticosteroids (notably, most patients were treated for 5 to 7 days). Our post hoc analysis suggests that this may represent a “rebound” in inflammation after discontinuation of corticosteroids. However, the clinical significance of this rebound and whether a longer course of steroids would improve outcomes is not known. Because corticosteroid therapy may be associated with adverse effects in some patients,⁴ it is possible that CRP nonresponders represent a subset of patients in whom corticosteroids are not effective and for whom alternative therapies should be considered. In one study looking at the usefulness of IL-1 inhibition for severe COVID-19 infection, patients who received IL-1 inhibitor therapy had improved mortality and a significant decrease in CRP concentration as compared with the historical group.¹⁹ Finally, it is worth noting that, in one large retrospective study, there was harm associated with corticosteroid therapy in patients with low levels of CRP, and in the RECOVERY trial there was a trend toward harm for patients with no oxygen requirement.^3,4 Serial measurement of CRP may further identify the subset of patients in whom corticosteroid therapy might be harmful.

This study has several limitations. First, the retrospective nature of this study is inherently prone to selection bias, and despite the large number of clinical variables accounted for, unmeasured confounders may still exist. This study was also conducted at a single clinical center operating under emergency circumstances at a time during which healthcare resources were limited. Overall in-hospital mortality was high but similar to mortality rates reported at other hospitals in the New York City area during the same months.²⁰ The strengths of this study include a large cohort of COVID-19 patients from New York City, an epicenter of COVID-19, who received corticosteroids.

CONCLUSION

We found that therapy with corticosteroids in patients with COVID-19 is associated with a substantial reduction in CRP levels within 72 hours of therapy, and for those patients in whom CRP levels decrease by 50% or more, there is a significantly lower risk of inpatient mortality. Future studies are needed to validate these findings in other cohorts and to determine if markers other than CRP levels may be predictors of a therapeutic response or if CRP nonresponders would benefit from other targeted therapies.

Confirmed cases of coronavirus disease 2019 (COVID-19) exceed 111 million, and the disease is responsible for approximately 2.4 million deaths worldwide.¹ In the United States, 28 million cases of COVID-19 have been reported, and the disease has caused more than 497,000 deaths.² The clinical presentation of COVID-19 varies widely, with the most severe presentation characterized by acute respiratory distress syndrome and a marked systemic inflammatory response. Corticosteroids have emerged as a potential therapeutic option in a subset of patients. Results from the recently published RECOVERY trial suggest a substantial mortality benefit of dexamethasone in patients who require mechanical ventilation, with a risk reduction of approximately 33%.³ In addition, a recent large retrospective study demonstrated a reduction in the risk of mechanical ventilation or mortality with corticosteroids in a prespecified subset of patients with C-reactive protein (CRP) ≥20 mg/dL, which indicates a high burden of inflammation.⁴

Some patients with severe COVID-19 experience a positive feedback cascade of proinflammatory cytokines, called the cytokine storm, which can worsen lung injury and, in some cases, progress to vasodilatory shock and multiorgan failure.⁵ This complication’s cytokine cascade includes interleukin (IL) 6, IL-1β, and CC chemokine ligand 3 (CCL3), which are released by airway macrophages and all of which are heavily implicated in the maladaptive forms of immune response to COVID-19.^6,7 The cytokine IL-6 is the primary signal for the production of CRP, and corticosteroids have been shown, both in vitro and in vivo, to reduce the production of IL-6 and other cytokines by airway macrophages.⁶ Levels of CRP have been shown to correlate with outcomes in COVID-19 and bacterial pneumonias.^7,8 Reduction in CRP levels following the institution of therapy, known as CRP response, has been shown to predict outcomes in other inflammatory conditions, such as osteomyelitis, hidradenitis suppurativa, and some cases of bacterial pneumonia.^8-10 Similar CRP response in hemophagocytic lymphohistiocytosis, an entity which closely resembles cytokine storm syndrome, has been shown to correlate with disease activity in patients following treatment with an IL-1 antagonist.¹¹ Whether the CRP response as a response to therapeutics in COVID-19 is associated with improved outcomes remains unknown.

Laboratory measurement of CRP levels offers several advantages over the measurement of interleukins. Notably, the half-life of CRP is approximately 19 hours, which is comparable across different age groups and inflammatory conditions because its concentration depends primarily on synthesis in the liver, and a decreased level suggests decreased stimulus for synthesis.⁸ This makes CRP a useful biomarker to assess response to therapy, in contrast to interleukins, which have short half-lives, are variable in heterogeneous populations, and can be difficult to measure. In addition, CRP measurement is rapid and relatively inexpensive.

We hypothesized that reduction in CRP levels by 50% or more within 72 hours after the initiation of corticosteroids in patients with COVID-19 is associated with reduced inpatient mortality and may be an early indicator of therapeutic response.

METHODS

Study Participants

In this retrospective cohort study, we reviewed all adult patients admitted to Montefiore Medical Center (Bronx, New York) for COVID-19 between March 10, 2020, and May 2, 2020. Patients must have been discharged (alive or deceased) by the administrative censor date (May 2, 2020) to be included. Patients who died within the first 48 hours of admission were excluded to allow sufficient time for corticosteroid treatment to take effect. For inclusion in the corticosteroid group, patients needed to have received at least 2 consecutive days of corticosteroid treatment beginning within the first 48 hours of admission with a total daily dose of 0.5 mg/kg prednisone equivalent or greater. Patients who received treatment-dose corticosteroids later in the hospital course were excluded (Appendix Figure).

Comparison Group and Outcome

We examined trends in CRP levels for patients who received corticosteroids vs trends among patients who did not receive corticosteroids. In addition, among patients who were treated with corticosteroids, we compared the inpatient mortality of those who did have a reduction in CRP level after treatment with inpatient mortality of those who did not have a reduction in CRP level after treatment. First, CRP level trends over time were examined in all patients, and compared between those who received corticosteroid treatment and those who did not. Then, patients who received corticosteroids were categorized based on changes in CRP levels after beginning corticosteroids. The first CRP level obtained during the first 48 hours of admission was used as the initial CRP level. For each patient, the last CRP level within the 72 hours after initiation of treatment was used to calculate the change in CRP level from admission. A patient was considered to be a “CRP responder” if their CRP level decreased by 50% or more within 72 hours after treatment and a “CRP nonresponder” if their CRP level did not drop by at least 50% within 72 hours of treatment. Patients who did not have a CRP level within the initial 48 hours of admission or a subsequent CRP measured in the 72 hours after treatment were considered to have an “undetermined CRP response” and excluded from the mortality analysis.

We observed a rise in CRP starting around day 6 among patients treated with corticosteroids and performed a post hoc analysis to determine if this was due to a selection effect whereby patients staying in the hospital longer had higher CRP levels or represented actual rise. In order to address this, we performed a stratified analysis comparing the trends in CRP levels among patients with a length of stay (LOS) of 7 or more days with trends among those with an LOS less than 7 days.

Statistical Analysis

To characterize differences in patients who received corticosteroids and those who did not, we examined their demographic, clinical characteristics, and admission laboratory values, using chi-square test for categorical variables and Kruskal-Wallis test for continuous variables (Table 1). The change in CRP levels from day 0 (presentation to the hospital) in both groups was plotted in a time-series analysis. For each day in the time series, the 95% CIs for the changes in CRP were computed using the t statistic for the corresponding distribution. The Kruskal-Wallis test was used to assess the significance of differences between groups at 72 hours after initiation of treatment.

After categorizing patients by CRP response, we compared demographic, clinical, and laboratory characteristics of patients who were CRP responsive with those of patients who were not, using the same tests of statistical inference mentioned above. To compare time to inpatient mortality differences between CRP response groups, Kaplan-Meier survival curves were generated and statistical significance determined via log-rank test. Univariable logistic regression was used to estimate the odds ratio of inpatient mortality between comparison groups in an unadjusted analysis. Last, to examine the independent association between CRP response and mortality, we constructed a multivariate model that included variables that were significantly associated with mortality in univariable analysis and considered to be important potential confounders by the authors. Details on variable selection for the model are listed in Appendix Table 1.

Data Collection

Data were directly extracted from our center’s electronic health record system. Data processing and recoding was performed using the Python programming language (version 2.7.17) and data analysis was done using Stata 12 (StataCorp LLC; 2011). This study was approved by the institutional review board of the Albert Einstein College of Medicine.

RESULTS

Corticosteroids vs No Corticosteroids

Between March 10, 2020, and May 2, 2020, a total of 3,382 adult patients were admitted for COVID-19 at Montefiore Medical Center. Of these, 2,707 patients met the study inclusion criteria, and 324 of those received corticosteroid treatment. Their demographic characteristics, comorbidities, and admission lab values are shown in Table 1. Patients who received corticosteroids were older, had higher comorbidity scores, were more likely to have asthma or chronic obstructive pulmonary disease, and were less likely to be full code status, compared with patients who did not receive corticosteroids. Patients who received corticosteroids also had higher initial white blood cell (WBC) and neutrophil counts but lower lymphocyte count. The two groups were comparable in initial creatinine level. Additional patient characteristics and addmission lab values are shown in Appendix Table 2.

Average change in CRP levels by hospital day for those who received corticosteroids and those who did not are shown in Figure 1A. Among patients who received corticosteroid treatment, there was a significant decrease in CRP level at 72 hours of treatment (P < .001). In the post hoc analysis of trends in CRP levels, we found that CRP levels among those treated with corticosteroids started to rise around day 6 after the initial drop. This trend was observed even after removing patients with shorter LOS (<7 days) (Figure 1B). The median durations of corticosteroid therapy were 3 days among patients whose LOS was less than 7 days and 6 days among those whose LOS was 7 days or greater. The rise in CRP level was seen at day 5 and day 7 within each group, respectively. Crude death rate was 41.7% among patients with LOS of less than 7 days and 40.6% in those with LOS of 7 days or greater.

CRP Responders vs Nonresponders

Among the 324 patients who received corticosteroids, 131 (40.4%) were classified as responders, 92 (28.4%) were classified as nonresponders, and 101 (31.2%) were undetermined. Characteristics of CRP responders and CRP nonresponders are shown in Table 2 and Appendix Table 3. CRP responders were more likely to have dementia, higher median admission platelet count, and fibrinogen level compared with CRP nonresponders. Patients whose CRP response was undetermined were excluded from the analysis. Their characteristics are shown in Appendix Table 4.

The observed inpatient mortality rate was 25.2% among CRP responders and 47.8% among CRP nonresponders. This was also demonstrated in the Kaplan-Meier survival curve (Figure 2). The odds of inpatient mortality among CRP responders was strongly and significantly reduced compared with those among nonresponders in an unadjusted analysis (odds ratio [OR], 0.37; 95% CI, 0.21-0.65; P = .001) and after adjustment for demographic and clinical characteristics including age, Charlson Comorbidity Index, initial WBC count, initial CRP level, and initial fibrinogen level (OR, 0.27; 95% CI, 0.14-0.54; P < .001). Details on how variables were operationalized and information on missing data are included in Appendix Table 1.

To explore whether this observed effect differed depending on severity of the respiratory illness, we examined the association between CRP response and mortality in subgroups stratified by intubation status. Within our cohort of 223 patients (92 CRP responders and 131 CRP nonresponders), 166 patients were never intubated, 50 patients were intubated in the first 48 hours, and 7 patients were intubated later on during the admission. The odds ratios for death among CRP responders vs nonresponders were 0.50 (P = .07) among patients never intubated and 0.46 (P = .2) among patients intubated within the initial 48 hours of admission.

DISCUSSION

In this retrospective study, we found that, on average, patients treated with corticosteroids had a swift and marked reduction in serum CRP. In addition, among patients treated with corticosteroids, those whose CRP was reduced by 50% or more within 72 hours after treatment had a dramatically reduced risk of inpatient mortality compared with the risk among nonresponders. This study contributes to a growing body of evidence that suggests that corticosteroids may be an efficacious treatment to reduce adverse events in patients with COVID-19 who have evidence of high levels of inflammation as measured by CRP level.^3,4,12,13

It remains unclear whether CRP is simply a biomarker of disease activity or if it plays a role in mediating inflammation. While CRP is commonly understood to be an acute phase reactant, it has been suggested that, after undergoing proteolysis, it functions as a chemoattractant for monocytes.¹⁴ In addition, it is now known that the inflammatory CD14+/CD16+ monocytes that express high levels of IL-6 are key drivers of the cytokine storm in COVID-19.¹⁵ Therefore, it may be possible that the high levels of circulating CRP in patients with cytokine storm recruits monocytes to the lungs, which leads to further lung injury.

Other mechanisms of immune dysregulation that may contribute to lung injury and respiratory failure in COVID-19, such as cytokine-induced T-cell suppression, have been proposed.^7,16 The related markers, such as levels of T-cells or specific cytokines, may therefore represent different but related underlying immune mechanisms affecting the clinical course of COVID-19 that may respond to different therapeutic modalities such as direct IL-6 blockade or chemokine receptor blockade, among others that are currently under investigation.^17,18

Regardless of the underlying mechanism of immune regulation, our study shows that serial measurement of CRP may serve as an early indicator of response to corticosteroids that correlates with decreased mortality. The association between CRP response and reduced risk of mortality was present in both subgroups, those requiring mechanical ventilation and those who did not. The risk reduction was similar in magnitude to the overall effect but was not statistically significant in either group. Interestingly, our time series analysis demonstrated a rise in CRP around day 6 among patients treated with corticosteroids (notably, most patients were treated for 5 to 7 days). Our post hoc analysis suggests that this may represent a “rebound” in inflammation after discontinuation of corticosteroids. However, the clinical significance of this rebound and whether a longer course of steroids would improve outcomes is not known. Because corticosteroid therapy may be associated with adverse effects in some patients,⁴ it is possible that CRP nonresponders represent a subset of patients in whom corticosteroids are not effective and for whom alternative therapies should be considered. In one study looking at the usefulness of IL-1 inhibition for severe COVID-19 infection, patients who received IL-1 inhibitor therapy had improved mortality and a significant decrease in CRP concentration as compared with the historical group.¹⁹ Finally, it is worth noting that, in one large retrospective study, there was harm associated with corticosteroid therapy in patients with low levels of CRP, and in the RECOVERY trial there was a trend toward harm for patients with no oxygen requirement.^3,4 Serial measurement of CRP may further identify the subset of patients in whom corticosteroid therapy might be harmful.

This study has several limitations. First, the retrospective nature of this study is inherently prone to selection bias, and despite the large number of clinical variables accounted for, unmeasured confounders may still exist. This study was also conducted at a single clinical center operating under emergency circumstances at a time during which healthcare resources were limited. Overall in-hospital mortality was high but similar to mortality rates reported at other hospitals in the New York City area during the same months.²⁰ The strengths of this study include a large cohort of COVID-19 patients from New York City, an epicenter of COVID-19, who received corticosteroids.

CONCLUSION

We found that therapy with corticosteroids in patients with COVID-19 is associated with a substantial reduction in CRP levels within 72 hours of therapy, and for those patients in whom CRP levels decrease by 50% or more, there is a significantly lower risk of inpatient mortality. Future studies are needed to validate these findings in other cohorts and to determine if markers other than CRP levels may be predictors of a therapeutic response or if CRP nonresponders would benefit from other targeted therapies.

Studies continue to demonstrate persistent gaps in equity for women and underrepresented minorities (URMs)¹ throughout nearly all aspects of academic medicine, including rank,^2-4 tenure,⁵ authorship,^6,7 funding opportunities,^8,9 awards,¹⁰ speakership,¹¹ leadership,^12,13 and salaries.^2,14,15 Hospital medicine, despite being a newer field,¹⁶ has also seen these disparities^17,18; however, there are numerous efforts in place to actively change our specialty’s course.^19-22 Hospital medicine is a field known for being a change agent in healthcare delivery,²² and its novel approaches are well poised to fundamentally shatter the glass ceilings imposed on traditionally underrepresented groups in medicine. The importance of diversity, equity, and inclusion (DEI) initiatives in healthcare has never been clearer,^23,24 particularly as they relate to cultural competence^25-28 and cultural humility,^29,30 implicit and explicit bias,²⁷ expanding care for underserved patient populations, supporting our workforce, and broadening research agendas.²⁸

In this article, we report DEI efforts within our division, focusing on the development of our strategic plan and specific outcomes related to compensation, recruitment, and policies.

METHODS

Our Division’s Framework to DEI—“It Takes a Village”

Our Division of Hospital Medicine (DHM), previously within the Division of General Internal Medicine, was founded in October 2017. The DHM at the University of Colorado Hospital (UCH) is composed of 100 faculty members (70 physicians and 30 advanced-practice providers; 58% women and 42% men). In 2018, we implemented a stepwise approach to critically assess DEI within our group and to build a strategic plan to address the issues. Key areas of focus included institutional structures, our people, our environments, and our core missions (Figure 1 and Appendix Figure 1). DHM members helped drive our work and partnered with departmental, hospital, and school of medicine committees; national organizations; and collaborators to enhance implementation and dissemination efforts. In addition to stakeholder engagement, we utilized strategic planning and rapid Plan-Do-Study-Act (PDSA) cycles to advance DEI work in our DHM.

Needs Assessment

As a new division, we sought stakeholder feedback from division members. All faculty within the division were invited to attend a meeting in which issues related to DEI were discussed. A literature review that spanned both medical and nonmedical fields was also completed. Search terms included salary equity, gender equity, diverse teams, diversity recruitment and retention, diversifying leadership, and diverse speakers. Salaries, internally funded time, and other processes, such as recruitment, promotion, and hiring for leadership positions, were evaluated during the first year we became a division.

Interventions

TThrough this work, and with stakeholder engagement, we developed a divisional strategic plan to address DEI globally. Our strategic plan included developing a DEI director role to assist with overseeing DEI efforts. We have highlighted the various methods utilized for each component (Figure 1). This work occurred from October 2017 to December 2018.

Our institutional structures

Using best practices from both medical and nonmedical fields, we developed evidence-based approaches to compensation,³¹ recruitment,³² and policies that support and foster a culture of DEI.³² These strategies were used to support the following initiatives:

Compensation: transparent and consistent approaches based upon benchmarking with a framework of equal pay for equal work and similar advanced training/academic rank. In conjunction with efforts within the School of Medicine (SOM), Department of Medicine (DOM), and the UCH, our division sought to study salaries across DHM faculty members. We had an open call for faculty to participate in a newly developed DHM Compensation Committee, with the intent of rigorously examining our compensation practices and goals. Through faculty feedback and committee work, salary equity was defined as equal pay (ie, base salary for one clinical full-time equivalent [FTE]) for equal work based on academic rank and/or years of practice/advanced training. We also compared DHM salaries to regional academic hospital medicine groups and concluded that DHM salaries were lower than local and national benchmarks. This information was used to create a two-phase approach to increasing salaries for all individuals below the American Association of Medical Colleges (AAMC) benchmarks³³ for academic hospitalists. We also developed a stipend system for external roles that came with additional compensation and roles within our own division that came with additional pay (ie, nocturnist). Phase 1 focused on those whose salaries were furthest away from and below benchmark, and phase 2 targeted all remaining individuals below benchmark.

A similar review of FTEs (based on required number of shifts for a full-time hospitalist) tied to our internal DHM leadership positions was completed by the division head and director of DEI. Specifically, the mission for each of our internally funded roles, job descriptions, and responsibilities was reviewed to ensure equity in funding.

Recruitment and advancement: processes to ensure equity and diversity in recruitment, tracking, and reporting, working to eliminate/mitigate bias. In collaboration with members of the AAMC Group on Women in Medicine and Science (GWIMS) and coauthors from various institutions, we developed toolkits and checklists aimed at achieving equity and diversity within candidate pools and on major committees, including, but not limited to, search and promotion committees.³² Additionally, a checklist was developed to help recruit more diverse speakers, including women and URMs, for local, regional, and national conferences.

Policies: evidence-based approaches, tracking and reporting, standardized approaches to eliminate/mitigate bias, embracing nontraditional paths. In partnership with our departmental efforts, members of our team led data collection and reporting for salary benchmarking, leadership roles, and committee membership. This included developing surveys and reporting templates that can be used to identify disparities and inform future efforts. We worked to ensure that we have faculty representing our field at the department and SOM levels. Specifically, we made sure to nominate division members during open calls for departmental and schoolwide committees, including the promotions committee.

Our People

The faculty and staff within our division have been instrumental in moving efforts forward in the following important areas.

Leadership: develop the position of director of DEI as well as leadership structures to support and increase DEI. One of the first steps in our strategic plan was creating a director of DEI leadership role (Appendix Figure 2). The director is responsible for researching, applying, and promoting a broad scope of DEI initiatives and best practices within the DHM, DOM, and SOM (in collaboration with their leaders), including recruitment, retention, and promotion of medical students, residents, and faculty; educational program development; health disparities research; and community-engaged scholarship.

Support: develop family leave policies/develop flexible work policies. Several members of our division worked on departmental committees and served in leadership roles on staff and faculty council. Estimated costs were assessed. Through collective efforts of department leadership and division head support, the department approved parental leave to employees following the birth of an employee’s child or the placement of a child with an employee in connection with adoption or permanent foster care.

Mentorship/sponsorship: enhance faculty advancement programs/develop pipeline and trainings/collaborate with student groups and organizations/invest in all of our people. Faculty across our divisional sites have held important roles in developing pipeline programs for undergraduate students bound for health professions, as well as programs developed specifically for medical students and internal medicine residents. This includes two programs, the CU Hospitalist Scholars Program (CUHSP) and Leadership Education for Aspiring Doctors (LEAD), in which undergraduate students have the opportunity to round with hospital medicine teams, work on quality-improvement projects, and receive extensive mentorship and advising from a diverse faculty team. Additionally, our faculty advancement team within the DHM has grown and been restructured to include more defined goals and to ensure each faculty member has at least one mentor in their area of interest.

Supportive: lactation space and support/diverse space options/inclusive and diverse environments. We worked closely with hospital leadership to advocate for adequately equipped lactation spaces, including equipment such as pumps, refrigerators, and computer workstations. Additionally, our team members conducted environmental scans (eg, identified pictures, artwork, or other images that were not representative of a diverse and inclusive environment and raised concerns when the environment was not inclusive).

Measures

Our measures focused on (1) development and implementation of our DEI strategic plan, including new policies, processes, and practices related to key components of the DEI program; and (2) assessment of specific DEI programs, including pre-post salary data disparities based on rank and pre-post disparities for protected time for similar roles.

Analysis

Through rapid PDSA cycles, we evaluated salary equity, equity in leadership allotment, and committee membership. We have developed a tracking board to track progress of the multiple projects in the strategic plan.

RESULTS

Strategic Plan Development and Tracking

From October 2017 to December 2018, we developed a robust strategic plan and stepwise approach to DEI (Figure 1 and Figure 2). The director of DEI position was developed (see Appendix Figure 2 for job description) to help oversee these efforts. Figure 3 highlights the specific efforts and the progress made on implementation (ie, high-level dashboard or “tracking board”). While outcomes are still pending in the areas of recruitment and advancement and environment, we have made measurable improvements in compensation, as outlined in the following section.

Compensation

One year after the salary-equity interventions, all of our physician faculty’s salaries were at the goal benchmark (Table), and differences in salary for those in similar years of rank were nearly eliminated. Similarly, after implementing an internally consistent approach to assigning FTE for new and established positions within the division (ie, those that fall within the purview of the division), all faculty in similar types of roles had similar amounts of protected time.

Recruitment and Advancement

Toolkits³² and committee recommendations have been incorporated into division goals, though some aspects are still in implementation phases, as division-wide implicit bias training was delayed secondary to the COVID-19 pandemic. Key goals include: (1) implicit bias training for all members of major committees; (2) aiming for a goal of at least 40% representation of women and 40% URMs on committees; (3) having a diversity expert serve on each committee in order to identify and discuss any potential bias in the search and candidate-selection processes; and (4) careful tracking of diversity metrics in regard to diversity of candidates at each step of the interview and selection process.

Surveys and reporting templates for equity on committees and leadership positions have been developed and deployed. Data dashboards for our division have been developed as well (for compensation, leadership, and committee membership). A divisional dashboard to report recruitment efforts is in progress.

We have successfully nominated several faculty members to the SOM promotions committee and departmental committees during open calls for these positions. At the division level, we have also adapted internal policies to ensure promotion occurs on time and offers alternative pathways for faculty that may primarily focus on clinical pathways. All faculty who have gone up for promotion thus far have been successfully promoted in their desired pathway.

Environment

We successfully advocated and achieved adequately equipped lactation spaces, including equipment such as pumps, refrigerators, and computer workstations. This achievement was possible because of our hospital partners. Our efforts helped us acquire sufficient space and facilities such that nursing mothers can pump and still be able to answer phones, enter orders, and document visits.

Our team members conducted environmental scans and raised concerns when the environment was not inclusive, such as conference rooms with portraits of leadership that do not show diversity. The all-male pictures were removed from one frequently used departmental conference room, which will eventually house a diverse group of pictures and achievements.

We aim to eliminate bias by offering implicit bias training for our faculty. While this is presently required for those who serve on committees, in leadership positions, or those involved in recruitment and interviewing for the DOM, our goal is to eventually provide this training to all faculty and staff in the division. We have also incorporated DEI topics into our educational conferences for faculty, including sessions on recognizing bias in medicine, how to be an upstander/ally, and the impact of race and racism on medicine.

DISCUSSION

The important findings of this work are: (1) that successes in DEI can be achieved with strategic planning and stakeholder engagement; (2) through simple modification of processes, we can improve equity in compensation and FTE allotted to leadership; (3) though it takes time, diversity recruitment can be improved using sound, sustainable, evidence-based processes; (4) this work is time-intensive and challenging, requiring ongoing efforts to improve, modify, and enhance current efforts and future successes.

We have certainly made some progress with DEI initiatives within our division and have also learned a great deal from this experience. First, change is difficult for all parties involved, including those leading change and those affected by the changes. We purposely made an effort to facilitate discussions with all of the DHM faculty and staff to ensure that everyone felt included in this work and that everyone’s voice was heard. This was exemplified by inviting all faculty members to a feedback session in which we discussed DEI within our division and areas that we wanted to improve on. Early on, we were able to define what diversity, equity, and inclusion meant to us as a division and then use these definitions to develop tangible goals for all the areas of highest importance to the group.

By increasing faculty presence on key committees, such as the promotions committee, we now have faculty members who are well versed in promotions processes. We are fortunate to have a promotions process that supports faculty advancement for faculty with diverse interests that spans from supporting highly clinical faculty, clinician educators, as well as more traditional researchers.³⁴ By having hospitalists serve in these roles, we help to add to the diverse perspectives on these committees, including emphasizing the scholarship that is associated with quality improvement, as well as DEI efforts which can often be viewed as service as opposed to scholarship.

Clear communication and transparency were key to all of our DEI initiatives. We had monthly updates on our DEI efforts during business meetings and also held impromptu meetings (also known as flash mobs³⁵) to answer questions and discuss concerns in real time. As with all DEI work, it is important to know where you are starting (having accurate data and a clear understanding of the data) and be able to communicate that data to the group. For example, using AAMC salary benchmarking³³ as well as other benchmarks allowed us to accurately calculate variance among salaries and identify the appropriate goal salary for each of our faculty members. Likewise, by completing an in-depth inventory on the work being done by all of our faculty in leadership roles, we were able to standardize the compensation/FTE for each of these roles. Tracking these changes over time, via the use of dashboards in our case, allows for real-time measurements and accountability for all of those involved. Our end goal will be to have all of these initiatives feed into one large dashboard.

Collaborating with leadership and stakeholders in the DOM, SOM, and hospital helped to make our DEI initiatives successful. Much too often, we work in silos when it comes to DEI work. However, we tend to have similar goals and can achieve much more if we work together. Collaboration with multiple stakeholders allowed for wider dissemination and resulted in a larger impact to the campus and community at large. This has been exemplified by the committee composition guidance that has been utilized by the DOM, as well as implementation of campus-wide policies, specifically the parental leave policy, which our faculty members played an important role in creating. Likewise, it is important to look outside of our institutions and work with other hospital medicine groups around the country who are interested in promoting DEI.

We still have much work ahead of us. We are continuing to measure outcomes status postimplementation of the toolkit and checklists being used for diversity recruitment and committee composition. Additionally, we are actively working on several initiatives, including:

• Instituting implicit bias training for all of our faculty
• Partnering with national leaders and our hospital systems to develop zero-tolerance policies regarding abusive behaviors (verbal, physical, and other), racism, and sexism in the hospital and other work settings
• Development of specific recruitment strategies as a means of diversifying our healthcare workforce (of note, based on a 2020 survey of our faculty, in which there was a 70% response rate, 8.5% of our faculty identified as URMs)
• Completion of a diversity dashboard to track our progress in all of these efforts over time
• Development of a more robust pipeline to promotion and leadership for our URM faculty

This study has several strengths. Many of the plans and strategies described here can be used to guide others interested in implementing this work. Figure 2 provides a stepwise
approach to addressing DEI in hospital medicine groups and divisions. We conducted this work at a large academic medical center, and while it may not be generalizable, it does offer some ideas for others to consider in their own work to advance DEI at their institutions. There are also several limitations to this work. Eliminating salary inequities with our approach did take resources. We took advantage of already lower salaries and the need to increase salaries closer to benchmark and paired this effort with our DEI efforts to achieve salary equity. This required partnerships with the department and hospital. Efforts to advance DEI also take a lot of time and effort, and thus commitment from the division, department, and institution as a whole is key. While we have outcomes for our efforts related to salary equity, recruitment efforts should be realized over time, as currently it is too early to tell. We have highlighted the efforts that have been put in place at this time.

CONCLUSION

Using a systematic evidence-based approach with key stakeholder involvement, a division-wide DEI strategy was developed and implemented. While this work is still ongoing, short-term wins are possible, in particular around salary equity and development of policies and structures to promote DEI.

Studies continue to demonstrate persistent gaps in equity for women and underrepresented minorities (URMs)¹ throughout nearly all aspects of academic medicine, including rank,^2-4 tenure,⁵ authorship,^6,7 funding opportunities,^8,9 awards,¹⁰ speakership,¹¹ leadership,^12,13 and salaries.^2,14,15 Hospital medicine, despite being a newer field,¹⁶ has also seen these disparities^17,18; however, there are numerous efforts in place to actively change our specialty’s course.^19-22 Hospital medicine is a field known for being a change agent in healthcare delivery,²² and its novel approaches are well poised to fundamentally shatter the glass ceilings imposed on traditionally underrepresented groups in medicine. The importance of diversity, equity, and inclusion (DEI) initiatives in healthcare has never been clearer,^23,24 particularly as they relate to cultural competence^25-28 and cultural humility,^29,30 implicit and explicit bias,²⁷ expanding care for underserved patient populations, supporting our workforce, and broadening research agendas.²⁸

In this article, we report DEI efforts within our division, focusing on the development of our strategic plan and specific outcomes related to compensation, recruitment, and policies.

METHODS

Our Division’s Framework to DEI—“It Takes a Village”

Our Division of Hospital Medicine (DHM), previously within the Division of General Internal Medicine, was founded in October 2017. The DHM at the University of Colorado Hospital (UCH) is composed of 100 faculty members (70 physicians and 30 advanced-practice providers; 58% women and 42% men). In 2018, we implemented a stepwise approach to critically assess DEI within our group and to build a strategic plan to address the issues. Key areas of focus included institutional structures, our people, our environments, and our core missions (Figure 1 and Appendix Figure 1). DHM members helped drive our work and partnered with departmental, hospital, and school of medicine committees; national organizations; and collaborators to enhance implementation and dissemination efforts. In addition to stakeholder engagement, we utilized strategic planning and rapid Plan-Do-Study-Act (PDSA) cycles to advance DEI work in our DHM.

Needs Assessment

As a new division, we sought stakeholder feedback from division members. All faculty within the division were invited to attend a meeting in which issues related to DEI were discussed. A literature review that spanned both medical and nonmedical fields was also completed. Search terms included salary equity, gender equity, diverse teams, diversity recruitment and retention, diversifying leadership, and diverse speakers. Salaries, internally funded time, and other processes, such as recruitment, promotion, and hiring for leadership positions, were evaluated during the first year we became a division.

Interventions

TThrough this work, and with stakeholder engagement, we developed a divisional strategic plan to address DEI globally. Our strategic plan included developing a DEI director role to assist with overseeing DEI efforts. We have highlighted the various methods utilized for each component (Figure 1). This work occurred from October 2017 to December 2018.

Our institutional structures

Using best practices from both medical and nonmedical fields, we developed evidence-based approaches to compensation,³¹ recruitment,³² and policies that support and foster a culture of DEI.³² These strategies were used to support the following initiatives:

Compensation: transparent and consistent approaches based upon benchmarking with a framework of equal pay for equal work and similar advanced training/academic rank. In conjunction with efforts within the School of Medicine (SOM), Department of Medicine (DOM), and the UCH, our division sought to study salaries across DHM faculty members. We had an open call for faculty to participate in a newly developed DHM Compensation Committee, with the intent of rigorously examining our compensation practices and goals. Through faculty feedback and committee work, salary equity was defined as equal pay (ie, base salary for one clinical full-time equivalent [FTE]) for equal work based on academic rank and/or years of practice/advanced training. We also compared DHM salaries to regional academic hospital medicine groups and concluded that DHM salaries were lower than local and national benchmarks. This information was used to create a two-phase approach to increasing salaries for all individuals below the American Association of Medical Colleges (AAMC) benchmarks³³ for academic hospitalists. We also developed a stipend system for external roles that came with additional compensation and roles within our own division that came with additional pay (ie, nocturnist). Phase 1 focused on those whose salaries were furthest away from and below benchmark, and phase 2 targeted all remaining individuals below benchmark.

A similar review of FTEs (based on required number of shifts for a full-time hospitalist) tied to our internal DHM leadership positions was completed by the division head and director of DEI. Specifically, the mission for each of our internally funded roles, job descriptions, and responsibilities was reviewed to ensure equity in funding.

Recruitment and advancement: processes to ensure equity and diversity in recruitment, tracking, and reporting, working to eliminate/mitigate bias. In collaboration with members of the AAMC Group on Women in Medicine and Science (GWIMS) and coauthors from various institutions, we developed toolkits and checklists aimed at achieving equity and diversity within candidate pools and on major committees, including, but not limited to, search and promotion committees.³² Additionally, a checklist was developed to help recruit more diverse speakers, including women and URMs, for local, regional, and national conferences.

Policies: evidence-based approaches, tracking and reporting, standardized approaches to eliminate/mitigate bias, embracing nontraditional paths. In partnership with our departmental efforts, members of our team led data collection and reporting for salary benchmarking, leadership roles, and committee membership. This included developing surveys and reporting templates that can be used to identify disparities and inform future efforts. We worked to ensure that we have faculty representing our field at the department and SOM levels. Specifically, we made sure to nominate division members during open calls for departmental and schoolwide committees, including the promotions committee.

Our People

The faculty and staff within our division have been instrumental in moving efforts forward in the following important areas.

Leadership: develop the position of director of DEI as well as leadership structures to support and increase DEI. One of the first steps in our strategic plan was creating a director of DEI leadership role (Appendix Figure 2). The director is responsible for researching, applying, and promoting a broad scope of DEI initiatives and best practices within the DHM, DOM, and SOM (in collaboration with their leaders), including recruitment, retention, and promotion of medical students, residents, and faculty; educational program development; health disparities research; and community-engaged scholarship.

Support: develop family leave policies/develop flexible work policies. Several members of our division worked on departmental committees and served in leadership roles on staff and faculty council. Estimated costs were assessed. Through collective efforts of department leadership and division head support, the department approved parental leave to employees following the birth of an employee’s child or the placement of a child with an employee in connection with adoption or permanent foster care.

Mentorship/sponsorship: enhance faculty advancement programs/develop pipeline and trainings/collaborate with student groups and organizations/invest in all of our people. Faculty across our divisional sites have held important roles in developing pipeline programs for undergraduate students bound for health professions, as well as programs developed specifically for medical students and internal medicine residents. This includes two programs, the CU Hospitalist Scholars Program (CUHSP) and Leadership Education for Aspiring Doctors (LEAD), in which undergraduate students have the opportunity to round with hospital medicine teams, work on quality-improvement projects, and receive extensive mentorship and advising from a diverse faculty team. Additionally, our faculty advancement team within the DHM has grown and been restructured to include more defined goals and to ensure each faculty member has at least one mentor in their area of interest.

Supportive: lactation space and support/diverse space options/inclusive and diverse environments. We worked closely with hospital leadership to advocate for adequately equipped lactation spaces, including equipment such as pumps, refrigerators, and computer workstations. Additionally, our team members conducted environmental scans (eg, identified pictures, artwork, or other images that were not representative of a diverse and inclusive environment and raised concerns when the environment was not inclusive).

Measures

Our measures focused on (1) development and implementation of our DEI strategic plan, including new policies, processes, and practices related to key components of the DEI program; and (2) assessment of specific DEI programs, including pre-post salary data disparities based on rank and pre-post disparities for protected time for similar roles.

Analysis

Through rapid PDSA cycles, we evaluated salary equity, equity in leadership allotment, and committee membership. We have developed a tracking board to track progress of the multiple projects in the strategic plan.

RESULTS

Strategic Plan Development and Tracking

From October 2017 to December 2018, we developed a robust strategic plan and stepwise approach to DEI (Figure 1 and Figure 2). The director of DEI position was developed (see Appendix Figure 2 for job description) to help oversee these efforts. Figure 3 highlights the specific efforts and the progress made on implementation (ie, high-level dashboard or “tracking board”). While outcomes are still pending in the areas of recruitment and advancement and environment, we have made measurable improvements in compensation, as outlined in the following section.

Compensation

One year after the salary-equity interventions, all of our physician faculty’s salaries were at the goal benchmark (Table), and differences in salary for those in similar years of rank were nearly eliminated. Similarly, after implementing an internally consistent approach to assigning FTE for new and established positions within the division (ie, those that fall within the purview of the division), all faculty in similar types of roles had similar amounts of protected time.

Recruitment and Advancement

Toolkits³² and committee recommendations have been incorporated into division goals, though some aspects are still in implementation phases, as division-wide implicit bias training was delayed secondary to the COVID-19 pandemic. Key goals include: (1) implicit bias training for all members of major committees; (2) aiming for a goal of at least 40% representation of women and 40% URMs on committees; (3) having a diversity expert serve on each committee in order to identify and discuss any potential bias in the search and candidate-selection processes; and (4) careful tracking of diversity metrics in regard to diversity of candidates at each step of the interview and selection process.

Surveys and reporting templates for equity on committees and leadership positions have been developed and deployed. Data dashboards for our division have been developed as well (for compensation, leadership, and committee membership). A divisional dashboard to report recruitment efforts is in progress.

We have successfully nominated several faculty members to the SOM promotions committee and departmental committees during open calls for these positions. At the division level, we have also adapted internal policies to ensure promotion occurs on time and offers alternative pathways for faculty that may primarily focus on clinical pathways. All faculty who have gone up for promotion thus far have been successfully promoted in their desired pathway.

Environment

We successfully advocated and achieved adequately equipped lactation spaces, including equipment such as pumps, refrigerators, and computer workstations. This achievement was possible because of our hospital partners. Our efforts helped us acquire sufficient space and facilities such that nursing mothers can pump and still be able to answer phones, enter orders, and document visits.

Our team members conducted environmental scans and raised concerns when the environment was not inclusive, such as conference rooms with portraits of leadership that do not show diversity. The all-male pictures were removed from one frequently used departmental conference room, which will eventually house a diverse group of pictures and achievements.

We aim to eliminate bias by offering implicit bias training for our faculty. While this is presently required for those who serve on committees, in leadership positions, or those involved in recruitment and interviewing for the DOM, our goal is to eventually provide this training to all faculty and staff in the division. We have also incorporated DEI topics into our educational conferences for faculty, including sessions on recognizing bias in medicine, how to be an upstander/ally, and the impact of race and racism on medicine.

DISCUSSION

The important findings of this work are: (1) that successes in DEI can be achieved with strategic planning and stakeholder engagement; (2) through simple modification of processes, we can improve equity in compensation and FTE allotted to leadership; (3) though it takes time, diversity recruitment can be improved using sound, sustainable, evidence-based processes; (4) this work is time-intensive and challenging, requiring ongoing efforts to improve, modify, and enhance current efforts and future successes.

We have certainly made some progress with DEI initiatives within our division and have also learned a great deal from this experience. First, change is difficult for all parties involved, including those leading change and those affected by the changes. We purposely made an effort to facilitate discussions with all of the DHM faculty and staff to ensure that everyone felt included in this work and that everyone’s voice was heard. This was exemplified by inviting all faculty members to a feedback session in which we discussed DEI within our division and areas that we wanted to improve on. Early on, we were able to define what diversity, equity, and inclusion meant to us as a division and then use these definitions to develop tangible goals for all the areas of highest importance to the group.

By increasing faculty presence on key committees, such as the promotions committee, we now have faculty members who are well versed in promotions processes. We are fortunate to have a promotions process that supports faculty advancement for faculty with diverse interests that spans from supporting highly clinical faculty, clinician educators, as well as more traditional researchers.³⁴ By having hospitalists serve in these roles, we help to add to the diverse perspectives on these committees, including emphasizing the scholarship that is associated with quality improvement, as well as DEI efforts which can often be viewed as service as opposed to scholarship.

Clear communication and transparency were key to all of our DEI initiatives. We had monthly updates on our DEI efforts during business meetings and also held impromptu meetings (also known as flash mobs³⁵) to answer questions and discuss concerns in real time. As with all DEI work, it is important to know where you are starting (having accurate data and a clear understanding of the data) and be able to communicate that data to the group. For example, using AAMC salary benchmarking³³ as well as other benchmarks allowed us to accurately calculate variance among salaries and identify the appropriate goal salary for each of our faculty members. Likewise, by completing an in-depth inventory on the work being done by all of our faculty in leadership roles, we were able to standardize the compensation/FTE for each of these roles. Tracking these changes over time, via the use of dashboards in our case, allows for real-time measurements and accountability for all of those involved. Our end goal will be to have all of these initiatives feed into one large dashboard.

Collaborating with leadership and stakeholders in the DOM, SOM, and hospital helped to make our DEI initiatives successful. Much too often, we work in silos when it comes to DEI work. However, we tend to have similar goals and can achieve much more if we work together. Collaboration with multiple stakeholders allowed for wider dissemination and resulted in a larger impact to the campus and community at large. This has been exemplified by the committee composition guidance that has been utilized by the DOM, as well as implementation of campus-wide policies, specifically the parental leave policy, which our faculty members played an important role in creating. Likewise, it is important to look outside of our institutions and work with other hospital medicine groups around the country who are interested in promoting DEI.

We still have much work ahead of us. We are continuing to measure outcomes status postimplementation of the toolkit and checklists being used for diversity recruitment and committee composition. Additionally, we are actively working on several initiatives, including:

• Instituting implicit bias training for all of our faculty
• Partnering with national leaders and our hospital systems to develop zero-tolerance policies regarding abusive behaviors (verbal, physical, and other), racism, and sexism in the hospital and other work settings
• Development of specific recruitment strategies as a means of diversifying our healthcare workforce (of note, based on a 2020 survey of our faculty, in which there was a 70% response rate, 8.5% of our faculty identified as URMs)
• Completion of a diversity dashboard to track our progress in all of these efforts over time
• Development of a more robust pipeline to promotion and leadership for our URM faculty

This study has several strengths. Many of the plans and strategies described here can be used to guide others interested in implementing this work. Figure 2 provides a stepwise
approach to addressing DEI in hospital medicine groups and divisions. We conducted this work at a large academic medical center, and while it may not be generalizable, it does offer some ideas for others to consider in their own work to advance DEI at their institutions. There are also several limitations to this work. Eliminating salary inequities with our approach did take resources. We took advantage of already lower salaries and the need to increase salaries closer to benchmark and paired this effort with our DEI efforts to achieve salary equity. This required partnerships with the department and hospital. Efforts to advance DEI also take a lot of time and effort, and thus commitment from the division, department, and institution as a whole is key. While we have outcomes for our efforts related to salary equity, recruitment efforts should be realized over time, as currently it is too early to tell. We have highlighted the efforts that have been put in place at this time.

CONCLUSION

Using a systematic evidence-based approach with key stakeholder involvement, a division-wide DEI strategy was developed and implemented. While this work is still ongoing, short-term wins are possible, in particular around salary equity and development of policies and structures to promote DEI.

Studies continue to demonstrate persistent gaps in equity for women and underrepresented minorities (URMs)¹ throughout nearly all aspects of academic medicine, including rank,^2-4 tenure,⁵ authorship,^6,7 funding opportunities,^8,9 awards,¹⁰ speakership,¹¹ leadership,^12,13 and salaries.^2,14,15 Hospital medicine, despite being a newer field,¹⁶ has also seen these disparities^17,18; however, there are numerous efforts in place to actively change our specialty’s course.^19-22 Hospital medicine is a field known for being a change agent in healthcare delivery,²² and its novel approaches are well poised to fundamentally shatter the glass ceilings imposed on traditionally underrepresented groups in medicine. The importance of diversity, equity, and inclusion (DEI) initiatives in healthcare has never been clearer,^23,24 particularly as they relate to cultural competence^25-28 and cultural humility,^29,30 implicit and explicit bias,²⁷ expanding care for underserved patient populations, supporting our workforce, and broadening research agendas.²⁸

In this article, we report DEI efforts within our division, focusing on the development of our strategic plan and specific outcomes related to compensation, recruitment, and policies.

METHODS

Our Division’s Framework to DEI—“It Takes a Village”

Our Division of Hospital Medicine (DHM), previously within the Division of General Internal Medicine, was founded in October 2017. The DHM at the University of Colorado Hospital (UCH) is composed of 100 faculty members (70 physicians and 30 advanced-practice providers; 58% women and 42% men). In 2018, we implemented a stepwise approach to critically assess DEI within our group and to build a strategic plan to address the issues. Key areas of focus included institutional structures, our people, our environments, and our core missions (Figure 1 and Appendix Figure 1). DHM members helped drive our work and partnered with departmental, hospital, and school of medicine committees; national organizations; and collaborators to enhance implementation and dissemination efforts. In addition to stakeholder engagement, we utilized strategic planning and rapid Plan-Do-Study-Act (PDSA) cycles to advance DEI work in our DHM.

Needs Assessment

As a new division, we sought stakeholder feedback from division members. All faculty within the division were invited to attend a meeting in which issues related to DEI were discussed. A literature review that spanned both medical and nonmedical fields was also completed. Search terms included salary equity, gender equity, diverse teams, diversity recruitment and retention, diversifying leadership, and diverse speakers. Salaries, internally funded time, and other processes, such as recruitment, promotion, and hiring for leadership positions, were evaluated during the first year we became a division.

Interventions

TThrough this work, and with stakeholder engagement, we developed a divisional strategic plan to address DEI globally. Our strategic plan included developing a DEI director role to assist with overseeing DEI efforts. We have highlighted the various methods utilized for each component (Figure 1). This work occurred from October 2017 to December 2018.

Our institutional structures

Using best practices from both medical and nonmedical fields, we developed evidence-based approaches to compensation,³¹ recruitment,³² and policies that support and foster a culture of DEI.³² These strategies were used to support the following initiatives:

Compensation: transparent and consistent approaches based upon benchmarking with a framework of equal pay for equal work and similar advanced training/academic rank. In conjunction with efforts within the School of Medicine (SOM), Department of Medicine (DOM), and the UCH, our division sought to study salaries across DHM faculty members. We had an open call for faculty to participate in a newly developed DHM Compensation Committee, with the intent of rigorously examining our compensation practices and goals. Through faculty feedback and committee work, salary equity was defined as equal pay (ie, base salary for one clinical full-time equivalent [FTE]) for equal work based on academic rank and/or years of practice/advanced training. We also compared DHM salaries to regional academic hospital medicine groups and concluded that DHM salaries were lower than local and national benchmarks. This information was used to create a two-phase approach to increasing salaries for all individuals below the American Association of Medical Colleges (AAMC) benchmarks³³ for academic hospitalists. We also developed a stipend system for external roles that came with additional compensation and roles within our own division that came with additional pay (ie, nocturnist). Phase 1 focused on those whose salaries were furthest away from and below benchmark, and phase 2 targeted all remaining individuals below benchmark.

A similar review of FTEs (based on required number of shifts for a full-time hospitalist) tied to our internal DHM leadership positions was completed by the division head and director of DEI. Specifically, the mission for each of our internally funded roles, job descriptions, and responsibilities was reviewed to ensure equity in funding.

Recruitment and advancement: processes to ensure equity and diversity in recruitment, tracking, and reporting, working to eliminate/mitigate bias. In collaboration with members of the AAMC Group on Women in Medicine and Science (GWIMS) and coauthors from various institutions, we developed toolkits and checklists aimed at achieving equity and diversity within candidate pools and on major committees, including, but not limited to, search and promotion committees.³² Additionally, a checklist was developed to help recruit more diverse speakers, including women and URMs, for local, regional, and national conferences.

Policies: evidence-based approaches, tracking and reporting, standardized approaches to eliminate/mitigate bias, embracing nontraditional paths. In partnership with our departmental efforts, members of our team led data collection and reporting for salary benchmarking, leadership roles, and committee membership. This included developing surveys and reporting templates that can be used to identify disparities and inform future efforts. We worked to ensure that we have faculty representing our field at the department and SOM levels. Specifically, we made sure to nominate division members during open calls for departmental and schoolwide committees, including the promotions committee.

Our People

The faculty and staff within our division have been instrumental in moving efforts forward in the following important areas.

Leadership: develop the position of director of DEI as well as leadership structures to support and increase DEI. One of the first steps in our strategic plan was creating a director of DEI leadership role (Appendix Figure 2). The director is responsible for researching, applying, and promoting a broad scope of DEI initiatives and best practices within the DHM, DOM, and SOM (in collaboration with their leaders), including recruitment, retention, and promotion of medical students, residents, and faculty; educational program development; health disparities research; and community-engaged scholarship.

Support: develop family leave policies/develop flexible work policies. Several members of our division worked on departmental committees and served in leadership roles on staff and faculty council. Estimated costs were assessed. Through collective efforts of department leadership and division head support, the department approved parental leave to employees following the birth of an employee’s child or the placement of a child with an employee in connection with adoption or permanent foster care.

Mentorship/sponsorship: enhance faculty advancement programs/develop pipeline and trainings/collaborate with student groups and organizations/invest in all of our people. Faculty across our divisional sites have held important roles in developing pipeline programs for undergraduate students bound for health professions, as well as programs developed specifically for medical students and internal medicine residents. This includes two programs, the CU Hospitalist Scholars Program (CUHSP) and Leadership Education for Aspiring Doctors (LEAD), in which undergraduate students have the opportunity to round with hospital medicine teams, work on quality-improvement projects, and receive extensive mentorship and advising from a diverse faculty team. Additionally, our faculty advancement team within the DHM has grown and been restructured to include more defined goals and to ensure each faculty member has at least one mentor in their area of interest.

Supportive: lactation space and support/diverse space options/inclusive and diverse environments. We worked closely with hospital leadership to advocate for adequately equipped lactation spaces, including equipment such as pumps, refrigerators, and computer workstations. Additionally, our team members conducted environmental scans (eg, identified pictures, artwork, or other images that were not representative of a diverse and inclusive environment and raised concerns when the environment was not inclusive).

Measures

Our measures focused on (1) development and implementation of our DEI strategic plan, including new policies, processes, and practices related to key components of the DEI program; and (2) assessment of specific DEI programs, including pre-post salary data disparities based on rank and pre-post disparities for protected time for similar roles.

Analysis

Through rapid PDSA cycles, we evaluated salary equity, equity in leadership allotment, and committee membership. We have developed a tracking board to track progress of the multiple projects in the strategic plan.

RESULTS

Strategic Plan Development and Tracking

From October 2017 to December 2018, we developed a robust strategic plan and stepwise approach to DEI (Figure 1 and Figure 2). The director of DEI position was developed (see Appendix Figure 2 for job description) to help oversee these efforts. Figure 3 highlights the specific efforts and the progress made on implementation (ie, high-level dashboard or “tracking board”). While outcomes are still pending in the areas of recruitment and advancement and environment, we have made measurable improvements in compensation, as outlined in the following section.

Compensation

One year after the salary-equity interventions, all of our physician faculty’s salaries were at the goal benchmark (Table), and differences in salary for those in similar years of rank were nearly eliminated. Similarly, after implementing an internally consistent approach to assigning FTE for new and established positions within the division (ie, those that fall within the purview of the division), all faculty in similar types of roles had similar amounts of protected time.

Recruitment and Advancement

Toolkits³² and committee recommendations have been incorporated into division goals, though some aspects are still in implementation phases, as division-wide implicit bias training was delayed secondary to the COVID-19 pandemic. Key goals include: (1) implicit bias training for all members of major committees; (2) aiming for a goal of at least 40% representation of women and 40% URMs on committees; (3) having a diversity expert serve on each committee in order to identify and discuss any potential bias in the search and candidate-selection processes; and (4) careful tracking of diversity metrics in regard to diversity of candidates at each step of the interview and selection process.

Surveys and reporting templates for equity on committees and leadership positions have been developed and deployed. Data dashboards for our division have been developed as well (for compensation, leadership, and committee membership). A divisional dashboard to report recruitment efforts is in progress.

We have successfully nominated several faculty members to the SOM promotions committee and departmental committees during open calls for these positions. At the division level, we have also adapted internal policies to ensure promotion occurs on time and offers alternative pathways for faculty that may primarily focus on clinical pathways. All faculty who have gone up for promotion thus far have been successfully promoted in their desired pathway.

Environment

We successfully advocated and achieved adequately equipped lactation spaces, including equipment such as pumps, refrigerators, and computer workstations. This achievement was possible because of our hospital partners. Our efforts helped us acquire sufficient space and facilities such that nursing mothers can pump and still be able to answer phones, enter orders, and document visits.

Our team members conducted environmental scans and raised concerns when the environment was not inclusive, such as conference rooms with portraits of leadership that do not show diversity. The all-male pictures were removed from one frequently used departmental conference room, which will eventually house a diverse group of pictures and achievements.

We aim to eliminate bias by offering implicit bias training for our faculty. While this is presently required for those who serve on committees, in leadership positions, or those involved in recruitment and interviewing for the DOM, our goal is to eventually provide this training to all faculty and staff in the division. We have also incorporated DEI topics into our educational conferences for faculty, including sessions on recognizing bias in medicine, how to be an upstander/ally, and the impact of race and racism on medicine.

DISCUSSION

The important findings of this work are: (1) that successes in DEI can be achieved with strategic planning and stakeholder engagement; (2) through simple modification of processes, we can improve equity in compensation and FTE allotted to leadership; (3) though it takes time, diversity recruitment can be improved using sound, sustainable, evidence-based processes; (4) this work is time-intensive and challenging, requiring ongoing efforts to improve, modify, and enhance current efforts and future successes.

We have certainly made some progress with DEI initiatives within our division and have also learned a great deal from this experience. First, change is difficult for all parties involved, including those leading change and those affected by the changes. We purposely made an effort to facilitate discussions with all of the DHM faculty and staff to ensure that everyone felt included in this work and that everyone’s voice was heard. This was exemplified by inviting all faculty members to a feedback session in which we discussed DEI within our division and areas that we wanted to improve on. Early on, we were able to define what diversity, equity, and inclusion meant to us as a division and then use these definitions to develop tangible goals for all the areas of highest importance to the group.

By increasing faculty presence on key committees, such as the promotions committee, we now have faculty members who are well versed in promotions processes. We are fortunate to have a promotions process that supports faculty advancement for faculty with diverse interests that spans from supporting highly clinical faculty, clinician educators, as well as more traditional researchers.³⁴ By having hospitalists serve in these roles, we help to add to the diverse perspectives on these committees, including emphasizing the scholarship that is associated with quality improvement, as well as DEI efforts which can often be viewed as service as opposed to scholarship.

Clear communication and transparency were key to all of our DEI initiatives. We had monthly updates on our DEI efforts during business meetings and also held impromptu meetings (also known as flash mobs³⁵) to answer questions and discuss concerns in real time. As with all DEI work, it is important to know where you are starting (having accurate data and a clear understanding of the data) and be able to communicate that data to the group. For example, using AAMC salary benchmarking³³ as well as other benchmarks allowed us to accurately calculate variance among salaries and identify the appropriate goal salary for each of our faculty members. Likewise, by completing an in-depth inventory on the work being done by all of our faculty in leadership roles, we were able to standardize the compensation/FTE for each of these roles. Tracking these changes over time, via the use of dashboards in our case, allows for real-time measurements and accountability for all of those involved. Our end goal will be to have all of these initiatives feed into one large dashboard.

Collaborating with leadership and stakeholders in the DOM, SOM, and hospital helped to make our DEI initiatives successful. Much too often, we work in silos when it comes to DEI work. However, we tend to have similar goals and can achieve much more if we work together. Collaboration with multiple stakeholders allowed for wider dissemination and resulted in a larger impact to the campus and community at large. This has been exemplified by the committee composition guidance that has been utilized by the DOM, as well as implementation of campus-wide policies, specifically the parental leave policy, which our faculty members played an important role in creating. Likewise, it is important to look outside of our institutions and work with other hospital medicine groups around the country who are interested in promoting DEI.

We still have much work ahead of us. We are continuing to measure outcomes status postimplementation of the toolkit and checklists being used for diversity recruitment and committee composition. Additionally, we are actively working on several initiatives, including:

• Instituting implicit bias training for all of our faculty
• Partnering with national leaders and our hospital systems to develop zero-tolerance policies regarding abusive behaviors (verbal, physical, and other), racism, and sexism in the hospital and other work settings
• Development of specific recruitment strategies as a means of diversifying our healthcare workforce (of note, based on a 2020 survey of our faculty, in which there was a 70% response rate, 8.5% of our faculty identified as URMs)
• Completion of a diversity dashboard to track our progress in all of these efforts over time
• Development of a more robust pipeline to promotion and leadership for our URM faculty

This study has several strengths. Many of the plans and strategies described here can be used to guide others interested in implementing this work. Figure 2 provides a stepwise
approach to addressing DEI in hospital medicine groups and divisions. We conducted this work at a large academic medical center, and while it may not be generalizable, it does offer some ideas for others to consider in their own work to advance DEI at their institutions. There are also several limitations to this work. Eliminating salary inequities with our approach did take resources. We took advantage of already lower salaries and the need to increase salaries closer to benchmark and paired this effort with our DEI efforts to achieve salary equity. This required partnerships with the department and hospital. Efforts to advance DEI also take a lot of time and effort, and thus commitment from the division, department, and institution as a whole is key. While we have outcomes for our efforts related to salary equity, recruitment efforts should be realized over time, as currently it is too early to tell. We have highlighted the efforts that have been put in place at this time.

CONCLUSION

Using a systematic evidence-based approach with key stakeholder involvement, a division-wide DEI strategy was developed and implemented. While this work is still ongoing, short-term wins are possible, in particular around salary equity and development of policies and structures to promote DEI.

Staphylococcal scalded skin syndrome (SSSS) is an exfoliative toxin-mediated dermatitis that predominantly occurs in young children. Multiple recent reports indicate a rising incidence of this disease.^1-4 Recommended treatment for SSSS includes antistaphylococcal antibiotics and supportive care measures.^5,6 Elimination or reduction of the toxin-producing Staphylococcus aureus is thought to help limit disease progression and promote recovery. Experts advocate for the use of antibiotics even when there is no apparent focal source of infection, such as an abscess.^6,7

Several factors may affect antibiotic selection, including the desire to inhibit toxin production and to target the causative pathogen in a bactericidal fashion. Because SSSS is toxin mediated, clindamycin is often recommended because of its inhibition of toxin synthesis.^5,8 The clinical utility of adding other antibiotics to clindamycin for coverage of methicillin-sensitive S aureus (MSSA) or methicillin-resistant S aureus (MRSA) is uncertain. Several studies report MSSA to be the predominant pathogen identified by culture^2,9; however, SSSS caused by MRSA has been reported.^9-11 Additionally, bactericidal antibiotics (eg, nafcillin) have been considered to hold potential clinical advantage as compared with bacteriostatic antibiotics (eg, clindamycin), even though clinical studies have not clearly demonstrated this advantage in the general population.^12,13 Some experts recommend additional MRSA or MSSA coverage (such as vancomycin or nafcillin) in patients with high illness severity or nonresponse to therapy, or in areas where there is high prevalence of staphylococcal resistance to clindamycin.^5,7,9,14 Alternatively, for areas with low MRSA prevalence, monotherapy with an anti-MSSA antibiotic is another potential option. No recent studies have compared patient outcomes among antibiotic regimens in children with SSSS.

Knowledge of the outcomes associated with different antibiotic regimens for children hospitalized with SSSS is needed and could be used to improve patient outcomes and potentially promote antibiotic stewardship. In this study, our objectives were to (1) describe antibiotic regimens given to children hospitalized with SSSS, and (2) examine the association of three antibiotic regimens commonly used for SSSS (clindamycin monotherapy, clindamycin plus additional MSSA coverage, and clindamycin plus additional MRSA coverage) with patient outcomes of length of stay (LOS), treatment failure, and cost in a large cohort of children at US children’s hospitals.

METHODS

We conducted a multicenter, retrospective cohort study utilizing data within the Pediatric Health Information System (PHIS) database from July 1, 2011, to June 30, 2016. Thirty-five free-standing tertiary care US children’s hospitals within 24 states were included. The Children’s Hospital Association (Lenexa, Kansas) maintains the PHIS database, which contains de-identified patient information, including diagnoses (with International Classification of Diseases, Ninth and Tenth Revision, Clinical Modification [ICD-9-CM, ICD-10-CM]), demographics, procedures, and daily billing records. Data quality and reliability are confirmed by participating institutions and the Children’s Hospital Association.¹⁵ The local institutional review board (IRB) deemed the study exempt from formal IRB review, as patient information was de-identified.

Study Population

We included hospitalized children aged newborn to 18 years with a primary or secondary diagnosis of SSSS (ICD-9, 695.81; ICD-10, L00). Children whose primary presentation and admission were to a PHIS hospital were included; children transferred from another hospital were excluded. The following exclusion criteria were based on previously published methodology.¹⁶ Children with complex chronic medical conditions as classified by Feudtner et al¹⁷ were excluded, since these children may require a different treatment approach than the general pediatric population. In order to decrease diagnostic ambiguity, we excluded children if an alternative dermatologic diagnosis was recorded as a principal or secondary diagnosis (eg, Stevens-Johnson syndrome or scarlet fever).¹⁶ Finally, hospitals with fewer than 10 children with SSSS during the study period were excluded.

Antibiotic Regimen Groups

We used PHIS daily billing codes to determine the antibiotics received by the study population. Children were classified into antibiotic regimen groups based on whether they received specific antibiotic combinations. Antibiotics received on any day during the hospitalization, including in the emergency department (ED), were used to assign patients to regimen groups. Antibiotics were classified into regimen groups based on consensus among study investigators, which included two board-certified pediatric infectious diseases specialists (A.C., R.M.). Antibiotic group definitions are listed in Table 1. Oral and intravenous (IV) therapies were grouped together for clindamycin, cephalexin/cefazolin, and linezolid because of good oral bioavailability in most situations.¹⁸ The three most common antistaphylococcal groups were chosen for further analysis: clindamycin alone, clindamycin plus MSSA coverage, and clindamycin plus MRSA coverage. The clindamycin group was defined as children with receipt of oral or IV clindamycin. Children who received clindamycin with additional MSSA coverage, such as cefazolin or nafcillin, were categorized as the clindamycin plus MSSA group. Children who received clindamycin with additional MRSA coverage, such as vancomycin or linezolid, were categorized as the clindamycin plus MRSA group. We chose not to include children who received the above regimens plus other antibiotics with partial antistaphylococcal activity, such as ampicillin, gentamicin, or ceftriaxone, in the clindamycin plus MSSA and clindamycin plus MRSA groups. We excluded these antibiotics to decrease the heterogeneity in the definition of regimen groups and allow a more direct comparison for effectiveness.

Covariates

Covariates included age, sex, ethnicity and/or race, payer type, level of care, illness severity, and region. The variable definitions below are in keeping with a prior study of SSSS.¹⁶ Age was categorized as: birth to 59 days, 2 to 11 months, 1 to 4 years (preschool age), 5 to 10 years (school age), and 11 to 18 years (adolescent). We examined infants younger than 60 days separately from older infants because this population may warrant additional treatment considerations. Race and ethnicity were categorized as White (non-Hispanic), African American (non-Hispanic), Hispanic, or other. Payer types included government, private, or other. Level of care was assigned as either intensive care or acute care. Illness severity was assigned using the All Patient Refined Diagnosis Related Group (APR-DRG; 3M Corporation, St. Paul, Minnesota) severity levels.¹⁹ In line with a prior study,¹⁶ we defined “low illness severity” as the APR-DRG minor (1) classification. The moderate (2), major (3), and extreme (4) classifications were defined as “moderate to high illness severity,” since there were very few classifications of major or extreme (<5%) illness severity. We categorized hospitals into the following US regions: Northeast, Midwest, South, and West.

Outcome Measures

The primary outcome was hospital LOS in days, and secondary outcomes were treatment failure and hospital costs. Hospital LOS was chosen as the primary outcome to represent the time needed for the child to show clinical improvement. Treatment failure was defined as a same-cause 14-day ED revisit or hospital readmission, and these were determined to be same-cause if a diagnosis for SSSS (ICD-9, 695.81; ICD-10, L00) was documented for the return encounter. The 14-day interval for readmission and ED revisit was chosen to measure any relapse of symptoms after completion of antibiotic therapy, similar to a prior study of treatment failure in skin and soft tissue infections.²⁰ Total costs of the hospitalization were estimated from charges using hospital- and year-specific cost-to-charge ratios. Subcategories of cost, including clinical, pharmacy, imaging, laboratory, supply, and other, were also compared among the three groups.

Statistical Analysis

Demographic and clinical characteristics of children were summarized using frequencies and percentages for categorical variables and medians with interquartile ranges (IQRs) for continuous variables. These were compared across antibiotic groups using chi-square and Kruskal–Wallis tests, respectively. In unadjusted analyses, outcomes were compared across antibiotic regimen groups using these same statistical tests. In order to account for patient clustering within hospitals, generalized linear mixed-effects models were used to model outcomes with a random intercept for each hospital. Models were adjusted for SSSS being listed as a principal or secondary diagnosis, race, illness severity, and level of care. We log-transformed LOS and cost data prior to modeling because of the nonnormal distributions for these data. Owing to the inability to measure the number of antibiotic doses, and to reduce the possibility of including children who received few regimen-defined combination antibiotics, a post hoc sensitivity analysis was performed. This analysis used an alternative definition for antibiotic regimen groups, for which children admitted for 2 or more calendar days must have received regimen-specified antibiotics on at least 2 days of the admission. Additionally, outcomes were stratified by low and moderate/high illness severity and compared across the three antibiotic regimen groups. All analyses were performed with SAS (SAS 9.4; SAS Institute, Cary, North Carolina), and P values of less than .05 were considered statistically significant.

RESULTS

Overall, 1,815 hospitalized children with SSSS were identified in the PHIS database, and after application of the exclusion criteria, 1,259 children remained, with 1,247 (99%) receiving antibiotics (Figure). The antibiotic regimens received by these children are described in Table 1. Of these, 828 children (66%) received one of the three most common antistaphylococcal regimens (clindamycin, clindamycin + MSSA, and clindamycin + MRSA) and were included for further analysis.

Characteristics of the 828 children are presented in Table 2. Most children (82%) were aged 4 years or younger, and distributions of age, sex, and insurance payer were similar among children receiving the three regimens. Thirty-two percent had moderate to high illness severity, and 3.5% required management in the intensive care setting. Of the three antibiotic regimens, clindamycin monotherapy was most common (47%), followed by clindamycin plus MSSA coverage (33%), and clindamycin plus MRSA coverage (20%). A higher proportion of children in the clindamycin plus MRSA group were African American and were hospitalized in the South. Children receiving clindamycin plus MRSA coverage had higher illness severity (44%) as compared with clindamycin monotherapy (28%) and clindamycin plus MSSA coverage (32%) (P = .001). Additionally, a larger proportion of children treated with clindamycin plus MRSA coverage were managed in the intensive care setting as compared with the clindamycin plus MSSA or clindamycin monotherapy groups.

Among the 828 children with SSSS, the median LOS was 2 days (IQR, 2-3), and treatment failure was 1.1% (95% CI, 0.4-1.8). After adjustment for illness severity, race, payer, and region (Table 3), the three antibiotic regimens were not associated with significant differences in LOS or treatment failure. Costs were significantly different among the three antibiotic regimens. Clindamycin plus MRSA coverage was associated with the greatest costs, whereas clindamycin monotherapy was associated with the lowest costs (mean, $5,348 vs $4,839, respectively; P < .001) (Table 3). In a sensitivity analysis using an alternative antibiotic regimen definition, we found results in line with the primary analysis, with no statistically significant differences in LOS (P = .44) or treatment failure (P = .54), but significant differences in cost (P < .001). Additionally, the same findings were present for LOS, treatment failure, and cost when outcomes were stratified by illness severity (Appendix Table). However, significant contributors to the higher cost in the clindamycin plus MRSA group did vary by illness severity stratification. Laboratory, supply, and pharmacy cost categories differed significantly among antibiotic groups for the low illness severity strata, whereas pharmacy was the only significant cost category difference in moderate/high illness severity.

DISCUSSION

Clindamycin monotherapy, clindamycin plus MSSA coverage, and clindamycin plus MRSA coverage are the most commonly administered antistaphylococcal antibiotic regimens for children hospitalized with SSSS at US children’s hospitals. Our multicenter study found that, across these antistaphylococcal antibiotic regimens, there were no associated differences in hospital LOS or treatment failure. However, the antibiotic regimens were associated with significant differences in overall hospital costs. These findings suggest that the use of clindamycin with additional MSSA or MRSA antibiotic coverage for children with SSSS may not be associated with additional clinical benefit, as compared with clindamycin monotherapy, and could potentially be more costly.

Prior literature describing LOS in relation to antibiotic use for children with SSSS is limited. Authors of a recent case series of 21 children in Philadelphia reported approximately 50% of children received clindamycin monotherapy or combination therapy, but patient outcomes such as LOS were not described.⁹ Clindamycin use and outcomes have been described in smaller studies and case reports of SSSS, which reported positive outcomes such as patient recovery and lack of disease recurrence.^2,9,21 A small retrospective, comparative effectiveness study of 30 neonates with SSSS examined beta-lactamase–resistant penicillin use with and without cephalosporins. They found no effect on LOS, but findings were limited by a small sample size.²² Our study cohort included relatively few neonates, and thus our findings may not be applicable to this population subgroup. We chose not to include regimens with third-generation cephalosporins or ampicillin, which may have limited the number of included neonates, because these antibiotics are frequently administered during evaluation for invasive bacterial infections.²³ We found a very low occurrence of treatment failure in our study cohort across all three groups, which is consistent with other studies of SSSS that report an overall good prognosis and low recurrence and/or readmission rates.^6,16,24 The low prevalence of treatment failure, however, precluded our ability to detect small differences among antibiotic regimen groups that may exist.

We observed that cost differed significantly across antibiotic regimen groups, with lowest cost associated with clindamycin monotherapy in adjusted analysis despite similar LOS. Even with our illness-severity adjustment, there may have been other unmeasured factors resulting in the higher cost associated with the combination groups. Hence, we also examined cost breakdown with a stratified analysis by illness severity. We found that pharmacy costs were significantly different among antibiotic groups in both illness severity strata, whereas those with low illness severity also differed by laboratory and supply costs. Thus, pharmacy cost differences may be the largest driver in the cost differential among groups. Lower cost in the clindamycin monotherapy group is likely due to administration of a single antibiotic. The reason for supply and laboratory cost differences is uncertain, but higher cost in the clindamycin plus MRSA group could possibly be from laboratory testing related to drug monitoring (eg, renal function testing or drug levels). While other studies have reported costs for hospitalized children with SSSS associated with different patient characteristics or diagnostic testing,^1,16 to our knowledge, no other studies have reported cost related to antibiotic regimens for SSSS. As healthcare reimbursements shift to value-based models, identifying treatment regimens with equal efficacy but lower cost will become increasingly important. Future studies should also examine other covariates and outcomes, such as oral vs parenteral antibiotic use, use of monitoring laboratories related to antibiotic choice, and adverse drug effects.

Several strengths and additional limitations apply to our study. Our study is one of the few to describe outcomes associated with antibiotic regimens for children with SSSS. With the PHIS database, we were able to include a large number of children with SSSS from children’s hospitals across the United States. Although the PHIS database affords these strengths, there are limitations inherent to administrative data. Children with SSSS were identified by documented ICD-9 and ICD-10 diagnostic codes, which might lead to misclassification. However, misclassification is less likely because only one ICD-9 and ICD-10 code exists for SSSS, and the characteristics of this condition are specific. Also, diagnostic codes for other dermatologic conditions (eg, scarlet fever) were excluded to further reduce the chance of misclassification. A limitation to our use of PHIS billing codes was the inability to confirm the dosage of antibiotics given, the number of doses, or whether antibiotics were prescribed upon discharge. Another limitation is that children whose antibiotic therapy was changed during hospitalization (eg, from clindamycin monotherapy to cefazolin monotherapy) were categorized into the combination groups. However, the sensitivity analysis performed based on a stricter antibiotic group definition (receipt of both antibiotics on at least 2 calendar days) did not alter the outcomes, which is reassuring. We were unable to assess the use of targeted antibiotic therapy because clinical data (eg, microbiology results) were not available. However, this may be less important because some literature suggests that cultures for S aureus are obtained infrequently² and may be difficult to interpret when obtained,²⁵ since culture growth can represent colonization rather than causative strains. An additional limitation is that administrative data do not include certain clinical outcomes, such as fever duration or degree of skin involvement, which could have differed among the groups. Last, the PHIS database only captures revisits or readmissions to PHIS hospitals, and so we are unable to exclude the possibility of a child being seen at or readmitted to another hospital.

Due to the observational design of this study and potential for incomplete measurement of illness severity, we recommend a future prospective trial with randomization to confirm these findings. One possible reason that LOS did not differ among groups is that the burden of clindamycin-resistant strains in our cohort could be low, and the addition of MSSA or MRSA coverage does not result in a clinically important increase in S aureus coverage. However, pooled pediatric hospital antibiogram data suggest the overall rate of clindamycin resistance is close to 20% in hospitals located in all US regions.²⁶ Limited studies also suggest that MSSA may be the predominant pathogen associated with SSSS.^2,9 To address this, future randomized trials could compare the effectiveness of clindamycin monotherapy to MSSA-specific agents like cefazolin or nafcillin. Unfortunately, anti-MSSA monotherapy was not evaluated in our study because very few children received this treatment. Using monotherapy as opposed to multiple antibiotics has the potential to promote antibiotic stewardship for antistaphylococcal antibiotics in the management of SSSS. Reducing unnecessary antibiotic use not only potentially affects antibiotic resistance, but could also benefit patients in reducing possible side effects, cost, and IV catheter complications.²⁷ However, acknowledging our study limitations, our findings should be applied cautiously in clinical settings, in the context of local antibiogram data, individual culture results, and specific patient factors. The local clindamycin resistance rate for both MSSA and MRSA should be considered. Many antibiotics chosen to treat MRSA—such as vancomycin and trimethoprim/sulfamethoxazole—will also have anti-MSSA activity and may have lower local resistance rates than clindamycin. Practitioners may also consider how each antibiotic kills bacteria; for example, beta-lactams rely on bacterial replication, but clindamycin does not. Each factor should influence how empiric treatment, whether monotherapy or combination, is chosen for children with SSSS.

CONCLUSION

In this large, multicenter cohort of hospitalized children with SSSS, we found that the addition of MSSA or MRSA coverage to clindamycin monotherapy was not associated with differences in outcomes of hospital LOS and treatment failure. Furthermore, clindamycin monotherapy was associated with lower overall cost. Prospective randomized studies are needed to confirm these findings and assess whether clindamycin monotherapy, monotherapy with an anti-MSSA antibiotic, or alternative regimens are most effective for treatment of children with SSSS.

Staphylococcal scalded skin syndrome (SSSS) is an exfoliative toxin-mediated dermatitis that predominantly occurs in young children. Multiple recent reports indicate a rising incidence of this disease.^1-4 Recommended treatment for SSSS includes antistaphylococcal antibiotics and supportive care measures.^5,6 Elimination or reduction of the toxin-producing Staphylococcus aureus is thought to help limit disease progression and promote recovery. Experts advocate for the use of antibiotics even when there is no apparent focal source of infection, such as an abscess.^6,7

Several factors may affect antibiotic selection, including the desire to inhibit toxin production and to target the causative pathogen in a bactericidal fashion. Because SSSS is toxin mediated, clindamycin is often recommended because of its inhibition of toxin synthesis.^5,8 The clinical utility of adding other antibiotics to clindamycin for coverage of methicillin-sensitive S aureus (MSSA) or methicillin-resistant S aureus (MRSA) is uncertain. Several studies report MSSA to be the predominant pathogen identified by culture^2,9; however, SSSS caused by MRSA has been reported.^9-11 Additionally, bactericidal antibiotics (eg, nafcillin) have been considered to hold potential clinical advantage as compared with bacteriostatic antibiotics (eg, clindamycin), even though clinical studies have not clearly demonstrated this advantage in the general population.^12,13 Some experts recommend additional MRSA or MSSA coverage (such as vancomycin or nafcillin) in patients with high illness severity or nonresponse to therapy, or in areas where there is high prevalence of staphylococcal resistance to clindamycin.^5,7,9,14 Alternatively, for areas with low MRSA prevalence, monotherapy with an anti-MSSA antibiotic is another potential option. No recent studies have compared patient outcomes among antibiotic regimens in children with SSSS.

Knowledge of the outcomes associated with different antibiotic regimens for children hospitalized with SSSS is needed and could be used to improve patient outcomes and potentially promote antibiotic stewardship. In this study, our objectives were to (1) describe antibiotic regimens given to children hospitalized with SSSS, and (2) examine the association of three antibiotic regimens commonly used for SSSS (clindamycin monotherapy, clindamycin plus additional MSSA coverage, and clindamycin plus additional MRSA coverage) with patient outcomes of length of stay (LOS), treatment failure, and cost in a large cohort of children at US children’s hospitals.

METHODS

We conducted a multicenter, retrospective cohort study utilizing data within the Pediatric Health Information System (PHIS) database from July 1, 2011, to June 30, 2016. Thirty-five free-standing tertiary care US children’s hospitals within 24 states were included. The Children’s Hospital Association (Lenexa, Kansas) maintains the PHIS database, which contains de-identified patient information, including diagnoses (with International Classification of Diseases, Ninth and Tenth Revision, Clinical Modification [ICD-9-CM, ICD-10-CM]), demographics, procedures, and daily billing records. Data quality and reliability are confirmed by participating institutions and the Children’s Hospital Association.¹⁵ The local institutional review board (IRB) deemed the study exempt from formal IRB review, as patient information was de-identified.

Study Population

We included hospitalized children aged newborn to 18 years with a primary or secondary diagnosis of SSSS (ICD-9, 695.81; ICD-10, L00). Children whose primary presentation and admission were to a PHIS hospital were included; children transferred from another hospital were excluded. The following exclusion criteria were based on previously published methodology.¹⁶ Children with complex chronic medical conditions as classified by Feudtner et al¹⁷ were excluded, since these children may require a different treatment approach than the general pediatric population. In order to decrease diagnostic ambiguity, we excluded children if an alternative dermatologic diagnosis was recorded as a principal or secondary diagnosis (eg, Stevens-Johnson syndrome or scarlet fever).¹⁶ Finally, hospitals with fewer than 10 children with SSSS during the study period were excluded.

Antibiotic Regimen Groups

We used PHIS daily billing codes to determine the antibiotics received by the study population. Children were classified into antibiotic regimen groups based on whether they received specific antibiotic combinations. Antibiotics received on any day during the hospitalization, including in the emergency department (ED), were used to assign patients to regimen groups. Antibiotics were classified into regimen groups based on consensus among study investigators, which included two board-certified pediatric infectious diseases specialists (A.C., R.M.). Antibiotic group definitions are listed in Table 1. Oral and intravenous (IV) therapies were grouped together for clindamycin, cephalexin/cefazolin, and linezolid because of good oral bioavailability in most situations.¹⁸ The three most common antistaphylococcal groups were chosen for further analysis: clindamycin alone, clindamycin plus MSSA coverage, and clindamycin plus MRSA coverage. The clindamycin group was defined as children with receipt of oral or IV clindamycin. Children who received clindamycin with additional MSSA coverage, such as cefazolin or nafcillin, were categorized as the clindamycin plus MSSA group. Children who received clindamycin with additional MRSA coverage, such as vancomycin or linezolid, were categorized as the clindamycin plus MRSA group. We chose not to include children who received the above regimens plus other antibiotics with partial antistaphylococcal activity, such as ampicillin, gentamicin, or ceftriaxone, in the clindamycin plus MSSA and clindamycin plus MRSA groups. We excluded these antibiotics to decrease the heterogeneity in the definition of regimen groups and allow a more direct comparison for effectiveness.

Covariates

Covariates included age, sex, ethnicity and/or race, payer type, level of care, illness severity, and region. The variable definitions below are in keeping with a prior study of SSSS.¹⁶ Age was categorized as: birth to 59 days, 2 to 11 months, 1 to 4 years (preschool age), 5 to 10 years (school age), and 11 to 18 years (adolescent). We examined infants younger than 60 days separately from older infants because this population may warrant additional treatment considerations. Race and ethnicity were categorized as White (non-Hispanic), African American (non-Hispanic), Hispanic, or other. Payer types included government, private, or other. Level of care was assigned as either intensive care or acute care. Illness severity was assigned using the All Patient Refined Diagnosis Related Group (APR-DRG; 3M Corporation, St. Paul, Minnesota) severity levels.¹⁹ In line with a prior study,¹⁶ we defined “low illness severity” as the APR-DRG minor (1) classification. The moderate (2), major (3), and extreme (4) classifications were defined as “moderate to high illness severity,” since there were very few classifications of major or extreme (<5%) illness severity. We categorized hospitals into the following US regions: Northeast, Midwest, South, and West.

Outcome Measures

The primary outcome was hospital LOS in days, and secondary outcomes were treatment failure and hospital costs. Hospital LOS was chosen as the primary outcome to represent the time needed for the child to show clinical improvement. Treatment failure was defined as a same-cause 14-day ED revisit or hospital readmission, and these were determined to be same-cause if a diagnosis for SSSS (ICD-9, 695.81; ICD-10, L00) was documented for the return encounter. The 14-day interval for readmission and ED revisit was chosen to measure any relapse of symptoms after completion of antibiotic therapy, similar to a prior study of treatment failure in skin and soft tissue infections.²⁰ Total costs of the hospitalization were estimated from charges using hospital- and year-specific cost-to-charge ratios. Subcategories of cost, including clinical, pharmacy, imaging, laboratory, supply, and other, were also compared among the three groups.

Statistical Analysis

Demographic and clinical characteristics of children were summarized using frequencies and percentages for categorical variables and medians with interquartile ranges (IQRs) for continuous variables. These were compared across antibiotic groups using chi-square and Kruskal–Wallis tests, respectively. In unadjusted analyses, outcomes were compared across antibiotic regimen groups using these same statistical tests. In order to account for patient clustering within hospitals, generalized linear mixed-effects models were used to model outcomes with a random intercept for each hospital. Models were adjusted for SSSS being listed as a principal or secondary diagnosis, race, illness severity, and level of care. We log-transformed LOS and cost data prior to modeling because of the nonnormal distributions for these data. Owing to the inability to measure the number of antibiotic doses, and to reduce the possibility of including children who received few regimen-defined combination antibiotics, a post hoc sensitivity analysis was performed. This analysis used an alternative definition for antibiotic regimen groups, for which children admitted for 2 or more calendar days must have received regimen-specified antibiotics on at least 2 days of the admission. Additionally, outcomes were stratified by low and moderate/high illness severity and compared across the three antibiotic regimen groups. All analyses were performed with SAS (SAS 9.4; SAS Institute, Cary, North Carolina), and P values of less than .05 were considered statistically significant.

RESULTS

Overall, 1,815 hospitalized children with SSSS were identified in the PHIS database, and after application of the exclusion criteria, 1,259 children remained, with 1,247 (99%) receiving antibiotics (Figure). The antibiotic regimens received by these children are described in Table 1. Of these, 828 children (66%) received one of the three most common antistaphylococcal regimens (clindamycin, clindamycin + MSSA, and clindamycin + MRSA) and were included for further analysis.

Characteristics of the 828 children are presented in Table 2. Most children (82%) were aged 4 years or younger, and distributions of age, sex, and insurance payer were similar among children receiving the three regimens. Thirty-two percent had moderate to high illness severity, and 3.5% required management in the intensive care setting. Of the three antibiotic regimens, clindamycin monotherapy was most common (47%), followed by clindamycin plus MSSA coverage (33%), and clindamycin plus MRSA coverage (20%). A higher proportion of children in the clindamycin plus MRSA group were African American and were hospitalized in the South. Children receiving clindamycin plus MRSA coverage had higher illness severity (44%) as compared with clindamycin monotherapy (28%) and clindamycin plus MSSA coverage (32%) (P = .001). Additionally, a larger proportion of children treated with clindamycin plus MRSA coverage were managed in the intensive care setting as compared with the clindamycin plus MSSA or clindamycin monotherapy groups.

Among the 828 children with SSSS, the median LOS was 2 days (IQR, 2-3), and treatment failure was 1.1% (95% CI, 0.4-1.8). After adjustment for illness severity, race, payer, and region (Table 3), the three antibiotic regimens were not associated with significant differences in LOS or treatment failure. Costs were significantly different among the three antibiotic regimens. Clindamycin plus MRSA coverage was associated with the greatest costs, whereas clindamycin monotherapy was associated with the lowest costs (mean, $5,348 vs $4,839, respectively; P < .001) (Table 3). In a sensitivity analysis using an alternative antibiotic regimen definition, we found results in line with the primary analysis, with no statistically significant differences in LOS (P = .44) or treatment failure (P = .54), but significant differences in cost (P < .001). Additionally, the same findings were present for LOS, treatment failure, and cost when outcomes were stratified by illness severity (Appendix Table). However, significant contributors to the higher cost in the clindamycin plus MRSA group did vary by illness severity stratification. Laboratory, supply, and pharmacy cost categories differed significantly among antibiotic groups for the low illness severity strata, whereas pharmacy was the only significant cost category difference in moderate/high illness severity.

DISCUSSION

Clindamycin monotherapy, clindamycin plus MSSA coverage, and clindamycin plus MRSA coverage are the most commonly administered antistaphylococcal antibiotic regimens for children hospitalized with SSSS at US children’s hospitals. Our multicenter study found that, across these antistaphylococcal antibiotic regimens, there were no associated differences in hospital LOS or treatment failure. However, the antibiotic regimens were associated with significant differences in overall hospital costs. These findings suggest that the use of clindamycin with additional MSSA or MRSA antibiotic coverage for children with SSSS may not be associated with additional clinical benefit, as compared with clindamycin monotherapy, and could potentially be more costly.

Prior literature describing LOS in relation to antibiotic use for children with SSSS is limited. Authors of a recent case series of 21 children in Philadelphia reported approximately 50% of children received clindamycin monotherapy or combination therapy, but patient outcomes such as LOS were not described.⁹ Clindamycin use and outcomes have been described in smaller studies and case reports of SSSS, which reported positive outcomes such as patient recovery and lack of disease recurrence.^2,9,21 A small retrospective, comparative effectiveness study of 30 neonates with SSSS examined beta-lactamase–resistant penicillin use with and without cephalosporins. They found no effect on LOS, but findings were limited by a small sample size.²² Our study cohort included relatively few neonates, and thus our findings may not be applicable to this population subgroup. We chose not to include regimens with third-generation cephalosporins or ampicillin, which may have limited the number of included neonates, because these antibiotics are frequently administered during evaluation for invasive bacterial infections.²³ We found a very low occurrence of treatment failure in our study cohort across all three groups, which is consistent with other studies of SSSS that report an overall good prognosis and low recurrence and/or readmission rates.^6,16,24 The low prevalence of treatment failure, however, precluded our ability to detect small differences among antibiotic regimen groups that may exist.

We observed that cost differed significantly across antibiotic regimen groups, with lowest cost associated with clindamycin monotherapy in adjusted analysis despite similar LOS. Even with our illness-severity adjustment, there may have been other unmeasured factors resulting in the higher cost associated with the combination groups. Hence, we also examined cost breakdown with a stratified analysis by illness severity. We found that pharmacy costs were significantly different among antibiotic groups in both illness severity strata, whereas those with low illness severity also differed by laboratory and supply costs. Thus, pharmacy cost differences may be the largest driver in the cost differential among groups. Lower cost in the clindamycin monotherapy group is likely due to administration of a single antibiotic. The reason for supply and laboratory cost differences is uncertain, but higher cost in the clindamycin plus MRSA group could possibly be from laboratory testing related to drug monitoring (eg, renal function testing or drug levels). While other studies have reported costs for hospitalized children with SSSS associated with different patient characteristics or diagnostic testing,^1,16 to our knowledge, no other studies have reported cost related to antibiotic regimens for SSSS. As healthcare reimbursements shift to value-based models, identifying treatment regimens with equal efficacy but lower cost will become increasingly important. Future studies should also examine other covariates and outcomes, such as oral vs parenteral antibiotic use, use of monitoring laboratories related to antibiotic choice, and adverse drug effects.

Several strengths and additional limitations apply to our study. Our study is one of the few to describe outcomes associated with antibiotic regimens for children with SSSS. With the PHIS database, we were able to include a large number of children with SSSS from children’s hospitals across the United States. Although the PHIS database affords these strengths, there are limitations inherent to administrative data. Children with SSSS were identified by documented ICD-9 and ICD-10 diagnostic codes, which might lead to misclassification. However, misclassification is less likely because only one ICD-9 and ICD-10 code exists for SSSS, and the characteristics of this condition are specific. Also, diagnostic codes for other dermatologic conditions (eg, scarlet fever) were excluded to further reduce the chance of misclassification. A limitation to our use of PHIS billing codes was the inability to confirm the dosage of antibiotics given, the number of doses, or whether antibiotics were prescribed upon discharge. Another limitation is that children whose antibiotic therapy was changed during hospitalization (eg, from clindamycin monotherapy to cefazolin monotherapy) were categorized into the combination groups. However, the sensitivity analysis performed based on a stricter antibiotic group definition (receipt of both antibiotics on at least 2 calendar days) did not alter the outcomes, which is reassuring. We were unable to assess the use of targeted antibiotic therapy because clinical data (eg, microbiology results) were not available. However, this may be less important because some literature suggests that cultures for S aureus are obtained infrequently² and may be difficult to interpret when obtained,²⁵ since culture growth can represent colonization rather than causative strains. An additional limitation is that administrative data do not include certain clinical outcomes, such as fever duration or degree of skin involvement, which could have differed among the groups. Last, the PHIS database only captures revisits or readmissions to PHIS hospitals, and so we are unable to exclude the possibility of a child being seen at or readmitted to another hospital.

Due to the observational design of this study and potential for incomplete measurement of illness severity, we recommend a future prospective trial with randomization to confirm these findings. One possible reason that LOS did not differ among groups is that the burden of clindamycin-resistant strains in our cohort could be low, and the addition of MSSA or MRSA coverage does not result in a clinically important increase in S aureus coverage. However, pooled pediatric hospital antibiogram data suggest the overall rate of clindamycin resistance is close to 20% in hospitals located in all US regions.²⁶ Limited studies also suggest that MSSA may be the predominant pathogen associated with SSSS.^2,9 To address this, future randomized trials could compare the effectiveness of clindamycin monotherapy to MSSA-specific agents like cefazolin or nafcillin. Unfortunately, anti-MSSA monotherapy was not evaluated in our study because very few children received this treatment. Using monotherapy as opposed to multiple antibiotics has the potential to promote antibiotic stewardship for antistaphylococcal antibiotics in the management of SSSS. Reducing unnecessary antibiotic use not only potentially affects antibiotic resistance, but could also benefit patients in reducing possible side effects, cost, and IV catheter complications.²⁷ However, acknowledging our study limitations, our findings should be applied cautiously in clinical settings, in the context of local antibiogram data, individual culture results, and specific patient factors. The local clindamycin resistance rate for both MSSA and MRSA should be considered. Many antibiotics chosen to treat MRSA—such as vancomycin and trimethoprim/sulfamethoxazole—will also have anti-MSSA activity and may have lower local resistance rates than clindamycin. Practitioners may also consider how each antibiotic kills bacteria; for example, beta-lactams rely on bacterial replication, but clindamycin does not. Each factor should influence how empiric treatment, whether monotherapy or combination, is chosen for children with SSSS.

CONCLUSION

In this large, multicenter cohort of hospitalized children with SSSS, we found that the addition of MSSA or MRSA coverage to clindamycin monotherapy was not associated with differences in outcomes of hospital LOS and treatment failure. Furthermore, clindamycin monotherapy was associated with lower overall cost. Prospective randomized studies are needed to confirm these findings and assess whether clindamycin monotherapy, monotherapy with an anti-MSSA antibiotic, or alternative regimens are most effective for treatment of children with SSSS.

Staphylococcal scalded skin syndrome (SSSS) is an exfoliative toxin-mediated dermatitis that predominantly occurs in young children. Multiple recent reports indicate a rising incidence of this disease.^1-4 Recommended treatment for SSSS includes antistaphylococcal antibiotics and supportive care measures.^5,6 Elimination or reduction of the toxin-producing Staphylococcus aureus is thought to help limit disease progression and promote recovery. Experts advocate for the use of antibiotics even when there is no apparent focal source of infection, such as an abscess.^6,7

Several factors may affect antibiotic selection, including the desire to inhibit toxin production and to target the causative pathogen in a bactericidal fashion. Because SSSS is toxin mediated, clindamycin is often recommended because of its inhibition of toxin synthesis.^5,8 The clinical utility of adding other antibiotics to clindamycin for coverage of methicillin-sensitive S aureus (MSSA) or methicillin-resistant S aureus (MRSA) is uncertain. Several studies report MSSA to be the predominant pathogen identified by culture^2,9; however, SSSS caused by MRSA has been reported.^9-11 Additionally, bactericidal antibiotics (eg, nafcillin) have been considered to hold potential clinical advantage as compared with bacteriostatic antibiotics (eg, clindamycin), even though clinical studies have not clearly demonstrated this advantage in the general population.^12,13 Some experts recommend additional MRSA or MSSA coverage (such as vancomycin or nafcillin) in patients with high illness severity or nonresponse to therapy, or in areas where there is high prevalence of staphylococcal resistance to clindamycin.^5,7,9,14 Alternatively, for areas with low MRSA prevalence, monotherapy with an anti-MSSA antibiotic is another potential option. No recent studies have compared patient outcomes among antibiotic regimens in children with SSSS.

Knowledge of the outcomes associated with different antibiotic regimens for children hospitalized with SSSS is needed and could be used to improve patient outcomes and potentially promote antibiotic stewardship. In this study, our objectives were to (1) describe antibiotic regimens given to children hospitalized with SSSS, and (2) examine the association of three antibiotic regimens commonly used for SSSS (clindamycin monotherapy, clindamycin plus additional MSSA coverage, and clindamycin plus additional MRSA coverage) with patient outcomes of length of stay (LOS), treatment failure, and cost in a large cohort of children at US children’s hospitals.

METHODS

We conducted a multicenter, retrospective cohort study utilizing data within the Pediatric Health Information System (PHIS) database from July 1, 2011, to June 30, 2016. Thirty-five free-standing tertiary care US children’s hospitals within 24 states were included. The Children’s Hospital Association (Lenexa, Kansas) maintains the PHIS database, which contains de-identified patient information, including diagnoses (with International Classification of Diseases, Ninth and Tenth Revision, Clinical Modification [ICD-9-CM, ICD-10-CM]), demographics, procedures, and daily billing records. Data quality and reliability are confirmed by participating institutions and the Children’s Hospital Association.¹⁵ The local institutional review board (IRB) deemed the study exempt from formal IRB review, as patient information was de-identified.

Study Population

We included hospitalized children aged newborn to 18 years with a primary or secondary diagnosis of SSSS (ICD-9, 695.81; ICD-10, L00). Children whose primary presentation and admission were to a PHIS hospital were included; children transferred from another hospital were excluded. The following exclusion criteria were based on previously published methodology.¹⁶ Children with complex chronic medical conditions as classified by Feudtner et al¹⁷ were excluded, since these children may require a different treatment approach than the general pediatric population. In order to decrease diagnostic ambiguity, we excluded children if an alternative dermatologic diagnosis was recorded as a principal or secondary diagnosis (eg, Stevens-Johnson syndrome or scarlet fever).¹⁶ Finally, hospitals with fewer than 10 children with SSSS during the study period were excluded.

Antibiotic Regimen Groups

We used PHIS daily billing codes to determine the antibiotics received by the study population. Children were classified into antibiotic regimen groups based on whether they received specific antibiotic combinations. Antibiotics received on any day during the hospitalization, including in the emergency department (ED), were used to assign patients to regimen groups. Antibiotics were classified into regimen groups based on consensus among study investigators, which included two board-certified pediatric infectious diseases specialists (A.C., R.M.). Antibiotic group definitions are listed in Table 1. Oral and intravenous (IV) therapies were grouped together for clindamycin, cephalexin/cefazolin, and linezolid because of good oral bioavailability in most situations.¹⁸ The three most common antistaphylococcal groups were chosen for further analysis: clindamycin alone, clindamycin plus MSSA coverage, and clindamycin plus MRSA coverage. The clindamycin group was defined as children with receipt of oral or IV clindamycin. Children who received clindamycin with additional MSSA coverage, such as cefazolin or nafcillin, were categorized as the clindamycin plus MSSA group. Children who received clindamycin with additional MRSA coverage, such as vancomycin or linezolid, were categorized as the clindamycin plus MRSA group. We chose not to include children who received the above regimens plus other antibiotics with partial antistaphylococcal activity, such as ampicillin, gentamicin, or ceftriaxone, in the clindamycin plus MSSA and clindamycin plus MRSA groups. We excluded these antibiotics to decrease the heterogeneity in the definition of regimen groups and allow a more direct comparison for effectiveness.

Covariates

Covariates included age, sex, ethnicity and/or race, payer type, level of care, illness severity, and region. The variable definitions below are in keeping with a prior study of SSSS.¹⁶ Age was categorized as: birth to 59 days, 2 to 11 months, 1 to 4 years (preschool age), 5 to 10 years (school age), and 11 to 18 years (adolescent). We examined infants younger than 60 days separately from older infants because this population may warrant additional treatment considerations. Race and ethnicity were categorized as White (non-Hispanic), African American (non-Hispanic), Hispanic, or other. Payer types included government, private, or other. Level of care was assigned as either intensive care or acute care. Illness severity was assigned using the All Patient Refined Diagnosis Related Group (APR-DRG; 3M Corporation, St. Paul, Minnesota) severity levels.¹⁹ In line with a prior study,¹⁶ we defined “low illness severity” as the APR-DRG minor (1) classification. The moderate (2), major (3), and extreme (4) classifications were defined as “moderate to high illness severity,” since there were very few classifications of major or extreme (<5%) illness severity. We categorized hospitals into the following US regions: Northeast, Midwest, South, and West.

Outcome Measures

The primary outcome was hospital LOS in days, and secondary outcomes were treatment failure and hospital costs. Hospital LOS was chosen as the primary outcome to represent the time needed for the child to show clinical improvement. Treatment failure was defined as a same-cause 14-day ED revisit or hospital readmission, and these were determined to be same-cause if a diagnosis for SSSS (ICD-9, 695.81; ICD-10, L00) was documented for the return encounter. The 14-day interval for readmission and ED revisit was chosen to measure any relapse of symptoms after completion of antibiotic therapy, similar to a prior study of treatment failure in skin and soft tissue infections.²⁰ Total costs of the hospitalization were estimated from charges using hospital- and year-specific cost-to-charge ratios. Subcategories of cost, including clinical, pharmacy, imaging, laboratory, supply, and other, were also compared among the three groups.

Statistical Analysis

Demographic and clinical characteristics of children were summarized using frequencies and percentages for categorical variables and medians with interquartile ranges (IQRs) for continuous variables. These were compared across antibiotic groups using chi-square and Kruskal–Wallis tests, respectively. In unadjusted analyses, outcomes were compared across antibiotic regimen groups using these same statistical tests. In order to account for patient clustering within hospitals, generalized linear mixed-effects models were used to model outcomes with a random intercept for each hospital. Models were adjusted for SSSS being listed as a principal or secondary diagnosis, race, illness severity, and level of care. We log-transformed LOS and cost data prior to modeling because of the nonnormal distributions for these data. Owing to the inability to measure the number of antibiotic doses, and to reduce the possibility of including children who received few regimen-defined combination antibiotics, a post hoc sensitivity analysis was performed. This analysis used an alternative definition for antibiotic regimen groups, for which children admitted for 2 or more calendar days must have received regimen-specified antibiotics on at least 2 days of the admission. Additionally, outcomes were stratified by low and moderate/high illness severity and compared across the three antibiotic regimen groups. All analyses were performed with SAS (SAS 9.4; SAS Institute, Cary, North Carolina), and P values of less than .05 were considered statistically significant.

RESULTS

Overall, 1,815 hospitalized children with SSSS were identified in the PHIS database, and after application of the exclusion criteria, 1,259 children remained, with 1,247 (99%) receiving antibiotics (Figure). The antibiotic regimens received by these children are described in Table 1. Of these, 828 children (66%) received one of the three most common antistaphylococcal regimens (clindamycin, clindamycin + MSSA, and clindamycin + MRSA) and were included for further analysis.

Characteristics of the 828 children are presented in Table 2. Most children (82%) were aged 4 years or younger, and distributions of age, sex, and insurance payer were similar among children receiving the three regimens. Thirty-two percent had moderate to high illness severity, and 3.5% required management in the intensive care setting. Of the three antibiotic regimens, clindamycin monotherapy was most common (47%), followed by clindamycin plus MSSA coverage (33%), and clindamycin plus MRSA coverage (20%). A higher proportion of children in the clindamycin plus MRSA group were African American and were hospitalized in the South. Children receiving clindamycin plus MRSA coverage had higher illness severity (44%) as compared with clindamycin monotherapy (28%) and clindamycin plus MSSA coverage (32%) (P = .001). Additionally, a larger proportion of children treated with clindamycin plus MRSA coverage were managed in the intensive care setting as compared with the clindamycin plus MSSA or clindamycin monotherapy groups.

Among the 828 children with SSSS, the median LOS was 2 days (IQR, 2-3), and treatment failure was 1.1% (95% CI, 0.4-1.8). After adjustment for illness severity, race, payer, and region (Table 3), the three antibiotic regimens were not associated with significant differences in LOS or treatment failure. Costs were significantly different among the three antibiotic regimens. Clindamycin plus MRSA coverage was associated with the greatest costs, whereas clindamycin monotherapy was associated with the lowest costs (mean, $5,348 vs $4,839, respectively; P < .001) (Table 3). In a sensitivity analysis using an alternative antibiotic regimen definition, we found results in line with the primary analysis, with no statistically significant differences in LOS (P = .44) or treatment failure (P = .54), but significant differences in cost (P < .001). Additionally, the same findings were present for LOS, treatment failure, and cost when outcomes were stratified by illness severity (Appendix Table). However, significant contributors to the higher cost in the clindamycin plus MRSA group did vary by illness severity stratification. Laboratory, supply, and pharmacy cost categories differed significantly among antibiotic groups for the low illness severity strata, whereas pharmacy was the only significant cost category difference in moderate/high illness severity.

DISCUSSION

Clindamycin monotherapy, clindamycin plus MSSA coverage, and clindamycin plus MRSA coverage are the most commonly administered antistaphylococcal antibiotic regimens for children hospitalized with SSSS at US children’s hospitals. Our multicenter study found that, across these antistaphylococcal antibiotic regimens, there were no associated differences in hospital LOS or treatment failure. However, the antibiotic regimens were associated with significant differences in overall hospital costs. These findings suggest that the use of clindamycin with additional MSSA or MRSA antibiotic coverage for children with SSSS may not be associated with additional clinical benefit, as compared with clindamycin monotherapy, and could potentially be more costly.

Prior literature describing LOS in relation to antibiotic use for children with SSSS is limited. Authors of a recent case series of 21 children in Philadelphia reported approximately 50% of children received clindamycin monotherapy or combination therapy, but patient outcomes such as LOS were not described.⁹ Clindamycin use and outcomes have been described in smaller studies and case reports of SSSS, which reported positive outcomes such as patient recovery and lack of disease recurrence.^2,9,21 A small retrospective, comparative effectiveness study of 30 neonates with SSSS examined beta-lactamase–resistant penicillin use with and without cephalosporins. They found no effect on LOS, but findings were limited by a small sample size.²² Our study cohort included relatively few neonates, and thus our findings may not be applicable to this population subgroup. We chose not to include regimens with third-generation cephalosporins or ampicillin, which may have limited the number of included neonates, because these antibiotics are frequently administered during evaluation for invasive bacterial infections.²³ We found a very low occurrence of treatment failure in our study cohort across all three groups, which is consistent with other studies of SSSS that report an overall good prognosis and low recurrence and/or readmission rates.^6,16,24 The low prevalence of treatment failure, however, precluded our ability to detect small differences among antibiotic regimen groups that may exist.

We observed that cost differed significantly across antibiotic regimen groups, with lowest cost associated with clindamycin monotherapy in adjusted analysis despite similar LOS. Even with our illness-severity adjustment, there may have been other unmeasured factors resulting in the higher cost associated with the combination groups. Hence, we also examined cost breakdown with a stratified analysis by illness severity. We found that pharmacy costs were significantly different among antibiotic groups in both illness severity strata, whereas those with low illness severity also differed by laboratory and supply costs. Thus, pharmacy cost differences may be the largest driver in the cost differential among groups. Lower cost in the clindamycin monotherapy group is likely due to administration of a single antibiotic. The reason for supply and laboratory cost differences is uncertain, but higher cost in the clindamycin plus MRSA group could possibly be from laboratory testing related to drug monitoring (eg, renal function testing or drug levels). While other studies have reported costs for hospitalized children with SSSS associated with different patient characteristics or diagnostic testing,^1,16 to our knowledge, no other studies have reported cost related to antibiotic regimens for SSSS. As healthcare reimbursements shift to value-based models, identifying treatment regimens with equal efficacy but lower cost will become increasingly important. Future studies should also examine other covariates and outcomes, such as oral vs parenteral antibiotic use, use of monitoring laboratories related to antibiotic choice, and adverse drug effects.

Several strengths and additional limitations apply to our study. Our study is one of the few to describe outcomes associated with antibiotic regimens for children with SSSS. With the PHIS database, we were able to include a large number of children with SSSS from children’s hospitals across the United States. Although the PHIS database affords these strengths, there are limitations inherent to administrative data. Children with SSSS were identified by documented ICD-9 and ICD-10 diagnostic codes, which might lead to misclassification. However, misclassification is less likely because only one ICD-9 and ICD-10 code exists for SSSS, and the characteristics of this condition are specific. Also, diagnostic codes for other dermatologic conditions (eg, scarlet fever) were excluded to further reduce the chance of misclassification. A limitation to our use of PHIS billing codes was the inability to confirm the dosage of antibiotics given, the number of doses, or whether antibiotics were prescribed upon discharge. Another limitation is that children whose antibiotic therapy was changed during hospitalization (eg, from clindamycin monotherapy to cefazolin monotherapy) were categorized into the combination groups. However, the sensitivity analysis performed based on a stricter antibiotic group definition (receipt of both antibiotics on at least 2 calendar days) did not alter the outcomes, which is reassuring. We were unable to assess the use of targeted antibiotic therapy because clinical data (eg, microbiology results) were not available. However, this may be less important because some literature suggests that cultures for S aureus are obtained infrequently² and may be difficult to interpret when obtained,²⁵ since culture growth can represent colonization rather than causative strains. An additional limitation is that administrative data do not include certain clinical outcomes, such as fever duration or degree of skin involvement, which could have differed among the groups. Last, the PHIS database only captures revisits or readmissions to PHIS hospitals, and so we are unable to exclude the possibility of a child being seen at or readmitted to another hospital.

Due to the observational design of this study and potential for incomplete measurement of illness severity, we recommend a future prospective trial with randomization to confirm these findings. One possible reason that LOS did not differ among groups is that the burden of clindamycin-resistant strains in our cohort could be low, and the addition of MSSA or MRSA coverage does not result in a clinically important increase in S aureus coverage. However, pooled pediatric hospital antibiogram data suggest the overall rate of clindamycin resistance is close to 20% in hospitals located in all US regions.²⁶ Limited studies also suggest that MSSA may be the predominant pathogen associated with SSSS.^2,9 To address this, future randomized trials could compare the effectiveness of clindamycin monotherapy to MSSA-specific agents like cefazolin or nafcillin. Unfortunately, anti-MSSA monotherapy was not evaluated in our study because very few children received this treatment. Using monotherapy as opposed to multiple antibiotics has the potential to promote antibiotic stewardship for antistaphylococcal antibiotics in the management of SSSS. Reducing unnecessary antibiotic use not only potentially affects antibiotic resistance, but could also benefit patients in reducing possible side effects, cost, and IV catheter complications.²⁷ However, acknowledging our study limitations, our findings should be applied cautiously in clinical settings, in the context of local antibiogram data, individual culture results, and specific patient factors. The local clindamycin resistance rate for both MSSA and MRSA should be considered. Many antibiotics chosen to treat MRSA—such as vancomycin and trimethoprim/sulfamethoxazole—will also have anti-MSSA activity and may have lower local resistance rates than clindamycin. Practitioners may also consider how each antibiotic kills bacteria; for example, beta-lactams rely on bacterial replication, but clindamycin does not. Each factor should influence how empiric treatment, whether monotherapy or combination, is chosen for children with SSSS.

CONCLUSION

In this large, multicenter cohort of hospitalized children with SSSS, we found that the addition of MSSA or MRSA coverage to clindamycin monotherapy was not associated with differences in outcomes of hospital LOS and treatment failure. Furthermore, clindamycin monotherapy was associated with lower overall cost. Prospective randomized studies are needed to confirm these findings and assess whether clindamycin monotherapy, monotherapy with an anti-MSSA antibiotic, or alternative regimens are most effective for treatment of children with SSSS.

Healthcare in rural areas faces increasing challenges due to community hospital closures, physician shortages, and a more concentrated population of older adults with higher rates of comorbid conditions than their urban counterparts.^1-3 Critical access hospitals (CAHs), which primarily serve rural areas, have fewer clinical capabilities, worse process-of-care measures, and higher mortality rates for some conditions when compared to non-CAHs.⁴ As such, CAHs are closing at record numbers across the United States,⁵ resulting in loss of available hospital beds and patient access to timely emergency services,⁶ which can worsen outcomes, further widening the rural-urban healthcare gap.^7,8 Furthermore, this strain on an overwhelmed health system in the most vulnerable areas restricts the ability to respond to healthcare crises like the coronavirus disease 2019 pandemic.⁹

Providing adequate staff for currently available hospital beds is also a problem in rural areas. Studies demonstrating improved outcomes, decreased length of stay (LOS), and increased quality with hospitalist services have resulted in a high demand for hospitalists nationwide.^10-12 Recruiting hospitalists to work in rural areas, however, has become increasingly challenging due to low-patient volumes, financial viability of hospitalist-model adoption, and provider shortages.^13,14 Recently, the Veterans Health Administration (VHA) reported a 28% nationwide shortage of hospitalists,¹⁵ which disproportionally affects rural VHA hospitals. Staffing difficulties and reliance on intermittent providers were reported by more than 80% of rural and low-complexity VHA facilities.¹⁶

Telehospitalist services (THS) can help deliver high-quality care to rural residents locally, decrease travel expenses, support hospital volume, and increase healthcare capacity in response to a pandemic.^14,17,18 Only a few studies have described THS (mostly with overnight or cross-coverage models directed to CAHs), and clinical outcomes have been inconsistently reported.^17,19-21 Furthermore, no program has been conducted within an integrated health system akin to the VHA. The primary objective of this quality improvement (QI) initiative was to perform a mixed-methods evaluation of THS between VHA hospitals to compare clinical outcomes and patient and staff satisfaction. Secondary outcomes included description of the implementation process, unexpected challenges, and subsequent QI initiatives. These results will expand the knowledge on feasibility of THS and provide implementation guidance.

METHODS

A mixed-methods approach was used to evaluate outcomes of this QI project. Reporting follows the revised Standards for Quality Improvement Reporting Excellence (SQUIRE 2.0).²²

Context

The VHA is the largest integrated healthcare system in the United States, with more than 8 million veterans enrolled, more than 30% of whom reside in a rural area. The VHA comprises more than 1,000 outpatient clinics and 170 acute care VA Medical Centers,^23,24 including more than 35 rural and low-complexity hospitals.²⁵ Low-complexity hospitals are those with the lowest volume and levels of patient complexity and minimal or no teaching programs, research, intensive care unit (ICU) beds, and subspecialists. Lack of reimbursement and interstate licensing, often cited as barriers to telemedicine, do not apply to the VHA. The hub site was a large tertiary care (high-complexity) VHA hospital located in Iowa City, Iowa. The spoke site was a low-complexity (10-bed acute inpatient unit with no ICU) rural VA hospital located in Tomah, Wisconsin.

Study Population

The preimplementation cohort for comparison included all patients admitted between January 1, 2018, and January 6, 2019. The postimplementation study cohort included all observation and acute care admissions during the pilot phase (January 7 to May 3, 2019) and sustainability phase (July 15 to December 31, 2019). The postimplementation analysis excluded the time period of May 4 to July 14, 2019, due to an interruption (gap) in THS. The gap period allowed for preliminary data analysis, optimization of the telecommunication system, and the recruitment and training of additional providers who could provide long-term staffing to the service.

Intervention

Preimplementation

Prior to THS implementation, Tomah’s inpatient ward was staffed by one physician per shift, who could be a hospitalist, medical officer of the day (MOD), or an intermittent provider (locum tenens). Hospitalists covering the acute inpatient ward prior to the THS transitioned to cover weekends, nights, and urgent care service shifts.

We visited the spoke site and held information-sharing sessions with key stakeholders (administrators, clinician leaders, nurses, and ancillary staff) prior to kick-off. Recurrent phone meetings addressed anticipated and emerging challenges. Telehospitalist and local providers underwent technology and service training.

Technology and Connectivity

A low-cost technology system using tablet computers provided Health Insurance Portability and Accountability Act–compliant videoconferencing with a telehospitalist at the hub site. An Eko-Core digital stethoscope^® with a web-based audio stream was available. Telehospitalists conducted encounters from a private office space with telehealth capabilities. A total of $9,000 was spent on equipment at both sites. Due to connectivity problems and data limits, the tablets were switched to mobile computer-on-wheels workstations and hospital-based Wi-Fi for the sustainability phase.

THS Description

An experienced hub hospitalist, together with an advanced practice provider (APP; nurse practitioner [NP] or physician assistant [PA]), cared for all patients admitted to the 10-bed inpatient unit at the spoke site, Monday through Friday from 8:00 AM to 4:30 PM. The APP had limited or no prior experience in acute inpatient medicine. The telehospitalist worked as a team with the APP. The APP was the main point of contact for nurses, performed physical examinations, and directed patient care to their level of comfort (in a similar manner as a teaching team). The telehospitalist conducted bedside patient rounds, participated in multidisciplinary huddles, and shared clinical documentation and administrative duties with the APP. The telehospitalist was the primary staff for admitted patients and had full access to the electronic health record (EHR). The THS was staffed by 10 hospitalists during the study period. Overnight and weekend cross-coverage and admissions were performed by MODs, who also covered the urgent care and cross-covered other nonmedical units.

Quantitative Evaluation Methods

Workload and Clinical Outcomes

An EHR query identified all patients admitted during the pre- and postimplementation periods. Demographic data, clinical Nosos risk scores,^26,27 and top admission diagnoses were reported. Workload was evaluated using the average number of encounters per day and self-reported telehospitalist worksheets, which were cross-referenced with EHR data. Clinical outcomes included LOS, 30-day hospital readmission rate, 30-day standardized mortality (SMR30), in-hospital mortality, and VHA-specific inpatient quality metrics. Independent sample t tests for continuous variables and chi-square tests or Fisher’s exact test (for patient class) for categorical variables were used to compare pre- and postimplementation groups. Statistical process control (SPC) charts evaluated changes over time. All analyses were conducted using Microsoft Excel and R.²⁸

Provider Satisfaction

Anonymous surveys were distributed to spoke-site inpatient and administrative staff at 1 month and 12 months postimplementation, assessing satisfaction, technology/connectivity, communication, and challenges (Appendix Figure 1). Satisfaction of the telehospitalist physicians at the hub site was measured 12 months postimplementation by a 26-question survey assessing the same domains, plus quality of care (Appendix Figure 2).

Patient Satisfaction

The VHA Survey of Healthcare Experiences of Patients (SHEP), a version of the Hospital Consumer Assessment of Healthcare Providers and Systems Survey,^29,30 was mailed to all patients after discharge. Survey responses concerning inpatient provider care (eg, care coordination, communication, hospital rating, willingness to recommend the hospital) during the pre- and postimplementation phases were compared using a two-sample test of independent proportions. Responses obtained during May and June 2019 were excluded.

Qualitative Evaluation Methods

The qualitative researcher observed information-sharing meetings and facilitated unstructured interviews with clinical and administrative staff during site visits preimplementation and 3 months after implementation. Interviews with administrators and clinical staff addressed their experiences with the THS, staff’s perception of patient and family response to THS implementation, administrative impacts, challenges, and strengths. All interviews and meetings were documented with handwritten notes and audio recordings. Interview summary notes were typed into a Microsoft Word document, verified by the physician-investigator, and synthesized by inductive themes into site-visit reports. Audio recordings were uploaded to a secure computer, transcribed, and reviewed for accuracy. The qualitative researcher also identified emerging themes from open-ended survey responses. Process evaluation findings were shared with administration at the spoke site.

The authors had full access to, and took full responsibility for, the integrity of the data. The project was evaluated by the University of Iowa Institutional Review Board and the Iowa City VA Research and Development Committee and was determined to be a non–human-subjects QI project.

RESULTS

Quantitative Workload and Clinical Outcomes

There were 822 admissions during the preimplementation period and 550 admissions during the postimplementation period (253 during the pilot and 297 during sustainability phase). Patient characteristics pre- and postimplementation were not significantly different (Table 1). The median patient age was 65 years; 96% of patients were male, and 83% were rural residents. The most common admission diagnosis was alcohol-related (36%); regarding patient disposition, 78% of admissions were discharged home.

Workload

There were 502 patient encounters staffed by the telehospitalist in the pilot phase, with an average of 6.25 encounters per day, and a telehospitalist-reported workload of 7 hours per day. There were 538 patient encounters, with an average of 4.67 encounters per day and a workload of 5.6 hours per day in the sustainability phase. The average daily census decreased from 5.0 (SD, 1.1) patients per day during preimplementation to 3.1 (SD, 0.5) patients per day during postimplementation (Table 2). In some of the months during the study period, admissions decreased below the lower SPC limit, suggesting a significant change (Figure). Adjusted LOS was significantly lower, with 3.0 (SD, 0.7) days vs 2.3 (SD, 0.3) days in the pre- and postimplementation periods, respectively. Bed occupancy rates were significantly lower in the sustainability phase compared with the pilot phase and the preimplementation period. Readmission rates varied, ranging from <10% to >30%, not significantly different but slightly higher in the postimplementation period. Readmission rates for heart failure, chronic obstructive pulmonary disease, and pneumonia remained unchanged; other medical readmissions (mostly alcohol-related) were slightly higher in the postimplementation period.

In-hospital mortality and SMR30 did not change significantly, but there was improvement in the 12-month rolling average of the observed/expected SMR30 from 1.40 to 1.08. Additional VHA-specific quality metrics were monitored and showed either small improvements or no change (data not shown).

Satisfaction at Hub and Spoke Sites

After sending two reminder communications via email, the telehospitalist satisfaction survey had a total response rate of 90% (9/10). Telehospitalists were satisfied or very satisfied (89%) with the program and the local providers (88.9%), rating their experience as good or excellent (100%) (Table 3). Communication with patients, families, and local staff was noted as being “positive” or “mostly positive.” Telehospitalists reported confidence in the accuracy of their diagnoses and rated the quality of care as being equal to that of a face-to-face encounter. Connectivity problems were prevalent, although most providers were able to resort to a back-up plan. Other challenges included differences in culture and concerns about liability. We received 27 responses from the spoke-site satisfaction survey; the response rate could not be determined because the survey was distributed by the spoke site for anonymity. Of the respondents, 37% identified as nurses, 25.9% as healthcare providers (APPs or physicians), and 33.3% as other staff (eg, social worker, nutritionist, physical therapist, utilization management, administrators); 3.7% did not respond. Among the participants, 88% had personally interacted with the THS. Most providers and other staff perceived THS as valuable (57.1% and 77.8%, respectively) and were satisfied or highly satisfied with THS (57.1% and 55.6%, respectively). On average, nurses provided lower ratings across all survey items than providers and other staff. Challenges noted by all staff included issues with communication, workflow, and technology/connectivity.

Regarding patient satisfaction, the SHEP survey showed a significant improvement in care coordination (18%; P = .02) and a nonsignificant improvement in communications about medications (5%; P =.054). The remaining items in the survey, including overall hospital rating and willingness to recommend the hospital, were unchanged (Appendix Table).

Qualitative Strengths

Our process evaluation identified high quality of care and teamwork as contributors to the success of the program. Overall, staff credited perceived improvements in quality of care to the quality of providers staffing the THS, including the local APPs. Noting the telehospitalists’ knowledge base and level of engagement as key attributes, one staff member commented: “I prefer a telehospitalist that really care[s] about patients than some provider that is physically here but does not engage.” Staff perceived improvements in the continuity of care, as well as care processes such as handoffs and transitions of care.

Improvements in teamwork were perceived compared with the previous model of care. Telehospitalists were lauded for their professionalism and communication skills. Overall, nurses felt providers in the THS listened more to their views. In addition, nurse respondents felt they could learn from several providers and said they enjoyed the telehospitalists’ disposition to teach and discuss patient care. The responsiveness of the THS staff was instrumental in building teamwork and acceptance. A bedside interdisciplinary protocol was established for appropriate patients. Local staff felt this was crucial for teamwork and patient satisfaction. Telehospitalists reported high-value in interdisciplinary rounds, facilitating interaction with nurses and ancillary staff. Handoff problems were identified, leading to QI initiatives to mitigate those issues.

Challenges

The survey identified administrative barriers, technical difficulties, workflow constraints, and clinical concerns. The credentialing process was complicated, delaying the onboarding of telehospitalists. Internet connectivity was inconsistent, leading to disruption in video communications; however, during the sustainability phase, updated technology improved communications. The communication workflow was resisted by some nurses, who wanted to phone the telehospitalist directly rather than having the local APP as the first contact. Secure messaging was enabled to allow nurses direct contact during the sustainability phase.

Workload was a concern among telehospitalists and local staff. Telehospitalists perceived the documentation requirements and administrative workload to be two to three times higher than at other hospitals—despite the lower number of encounters. Finally, clinical concerns from spoke-site clinicians included a perceived rise in the acuity of patients (which was not evident by the Nosos score) and delayed decisions to transfer-out patients. These concerns were addressed with educational sessions for telehospitalists during the sustainability phase.

Additional Quality Improvement Projects

The implementation of THS resulted in QI initiatives at the spoke site, including an EHR-integrated handoff tool; a documentation evaluation that led to the elimination of duplicative, inefficient, and error-prone templates; and a revision of the alcohol withdrawal treatment protocol during the sustainability phase to reduce the use of intravenous benzodiazepines. A more comprehensive benzodiazepine-sparing alcohol withdrawal treatment protocol was also developed but was not implemented until after the study period (January 2020).

DISCUSSION

Our pre-post study evaluation found implementation of a THS to be noninferior to face-to-face care, with no significant change in mortality, readmission rate, or patient satisfaction. The significant improvement observed in LOS is consistent with the adoption of hospitalist models in other medical care settings,¹¹ but had not been reported by previous telehospitalist studies. For example, in their retrospective chart review comparing an NP-supported telehospitalist model to locum tenens hospitalists, Boltz et al found no difference in LOS.³¹ Moreover, as in our study, they found no differences in readmissions, mortality, and patient satisfaction.³¹ Similarly, Kuperman et al reported unchanged daily census, LOS, and transfer rates from a CAH with their virtual hospitalist program, but a decrease in the percentage of patients transferred-out from the emergency department, suggesting that more patients were treated locally.¹⁹

Reduction in LOS is one of the primary measures of efficiency in hospital care³¹; reducing LOS while maintaining the quality of care lowers hospital costs. The reduction in LOS in our study could be attributed to greater continuity of care, engagement/experience of the telehospitalists, or other factors. This decrease in LOS and slight reduction in admissions resulted in an overall lower daily census during the study period and impacted efficiency. Our study was unable to determine the cause for the reduction in admissions; however, several concurrent events, including the expansion of community-care options for veterans under the MISSION ACT (Maintaining Internal Systems and Strengthening Integrated Outside Networks Act) in June 2019, a nationwide smoking ban at VA facilities (October 2019), and a modification in the alcohol withdrawal treatment protocol might have influenced veterans’ choice of hospital.

Readmission rates were slightly higher, though nonsignificant, in the postimplementation period. Alcohol-related readmissions accounted for most readmissions; some of the protocol changes, such as admitting all patients with alcohol withdrawal to inpatient class instead of admitting some to the observation class, accounted for part of the increase in readmission rates. Readmission rates for other conditions such as chronic obstructive pulmonary disease, chronic heart failure, or pneumonia were not significantly different, suggesting that the reduction in LOS did not result in an unintended increased readmission rate for those conditions.

Rural hospitals are struggling with staffing and finances. Resorting to locum tenens staffing is costly and can result in variable quality of care.^32,33 APPs are increasingly taking on hospitalist positions, with 65% of adult hospitalist programs, including half of all VHA hospitals, employing NPs and PAs.^34,35 In response to this expanded scope of practice, hospitals employing APPs in hospitalist roles must comply with state and federal laws, which often require that APPs be supervised by or work in collaboration with an on-site or off-site physician. The THS is a great model to support APPs and address staffing and cost challenges in low-volume rural facilities, while maintaining quality of care. Some APP-telehospitalist programs similar to ours have reported cost reductions of up 58% compared to programs that employ locum tenens physicians.³¹ In our model, we assume that a single telehospitalist hub could provide coverage to two or three spoke sites with APP support, reducing staffing costs.

Hub telehospitalists reported satisfaction with the program, and they perceived the quality of care to be comparable to face-to-face encounters; their responses were consistent with those previously reported in an evaluation of telemedicine acute care by JaKa et al.²⁰ Spoke-site staff, however, had a mixed level of satisfaction, which was different from responses reported by JaKa et al.²⁰ The primary challenges encountered were technological and communication issues, differences in cultures of care between the hub and the spoke sites, and buy-in from frontline staff. Differences in expectations and unclear role definitions between the local APP and the telehospitalist were identified as contributors to dissatisfaction with the program by the nursing staff. Modifications to the communication processes between nurses and telehospitalists and role clarification improved the experience. Culture and practice differences between spoke physicians and the telehospitalist persisted throughout the program implementation, and likely affected the hub providers’ perception of the THS. This was evidenced by reluctance from spoke physicians to implement warm handoffs or participate in THS meetings and resistance to protocol changes. Additional evaluations, collaborations. and interventions are required to improve satisfaction of spoke-site staff.

This study has several limitations. First, the VHA is an integrated health system, one that serves an older, predominantly male patient population. Also, the lack of reimbursement and interstate licensing restrictions limit generalizability of these results to other CAHs or healthcare systems. Furthermore, the intervention was limited to a single rural site; while this allowed for a detailed evaluation, unique barriers or facilitators might exist that limit its applicability. In addition, QI initiatives implemented by the VHA during the project period might have confounded some of our results. Last, patient satisfaction survey data are overall limited in their ability to fully assess patient’s experience and satisfaction with the program. Further qualitative studies are needed to gain deeper insight into patient perspectives with the THS and whether modality of care delivery influences patients’ care decisions. Future studies should consider a multisite design with one or more hubs and multiple spoke sites.

CONCLUSION

Telehospitalist services are a feasible and safe approach to provide inpatient services and address staffing needs of rural hospitals. To enhance program performance, it is essential to ensure adequate technological quality, clearly delineate and define roles and responsibilities of the care team, and address communication issues or staff concerns in a timely manner.

Acknowledgments

The authors thank the staff, administration, and leadership at the Tomah and Iowa City VA Medical Centers for working with us on this project. They offer special thanks to Kevin Glenn, MD, MS, Ethan Kuperman, MD, MS, FHM, and Jennifer Chapin, MSN, RN, for sharing their expertise, and the telehealth team, including Nathaniel Samuelson, Angela McDowell, and Katrin Metcalf.

Disclaimer

The views expressed in this article are those of the authors and do not necessarily represent the views of the Department of Veterans Affairs.

Healthcare in rural areas faces increasing challenges due to community hospital closures, physician shortages, and a more concentrated population of older adults with higher rates of comorbid conditions than their urban counterparts.^1-3 Critical access hospitals (CAHs), which primarily serve rural areas, have fewer clinical capabilities, worse process-of-care measures, and higher mortality rates for some conditions when compared to non-CAHs.⁴ As such, CAHs are closing at record numbers across the United States,⁵ resulting in loss of available hospital beds and patient access to timely emergency services,⁶ which can worsen outcomes, further widening the rural-urban healthcare gap.^7,8 Furthermore, this strain on an overwhelmed health system in the most vulnerable areas restricts the ability to respond to healthcare crises like the coronavirus disease 2019 pandemic.⁹

Providing adequate staff for currently available hospital beds is also a problem in rural areas. Studies demonstrating improved outcomes, decreased length of stay (LOS), and increased quality with hospitalist services have resulted in a high demand for hospitalists nationwide.^10-12 Recruiting hospitalists to work in rural areas, however, has become increasingly challenging due to low-patient volumes, financial viability of hospitalist-model adoption, and provider shortages.^13,14 Recently, the Veterans Health Administration (VHA) reported a 28% nationwide shortage of hospitalists,¹⁵ which disproportionally affects rural VHA hospitals. Staffing difficulties and reliance on intermittent providers were reported by more than 80% of rural and low-complexity VHA facilities.¹⁶

Telehospitalist services (THS) can help deliver high-quality care to rural residents locally, decrease travel expenses, support hospital volume, and increase healthcare capacity in response to a pandemic.^14,17,18 Only a few studies have described THS (mostly with overnight or cross-coverage models directed to CAHs), and clinical outcomes have been inconsistently reported.^17,19-21 Furthermore, no program has been conducted within an integrated health system akin to the VHA. The primary objective of this quality improvement (QI) initiative was to perform a mixed-methods evaluation of THS between VHA hospitals to compare clinical outcomes and patient and staff satisfaction. Secondary outcomes included description of the implementation process, unexpected challenges, and subsequent QI initiatives. These results will expand the knowledge on feasibility of THS and provide implementation guidance.

METHODS

A mixed-methods approach was used to evaluate outcomes of this QI project. Reporting follows the revised Standards for Quality Improvement Reporting Excellence (SQUIRE 2.0).²²

Context

The VHA is the largest integrated healthcare system in the United States, with more than 8 million veterans enrolled, more than 30% of whom reside in a rural area. The VHA comprises more than 1,000 outpatient clinics and 170 acute care VA Medical Centers,^23,24 including more than 35 rural and low-complexity hospitals.²⁵ Low-complexity hospitals are those with the lowest volume and levels of patient complexity and minimal or no teaching programs, research, intensive care unit (ICU) beds, and subspecialists. Lack of reimbursement and interstate licensing, often cited as barriers to telemedicine, do not apply to the VHA. The hub site was a large tertiary care (high-complexity) VHA hospital located in Iowa City, Iowa. The spoke site was a low-complexity (10-bed acute inpatient unit with no ICU) rural VA hospital located in Tomah, Wisconsin.

Study Population

The preimplementation cohort for comparison included all patients admitted between January 1, 2018, and January 6, 2019. The postimplementation study cohort included all observation and acute care admissions during the pilot phase (January 7 to May 3, 2019) and sustainability phase (July 15 to December 31, 2019). The postimplementation analysis excluded the time period of May 4 to July 14, 2019, due to an interruption (gap) in THS. The gap period allowed for preliminary data analysis, optimization of the telecommunication system, and the recruitment and training of additional providers who could provide long-term staffing to the service.

Intervention

Preimplementation

Prior to THS implementation, Tomah’s inpatient ward was staffed by one physician per shift, who could be a hospitalist, medical officer of the day (MOD), or an intermittent provider (locum tenens). Hospitalists covering the acute inpatient ward prior to the THS transitioned to cover weekends, nights, and urgent care service shifts.

We visited the spoke site and held information-sharing sessions with key stakeholders (administrators, clinician leaders, nurses, and ancillary staff) prior to kick-off. Recurrent phone meetings addressed anticipated and emerging challenges. Telehospitalist and local providers underwent technology and service training.

Technology and Connectivity

A low-cost technology system using tablet computers provided Health Insurance Portability and Accountability Act–compliant videoconferencing with a telehospitalist at the hub site. An Eko-Core digital stethoscope^® with a web-based audio stream was available. Telehospitalists conducted encounters from a private office space with telehealth capabilities. A total of $9,000 was spent on equipment at both sites. Due to connectivity problems and data limits, the tablets were switched to mobile computer-on-wheels workstations and hospital-based Wi-Fi for the sustainability phase.

THS Description

An experienced hub hospitalist, together with an advanced practice provider (APP; nurse practitioner [NP] or physician assistant [PA]), cared for all patients admitted to the 10-bed inpatient unit at the spoke site, Monday through Friday from 8:00 AM to 4:30 PM. The APP had limited or no prior experience in acute inpatient medicine. The telehospitalist worked as a team with the APP. The APP was the main point of contact for nurses, performed physical examinations, and directed patient care to their level of comfort (in a similar manner as a teaching team). The telehospitalist conducted bedside patient rounds, participated in multidisciplinary huddles, and shared clinical documentation and administrative duties with the APP. The telehospitalist was the primary staff for admitted patients and had full access to the electronic health record (EHR). The THS was staffed by 10 hospitalists during the study period. Overnight and weekend cross-coverage and admissions were performed by MODs, who also covered the urgent care and cross-covered other nonmedical units.

Quantitative Evaluation Methods

Workload and Clinical Outcomes

An EHR query identified all patients admitted during the pre- and postimplementation periods. Demographic data, clinical Nosos risk scores,^26,27 and top admission diagnoses were reported. Workload was evaluated using the average number of encounters per day and self-reported telehospitalist worksheets, which were cross-referenced with EHR data. Clinical outcomes included LOS, 30-day hospital readmission rate, 30-day standardized mortality (SMR30), in-hospital mortality, and VHA-specific inpatient quality metrics. Independent sample t tests for continuous variables and chi-square tests or Fisher’s exact test (for patient class) for categorical variables were used to compare pre- and postimplementation groups. Statistical process control (SPC) charts evaluated changes over time. All analyses were conducted using Microsoft Excel and R.²⁸

Provider Satisfaction

Anonymous surveys were distributed to spoke-site inpatient and administrative staff at 1 month and 12 months postimplementation, assessing satisfaction, technology/connectivity, communication, and challenges (Appendix Figure 1). Satisfaction of the telehospitalist physicians at the hub site was measured 12 months postimplementation by a 26-question survey assessing the same domains, plus quality of care (Appendix Figure 2).

Patient Satisfaction

The VHA Survey of Healthcare Experiences of Patients (SHEP), a version of the Hospital Consumer Assessment of Healthcare Providers and Systems Survey,^29,30 was mailed to all patients after discharge. Survey responses concerning inpatient provider care (eg, care coordination, communication, hospital rating, willingness to recommend the hospital) during the pre- and postimplementation phases were compared using a two-sample test of independent proportions. Responses obtained during May and June 2019 were excluded.

Qualitative Evaluation Methods

The qualitative researcher observed information-sharing meetings and facilitated unstructured interviews with clinical and administrative staff during site visits preimplementation and 3 months after implementation. Interviews with administrators and clinical staff addressed their experiences with the THS, staff’s perception of patient and family response to THS implementation, administrative impacts, challenges, and strengths. All interviews and meetings were documented with handwritten notes and audio recordings. Interview summary notes were typed into a Microsoft Word document, verified by the physician-investigator, and synthesized by inductive themes into site-visit reports. Audio recordings were uploaded to a secure computer, transcribed, and reviewed for accuracy. The qualitative researcher also identified emerging themes from open-ended survey responses. Process evaluation findings were shared with administration at the spoke site.

The authors had full access to, and took full responsibility for, the integrity of the data. The project was evaluated by the University of Iowa Institutional Review Board and the Iowa City VA Research and Development Committee and was determined to be a non–human-subjects QI project.

RESULTS

Quantitative Workload and Clinical Outcomes

There were 822 admissions during the preimplementation period and 550 admissions during the postimplementation period (253 during the pilot and 297 during sustainability phase). Patient characteristics pre- and postimplementation were not significantly different (Table 1). The median patient age was 65 years; 96% of patients were male, and 83% were rural residents. The most common admission diagnosis was alcohol-related (36%); regarding patient disposition, 78% of admissions were discharged home.

Workload

There were 502 patient encounters staffed by the telehospitalist in the pilot phase, with an average of 6.25 encounters per day, and a telehospitalist-reported workload of 7 hours per day. There were 538 patient encounters, with an average of 4.67 encounters per day and a workload of 5.6 hours per day in the sustainability phase. The average daily census decreased from 5.0 (SD, 1.1) patients per day during preimplementation to 3.1 (SD, 0.5) patients per day during postimplementation (Table 2). In some of the months during the study period, admissions decreased below the lower SPC limit, suggesting a significant change (Figure). Adjusted LOS was significantly lower, with 3.0 (SD, 0.7) days vs 2.3 (SD, 0.3) days in the pre- and postimplementation periods, respectively. Bed occupancy rates were significantly lower in the sustainability phase compared with the pilot phase and the preimplementation period. Readmission rates varied, ranging from <10% to >30%, not significantly different but slightly higher in the postimplementation period. Readmission rates for heart failure, chronic obstructive pulmonary disease, and pneumonia remained unchanged; other medical readmissions (mostly alcohol-related) were slightly higher in the postimplementation period.

In-hospital mortality and SMR30 did not change significantly, but there was improvement in the 12-month rolling average of the observed/expected SMR30 from 1.40 to 1.08. Additional VHA-specific quality metrics were monitored and showed either small improvements or no change (data not shown).

Satisfaction at Hub and Spoke Sites

After sending two reminder communications via email, the telehospitalist satisfaction survey had a total response rate of 90% (9/10). Telehospitalists were satisfied or very satisfied (89%) with the program and the local providers (88.9%), rating their experience as good or excellent (100%) (Table 3). Communication with patients, families, and local staff was noted as being “positive” or “mostly positive.” Telehospitalists reported confidence in the accuracy of their diagnoses and rated the quality of care as being equal to that of a face-to-face encounter. Connectivity problems were prevalent, although most providers were able to resort to a back-up plan. Other challenges included differences in culture and concerns about liability. We received 27 responses from the spoke-site satisfaction survey; the response rate could not be determined because the survey was distributed by the spoke site for anonymity. Of the respondents, 37% identified as nurses, 25.9% as healthcare providers (APPs or physicians), and 33.3% as other staff (eg, social worker, nutritionist, physical therapist, utilization management, administrators); 3.7% did not respond. Among the participants, 88% had personally interacted with the THS. Most providers and other staff perceived THS as valuable (57.1% and 77.8%, respectively) and were satisfied or highly satisfied with THS (57.1% and 55.6%, respectively). On average, nurses provided lower ratings across all survey items than providers and other staff. Challenges noted by all staff included issues with communication, workflow, and technology/connectivity.

Regarding patient satisfaction, the SHEP survey showed a significant improvement in care coordination (18%; P = .02) and a nonsignificant improvement in communications about medications (5%; P =.054). The remaining items in the survey, including overall hospital rating and willingness to recommend the hospital, were unchanged (Appendix Table).

Qualitative Strengths

Our process evaluation identified high quality of care and teamwork as contributors to the success of the program. Overall, staff credited perceived improvements in quality of care to the quality of providers staffing the THS, including the local APPs. Noting the telehospitalists’ knowledge base and level of engagement as key attributes, one staff member commented: “I prefer a telehospitalist that really care[s] about patients than some provider that is physically here but does not engage.” Staff perceived improvements in the continuity of care, as well as care processes such as handoffs and transitions of care.

Improvements in teamwork were perceived compared with the previous model of care. Telehospitalists were lauded for their professionalism and communication skills. Overall, nurses felt providers in the THS listened more to their views. In addition, nurse respondents felt they could learn from several providers and said they enjoyed the telehospitalists’ disposition to teach and discuss patient care. The responsiveness of the THS staff was instrumental in building teamwork and acceptance. A bedside interdisciplinary protocol was established for appropriate patients. Local staff felt this was crucial for teamwork and patient satisfaction. Telehospitalists reported high-value in interdisciplinary rounds, facilitating interaction with nurses and ancillary staff. Handoff problems were identified, leading to QI initiatives to mitigate those issues.

Challenges

The survey identified administrative barriers, technical difficulties, workflow constraints, and clinical concerns. The credentialing process was complicated, delaying the onboarding of telehospitalists. Internet connectivity was inconsistent, leading to disruption in video communications; however, during the sustainability phase, updated technology improved communications. The communication workflow was resisted by some nurses, who wanted to phone the telehospitalist directly rather than having the local APP as the first contact. Secure messaging was enabled to allow nurses direct contact during the sustainability phase.

Workload was a concern among telehospitalists and local staff. Telehospitalists perceived the documentation requirements and administrative workload to be two to three times higher than at other hospitals—despite the lower number of encounters. Finally, clinical concerns from spoke-site clinicians included a perceived rise in the acuity of patients (which was not evident by the Nosos score) and delayed decisions to transfer-out patients. These concerns were addressed with educational sessions for telehospitalists during the sustainability phase.

Additional Quality Improvement Projects

The implementation of THS resulted in QI initiatives at the spoke site, including an EHR-integrated handoff tool; a documentation evaluation that led to the elimination of duplicative, inefficient, and error-prone templates; and a revision of the alcohol withdrawal treatment protocol during the sustainability phase to reduce the use of intravenous benzodiazepines. A more comprehensive benzodiazepine-sparing alcohol withdrawal treatment protocol was also developed but was not implemented until after the study period (January 2020).

DISCUSSION

Our pre-post study evaluation found implementation of a THS to be noninferior to face-to-face care, with no significant change in mortality, readmission rate, or patient satisfaction. The significant improvement observed in LOS is consistent with the adoption of hospitalist models in other medical care settings,¹¹ but had not been reported by previous telehospitalist studies. For example, in their retrospective chart review comparing an NP-supported telehospitalist model to locum tenens hospitalists, Boltz et al found no difference in LOS.³¹ Moreover, as in our study, they found no differences in readmissions, mortality, and patient satisfaction.³¹ Similarly, Kuperman et al reported unchanged daily census, LOS, and transfer rates from a CAH with their virtual hospitalist program, but a decrease in the percentage of patients transferred-out from the emergency department, suggesting that more patients were treated locally.¹⁹

Reduction in LOS is one of the primary measures of efficiency in hospital care³¹; reducing LOS while maintaining the quality of care lowers hospital costs. The reduction in LOS in our study could be attributed to greater continuity of care, engagement/experience of the telehospitalists, or other factors. This decrease in LOS and slight reduction in admissions resulted in an overall lower daily census during the study period and impacted efficiency. Our study was unable to determine the cause for the reduction in admissions; however, several concurrent events, including the expansion of community-care options for veterans under the MISSION ACT (Maintaining Internal Systems and Strengthening Integrated Outside Networks Act) in June 2019, a nationwide smoking ban at VA facilities (October 2019), and a modification in the alcohol withdrawal treatment protocol might have influenced veterans’ choice of hospital.

Readmission rates were slightly higher, though nonsignificant, in the postimplementation period. Alcohol-related readmissions accounted for most readmissions; some of the protocol changes, such as admitting all patients with alcohol withdrawal to inpatient class instead of admitting some to the observation class, accounted for part of the increase in readmission rates. Readmission rates for other conditions such as chronic obstructive pulmonary disease, chronic heart failure, or pneumonia were not significantly different, suggesting that the reduction in LOS did not result in an unintended increased readmission rate for those conditions.

Rural hospitals are struggling with staffing and finances. Resorting to locum tenens staffing is costly and can result in variable quality of care.^32,33 APPs are increasingly taking on hospitalist positions, with 65% of adult hospitalist programs, including half of all VHA hospitals, employing NPs and PAs.^34,35 In response to this expanded scope of practice, hospitals employing APPs in hospitalist roles must comply with state and federal laws, which often require that APPs be supervised by or work in collaboration with an on-site or off-site physician. The THS is a great model to support APPs and address staffing and cost challenges in low-volume rural facilities, while maintaining quality of care. Some APP-telehospitalist programs similar to ours have reported cost reductions of up 58% compared to programs that employ locum tenens physicians.³¹ In our model, we assume that a single telehospitalist hub could provide coverage to two or three spoke sites with APP support, reducing staffing costs.

Hub telehospitalists reported satisfaction with the program, and they perceived the quality of care to be comparable to face-to-face encounters; their responses were consistent with those previously reported in an evaluation of telemedicine acute care by JaKa et al.²⁰ Spoke-site staff, however, had a mixed level of satisfaction, which was different from responses reported by JaKa et al.²⁰ The primary challenges encountered were technological and communication issues, differences in cultures of care between the hub and the spoke sites, and buy-in from frontline staff. Differences in expectations and unclear role definitions between the local APP and the telehospitalist were identified as contributors to dissatisfaction with the program by the nursing staff. Modifications to the communication processes between nurses and telehospitalists and role clarification improved the experience. Culture and practice differences between spoke physicians and the telehospitalist persisted throughout the program implementation, and likely affected the hub providers’ perception of the THS. This was evidenced by reluctance from spoke physicians to implement warm handoffs or participate in THS meetings and resistance to protocol changes. Additional evaluations, collaborations. and interventions are required to improve satisfaction of spoke-site staff.

This study has several limitations. First, the VHA is an integrated health system, one that serves an older, predominantly male patient population. Also, the lack of reimbursement and interstate licensing restrictions limit generalizability of these results to other CAHs or healthcare systems. Furthermore, the intervention was limited to a single rural site; while this allowed for a detailed evaluation, unique barriers or facilitators might exist that limit its applicability. In addition, QI initiatives implemented by the VHA during the project period might have confounded some of our results. Last, patient satisfaction survey data are overall limited in their ability to fully assess patient’s experience and satisfaction with the program. Further qualitative studies are needed to gain deeper insight into patient perspectives with the THS and whether modality of care delivery influences patients’ care decisions. Future studies should consider a multisite design with one or more hubs and multiple spoke sites.

CONCLUSION

Telehospitalist services are a feasible and safe approach to provide inpatient services and address staffing needs of rural hospitals. To enhance program performance, it is essential to ensure adequate technological quality, clearly delineate and define roles and responsibilities of the care team, and address communication issues or staff concerns in a timely manner.

Acknowledgments

The authors thank the staff, administration, and leadership at the Tomah and Iowa City VA Medical Centers for working with us on this project. They offer special thanks to Kevin Glenn, MD, MS, Ethan Kuperman, MD, MS, FHM, and Jennifer Chapin, MSN, RN, for sharing their expertise, and the telehealth team, including Nathaniel Samuelson, Angela McDowell, and Katrin Metcalf.

Disclaimer

The views expressed in this article are those of the authors and do not necessarily represent the views of the Department of Veterans Affairs.

Healthcare in rural areas faces increasing challenges due to community hospital closures, physician shortages, and a more concentrated population of older adults with higher rates of comorbid conditions than their urban counterparts.^1-3 Critical access hospitals (CAHs), which primarily serve rural areas, have fewer clinical capabilities, worse process-of-care measures, and higher mortality rates for some conditions when compared to non-CAHs.⁴ As such, CAHs are closing at record numbers across the United States,⁵ resulting in loss of available hospital beds and patient access to timely emergency services,⁶ which can worsen outcomes, further widening the rural-urban healthcare gap.^7,8 Furthermore, this strain on an overwhelmed health system in the most vulnerable areas restricts the ability to respond to healthcare crises like the coronavirus disease 2019 pandemic.⁹

Providing adequate staff for currently available hospital beds is also a problem in rural areas. Studies demonstrating improved outcomes, decreased length of stay (LOS), and increased quality with hospitalist services have resulted in a high demand for hospitalists nationwide.^10-12 Recruiting hospitalists to work in rural areas, however, has become increasingly challenging due to low-patient volumes, financial viability of hospitalist-model adoption, and provider shortages.^13,14 Recently, the Veterans Health Administration (VHA) reported a 28% nationwide shortage of hospitalists,¹⁵ which disproportionally affects rural VHA hospitals. Staffing difficulties and reliance on intermittent providers were reported by more than 80% of rural and low-complexity VHA facilities.¹⁶

Telehospitalist services (THS) can help deliver high-quality care to rural residents locally, decrease travel expenses, support hospital volume, and increase healthcare capacity in response to a pandemic.^14,17,18 Only a few studies have described THS (mostly with overnight or cross-coverage models directed to CAHs), and clinical outcomes have been inconsistently reported.^17,19-21 Furthermore, no program has been conducted within an integrated health system akin to the VHA. The primary objective of this quality improvement (QI) initiative was to perform a mixed-methods evaluation of THS between VHA hospitals to compare clinical outcomes and patient and staff satisfaction. Secondary outcomes included description of the implementation process, unexpected challenges, and subsequent QI initiatives. These results will expand the knowledge on feasibility of THS and provide implementation guidance.

METHODS

A mixed-methods approach was used to evaluate outcomes of this QI project. Reporting follows the revised Standards for Quality Improvement Reporting Excellence (SQUIRE 2.0).²²

Context

The VHA is the largest integrated healthcare system in the United States, with more than 8 million veterans enrolled, more than 30% of whom reside in a rural area. The VHA comprises more than 1,000 outpatient clinics and 170 acute care VA Medical Centers,^23,24 including more than 35 rural and low-complexity hospitals.²⁵ Low-complexity hospitals are those with the lowest volume and levels of patient complexity and minimal or no teaching programs, research, intensive care unit (ICU) beds, and subspecialists. Lack of reimbursement and interstate licensing, often cited as barriers to telemedicine, do not apply to the VHA. The hub site was a large tertiary care (high-complexity) VHA hospital located in Iowa City, Iowa. The spoke site was a low-complexity (10-bed acute inpatient unit with no ICU) rural VA hospital located in Tomah, Wisconsin.

Study Population

The preimplementation cohort for comparison included all patients admitted between January 1, 2018, and January 6, 2019. The postimplementation study cohort included all observation and acute care admissions during the pilot phase (January 7 to May 3, 2019) and sustainability phase (July 15 to December 31, 2019). The postimplementation analysis excluded the time period of May 4 to July 14, 2019, due to an interruption (gap) in THS. The gap period allowed for preliminary data analysis, optimization of the telecommunication system, and the recruitment and training of additional providers who could provide long-term staffing to the service.

Intervention

Preimplementation

Prior to THS implementation, Tomah’s inpatient ward was staffed by one physician per shift, who could be a hospitalist, medical officer of the day (MOD), or an intermittent provider (locum tenens). Hospitalists covering the acute inpatient ward prior to the THS transitioned to cover weekends, nights, and urgent care service shifts.

We visited the spoke site and held information-sharing sessions with key stakeholders (administrators, clinician leaders, nurses, and ancillary staff) prior to kick-off. Recurrent phone meetings addressed anticipated and emerging challenges. Telehospitalist and local providers underwent technology and service training.

Technology and Connectivity

A low-cost technology system using tablet computers provided Health Insurance Portability and Accountability Act–compliant videoconferencing with a telehospitalist at the hub site. An Eko-Core digital stethoscope^® with a web-based audio stream was available. Telehospitalists conducted encounters from a private office space with telehealth capabilities. A total of $9,000 was spent on equipment at both sites. Due to connectivity problems and data limits, the tablets were switched to mobile computer-on-wheels workstations and hospital-based Wi-Fi for the sustainability phase.

THS Description

An experienced hub hospitalist, together with an advanced practice provider (APP; nurse practitioner [NP] or physician assistant [PA]), cared for all patients admitted to the 10-bed inpatient unit at the spoke site, Monday through Friday from 8:00 AM to 4:30 PM. The APP had limited or no prior experience in acute inpatient medicine. The telehospitalist worked as a team with the APP. The APP was the main point of contact for nurses, performed physical examinations, and directed patient care to their level of comfort (in a similar manner as a teaching team). The telehospitalist conducted bedside patient rounds, participated in multidisciplinary huddles, and shared clinical documentation and administrative duties with the APP. The telehospitalist was the primary staff for admitted patients and had full access to the electronic health record (EHR). The THS was staffed by 10 hospitalists during the study period. Overnight and weekend cross-coverage and admissions were performed by MODs, who also covered the urgent care and cross-covered other nonmedical units.

Quantitative Evaluation Methods

Workload and Clinical Outcomes

An EHR query identified all patients admitted during the pre- and postimplementation periods. Demographic data, clinical Nosos risk scores,^26,27 and top admission diagnoses were reported. Workload was evaluated using the average number of encounters per day and self-reported telehospitalist worksheets, which were cross-referenced with EHR data. Clinical outcomes included LOS, 30-day hospital readmission rate, 30-day standardized mortality (SMR30), in-hospital mortality, and VHA-specific inpatient quality metrics. Independent sample t tests for continuous variables and chi-square tests or Fisher’s exact test (for patient class) for categorical variables were used to compare pre- and postimplementation groups. Statistical process control (SPC) charts evaluated changes over time. All analyses were conducted using Microsoft Excel and R.²⁸

Provider Satisfaction

Anonymous surveys were distributed to spoke-site inpatient and administrative staff at 1 month and 12 months postimplementation, assessing satisfaction, technology/connectivity, communication, and challenges (Appendix Figure 1). Satisfaction of the telehospitalist physicians at the hub site was measured 12 months postimplementation by a 26-question survey assessing the same domains, plus quality of care (Appendix Figure 2).

Patient Satisfaction

The VHA Survey of Healthcare Experiences of Patients (SHEP), a version of the Hospital Consumer Assessment of Healthcare Providers and Systems Survey,^29,30 was mailed to all patients after discharge. Survey responses concerning inpatient provider care (eg, care coordination, communication, hospital rating, willingness to recommend the hospital) during the pre- and postimplementation phases were compared using a two-sample test of independent proportions. Responses obtained during May and June 2019 were excluded.

Qualitative Evaluation Methods

The qualitative researcher observed information-sharing meetings and facilitated unstructured interviews with clinical and administrative staff during site visits preimplementation and 3 months after implementation. Interviews with administrators and clinical staff addressed their experiences with the THS, staff’s perception of patient and family response to THS implementation, administrative impacts, challenges, and strengths. All interviews and meetings were documented with handwritten notes and audio recordings. Interview summary notes were typed into a Microsoft Word document, verified by the physician-investigator, and synthesized by inductive themes into site-visit reports. Audio recordings were uploaded to a secure computer, transcribed, and reviewed for accuracy. The qualitative researcher also identified emerging themes from open-ended survey responses. Process evaluation findings were shared with administration at the spoke site.

The authors had full access to, and took full responsibility for, the integrity of the data. The project was evaluated by the University of Iowa Institutional Review Board and the Iowa City VA Research and Development Committee and was determined to be a non–human-subjects QI project.

RESULTS

Quantitative Workload and Clinical Outcomes

There were 822 admissions during the preimplementation period and 550 admissions during the postimplementation period (253 during the pilot and 297 during sustainability phase). Patient characteristics pre- and postimplementation were not significantly different (Table 1). The median patient age was 65 years; 96% of patients were male, and 83% were rural residents. The most common admission diagnosis was alcohol-related (36%); regarding patient disposition, 78% of admissions were discharged home.

Workload

There were 502 patient encounters staffed by the telehospitalist in the pilot phase, with an average of 6.25 encounters per day, and a telehospitalist-reported workload of 7 hours per day. There were 538 patient encounters, with an average of 4.67 encounters per day and a workload of 5.6 hours per day in the sustainability phase. The average daily census decreased from 5.0 (SD, 1.1) patients per day during preimplementation to 3.1 (SD, 0.5) patients per day during postimplementation (Table 2). In some of the months during the study period, admissions decreased below the lower SPC limit, suggesting a significant change (Figure). Adjusted LOS was significantly lower, with 3.0 (SD, 0.7) days vs 2.3 (SD, 0.3) days in the pre- and postimplementation periods, respectively. Bed occupancy rates were significantly lower in the sustainability phase compared with the pilot phase and the preimplementation period. Readmission rates varied, ranging from <10% to >30%, not significantly different but slightly higher in the postimplementation period. Readmission rates for heart failure, chronic obstructive pulmonary disease, and pneumonia remained unchanged; other medical readmissions (mostly alcohol-related) were slightly higher in the postimplementation period.

In-hospital mortality and SMR30 did not change significantly, but there was improvement in the 12-month rolling average of the observed/expected SMR30 from 1.40 to 1.08. Additional VHA-specific quality metrics were monitored and showed either small improvements or no change (data not shown).

Satisfaction at Hub and Spoke Sites

After sending two reminder communications via email, the telehospitalist satisfaction survey had a total response rate of 90% (9/10). Telehospitalists were satisfied or very satisfied (89%) with the program and the local providers (88.9%), rating their experience as good or excellent (100%) (Table 3). Communication with patients, families, and local staff was noted as being “positive” or “mostly positive.” Telehospitalists reported confidence in the accuracy of their diagnoses and rated the quality of care as being equal to that of a face-to-face encounter. Connectivity problems were prevalent, although most providers were able to resort to a back-up plan. Other challenges included differences in culture and concerns about liability. We received 27 responses from the spoke-site satisfaction survey; the response rate could not be determined because the survey was distributed by the spoke site for anonymity. Of the respondents, 37% identified as nurses, 25.9% as healthcare providers (APPs or physicians), and 33.3% as other staff (eg, social worker, nutritionist, physical therapist, utilization management, administrators); 3.7% did not respond. Among the participants, 88% had personally interacted with the THS. Most providers and other staff perceived THS as valuable (57.1% and 77.8%, respectively) and were satisfied or highly satisfied with THS (57.1% and 55.6%, respectively). On average, nurses provided lower ratings across all survey items than providers and other staff. Challenges noted by all staff included issues with communication, workflow, and technology/connectivity.

Regarding patient satisfaction, the SHEP survey showed a significant improvement in care coordination (18%; P = .02) and a nonsignificant improvement in communications about medications (5%; P =.054). The remaining items in the survey, including overall hospital rating and willingness to recommend the hospital, were unchanged (Appendix Table).

Qualitative Strengths

Our process evaluation identified high quality of care and teamwork as contributors to the success of the program. Overall, staff credited perceived improvements in quality of care to the quality of providers staffing the THS, including the local APPs. Noting the telehospitalists’ knowledge base and level of engagement as key attributes, one staff member commented: “I prefer a telehospitalist that really care[s] about patients than some provider that is physically here but does not engage.” Staff perceived improvements in the continuity of care, as well as care processes such as handoffs and transitions of care.

Improvements in teamwork were perceived compared with the previous model of care. Telehospitalists were lauded for their professionalism and communication skills. Overall, nurses felt providers in the THS listened more to their views. In addition, nurse respondents felt they could learn from several providers and said they enjoyed the telehospitalists’ disposition to teach and discuss patient care. The responsiveness of the THS staff was instrumental in building teamwork and acceptance. A bedside interdisciplinary protocol was established for appropriate patients. Local staff felt this was crucial for teamwork and patient satisfaction. Telehospitalists reported high-value in interdisciplinary rounds, facilitating interaction with nurses and ancillary staff. Handoff problems were identified, leading to QI initiatives to mitigate those issues.

Challenges

The survey identified administrative barriers, technical difficulties, workflow constraints, and clinical concerns. The credentialing process was complicated, delaying the onboarding of telehospitalists. Internet connectivity was inconsistent, leading to disruption in video communications; however, during the sustainability phase, updated technology improved communications. The communication workflow was resisted by some nurses, who wanted to phone the telehospitalist directly rather than having the local APP as the first contact. Secure messaging was enabled to allow nurses direct contact during the sustainability phase.

Workload was a concern among telehospitalists and local staff. Telehospitalists perceived the documentation requirements and administrative workload to be two to three times higher than at other hospitals—despite the lower number of encounters. Finally, clinical concerns from spoke-site clinicians included a perceived rise in the acuity of patients (which was not evident by the Nosos score) and delayed decisions to transfer-out patients. These concerns were addressed with educational sessions for telehospitalists during the sustainability phase.

Additional Quality Improvement Projects

The implementation of THS resulted in QI initiatives at the spoke site, including an EHR-integrated handoff tool; a documentation evaluation that led to the elimination of duplicative, inefficient, and error-prone templates; and a revision of the alcohol withdrawal treatment protocol during the sustainability phase to reduce the use of intravenous benzodiazepines. A more comprehensive benzodiazepine-sparing alcohol withdrawal treatment protocol was also developed but was not implemented until after the study period (January 2020).

DISCUSSION

Our pre-post study evaluation found implementation of a THS to be noninferior to face-to-face care, with no significant change in mortality, readmission rate, or patient satisfaction. The significant improvement observed in LOS is consistent with the adoption of hospitalist models in other medical care settings,¹¹ but had not been reported by previous telehospitalist studies. For example, in their retrospective chart review comparing an NP-supported telehospitalist model to locum tenens hospitalists, Boltz et al found no difference in LOS.³¹ Moreover, as in our study, they found no differences in readmissions, mortality, and patient satisfaction.³¹ Similarly, Kuperman et al reported unchanged daily census, LOS, and transfer rates from a CAH with their virtual hospitalist program, but a decrease in the percentage of patients transferred-out from the emergency department, suggesting that more patients were treated locally.¹⁹

Reduction in LOS is one of the primary measures of efficiency in hospital care³¹; reducing LOS while maintaining the quality of care lowers hospital costs. The reduction in LOS in our study could be attributed to greater continuity of care, engagement/experience of the telehospitalists, or other factors. This decrease in LOS and slight reduction in admissions resulted in an overall lower daily census during the study period and impacted efficiency. Our study was unable to determine the cause for the reduction in admissions; however, several concurrent events, including the expansion of community-care options for veterans under the MISSION ACT (Maintaining Internal Systems and Strengthening Integrated Outside Networks Act) in June 2019, a nationwide smoking ban at VA facilities (October 2019), and a modification in the alcohol withdrawal treatment protocol might have influenced veterans’ choice of hospital.

Readmission rates were slightly higher, though nonsignificant, in the postimplementation period. Alcohol-related readmissions accounted for most readmissions; some of the protocol changes, such as admitting all patients with alcohol withdrawal to inpatient class instead of admitting some to the observation class, accounted for part of the increase in readmission rates. Readmission rates for other conditions such as chronic obstructive pulmonary disease, chronic heart failure, or pneumonia were not significantly different, suggesting that the reduction in LOS did not result in an unintended increased readmission rate for those conditions.

Rural hospitals are struggling with staffing and finances. Resorting to locum tenens staffing is costly and can result in variable quality of care.^32,33 APPs are increasingly taking on hospitalist positions, with 65% of adult hospitalist programs, including half of all VHA hospitals, employing NPs and PAs.^34,35 In response to this expanded scope of practice, hospitals employing APPs in hospitalist roles must comply with state and federal laws, which often require that APPs be supervised by or work in collaboration with an on-site or off-site physician. The THS is a great model to support APPs and address staffing and cost challenges in low-volume rural facilities, while maintaining quality of care. Some APP-telehospitalist programs similar to ours have reported cost reductions of up 58% compared to programs that employ locum tenens physicians.³¹ In our model, we assume that a single telehospitalist hub could provide coverage to two or three spoke sites with APP support, reducing staffing costs.

Hub telehospitalists reported satisfaction with the program, and they perceived the quality of care to be comparable to face-to-face encounters; their responses were consistent with those previously reported in an evaluation of telemedicine acute care by JaKa et al.²⁰ Spoke-site staff, however, had a mixed level of satisfaction, which was different from responses reported by JaKa et al.²⁰ The primary challenges encountered were technological and communication issues, differences in cultures of care between the hub and the spoke sites, and buy-in from frontline staff. Differences in expectations and unclear role definitions between the local APP and the telehospitalist were identified as contributors to dissatisfaction with the program by the nursing staff. Modifications to the communication processes between nurses and telehospitalists and role clarification improved the experience. Culture and practice differences between spoke physicians and the telehospitalist persisted throughout the program implementation, and likely affected the hub providers’ perception of the THS. This was evidenced by reluctance from spoke physicians to implement warm handoffs or participate in THS meetings and resistance to protocol changes. Additional evaluations, collaborations. and interventions are required to improve satisfaction of spoke-site staff.

This study has several limitations. First, the VHA is an integrated health system, one that serves an older, predominantly male patient population. Also, the lack of reimbursement and interstate licensing restrictions limit generalizability of these results to other CAHs or healthcare systems. Furthermore, the intervention was limited to a single rural site; while this allowed for a detailed evaluation, unique barriers or facilitators might exist that limit its applicability. In addition, QI initiatives implemented by the VHA during the project period might have confounded some of our results. Last, patient satisfaction survey data are overall limited in their ability to fully assess patient’s experience and satisfaction with the program. Further qualitative studies are needed to gain deeper insight into patient perspectives with the THS and whether modality of care delivery influences patients’ care decisions. Future studies should consider a multisite design with one or more hubs and multiple spoke sites.

CONCLUSION

Telehospitalist services are a feasible and safe approach to provide inpatient services and address staffing needs of rural hospitals. To enhance program performance, it is essential to ensure adequate technological quality, clearly delineate and define roles and responsibilities of the care team, and address communication issues or staff concerns in a timely manner.

Acknowledgments

The authors thank the staff, administration, and leadership at the Tomah and Iowa City VA Medical Centers for working with us on this project. They offer special thanks to Kevin Glenn, MD, MS, Ethan Kuperman, MD, MS, FHM, and Jennifer Chapin, MSN, RN, for sharing their expertise, and the telehealth team, including Nathaniel Samuelson, Angela McDowell, and Katrin Metcalf.

Disclaimer

The views expressed in this article are those of the authors and do not necessarily represent the views of the Department of Veterans Affairs.

In January 2017, the US Department of Veterans Affairs (VA), led by the National Center of Ethics in Health Care, created the Life-Sustaining Treatment Decisions Initiative (LSTDI). The VA gradually implemented the LSTDI in its facilities nationwide. In a format similar to the standardized form of portable medical orders, provider orders for life-sustaining treatments (POLST), the initiative promotes discussions with veterans and encourages but does not require health care professionals (HCPs) to complete a template for documentation (life-sustaining treatment [LST] note) of a patient’s preferences.¹ The HCP enters a code status into the electronic health record (EHR), creating a portable and durable note and order.

With a new durable code status, the HCPs performing these procedures (eg, colonoscopies, coronary catheterization, or percutaneous biopsies) need to acknowledge and can potentially rescind a do not resuscitate (DNR) order. Although the risk of cardiac arrest or intubation is low, all invasive procedures carry these risks to some degree.^2,3 Some HCPs advocate the automatic discontinuation of DNR orders before any procedure, but multiple professional societies recommend that patients be included in these discussions to honor their wishes.^4-7 Although no procedures at the VA require the suspension of a DNR status, it is important to establish which life-sustaining measures are acceptable to patients.

As part of the informed consent process, proceduralists (HCPs who perform a procedure) should discuss the option of temporary suspension of DNR in the periprocedural period and document the outcome of this discussion (eg, rescinded DNR, acknowledgment of continued DNR status). These discussions need to be documented clearly to ensure accurate communication with other HCPs, particularly those caring for the patient postprocedure. Without the documentation, the risk that the patient’s wishes will not be honored is high.⁸ Code status is usually addressed before intubation of general anesthesia; however, nonsurgical procedures have a lower likelihood of DNR acknowledgment.

This study aimed to examine and improve the rate of acknowledgment of DNR status before nonsurgical procedures. We hypothesized that the rate of DNR acknowledgment before nonsurgical invasive procedures is low; and the rate can be raised with an intervention designed to educate proceduralists and improve and simplify this documentation.⁹

Methods

This was a single center, before/after quasi-experimental study. The study was considered clinical operations and institutional review board approval was unnecessary.

A retrospective chart review was performed of patients who underwent an inpatient or outpatient, nonsurgical invasive procedure at the Minneapolis VA Medical Center in Minnesota. The preintervention period was defined as the first 6 months after implementation of the LSTDI between May 8, 2018 and October 31, 2018. The intervention was presented in December 2018 and January 2019. The postintervention period was from February 1, 2019 to April 30, 2019.

Patients who underwent a nonsurgical invasive procedure were reviewed in 3 procedural areas. These areas were chosen based on high patient volumes and the need for rapid patient turnover, including gastroenterology, cardiology, and interventional radiology. An invasive procedure was defined as any procedure requiring patient consent. Those patients who had a completed LST note and who had a DNR order were recorded.

 

The intervention was composed of 2 elements: (1) an addendum to the LST note, which temporarily suspended resuscitation orders (Figure). We developed the addendum based on templates and orders in use before LSTDI implementation. Physicians from the procedural areas reviewed the addendum and provided feedback and the facility chief-of-staff provided approval. Part 2 was an educational presentation to proceduralists in each procedural area. The presentation included a brief introduction to the LSTDI, where to find a life-sustaining treatment note, code status, the importance of addressing code status, and a description of the addendum. The proceduralists were advised to use the addendum only after discussion with the patient and obtaining verbal consent for DNR suspension. If the patient elected to remain DNR, proceduralists were encouraged to document the conversation acknowledging the DNR.

Outcomes

The primary outcome of the study was proceduralist acknowledgment of DNR status before nonsurgical invasive procedures. DNR status was considered acknowledged if the proceduralist provided any type of documentation.

Statistical Analysis

Model predicted percentages of DNR acknowledgment are reported from a logistic regression model with both procedural area, time (before vs after) and the interaction between these 2 variables in the model. The simple main effects comparing before vs after within the procedural area based on post hoc contrasts of the interaction term also are shown.

Results

During the first 6 months following LSTDI implementation (the preintervention phase), 5,362 invasive procedures were performed in gastroenterology, interventional radiology, and cardiology. A total of 211 procedures were performed on patients who had a prior LST note indicating DNR. Of those, 68 (32.2%) had documentation acknowledging their DNR status. The educational presentation was given to each of the 3 departments with about 75% faculty attendance in each department. After the intervention, 1,932 invasive procedures were performed, identifying 143 LST notes with a DNR status. Sixty-five (45.5%) had documentation of a discussion regarding their DNR status.

The interaction between procedural areas and time (before, after) was examined. Of the 3 procedural areas, only interventional radiology had significant differences before vs after, 7.5% vs 26.3%, respectively (P = .01). Model-adjusted percentages before vs after for cardiology were 75.6% vs 91.7% (P = .12) and for gastroenterology were 46% vs 53.5% (P = .40) (Table). When all 3 procedural areas were combined, there was a significant improvement in the overall percentage of DNR acknowledgment postintervention from 38.6% to 61.1.% (P = .01).

Discussion

With the LSTDI, DNR orders remain in place and are valid in the inpatient and outpatient setting until reversed by the patient. This creates new challenges for proceduralists. Before our intervention, only about one-third of proceduralists’ recognized DNR status before procedures. This low rate of preprocedural DNR acknowledgments is not unique to the VA. A pilot study assessing rate of documentation of code status discussions in patients undergoing venting gastrostomy tube for malignant bowel obstruction showed documentation in only 22% of cases before the procedure.¹⁰ Another simulation-based study of anesthesiologist showed only 57% of subjects addressed resuscitation before starting the procedure.¹¹

Despite the low initial rates of DNR acknowledgment, our intervention successfully improved these rates, although with variation between procedural areas. Prior studies looking at improving adherence to guidelines have shown the benefit of physician education.^12,13 Improving code status acknowledgment before an invasive procedure not only involves increasing awareness of a preexisting code status, but also developing a system to incorporate the documentation process efficiently into the procedural workflow and ensuring that providers are aware of the appropriate process. Although the largest improvement was in interventional radiology, many patients postintervention still did not have their DNR orders acknowledged. Confusion is created when the patient is cared for by a different HCP or when the resuscitation team is called during a cardiac arrest. Cardiopulmonary resuscitation may be started or withheld incorrectly if the patient’s most recent wishes for resuscitation are unclear.¹⁴

 

Outside of using education to raise awareness, other improvements could utilize informatics solutions, such as developing an alert on opening a patient chart if a DNR status exists (such as a pop-up screen) or adding code status as an item to a preprocedural checklist. Similar to our study, previous studies also have found that a systematic approach with guidelines and templates improved rates of documentation of code status and DNR decisions.^15,16 A large proportion of the LST notes and procedures done on patients with a DNR in our study occurred in the inpatient setting without any involvement of the primary care provider in the discussion. Having an automated way to alert the primary care provider that a new LST note has been completed may be helpful in guiding future care. Future work could identify additional systematic methods to increase acknowledgment of DNR.

Limitations

Our single-center results may not be generalizable. Although the interaction between procedural area and time was tested, it is possible that improvement in DNR acknowledgment was attributable to secular trends and not the intervention. Other limitations included the decreased generalizability of a VA health care initiative and its unique electronic health record, incomplete attendance rates at our educational sessions, and a lack of patient-centered outcomes.

Conclusions

A templated addendum combined with targeted staff education improved the percentage of DNR acknowledgments before nonsurgical invasive procedures, an important step in establishing patient preferences for life-sustaining treatment in procedures with potential complications. Further research is needed to assess whether these improvements also lead to improved patient-centered outcomes.

Acknowledgments

The authors would like to acknowledge the invaluable help of Dr. Kathryn Rice and Dr. Anne Melzer for their guidance in the manuscript revision process

In January 2017, the US Department of Veterans Affairs (VA), led by the National Center of Ethics in Health Care, created the Life-Sustaining Treatment Decisions Initiative (LSTDI). The VA gradually implemented the LSTDI in its facilities nationwide. In a format similar to the standardized form of portable medical orders, provider orders for life-sustaining treatments (POLST), the initiative promotes discussions with veterans and encourages but does not require health care professionals (HCPs) to complete a template for documentation (life-sustaining treatment [LST] note) of a patient’s preferences.¹ The HCP enters a code status into the electronic health record (EHR), creating a portable and durable note and order.

With a new durable code status, the HCPs performing these procedures (eg, colonoscopies, coronary catheterization, or percutaneous biopsies) need to acknowledge and can potentially rescind a do not resuscitate (DNR) order. Although the risk of cardiac arrest or intubation is low, all invasive procedures carry these risks to some degree.^2,3 Some HCPs advocate the automatic discontinuation of DNR orders before any procedure, but multiple professional societies recommend that patients be included in these discussions to honor their wishes.^4-7 Although no procedures at the VA require the suspension of a DNR status, it is important to establish which life-sustaining measures are acceptable to patients.

As part of the informed consent process, proceduralists (HCPs who perform a procedure) should discuss the option of temporary suspension of DNR in the periprocedural period and document the outcome of this discussion (eg, rescinded DNR, acknowledgment of continued DNR status). These discussions need to be documented clearly to ensure accurate communication with other HCPs, particularly those caring for the patient postprocedure. Without the documentation, the risk that the patient’s wishes will not be honored is high.⁸ Code status is usually addressed before intubation of general anesthesia; however, nonsurgical procedures have a lower likelihood of DNR acknowledgment.

This study aimed to examine and improve the rate of acknowledgment of DNR status before nonsurgical procedures. We hypothesized that the rate of DNR acknowledgment before nonsurgical invasive procedures is low; and the rate can be raised with an intervention designed to educate proceduralists and improve and simplify this documentation.⁹

Methods

This was a single center, before/after quasi-experimental study. The study was considered clinical operations and institutional review board approval was unnecessary.

A retrospective chart review was performed of patients who underwent an inpatient or outpatient, nonsurgical invasive procedure at the Minneapolis VA Medical Center in Minnesota. The preintervention period was defined as the first 6 months after implementation of the LSTDI between May 8, 2018 and October 31, 2018. The intervention was presented in December 2018 and January 2019. The postintervention period was from February 1, 2019 to April 30, 2019.

Patients who underwent a nonsurgical invasive procedure were reviewed in 3 procedural areas. These areas were chosen based on high patient volumes and the need for rapid patient turnover, including gastroenterology, cardiology, and interventional radiology. An invasive procedure was defined as any procedure requiring patient consent. Those patients who had a completed LST note and who had a DNR order were recorded.

 

The intervention was composed of 2 elements: (1) an addendum to the LST note, which temporarily suspended resuscitation orders (Figure). We developed the addendum based on templates and orders in use before LSTDI implementation. Physicians from the procedural areas reviewed the addendum and provided feedback and the facility chief-of-staff provided approval. Part 2 was an educational presentation to proceduralists in each procedural area. The presentation included a brief introduction to the LSTDI, where to find a life-sustaining treatment note, code status, the importance of addressing code status, and a description of the addendum. The proceduralists were advised to use the addendum only after discussion with the patient and obtaining verbal consent for DNR suspension. If the patient elected to remain DNR, proceduralists were encouraged to document the conversation acknowledging the DNR.

Outcomes

The primary outcome of the study was proceduralist acknowledgment of DNR status before nonsurgical invasive procedures. DNR status was considered acknowledged if the proceduralist provided any type of documentation.

Statistical Analysis

Model predicted percentages of DNR acknowledgment are reported from a logistic regression model with both procedural area, time (before vs after) and the interaction between these 2 variables in the model. The simple main effects comparing before vs after within the procedural area based on post hoc contrasts of the interaction term also are shown.

Results

During the first 6 months following LSTDI implementation (the preintervention phase), 5,362 invasive procedures were performed in gastroenterology, interventional radiology, and cardiology. A total of 211 procedures were performed on patients who had a prior LST note indicating DNR. Of those, 68 (32.2%) had documentation acknowledging their DNR status. The educational presentation was given to each of the 3 departments with about 75% faculty attendance in each department. After the intervention, 1,932 invasive procedures were performed, identifying 143 LST notes with a DNR status. Sixty-five (45.5%) had documentation of a discussion regarding their DNR status.

The interaction between procedural areas and time (before, after) was examined. Of the 3 procedural areas, only interventional radiology had significant differences before vs after, 7.5% vs 26.3%, respectively (P = .01). Model-adjusted percentages before vs after for cardiology were 75.6% vs 91.7% (P = .12) and for gastroenterology were 46% vs 53.5% (P = .40) (Table). When all 3 procedural areas were combined, there was a significant improvement in the overall percentage of DNR acknowledgment postintervention from 38.6% to 61.1.% (P = .01).

Discussion

With the LSTDI, DNR orders remain in place and are valid in the inpatient and outpatient setting until reversed by the patient. This creates new challenges for proceduralists. Before our intervention, only about one-third of proceduralists’ recognized DNR status before procedures. This low rate of preprocedural DNR acknowledgments is not unique to the VA. A pilot study assessing rate of documentation of code status discussions in patients undergoing venting gastrostomy tube for malignant bowel obstruction showed documentation in only 22% of cases before the procedure.¹⁰ Another simulation-based study of anesthesiologist showed only 57% of subjects addressed resuscitation before starting the procedure.¹¹

Despite the low initial rates of DNR acknowledgment, our intervention successfully improved these rates, although with variation between procedural areas. Prior studies looking at improving adherence to guidelines have shown the benefit of physician education.^12,13 Improving code status acknowledgment before an invasive procedure not only involves increasing awareness of a preexisting code status, but also developing a system to incorporate the documentation process efficiently into the procedural workflow and ensuring that providers are aware of the appropriate process. Although the largest improvement was in interventional radiology, many patients postintervention still did not have their DNR orders acknowledged. Confusion is created when the patient is cared for by a different HCP or when the resuscitation team is called during a cardiac arrest. Cardiopulmonary resuscitation may be started or withheld incorrectly if the patient’s most recent wishes for resuscitation are unclear.¹⁴

 

Outside of using education to raise awareness, other improvements could utilize informatics solutions, such as developing an alert on opening a patient chart if a DNR status exists (such as a pop-up screen) or adding code status as an item to a preprocedural checklist. Similar to our study, previous studies also have found that a systematic approach with guidelines and templates improved rates of documentation of code status and DNR decisions.^15,16 A large proportion of the LST notes and procedures done on patients with a DNR in our study occurred in the inpatient setting without any involvement of the primary care provider in the discussion. Having an automated way to alert the primary care provider that a new LST note has been completed may be helpful in guiding future care. Future work could identify additional systematic methods to increase acknowledgment of DNR.

Limitations

Our single-center results may not be generalizable. Although the interaction between procedural area and time was tested, it is possible that improvement in DNR acknowledgment was attributable to secular trends and not the intervention. Other limitations included the decreased generalizability of a VA health care initiative and its unique electronic health record, incomplete attendance rates at our educational sessions, and a lack of patient-centered outcomes.

Conclusions

A templated addendum combined with targeted staff education improved the percentage of DNR acknowledgments before nonsurgical invasive procedures, an important step in establishing patient preferences for life-sustaining treatment in procedures with potential complications. Further research is needed to assess whether these improvements also lead to improved patient-centered outcomes.

Acknowledgments

The authors would like to acknowledge the invaluable help of Dr. Kathryn Rice and Dr. Anne Melzer for their guidance in the manuscript revision process

In January 2017, the US Department of Veterans Affairs (VA), led by the National Center of Ethics in Health Care, created the Life-Sustaining Treatment Decisions Initiative (LSTDI). The VA gradually implemented the LSTDI in its facilities nationwide. In a format similar to the standardized form of portable medical orders, provider orders for life-sustaining treatments (POLST), the initiative promotes discussions with veterans and encourages but does not require health care professionals (HCPs) to complete a template for documentation (life-sustaining treatment [LST] note) of a patient’s preferences.¹ The HCP enters a code status into the electronic health record (EHR), creating a portable and durable note and order.

With a new durable code status, the HCPs performing these procedures (eg, colonoscopies, coronary catheterization, or percutaneous biopsies) need to acknowledge and can potentially rescind a do not resuscitate (DNR) order. Although the risk of cardiac arrest or intubation is low, all invasive procedures carry these risks to some degree.^2,3 Some HCPs advocate the automatic discontinuation of DNR orders before any procedure, but multiple professional societies recommend that patients be included in these discussions to honor their wishes.^4-7 Although no procedures at the VA require the suspension of a DNR status, it is important to establish which life-sustaining measures are acceptable to patients.

As part of the informed consent process, proceduralists (HCPs who perform a procedure) should discuss the option of temporary suspension of DNR in the periprocedural period and document the outcome of this discussion (eg, rescinded DNR, acknowledgment of continued DNR status). These discussions need to be documented clearly to ensure accurate communication with other HCPs, particularly those caring for the patient postprocedure. Without the documentation, the risk that the patient’s wishes will not be honored is high.⁸ Code status is usually addressed before intubation of general anesthesia; however, nonsurgical procedures have a lower likelihood of DNR acknowledgment.

This study aimed to examine and improve the rate of acknowledgment of DNR status before nonsurgical procedures. We hypothesized that the rate of DNR acknowledgment before nonsurgical invasive procedures is low; and the rate can be raised with an intervention designed to educate proceduralists and improve and simplify this documentation.⁹

Methods

This was a single center, before/after quasi-experimental study. The study was considered clinical operations and institutional review board approval was unnecessary.

A retrospective chart review was performed of patients who underwent an inpatient or outpatient, nonsurgical invasive procedure at the Minneapolis VA Medical Center in Minnesota. The preintervention period was defined as the first 6 months after implementation of the LSTDI between May 8, 2018 and October 31, 2018. The intervention was presented in December 2018 and January 2019. The postintervention period was from February 1, 2019 to April 30, 2019.

Patients who underwent a nonsurgical invasive procedure were reviewed in 3 procedural areas. These areas were chosen based on high patient volumes and the need for rapid patient turnover, including gastroenterology, cardiology, and interventional radiology. An invasive procedure was defined as any procedure requiring patient consent. Those patients who had a completed LST note and who had a DNR order were recorded.

 

The intervention was composed of 2 elements: (1) an addendum to the LST note, which temporarily suspended resuscitation orders (Figure). We developed the addendum based on templates and orders in use before LSTDI implementation. Physicians from the procedural areas reviewed the addendum and provided feedback and the facility chief-of-staff provided approval. Part 2 was an educational presentation to proceduralists in each procedural area. The presentation included a brief introduction to the LSTDI, where to find a life-sustaining treatment note, code status, the importance of addressing code status, and a description of the addendum. The proceduralists were advised to use the addendum only after discussion with the patient and obtaining verbal consent for DNR suspension. If the patient elected to remain DNR, proceduralists were encouraged to document the conversation acknowledging the DNR.

Outcomes

The primary outcome of the study was proceduralist acknowledgment of DNR status before nonsurgical invasive procedures. DNR status was considered acknowledged if the proceduralist provided any type of documentation.

Statistical Analysis

Model predicted percentages of DNR acknowledgment are reported from a logistic regression model with both procedural area, time (before vs after) and the interaction between these 2 variables in the model. The simple main effects comparing before vs after within the procedural area based on post hoc contrasts of the interaction term also are shown.

Results

During the first 6 months following LSTDI implementation (the preintervention phase), 5,362 invasive procedures were performed in gastroenterology, interventional radiology, and cardiology. A total of 211 procedures were performed on patients who had a prior LST note indicating DNR. Of those, 68 (32.2%) had documentation acknowledging their DNR status. The educational presentation was given to each of the 3 departments with about 75% faculty attendance in each department. After the intervention, 1,932 invasive procedures were performed, identifying 143 LST notes with a DNR status. Sixty-five (45.5%) had documentation of a discussion regarding their DNR status.

The interaction between procedural areas and time (before, after) was examined. Of the 3 procedural areas, only interventional radiology had significant differences before vs after, 7.5% vs 26.3%, respectively (P = .01). Model-adjusted percentages before vs after for cardiology were 75.6% vs 91.7% (P = .12) and for gastroenterology were 46% vs 53.5% (P = .40) (Table). When all 3 procedural areas were combined, there was a significant improvement in the overall percentage of DNR acknowledgment postintervention from 38.6% to 61.1.% (P = .01).

Discussion

With the LSTDI, DNR orders remain in place and are valid in the inpatient and outpatient setting until reversed by the patient. This creates new challenges for proceduralists. Before our intervention, only about one-third of proceduralists’ recognized DNR status before procedures. This low rate of preprocedural DNR acknowledgments is not unique to the VA. A pilot study assessing rate of documentation of code status discussions in patients undergoing venting gastrostomy tube for malignant bowel obstruction showed documentation in only 22% of cases before the procedure.¹⁰ Another simulation-based study of anesthesiologist showed only 57% of subjects addressed resuscitation before starting the procedure.¹¹

Despite the low initial rates of DNR acknowledgment, our intervention successfully improved these rates, although with variation between procedural areas. Prior studies looking at improving adherence to guidelines have shown the benefit of physician education.^12,13 Improving code status acknowledgment before an invasive procedure not only involves increasing awareness of a preexisting code status, but also developing a system to incorporate the documentation process efficiently into the procedural workflow and ensuring that providers are aware of the appropriate process. Although the largest improvement was in interventional radiology, many patients postintervention still did not have their DNR orders acknowledged. Confusion is created when the patient is cared for by a different HCP or when the resuscitation team is called during a cardiac arrest. Cardiopulmonary resuscitation may be started or withheld incorrectly if the patient’s most recent wishes for resuscitation are unclear.¹⁴

 

Outside of using education to raise awareness, other improvements could utilize informatics solutions, such as developing an alert on opening a patient chart if a DNR status exists (such as a pop-up screen) or adding code status as an item to a preprocedural checklist. Similar to our study, previous studies also have found that a systematic approach with guidelines and templates improved rates of documentation of code status and DNR decisions.^15,16 A large proportion of the LST notes and procedures done on patients with a DNR in our study occurred in the inpatient setting without any involvement of the primary care provider in the discussion. Having an automated way to alert the primary care provider that a new LST note has been completed may be helpful in guiding future care. Future work could identify additional systematic methods to increase acknowledgment of DNR.

Limitations

Our single-center results may not be generalizable. Although the interaction between procedural area and time was tested, it is possible that improvement in DNR acknowledgment was attributable to secular trends and not the intervention. Other limitations included the decreased generalizability of a VA health care initiative and its unique electronic health record, incomplete attendance rates at our educational sessions, and a lack of patient-centered outcomes.

Conclusions

A templated addendum combined with targeted staff education improved the percentage of DNR acknowledgments before nonsurgical invasive procedures, an important step in establishing patient preferences for life-sustaining treatment in procedures with potential complications. Further research is needed to assess whether these improvements also lead to improved patient-centered outcomes.

Acknowledgments

The authors would like to acknowledge the invaluable help of Dr. Kathryn Rice and Dr. Anne Melzer for their guidance in the manuscript revision process

Patients with complex medical and psychosocial needs are at the highest risk for fragmented care and adverse health outcomes.^1,2 Although these high-risk patients make up only about 5% of the US patient population, they can account for as much as half of total health care costs.¹ High-risk patients are complicated to treat because most have multiple chronic medical conditions, and many have a wide variety of psychological and social needs. Thus, physician, physician assistant, and nurse practitioner primary care providers (PCPs), and nurses (registered nurses, licensed vocational nurses, and licensed practical nurses) must address the complexity of the human condition in conjunction with health problems.²

Background

Caring for high-risk patients within a tight clinic schedule geared to the provision of comprehensive care to large panels of less complex patients can be a source of stress for PCPs and nurses.^3-5 These conditions may lead to reduced well-being among primary care team members and to potential turnover.⁶ Furthermore, primary care staff may feel uncomfortable or lack the ability to address nonmedical concerns because of “person-specific factors that interfere with the delivery of usual care and decision making for whatever condition the patient has.”^7,8 Having additional support for complex patients, such as intensive outpatient management teams, may be protective both by reducing health care provider (HCP) stress and improving patient outcomes.^3,4

Caring for high-risk patients is challenging.^9-11 High-risk patient care may require additional, often unpaid, work hoursand may be discouraging because these patients can be difficult to engage in care.^7,12 Furthermore, high-risk patient care is challenging for primary care teams, since these complex patients may fall through the cracks and experience potentially preventable hospitalization or even death. Avoiding these negative consequences typically requires substantial time for the primary care team to engage and counsel the patient, family, and caregiver, through more frequent visits and additional communication. Furthermore, the primary care team typically must coordinate with other HCPs and resources—as many as 16 in a single year and as much as 12 for a single patient over an 80-day period.^13,14 Not surprisingly, primary care teams identify help with care coordination as a critical need that may be addressed with intensive management support.

Primary care at the US Department of Veterans Affairs (VA) Veterans Health Administration (VHA) provides care for a large proportion of high-risk patients.¹⁵ Accordingly, VHA provides a variety of intensive management options for equipping primary care teams with expanded resources for caring for high-risk patients, including those offered in a few sites by a pilot intensive management program.¹⁶ As part of the pilot’s evaluation, we studied the work experiences of PCPs and nurses, some of whom had experienced the pilot program and some of whom only had access to typical VHA intensive management resources, such as telehealth and specialty medical homes (referred to in the VA as patient aligned care teams, or PACT), eg, for women patients, for patients who are homeless, or for older adults.¹⁷ Surveys assessed whether HCPs who indicated they were likely to seek help from PACT intensive management (PIM) teams to care for high-risk patients had higher job satisfaction and intention to stay at VHA compared with those who were not likely to seek help.

While substantial research on high-risk patients’ intensive management needs has focused on patient-level outcomes of interventions for meeting those needs,little research has examined links between primary care team access to intensive management resources and experiences, such as job satisfaction and job retention.¹⁸ In the work presented here, our objectives were to (1) assess the likelihood that a PCP or nurse intent to manage high-risk patients by seeking care coordination help from or transferring care to an intensive management team; and (2) evaluate the relationship between PCP or nurse intentions regarding using intensive management help for high-risk patients and their job satisfaction and likelihood of leaving VA primary care. We hypothesized that the accessibility of intensive management resources and PCP and nurse receptivity to accessing those resources may affect job-related experiences.

Methods

This study was conducted as part of the evaluation of a VA pilot project to provide general primary care teams with intensive management support from interdisciplinary teams for high-risk patients in 5 VHA systems in 5 states (Ohio, Georgia, North Carolina, Wisconsin, and California).⁶ We sampled primary care staff at 39 primary care clinics within those systems, all of whom had access to VA intensive management resources. These included telehealth, health coaches, integrated mental health providers, and specialty PACTs for specific populations (eg, those who are women, elderly, homeless, HIV-positive, or who have serious mental illness). Of the 39 primary care clinics that participated in the survey, 8 also participated in the pilot program offering an intensive management team to support general primary care in their care of high-risk patients.

 

Data are from PCPs and nurses who completed 2 cross-sectional surveys (online or hard copy). We invited 1,000 PCPs and nurses to complete the first survey (fielded December 2014 to May 2015) and 863 to complete the second survey (fielded October 2016 to January 2017). A total of 436 completed the first survey for a response rate of 44%, and 313 completed the second survey for a response rate of 36%. We constructed a longitudinal cohort of 144 PCPs and nurses who completed both surveys and had data at 2 timepoints. This longitudinal cohort represents 33% of the 442 unique respondents who completed either the first or second survey. Overlap across surveys was low because of high staff turnover between survey waves.

Measures

Outcomes. We examined 2 single-item outcome measures to assess job satisfaction and retention (ie, intent to stay in primary care at the VA) measured in both surveys. These items were worded “Overall, I am satisfied with my job.” and “I intend to continue working in primary care at the VA for the next two years.” Both items were rated on a 5-point Likert scale.

Independent Variable. We assessed proclivity to seek assistance in caring for high-risk patients based on PCPs or nurses indicating that they are likely to either “manage these patients with ongoing care coordination assistance from an intensive management team” and/or “transfer these patients from primary care to another intensive management team or program specializing in high-risk patients.” These 2 items were rated on a 5-point Likert scale; we dichotomized the scale with likely or very likely indicating high proclivity (likelihood) for ease of interpretation of the combined items.

Covariates. We also controlled for indicators of staff demographic and practice characteristics in multivariate analyses. These included gender, staff type (PCP vs nurse), years practicing at a VA clinic, team staffing level (full vs partial), proportion of the panel consisting of high-risk patients (using binary indicators: 11 to 20% or > 20% compared with 0 to 10% as the reference group), and whether or not the site participated in the pilot program offering an intensive management team to support primary care for high-risk patients to distinguish the 8 pilot sites from nonpilot sites.

Statistical Analysis

We used ordinary least squares regression analysis to examine associations between the independent variable measured at time 1 and outcomes measured at time 2, controlling for time 1 outcomes among staff who completed both surveys (eg, the longitudinal cohort). We adjusted for time 1 covariates and clustering of staff within clinics, assuming a random effect with robust standard errors, and conducted multiple imputations for item-level missing data. Poststratification weights were used to adjust for survey nonresponse by staff type, gender, facilities participating in the innovations, and type of specialty PACT. We calculated weights based on the sampling frame of all PCPs and nurses for each survey.

Results

Table 1 shows the proportion of primary care staff responding to the surveys. For the longitudinal cohort, the response by staff type was similar to the sample of staff that responded only to a single survey, but the sample that did not respond to either survey included more physicians. There was also some variation by medical center. For example, a smaller proportion of the cohort was from site D and more was from site E compared with the other samples. The proportion of primary care staff in facilities that participated in the intensive management pilot was higher than the proportion in other facilities. More women (81.9%) were in the longitudinal cohort compared with 77.4% in the single-survey sample and 69.2% in the sample that responded to neither survey.

Both surveys were completed by 144 respondents while 442 completed 1 survey and 645 did not respond to either survey. The cohort was predominantly nurses (64.6%); Of the PCPs, 25% were physicians. Most staff were women (81.9%) and aged > 45 years (72.2%). Staff had practiced at their current VA clinics for a mean of 7.4 years, and most reported being on a fully-staffed primary care team (70%).

 

Multivariate Analyses

In the multivariable regression analyses, we found that the primary care staff, which reported being more likely to use intensive management teams to help care for high-risk patients at time 1, reported significantly higher satisfaction (0.63 points higher on a 5-point scale) and intention to stay (0.41 points higher) at VA primary care (both P < .05) at time 2, 18 months later (Table 2). These effect sizes are equivalent to nearly two-thirds and half of a standard deviation, respectively. Among our control variables, years practicing in the VA was significantly associated with a lower likelihood of intent to stay at the VA. Models account for 28% of the variation in satisfaction and 22% of the variation in retention. The Figure shows the adjusted means based on parameters from the regression models for job satisfaction and intent to stay at the VA as well as likelihood of using an intensive management team for high-risk patients. Job satisfaction is nearly 1 point higher among those who report being likely to draw on support from an intensive management team to care for high-risk patients compared with those who reported that they were unlikely to use such a team. The pattern for intent to stay at the VA, while less pronounced, is similar to that for satisfaction.

Discussion

Our findings are consistent with our hypothesis that augmenting primary care with high-risk patient intensive management assistance would improve primary care staff job satisfaction and retention. Findings also mirror recent qualitative studies, which have found that systemic approaches to augment primary care of high-risk patients are likely needed to maintain well-being.^7,19 We found a positive relationship between the likelihood of using intensive management teams to help care for their high-risk patients and reported job satisfaction and intent to continue to work within VA primary care 18 months later. To our knowledge, this study is the first to examine the potential impact on PCPs and nurses of using intensive management teams to help care for high-risk patients.

Our study suggests that this approach has the potential to alleviate PCP and nurse stress by incorporating intensive management teams as an extension of the medical home. Even high-functioning medical homes may find it challenging to meet the needs of their high-risk patients.^3,7,8 Time constraints and a structured clinic schedule may limit the ability of medical homes to balance the needs of the general panel vs the individual needs of high-risk patients who might benefit from intensive services. Limited knowledge and lack of training to address the broad array of problems faced by high-risk patients also makes care challenging.²

Intensive management services often include interdisciplinary and comprehensive assessments, care coordination, health care system navigation, and linkages to social and home care services.²⁰ Medical homes may benefit from these services, especially resources to support care coordination and communication with specialists and social services in large medical neighborhoods.²¹ For example, including a social worker on the intensive patient care team can help primary care staff by focusing specialized resources on nonmedical issues, such as chronic homelessness, substance use disorders, food insecurity, access to transportation, and poverty.¹⁸

Limitations

This study is subject to some limitations, including those typical of surveys, such as reliance on self-reported data. The longitudinal sample we studied had response rates that varied by site, participation in the pilot program, and gender relative to those who did not respond to both surveys; selection bias is possible. While we use a longitudinal cohort, we cannot attribute causality; it is possible that more satisfied staff are more likely to use intensive management teams rather than the use of intensive management teams contributing to higher satisfaction. Although each study site includes at least 1 type of intensive management resource, we cannot ascertain which intensive management resource primary care staff accessed, if any. While our sample size for the longitudinal cohort responders was limited, focusing on our longitudinal cohort provides more valid and reliable estimates than does using 2 cross-sectional samples with all responders. In addition, our models do not completely explain variation in the outcomes (R²= 0.28 and 0.22), although we included major explanatory factors, such as team staffing and professional type; other unmeasured factors may explain our outcomes. Finally, our provider sample may not generalize to HCPs in non-VA settings.

Conclusions

Our study expands on the limited data regarding the primary care staff experience of caring for high-risk patients and the potential impact of using interdisciplinary assistance to help care for this population. A strength of this study is the longitudinal cohort design that allowed us to understand staff receptivity to having access to intensive management resources to help care for high-risk patients over time among the same group of primary care staff. Given that an economic analysis has determined that the addition of the pilot intensive management program has been cost neutral to the VA, the possibility of its benefit, as suggested by our study findings, would support further implementation and evaluation of intensive management teams as a resource for PCPs caring for high-risk patients.²²

Understanding the mechanisms by which primary care staff benefit most from high-risk patient assistance, and how to optimize communication and coordination between primary care staff and intensive management teams for high-risk patients might further increase primary care satisfaction and retention.

Patients with complex medical and psychosocial needs are at the highest risk for fragmented care and adverse health outcomes.^1,2 Although these high-risk patients make up only about 5% of the US patient population, they can account for as much as half of total health care costs.¹ High-risk patients are complicated to treat because most have multiple chronic medical conditions, and many have a wide variety of psychological and social needs. Thus, physician, physician assistant, and nurse practitioner primary care providers (PCPs), and nurses (registered nurses, licensed vocational nurses, and licensed practical nurses) must address the complexity of the human condition in conjunction with health problems.²

Background

Caring for high-risk patients within a tight clinic schedule geared to the provision of comprehensive care to large panels of less complex patients can be a source of stress for PCPs and nurses.^3-5 These conditions may lead to reduced well-being among primary care team members and to potential turnover.⁶ Furthermore, primary care staff may feel uncomfortable or lack the ability to address nonmedical concerns because of “person-specific factors that interfere with the delivery of usual care and decision making for whatever condition the patient has.”^7,8 Having additional support for complex patients, such as intensive outpatient management teams, may be protective both by reducing health care provider (HCP) stress and improving patient outcomes.^3,4

Caring for high-risk patients is challenging.^9-11 High-risk patient care may require additional, often unpaid, work hoursand may be discouraging because these patients can be difficult to engage in care.^7,12 Furthermore, high-risk patient care is challenging for primary care teams, since these complex patients may fall through the cracks and experience potentially preventable hospitalization or even death. Avoiding these negative consequences typically requires substantial time for the primary care team to engage and counsel the patient, family, and caregiver, through more frequent visits and additional communication. Furthermore, the primary care team typically must coordinate with other HCPs and resources—as many as 16 in a single year and as much as 12 for a single patient over an 80-day period.^13,14 Not surprisingly, primary care teams identify help with care coordination as a critical need that may be addressed with intensive management support.

Primary care at the US Department of Veterans Affairs (VA) Veterans Health Administration (VHA) provides care for a large proportion of high-risk patients.¹⁵ Accordingly, VHA provides a variety of intensive management options for equipping primary care teams with expanded resources for caring for high-risk patients, including those offered in a few sites by a pilot intensive management program.¹⁶ As part of the pilot’s evaluation, we studied the work experiences of PCPs and nurses, some of whom had experienced the pilot program and some of whom only had access to typical VHA intensive management resources, such as telehealth and specialty medical homes (referred to in the VA as patient aligned care teams, or PACT), eg, for women patients, for patients who are homeless, or for older adults.¹⁷ Surveys assessed whether HCPs who indicated they were likely to seek help from PACT intensive management (PIM) teams to care for high-risk patients had higher job satisfaction and intention to stay at VHA compared with those who were not likely to seek help.

While substantial research on high-risk patients’ intensive management needs has focused on patient-level outcomes of interventions for meeting those needs,little research has examined links between primary care team access to intensive management resources and experiences, such as job satisfaction and job retention.¹⁸ In the work presented here, our objectives were to (1) assess the likelihood that a PCP or nurse intent to manage high-risk patients by seeking care coordination help from or transferring care to an intensive management team; and (2) evaluate the relationship between PCP or nurse intentions regarding using intensive management help for high-risk patients and their job satisfaction and likelihood of leaving VA primary care. We hypothesized that the accessibility of intensive management resources and PCP and nurse receptivity to accessing those resources may affect job-related experiences.

Methods

This study was conducted as part of the evaluation of a VA pilot project to provide general primary care teams with intensive management support from interdisciplinary teams for high-risk patients in 5 VHA systems in 5 states (Ohio, Georgia, North Carolina, Wisconsin, and California).⁶ We sampled primary care staff at 39 primary care clinics within those systems, all of whom had access to VA intensive management resources. These included telehealth, health coaches, integrated mental health providers, and specialty PACTs for specific populations (eg, those who are women, elderly, homeless, HIV-positive, or who have serious mental illness). Of the 39 primary care clinics that participated in the survey, 8 also participated in the pilot program offering an intensive management team to support general primary care in their care of high-risk patients.

 

Data are from PCPs and nurses who completed 2 cross-sectional surveys (online or hard copy). We invited 1,000 PCPs and nurses to complete the first survey (fielded December 2014 to May 2015) and 863 to complete the second survey (fielded October 2016 to January 2017). A total of 436 completed the first survey for a response rate of 44%, and 313 completed the second survey for a response rate of 36%. We constructed a longitudinal cohort of 144 PCPs and nurses who completed both surveys and had data at 2 timepoints. This longitudinal cohort represents 33% of the 442 unique respondents who completed either the first or second survey. Overlap across surveys was low because of high staff turnover between survey waves.

Measures

Outcomes. We examined 2 single-item outcome measures to assess job satisfaction and retention (ie, intent to stay in primary care at the VA) measured in both surveys. These items were worded “Overall, I am satisfied with my job.” and “I intend to continue working in primary care at the VA for the next two years.” Both items were rated on a 5-point Likert scale.

Independent Variable. We assessed proclivity to seek assistance in caring for high-risk patients based on PCPs or nurses indicating that they are likely to either “manage these patients with ongoing care coordination assistance from an intensive management team” and/or “transfer these patients from primary care to another intensive management team or program specializing in high-risk patients.” These 2 items were rated on a 5-point Likert scale; we dichotomized the scale with likely or very likely indicating high proclivity (likelihood) for ease of interpretation of the combined items.

Covariates. We also controlled for indicators of staff demographic and practice characteristics in multivariate analyses. These included gender, staff type (PCP vs nurse), years practicing at a VA clinic, team staffing level (full vs partial), proportion of the panel consisting of high-risk patients (using binary indicators: 11 to 20% or > 20% compared with 0 to 10% as the reference group), and whether or not the site participated in the pilot program offering an intensive management team to support primary care for high-risk patients to distinguish the 8 pilot sites from nonpilot sites.

Statistical Analysis

We used ordinary least squares regression analysis to examine associations between the independent variable measured at time 1 and outcomes measured at time 2, controlling for time 1 outcomes among staff who completed both surveys (eg, the longitudinal cohort). We adjusted for time 1 covariates and clustering of staff within clinics, assuming a random effect with robust standard errors, and conducted multiple imputations for item-level missing data. Poststratification weights were used to adjust for survey nonresponse by staff type, gender, facilities participating in the innovations, and type of specialty PACT. We calculated weights based on the sampling frame of all PCPs and nurses for each survey.

Results

Table 1 shows the proportion of primary care staff responding to the surveys. For the longitudinal cohort, the response by staff type was similar to the sample of staff that responded only to a single survey, but the sample that did not respond to either survey included more physicians. There was also some variation by medical center. For example, a smaller proportion of the cohort was from site D and more was from site E compared with the other samples. The proportion of primary care staff in facilities that participated in the intensive management pilot was higher than the proportion in other facilities. More women (81.9%) were in the longitudinal cohort compared with 77.4% in the single-survey sample and 69.2% in the sample that responded to neither survey.

Both surveys were completed by 144 respondents while 442 completed 1 survey and 645 did not respond to either survey. The cohort was predominantly nurses (64.6%); Of the PCPs, 25% were physicians. Most staff were women (81.9%) and aged > 45 years (72.2%). Staff had practiced at their current VA clinics for a mean of 7.4 years, and most reported being on a fully-staffed primary care team (70%).

 

Multivariate Analyses

In the multivariable regression analyses, we found that the primary care staff, which reported being more likely to use intensive management teams to help care for high-risk patients at time 1, reported significantly higher satisfaction (0.63 points higher on a 5-point scale) and intention to stay (0.41 points higher) at VA primary care (both P < .05) at time 2, 18 months later (Table 2). These effect sizes are equivalent to nearly two-thirds and half of a standard deviation, respectively. Among our control variables, years practicing in the VA was significantly associated with a lower likelihood of intent to stay at the VA. Models account for 28% of the variation in satisfaction and 22% of the variation in retention. The Figure shows the adjusted means based on parameters from the regression models for job satisfaction and intent to stay at the VA as well as likelihood of using an intensive management team for high-risk patients. Job satisfaction is nearly 1 point higher among those who report being likely to draw on support from an intensive management team to care for high-risk patients compared with those who reported that they were unlikely to use such a team. The pattern for intent to stay at the VA, while less pronounced, is similar to that for satisfaction.

Discussion

Our findings are consistent with our hypothesis that augmenting primary care with high-risk patient intensive management assistance would improve primary care staff job satisfaction and retention. Findings also mirror recent qualitative studies, which have found that systemic approaches to augment primary care of high-risk patients are likely needed to maintain well-being.^7,19 We found a positive relationship between the likelihood of using intensive management teams to help care for their high-risk patients and reported job satisfaction and intent to continue to work within VA primary care 18 months later. To our knowledge, this study is the first to examine the potential impact on PCPs and nurses of using intensive management teams to help care for high-risk patients.

Our study suggests that this approach has the potential to alleviate PCP and nurse stress by incorporating intensive management teams as an extension of the medical home. Even high-functioning medical homes may find it challenging to meet the needs of their high-risk patients.^3,7,8 Time constraints and a structured clinic schedule may limit the ability of medical homes to balance the needs of the general panel vs the individual needs of high-risk patients who might benefit from intensive services. Limited knowledge and lack of training to address the broad array of problems faced by high-risk patients also makes care challenging.²

Intensive management services often include interdisciplinary and comprehensive assessments, care coordination, health care system navigation, and linkages to social and home care services.²⁰ Medical homes may benefit from these services, especially resources to support care coordination and communication with specialists and social services in large medical neighborhoods.²¹ For example, including a social worker on the intensive patient care team can help primary care staff by focusing specialized resources on nonmedical issues, such as chronic homelessness, substance use disorders, food insecurity, access to transportation, and poverty.¹⁸

Limitations

This study is subject to some limitations, including those typical of surveys, such as reliance on self-reported data. The longitudinal sample we studied had response rates that varied by site, participation in the pilot program, and gender relative to those who did not respond to both surveys; selection bias is possible. While we use a longitudinal cohort, we cannot attribute causality; it is possible that more satisfied staff are more likely to use intensive management teams rather than the use of intensive management teams contributing to higher satisfaction. Although each study site includes at least 1 type of intensive management resource, we cannot ascertain which intensive management resource primary care staff accessed, if any. While our sample size for the longitudinal cohort responders was limited, focusing on our longitudinal cohort provides more valid and reliable estimates than does using 2 cross-sectional samples with all responders. In addition, our models do not completely explain variation in the outcomes (R²= 0.28 and 0.22), although we included major explanatory factors, such as team staffing and professional type; other unmeasured factors may explain our outcomes. Finally, our provider sample may not generalize to HCPs in non-VA settings.

Conclusions

Our study expands on the limited data regarding the primary care staff experience of caring for high-risk patients and the potential impact of using interdisciplinary assistance to help care for this population. A strength of this study is the longitudinal cohort design that allowed us to understand staff receptivity to having access to intensive management resources to help care for high-risk patients over time among the same group of primary care staff. Given that an economic analysis has determined that the addition of the pilot intensive management program has been cost neutral to the VA, the possibility of its benefit, as suggested by our study findings, would support further implementation and evaluation of intensive management teams as a resource for PCPs caring for high-risk patients.²²

Understanding the mechanisms by which primary care staff benefit most from high-risk patient assistance, and how to optimize communication and coordination between primary care staff and intensive management teams for high-risk patients might further increase primary care satisfaction and retention.

Patients with complex medical and psychosocial needs are at the highest risk for fragmented care and adverse health outcomes.^1,2 Although these high-risk patients make up only about 5% of the US patient population, they can account for as much as half of total health care costs.¹ High-risk patients are complicated to treat because most have multiple chronic medical conditions, and many have a wide variety of psychological and social needs. Thus, physician, physician assistant, and nurse practitioner primary care providers (PCPs), and nurses (registered nurses, licensed vocational nurses, and licensed practical nurses) must address the complexity of the human condition in conjunction with health problems.²

Background

Caring for high-risk patients within a tight clinic schedule geared to the provision of comprehensive care to large panels of less complex patients can be a source of stress for PCPs and nurses.^3-5 These conditions may lead to reduced well-being among primary care team members and to potential turnover.⁶ Furthermore, primary care staff may feel uncomfortable or lack the ability to address nonmedical concerns because of “person-specific factors that interfere with the delivery of usual care and decision making for whatever condition the patient has.”^7,8 Having additional support for complex patients, such as intensive outpatient management teams, may be protective both by reducing health care provider (HCP) stress and improving patient outcomes.^3,4

Caring for high-risk patients is challenging.^9-11 High-risk patient care may require additional, often unpaid, work hoursand may be discouraging because these patients can be difficult to engage in care.^7,12 Furthermore, high-risk patient care is challenging for primary care teams, since these complex patients may fall through the cracks and experience potentially preventable hospitalization or even death. Avoiding these negative consequences typically requires substantial time for the primary care team to engage and counsel the patient, family, and caregiver, through more frequent visits and additional communication. Furthermore, the primary care team typically must coordinate with other HCPs and resources—as many as 16 in a single year and as much as 12 for a single patient over an 80-day period.^13,14 Not surprisingly, primary care teams identify help with care coordination as a critical need that may be addressed with intensive management support.

Primary care at the US Department of Veterans Affairs (VA) Veterans Health Administration (VHA) provides care for a large proportion of high-risk patients.¹⁵ Accordingly, VHA provides a variety of intensive management options for equipping primary care teams with expanded resources for caring for high-risk patients, including those offered in a few sites by a pilot intensive management program.¹⁶ As part of the pilot’s evaluation, we studied the work experiences of PCPs and nurses, some of whom had experienced the pilot program and some of whom only had access to typical VHA intensive management resources, such as telehealth and specialty medical homes (referred to in the VA as patient aligned care teams, or PACT), eg, for women patients, for patients who are homeless, or for older adults.¹⁷ Surveys assessed whether HCPs who indicated they were likely to seek help from PACT intensive management (PIM) teams to care for high-risk patients had higher job satisfaction and intention to stay at VHA compared with those who were not likely to seek help.

While substantial research on high-risk patients’ intensive management needs has focused on patient-level outcomes of interventions for meeting those needs,little research has examined links between primary care team access to intensive management resources and experiences, such as job satisfaction and job retention.¹⁸ In the work presented here, our objectives were to (1) assess the likelihood that a PCP or nurse intent to manage high-risk patients by seeking care coordination help from or transferring care to an intensive management team; and (2) evaluate the relationship between PCP or nurse intentions regarding using intensive management help for high-risk patients and their job satisfaction and likelihood of leaving VA primary care. We hypothesized that the accessibility of intensive management resources and PCP and nurse receptivity to accessing those resources may affect job-related experiences.

Methods

This study was conducted as part of the evaluation of a VA pilot project to provide general primary care teams with intensive management support from interdisciplinary teams for high-risk patients in 5 VHA systems in 5 states (Ohio, Georgia, North Carolina, Wisconsin, and California).⁶ We sampled primary care staff at 39 primary care clinics within those systems, all of whom had access to VA intensive management resources. These included telehealth, health coaches, integrated mental health providers, and specialty PACTs for specific populations (eg, those who are women, elderly, homeless, HIV-positive, or who have serious mental illness). Of the 39 primary care clinics that participated in the survey, 8 also participated in the pilot program offering an intensive management team to support general primary care in their care of high-risk patients.

 

Data are from PCPs and nurses who completed 2 cross-sectional surveys (online or hard copy). We invited 1,000 PCPs and nurses to complete the first survey (fielded December 2014 to May 2015) and 863 to complete the second survey (fielded October 2016 to January 2017). A total of 436 completed the first survey for a response rate of 44%, and 313 completed the second survey for a response rate of 36%. We constructed a longitudinal cohort of 144 PCPs and nurses who completed both surveys and had data at 2 timepoints. This longitudinal cohort represents 33% of the 442 unique respondents who completed either the first or second survey. Overlap across surveys was low because of high staff turnover between survey waves.

Measures

Outcomes. We examined 2 single-item outcome measures to assess job satisfaction and retention (ie, intent to stay in primary care at the VA) measured in both surveys. These items were worded “Overall, I am satisfied with my job.” and “I intend to continue working in primary care at the VA for the next two years.” Both items were rated on a 5-point Likert scale.

Independent Variable. We assessed proclivity to seek assistance in caring for high-risk patients based on PCPs or nurses indicating that they are likely to either “manage these patients with ongoing care coordination assistance from an intensive management team” and/or “transfer these patients from primary care to another intensive management team or program specializing in high-risk patients.” These 2 items were rated on a 5-point Likert scale; we dichotomized the scale with likely or very likely indicating high proclivity (likelihood) for ease of interpretation of the combined items.

Covariates. We also controlled for indicators of staff demographic and practice characteristics in multivariate analyses. These included gender, staff type (PCP vs nurse), years practicing at a VA clinic, team staffing level (full vs partial), proportion of the panel consisting of high-risk patients (using binary indicators: 11 to 20% or > 20% compared with 0 to 10% as the reference group), and whether or not the site participated in the pilot program offering an intensive management team to support primary care for high-risk patients to distinguish the 8 pilot sites from nonpilot sites.

Statistical Analysis

We used ordinary least squares regression analysis to examine associations between the independent variable measured at time 1 and outcomes measured at time 2, controlling for time 1 outcomes among staff who completed both surveys (eg, the longitudinal cohort). We adjusted for time 1 covariates and clustering of staff within clinics, assuming a random effect with robust standard errors, and conducted multiple imputations for item-level missing data. Poststratification weights were used to adjust for survey nonresponse by staff type, gender, facilities participating in the innovations, and type of specialty PACT. We calculated weights based on the sampling frame of all PCPs and nurses for each survey.

Results

Table 1 shows the proportion of primary care staff responding to the surveys. For the longitudinal cohort, the response by staff type was similar to the sample of staff that responded only to a single survey, but the sample that did not respond to either survey included more physicians. There was also some variation by medical center. For example, a smaller proportion of the cohort was from site D and more was from site E compared with the other samples. The proportion of primary care staff in facilities that participated in the intensive management pilot was higher than the proportion in other facilities. More women (81.9%) were in the longitudinal cohort compared with 77.4% in the single-survey sample and 69.2% in the sample that responded to neither survey.

Both surveys were completed by 144 respondents while 442 completed 1 survey and 645 did not respond to either survey. The cohort was predominantly nurses (64.6%); Of the PCPs, 25% were physicians. Most staff were women (81.9%) and aged > 45 years (72.2%). Staff had practiced at their current VA clinics for a mean of 7.4 years, and most reported being on a fully-staffed primary care team (70%).

 

Multivariate Analyses

In the multivariable regression analyses, we found that the primary care staff, which reported being more likely to use intensive management teams to help care for high-risk patients at time 1, reported significantly higher satisfaction (0.63 points higher on a 5-point scale) and intention to stay (0.41 points higher) at VA primary care (both P < .05) at time 2, 18 months later (Table 2). These effect sizes are equivalent to nearly two-thirds and half of a standard deviation, respectively. Among our control variables, years practicing in the VA was significantly associated with a lower likelihood of intent to stay at the VA. Models account for 28% of the variation in satisfaction and 22% of the variation in retention. The Figure shows the adjusted means based on parameters from the regression models for job satisfaction and intent to stay at the VA as well as likelihood of using an intensive management team for high-risk patients. Job satisfaction is nearly 1 point higher among those who report being likely to draw on support from an intensive management team to care for high-risk patients compared with those who reported that they were unlikely to use such a team. The pattern for intent to stay at the VA, while less pronounced, is similar to that for satisfaction.

Discussion

Our findings are consistent with our hypothesis that augmenting primary care with high-risk patient intensive management assistance would improve primary care staff job satisfaction and retention. Findings also mirror recent qualitative studies, which have found that systemic approaches to augment primary care of high-risk patients are likely needed to maintain well-being.^7,19 We found a positive relationship between the likelihood of using intensive management teams to help care for their high-risk patients and reported job satisfaction and intent to continue to work within VA primary care 18 months later. To our knowledge, this study is the first to examine the potential impact on PCPs and nurses of using intensive management teams to help care for high-risk patients.

Our study suggests that this approach has the potential to alleviate PCP and nurse stress by incorporating intensive management teams as an extension of the medical home. Even high-functioning medical homes may find it challenging to meet the needs of their high-risk patients.^3,7,8 Time constraints and a structured clinic schedule may limit the ability of medical homes to balance the needs of the general panel vs the individual needs of high-risk patients who might benefit from intensive services. Limited knowledge and lack of training to address the broad array of problems faced by high-risk patients also makes care challenging.²

Intensive management services often include interdisciplinary and comprehensive assessments, care coordination, health care system navigation, and linkages to social and home care services.²⁰ Medical homes may benefit from these services, especially resources to support care coordination and communication with specialists and social services in large medical neighborhoods.²¹ For example, including a social worker on the intensive patient care team can help primary care staff by focusing specialized resources on nonmedical issues, such as chronic homelessness, substance use disorders, food insecurity, access to transportation, and poverty.¹⁸

Limitations

This study is subject to some limitations, including those typical of surveys, such as reliance on self-reported data. The longitudinal sample we studied had response rates that varied by site, participation in the pilot program, and gender relative to those who did not respond to both surveys; selection bias is possible. While we use a longitudinal cohort, we cannot attribute causality; it is possible that more satisfied staff are more likely to use intensive management teams rather than the use of intensive management teams contributing to higher satisfaction. Although each study site includes at least 1 type of intensive management resource, we cannot ascertain which intensive management resource primary care staff accessed, if any. While our sample size for the longitudinal cohort responders was limited, focusing on our longitudinal cohort provides more valid and reliable estimates than does using 2 cross-sectional samples with all responders. In addition, our models do not completely explain variation in the outcomes (R²= 0.28 and 0.22), although we included major explanatory factors, such as team staffing and professional type; other unmeasured factors may explain our outcomes. Finally, our provider sample may not generalize to HCPs in non-VA settings.

Conclusions

Our study expands on the limited data regarding the primary care staff experience of caring for high-risk patients and the potential impact of using interdisciplinary assistance to help care for this population. A strength of this study is the longitudinal cohort design that allowed us to understand staff receptivity to having access to intensive management resources to help care for high-risk patients over time among the same group of primary care staff. Given that an economic analysis has determined that the addition of the pilot intensive management program has been cost neutral to the VA, the possibility of its benefit, as suggested by our study findings, would support further implementation and evaluation of intensive management teams as a resource for PCPs caring for high-risk patients.²²

Understanding the mechanisms by which primary care staff benefit most from high-risk patient assistance, and how to optimize communication and coordination between primary care staff and intensive management teams for high-risk patients might further increase primary care satisfaction and retention.

For decades, opioids have been a mainstay in the management of pain after total joint arthroplasty. In the past 10 years, however, opioid prescribing has come under increased scrutiny due to a rise in rates of opioid abuse, pill diversion, and opioid-related deaths.^1,2 Opioids are associated with adverse effects, including nausea, vomiting, constipation, apathy, and respiratory depression, all of which influence arthroplasty outcomes and affect the patient experience. Although primary care groups account for nearly half of prescriptions written, orthopedic surgeons have the third highest per capita rate of opioid prescribing of all medical specialties.^3,4 This puts orthopedic surgeons, particularly those who perform routine procedures, in an opportune but challenging position to confront this problem through novel pain management strategies.

Approximately 1 million total knee arthroplasties (TKAs) are performed in the US every year, and the US Department of Veterans Affairs (VA) health system performs about 10,000 hip and knee joint replacements.^5,6 There is no standardization of opioid prescribing in the postoperative period following these procedures, and studies have reported a wide variation in prescribing habits even within a single institution for a specific surgery.⁷ Patients who undergo TKA are at particularly high risk of long-term opioid use if they are on continuous opioids at the time of surgery; this is problematic in a VA patient population in which at least 16% of patients are prescribed opioids in a given year.⁸ Furthermore, veterans are twice as likely as nonveterans to die of an accidental overdose.⁹ Despite these risks, opioids remain a cornerstone of postoperative pain management both within and outside of the VA.¹⁰

In 2018, to limit unnecessary prescribing of opioid pain medication, the total joint service at the VA Portland Health Care System (VAPHCS) in Oregon implemented the Minimizing Opioids after Joint Operation (MOJO) postoperative pain protocol. The goal of the protocol was to reduce opioid use following TKA. The objectives were to provide safe, appropriate analgesia while allowing early mobilization and discharge without a concomitant increase in readmissions or emergency department (ED) visits. The purpose of this retrospective chart review was to compare the efficacy of the MOJO protocol with our historical experience and report our preliminary results.

Methods

Institutional review board approval was obtained to retrospectively review the medical records of patients who had undergone TKA surgery during 2018 at VAPHCS. The MOJO protocol was composed of several simultaneous changes. The centerpiece of the new protocol was a drastic decrease in routine prescription of postoperative opioids (Table 1). Other changes included instructing patients to reduce the use of preoperative opioid pain medication 6 weeks before surgery with a goal of no opioid consumption, perform daily sets of preoperative exercises, and attend a preoperative consultation/education session with a nurse coordinator to emphasize early recovery and discharge. In patients with chronic use of opioid pain medication (particularly those for whom the medication had been prescribed for other sources of pain, such as lumbar back pain), the goal was daily opioid use of ≤ 30 morphine equivalent doses (MEDs). During the inpatient stay, we stopped prescribing prophylactic pain medication prior to physical therapy (PT).

We encouraged preoperative optimization of muscle strength by giving instructions for 4 to 8 weeks of daily exercises (Appendix). We introduced perioperative adductor canal blocks (at the discretion of the anesthesia team) and transitioned to surgery without a tourniquet. Patients in both groups received intraoperative antibiotics and IV tranexamic acid (TXA); the MOJO group also received topical TXA.

Further patient care optimization included providing patients with a team-based approach, which consisted of nurse coordinators, physician assistants and nurse practitioners, residents, and the attending surgeon. Our team reviews the planned pain management protocol, perioperative expectations, criteria for discharge, and anticipated surgical outcomes with the patient during their preoperative visits. On postoperative day 1, these members round as a team to encourage patients in their immediate postoperative recovery and rehabilitation. During rounds, the team assesses whether the patient meets the criteria for discharge, adjusting the pain management protocol if necessary.

Changes in surgical technique included arthrotomy with electrocautery, minimizing traumatic dissection or resection of the synovial tissue, and intra-articular injection of a cocktail of ropivacaine 5 mg/mL 40 mL, epinephrine 1:1,000 0.5 mL, and methylprednisolone sodium 40 mg diluted with normal saline to a total volume of 120 mL.

The new routine was gradually implemented beginning January 2017 and fully implemented by July 2018. This study compared the first 20 consecutive patients undergoing primary TKA after July 2018 to the last 20 consecutive patients undergoing primary TKA prior to January 2017. Exclusion criteria included bilateral TKA, death before 90 days, and revision as the indication for surgery. The senior attending surgeon performed all surgeries using a standard midline approach. The majority of surgeries were performed using a cemented Vanguard total knee system (Zimmer Biomet); 4 patients in the historical group had a NexGen knee system, cementless monoblock tibial components (Zimmer Biomet); and 1 patient had a Logic knee system (Exactech). Surgical selection criteria for patients did not differ between groups.

 

Electronic health records were reviewed and data were abstracted. The data included demographic information (age, gender, body mass index [BMI], diagnosis, and procedure), surgical factors (American Society of Anesthesiologists score, Risk Assessment and Predictive Tool score, operative time, tourniquet time, estimated blood loss), hospital factors (length of stay [LOS], discharge location), postoperative pain scores (measured on postoperative day 1 and on day of discharge), and postdischarge events (90-day complications, telephone calls reporting pain, reoperations, returns to the ED, 90-day readmissions).

The primary outcome was the mean postoperative daily MED during the inpatient stay. Secondary outcomes included pain on postoperative day 1, pain at the time of discharge, LOS, hospital readmissions, and ED visits within 90 days of surgery. Because different opioid pain medications were used by patients postoperatively, all opioids were converted to MED prior to the final analysis. Collected patient data were de-identified prior to analysis.

Power analysis was conducted to determine whether the study had sufficient population size to reject the null hypothesis for the primary outcome measure. Because practitioners controlled postoperative opioid use, a Cohen’s d of 1.0 was used so that a very large effect size was needed to reach clinical significance. Statistical significance was set to 0.05, and patient groups were set at 20 patients each. This yielded an appropriate power of 0.87. Population characteristics were compared between groups using t tests and χ² tests as appropriate. To analyze the primary outcome, comparisons were made between the 2 cohorts using 2-tailed t tests. Secondary outcomes were compared between groups using t tests or χ² tests. All statistics were performed using R version 3.5.2. Power analysis was conducted using the package pwr.¹¹ Statistical significance was set at P < .05.

Results

Forty patients met the inclusion criteria, evenly divided between those undergoing TKA before and after instituting the MOJO protocol (Table 2). A single patient in the MOJO group died and was excluded. A patient who underwent bilateral TKA also was excluded. Both groups reflected the male predominance of the VA patient population. MOJO patients tended to have lower BMIs (34 vs 30, P < .01). All patients indicated for surgery with preoperative opioid use were able to titrate down to their preoperative goal as verified by prescriptions filled at VA pharmacies. Twelve of the patients in the MOJO group received adductor canal blocks.

Results of t tests and χ² tests comparing primary and secondary endpoints are listed in Table 3. Differences between the daily MEDs given in the historical and MOJO groups are shown. There were significant differences between the pre-MOJO and MOJO groups with regard to daily inpatient MEDs (82 mg vs 29 mg, P < .01) and total inpatient MEDs (306 mg vs 32 mg, P < .01). There was less self-reported pain on postoperative day 1 in the MOJO group (5.5 vs 3.9, P < .01), decreased LOS (4.4 days vs 1.2 days, P < .01), a trend toward fewer total ED visits (6 vs 2, P = .24), and fewer discharges to skilled nursing facilities (12 vs 0, P < .01). There were no blood transfusions in either group.

There were no readmissions due to uncontrolled pain. There was 1 readmission for shortness of breath in the MOJO group. The patient was discharged home the following day after ruling out thromboembolic and cardiovascular events. One patient from the control group was readmitted after missing a step on a staircase and falling. The patient sustained a quadriceps tendon rupture and underwent primary suture repair.

Discussion

Our results demonstrate that a multimodal approach to significantly reduce postoperative opioid use in patients with TKA is possible without increasing readmissions or ED visits for pain control. The patients in the MOJO group had a faster recovery, earlier discharge, and less use of postoperative opioid medication. Our approach to postoperative pain management was divided into 2 main categories: patient optimization and surgical optimization.

Patient Selection

Besides the standard evaluation and optimization of patients’ medical conditions, identifying and optimizing at-risk patients before surgery was a critical component of our protocol. Managing postoperative pain in patients with prior opioid use is an intractable challenge in orthopedic surgery. Patients with a history of chronic pain and preoperative use of opioid medications remain at higher risk of postoperative chronic pain and persistent use of opioid medication despite no obvious surgical complications.⁸ In a sample of > 6,000 veterans who underwent TKA at VA hospitals in 2014, 57% of the patients with daily use of opioids in the 90 days before surgery remained on opioids 1 year after surgery (vs 2 % in patients not on long-term opioids).⁸ This relationship between pre- and postoperative opioid use also was dose dependent.¹²

 

Furthermore, those with high preoperative use may experience worse outcomes relative to the opioid naive population as measured by arthritis-specific pain indices.¹³ In a well-powered retrospective study of patients who underwent elective orthopedic procedures, preoperative opioid abuse or dependence (determined by the International Classification of Diseases, Ninth Revision diagnosis) increased inpatient mortality, aggregate morbidity, surgical site infection, myocardial infarction, and LOS.¹⁴ Preoperative opioid use also has been associated with increased risk of ED visits, readmission, infection, stiffness, and aseptic revision.¹⁵ In patients with TKA in the VA specifically, preoperative opioid use (> 3 months in the prior year) was associated with increased revision rates that were even higher than those for patients with diabetes mellitus.¹⁶

Patient Education

Based on this evidence, we instruct patients to reduce their preoperative opioid dosing to zero (for patients with joint pain) or < 30 MED (for patients using opioids for other reasons). Although preoperative reduction of opioid use has been shown to improve outcomes after TKA, pain subspecialty recommendations for patients with chronic opioid use recommend considering adjunctive therapies, including transcutaneous electrical nerve stimulation, cognitive behavioral therapy, gabapentin, or ketamine.^17,18 Through patient education our team has been successful in decreasing preoperative opioid use without adding other drugs or modalities.

Patient Optimization

Preoperative patient optimization included 4 to 8 weeks of daily sets of physical activity instructions (prehab) to improve the musculoskeletal function. These instructions are given to patients 4 to 8 weeks before surgery and aim to improve the patient’s balance, mobility, and functional ability (Appendix). Meta-analysis has shown that patients who undergo preoperative PT have a small but statistically significant decrease in postoperative pain at 4 weeks, though this does not persist beyond that period.¹⁹

We did note a lower BMI in patients in the MOJO group. Though this has the potential to be a confounder, a study of BMI in > 4,000 patients who underwent joint replacement surgery has shown that BMI is not associated with differences in postoperative pain.²⁰

Surgeon and Surgical-Related Variables

Patients in the MOJO group had increased use of adductor canal blocks. A 2017 meta-analysis of 12,530 patients comparing analgesic modalities found that peripheral nerve blocks targeting multiple nerves (eg, femoral/sciatic) decreased pain at rest, decreased opioid consumption, and improved range of motion postoperatively.²¹ Also, these were found to be superior to single nerve blocks, periarticular infiltration, and epidural blocks.²¹ However, major nerve and epidural blocks affecting the lower extremity may increase the risk of falls and prolong LOS.^22,23 The preferred peripheral block at VAPHCS is a single shot ultrasound-guided adductor canal block before the induction of general or spinal anesthesia. A randomized controlled trial has demonstrated superiority of this block to the femoral nerve block with regard to postoperative quadriceps strength, conferring the theoretical advantage of decreased fall risk and ability to participate in immediate PT.²⁴ Although we are unable to confirm an association between anesthetic modalities and opioid burden, our clinical impression is that blocks were effective at reducing immediate postoperative pain. However, among MOJO patients there were no differences in patients with and without blocks for either pain (4.2 vs 3.8, P = .69) or opioid consumption (28.8 vs 33.0, P = .72) after surgery, though our study was not powered to detect a difference in this restricted subgroup.

Patients who frequently had reported postoperative thigh pain prompted us to make changes in our surgical technique, performing TKA without use of a tourniquet. Tourniquet use has been associated with an increased risk of thigh pain after TKA by multiple authors.^25,26 Postoperative thigh pain also is pressure dependent.²⁷ In addition, its use may be associated with a slightly increased risk of thromboembolic events and delayed functional recovery.^28,29

Because postoperative hemarthrosis is associated with more pain and reduced joint recovery function, we used topical TXA to reduce postoperative surgical site and joint hematoma. TXA (either oral, IV, or topical) during TKA is used to control postoperative bleeding primarily and decrease the need for transfusion without concomitant increase in thromboembolic events.^30,31 Topical TXA may be more effective than IV, particularly in the immediate postoperative period.³² Although pain typically is not an endpoint in studies of TXA, a prospective study of 48 patients showed evidence that its use may be associated with decreased postoperative pain in the first 24 hours after surgery (though not after).³³ Finally, the use of intra-articular injection has evolved in our clinical practice, but literature is lacking with regard to its efficacy; more studies are needed to determine its effect relative to no injection. We have not seen any benefits to using cryotherapy in our practice; considering the costs for equipment and health care provider time, cryotherapy was not included in our new protocol.

Limitations

This is a nonrandomized retrospective single-institution study. Our study population is composed of mostly males with military experience and is not necessarily a representative sample of the general population eligible for joint arthroplasty. Our primary endpoint (reduction of opioid use postoperatively) also was a cornerstone of our intervention. To account for this, we set a very large effect size in our power analysis and evaluated multiple secondary endpoints to determine whether postoperative pain remained well controlled and complications/readmission minimized with our interventions. Because our intervention was multimodal, our study cannot make conclusions about the effect of a particular component of our treatment strategy. We did not measure or compare functional outcomes between both groups, which offers an opportunity for further research.

 

These limitations are balanced by several strengths. Our cohort was well controlled with respect to the dose and type of drug used. There is staff dedicated to postoperative telephone follow-up after discharge, and veterans are apt to seek care within the VA health care system, which improves case finding for complications and ED visits. No patients were lost to follow-up. Moreover, our drastic reduction in opioid use is promising enough to warrant reporting, while the broader orthopedic literature explores the relative impact of each variable.

Conclusions

The MOJO protocol has been effective for reducing postoperative opioid use after TKA without compromising effective pain management. The drastic reduction in the postoperative use of opioid pain medications and LOS have contributed to a cultural shift within our department, comprehensive team approach, multimodal pain management, and preoperative patient optimization. Further investigations are required to assess the impact of each intervention on observed outcomes. However, the framework and routines are applicable to other institutions and surgical specialties.

Acknowledgments

The authors recognize Derek Bond, MD, for his help in creating the MOJO acronym.

For decades, opioids have been a mainstay in the management of pain after total joint arthroplasty. In the past 10 years, however, opioid prescribing has come under increased scrutiny due to a rise in rates of opioid abuse, pill diversion, and opioid-related deaths.^1,2 Opioids are associated with adverse effects, including nausea, vomiting, constipation, apathy, and respiratory depression, all of which influence arthroplasty outcomes and affect the patient experience. Although primary care groups account for nearly half of prescriptions written, orthopedic surgeons have the third highest per capita rate of opioid prescribing of all medical specialties.^3,4 This puts orthopedic surgeons, particularly those who perform routine procedures, in an opportune but challenging position to confront this problem through novel pain management strategies.

Approximately 1 million total knee arthroplasties (TKAs) are performed in the US every year, and the US Department of Veterans Affairs (VA) health system performs about 10,000 hip and knee joint replacements.^5,6 There is no standardization of opioid prescribing in the postoperative period following these procedures, and studies have reported a wide variation in prescribing habits even within a single institution for a specific surgery.⁷ Patients who undergo TKA are at particularly high risk of long-term opioid use if they are on continuous opioids at the time of surgery; this is problematic in a VA patient population in which at least 16% of patients are prescribed opioids in a given year.⁸ Furthermore, veterans are twice as likely as nonveterans to die of an accidental overdose.⁹ Despite these risks, opioids remain a cornerstone of postoperative pain management both within and outside of the VA.¹⁰

In 2018, to limit unnecessary prescribing of opioid pain medication, the total joint service at the VA Portland Health Care System (VAPHCS) in Oregon implemented the Minimizing Opioids after Joint Operation (MOJO) postoperative pain protocol. The goal of the protocol was to reduce opioid use following TKA. The objectives were to provide safe, appropriate analgesia while allowing early mobilization and discharge without a concomitant increase in readmissions or emergency department (ED) visits. The purpose of this retrospective chart review was to compare the efficacy of the MOJO protocol with our historical experience and report our preliminary results.

Methods

Institutional review board approval was obtained to retrospectively review the medical records of patients who had undergone TKA surgery during 2018 at VAPHCS. The MOJO protocol was composed of several simultaneous changes. The centerpiece of the new protocol was a drastic decrease in routine prescription of postoperative opioids (Table 1). Other changes included instructing patients to reduce the use of preoperative opioid pain medication 6 weeks before surgery with a goal of no opioid consumption, perform daily sets of preoperative exercises, and attend a preoperative consultation/education session with a nurse coordinator to emphasize early recovery and discharge. In patients with chronic use of opioid pain medication (particularly those for whom the medication had been prescribed for other sources of pain, such as lumbar back pain), the goal was daily opioid use of ≤ 30 morphine equivalent doses (MEDs). During the inpatient stay, we stopped prescribing prophylactic pain medication prior to physical therapy (PT).

We encouraged preoperative optimization of muscle strength by giving instructions for 4 to 8 weeks of daily exercises (Appendix). We introduced perioperative adductor canal blocks (at the discretion of the anesthesia team) and transitioned to surgery without a tourniquet. Patients in both groups received intraoperative antibiotics and IV tranexamic acid (TXA); the MOJO group also received topical TXA.

Further patient care optimization included providing patients with a team-based approach, which consisted of nurse coordinators, physician assistants and nurse practitioners, residents, and the attending surgeon. Our team reviews the planned pain management protocol, perioperative expectations, criteria for discharge, and anticipated surgical outcomes with the patient during their preoperative visits. On postoperative day 1, these members round as a team to encourage patients in their immediate postoperative recovery and rehabilitation. During rounds, the team assesses whether the patient meets the criteria for discharge, adjusting the pain management protocol if necessary.

Changes in surgical technique included arthrotomy with electrocautery, minimizing traumatic dissection or resection of the synovial tissue, and intra-articular injection of a cocktail of ropivacaine 5 mg/mL 40 mL, epinephrine 1:1,000 0.5 mL, and methylprednisolone sodium 40 mg diluted with normal saline to a total volume of 120 mL.

The new routine was gradually implemented beginning January 2017 and fully implemented by July 2018. This study compared the first 20 consecutive patients undergoing primary TKA after July 2018 to the last 20 consecutive patients undergoing primary TKA prior to January 2017. Exclusion criteria included bilateral TKA, death before 90 days, and revision as the indication for surgery. The senior attending surgeon performed all surgeries using a standard midline approach. The majority of surgeries were performed using a cemented Vanguard total knee system (Zimmer Biomet); 4 patients in the historical group had a NexGen knee system, cementless monoblock tibial components (Zimmer Biomet); and 1 patient had a Logic knee system (Exactech). Surgical selection criteria for patients did not differ between groups.

 

Electronic health records were reviewed and data were abstracted. The data included demographic information (age, gender, body mass index [BMI], diagnosis, and procedure), surgical factors (American Society of Anesthesiologists score, Risk Assessment and Predictive Tool score, operative time, tourniquet time, estimated blood loss), hospital factors (length of stay [LOS], discharge location), postoperative pain scores (measured on postoperative day 1 and on day of discharge), and postdischarge events (90-day complications, telephone calls reporting pain, reoperations, returns to the ED, 90-day readmissions).

The primary outcome was the mean postoperative daily MED during the inpatient stay. Secondary outcomes included pain on postoperative day 1, pain at the time of discharge, LOS, hospital readmissions, and ED visits within 90 days of surgery. Because different opioid pain medications were used by patients postoperatively, all opioids were converted to MED prior to the final analysis. Collected patient data were de-identified prior to analysis.

Power analysis was conducted to determine whether the study had sufficient population size to reject the null hypothesis for the primary outcome measure. Because practitioners controlled postoperative opioid use, a Cohen’s d of 1.0 was used so that a very large effect size was needed to reach clinical significance. Statistical significance was set to 0.05, and patient groups were set at 20 patients each. This yielded an appropriate power of 0.87. Population characteristics were compared between groups using t tests and χ² tests as appropriate. To analyze the primary outcome, comparisons were made between the 2 cohorts using 2-tailed t tests. Secondary outcomes were compared between groups using t tests or χ² tests. All statistics were performed using R version 3.5.2. Power analysis was conducted using the package pwr.¹¹ Statistical significance was set at P < .05.

Results

Forty patients met the inclusion criteria, evenly divided between those undergoing TKA before and after instituting the MOJO protocol (Table 2). A single patient in the MOJO group died and was excluded. A patient who underwent bilateral TKA also was excluded. Both groups reflected the male predominance of the VA patient population. MOJO patients tended to have lower BMIs (34 vs 30, P < .01). All patients indicated for surgery with preoperative opioid use were able to titrate down to their preoperative goal as verified by prescriptions filled at VA pharmacies. Twelve of the patients in the MOJO group received adductor canal blocks.

Results of t tests and χ² tests comparing primary and secondary endpoints are listed in Table 3. Differences between the daily MEDs given in the historical and MOJO groups are shown. There were significant differences between the pre-MOJO and MOJO groups with regard to daily inpatient MEDs (82 mg vs 29 mg, P < .01) and total inpatient MEDs (306 mg vs 32 mg, P < .01). There was less self-reported pain on postoperative day 1 in the MOJO group (5.5 vs 3.9, P < .01), decreased LOS (4.4 days vs 1.2 days, P < .01), a trend toward fewer total ED visits (6 vs 2, P = .24), and fewer discharges to skilled nursing facilities (12 vs 0, P < .01). There were no blood transfusions in either group.

There were no readmissions due to uncontrolled pain. There was 1 readmission for shortness of breath in the MOJO group. The patient was discharged home the following day after ruling out thromboembolic and cardiovascular events. One patient from the control group was readmitted after missing a step on a staircase and falling. The patient sustained a quadriceps tendon rupture and underwent primary suture repair.

Discussion

Our results demonstrate that a multimodal approach to significantly reduce postoperative opioid use in patients with TKA is possible without increasing readmissions or ED visits for pain control. The patients in the MOJO group had a faster recovery, earlier discharge, and less use of postoperative opioid medication. Our approach to postoperative pain management was divided into 2 main categories: patient optimization and surgical optimization.

Patient Selection

Besides the standard evaluation and optimization of patients’ medical conditions, identifying and optimizing at-risk patients before surgery was a critical component of our protocol. Managing postoperative pain in patients with prior opioid use is an intractable challenge in orthopedic surgery. Patients with a history of chronic pain and preoperative use of opioid medications remain at higher risk of postoperative chronic pain and persistent use of opioid medication despite no obvious surgical complications.⁸ In a sample of > 6,000 veterans who underwent TKA at VA hospitals in 2014, 57% of the patients with daily use of opioids in the 90 days before surgery remained on opioids 1 year after surgery (vs 2 % in patients not on long-term opioids).⁸ This relationship between pre- and postoperative opioid use also was dose dependent.¹²

 

Furthermore, those with high preoperative use may experience worse outcomes relative to the opioid naive population as measured by arthritis-specific pain indices.¹³ In a well-powered retrospective study of patients who underwent elective orthopedic procedures, preoperative opioid abuse or dependence (determined by the International Classification of Diseases, Ninth Revision diagnosis) increased inpatient mortality, aggregate morbidity, surgical site infection, myocardial infarction, and LOS.¹⁴ Preoperative opioid use also has been associated with increased risk of ED visits, readmission, infection, stiffness, and aseptic revision.¹⁵ In patients with TKA in the VA specifically, preoperative opioid use (> 3 months in the prior year) was associated with increased revision rates that were even higher than those for patients with diabetes mellitus.¹⁶

Patient Education

Based on this evidence, we instruct patients to reduce their preoperative opioid dosing to zero (for patients with joint pain) or < 30 MED (for patients using opioids for other reasons). Although preoperative reduction of opioid use has been shown to improve outcomes after TKA, pain subspecialty recommendations for patients with chronic opioid use recommend considering adjunctive therapies, including transcutaneous electrical nerve stimulation, cognitive behavioral therapy, gabapentin, or ketamine.^17,18 Through patient education our team has been successful in decreasing preoperative opioid use without adding other drugs or modalities.

Patient Optimization

Preoperative patient optimization included 4 to 8 weeks of daily sets of physical activity instructions (prehab) to improve the musculoskeletal function. These instructions are given to patients 4 to 8 weeks before surgery and aim to improve the patient’s balance, mobility, and functional ability (Appendix). Meta-analysis has shown that patients who undergo preoperative PT have a small but statistically significant decrease in postoperative pain at 4 weeks, though this does not persist beyond that period.¹⁹

We did note a lower BMI in patients in the MOJO group. Though this has the potential to be a confounder, a study of BMI in > 4,000 patients who underwent joint replacement surgery has shown that BMI is not associated with differences in postoperative pain.²⁰

Surgeon and Surgical-Related Variables

Patients in the MOJO group had increased use of adductor canal blocks. A 2017 meta-analysis of 12,530 patients comparing analgesic modalities found that peripheral nerve blocks targeting multiple nerves (eg, femoral/sciatic) decreased pain at rest, decreased opioid consumption, and improved range of motion postoperatively.²¹ Also, these were found to be superior to single nerve blocks, periarticular infiltration, and epidural blocks.²¹ However, major nerve and epidural blocks affecting the lower extremity may increase the risk of falls and prolong LOS.^22,23 The preferred peripheral block at VAPHCS is a single shot ultrasound-guided adductor canal block before the induction of general or spinal anesthesia. A randomized controlled trial has demonstrated superiority of this block to the femoral nerve block with regard to postoperative quadriceps strength, conferring the theoretical advantage of decreased fall risk and ability to participate in immediate PT.²⁴ Although we are unable to confirm an association between anesthetic modalities and opioid burden, our clinical impression is that blocks were effective at reducing immediate postoperative pain. However, among MOJO patients there were no differences in patients with and without blocks for either pain (4.2 vs 3.8, P = .69) or opioid consumption (28.8 vs 33.0, P = .72) after surgery, though our study was not powered to detect a difference in this restricted subgroup.

Patients who frequently had reported postoperative thigh pain prompted us to make changes in our surgical technique, performing TKA without use of a tourniquet. Tourniquet use has been associated with an increased risk of thigh pain after TKA by multiple authors.^25,26 Postoperative thigh pain also is pressure dependent.²⁷ In addition, its use may be associated with a slightly increased risk of thromboembolic events and delayed functional recovery.^28,29

Because postoperative hemarthrosis is associated with more pain and reduced joint recovery function, we used topical TXA to reduce postoperative surgical site and joint hematoma. TXA (either oral, IV, or topical) during TKA is used to control postoperative bleeding primarily and decrease the need for transfusion without concomitant increase in thromboembolic events.^30,31 Topical TXA may be more effective than IV, particularly in the immediate postoperative period.³² Although pain typically is not an endpoint in studies of TXA, a prospective study of 48 patients showed evidence that its use may be associated with decreased postoperative pain in the first 24 hours after surgery (though not after).³³ Finally, the use of intra-articular injection has evolved in our clinical practice, but literature is lacking with regard to its efficacy; more studies are needed to determine its effect relative to no injection. We have not seen any benefits to using cryotherapy in our practice; considering the costs for equipment and health care provider time, cryotherapy was not included in our new protocol.

Limitations

This is a nonrandomized retrospective single-institution study. Our study population is composed of mostly males with military experience and is not necessarily a representative sample of the general population eligible for joint arthroplasty. Our primary endpoint (reduction of opioid use postoperatively) also was a cornerstone of our intervention. To account for this, we set a very large effect size in our power analysis and evaluated multiple secondary endpoints to determine whether postoperative pain remained well controlled and complications/readmission minimized with our interventions. Because our intervention was multimodal, our study cannot make conclusions about the effect of a particular component of our treatment strategy. We did not measure or compare functional outcomes between both groups, which offers an opportunity for further research.

 

These limitations are balanced by several strengths. Our cohort was well controlled with respect to the dose and type of drug used. There is staff dedicated to postoperative telephone follow-up after discharge, and veterans are apt to seek care within the VA health care system, which improves case finding for complications and ED visits. No patients were lost to follow-up. Moreover, our drastic reduction in opioid use is promising enough to warrant reporting, while the broader orthopedic literature explores the relative impact of each variable.

Conclusions

The MOJO protocol has been effective for reducing postoperative opioid use after TKA without compromising effective pain management. The drastic reduction in the postoperative use of opioid pain medications and LOS have contributed to a cultural shift within our department, comprehensive team approach, multimodal pain management, and preoperative patient optimization. Further investigations are required to assess the impact of each intervention on observed outcomes. However, the framework and routines are applicable to other institutions and surgical specialties.

Acknowledgments

The authors recognize Derek Bond, MD, for his help in creating the MOJO acronym.

For decades, opioids have been a mainstay in the management of pain after total joint arthroplasty. In the past 10 years, however, opioid prescribing has come under increased scrutiny due to a rise in rates of opioid abuse, pill diversion, and opioid-related deaths.^1,2 Opioids are associated with adverse effects, including nausea, vomiting, constipation, apathy, and respiratory depression, all of which influence arthroplasty outcomes and affect the patient experience. Although primary care groups account for nearly half of prescriptions written, orthopedic surgeons have the third highest per capita rate of opioid prescribing of all medical specialties.^3,4 This puts orthopedic surgeons, particularly those who perform routine procedures, in an opportune but challenging position to confront this problem through novel pain management strategies.

Approximately 1 million total knee arthroplasties (TKAs) are performed in the US every year, and the US Department of Veterans Affairs (VA) health system performs about 10,000 hip and knee joint replacements.^5,6 There is no standardization of opioid prescribing in the postoperative period following these procedures, and studies have reported a wide variation in prescribing habits even within a single institution for a specific surgery.⁷ Patients who undergo TKA are at particularly high risk of long-term opioid use if they are on continuous opioids at the time of surgery; this is problematic in a VA patient population in which at least 16% of patients are prescribed opioids in a given year.⁸ Furthermore, veterans are twice as likely as nonveterans to die of an accidental overdose.⁹ Despite these risks, opioids remain a cornerstone of postoperative pain management both within and outside of the VA.¹⁰

In 2018, to limit unnecessary prescribing of opioid pain medication, the total joint service at the VA Portland Health Care System (VAPHCS) in Oregon implemented the Minimizing Opioids after Joint Operation (MOJO) postoperative pain protocol. The goal of the protocol was to reduce opioid use following TKA. The objectives were to provide safe, appropriate analgesia while allowing early mobilization and discharge without a concomitant increase in readmissions or emergency department (ED) visits. The purpose of this retrospective chart review was to compare the efficacy of the MOJO protocol with our historical experience and report our preliminary results.

Methods

Institutional review board approval was obtained to retrospectively review the medical records of patients who had undergone TKA surgery during 2018 at VAPHCS. The MOJO protocol was composed of several simultaneous changes. The centerpiece of the new protocol was a drastic decrease in routine prescription of postoperative opioids (Table 1). Other changes included instructing patients to reduce the use of preoperative opioid pain medication 6 weeks before surgery with a goal of no opioid consumption, perform daily sets of preoperative exercises, and attend a preoperative consultation/education session with a nurse coordinator to emphasize early recovery and discharge. In patients with chronic use of opioid pain medication (particularly those for whom the medication had been prescribed for other sources of pain, such as lumbar back pain), the goal was daily opioid use of ≤ 30 morphine equivalent doses (MEDs). During the inpatient stay, we stopped prescribing prophylactic pain medication prior to physical therapy (PT).

We encouraged preoperative optimization of muscle strength by giving instructions for 4 to 8 weeks of daily exercises (Appendix). We introduced perioperative adductor canal blocks (at the discretion of the anesthesia team) and transitioned to surgery without a tourniquet. Patients in both groups received intraoperative antibiotics and IV tranexamic acid (TXA); the MOJO group also received topical TXA.

Further patient care optimization included providing patients with a team-based approach, which consisted of nurse coordinators, physician assistants and nurse practitioners, residents, and the attending surgeon. Our team reviews the planned pain management protocol, perioperative expectations, criteria for discharge, and anticipated surgical outcomes with the patient during their preoperative visits. On postoperative day 1, these members round as a team to encourage patients in their immediate postoperative recovery and rehabilitation. During rounds, the team assesses whether the patient meets the criteria for discharge, adjusting the pain management protocol if necessary.

Changes in surgical technique included arthrotomy with electrocautery, minimizing traumatic dissection or resection of the synovial tissue, and intra-articular injection of a cocktail of ropivacaine 5 mg/mL 40 mL, epinephrine 1:1,000 0.5 mL, and methylprednisolone sodium 40 mg diluted with normal saline to a total volume of 120 mL.

The new routine was gradually implemented beginning January 2017 and fully implemented by July 2018. This study compared the first 20 consecutive patients undergoing primary TKA after July 2018 to the last 20 consecutive patients undergoing primary TKA prior to January 2017. Exclusion criteria included bilateral TKA, death before 90 days, and revision as the indication for surgery. The senior attending surgeon performed all surgeries using a standard midline approach. The majority of surgeries were performed using a cemented Vanguard total knee system (Zimmer Biomet); 4 patients in the historical group had a NexGen knee system, cementless monoblock tibial components (Zimmer Biomet); and 1 patient had a Logic knee system (Exactech). Surgical selection criteria for patients did not differ between groups.

 

Electronic health records were reviewed and data were abstracted. The data included demographic information (age, gender, body mass index [BMI], diagnosis, and procedure), surgical factors (American Society of Anesthesiologists score, Risk Assessment and Predictive Tool score, operative time, tourniquet time, estimated blood loss), hospital factors (length of stay [LOS], discharge location), postoperative pain scores (measured on postoperative day 1 and on day of discharge), and postdischarge events (90-day complications, telephone calls reporting pain, reoperations, returns to the ED, 90-day readmissions).

The primary outcome was the mean postoperative daily MED during the inpatient stay. Secondary outcomes included pain on postoperative day 1, pain at the time of discharge, LOS, hospital readmissions, and ED visits within 90 days of surgery. Because different opioid pain medications were used by patients postoperatively, all opioids were converted to MED prior to the final analysis. Collected patient data were de-identified prior to analysis.

Power analysis was conducted to determine whether the study had sufficient population size to reject the null hypothesis for the primary outcome measure. Because practitioners controlled postoperative opioid use, a Cohen’s d of 1.0 was used so that a very large effect size was needed to reach clinical significance. Statistical significance was set to 0.05, and patient groups were set at 20 patients each. This yielded an appropriate power of 0.87. Population characteristics were compared between groups using t tests and χ² tests as appropriate. To analyze the primary outcome, comparisons were made between the 2 cohorts using 2-tailed t tests. Secondary outcomes were compared between groups using t tests or χ² tests. All statistics were performed using R version 3.5.2. Power analysis was conducted using the package pwr.¹¹ Statistical significance was set at P < .05.

Results

Forty patients met the inclusion criteria, evenly divided between those undergoing TKA before and after instituting the MOJO protocol (Table 2). A single patient in the MOJO group died and was excluded. A patient who underwent bilateral TKA also was excluded. Both groups reflected the male predominance of the VA patient population. MOJO patients tended to have lower BMIs (34 vs 30, P < .01). All patients indicated for surgery with preoperative opioid use were able to titrate down to their preoperative goal as verified by prescriptions filled at VA pharmacies. Twelve of the patients in the MOJO group received adductor canal blocks.

Results of t tests and χ² tests comparing primary and secondary endpoints are listed in Table 3. Differences between the daily MEDs given in the historical and MOJO groups are shown. There were significant differences between the pre-MOJO and MOJO groups with regard to daily inpatient MEDs (82 mg vs 29 mg, P < .01) and total inpatient MEDs (306 mg vs 32 mg, P < .01). There was less self-reported pain on postoperative day 1 in the MOJO group (5.5 vs 3.9, P < .01), decreased LOS (4.4 days vs 1.2 days, P < .01), a trend toward fewer total ED visits (6 vs 2, P = .24), and fewer discharges to skilled nursing facilities (12 vs 0, P < .01). There were no blood transfusions in either group.

There were no readmissions due to uncontrolled pain. There was 1 readmission for shortness of breath in the MOJO group. The patient was discharged home the following day after ruling out thromboembolic and cardiovascular events. One patient from the control group was readmitted after missing a step on a staircase and falling. The patient sustained a quadriceps tendon rupture and underwent primary suture repair.

Discussion

Our results demonstrate that a multimodal approach to significantly reduce postoperative opioid use in patients with TKA is possible without increasing readmissions or ED visits for pain control. The patients in the MOJO group had a faster recovery, earlier discharge, and less use of postoperative opioid medication. Our approach to postoperative pain management was divided into 2 main categories: patient optimization and surgical optimization.

Patient Selection

Besides the standard evaluation and optimization of patients’ medical conditions, identifying and optimizing at-risk patients before surgery was a critical component of our protocol. Managing postoperative pain in patients with prior opioid use is an intractable challenge in orthopedic surgery. Patients with a history of chronic pain and preoperative use of opioid medications remain at higher risk of postoperative chronic pain and persistent use of opioid medication despite no obvious surgical complications.⁸ In a sample of > 6,000 veterans who underwent TKA at VA hospitals in 2014, 57% of the patients with daily use of opioids in the 90 days before surgery remained on opioids 1 year after surgery (vs 2 % in patients not on long-term opioids).⁸ This relationship between pre- and postoperative opioid use also was dose dependent.¹²

 

Furthermore, those with high preoperative use may experience worse outcomes relative to the opioid naive population as measured by arthritis-specific pain indices.¹³ In a well-powered retrospective study of patients who underwent elective orthopedic procedures, preoperative opioid abuse or dependence (determined by the International Classification of Diseases, Ninth Revision diagnosis) increased inpatient mortality, aggregate morbidity, surgical site infection, myocardial infarction, and LOS.¹⁴ Preoperative opioid use also has been associated with increased risk of ED visits, readmission, infection, stiffness, and aseptic revision.¹⁵ In patients with TKA in the VA specifically, preoperative opioid use (> 3 months in the prior year) was associated with increased revision rates that were even higher than those for patients with diabetes mellitus.¹⁶

Patient Education

Based on this evidence, we instruct patients to reduce their preoperative opioid dosing to zero (for patients with joint pain) or < 30 MED (for patients using opioids for other reasons). Although preoperative reduction of opioid use has been shown to improve outcomes after TKA, pain subspecialty recommendations for patients with chronic opioid use recommend considering adjunctive therapies, including transcutaneous electrical nerve stimulation, cognitive behavioral therapy, gabapentin, or ketamine.^17,18 Through patient education our team has been successful in decreasing preoperative opioid use without adding other drugs or modalities.

Patient Optimization

Preoperative patient optimization included 4 to 8 weeks of daily sets of physical activity instructions (prehab) to improve the musculoskeletal function. These instructions are given to patients 4 to 8 weeks before surgery and aim to improve the patient’s balance, mobility, and functional ability (Appendix). Meta-analysis has shown that patients who undergo preoperative PT have a small but statistically significant decrease in postoperative pain at 4 weeks, though this does not persist beyond that period.¹⁹

We did note a lower BMI in patients in the MOJO group. Though this has the potential to be a confounder, a study of BMI in > 4,000 patients who underwent joint replacement surgery has shown that BMI is not associated with differences in postoperative pain.²⁰

Surgeon and Surgical-Related Variables

Patients in the MOJO group had increased use of adductor canal blocks. A 2017 meta-analysis of 12,530 patients comparing analgesic modalities found that peripheral nerve blocks targeting multiple nerves (eg, femoral/sciatic) decreased pain at rest, decreased opioid consumption, and improved range of motion postoperatively.²¹ Also, these were found to be superior to single nerve blocks, periarticular infiltration, and epidural blocks.²¹ However, major nerve and epidural blocks affecting the lower extremity may increase the risk of falls and prolong LOS.^22,23 The preferred peripheral block at VAPHCS is a single shot ultrasound-guided adductor canal block before the induction of general or spinal anesthesia. A randomized controlled trial has demonstrated superiority of this block to the femoral nerve block with regard to postoperative quadriceps strength, conferring the theoretical advantage of decreased fall risk and ability to participate in immediate PT.²⁴ Although we are unable to confirm an association between anesthetic modalities and opioid burden, our clinical impression is that blocks were effective at reducing immediate postoperative pain. However, among MOJO patients there were no differences in patients with and without blocks for either pain (4.2 vs 3.8, P = .69) or opioid consumption (28.8 vs 33.0, P = .72) after surgery, though our study was not powered to detect a difference in this restricted subgroup.

Patients who frequently had reported postoperative thigh pain prompted us to make changes in our surgical technique, performing TKA without use of a tourniquet. Tourniquet use has been associated with an increased risk of thigh pain after TKA by multiple authors.^25,26 Postoperative thigh pain also is pressure dependent.²⁷ In addition, its use may be associated with a slightly increased risk of thromboembolic events and delayed functional recovery.^28,29

Because postoperative hemarthrosis is associated with more pain and reduced joint recovery function, we used topical TXA to reduce postoperative surgical site and joint hematoma. TXA (either oral, IV, or topical) during TKA is used to control postoperative bleeding primarily and decrease the need for transfusion without concomitant increase in thromboembolic events.^30,31 Topical TXA may be more effective than IV, particularly in the immediate postoperative period.³² Although pain typically is not an endpoint in studies of TXA, a prospective study of 48 patients showed evidence that its use may be associated with decreased postoperative pain in the first 24 hours after surgery (though not after).³³ Finally, the use of intra-articular injection has evolved in our clinical practice, but literature is lacking with regard to its efficacy; more studies are needed to determine its effect relative to no injection. We have not seen any benefits to using cryotherapy in our practice; considering the costs for equipment and health care provider time, cryotherapy was not included in our new protocol.

Limitations

This is a nonrandomized retrospective single-institution study. Our study population is composed of mostly males with military experience and is not necessarily a representative sample of the general population eligible for joint arthroplasty. Our primary endpoint (reduction of opioid use postoperatively) also was a cornerstone of our intervention. To account for this, we set a very large effect size in our power analysis and evaluated multiple secondary endpoints to determine whether postoperative pain remained well controlled and complications/readmission minimized with our interventions. Because our intervention was multimodal, our study cannot make conclusions about the effect of a particular component of our treatment strategy. We did not measure or compare functional outcomes between both groups, which offers an opportunity for further research.

 

These limitations are balanced by several strengths. Our cohort was well controlled with respect to the dose and type of drug used. There is staff dedicated to postoperative telephone follow-up after discharge, and veterans are apt to seek care within the VA health care system, which improves case finding for complications and ED visits. No patients were lost to follow-up. Moreover, our drastic reduction in opioid use is promising enough to warrant reporting, while the broader orthopedic literature explores the relative impact of each variable.

Conclusions

The MOJO protocol has been effective for reducing postoperative opioid use after TKA without compromising effective pain management. The drastic reduction in the postoperative use of opioid pain medications and LOS have contributed to a cultural shift within our department, comprehensive team approach, multimodal pain management, and preoperative patient optimization. Further investigations are required to assess the impact of each intervention on observed outcomes. However, the framework and routines are applicable to other institutions and surgical specialties.

Acknowledgments

The authors recognize Derek Bond, MD, for his help in creating the MOJO acronym.