Nader Najafi, MD

Elevated blood pressure (BP) is common among hospitalized adults, with prevalence estimates between 50% and 70%.¹ Many factors can cause or exacerbate BP elevations in the setting of acute illness, such as pain, anxiety, medication withdrawal, and volume status, among others.² While there are clear evidence-based recommendations for treating hypertension (HTN) in the ambulatory setting,³ guidelines for the management of elevated BP in the hospital are lacking.^4,5

Hypertensive crises are generally recognized as warranting rapid reduction in BP;^6-8 however, these represent the minority of cases.^9,10 Far more common in the hospital are patients with asymptomatic elevated BP, a population for which there is no high-quality evidence and no guidelines supporting the use of intravenous (IV) antihypertensives.^11,12 Treatment with such medications has been associated with highly variable clinical responses^13-15 and may result in adverse events, such as hypotension.¹⁰

To date, only a small number of studies have investigated the treatment of asymptomatic elevated BP among hospitalized adults.^10,13-15 These have suggested that IV antihypertensives are utilized frequently in this setting, often for only modestly elevated BPs; however, the studies have tended to be small, not racially diverse, and limited to noncritically ill patients. Furthermore, while it is generally accepted that reducing the use of IV antihypertensives among asymptomatic patients would have no adverse impact, to our knowledge there have been no published studies which have instituted such an initiative while measuring balancing outcomes.

The purpose of this study was to further the existing literature by defining the prevalence and effects of IV antihypertensive medication utilization among a medically complex, multiracial population of asymptomatic medical inpatients using a large electronic dataset and to evaluate the impact of a division-wide, two-tiered quality improvement (QI) initiative on the rates of IV antihypertensive utilization and patient outcomes.

Setting

The study was conducted at the University of California, San Francisco (UCSF), an 800-bed tertiary care, academic medical center. It was approved by the UCSF Institutional Review Board. General medicine patients at UCSF are distributed between teaching and direct-care (hospitalist) services. The intensive care unit (ICU) is “open,” meaning the medicine service acts as the primary team for all nonsurgical ICU patients. This study included all adult general medicine patients admitted to UCSF Medical Center between January 1, 2017 and March 1, 2018, including those in the ICU.

Study Population and Data Collection

The UCSF Medical Center uses the electronic health record (EHR) Epic (Epic 2017, Epic Systems Corporation, Verona, Wisconsin) for all clinical care. We obtained computerized EHR data from Clarity, the relational database that stores Epic’s inpatient data in thousands of tables, including orders, medications, laboratory and radiology results, vital signs, patient demographics, and notes. We identified all adult patients hospitalized on the general medicine service with ≥1 episode of elevated BP (>160/90 mm Hg) at any point during their hospitalization who were not on a vasopressor medication at the time of the vital sign recording.

We further identified all instances in which either IV labetalol or hydralazine were administered to these patients. These two agents were chosen because they are the only IV antihypertensives used commonly at our institution for the treatment of asymptomatic elevated BP among internal medicine patients. Only those orders placed by a general medicine provider or reconciled by a general medicine provider upon transfer from another service were included. For each medication administration timestamp, we collected vital signs before and after the administration, along with the ordering provider and the clinical indication that was documented in the electronic order. To determine if a medication was administered with concern for end-organ injury, we also extracted orders that could serve as a proxy for the provider’s clinical assessment—namely electrocardiograms, serum troponins, chest x-rays, and computerized tomography scans of the head—which were placed in the one hour preceding or 15 minutes following administration of an IV antihypertensive medication.

To assess for comorbid conditions, including a preexisting diagnosis of HTN, we collected International Classification of Diseases (ICD)-9/10 diagnosis codes. Further, we also extracted All Patient Refined Diagnosis-Related Group (APR-DRG) weights, which are a standardized measure of illness severity based on relative resource consumption during hospitalization.^16,17

Patients were categorized as having either “symptomatic” or “asymptomatic” elevated BP. We defined symptomatic elevated BP as having received treatment with an IV medication with provider concern for end-organ injury, as defined above. We further identified all patients in which tight BP control may be clinically indicated on the basis of the presence of any of the following ICD-9/10 diagnosis codes at the time of hospital discharge: myocardial infarction, ischemic stroke, intracranial hemorrhage, subarachnoid hemorrhage, subdural hematoma, aortic dissection, hypertensive emergency, or hypertensive encephalopathy. All patients with symptomatic elevated BP or any of the above ICD-9/10 diagnoses were excluded from the analysis, since administration of IV antihypertensive medications would plausibly be warranted in these clinical scenarios.

The encounter numbers from the dataset were used to link to patient demographic data, which included age, sex, race, ethnicity, primary language, and insurance status. Finally, we identified all instances of rapid response calls, ICU transfers, and code blues (cardiopulmonary arrests) for each patient in the dataset.

Blood Pressure Measurements

BP data were collected from invasive BP (IBP) monitoring devices and noninvasive BP cuffs. For patients with BP measurements recorded concomitantly from both IBP (ie, arterial lines) in addition to noninvasive BP cuffs, the arterial line reading was favored. All systolic BP (SBP) readings >240 mm Hg from arterial lines were excluded, as this has previously been described as the upper physiologic limit for IBP readings.¹⁸

Primary Outcome

The primary outcome for the study was the proportion of patients treated with IV antihypertensive medications (labetalol or hydralazine). Using aggregate data, we calculated the number of patients who were treated at least once with an IV antihypertensive in a given month (numerator), divided by the number of patients with ≥1 episode of asymptomatic elevated BP that month (denominator). The denominator was considered to be the population of patients “at risk” of being treated with IV antihypertensive medications. For patients with multiple admissions during the study period, each admission was considered separately. These results are displayed in the upper portion of the run chart (Figure).

Secondary Outcomes

To investigate blood pressure trends over time, we analyzed BP in three ways. First, we analyzed the median SBP for the entire population. Second, to determine clinical responses to IV antihypertensive medications among patients receiving treatment, we calculated the population medians for the pretreatment SBP, the change in SBP from pretreatment baseline, and the posttreatment SBP. Third, we calculated the average median SBP on a monthly basis for the duration of the study. This was achieved by calculating the median value of all SBPs for an individual patient, then averaging across all patients in a given month. The average monthly median SBPs are displayed in the lower portion of the Figure.

To investigate whether the intervention was associated with negative patient outcomes, the proportions of several balancing outcomes were compared between pre- and postintervention periods, including ICU transfers, rapid response calls, and code blues (cardiopulmonary arrests).

Development and Implementation of an Intervention to Reduce Excessive IV Antihypertensive Use

After establishing the baseline prevalence of IV antihypertensive medication use at our institution, we developed a QI initiative with the goal of reducing IV antihypertensive medication utilization by the general medicine service for the treatment of asymptomatic patients. We hypothesized that potential contributors to overutilization might include lack of education, provider/nursing discomfort, and a system designed to mandate provider notification for even modestly elevated BPs. The QI initiative, which took place between October 2017 and December 2017, was designed to address these potential contributors and was comprised of a division-wide, two-tiered, bundled intervention. Our choice of a two-tiered approach was based on the fact that successful culture change is challenging, along with the existing evidence that multifaceted QI interventions are more often successful than single-tiered approaches.¹⁹

The first tier of the initiative included an educational campaign referred to colloquially as “NoIVForHighBP,” which targeted residents, hospitalists, and nursing staff. The campaign consisted of a series of presentations, best practice updates, handouts, and posters displayed prominently in shared workspaces. The educational content focused on alternative approaches to the management of asymptomatic elevated BP in the hospital, such as identification and treatment of pain, anxiety, volume overload, or other contributing factors (see supplemental materials). These educational outreaches occurred periodically between October 4, 2017 and November 20, 2017, with the bulk of the educational efforts taking place during November. Therefore, November 1, 2017 was designated the start date for the intervention period.

The second tier of the intervention included the liberalization of the EHR BP notification parameters on the standard inpatient admission order set from >160/90 mm Hg to >180/90 mm Hg. This change took effect on 12/6/2017. The decision to modify the BP notification parameters was based on the hypothesis that mandatory notifications for modestly elevated BPs may prompt providers to reflexively order IV antihypertensive medications, especially during times of cross-coverage or high clinical workload.

Statistical Analysis

All statistical analyses were performed using Stata software version 15 (StataCorp. 2017. Stata Statistical Software: Release 15. College Station, Texas: StataCorp LLC). Baseline patient characteristics were compared using nonparametric tests of significance. Population median SBPs were compared between pre- and postintervention periods using Mood’s Median Test, which was selected because the data were distributed nonnormally, and variances between samples were unequal.

Among patients treated with IV antihypertensive medications, we compared the proportion of pretreatment SBPs falling into each of three specified ranges (SBP <180 mm Hg, SBP 180-199 mm Hg, and SBP >200 mm Hg) between baseline and intervention periods using chi-squared tests.

Using aggregate data, we compared the unadjusted proportion of patients treated with IV antihypertensive medications between pre- and postintervention periods using a chi-squared test. Next, using patient-level data, a logistic regression analysis was performed to examine the association between receipt of IV antihypertensive medications and time (dichotomized between pre- and postintervention periods) while adjusting for age, sex, race, ethnicity, primary language, insurance status, preexisting HTN, length of stay, and APR-DRG weight.

Rates of balancing outcomes were compared using chi-squared tests. A logistic regression analysis using patient-level data was also performed to investigate the association between each of these outcomes and the intervention period (pre vs post) while adjusting for age, sex, race, ethnicity, primary language, insurance status, preexisting HTN, length of stay, and APR-DRG weight.

Baseline Period

We identified 2,306 patients with ≥1 episode of asymptomatic elevated BP during the 10-month preintervention period. Patients on average experienced 9 episodes of elevated BP per hospitalization, representing 21,207 potential opportunities for treatment. Baseline characteristics are summarized in Table 1. In general, this represents an older population that was medically complex and multiracial.

Of these patients, 251 (11%) received IV hydralazine and/or labetalol at least once during their hospitalization, with a total of 597 doses administered. Among those treated, a median of 2 doses were given per patient (IQR: 1-4), 64% of which were hydralazine. The majority (380 [64%]) were ordered on an “as needed” basis, while 217 (36%) were administered as a one-time dose. Three-quarters of all doses were ordered by the teaching service (456 [76%]), with the remaining 24% ordered by the direct-care (hospitalist) service.

During the baseline period among patients receiving IV antihypertensive medications, the median SBP of the population prior to treatment was 187 mm Hg (IQR 177-199; Table 2). Treatment was initiated in 30% of patients for an SBP <180 mm Hg and in 75% for an SBP <200 mm Hg. The median time to follow-up BP check was 34 minutes (IQR 15-58). The median decrement in SBP was 20 mm Hg (IQR 5-37); however, the response to treatment was highly variable, with 2% of patients experiencing no change and 14% experiencing an increase in SBP. Seventy-nine patients (14%) had a decrement in SBP >25% following treatment.

Description of Quality Improvement Results

Following the QI initiative, a total of 934 patients experienced 9,743 episodes of asymptomatic elevated blood pressure over a 4-month period (November 1, 2017 to February 28, 2018). As shown in Table 1, patients in the postintervention period had a slightly higher median age (67 [IQR 55-80] vs 69 [IQR 57-83]; P = .01), a higher median APR-DRG weight (1.34 [IQR 0.99-1.77] vs 1.48 [1.00-1.82]; P < .001), and a longer median length of stay (4.6 [2.8-8.0] days vs 5.1 [2.9-9.2] days; P = .004). There was also a higher proportion of nonEnglish speakers, fewer Black patients, and a lower proportion of preexisting HTN, in the postintervention period.

Of the 934 patients with ≥1 episode of asymptomatic elevated BP, 70 (7%) were treated with IV antihypertensive medications, with a total of 196 doses administered. The proportion of patients treated per month during the postintervention period ranged from 6% to 8%, which was the lowest of the entire study period and below the baseline average of 10% (Figure).

In a patient-level logistic regression pre-post analysis adjusting for age, sex, race, ethnicity, primary language, insurance status, preexisting HTN, length of stay, and APR-DRG weight, patients admitted to the general medicine service during the postintervention period had 38% lower odds of receiving IV antihypertensive medications than those admitted during the baseline period (OR = 0.62; 95% CI 0.47-0.83; P = .001). In this adjusted model, the following factors were independently associated with increased odds of receiving treatment: APR-DRG weight (OR 1.13; 95% CI 1.07-1.20; P < .001), Black race (OR 1.81; 95% CI 1.29-2.53; P = .001), length of stay (OR 1.02; 95% CI 1.01-1.03; P < .001), and preexisting HTN (OR 4.25; 95% CI 2.75-6.56; P < .001). Older age was associated with lower odds of treatment (Table 2).

Among patients who received treatment, there were no differences between pre- and postintervention periods in the proportion of pretreatment SBP <180 mm Hg (29% vs 32%; P = .40), 180-199 mm Hg (47% vs 40%; P = .10), or >200 mm Hg (25% vs 28%; P = .31; Table 3).

Our results suggest that treatment of asymptomatic elevated BP using IV antihypertensive medications is common practice at our institution. We found that treatment is often initiated for only modestly elevated BPs and that the clinical response to these medications is highly variable. In the baseline period, one in seven patients experienced a decrement in BP >25% following treatment, which could potentially cause harm.¹¹ There is no evidence, neither are there any consensus guidelines, to support the rapid reduction of BP among asymptomatic patients, making this a potential valuable opportunity for reducing unnecessary treatment, minimizing waste, and avoiding harm.

While there are a few previously published studies with similar results, we add to the existing literature by studying a larger population of more than 3,000 total patients, which was uniquely multiracial, including a high proportion of non-English speakers. Furthermore, our cohort included patients in the ICU, which is reflected in the higher-than-average APR-DRG weights. Despite being critically ill, these patients arguably still do not warrant aggressive treatment of elevated BP when asymptomatic. By excluding symptomatic BP elevations using surrogate markers for end-organ damage in addition to discharge diagnosis codes indicative of conditions in which tight BP control may be warranted, we were able to study a more critically ill patient population. We were also able to describe which baseline patient characteristics convey higher adjusted odds of receiving treatment, such as preexisting HTN, younger age, illness severity, and black race.

Perhaps most significantly, our study is the first to demonstrate an effective QI intervention aimed at reducing unnecessary utilization of IV antihypertensives. We found that this can feasibly be accomplished through a combination of educational efforts and systems changes, which could easily be replicated at other institutions. While the absolute reduction in the number of patients receiving treatment was modest, if these findings were to be widely accepted and resulted in a wide-spread change in culture, there would be a potential for greater impact.

Despite the reduction in the proportion of patients receiving IV antihypertensive medications, we found no change in the median SBP compared with the baseline period, which seems to support that the intervention was well tolerated. We also found no difference in the number of ICU transfers, rapid response calls, and cardiopulmonary arrests between groups. While these findings are both reassuring, it is impossible to draw definitive conclusions about safety given the small absolute number of patients having received treatment in each group. Fortunately, current guidelines and literature support the safety of such an intervention, as there is no existing evidence to suggest that failing to rapidly lower BP among asymptomatic patients is potentially harmful.¹¹

There are several limitations to our study. First, by utilizing a large electronic dataset, the quality of our analyses was reliant on the accuracy of the recorded EHR data. Second, in the absence of a controlled trial or control group, we cannot say definitively that our QI initiative was the direct cause of the improved rates of IV antihypertensive utilization, though the effect did persist after adjusting for baseline characteristics in patient-level models. Third, our follow-up period was relatively short, with fewer than half as many patients as in the preintervention period. This is an important limitation, since the impact of QI interventions often diminishes over time. We plan to continually monitor IV antihypertensive use, feed those data back to our group, and revitalize educational efforts should rates begin to rise. Fourth, we were unable to directly measure which patients had true end-organ injury and instead used orders placed around the time of medication administration as a surrogate marker. While this is an imperfect measure, we feel that in cases where a provider was concerned enough to even test for end-organ injury, the use of IV antihypertensives was likely justified and was therefore appropriately excluded from the analysis. Lastly, we were limited in our ability to describe associations with true clinical outcomes, such as stroke or myocardial infarction, which could theoretically be propagated by either the use or the avoidance of IV antihypertensive medications. Fortunately, based on clinical guidelines and existing evidence, there is no reason to believe that reducing IV antihypertensive use would result in increased rates of these outcomes.

Our study reaffirms the fact that overutilization of IV antihypertensive medications among asymptomatic hospitalized patients is pervasive across hospital systems. This represents a potential target for a concerted change in culture, which we have demonstrated can be feasibly accomplished through education and systems changes.

Disclosures

Dr. Auerbach has current or pending grants from the CDC, PCORI, and FDA that are unrelated to this research manuscript. He also receives royalties from UpToDate, and received an honorarium for being editor of JHM. Dr. Jacobs received a $1,000 Resident/Fellow Travel Grant from the Society of Hospital Medicine to support the cost of travel to SHM, where he presented this research as a poster in 2018. Dr. Prasad receives money from EpiExcellence, LLC for consultation, which is unrelated to this research manuscript. All other authors have nothing to disclose.

Elevated blood pressure (BP) is common among hospitalized adults, with prevalence estimates between 50% and 70%.¹ Many factors can cause or exacerbate BP elevations in the setting of acute illness, such as pain, anxiety, medication withdrawal, and volume status, among others.² While there are clear evidence-based recommendations for treating hypertension (HTN) in the ambulatory setting,³ guidelines for the management of elevated BP in the hospital are lacking.^4,5

Hypertensive crises are generally recognized as warranting rapid reduction in BP;^6-8 however, these represent the minority of cases.^9,10 Far more common in the hospital are patients with asymptomatic elevated BP, a population for which there is no high-quality evidence and no guidelines supporting the use of intravenous (IV) antihypertensives.^11,12 Treatment with such medications has been associated with highly variable clinical responses^13-15 and may result in adverse events, such as hypotension.¹⁰

To date, only a small number of studies have investigated the treatment of asymptomatic elevated BP among hospitalized adults.^10,13-15 These have suggested that IV antihypertensives are utilized frequently in this setting, often for only modestly elevated BPs; however, the studies have tended to be small, not racially diverse, and limited to noncritically ill patients. Furthermore, while it is generally accepted that reducing the use of IV antihypertensives among asymptomatic patients would have no adverse impact, to our knowledge there have been no published studies which have instituted such an initiative while measuring balancing outcomes.

The purpose of this study was to further the existing literature by defining the prevalence and effects of IV antihypertensive medication utilization among a medically complex, multiracial population of asymptomatic medical inpatients using a large electronic dataset and to evaluate the impact of a division-wide, two-tiered quality improvement (QI) initiative on the rates of IV antihypertensive utilization and patient outcomes.

Setting

The study was conducted at the University of California, San Francisco (UCSF), an 800-bed tertiary care, academic medical center. It was approved by the UCSF Institutional Review Board. General medicine patients at UCSF are distributed between teaching and direct-care (hospitalist) services. The intensive care unit (ICU) is “open,” meaning the medicine service acts as the primary team for all nonsurgical ICU patients. This study included all adult general medicine patients admitted to UCSF Medical Center between January 1, 2017 and March 1, 2018, including those in the ICU.

Study Population and Data Collection

The UCSF Medical Center uses the electronic health record (EHR) Epic (Epic 2017, Epic Systems Corporation, Verona, Wisconsin) for all clinical care. We obtained computerized EHR data from Clarity, the relational database that stores Epic’s inpatient data in thousands of tables, including orders, medications, laboratory and radiology results, vital signs, patient demographics, and notes. We identified all adult patients hospitalized on the general medicine service with ≥1 episode of elevated BP (>160/90 mm Hg) at any point during their hospitalization who were not on a vasopressor medication at the time of the vital sign recording.

We further identified all instances in which either IV labetalol or hydralazine were administered to these patients. These two agents were chosen because they are the only IV antihypertensives used commonly at our institution for the treatment of asymptomatic elevated BP among internal medicine patients. Only those orders placed by a general medicine provider or reconciled by a general medicine provider upon transfer from another service were included. For each medication administration timestamp, we collected vital signs before and after the administration, along with the ordering provider and the clinical indication that was documented in the electronic order. To determine if a medication was administered with concern for end-organ injury, we also extracted orders that could serve as a proxy for the provider’s clinical assessment—namely electrocardiograms, serum troponins, chest x-rays, and computerized tomography scans of the head—which were placed in the one hour preceding or 15 minutes following administration of an IV antihypertensive medication.

To assess for comorbid conditions, including a preexisting diagnosis of HTN, we collected International Classification of Diseases (ICD)-9/10 diagnosis codes. Further, we also extracted All Patient Refined Diagnosis-Related Group (APR-DRG) weights, which are a standardized measure of illness severity based on relative resource consumption during hospitalization.^16,17

Patients were categorized as having either “symptomatic” or “asymptomatic” elevated BP. We defined symptomatic elevated BP as having received treatment with an IV medication with provider concern for end-organ injury, as defined above. We further identified all patients in which tight BP control may be clinically indicated on the basis of the presence of any of the following ICD-9/10 diagnosis codes at the time of hospital discharge: myocardial infarction, ischemic stroke, intracranial hemorrhage, subarachnoid hemorrhage, subdural hematoma, aortic dissection, hypertensive emergency, or hypertensive encephalopathy. All patients with symptomatic elevated BP or any of the above ICD-9/10 diagnoses were excluded from the analysis, since administration of IV antihypertensive medications would plausibly be warranted in these clinical scenarios.

The encounter numbers from the dataset were used to link to patient demographic data, which included age, sex, race, ethnicity, primary language, and insurance status. Finally, we identified all instances of rapid response calls, ICU transfers, and code blues (cardiopulmonary arrests) for each patient in the dataset.

Blood Pressure Measurements

BP data were collected from invasive BP (IBP) monitoring devices and noninvasive BP cuffs. For patients with BP measurements recorded concomitantly from both IBP (ie, arterial lines) in addition to noninvasive BP cuffs, the arterial line reading was favored. All systolic BP (SBP) readings >240 mm Hg from arterial lines were excluded, as this has previously been described as the upper physiologic limit for IBP readings.¹⁸

Primary Outcome

The primary outcome for the study was the proportion of patients treated with IV antihypertensive medications (labetalol or hydralazine). Using aggregate data, we calculated the number of patients who were treated at least once with an IV antihypertensive in a given month (numerator), divided by the number of patients with ≥1 episode of asymptomatic elevated BP that month (denominator). The denominator was considered to be the population of patients “at risk” of being treated with IV antihypertensive medications. For patients with multiple admissions during the study period, each admission was considered separately. These results are displayed in the upper portion of the run chart (Figure).

Secondary Outcomes

To investigate blood pressure trends over time, we analyzed BP in three ways. First, we analyzed the median SBP for the entire population. Second, to determine clinical responses to IV antihypertensive medications among patients receiving treatment, we calculated the population medians for the pretreatment SBP, the change in SBP from pretreatment baseline, and the posttreatment SBP. Third, we calculated the average median SBP on a monthly basis for the duration of the study. This was achieved by calculating the median value of all SBPs for an individual patient, then averaging across all patients in a given month. The average monthly median SBPs are displayed in the lower portion of the Figure.

To investigate whether the intervention was associated with negative patient outcomes, the proportions of several balancing outcomes were compared between pre- and postintervention periods, including ICU transfers, rapid response calls, and code blues (cardiopulmonary arrests).

Development and Implementation of an Intervention to Reduce Excessive IV Antihypertensive Use

After establishing the baseline prevalence of IV antihypertensive medication use at our institution, we developed a QI initiative with the goal of reducing IV antihypertensive medication utilization by the general medicine service for the treatment of asymptomatic patients. We hypothesized that potential contributors to overutilization might include lack of education, provider/nursing discomfort, and a system designed to mandate provider notification for even modestly elevated BPs. The QI initiative, which took place between October 2017 and December 2017, was designed to address these potential contributors and was comprised of a division-wide, two-tiered, bundled intervention. Our choice of a two-tiered approach was based on the fact that successful culture change is challenging, along with the existing evidence that multifaceted QI interventions are more often successful than single-tiered approaches.¹⁹

The first tier of the initiative included an educational campaign referred to colloquially as “NoIVForHighBP,” which targeted residents, hospitalists, and nursing staff. The campaign consisted of a series of presentations, best practice updates, handouts, and posters displayed prominently in shared workspaces. The educational content focused on alternative approaches to the management of asymptomatic elevated BP in the hospital, such as identification and treatment of pain, anxiety, volume overload, or other contributing factors (see supplemental materials). These educational outreaches occurred periodically between October 4, 2017 and November 20, 2017, with the bulk of the educational efforts taking place during November. Therefore, November 1, 2017 was designated the start date for the intervention period.

The second tier of the intervention included the liberalization of the EHR BP notification parameters on the standard inpatient admission order set from >160/90 mm Hg to >180/90 mm Hg. This change took effect on 12/6/2017. The decision to modify the BP notification parameters was based on the hypothesis that mandatory notifications for modestly elevated BPs may prompt providers to reflexively order IV antihypertensive medications, especially during times of cross-coverage or high clinical workload.

Statistical Analysis

All statistical analyses were performed using Stata software version 15 (StataCorp. 2017. Stata Statistical Software: Release 15. College Station, Texas: StataCorp LLC). Baseline patient characteristics were compared using nonparametric tests of significance. Population median SBPs were compared between pre- and postintervention periods using Mood’s Median Test, which was selected because the data were distributed nonnormally, and variances between samples were unequal.

Among patients treated with IV antihypertensive medications, we compared the proportion of pretreatment SBPs falling into each of three specified ranges (SBP <180 mm Hg, SBP 180-199 mm Hg, and SBP >200 mm Hg) between baseline and intervention periods using chi-squared tests.

Using aggregate data, we compared the unadjusted proportion of patients treated with IV antihypertensive medications between pre- and postintervention periods using a chi-squared test. Next, using patient-level data, a logistic regression analysis was performed to examine the association between receipt of IV antihypertensive medications and time (dichotomized between pre- and postintervention periods) while adjusting for age, sex, race, ethnicity, primary language, insurance status, preexisting HTN, length of stay, and APR-DRG weight.

Rates of balancing outcomes were compared using chi-squared tests. A logistic regression analysis using patient-level data was also performed to investigate the association between each of these outcomes and the intervention period (pre vs post) while adjusting for age, sex, race, ethnicity, primary language, insurance status, preexisting HTN, length of stay, and APR-DRG weight.

Baseline Period

We identified 2,306 patients with ≥1 episode of asymptomatic elevated BP during the 10-month preintervention period. Patients on average experienced 9 episodes of elevated BP per hospitalization, representing 21,207 potential opportunities for treatment. Baseline characteristics are summarized in Table 1. In general, this represents an older population that was medically complex and multiracial.

Of these patients, 251 (11%) received IV hydralazine and/or labetalol at least once during their hospitalization, with a total of 597 doses administered. Among those treated, a median of 2 doses were given per patient (IQR: 1-4), 64% of which were hydralazine. The majority (380 [64%]) were ordered on an “as needed” basis, while 217 (36%) were administered as a one-time dose. Three-quarters of all doses were ordered by the teaching service (456 [76%]), with the remaining 24% ordered by the direct-care (hospitalist) service.

During the baseline period among patients receiving IV antihypertensive medications, the median SBP of the population prior to treatment was 187 mm Hg (IQR 177-199; Table 2). Treatment was initiated in 30% of patients for an SBP <180 mm Hg and in 75% for an SBP <200 mm Hg. The median time to follow-up BP check was 34 minutes (IQR 15-58). The median decrement in SBP was 20 mm Hg (IQR 5-37); however, the response to treatment was highly variable, with 2% of patients experiencing no change and 14% experiencing an increase in SBP. Seventy-nine patients (14%) had a decrement in SBP >25% following treatment.

Description of Quality Improvement Results

Following the QI initiative, a total of 934 patients experienced 9,743 episodes of asymptomatic elevated blood pressure over a 4-month period (November 1, 2017 to February 28, 2018). As shown in Table 1, patients in the postintervention period had a slightly higher median age (67 [IQR 55-80] vs 69 [IQR 57-83]; P = .01), a higher median APR-DRG weight (1.34 [IQR 0.99-1.77] vs 1.48 [1.00-1.82]; P < .001), and a longer median length of stay (4.6 [2.8-8.0] days vs 5.1 [2.9-9.2] days; P = .004). There was also a higher proportion of nonEnglish speakers, fewer Black patients, and a lower proportion of preexisting HTN, in the postintervention period.

Of the 934 patients with ≥1 episode of asymptomatic elevated BP, 70 (7%) were treated with IV antihypertensive medications, with a total of 196 doses administered. The proportion of patients treated per month during the postintervention period ranged from 6% to 8%, which was the lowest of the entire study period and below the baseline average of 10% (Figure).

In a patient-level logistic regression pre-post analysis adjusting for age, sex, race, ethnicity, primary language, insurance status, preexisting HTN, length of stay, and APR-DRG weight, patients admitted to the general medicine service during the postintervention period had 38% lower odds of receiving IV antihypertensive medications than those admitted during the baseline period (OR = 0.62; 95% CI 0.47-0.83; P = .001). In this adjusted model, the following factors were independently associated with increased odds of receiving treatment: APR-DRG weight (OR 1.13; 95% CI 1.07-1.20; P < .001), Black race (OR 1.81; 95% CI 1.29-2.53; P = .001), length of stay (OR 1.02; 95% CI 1.01-1.03; P < .001), and preexisting HTN (OR 4.25; 95% CI 2.75-6.56; P < .001). Older age was associated with lower odds of treatment (Table 2).

Among patients who received treatment, there were no differences between pre- and postintervention periods in the proportion of pretreatment SBP <180 mm Hg (29% vs 32%; P = .40), 180-199 mm Hg (47% vs 40%; P = .10), or >200 mm Hg (25% vs 28%; P = .31; Table 3).

Our results suggest that treatment of asymptomatic elevated BP using IV antihypertensive medications is common practice at our institution. We found that treatment is often initiated for only modestly elevated BPs and that the clinical response to these medications is highly variable. In the baseline period, one in seven patients experienced a decrement in BP >25% following treatment, which could potentially cause harm.¹¹ There is no evidence, neither are there any consensus guidelines, to support the rapid reduction of BP among asymptomatic patients, making this a potential valuable opportunity for reducing unnecessary treatment, minimizing waste, and avoiding harm.

While there are a few previously published studies with similar results, we add to the existing literature by studying a larger population of more than 3,000 total patients, which was uniquely multiracial, including a high proportion of non-English speakers. Furthermore, our cohort included patients in the ICU, which is reflected in the higher-than-average APR-DRG weights. Despite being critically ill, these patients arguably still do not warrant aggressive treatment of elevated BP when asymptomatic. By excluding symptomatic BP elevations using surrogate markers for end-organ damage in addition to discharge diagnosis codes indicative of conditions in which tight BP control may be warranted, we were able to study a more critically ill patient population. We were also able to describe which baseline patient characteristics convey higher adjusted odds of receiving treatment, such as preexisting HTN, younger age, illness severity, and black race.

Perhaps most significantly, our study is the first to demonstrate an effective QI intervention aimed at reducing unnecessary utilization of IV antihypertensives. We found that this can feasibly be accomplished through a combination of educational efforts and systems changes, which could easily be replicated at other institutions. While the absolute reduction in the number of patients receiving treatment was modest, if these findings were to be widely accepted and resulted in a wide-spread change in culture, there would be a potential for greater impact.

Despite the reduction in the proportion of patients receiving IV antihypertensive medications, we found no change in the median SBP compared with the baseline period, which seems to support that the intervention was well tolerated. We also found no difference in the number of ICU transfers, rapid response calls, and cardiopulmonary arrests between groups. While these findings are both reassuring, it is impossible to draw definitive conclusions about safety given the small absolute number of patients having received treatment in each group. Fortunately, current guidelines and literature support the safety of such an intervention, as there is no existing evidence to suggest that failing to rapidly lower BP among asymptomatic patients is potentially harmful.¹¹

There are several limitations to our study. First, by utilizing a large electronic dataset, the quality of our analyses was reliant on the accuracy of the recorded EHR data. Second, in the absence of a controlled trial or control group, we cannot say definitively that our QI initiative was the direct cause of the improved rates of IV antihypertensive utilization, though the effect did persist after adjusting for baseline characteristics in patient-level models. Third, our follow-up period was relatively short, with fewer than half as many patients as in the preintervention period. This is an important limitation, since the impact of QI interventions often diminishes over time. We plan to continually monitor IV antihypertensive use, feed those data back to our group, and revitalize educational efforts should rates begin to rise. Fourth, we were unable to directly measure which patients had true end-organ injury and instead used orders placed around the time of medication administration as a surrogate marker. While this is an imperfect measure, we feel that in cases where a provider was concerned enough to even test for end-organ injury, the use of IV antihypertensives was likely justified and was therefore appropriately excluded from the analysis. Lastly, we were limited in our ability to describe associations with true clinical outcomes, such as stroke or myocardial infarction, which could theoretically be propagated by either the use or the avoidance of IV antihypertensive medications. Fortunately, based on clinical guidelines and existing evidence, there is no reason to believe that reducing IV antihypertensive use would result in increased rates of these outcomes.

Our study reaffirms the fact that overutilization of IV antihypertensive medications among asymptomatic hospitalized patients is pervasive across hospital systems. This represents a potential target for a concerted change in culture, which we have demonstrated can be feasibly accomplished through education and systems changes.

Disclosures

Dr. Auerbach has current or pending grants from the CDC, PCORI, and FDA that are unrelated to this research manuscript. He also receives royalties from UpToDate, and received an honorarium for being editor of JHM. Dr. Jacobs received a $1,000 Resident/Fellow Travel Grant from the Society of Hospital Medicine to support the cost of travel to SHM, where he presented this research as a poster in 2018. Dr. Prasad receives money from EpiExcellence, LLC for consultation, which is unrelated to this research manuscript. All other authors have nothing to disclose.

Elevated blood pressure (BP) is common among hospitalized adults, with prevalence estimates between 50% and 70%.¹ Many factors can cause or exacerbate BP elevations in the setting of acute illness, such as pain, anxiety, medication withdrawal, and volume status, among others.² While there are clear evidence-based recommendations for treating hypertension (HTN) in the ambulatory setting,³ guidelines for the management of elevated BP in the hospital are lacking.^4,5

Hypertensive crises are generally recognized as warranting rapid reduction in BP;^6-8 however, these represent the minority of cases.^9,10 Far more common in the hospital are patients with asymptomatic elevated BP, a population for which there is no high-quality evidence and no guidelines supporting the use of intravenous (IV) antihypertensives.^11,12 Treatment with such medications has been associated with highly variable clinical responses^13-15 and may result in adverse events, such as hypotension.¹⁰

To date, only a small number of studies have investigated the treatment of asymptomatic elevated BP among hospitalized adults.^10,13-15 These have suggested that IV antihypertensives are utilized frequently in this setting, often for only modestly elevated BPs; however, the studies have tended to be small, not racially diverse, and limited to noncritically ill patients. Furthermore, while it is generally accepted that reducing the use of IV antihypertensives among asymptomatic patients would have no adverse impact, to our knowledge there have been no published studies which have instituted such an initiative while measuring balancing outcomes.

The purpose of this study was to further the existing literature by defining the prevalence and effects of IV antihypertensive medication utilization among a medically complex, multiracial population of asymptomatic medical inpatients using a large electronic dataset and to evaluate the impact of a division-wide, two-tiered quality improvement (QI) initiative on the rates of IV antihypertensive utilization and patient outcomes.

Setting

The study was conducted at the University of California, San Francisco (UCSF), an 800-bed tertiary care, academic medical center. It was approved by the UCSF Institutional Review Board. General medicine patients at UCSF are distributed between teaching and direct-care (hospitalist) services. The intensive care unit (ICU) is “open,” meaning the medicine service acts as the primary team for all nonsurgical ICU patients. This study included all adult general medicine patients admitted to UCSF Medical Center between January 1, 2017 and March 1, 2018, including those in the ICU.

Study Population and Data Collection

The UCSF Medical Center uses the electronic health record (EHR) Epic (Epic 2017, Epic Systems Corporation, Verona, Wisconsin) for all clinical care. We obtained computerized EHR data from Clarity, the relational database that stores Epic’s inpatient data in thousands of tables, including orders, medications, laboratory and radiology results, vital signs, patient demographics, and notes. We identified all adult patients hospitalized on the general medicine service with ≥1 episode of elevated BP (>160/90 mm Hg) at any point during their hospitalization who were not on a vasopressor medication at the time of the vital sign recording.

We further identified all instances in which either IV labetalol or hydralazine were administered to these patients. These two agents were chosen because they are the only IV antihypertensives used commonly at our institution for the treatment of asymptomatic elevated BP among internal medicine patients. Only those orders placed by a general medicine provider or reconciled by a general medicine provider upon transfer from another service were included. For each medication administration timestamp, we collected vital signs before and after the administration, along with the ordering provider and the clinical indication that was documented in the electronic order. To determine if a medication was administered with concern for end-organ injury, we also extracted orders that could serve as a proxy for the provider’s clinical assessment—namely electrocardiograms, serum troponins, chest x-rays, and computerized tomography scans of the head—which were placed in the one hour preceding or 15 minutes following administration of an IV antihypertensive medication.

To assess for comorbid conditions, including a preexisting diagnosis of HTN, we collected International Classification of Diseases (ICD)-9/10 diagnosis codes. Further, we also extracted All Patient Refined Diagnosis-Related Group (APR-DRG) weights, which are a standardized measure of illness severity based on relative resource consumption during hospitalization.^16,17

Patients were categorized as having either “symptomatic” or “asymptomatic” elevated BP. We defined symptomatic elevated BP as having received treatment with an IV medication with provider concern for end-organ injury, as defined above. We further identified all patients in which tight BP control may be clinically indicated on the basis of the presence of any of the following ICD-9/10 diagnosis codes at the time of hospital discharge: myocardial infarction, ischemic stroke, intracranial hemorrhage, subarachnoid hemorrhage, subdural hematoma, aortic dissection, hypertensive emergency, or hypertensive encephalopathy. All patients with symptomatic elevated BP or any of the above ICD-9/10 diagnoses were excluded from the analysis, since administration of IV antihypertensive medications would plausibly be warranted in these clinical scenarios.

The encounter numbers from the dataset were used to link to patient demographic data, which included age, sex, race, ethnicity, primary language, and insurance status. Finally, we identified all instances of rapid response calls, ICU transfers, and code blues (cardiopulmonary arrests) for each patient in the dataset.

Blood Pressure Measurements

BP data were collected from invasive BP (IBP) monitoring devices and noninvasive BP cuffs. For patients with BP measurements recorded concomitantly from both IBP (ie, arterial lines) in addition to noninvasive BP cuffs, the arterial line reading was favored. All systolic BP (SBP) readings >240 mm Hg from arterial lines were excluded, as this has previously been described as the upper physiologic limit for IBP readings.¹⁸

Primary Outcome

The primary outcome for the study was the proportion of patients treated with IV antihypertensive medications (labetalol or hydralazine). Using aggregate data, we calculated the number of patients who were treated at least once with an IV antihypertensive in a given month (numerator), divided by the number of patients with ≥1 episode of asymptomatic elevated BP that month (denominator). The denominator was considered to be the population of patients “at risk” of being treated with IV antihypertensive medications. For patients with multiple admissions during the study period, each admission was considered separately. These results are displayed in the upper portion of the run chart (Figure).

Secondary Outcomes

To investigate blood pressure trends over time, we analyzed BP in three ways. First, we analyzed the median SBP for the entire population. Second, to determine clinical responses to IV antihypertensive medications among patients receiving treatment, we calculated the population medians for the pretreatment SBP, the change in SBP from pretreatment baseline, and the posttreatment SBP. Third, we calculated the average median SBP on a monthly basis for the duration of the study. This was achieved by calculating the median value of all SBPs for an individual patient, then averaging across all patients in a given month. The average monthly median SBPs are displayed in the lower portion of the Figure.

To investigate whether the intervention was associated with negative patient outcomes, the proportions of several balancing outcomes were compared between pre- and postintervention periods, including ICU transfers, rapid response calls, and code blues (cardiopulmonary arrests).

Development and Implementation of an Intervention to Reduce Excessive IV Antihypertensive Use

After establishing the baseline prevalence of IV antihypertensive medication use at our institution, we developed a QI initiative with the goal of reducing IV antihypertensive medication utilization by the general medicine service for the treatment of asymptomatic patients. We hypothesized that potential contributors to overutilization might include lack of education, provider/nursing discomfort, and a system designed to mandate provider notification for even modestly elevated BPs. The QI initiative, which took place between October 2017 and December 2017, was designed to address these potential contributors and was comprised of a division-wide, two-tiered, bundled intervention. Our choice of a two-tiered approach was based on the fact that successful culture change is challenging, along with the existing evidence that multifaceted QI interventions are more often successful than single-tiered approaches.¹⁹

The first tier of the initiative included an educational campaign referred to colloquially as “NoIVForHighBP,” which targeted residents, hospitalists, and nursing staff. The campaign consisted of a series of presentations, best practice updates, handouts, and posters displayed prominently in shared workspaces. The educational content focused on alternative approaches to the management of asymptomatic elevated BP in the hospital, such as identification and treatment of pain, anxiety, volume overload, or other contributing factors (see supplemental materials). These educational outreaches occurred periodically between October 4, 2017 and November 20, 2017, with the bulk of the educational efforts taking place during November. Therefore, November 1, 2017 was designated the start date for the intervention period.

The second tier of the intervention included the liberalization of the EHR BP notification parameters on the standard inpatient admission order set from >160/90 mm Hg to >180/90 mm Hg. This change took effect on 12/6/2017. The decision to modify the BP notification parameters was based on the hypothesis that mandatory notifications for modestly elevated BPs may prompt providers to reflexively order IV antihypertensive medications, especially during times of cross-coverage or high clinical workload.

Statistical Analysis

All statistical analyses were performed using Stata software version 15 (StataCorp. 2017. Stata Statistical Software: Release 15. College Station, Texas: StataCorp LLC). Baseline patient characteristics were compared using nonparametric tests of significance. Population median SBPs were compared between pre- and postintervention periods using Mood’s Median Test, which was selected because the data were distributed nonnormally, and variances between samples were unequal.

Among patients treated with IV antihypertensive medications, we compared the proportion of pretreatment SBPs falling into each of three specified ranges (SBP <180 mm Hg, SBP 180-199 mm Hg, and SBP >200 mm Hg) between baseline and intervention periods using chi-squared tests.

Using aggregate data, we compared the unadjusted proportion of patients treated with IV antihypertensive medications between pre- and postintervention periods using a chi-squared test. Next, using patient-level data, a logistic regression analysis was performed to examine the association between receipt of IV antihypertensive medications and time (dichotomized between pre- and postintervention periods) while adjusting for age, sex, race, ethnicity, primary language, insurance status, preexisting HTN, length of stay, and APR-DRG weight.

Rates of balancing outcomes were compared using chi-squared tests. A logistic regression analysis using patient-level data was also performed to investigate the association between each of these outcomes and the intervention period (pre vs post) while adjusting for age, sex, race, ethnicity, primary language, insurance status, preexisting HTN, length of stay, and APR-DRG weight.

Baseline Period

We identified 2,306 patients with ≥1 episode of asymptomatic elevated BP during the 10-month preintervention period. Patients on average experienced 9 episodes of elevated BP per hospitalization, representing 21,207 potential opportunities for treatment. Baseline characteristics are summarized in Table 1. In general, this represents an older population that was medically complex and multiracial.

Of these patients, 251 (11%) received IV hydralazine and/or labetalol at least once during their hospitalization, with a total of 597 doses administered. Among those treated, a median of 2 doses were given per patient (IQR: 1-4), 64% of which were hydralazine. The majority (380 [64%]) were ordered on an “as needed” basis, while 217 (36%) were administered as a one-time dose. Three-quarters of all doses were ordered by the teaching service (456 [76%]), with the remaining 24% ordered by the direct-care (hospitalist) service.

During the baseline period among patients receiving IV antihypertensive medications, the median SBP of the population prior to treatment was 187 mm Hg (IQR 177-199; Table 2). Treatment was initiated in 30% of patients for an SBP <180 mm Hg and in 75% for an SBP <200 mm Hg. The median time to follow-up BP check was 34 minutes (IQR 15-58). The median decrement in SBP was 20 mm Hg (IQR 5-37); however, the response to treatment was highly variable, with 2% of patients experiencing no change and 14% experiencing an increase in SBP. Seventy-nine patients (14%) had a decrement in SBP >25% following treatment.

Description of Quality Improvement Results

Following the QI initiative, a total of 934 patients experienced 9,743 episodes of asymptomatic elevated blood pressure over a 4-month period (November 1, 2017 to February 28, 2018). As shown in Table 1, patients in the postintervention period had a slightly higher median age (67 [IQR 55-80] vs 69 [IQR 57-83]; P = .01), a higher median APR-DRG weight (1.34 [IQR 0.99-1.77] vs 1.48 [1.00-1.82]; P < .001), and a longer median length of stay (4.6 [2.8-8.0] days vs 5.1 [2.9-9.2] days; P = .004). There was also a higher proportion of nonEnglish speakers, fewer Black patients, and a lower proportion of preexisting HTN, in the postintervention period.

Of the 934 patients with ≥1 episode of asymptomatic elevated BP, 70 (7%) were treated with IV antihypertensive medications, with a total of 196 doses administered. The proportion of patients treated per month during the postintervention period ranged from 6% to 8%, which was the lowest of the entire study period and below the baseline average of 10% (Figure).

In a patient-level logistic regression pre-post analysis adjusting for age, sex, race, ethnicity, primary language, insurance status, preexisting HTN, length of stay, and APR-DRG weight, patients admitted to the general medicine service during the postintervention period had 38% lower odds of receiving IV antihypertensive medications than those admitted during the baseline period (OR = 0.62; 95% CI 0.47-0.83; P = .001). In this adjusted model, the following factors were independently associated with increased odds of receiving treatment: APR-DRG weight (OR 1.13; 95% CI 1.07-1.20; P < .001), Black race (OR 1.81; 95% CI 1.29-2.53; P = .001), length of stay (OR 1.02; 95% CI 1.01-1.03; P < .001), and preexisting HTN (OR 4.25; 95% CI 2.75-6.56; P < .001). Older age was associated with lower odds of treatment (Table 2).

Among patients who received treatment, there were no differences between pre- and postintervention periods in the proportion of pretreatment SBP <180 mm Hg (29% vs 32%; P = .40), 180-199 mm Hg (47% vs 40%; P = .10), or >200 mm Hg (25% vs 28%; P = .31; Table 3).

Our results suggest that treatment of asymptomatic elevated BP using IV antihypertensive medications is common practice at our institution. We found that treatment is often initiated for only modestly elevated BPs and that the clinical response to these medications is highly variable. In the baseline period, one in seven patients experienced a decrement in BP >25% following treatment, which could potentially cause harm.¹¹ There is no evidence, neither are there any consensus guidelines, to support the rapid reduction of BP among asymptomatic patients, making this a potential valuable opportunity for reducing unnecessary treatment, minimizing waste, and avoiding harm.

While there are a few previously published studies with similar results, we add to the existing literature by studying a larger population of more than 3,000 total patients, which was uniquely multiracial, including a high proportion of non-English speakers. Furthermore, our cohort included patients in the ICU, which is reflected in the higher-than-average APR-DRG weights. Despite being critically ill, these patients arguably still do not warrant aggressive treatment of elevated BP when asymptomatic. By excluding symptomatic BP elevations using surrogate markers for end-organ damage in addition to discharge diagnosis codes indicative of conditions in which tight BP control may be warranted, we were able to study a more critically ill patient population. We were also able to describe which baseline patient characteristics convey higher adjusted odds of receiving treatment, such as preexisting HTN, younger age, illness severity, and black race.

Perhaps most significantly, our study is the first to demonstrate an effective QI intervention aimed at reducing unnecessary utilization of IV antihypertensives. We found that this can feasibly be accomplished through a combination of educational efforts and systems changes, which could easily be replicated at other institutions. While the absolute reduction in the number of patients receiving treatment was modest, if these findings were to be widely accepted and resulted in a wide-spread change in culture, there would be a potential for greater impact.

Despite the reduction in the proportion of patients receiving IV antihypertensive medications, we found no change in the median SBP compared with the baseline period, which seems to support that the intervention was well tolerated. We also found no difference in the number of ICU transfers, rapid response calls, and cardiopulmonary arrests between groups. While these findings are both reassuring, it is impossible to draw definitive conclusions about safety given the small absolute number of patients having received treatment in each group. Fortunately, current guidelines and literature support the safety of such an intervention, as there is no existing evidence to suggest that failing to rapidly lower BP among asymptomatic patients is potentially harmful.¹¹

There are several limitations to our study. First, by utilizing a large electronic dataset, the quality of our analyses was reliant on the accuracy of the recorded EHR data. Second, in the absence of a controlled trial or control group, we cannot say definitively that our QI initiative was the direct cause of the improved rates of IV antihypertensive utilization, though the effect did persist after adjusting for baseline characteristics in patient-level models. Third, our follow-up period was relatively short, with fewer than half as many patients as in the preintervention period. This is an important limitation, since the impact of QI interventions often diminishes over time. We plan to continually monitor IV antihypertensive use, feed those data back to our group, and revitalize educational efforts should rates begin to rise. Fourth, we were unable to directly measure which patients had true end-organ injury and instead used orders placed around the time of medication administration as a surrogate marker. While this is an imperfect measure, we feel that in cases where a provider was concerned enough to even test for end-organ injury, the use of IV antihypertensives was likely justified and was therefore appropriately excluded from the analysis. Lastly, we were limited in our ability to describe associations with true clinical outcomes, such as stroke or myocardial infarction, which could theoretically be propagated by either the use or the avoidance of IV antihypertensive medications. Fortunately, based on clinical guidelines and existing evidence, there is no reason to believe that reducing IV antihypertensive use would result in increased rates of these outcomes.

Our study reaffirms the fact that overutilization of IV antihypertensive medications among asymptomatic hospitalized patients is pervasive across hospital systems. This represents a potential target for a concerted change in culture, which we have demonstrated can be feasibly accomplished through education and systems changes.

Disclosures

Dr. Auerbach has current or pending grants from the CDC, PCORI, and FDA that are unrelated to this research manuscript. He also receives royalties from UpToDate, and received an honorarium for being editor of JHM. Dr. Jacobs received a $1,000 Resident/Fellow Travel Grant from the Society of Hospital Medicine to support the cost of travel to SHM, where he presented this research as a poster in 2018. Dr. Prasad receives money from EpiExcellence, LLC for consultation, which is unrelated to this research manuscript. All other authors have nothing to disclose.

Resident physician workload has traditionally been measured by patient census.^1,2 However, census and other volume-based metrics such as daily admissions may not accurately reflect workload due to variation in patient complexity. Relative value units (RVUs) are another commonly used marker of workload, but the validity of this metric relies on accurate coding, usually done by the attending physician, and is less directly related to resident physician workload. Because much of hospital-based medicine is mediated through the electronic health record (EHR), which can capture differences in patient complexity,³ electronic records could be harnessed to more comprehensively describe residents’ work. Current government estimates indicate that several hundred companies offer certified EHRs, thanks in large part to the Health Information Technology for Economic and Clinical Health (HITECH) Act of 2009, which aimed to promote adoption and meaningful use of health information technology.^{4, 5} These systems can collect important data about the usage and operating patterns of physicians, which may provide an insight into workload.^6-8

Accurately measuring workload is important because of the direct link that has been drawn between physician workload and quality metrics. In a study of attending hospitalists, higher workload, as measured by patient census and RVUs, was associated with longer lengths of stay and higher costs of hospitalization.⁹ Another study among medical residents found that as daily admissions increased, length of stay, cost, and inpatient mortality appeared to rise.¹⁰ Although these studies used only volume-based workload metrics, the implication that high workload may negatively impact patient care hints at a possible trade-off between the two that should inform discussions of physician productivity.

In the current study, we examine whether data obtained from the EHR, particularly electronic order volume, could provide valuable information, in addition to patient volume, about resident physician workload. We first tested the feasibility and validity of using electronic order volume as an important component of clinical workload by examining the relationship between electronic order volume and well-established factors that are likely to increase the workload of residents, including patient level of care and severity of illness. Then, using order volume as a marker for workload, we sought to describe whether higher order volumes were associated with two discharge-related quality metrics, completion of a high-quality after-visit summary and timely discharge summary, postulating that quality metrics may suffer when residents are busier.

Study Design and Setting

We performed a single-center retrospective cohort study of patients admitted to the internal medicine service at the University of California, San Francisco (UCSF) Medical Center between May 1, 2015 and July 31, 2016. UCSF is a 600-bed academic medical center, and the inpatient internal medicine teaching service manages an average daily census of 80-90 patients. Medicine teams care for patients on the general acute-care wards, the step-down units (for patients with higher acuity of care), and also patients in the intensive care unit (ICU). ICU patients are comanaged by general medicine teams and intensive care teams; internal medicine teams enter all electronic orders for ICU patients, except for orders for respiratory care or sedating medications. The inpatient internal medicine teaching service comprises eight teams each supervised by an attending physician, a senior resident (in the second or third year of residency training), two interns, and a third- and/or fourth-year medical student. Residents place all clinical orders and complete all clinical documentation through the EHR (Epic Systems, Verona, Wisconsin).¹¹ Typically, the bulk of the orders and documentation, including discharge documentation, is performed by interns; however, the degree of senior resident involvement in these tasks is variable and team-dependent. In addition to the eight resident teams, there are also four attending hospitalist-only internal medicine teams, who manage a service of ~30-40 patients.

Study Population

Our study population comprised all hospitalized adults admitted to the eight resident-run teams on the internal medicine teaching service. Patients cared for by hospitalist-only teams were not included in this analysis. Because the focus of our study was on hospitalizations, individual patients may have been included multiple times over the course of the study. Hospitalizations were excluded if they did not have complete Medicare Severity-Diagnosis Related Group (MS-DRG) data,¹² since this was used as our severity of illness marker. This occurred either because patients were not discharged by the end of the study period or because they had a length of stay of less than one day, because this metric was not assigned to these short-stay (observation) patients.

Data Collection

All electronic orders placed during the study period were obtained by extracting data from Epic’s Clarity database. Our EHR allows for the use of order sets; each order in these sets was counted individually, so that an order set with several orders would not be identified as one order. We identified the time and date that the order was placed, the ordering physician, the identity of the patient for which the order was placed, and the location of the patient when the order was placed, to determine the level of care (ICU, step-down, or general medicine unit). To track the composite volume of orders placed by resident teams, we matched each ordering physician to his or her corresponding resident team using our physician scheduling database, Amion (Spiral Software). We obtained team census by tabulating the total number of patients that a single resident team placed orders on over the course of a given calendar day. From billing data, we identified the MS-DRG weight that was assigned at the end of each hospitalization. Finally, we collected data on adherence to two discharge-related quality metrics to determine whether increased order volume was associated with decreased rates of adherence to these metrics. Using departmental patient-level quality improvement data, we determined whether each metric was met on discharge at the patient level. We also extracted patient-level demographic data, including age, sex, and insurance status, from this departmental quality improvement database.

Discharge Quality Outcome Metrics

We hypothesized that as the total daily electronic orders of a resident team increased, the rate of completion of two discharge-related quality metrics would decline due to the greater time constraints placed on the teams. The first metric we used was the completion of a high-quality after-visit summary (AVS), which has been described by the Centers for Medicare and Medicaid Services as part of its Meaningful Use Initiative.¹³ It was selected by the residents in our program as a particularly high-priority quality metric. Our institution specifically defines a “high-quality” AVS as including the following three components: a principal hospital problem, patient instructions, and follow-up information. The second discharge-related quality metric was the completion of a timely discharge summary, another measure recognized as a critical component in high-quality care.¹⁴ To be considered timely, the discharge summary had to be filed no later than 24 hours after the discharge order was entered into the EHR. This metric was more recently tracked by the internal medicine department and was not selected by the residents as a high-priority metric.

Statistical Analysis

To examine how the order volume per day changed throughout each sequential day of hospital admission, mean orders per hospital day with 95% CIs were plotted. We performed an aggregate analysis of all orders placed for each patient per day across three different levels of care (ICU, step-down, and general medicine). For each day of the study period, we summed all orders for all patients according to their location and divided by the number of total patients in each location to identify the average number of orders written for an ICU, step-down, and general medicine patient that day. We then calculated the mean daily orders for an ICU, step-down, and general medicine patient over the entire study period. We used ANOVA to test for statistically significant differences between the mean daily orders between these locations.

To examine the relationship between severity of illness and order volume, we performed an unadjusted patient-level analysis of orders per patient in the first three days of each hospitalization and stratified the data by the MS-DRG payment weight, which we divided into four quartiles. For each quartile, we calculated the mean number of orders placed in the first three days of admission and used ANOVA to test for statistically significant differences. We restricted the orders to the first three days of hospitalization instead of calculating mean orders per day of hospitalization because we postulated that the majority of orders were entered in these first few days and that with increasing length of stay (which we expected to occur with higher MS-DRG weight), the order volume becomes highly variable, which would tend to skew the mean orders per day.

We used multivariable logistic regression to determine whether the volume of electronic orders on the day of a given patient’s discharge, and also on the day before a given patient’s discharge, was a significant predictor of receiving a high-quality AVS. We adjusted for team census on the day of discharge, MS-DRG weight, age, sex, and insurance status. We then conducted a separate analysis of the association between electronic order volume and likelihood of completing a timely discharge summary among patients where discharge summary data were available. Logistic regression for each case was performed independently, so that team orders on the day prior to a patient’s discharge were not included in the model for the relationship between team orders on the day of a patient’s discharge and the discharge-related quality metric of interest, and vice versa, since including both in the model would be potentially disruptive given that orders on the day before and day of a patient’s discharge are likely correlated.

We also performed a subanalysis in which we restricted orders to only those placed during the daytime hours (7 am-7 pm), since these reflect the work performed by the primary team, and excluded those placed by covering night-shift residents.

IRB Approval

The study was approved by the UCSF Institutional Review Board and was granted a waiver of informed consent.

Population

We identified 7,296 eligible hospitalizations during the study period. After removing hospitalizations according to our exclusion criteria (Figure 1), there were 5,032 hospitalizations that were used in the analysis for which a total of 929,153 orders were written. The vast majority of patients received at least one order per day; fewer than 1% of encounter-days had zero associated orders. The top 10 discharge diagnoses identified in the cohort are listed in Appendix Table 1. A breakdown of orders by order type, across the entire cohort, is displayed in Appendix Table 2. The mean number of orders per patient per day of hospitalization is plotted in the Appendix Figure, which indicates that the number of orders is highest on the day of admission, decreases significantly after the first few days, and becomes increasingly variable with longer lengths of stay.

Patient Level of Care and Severity of Illness Metrics

Patients at a higher level of care had, on average, more orders entered per day. The mean order frequency was 40 orders per day for an ICU patient (standard deviation [SD] 13, range 13-134), 24 for a step-down patient (SD 6, range 11-48), and 19 for a general medicine unit patient (SD 3, range 10-31). The difference in mean daily orders was statistically significant (P < .001, Figure 2a).

Orders also correlated with increasing severity of illness. Patients in the lowest quartile of MS-DRG weight received, on average, 98 orders in the first three days of hospitalization (SD 35, range 2-349), those in the second quartile received 105 orders (SD 38, range 10-380), those in the third quartile received 132 orders (SD 51, range 17-436), and those in the fourth and highest quartile received 149 orders (SD 59, range 32-482). Comparisons between each of these severity of illness categories were significant (P < .001, Figure 2b).

Discharge-Related Quality Metrics

The median number of orders per internal medicine team per day was 343 (IQR 261- 446). Of the 5,032 total discharged patients, 3,657 (73%) received a high-quality AVS on discharge. After controlling for team census, severity of illness, and demographic factors, there was no statistically significant association between total orders on the day of discharge and odds of receiving a high-quality AVS (OR 1.01; 95% CI 0.96-1.06), or between team orders placed the day prior to discharge and odds of receiving a high-quality AVS (OR 0.99; 95% CI 0.95-1.04; Table 1). When we restricted our analysis to orders placed during daytime hours (7 am-7 pm), these findings were largely unchanged (OR 1.05; 95% CI 0.97-1.14 for orders on the day of discharge; OR 1.02; 95% CI 0.95-1.10 for orders on the day before discharge).

There were 3,835 patients for whom data on timing of discharge summary were available. Of these, 3,455 (91.2%) had a discharge summary completed within 24 hours. After controlling for team census, severity of illness, and demographic factors, there was no statistically significant association between total orders placed by the team on a patient’s day of discharge and odds of receiving a timely discharge summary (OR 0.96; 95% CI 0.88-1.05). However, patients were 12% less likely to receive a timely discharge summary for every 100 extra orders the team placed on the day prior to discharge (OR 0.88, 95% CI 0.82-0.95). Patients who received a timely discharge summary were cared for by teams who placed a median of 345 orders the day prior to their discharge, whereas those that did not receive a timely discharge summary were cared for by teams who placed a significantly higher number of orders (375) on the day prior to discharge (Table 2). When we restricted our analysis to only daytime orders, there were no significant changes in the findings (OR 1.00; 95% CI 0.88-1.14 for orders on the day of discharge; OR 0.84; 95% CI 0.75-0.95 for orders on the day prior to discharge).

We found that electronic order volume may be a marker for patient complexity, which encompasses both level of care and severity of illness, and could be a marker of resident physician workload that harnesses readily available data from an EHR. Recent time-motion studies of internal medicine residents indicate that the majority of trainees’ time is spent on computers, engaged in indirect patient care activities such as reading electronic charts, entering electronic orders, and writing computerized notes.^15-18 Capturing these tasks through metrics such as electronic order volume, as we did in this study, can provide valuable insights into resident physician workflow.

We found that ICU patients received more than twice as many orders per day than did general acute care-level patients. Furthermore, we found that patients whose hospitalizations fell into the highest MS-DRG weight quartile received approximately 50% more orders during the first three days of admission compared to that of patients whose hospitalizations fell into the lowest quartile. This strong association indicates that electronic order volume could provide meaningful additional information, in concert with other factors such as census, to describe resident physician workload.

We did not find that our workload measure was significantly associated with high-quality AVS completion. There are several possible explanations for this finding. First, adherence to this quality metric may be independent of workload, possibly because it is highly prioritized by residents at our institution. Second, adherence may only be impacted at levels of workload greater than what was experienced by the residents in our study. Finally, electronic order volume may not encompass enough of total workload to be reliably representative of resident work. However, the tight correlation between electronic order volume with severity of illness and level of care, in conjunction with the finding that patients were less likely to receive a timely discharge summary when workload was high on the day prior to a patient’s discharge, suggests that electronic order volume does indeed encompass a meaningful component of workload, and that with higher workload, adherence to some quality metrics may decline. We found that patients who received a timely discharge summary were discharged by teams who entered 30 fewer orders on the day before discharge compared with patients who did not receive a timely discharge summary. In addition to being statistically significant, it is also likely that this difference is clinically significant, although a determination of clinical significance is outside the scope of this study. Further exploration into the relationship between order volume and other quality metrics that are perhaps more sensitive to workload would be interesting.

The primary strength of our study is in how it demonstrates that EHRs can be harnessed to provide additional insights into clinical workload in a quantifiable and automated manner. Although there are a wide range of EHRs currently in use across the country, the capability to track electronic orders is common and could therefore be used broadly across institutions, with tailoring and standardization specific to each site. This technique is similar to that used by prior investigators who characterized the workload of pediatric residents by orders entered and notes written in the electronic medical record.¹⁹ However, our study is unique, in that we explored the relationship between electronic order volume and patient-level severity metrics as well as discharge-related quality metrics.

Our study is limited by several factors. When conceptualizing resident workload, several other elements that contribute to a sense of “busyness” may be independent of electronic orders and were not measured in our study.²⁰ These include communication factors (such as language discordance, discussion with consulting services, and difficult end-of-life discussions), environmental factors (such as geographic localization), resident physician team factors (such as competing clinical or educational responsibilities), timing (in terms of day of week as well as time of year, since residents in July likely feel “busier” than residents in May), and ultimate discharge destination for patients (those going to a skilled nursing facility may require discharge documentation more urgently). Additionally, we chose to focus on the workload of resident teams, as represented by team orders, as opposed to individual work, which may be more directly correlated to our outcomes of interest, completion of a high-quality AVS, and timely discharge summary, which are usually performed by individuals.

Furthermore, we did not measure the relationship between our objective measure of workload and clinical endpoints. Instead, we chose to focus on process measures because they are less likely to be confounded by clinical factors independent of physician workload.²¹ Future studies should also consider obtaining direct resident-level measures of “busyness” or burnout, or other resident-centered endpoints, such as whether residents left the hospital at times consistent with duty hour regulations or whether they were able to attend educational conferences.

These limitations pose opportunities for further efforts to more comprehensively characterize clinical workload. Additional research is needed to understand and quantify the impact of patient, physician, and environmental factors that are not reflected by electronic order volume. Furthermore, an exploration of other electronic surrogates for clinical workload, such as paging volume and other EHR-derived data points, could also prove valuable in further describing the clinical workload. Future studies should also examine whether there is a relationship between these novel markers of workload and further outcomes, including both process measures and clinical endpoints.

Electronic order volume may provide valuable additional information for estimating the workload of resident physicians caring for hospitalized patients. Further investigation to determine whether the statistically significant differences identified in this study are clinically significant, how the technique used in this work may be applied to different EHRs, an examination of other EHR-derived metrics that may represent workload, and an exploration of additional patient-centered outcomes may be warranted.

Disclosures

Rajkomar reports personal fees from Google LLC, outside the submitted work. Dr. Khanna reports that during the conduct of the study, his salary, and the development of CareWeb (a communication platform that includes a smartphone-based paging application in use in several inpatient clinical units at University of California, San Francisco [UCSF] Medical Center) were supported by funding from the Center for Digital Health Innovation at UCSF. The CareWeb software has been licensed by Voalte.

Disclaimer

The views expressed in the submitted article are of the authors and not an official position of the institution.

Resident physician workload has traditionally been measured by patient census.^1,2 However, census and other volume-based metrics such as daily admissions may not accurately reflect workload due to variation in patient complexity. Relative value units (RVUs) are another commonly used marker of workload, but the validity of this metric relies on accurate coding, usually done by the attending physician, and is less directly related to resident physician workload. Because much of hospital-based medicine is mediated through the electronic health record (EHR), which can capture differences in patient complexity,³ electronic records could be harnessed to more comprehensively describe residents’ work. Current government estimates indicate that several hundred companies offer certified EHRs, thanks in large part to the Health Information Technology for Economic and Clinical Health (HITECH) Act of 2009, which aimed to promote adoption and meaningful use of health information technology.^{4, 5} These systems can collect important data about the usage and operating patterns of physicians, which may provide an insight into workload.^6-8

Accurately measuring workload is important because of the direct link that has been drawn between physician workload and quality metrics. In a study of attending hospitalists, higher workload, as measured by patient census and RVUs, was associated with longer lengths of stay and higher costs of hospitalization.⁹ Another study among medical residents found that as daily admissions increased, length of stay, cost, and inpatient mortality appeared to rise.¹⁰ Although these studies used only volume-based workload metrics, the implication that high workload may negatively impact patient care hints at a possible trade-off between the two that should inform discussions of physician productivity.

In the current study, we examine whether data obtained from the EHR, particularly electronic order volume, could provide valuable information, in addition to patient volume, about resident physician workload. We first tested the feasibility and validity of using electronic order volume as an important component of clinical workload by examining the relationship between electronic order volume and well-established factors that are likely to increase the workload of residents, including patient level of care and severity of illness. Then, using order volume as a marker for workload, we sought to describe whether higher order volumes were associated with two discharge-related quality metrics, completion of a high-quality after-visit summary and timely discharge summary, postulating that quality metrics may suffer when residents are busier.

Study Design and Setting

We performed a single-center retrospective cohort study of patients admitted to the internal medicine service at the University of California, San Francisco (UCSF) Medical Center between May 1, 2015 and July 31, 2016. UCSF is a 600-bed academic medical center, and the inpatient internal medicine teaching service manages an average daily census of 80-90 patients. Medicine teams care for patients on the general acute-care wards, the step-down units (for patients with higher acuity of care), and also patients in the intensive care unit (ICU). ICU patients are comanaged by general medicine teams and intensive care teams; internal medicine teams enter all electronic orders for ICU patients, except for orders for respiratory care or sedating medications. The inpatient internal medicine teaching service comprises eight teams each supervised by an attending physician, a senior resident (in the second or third year of residency training), two interns, and a third- and/or fourth-year medical student. Residents place all clinical orders and complete all clinical documentation through the EHR (Epic Systems, Verona, Wisconsin).¹¹ Typically, the bulk of the orders and documentation, including discharge documentation, is performed by interns; however, the degree of senior resident involvement in these tasks is variable and team-dependent. In addition to the eight resident teams, there are also four attending hospitalist-only internal medicine teams, who manage a service of ~30-40 patients.

Study Population

Our study population comprised all hospitalized adults admitted to the eight resident-run teams on the internal medicine teaching service. Patients cared for by hospitalist-only teams were not included in this analysis. Because the focus of our study was on hospitalizations, individual patients may have been included multiple times over the course of the study. Hospitalizations were excluded if they did not have complete Medicare Severity-Diagnosis Related Group (MS-DRG) data,¹² since this was used as our severity of illness marker. This occurred either because patients were not discharged by the end of the study period or because they had a length of stay of less than one day, because this metric was not assigned to these short-stay (observation) patients.

Data Collection

All electronic orders placed during the study period were obtained by extracting data from Epic’s Clarity database. Our EHR allows for the use of order sets; each order in these sets was counted individually, so that an order set with several orders would not be identified as one order. We identified the time and date that the order was placed, the ordering physician, the identity of the patient for which the order was placed, and the location of the patient when the order was placed, to determine the level of care (ICU, step-down, or general medicine unit). To track the composite volume of orders placed by resident teams, we matched each ordering physician to his or her corresponding resident team using our physician scheduling database, Amion (Spiral Software). We obtained team census by tabulating the total number of patients that a single resident team placed orders on over the course of a given calendar day. From billing data, we identified the MS-DRG weight that was assigned at the end of each hospitalization. Finally, we collected data on adherence to two discharge-related quality metrics to determine whether increased order volume was associated with decreased rates of adherence to these metrics. Using departmental patient-level quality improvement data, we determined whether each metric was met on discharge at the patient level. We also extracted patient-level demographic data, including age, sex, and insurance status, from this departmental quality improvement database.

Discharge Quality Outcome Metrics

We hypothesized that as the total daily electronic orders of a resident team increased, the rate of completion of two discharge-related quality metrics would decline due to the greater time constraints placed on the teams. The first metric we used was the completion of a high-quality after-visit summary (AVS), which has been described by the Centers for Medicare and Medicaid Services as part of its Meaningful Use Initiative.¹³ It was selected by the residents in our program as a particularly high-priority quality metric. Our institution specifically defines a “high-quality” AVS as including the following three components: a principal hospital problem, patient instructions, and follow-up information. The second discharge-related quality metric was the completion of a timely discharge summary, another measure recognized as a critical component in high-quality care.¹⁴ To be considered timely, the discharge summary had to be filed no later than 24 hours after the discharge order was entered into the EHR. This metric was more recently tracked by the internal medicine department and was not selected by the residents as a high-priority metric.

Statistical Analysis

To examine how the order volume per day changed throughout each sequential day of hospital admission, mean orders per hospital day with 95% CIs were plotted. We performed an aggregate analysis of all orders placed for each patient per day across three different levels of care (ICU, step-down, and general medicine). For each day of the study period, we summed all orders for all patients according to their location and divided by the number of total patients in each location to identify the average number of orders written for an ICU, step-down, and general medicine patient that day. We then calculated the mean daily orders for an ICU, step-down, and general medicine patient over the entire study period. We used ANOVA to test for statistically significant differences between the mean daily orders between these locations.

To examine the relationship between severity of illness and order volume, we performed an unadjusted patient-level analysis of orders per patient in the first three days of each hospitalization and stratified the data by the MS-DRG payment weight, which we divided into four quartiles. For each quartile, we calculated the mean number of orders placed in the first three days of admission and used ANOVA to test for statistically significant differences. We restricted the orders to the first three days of hospitalization instead of calculating mean orders per day of hospitalization because we postulated that the majority of orders were entered in these first few days and that with increasing length of stay (which we expected to occur with higher MS-DRG weight), the order volume becomes highly variable, which would tend to skew the mean orders per day.

We used multivariable logistic regression to determine whether the volume of electronic orders on the day of a given patient’s discharge, and also on the day before a given patient’s discharge, was a significant predictor of receiving a high-quality AVS. We adjusted for team census on the day of discharge, MS-DRG weight, age, sex, and insurance status. We then conducted a separate analysis of the association between electronic order volume and likelihood of completing a timely discharge summary among patients where discharge summary data were available. Logistic regression for each case was performed independently, so that team orders on the day prior to a patient’s discharge were not included in the model for the relationship between team orders on the day of a patient’s discharge and the discharge-related quality metric of interest, and vice versa, since including both in the model would be potentially disruptive given that orders on the day before and day of a patient’s discharge are likely correlated.

We also performed a subanalysis in which we restricted orders to only those placed during the daytime hours (7 am-7 pm), since these reflect the work performed by the primary team, and excluded those placed by covering night-shift residents.

IRB Approval

The study was approved by the UCSF Institutional Review Board and was granted a waiver of informed consent.

Population

We identified 7,296 eligible hospitalizations during the study period. After removing hospitalizations according to our exclusion criteria (Figure 1), there were 5,032 hospitalizations that were used in the analysis for which a total of 929,153 orders were written. The vast majority of patients received at least one order per day; fewer than 1% of encounter-days had zero associated orders. The top 10 discharge diagnoses identified in the cohort are listed in Appendix Table 1. A breakdown of orders by order type, across the entire cohort, is displayed in Appendix Table 2. The mean number of orders per patient per day of hospitalization is plotted in the Appendix Figure, which indicates that the number of orders is highest on the day of admission, decreases significantly after the first few days, and becomes increasingly variable with longer lengths of stay.

Patient Level of Care and Severity of Illness Metrics

Patients at a higher level of care had, on average, more orders entered per day. The mean order frequency was 40 orders per day for an ICU patient (standard deviation [SD] 13, range 13-134), 24 for a step-down patient (SD 6, range 11-48), and 19 for a general medicine unit patient (SD 3, range 10-31). The difference in mean daily orders was statistically significant (P < .001, Figure 2a).

Orders also correlated with increasing severity of illness. Patients in the lowest quartile of MS-DRG weight received, on average, 98 orders in the first three days of hospitalization (SD 35, range 2-349), those in the second quartile received 105 orders (SD 38, range 10-380), those in the third quartile received 132 orders (SD 51, range 17-436), and those in the fourth and highest quartile received 149 orders (SD 59, range 32-482). Comparisons between each of these severity of illness categories were significant (P < .001, Figure 2b).

Discharge-Related Quality Metrics

The median number of orders per internal medicine team per day was 343 (IQR 261- 446). Of the 5,032 total discharged patients, 3,657 (73%) received a high-quality AVS on discharge. After controlling for team census, severity of illness, and demographic factors, there was no statistically significant association between total orders on the day of discharge and odds of receiving a high-quality AVS (OR 1.01; 95% CI 0.96-1.06), or between team orders placed the day prior to discharge and odds of receiving a high-quality AVS (OR 0.99; 95% CI 0.95-1.04; Table 1). When we restricted our analysis to orders placed during daytime hours (7 am-7 pm), these findings were largely unchanged (OR 1.05; 95% CI 0.97-1.14 for orders on the day of discharge; OR 1.02; 95% CI 0.95-1.10 for orders on the day before discharge).

There were 3,835 patients for whom data on timing of discharge summary were available. Of these, 3,455 (91.2%) had a discharge summary completed within 24 hours. After controlling for team census, severity of illness, and demographic factors, there was no statistically significant association between total orders placed by the team on a patient’s day of discharge and odds of receiving a timely discharge summary (OR 0.96; 95% CI 0.88-1.05). However, patients were 12% less likely to receive a timely discharge summary for every 100 extra orders the team placed on the day prior to discharge (OR 0.88, 95% CI 0.82-0.95). Patients who received a timely discharge summary were cared for by teams who placed a median of 345 orders the day prior to their discharge, whereas those that did not receive a timely discharge summary were cared for by teams who placed a significantly higher number of orders (375) on the day prior to discharge (Table 2). When we restricted our analysis to only daytime orders, there were no significant changes in the findings (OR 1.00; 95% CI 0.88-1.14 for orders on the day of discharge; OR 0.84; 95% CI 0.75-0.95 for orders on the day prior to discharge).

We found that electronic order volume may be a marker for patient complexity, which encompasses both level of care and severity of illness, and could be a marker of resident physician workload that harnesses readily available data from an EHR. Recent time-motion studies of internal medicine residents indicate that the majority of trainees’ time is spent on computers, engaged in indirect patient care activities such as reading electronic charts, entering electronic orders, and writing computerized notes.^15-18 Capturing these tasks through metrics such as electronic order volume, as we did in this study, can provide valuable insights into resident physician workflow.

We found that ICU patients received more than twice as many orders per day than did general acute care-level patients. Furthermore, we found that patients whose hospitalizations fell into the highest MS-DRG weight quartile received approximately 50% more orders during the first three days of admission compared to that of patients whose hospitalizations fell into the lowest quartile. This strong association indicates that electronic order volume could provide meaningful additional information, in concert with other factors such as census, to describe resident physician workload.

We did not find that our workload measure was significantly associated with high-quality AVS completion. There are several possible explanations for this finding. First, adherence to this quality metric may be independent of workload, possibly because it is highly prioritized by residents at our institution. Second, adherence may only be impacted at levels of workload greater than what was experienced by the residents in our study. Finally, electronic order volume may not encompass enough of total workload to be reliably representative of resident work. However, the tight correlation between electronic order volume with severity of illness and level of care, in conjunction with the finding that patients were less likely to receive a timely discharge summary when workload was high on the day prior to a patient’s discharge, suggests that electronic order volume does indeed encompass a meaningful component of workload, and that with higher workload, adherence to some quality metrics may decline. We found that patients who received a timely discharge summary were discharged by teams who entered 30 fewer orders on the day before discharge compared with patients who did not receive a timely discharge summary. In addition to being statistically significant, it is also likely that this difference is clinically significant, although a determination of clinical significance is outside the scope of this study. Further exploration into the relationship between order volume and other quality metrics that are perhaps more sensitive to workload would be interesting.

The primary strength of our study is in how it demonstrates that EHRs can be harnessed to provide additional insights into clinical workload in a quantifiable and automated manner. Although there are a wide range of EHRs currently in use across the country, the capability to track electronic orders is common and could therefore be used broadly across institutions, with tailoring and standardization specific to each site. This technique is similar to that used by prior investigators who characterized the workload of pediatric residents by orders entered and notes written in the electronic medical record.¹⁹ However, our study is unique, in that we explored the relationship between electronic order volume and patient-level severity metrics as well as discharge-related quality metrics.

Our study is limited by several factors. When conceptualizing resident workload, several other elements that contribute to a sense of “busyness” may be independent of electronic orders and were not measured in our study.²⁰ These include communication factors (such as language discordance, discussion with consulting services, and difficult end-of-life discussions), environmental factors (such as geographic localization), resident physician team factors (such as competing clinical or educational responsibilities), timing (in terms of day of week as well as time of year, since residents in July likely feel “busier” than residents in May), and ultimate discharge destination for patients (those going to a skilled nursing facility may require discharge documentation more urgently). Additionally, we chose to focus on the workload of resident teams, as represented by team orders, as opposed to individual work, which may be more directly correlated to our outcomes of interest, completion of a high-quality AVS, and timely discharge summary, which are usually performed by individuals.

Furthermore, we did not measure the relationship between our objective measure of workload and clinical endpoints. Instead, we chose to focus on process measures because they are less likely to be confounded by clinical factors independent of physician workload.²¹ Future studies should also consider obtaining direct resident-level measures of “busyness” or burnout, or other resident-centered endpoints, such as whether residents left the hospital at times consistent with duty hour regulations or whether they were able to attend educational conferences.

These limitations pose opportunities for further efforts to more comprehensively characterize clinical workload. Additional research is needed to understand and quantify the impact of patient, physician, and environmental factors that are not reflected by electronic order volume. Furthermore, an exploration of other electronic surrogates for clinical workload, such as paging volume and other EHR-derived data points, could also prove valuable in further describing the clinical workload. Future studies should also examine whether there is a relationship between these novel markers of workload and further outcomes, including both process measures and clinical endpoints.

Electronic order volume may provide valuable additional information for estimating the workload of resident physicians caring for hospitalized patients. Further investigation to determine whether the statistically significant differences identified in this study are clinically significant, how the technique used in this work may be applied to different EHRs, an examination of other EHR-derived metrics that may represent workload, and an exploration of additional patient-centered outcomes may be warranted.

Disclosures

Rajkomar reports personal fees from Google LLC, outside the submitted work. Dr. Khanna reports that during the conduct of the study, his salary, and the development of CareWeb (a communication platform that includes a smartphone-based paging application in use in several inpatient clinical units at University of California, San Francisco [UCSF] Medical Center) were supported by funding from the Center for Digital Health Innovation at UCSF. The CareWeb software has been licensed by Voalte.

Disclaimer

The views expressed in the submitted article are of the authors and not an official position of the institution.

Resident physician workload has traditionally been measured by patient census.^1,2 However, census and other volume-based metrics such as daily admissions may not accurately reflect workload due to variation in patient complexity. Relative value units (RVUs) are another commonly used marker of workload, but the validity of this metric relies on accurate coding, usually done by the attending physician, and is less directly related to resident physician workload. Because much of hospital-based medicine is mediated through the electronic health record (EHR), which can capture differences in patient complexity,³ electronic records could be harnessed to more comprehensively describe residents’ work. Current government estimates indicate that several hundred companies offer certified EHRs, thanks in large part to the Health Information Technology for Economic and Clinical Health (HITECH) Act of 2009, which aimed to promote adoption and meaningful use of health information technology.^{4, 5} These systems can collect important data about the usage and operating patterns of physicians, which may provide an insight into workload.^6-8

Accurately measuring workload is important because of the direct link that has been drawn between physician workload and quality metrics. In a study of attending hospitalists, higher workload, as measured by patient census and RVUs, was associated with longer lengths of stay and higher costs of hospitalization.⁹ Another study among medical residents found that as daily admissions increased, length of stay, cost, and inpatient mortality appeared to rise.¹⁰ Although these studies used only volume-based workload metrics, the implication that high workload may negatively impact patient care hints at a possible trade-off between the two that should inform discussions of physician productivity.

In the current study, we examine whether data obtained from the EHR, particularly electronic order volume, could provide valuable information, in addition to patient volume, about resident physician workload. We first tested the feasibility and validity of using electronic order volume as an important component of clinical workload by examining the relationship between electronic order volume and well-established factors that are likely to increase the workload of residents, including patient level of care and severity of illness. Then, using order volume as a marker for workload, we sought to describe whether higher order volumes were associated with two discharge-related quality metrics, completion of a high-quality after-visit summary and timely discharge summary, postulating that quality metrics may suffer when residents are busier.

Study Design and Setting

We performed a single-center retrospective cohort study of patients admitted to the internal medicine service at the University of California, San Francisco (UCSF) Medical Center between May 1, 2015 and July 31, 2016. UCSF is a 600-bed academic medical center, and the inpatient internal medicine teaching service manages an average daily census of 80-90 patients. Medicine teams care for patients on the general acute-care wards, the step-down units (for patients with higher acuity of care), and also patients in the intensive care unit (ICU). ICU patients are comanaged by general medicine teams and intensive care teams; internal medicine teams enter all electronic orders for ICU patients, except for orders for respiratory care or sedating medications. The inpatient internal medicine teaching service comprises eight teams each supervised by an attending physician, a senior resident (in the second or third year of residency training), two interns, and a third- and/or fourth-year medical student. Residents place all clinical orders and complete all clinical documentation through the EHR (Epic Systems, Verona, Wisconsin).¹¹ Typically, the bulk of the orders and documentation, including discharge documentation, is performed by interns; however, the degree of senior resident involvement in these tasks is variable and team-dependent. In addition to the eight resident teams, there are also four attending hospitalist-only internal medicine teams, who manage a service of ~30-40 patients.

Study Population

Our study population comprised all hospitalized adults admitted to the eight resident-run teams on the internal medicine teaching service. Patients cared for by hospitalist-only teams were not included in this analysis. Because the focus of our study was on hospitalizations, individual patients may have been included multiple times over the course of the study. Hospitalizations were excluded if they did not have complete Medicare Severity-Diagnosis Related Group (MS-DRG) data,¹² since this was used as our severity of illness marker. This occurred either because patients were not discharged by the end of the study period or because they had a length of stay of less than one day, because this metric was not assigned to these short-stay (observation) patients.

Data Collection

All electronic orders placed during the study period were obtained by extracting data from Epic’s Clarity database. Our EHR allows for the use of order sets; each order in these sets was counted individually, so that an order set with several orders would not be identified as one order. We identified the time and date that the order was placed, the ordering physician, the identity of the patient for which the order was placed, and the location of the patient when the order was placed, to determine the level of care (ICU, step-down, or general medicine unit). To track the composite volume of orders placed by resident teams, we matched each ordering physician to his or her corresponding resident team using our physician scheduling database, Amion (Spiral Software). We obtained team census by tabulating the total number of patients that a single resident team placed orders on over the course of a given calendar day. From billing data, we identified the MS-DRG weight that was assigned at the end of each hospitalization. Finally, we collected data on adherence to two discharge-related quality metrics to determine whether increased order volume was associated with decreased rates of adherence to these metrics. Using departmental patient-level quality improvement data, we determined whether each metric was met on discharge at the patient level. We also extracted patient-level demographic data, including age, sex, and insurance status, from this departmental quality improvement database.

Discharge Quality Outcome Metrics

We hypothesized that as the total daily electronic orders of a resident team increased, the rate of completion of two discharge-related quality metrics would decline due to the greater time constraints placed on the teams. The first metric we used was the completion of a high-quality after-visit summary (AVS), which has been described by the Centers for Medicare and Medicaid Services as part of its Meaningful Use Initiative.¹³ It was selected by the residents in our program as a particularly high-priority quality metric. Our institution specifically defines a “high-quality” AVS as including the following three components: a principal hospital problem, patient instructions, and follow-up information. The second discharge-related quality metric was the completion of a timely discharge summary, another measure recognized as a critical component in high-quality care.¹⁴ To be considered timely, the discharge summary had to be filed no later than 24 hours after the discharge order was entered into the EHR. This metric was more recently tracked by the internal medicine department and was not selected by the residents as a high-priority metric.

Statistical Analysis

To examine how the order volume per day changed throughout each sequential day of hospital admission, mean orders per hospital day with 95% CIs were plotted. We performed an aggregate analysis of all orders placed for each patient per day across three different levels of care (ICU, step-down, and general medicine). For each day of the study period, we summed all orders for all patients according to their location and divided by the number of total patients in each location to identify the average number of orders written for an ICU, step-down, and general medicine patient that day. We then calculated the mean daily orders for an ICU, step-down, and general medicine patient over the entire study period. We used ANOVA to test for statistically significant differences between the mean daily orders between these locations.

To examine the relationship between severity of illness and order volume, we performed an unadjusted patient-level analysis of orders per patient in the first three days of each hospitalization and stratified the data by the MS-DRG payment weight, which we divided into four quartiles. For each quartile, we calculated the mean number of orders placed in the first three days of admission and used ANOVA to test for statistically significant differences. We restricted the orders to the first three days of hospitalization instead of calculating mean orders per day of hospitalization because we postulated that the majority of orders were entered in these first few days and that with increasing length of stay (which we expected to occur with higher MS-DRG weight), the order volume becomes highly variable, which would tend to skew the mean orders per day.

We used multivariable logistic regression to determine whether the volume of electronic orders on the day of a given patient’s discharge, and also on the day before a given patient’s discharge, was a significant predictor of receiving a high-quality AVS. We adjusted for team census on the day of discharge, MS-DRG weight, age, sex, and insurance status. We then conducted a separate analysis of the association between electronic order volume and likelihood of completing a timely discharge summary among patients where discharge summary data were available. Logistic regression for each case was performed independently, so that team orders on the day prior to a patient’s discharge were not included in the model for the relationship between team orders on the day of a patient’s discharge and the discharge-related quality metric of interest, and vice versa, since including both in the model would be potentially disruptive given that orders on the day before and day of a patient’s discharge are likely correlated.

We also performed a subanalysis in which we restricted orders to only those placed during the daytime hours (7 am-7 pm), since these reflect the work performed by the primary team, and excluded those placed by covering night-shift residents.

IRB Approval

The study was approved by the UCSF Institutional Review Board and was granted a waiver of informed consent.

Population

We identified 7,296 eligible hospitalizations during the study period. After removing hospitalizations according to our exclusion criteria (Figure 1), there were 5,032 hospitalizations that were used in the analysis for which a total of 929,153 orders were written. The vast majority of patients received at least one order per day; fewer than 1% of encounter-days had zero associated orders. The top 10 discharge diagnoses identified in the cohort are listed in Appendix Table 1. A breakdown of orders by order type, across the entire cohort, is displayed in Appendix Table 2. The mean number of orders per patient per day of hospitalization is plotted in the Appendix Figure, which indicates that the number of orders is highest on the day of admission, decreases significantly after the first few days, and becomes increasingly variable with longer lengths of stay.

Patient Level of Care and Severity of Illness Metrics

Patients at a higher level of care had, on average, more orders entered per day. The mean order frequency was 40 orders per day for an ICU patient (standard deviation [SD] 13, range 13-134), 24 for a step-down patient (SD 6, range 11-48), and 19 for a general medicine unit patient (SD 3, range 10-31). The difference in mean daily orders was statistically significant (P < .001, Figure 2a).

Orders also correlated with increasing severity of illness. Patients in the lowest quartile of MS-DRG weight received, on average, 98 orders in the first three days of hospitalization (SD 35, range 2-349), those in the second quartile received 105 orders (SD 38, range 10-380), those in the third quartile received 132 orders (SD 51, range 17-436), and those in the fourth and highest quartile received 149 orders (SD 59, range 32-482). Comparisons between each of these severity of illness categories were significant (P < .001, Figure 2b).

Discharge-Related Quality Metrics

The median number of orders per internal medicine team per day was 343 (IQR 261- 446). Of the 5,032 total discharged patients, 3,657 (73%) received a high-quality AVS on discharge. After controlling for team census, severity of illness, and demographic factors, there was no statistically significant association between total orders on the day of discharge and odds of receiving a high-quality AVS (OR 1.01; 95% CI 0.96-1.06), or between team orders placed the day prior to discharge and odds of receiving a high-quality AVS (OR 0.99; 95% CI 0.95-1.04; Table 1). When we restricted our analysis to orders placed during daytime hours (7 am-7 pm), these findings were largely unchanged (OR 1.05; 95% CI 0.97-1.14 for orders on the day of discharge; OR 1.02; 95% CI 0.95-1.10 for orders on the day before discharge).

There were 3,835 patients for whom data on timing of discharge summary were available. Of these, 3,455 (91.2%) had a discharge summary completed within 24 hours. After controlling for team census, severity of illness, and demographic factors, there was no statistically significant association between total orders placed by the team on a patient’s day of discharge and odds of receiving a timely discharge summary (OR 0.96; 95% CI 0.88-1.05). However, patients were 12% less likely to receive a timely discharge summary for every 100 extra orders the team placed on the day prior to discharge (OR 0.88, 95% CI 0.82-0.95). Patients who received a timely discharge summary were cared for by teams who placed a median of 345 orders the day prior to their discharge, whereas those that did not receive a timely discharge summary were cared for by teams who placed a significantly higher number of orders (375) on the day prior to discharge (Table 2). When we restricted our analysis to only daytime orders, there were no significant changes in the findings (OR 1.00; 95% CI 0.88-1.14 for orders on the day of discharge; OR 0.84; 95% CI 0.75-0.95 for orders on the day prior to discharge).

We found that electronic order volume may be a marker for patient complexity, which encompasses both level of care and severity of illness, and could be a marker of resident physician workload that harnesses readily available data from an EHR. Recent time-motion studies of internal medicine residents indicate that the majority of trainees’ time is spent on computers, engaged in indirect patient care activities such as reading electronic charts, entering electronic orders, and writing computerized notes.^15-18 Capturing these tasks through metrics such as electronic order volume, as we did in this study, can provide valuable insights into resident physician workflow.

We found that ICU patients received more than twice as many orders per day than did general acute care-level patients. Furthermore, we found that patients whose hospitalizations fell into the highest MS-DRG weight quartile received approximately 50% more orders during the first three days of admission compared to that of patients whose hospitalizations fell into the lowest quartile. This strong association indicates that electronic order volume could provide meaningful additional information, in concert with other factors such as census, to describe resident physician workload.

We did not find that our workload measure was significantly associated with high-quality AVS completion. There are several possible explanations for this finding. First, adherence to this quality metric may be independent of workload, possibly because it is highly prioritized by residents at our institution. Second, adherence may only be impacted at levels of workload greater than what was experienced by the residents in our study. Finally, electronic order volume may not encompass enough of total workload to be reliably representative of resident work. However, the tight correlation between electronic order volume with severity of illness and level of care, in conjunction with the finding that patients were less likely to receive a timely discharge summary when workload was high on the day prior to a patient’s discharge, suggests that electronic order volume does indeed encompass a meaningful component of workload, and that with higher workload, adherence to some quality metrics may decline. We found that patients who received a timely discharge summary were discharged by teams who entered 30 fewer orders on the day before discharge compared with patients who did not receive a timely discharge summary. In addition to being statistically significant, it is also likely that this difference is clinically significant, although a determination of clinical significance is outside the scope of this study. Further exploration into the relationship between order volume and other quality metrics that are perhaps more sensitive to workload would be interesting.

The primary strength of our study is in how it demonstrates that EHRs can be harnessed to provide additional insights into clinical workload in a quantifiable and automated manner. Although there are a wide range of EHRs currently in use across the country, the capability to track electronic orders is common and could therefore be used broadly across institutions, with tailoring and standardization specific to each site. This technique is similar to that used by prior investigators who characterized the workload of pediatric residents by orders entered and notes written in the electronic medical record.¹⁹ However, our study is unique, in that we explored the relationship between electronic order volume and patient-level severity metrics as well as discharge-related quality metrics.

Our study is limited by several factors. When conceptualizing resident workload, several other elements that contribute to a sense of “busyness” may be independent of electronic orders and were not measured in our study.²⁰ These include communication factors (such as language discordance, discussion with consulting services, and difficult end-of-life discussions), environmental factors (such as geographic localization), resident physician team factors (such as competing clinical or educational responsibilities), timing (in terms of day of week as well as time of year, since residents in July likely feel “busier” than residents in May), and ultimate discharge destination for patients (those going to a skilled nursing facility may require discharge documentation more urgently). Additionally, we chose to focus on the workload of resident teams, as represented by team orders, as opposed to individual work, which may be more directly correlated to our outcomes of interest, completion of a high-quality AVS, and timely discharge summary, which are usually performed by individuals.

Furthermore, we did not measure the relationship between our objective measure of workload and clinical endpoints. Instead, we chose to focus on process measures because they are less likely to be confounded by clinical factors independent of physician workload.²¹ Future studies should also consider obtaining direct resident-level measures of “busyness” or burnout, or other resident-centered endpoints, such as whether residents left the hospital at times consistent with duty hour regulations or whether they were able to attend educational conferences.

These limitations pose opportunities for further efforts to more comprehensively characterize clinical workload. Additional research is needed to understand and quantify the impact of patient, physician, and environmental factors that are not reflected by electronic order volume. Furthermore, an exploration of other electronic surrogates for clinical workload, such as paging volume and other EHR-derived data points, could also prove valuable in further describing the clinical workload. Future studies should also examine whether there is a relationship between these novel markers of workload and further outcomes, including both process measures and clinical endpoints.

Electronic order volume may provide valuable additional information for estimating the workload of resident physicians caring for hospitalized patients. Further investigation to determine whether the statistically significant differences identified in this study are clinically significant, how the technique used in this work may be applied to different EHRs, an examination of other EHR-derived metrics that may represent workload, and an exploration of additional patient-centered outcomes may be warranted.

Disclosures

Rajkomar reports personal fees from Google LLC, outside the submitted work. Dr. Khanna reports that during the conduct of the study, his salary, and the development of CareWeb (a communication platform that includes a smartphone-based paging application in use in several inpatient clinical units at University of California, San Francisco [UCSF] Medical Center) were supported by funding from the Center for Digital Health Innovation at UCSF. The CareWeb software has been licensed by Voalte.

Disclaimer

The views expressed in the submitted article are of the authors and not an official position of the institution.

Electronic consultation (e-consult) in the outpatient setting allows subspecialists to provide assessment and recommendations for patients without in-person visits.¹ An e-consult is an asynchronous communication that uses the electronic medical record (EMR) and typically involves an electronic order from a requesting provider and an electronic note from a consulting provider. The initial motivation for developing this consultation modality was to improve access to subspecialty care for patients in the primary care setting, and findings of studies at several sites support this claim.^1-4 In addition, e-consult may also reduce cost because converting unnecessary face-to-face encounters into e-consults reduces patients’ travel costs and healthcare organizations’ expensive subspecialty clinic time.^3,5 Moreover, instead of addressing less complex clinical questions in informal, undocumented face-to-face or telephone “curbside” consultations with specialists, providers can instead ask for e-consults and thereby ensure thorough chart review and proper documentation.⁶

Use of e-consults in the inpatient setting is relatively novel.⁷ In addition to having the advantages already mentioned, e-consults are faster than in-person bedside consultations and may be beneficial in the fast-moving inpatient care setting. Finally, healthcare systems with multiple hospital sites may not have the capacity to physically locate subspecialists at each site, which makes e-consults attractive for avoiding unnecessary travel time.

In this article, we describe how we developed an inpatient e-consult protocol for a new, remote hospital within our healthcare system and explore data on safety and physician attitudes after e-consult implementation.

The Institutional Review Board of the University of California San Francisco (UCSF) approved this study.

Setting

In February 2015, UCSF opened a new hospital in the Mission Bay neighborhood of San Francisco, 4 miles from the existing hospital. The new hospital is home to several adult inpatient services: urology, otolaryngology, colorectal surgery, obstetrics, and gynecologic surgery. A hospitalist is on-site 24 hours a day to provide consultation for these services around issues that relate to internal medicine. A hospitalist who requires subspecialty expertise to answer a clinical question can request a consultation by in-person visit, video telemedicine, or e-consult, each of which is available 24/7. Almost all of the medicine subspecialists work on the existing campus, not in Mission Bay.

Protocol Development and Implementation

The protocol for the e-consult program was developed over several months by an interdisciplinary group that included 3 hospitalists, 1 obstetrician, 1 project manager, and 1 informaticist. The group outlined the process for requesting and completing an e-consult (Figure), designed a note template for consultants to use for EMR documentation, conducted outreach with subspecialty groups to discuss the protocol, and developed an EMR report to track e-consult use and content over time. As our medical center does not bill payers for inpatient e-consults, e-consult note tracking is used to provide reimbursement internally, from the medical center to the respective departments of the consultants. Reimbursement is made at a set rate per e-consult note, with the rate set to approximate the reimbursement of a low-acuity in-person consult on the main campus.

The workflow of an e-consult is as follows: (1) When a primary team requires a consultation on an issue that falls within the purview of internal medicine, it pages the on-site hospitalist. (2) The hospitalist accepts the consultation by phone, reviews the chart, and examines the patient. (3) If the hospitalist requires subspecialty assistance to answer a clinical question, he or she contacts the appropriate subspecialty service by pager. (4) The subspecialist speaks with the hospitalist about the consultation question, and together they decide if an e-consult is appropriate, based on the complexity of the clinical scenario. (5) The subspecialist reviews the patient’s chart and documents their care plan recommendations in an e-consult note. Consultants can use e-consult for both initial and follow-up assessment, and there is no strict requirement that they also contact the hospitalist or the primary team by phone in addition to consultation note. Given their novelty, almost all e-consults are performed by subspecialist attendings, not residents or fellows.

Evaluation

Each month, we tracked e-consult use using an EMR report built as part of the implementation of the program. For the first four months of implementation, every patient who received an e-consult also had a manual chart review of the period around the e-consult, performed by a hospitalist, in order to audit for any potential safety issues. These issues included, for example, an e-consult performed for a patient whose complexity or severity of illness was felt to be too great to defer an in-person visit, or a patient who received e-consult recommendations that were significantly retracted in a follow-up in-person note.

Eight months after the program started, we assessed experience by electronically surveying the 9 hospitalists and 11 consultants who had requested or performed at least 2 e-consults.⁸ Survey items were measured on a 5-point Likert scale: strongly disagree to strongly agree. The items, which related to ease of calling for a consultation, quality of e-consults, impact on clinical care, safety concerns, and satisfaction, were inspired by themes identified in a systematic review of the literature on e-consults in the outpatient setting.² We sent 2 reminders to responders. Data were summarized using descriptive statistics. Analysis was performed in SPSS version 22.0 (IBM).

There were 143 initial subspecialty consultations by e-consult between program launch in February 2015 and manuscript preparation in February 2016, an average of 11 e-consults per month. There were 313 total e-consult notes (these included both initial and follow-up e-consult notes). By comparison, 240 initial in-person consultations occurred during the same period, and there were 435 total in-person consultation notes (46% new or initial notes, 54% follow-up notes). The top 5 subspecialties by volume of e-consults were infectious disease (35%), hematology (20%), endocrinology (14%), nephrology (13%), and cardiology (8%). For reference, e-consults are also available from psychiatry, neurology, oncology, gastroenterology, pulmonology, and rheumatology. Percentage of consultations performed during daytime hours (defined as 8 a.m. to 5 p.m.) was 92% for e-consults and 96% for in-person consultations.

There were no e-consult–related patient safety issues reported through the medical center’s incident reporting system during the study period. There were also no patient safety issues identified in the manual audits of 80 charts during the first 4 months of the program.

Seven (78%) of 9 hospitalists and 7 (64%) of 11 consultants completed the survey. Both groups agreed that e-consults were easy to use and efficient (Table). All hospitalists were satisfied with the quality of e-consult recommendations, but only 3 (43%) of the 7 consultants agreed they could provide high-quality consultation by e-consult. In their comments, 2 consultants expressed concerns. One concern was about missing crucial information by performing only a chart review, and the other was about being tempted to perform an e-consult simply because it is expedient.

Although use of e-consults in the outpatient setting is relatively commonplace, our program represents a novel use of e-consults in safely and efficiently providing subspecialty consultation to inpatients at a remote hospital.

For hospitalists, an e-consult system offers numerous benefits. Clinical questions beyond an internists’ scope of practice come up often, and simple questions might traditionally result in an informal curbside consult. While a curbside consult provides answers faster than an in-person visit, it creates risks for the requesting hospitalists: the consultants only know what they are told, whether the information is incomplete or erroneous; their opinions are given without documentation or compensation, which reduces a sense of accountability; and the lack of documentation does not allow their advice to persist in the chart as a reference for future providers.⁹ Our e-consult program solves these problems by requiring that consultants perform chart review and provide documentation as well as obligating the medical center to pay a small compensation to consultants for their time. We hope this lowers the bar to requesting consultation for remote sites, where the alternative would be burdensome travel time to do an in-person visit.

In our study, hospitalists were universally pleased with the quality of e-consult recommendations, but only 43% of consultants agreed. These findings correlate with the literature on e-consults in the outpatient setting.² Unfortunately, our survey comments did not shed further light on this sentiment. In the outpatient literature, consultants were most concerned with having a clear clinical question, facing the liability of providing recommendations without performing an examination, and receiving appropriate compensation for answering e-consults.

The generalizability of our program findings is limited most significantly by the particular arrangement of our clinical services: Our remote site is home to a select group of adult inpatient services, a hospitalist is available on-site for these services 24 hours a day, and the distance to the remote site can be overcome with modest effort should a patient require an in-person visit in the initial or follow-up period. The generalizability of our safety findings is limited by the use of a single reviewer for chart auditing.

Given the rise of accountable care organizations and the prevalence of hospital mergers in the healthcare landscape, we believe that healthcare systems that operate remote sites under constrained budgets could look to e-consults to more cost-effectively extend subspecialty expertise across the inpatient enterprise. With improvements in health information exchange, it may also become feasible for consultants to offer e-consults to hospitals outside a medical center’s network. Our study showed that inpatient e-consult programs can be developed and implemented, that they appear not to pose any significant safety issues, and that they can facilitate delivery of timely clinical care.

Acknowledgment

The authors thank Raphaela Levy-Moore for creating and implementing the e-consult note template for our electronic medical record.

Disclosure

Nothing to report.

Electronic consultation (e-consult) in the outpatient setting allows subspecialists to provide assessment and recommendations for patients without in-person visits.¹ An e-consult is an asynchronous communication that uses the electronic medical record (EMR) and typically involves an electronic order from a requesting provider and an electronic note from a consulting provider. The initial motivation for developing this consultation modality was to improve access to subspecialty care for patients in the primary care setting, and findings of studies at several sites support this claim.^1-4 In addition, e-consult may also reduce cost because converting unnecessary face-to-face encounters into e-consults reduces patients’ travel costs and healthcare organizations’ expensive subspecialty clinic time.^3,5 Moreover, instead of addressing less complex clinical questions in informal, undocumented face-to-face or telephone “curbside” consultations with specialists, providers can instead ask for e-consults and thereby ensure thorough chart review and proper documentation.⁶

Use of e-consults in the inpatient setting is relatively novel.⁷ In addition to having the advantages already mentioned, e-consults are faster than in-person bedside consultations and may be beneficial in the fast-moving inpatient care setting. Finally, healthcare systems with multiple hospital sites may not have the capacity to physically locate subspecialists at each site, which makes e-consults attractive for avoiding unnecessary travel time.

In this article, we describe how we developed an inpatient e-consult protocol for a new, remote hospital within our healthcare system and explore data on safety and physician attitudes after e-consult implementation.

The Institutional Review Board of the University of California San Francisco (UCSF) approved this study.

Setting

In February 2015, UCSF opened a new hospital in the Mission Bay neighborhood of San Francisco, 4 miles from the existing hospital. The new hospital is home to several adult inpatient services: urology, otolaryngology, colorectal surgery, obstetrics, and gynecologic surgery. A hospitalist is on-site 24 hours a day to provide consultation for these services around issues that relate to internal medicine. A hospitalist who requires subspecialty expertise to answer a clinical question can request a consultation by in-person visit, video telemedicine, or e-consult, each of which is available 24/7. Almost all of the medicine subspecialists work on the existing campus, not in Mission Bay.

Protocol Development and Implementation

The protocol for the e-consult program was developed over several months by an interdisciplinary group that included 3 hospitalists, 1 obstetrician, 1 project manager, and 1 informaticist. The group outlined the process for requesting and completing an e-consult (Figure), designed a note template for consultants to use for EMR documentation, conducted outreach with subspecialty groups to discuss the protocol, and developed an EMR report to track e-consult use and content over time. As our medical center does not bill payers for inpatient e-consults, e-consult note tracking is used to provide reimbursement internally, from the medical center to the respective departments of the consultants. Reimbursement is made at a set rate per e-consult note, with the rate set to approximate the reimbursement of a low-acuity in-person consult on the main campus.

The workflow of an e-consult is as follows: (1) When a primary team requires a consultation on an issue that falls within the purview of internal medicine, it pages the on-site hospitalist. (2) The hospitalist accepts the consultation by phone, reviews the chart, and examines the patient. (3) If the hospitalist requires subspecialty assistance to answer a clinical question, he or she contacts the appropriate subspecialty service by pager. (4) The subspecialist speaks with the hospitalist about the consultation question, and together they decide if an e-consult is appropriate, based on the complexity of the clinical scenario. (5) The subspecialist reviews the patient’s chart and documents their care plan recommendations in an e-consult note. Consultants can use e-consult for both initial and follow-up assessment, and there is no strict requirement that they also contact the hospitalist or the primary team by phone in addition to consultation note. Given their novelty, almost all e-consults are performed by subspecialist attendings, not residents or fellows.

Evaluation

Each month, we tracked e-consult use using an EMR report built as part of the implementation of the program. For the first four months of implementation, every patient who received an e-consult also had a manual chart review of the period around the e-consult, performed by a hospitalist, in order to audit for any potential safety issues. These issues included, for example, an e-consult performed for a patient whose complexity or severity of illness was felt to be too great to defer an in-person visit, or a patient who received e-consult recommendations that were significantly retracted in a follow-up in-person note.

Eight months after the program started, we assessed experience by electronically surveying the 9 hospitalists and 11 consultants who had requested or performed at least 2 e-consults.⁸ Survey items were measured on a 5-point Likert scale: strongly disagree to strongly agree. The items, which related to ease of calling for a consultation, quality of e-consults, impact on clinical care, safety concerns, and satisfaction, were inspired by themes identified in a systematic review of the literature on e-consults in the outpatient setting.² We sent 2 reminders to responders. Data were summarized using descriptive statistics. Analysis was performed in SPSS version 22.0 (IBM).

There were 143 initial subspecialty consultations by e-consult between program launch in February 2015 and manuscript preparation in February 2016, an average of 11 e-consults per month. There were 313 total e-consult notes (these included both initial and follow-up e-consult notes). By comparison, 240 initial in-person consultations occurred during the same period, and there were 435 total in-person consultation notes (46% new or initial notes, 54% follow-up notes). The top 5 subspecialties by volume of e-consults were infectious disease (35%), hematology (20%), endocrinology (14%), nephrology (13%), and cardiology (8%). For reference, e-consults are also available from psychiatry, neurology, oncology, gastroenterology, pulmonology, and rheumatology. Percentage of consultations performed during daytime hours (defined as 8 a.m. to 5 p.m.) was 92% for e-consults and 96% for in-person consultations.

There were no e-consult–related patient safety issues reported through the medical center’s incident reporting system during the study period. There were also no patient safety issues identified in the manual audits of 80 charts during the first 4 months of the program.

Seven (78%) of 9 hospitalists and 7 (64%) of 11 consultants completed the survey. Both groups agreed that e-consults were easy to use and efficient (Table). All hospitalists were satisfied with the quality of e-consult recommendations, but only 3 (43%) of the 7 consultants agreed they could provide high-quality consultation by e-consult. In their comments, 2 consultants expressed concerns. One concern was about missing crucial information by performing only a chart review, and the other was about being tempted to perform an e-consult simply because it is expedient.

Although use of e-consults in the outpatient setting is relatively commonplace, our program represents a novel use of e-consults in safely and efficiently providing subspecialty consultation to inpatients at a remote hospital.

For hospitalists, an e-consult system offers numerous benefits. Clinical questions beyond an internists’ scope of practice come up often, and simple questions might traditionally result in an informal curbside consult. While a curbside consult provides answers faster than an in-person visit, it creates risks for the requesting hospitalists: the consultants only know what they are told, whether the information is incomplete or erroneous; their opinions are given without documentation or compensation, which reduces a sense of accountability; and the lack of documentation does not allow their advice to persist in the chart as a reference for future providers.⁹ Our e-consult program solves these problems by requiring that consultants perform chart review and provide documentation as well as obligating the medical center to pay a small compensation to consultants for their time. We hope this lowers the bar to requesting consultation for remote sites, where the alternative would be burdensome travel time to do an in-person visit.

In our study, hospitalists were universally pleased with the quality of e-consult recommendations, but only 43% of consultants agreed. These findings correlate with the literature on e-consults in the outpatient setting.² Unfortunately, our survey comments did not shed further light on this sentiment. In the outpatient literature, consultants were most concerned with having a clear clinical question, facing the liability of providing recommendations without performing an examination, and receiving appropriate compensation for answering e-consults.

The generalizability of our program findings is limited most significantly by the particular arrangement of our clinical services: Our remote site is home to a select group of adult inpatient services, a hospitalist is available on-site for these services 24 hours a day, and the distance to the remote site can be overcome with modest effort should a patient require an in-person visit in the initial or follow-up period. The generalizability of our safety findings is limited by the use of a single reviewer for chart auditing.

Given the rise of accountable care organizations and the prevalence of hospital mergers in the healthcare landscape, we believe that healthcare systems that operate remote sites under constrained budgets could look to e-consults to more cost-effectively extend subspecialty expertise across the inpatient enterprise. With improvements in health information exchange, it may also become feasible for consultants to offer e-consults to hospitals outside a medical center’s network. Our study showed that inpatient e-consult programs can be developed and implemented, that they appear not to pose any significant safety issues, and that they can facilitate delivery of timely clinical care.

Acknowledgment

The authors thank Raphaela Levy-Moore for creating and implementing the e-consult note template for our electronic medical record.

Disclosure

Nothing to report.

Electronic consultation (e-consult) in the outpatient setting allows subspecialists to provide assessment and recommendations for patients without in-person visits.¹ An e-consult is an asynchronous communication that uses the electronic medical record (EMR) and typically involves an electronic order from a requesting provider and an electronic note from a consulting provider. The initial motivation for developing this consultation modality was to improve access to subspecialty care for patients in the primary care setting, and findings of studies at several sites support this claim.^1-4 In addition, e-consult may also reduce cost because converting unnecessary face-to-face encounters into e-consults reduces patients’ travel costs and healthcare organizations’ expensive subspecialty clinic time.^3,5 Moreover, instead of addressing less complex clinical questions in informal, undocumented face-to-face or telephone “curbside” consultations with specialists, providers can instead ask for e-consults and thereby ensure thorough chart review and proper documentation.⁶

Use of e-consults in the inpatient setting is relatively novel.⁷ In addition to having the advantages already mentioned, e-consults are faster than in-person bedside consultations and may be beneficial in the fast-moving inpatient care setting. Finally, healthcare systems with multiple hospital sites may not have the capacity to physically locate subspecialists at each site, which makes e-consults attractive for avoiding unnecessary travel time.

In this article, we describe how we developed an inpatient e-consult protocol for a new, remote hospital within our healthcare system and explore data on safety and physician attitudes after e-consult implementation.

The Institutional Review Board of the University of California San Francisco (UCSF) approved this study.

Setting

In February 2015, UCSF opened a new hospital in the Mission Bay neighborhood of San Francisco, 4 miles from the existing hospital. The new hospital is home to several adult inpatient services: urology, otolaryngology, colorectal surgery, obstetrics, and gynecologic surgery. A hospitalist is on-site 24 hours a day to provide consultation for these services around issues that relate to internal medicine. A hospitalist who requires subspecialty expertise to answer a clinical question can request a consultation by in-person visit, video telemedicine, or e-consult, each of which is available 24/7. Almost all of the medicine subspecialists work on the existing campus, not in Mission Bay.

Protocol Development and Implementation

The protocol for the e-consult program was developed over several months by an interdisciplinary group that included 3 hospitalists, 1 obstetrician, 1 project manager, and 1 informaticist. The group outlined the process for requesting and completing an e-consult (Figure), designed a note template for consultants to use for EMR documentation, conducted outreach with subspecialty groups to discuss the protocol, and developed an EMR report to track e-consult use and content over time. As our medical center does not bill payers for inpatient e-consults, e-consult note tracking is used to provide reimbursement internally, from the medical center to the respective departments of the consultants. Reimbursement is made at a set rate per e-consult note, with the rate set to approximate the reimbursement of a low-acuity in-person consult on the main campus.

The workflow of an e-consult is as follows: (1) When a primary team requires a consultation on an issue that falls within the purview of internal medicine, it pages the on-site hospitalist. (2) The hospitalist accepts the consultation by phone, reviews the chart, and examines the patient. (3) If the hospitalist requires subspecialty assistance to answer a clinical question, he or she contacts the appropriate subspecialty service by pager. (4) The subspecialist speaks with the hospitalist about the consultation question, and together they decide if an e-consult is appropriate, based on the complexity of the clinical scenario. (5) The subspecialist reviews the patient’s chart and documents their care plan recommendations in an e-consult note. Consultants can use e-consult for both initial and follow-up assessment, and there is no strict requirement that they also contact the hospitalist or the primary team by phone in addition to consultation note. Given their novelty, almost all e-consults are performed by subspecialist attendings, not residents or fellows.

Evaluation

Each month, we tracked e-consult use using an EMR report built as part of the implementation of the program. For the first four months of implementation, every patient who received an e-consult also had a manual chart review of the period around the e-consult, performed by a hospitalist, in order to audit for any potential safety issues. These issues included, for example, an e-consult performed for a patient whose complexity or severity of illness was felt to be too great to defer an in-person visit, or a patient who received e-consult recommendations that were significantly retracted in a follow-up in-person note.

Eight months after the program started, we assessed experience by electronically surveying the 9 hospitalists and 11 consultants who had requested or performed at least 2 e-consults.⁸ Survey items were measured on a 5-point Likert scale: strongly disagree to strongly agree. The items, which related to ease of calling for a consultation, quality of e-consults, impact on clinical care, safety concerns, and satisfaction, were inspired by themes identified in a systematic review of the literature on e-consults in the outpatient setting.² We sent 2 reminders to responders. Data were summarized using descriptive statistics. Analysis was performed in SPSS version 22.0 (IBM).

There were 143 initial subspecialty consultations by e-consult between program launch in February 2015 and manuscript preparation in February 2016, an average of 11 e-consults per month. There were 313 total e-consult notes (these included both initial and follow-up e-consult notes). By comparison, 240 initial in-person consultations occurred during the same period, and there were 435 total in-person consultation notes (46% new or initial notes, 54% follow-up notes). The top 5 subspecialties by volume of e-consults were infectious disease (35%), hematology (20%), endocrinology (14%), nephrology (13%), and cardiology (8%). For reference, e-consults are also available from psychiatry, neurology, oncology, gastroenterology, pulmonology, and rheumatology. Percentage of consultations performed during daytime hours (defined as 8 a.m. to 5 p.m.) was 92% for e-consults and 96% for in-person consultations.

There were no e-consult–related patient safety issues reported through the medical center’s incident reporting system during the study period. There were also no patient safety issues identified in the manual audits of 80 charts during the first 4 months of the program.

Seven (78%) of 9 hospitalists and 7 (64%) of 11 consultants completed the survey. Both groups agreed that e-consults were easy to use and efficient (Table). All hospitalists were satisfied with the quality of e-consult recommendations, but only 3 (43%) of the 7 consultants agreed they could provide high-quality consultation by e-consult. In their comments, 2 consultants expressed concerns. One concern was about missing crucial information by performing only a chart review, and the other was about being tempted to perform an e-consult simply because it is expedient.

Although use of e-consults in the outpatient setting is relatively commonplace, our program represents a novel use of e-consults in safely and efficiently providing subspecialty consultation to inpatients at a remote hospital.

For hospitalists, an e-consult system offers numerous benefits. Clinical questions beyond an internists’ scope of practice come up often, and simple questions might traditionally result in an informal curbside consult. While a curbside consult provides answers faster than an in-person visit, it creates risks for the requesting hospitalists: the consultants only know what they are told, whether the information is incomplete or erroneous; their opinions are given without documentation or compensation, which reduces a sense of accountability; and the lack of documentation does not allow their advice to persist in the chart as a reference for future providers.⁹ Our e-consult program solves these problems by requiring that consultants perform chart review and provide documentation as well as obligating the medical center to pay a small compensation to consultants for their time. We hope this lowers the bar to requesting consultation for remote sites, where the alternative would be burdensome travel time to do an in-person visit.

In our study, hospitalists were universally pleased with the quality of e-consult recommendations, but only 43% of consultants agreed. These findings correlate with the literature on e-consults in the outpatient setting.² Unfortunately, our survey comments did not shed further light on this sentiment. In the outpatient literature, consultants were most concerned with having a clear clinical question, facing the liability of providing recommendations without performing an examination, and receiving appropriate compensation for answering e-consults.

The generalizability of our program findings is limited most significantly by the particular arrangement of our clinical services: Our remote site is home to a select group of adult inpatient services, a hospitalist is available on-site for these services 24 hours a day, and the distance to the remote site can be overcome with modest effort should a patient require an in-person visit in the initial or follow-up period. The generalizability of our safety findings is limited by the use of a single reviewer for chart auditing.

Given the rise of accountable care organizations and the prevalence of hospital mergers in the healthcare landscape, we believe that healthcare systems that operate remote sites under constrained budgets could look to e-consults to more cost-effectively extend subspecialty expertise across the inpatient enterprise. With improvements in health information exchange, it may also become feasible for consultants to offer e-consults to hospitals outside a medical center’s network. Our study showed that inpatient e-consult programs can be developed and implemented, that they appear not to pose any significant safety issues, and that they can facilitate delivery of timely clinical care.

Acknowledgment

The authors thank Raphaela Levy-Moore for creating and implementing the e-consult note template for our electronic medical record.

Disclosure

Nothing to report.