Affiliations
Department of Medicine, Northwestern University Feinberg School of Medicine, Chicago, Illinois
Given name(s)
Joseph
Family name
Feinglass
Degrees
PhD

Night or Weekend Admission and Outcomes

Article Type
Changed
Thu, 05/25/2017 - 21:52
Display Headline
The association between night or weekend admission and hospitalization‐relevant patient outcomes

The hospitalist movement and increasingly stringent resident work hour restrictions have led to the utilization of shift work in many hospitals.1 Use of nocturnist and night float systems, while often necessary, results in increased patient hand‐offs. Research suggests that hand‐offs in the inpatient setting can adversely affect patient outcomes as lack of continuity may increase the possibility of medical error.2, 3 In 2001, Bell et al.4 found that mortality was higher among patients admitted on weekends as compared to weekdays. Uneven staffing, lack of supervision, and fragmented care were cited as potential contributing factors.4 Similarly, Peberdy et al.5 in 2008 revealed that patients were less likely to survive a cardiac arrest if it occurred at night or on weekends, again attributed in part to fragmented patient care and understaffing.

The results of these studies raise concerns as to whether increased reliance on shift work and resulting handoffs compromises patient care.6, 7 The aim of this study was to evaluate the potential association between night admission and hospitalization‐relevant outcomes (length of stay [LOS], hospital charges, intensive care unit [ICU] transfer during hospitalization, repeat emergency department [ED] visit within 30 days of discharge, readmission within 30 days of discharge, and poor outcome [transfer to the ICU, cardiac arrest, or death] within the first 24 hours of admission) at an institution that exclusively uses nocturnists (night‐shift based hospitalists) and a resident night float system for patients admitted at night to the general medicine service. A secondary aim was to determine the potential association between weekend admission and hospitalization‐relevant outcomes.

Methods

Study Sample and Selection

We conducted a retrospective medical record review at a large urban academic hospital. Using an administrative hospital data set, we assembled a list of approximately 9000 admissions to the general medicine service from the ED between January 2008 and October 2008. We sampled consecutive admissions from 3 distinct periods beginning in January, April, and July to capture outcomes at various points in the academic year. We attempted to review approximately 10% of all charts equally distributed among the 3 sampling periods (ie, 900 charts total with one‐third from each period) based on time available to the reviewers. We excluded patients not admitted to the general medicine service and patients without complete demographic or outcome information. We also excluded patients not admitted from the ED given that the vast majority of admissions to our hospital during the night (96%) or weekend (93%) are from the ED. Patients admitted to the general medicine service are cared for either by a hospitalist or by a teaching team comprised of 1 attending (about 40% of whom are hospitalists), 1 resident, 1 to 2 interns, and 1 to 3 medical students. From 7 am to 6:59 pm patients are admitted to the care of 1 of the primary daytime admitting teams. From 7 pm to 6:59 am patients are admitted by nocturnists (hospitalist service) or night float residents (teaching service). These patients are handed off to day teams at 7 am. Hospitalist teams change service on a weekly to biweekly basis and resident teams switch on a monthly basis; there is no difference in physician staffing between the weekend and weekdays. The Northwestern University Institutional Review Board approved this study.

Data Acquisition and Medical Records Reviews

We obtained demographic data including gender, age, race and ethnicity, patient insurance, admission day (weekday vs. weekend), admission time (defined as the time that a patient receives a hospital bed, which at our institution is also the time that admitting teams receive report and assume care for the patient), and the International Classification of Disease codes required to determine the Major Diagnostic Category (MDC) and calculate the Charlson Comorbidity Index8, 9 as part of an administrative data set. We divided the admission time into night admission (defined as 7 pm to 6:59 am) and day admission (defined as 7:00 am to 6:59 pm). We created a chart abstraction tool to allow manual recording of the additional fields of admitting team (hospitalist vs. resident), 30 day repeat ED visit, 30 day readmission, and poor outcomes within the first 24 hours of admission, directly from the electronic record.

Study Outcomes

We evaluated each admission for the following 6 primary outcomes which were specified a priori: LOS (defined as discharge date and time minus admission date and time), hospital charges (defined as charges billed as recorded in the administrative data set), ICU transfer during hospitalization (defined as 1 ICU day in the administrative data set), 30 day repeat ED visit (defined as a visit to our ED within 30 days of discharge as assessed by chart abstraction), 30 day readmission (defined as any planned or unplanned admission to any inpatient service at our institution within 30 days of discharge as assessed by chart abstraction), and poor outcome within 24 hours of admission (defined as transfer to the ICU, cardiac arrest, or death as assessed by chart abstraction). Each of these outcomes has been used in prior work to assess the quality of inpatient care.10, 11

Statistical Analysis

Interrater reliability between the 3 physician reviewers was assessed for 20 randomly selected admissions across the 4 separate review measures using interclass correlation coefficients. Comparisons between night admissions and day admissions, and between weekend and weekday admissions, for the continuous primary outcomes (LOS, hospital charges) were assessed using 2‐tailed t‐tests as well as Wilcoxon rank sum test. In the multivariable modeling, these outcomes were assessed by linear regression controlling for age, gender, race and ethnicity, Medicaid or self‐pay insurance, admission to the hospitalist or teaching service, most common MDC categories, and Charlson Comorbidity Index. Because both outcomes were right‐skewed, we separately assessed each after log‐transformation controlling for the same variables.

All comparisons of the dichotomous primary outcomes (ICU transfer during hospitalization, 30 day repeat ED visit, 30 day readmission, and poor outcome within the first 24 hours after admission) were assessed at the univariate level by chi‐squared test, and in the multivariable models using logistic regression, controlling for the same variables as the linear models above. All adjustments were specified a priori. All data analyses were conducted using Stata (College Station, TX; Version 11).

Results

We reviewed 857 records. After excluding 33 records lacking administrative data regarding gender, race and ethnicity, and other demographic variables, there were 824 medical records available for analysis. We reviewed a similar number of records from each time period: 274 from January 2008, 265 from April 2008, and 285 from July 2008. A total of 345 (42%) patients were admitted during the day, and 479 (58%) at night; 641 (78%) were admitted on weekdays, and 183 (22%) on weekends. The 33 excluded charts were similar to the included charts for both time of admission and outcomes. Results for parametric testing and nonparametric testing, as well as for log‐transformation and non‐log‐transformation of the continuous outcomes were similar in both magnitude and statistical significance, so we present the parametric and nonlog‐transformed results below for ease of interpretation.

Interrater reliability among the 3 reviewers was very high. There were no disagreements among the 20 multiple reviews for either poor outcomes within 24 hours of admission or admitting service; the interclass correlation coefficients for 30 day repeat ED visit and 30 day readmission were 0.97 and 0.87, respectively.

Patients admitted at night or on the weekend were similar to patients admitted during the day and week across age, gender, insurance class, MDC, and Charlson Comorbidity Index (Table 1). For unadjusted outcomes, patients admitted at night has a similar LOS, hospital charges, 30 day repeat ED visits, 30 day readmissions, and poor outcome within 24 hours of admission as those patients admitted during the day. They had a potentially lower chance of any ICU transfer during hospitalization though this did not reach statistical significance at P < 0.05 (night admission 6%, day admission 3%, P = 0.06) (Table 2).

Baseline Characteristics of Patients
CharacteristicsTime of DayDay of the Week
Day Admission (n = 345)Night Admission (n = 479)Weekday Admission (n = 641)Weekend Admission (n = 183)
  • NOTE: All P values > 0.05.

  • Abbreviation: ED, emergency department.

Age (years)60.859.760.658.7
Gender (% male)47434546
Race/Ethnicity (%)
White, Asian, other61545755
Black34383734
Hispanic58610
Medicaid or self pay (%)9101011
Major diagnostic category (%)
Respiratory disease14131413
Circulatory disease28232624
Digestive disease12121212
Other45524851
Charlson Comorbidity Index3.713.603.663.60
Outcomes, Unadjusted
OutcomesTime of DayDay of the Week
Day Admission (n = 345)Night Admission (n = 479)Weekday Admission (n = 641)Weekend Admission (n = 183)
  • Abbreviations: ED, emergency department; ICU, intensive care unit.

  • P < 0.05.

  • P= 0.06.

Length of stay4.34.14.33.8
Hospital charges$27,500$25,200$27,200*$22,700*
ICU transfer during hospitalization (%)635*1*
Repeat ED visit at 30 days (%)20222221
Readmission at 30 days (%)17202017
Poor outcome at 24 hours (ICU transfer, cardiac arrest, or death)(%)2121

Patients admitted to the hospital during the weekend were similar to patients admitted during the week for unadjusted LOS, 30 day repeat ED visit or readmission rate, and poor outcomes within 24 hours of admission as those admitted during the week; however, they had lower hospital charges (weekend admission $22,700, weekday admission $27,200; P = 0.02), and a lower chance of ICU transfer during hospitalization (weekend admission 1%, weekday admission 5%; P = 0.02) (Table 2).

In the multivariable linear and logistic regression models (Tables 3 and 4), we assessed the independent association between night admission or weekend admission and each hospitalization‐relevant outcome except for poor outcome within 24 hours of admission (poor outcome within 24 hours of admission was not modeled to avoid the risk of overfitting because there were only 13 total events). After adjustment for age, gender, race and ethnicity, admitting service (hospitalist or teaching), Medicaid or self‐pay insurance, MDC, and Charlson Comorbidity Index, there was no statistically significant association between night admission and worse outcomes for LOS, hospital charges, 30 day repeat ED visit, or 30 day readmission. Night admission was associated with a decreased chance of ICU transfer during hospitalization, but the difference was not statistically significant (odds ratio, 0.54; 95% confidence interval [CI], 0.26‐1.11, P = 0.09). Weekend admission was not associated with worse outcomes for LOS or 30 day repeat ED visit or readmission; however, weekend admission was associated with a decrease in overall charges ($4400; 95% CI, $8300 to $600) and a decreased chance of ICU transfer during hospitalization (odds ratio, 0.20; 95% CI, 0.050.88).

Linear Regressions for Continuous Outcomes (With Coefficients)
PredictorsLength of Stay (days), Coefficient (95% CI)Hospital Charges (dollars), Coefficient (95% CI)
  • Abbreviations: CI, confidence intervals; ICU, intensive care unit; MDC, major diagnostic category: comparison to other.

  • P < 0.05.

Night admission0.23 (0.77 to 0.32)2100 (5400 to 1100)
Weekend admission0.42 (1.07 to 0.23)4400 (8300 to 600)*
Age0.01 (0.01 to 0.03)0 (100 to 100)
Male gender0.15 (0.70 to 0.39)400 (3700 to 2800)
Race, Black0.18 (0.41 to 0.78)200 (3700 to 3400)
Ethnicity, Hispanic0.62 (1.73 to 0.49)2300 (8900 to 4300)
Medicaid or self‐pay insurance1.87 (0.93 to 2.82)*8900 (3300 to 14600)*
Hospitalist service0.26 (0.29 to 0.81)600 (3900 to 2700)
MDC: respiratory0.36 (1.18 to 0.46)700 (4200 to 5600)
MDC: circulatory1.36 (2.04 to 0.68)*600 (4600 to 3400)
MDC: digestive1.22 (2.08 to 0.35)*6800 (12000 to 1700)*
Charlson Comorbidity Index0.35 (0.22 to 0.49)*2200 (1400 to 3000)*
Logistic Regressions for Dichotomous Outcomes (With Odds Ratios)
PredictorsICU Transfer during Hospitalization, Odds Ratio (95% CI)Repeat ED Visit at 30 days, Odds Ratio (95% CI)Readmission at 30 days, Odds Ratio (95% CI)
  • Abbreviations: CI, confidence intervals; ICU, intensive care unit; MDC, major diagnostic category: comparison to other.

  • P < 0.05.

Night admission0.53 (0.26 to 1.11)1.13 (0.80 to 1.60)1.23 (0.86 to 1.78)
Weekend admission0.20 (0.05 to 0.88)*0.95 (0.63 to 1.44)0.80 (0.51 to 1.25)
Age1.00 (0.98 to 1.02)0.99 (0.98 to 1.002)1.00 (0.99 to 1.01)
Male gender0.98 (0.47 to 2.02)1.09 (0.78 to 1.54)0.91 (0.64 to 1.31)
Race, Black0.75 (0.33 to 1.70)1.48 (1.02 to 2.14)*1.12 (0.76 to 1.65)
Ethnicity, Hispanic0.76 (0.16 to 3.73)1.09 (0.55 to 2.17)1.11 (0.55 to 2.22)
Medicaid or self‐pay insurance0.75 (0.16 to 3.49)1.61 (0.95 to 2.72)2.14 (1.24 to 3.67)*
Hospitalist service0.68 (0.33 to 1.44)1.15 (0.81 to 1.63)0.99 (0.69 to 1.43)
MDC: respiratory1.18 (0.41 to 3.38)1.02 (0.61 to 1.69)1.16 (0.69 to 1.95)
MDC: circulatory1.22 (0.52 to 2.87)0.79 (0.51 to 1.22)0.80 (0.51 to 1.27)
MDC: digestive0.51 (0.11 to 2.32)0.83 (0.47 to 1.46)1.08 (0.62 to 1.91)
Charlson Comobrbidity Index1.25 (1.09 to 1.45)*1.09 (1.01 to 1.19)*1.11 (1.02 to 1.21)*

Our multivariate models explained very little of the variance in patient outcomes. For LOS and hospital charges, adjusted R2 values were 0.06 and 0.05, respectively. For ICU transfer during hospitalization, 30 day repeat ED visit, and 30 day readmission, the areas under the receiver operator curves were 0.75, 0.51, and 0.61 respectively.

To assess the robustness of our conclusions regarding night admission, we redefined night to include only patients admitted between the hours of 8 pm and 5:59 am. This did not change our conclusions. We also tested for interaction between night admission and weekend admission for all outcomes to assess whether night admissions on the weekend were in fact at increased risk of worse outcomes; we found no evidence of interaction (P > 0.3 for the interaction terms in each model).

Discussion

Among patients admitted to the medicine services at our academic medical center, night or weekend admission was not associated with worse hospitalization‐relevant outcomes. In some cases, night or weekend admission was associated with better outcomes, particularly in terms of ICU transfer during hospitalization and hospital charges. Prior research indicates worse outcomes during off‐hours,5 but we did not replicate this finding in our study.

The finding that admission at night was not associated with worse outcomes, particularly proximal outcomes such as LOS or ICU transfer during hospitalization, was surprising, though reassuring in view of the fact that more than half of our patients are admitted at night. We believe a few factors may be responsible. First, our general medicine service is staffed during the night (7 pm to 7 am) by in‐house nocturnists and night float residents. Second, our staffing ratio, while lower at night than during the day, remains the same on weekends and may be higher than in other settings. In continuously well‐staffed settings such as the ED12 and ICU,13 night and weekend admissions are only inconsistently associated with worse outcomes, which may be the same phenomena we observed in the current study. Third, the hospital used as the site of this study has received Nursing Magnet recognition and numerous quality awards such as the National Research Corporation's Consumer Choice Award and recognition as a Distinguished Hospital for Clinical Excellence by HealthGrades. Fourth, our integrated electronic medical record, computerized physician order entry system, and automatically generated sign out serve as complements to the morning hand off. Fifth, hospitalists and teaching teams rotate on a weekly, biweekly, or every 4 week basis, which may protect against discontinuity associated with the weekend. We believe that all of these factors may facilitate alert, comprehensive care during the night and weekend as well as safe and efficient transfer of patients from the night to the day providers.

We were also surprised by the association between weekend admission and lower charges and a lower chance of ICU transfer during hospitalization. We believe many of the same factors noted above may have played a role in these findings. In terms of hospital charges, it is possible that some workups were completed outside of the hospital rather than during the hospitalization, and that some tests were not ordered at all due to unavailability on weekends. The decreased chance of ICU transfer is unexplained. We hypothesize that there may have been a more conservative admission strategy within the ED, such that patients with high baseline severity were admitted directly to the ICU on the weekend rather than being admitted first to the general medicine floor. This hypothesis requires further study.

Our study had important limitations. It was a retrospective study from a single academic hospital. The sample size lacked sufficient power to detect differences in the low frequency of certain outcomes such as poor outcomes within 24 hours of admission (2% vs. 1%), and also for more frequent outcomes such as 30 day readmission; it is possible that with a larger sample there would have been statistically significant differences. Further, we recognize that the Charlson Comorbidity Index, which was developed to predict 1‐year mortality for medicine service patients, does not adjust for severity of illness at presentation, particularly for outcomes such as readmission. If patients admitted at night and during the weekend were less acutely ill despite having similar comorbidities and MDCs at admission, true associations between time of admission and worse outcomes could have been masked. Furthermore, the multivariable modeling explained very little of the variance in patient outcomes such that significant unmeasured confounding may still be present, and consequently our results cannot be interpreted in a causal way. Data was collected from electronic records, so it is possible that some adverse events were not recorded. However, it seems unlikely that major events such as death and transfer to an ICU would have been missed.

Several aspects of the study strengthen our confidence in the findings, including a large sample size, relevance of the outcomes, the adjustment for confounders, and an assessment for robustness of the conclusions based on restricting the definition of night and also testing for interaction between night and weekend admission. Our patient demographics and insurance mix resemble that of other academic hospitals,10 and perhaps our results may be generalizable to these settings, if not to non‐urban or community hospitals. Furthermore, the Charlson Comorbidity Index was associated with all 5 of the modeled outcomes we chose for our study, reaffirming their utility in assessing the quality of hospital care. Future directions for investigation may include examining the association of night admission with hospitalization‐relevant outcomes in nonacademic, nonurban settings, and examining whether the lack of association between night and weekend admission and worse outcomes persists with adjustment for initial severity of illness.

In summary, at a large, well‐staffed urban academic hospital, day or time of admission were not associated with worse hospitalization‐relevant outcomes. The use of nocturnists and night float teams for night admissions and continuity across weekends appears to be a safe approach to handling the increased volume of patients admitted at night, and a viable alternative to overnight call in the era of work hour restrictions.

References
  1. Vaughn DM,Stout CL,McCampbell BL, et al.Three‐year results of mandated work hour restrictions: attending and resident perspectives and effects in a community hospital.Am Surg.2008;74(6):542546; discussion 546–547.
  2. Kitch BT,Cooper JB,Zapol WM, et al.Handoffs causing patient harm: a survey of medical and surgical house staff.Jt Comm J Qual Patient Saf.2008;34(10):563570.
  3. Petersen LA,Brennan TA,O'Neil AC,Cook EF,Lee TH.Does housestaff discontinuity of care increase the risk for preventable adverse events?Ann Intern Med.1994;121(11):866872.
  4. Bell CM,Redelmeier DA.Mortality among patients admitted to hospitals on weekends as compared with weekdays.N Engl J Med.2001;345(9):663668.
  5. Peberdy MA,Ornato JP,Larkin GL, et al.Survival from in‐hospital cardiac arrest during nights and weekends.JAMA.2008;299(7):785792.
  6. Sharma G,Freeman J,Zhang D,Goodwin JS.Continuity of care and intensive care unit use at the end of life.Arch Intern Med.2009;169(1):8186.
  7. Sharma G,Fletcher KE,Zhang D,Kuo YF,Freeman JL,Goodwin JS.Continuity of outpatient and inpatient care by primary care physicians for hospitalized older adults.JAMA.2009;301(16):16711680.
  8. Charlson ME,Ales KL,Simon R,MacKenzie CR.Why predictive indexes perform less well in validation studies: is it magic or methods?Arch Intern Med.1987;147:21552161.
  9. Deyo RA,Cherkin DC,Ciol MA.Adapting a clinical comorbidity index for use with ICD‐9‐CM administrative databases.J Clin Epidemiol.1992;45(6):613619.
  10. Roy CL,Liang CL,Lund M, et al.Implementation of a physician assistant/hospitalist service in an academic medical center: impact on efficiency and patient outcomes.J Hosp Med.2008;3(5):361368.
  11. Groarke JD,Gallagher J,Stack J, et al.Use of an admission early warning score to predict patient morbidity and mortality and treatment success.Emerg Med J.2008;25(12):803806.
  12. Schmulewitz L,Proudfoot A,Bell D.The impact of weekends on outcome for emergency patients.Clin Med.2005;5(6):621625.
  13. Meynaar IA,van der Spoel JI,Rommes JH,van Spreuwel‐Verheijen M,Bosman RJ,Spronk PE.Off hour admission to an intensivist‐led ICU is not associated with increased mortality.Crit Care.2009;13(3):R84.
Article PDF
Issue
Journal of Hospital Medicine - 6(1)
Publications
Page Number
10-14
Legacy Keywords
communication, continuity of care transition and discharge planning, education, outcomes measurement, patient safety, resident
Sections
Article PDF
Article PDF

The hospitalist movement and increasingly stringent resident work hour restrictions have led to the utilization of shift work in many hospitals.1 Use of nocturnist and night float systems, while often necessary, results in increased patient hand‐offs. Research suggests that hand‐offs in the inpatient setting can adversely affect patient outcomes as lack of continuity may increase the possibility of medical error.2, 3 In 2001, Bell et al.4 found that mortality was higher among patients admitted on weekends as compared to weekdays. Uneven staffing, lack of supervision, and fragmented care were cited as potential contributing factors.4 Similarly, Peberdy et al.5 in 2008 revealed that patients were less likely to survive a cardiac arrest if it occurred at night or on weekends, again attributed in part to fragmented patient care and understaffing.

The results of these studies raise concerns as to whether increased reliance on shift work and resulting handoffs compromises patient care.6, 7 The aim of this study was to evaluate the potential association between night admission and hospitalization‐relevant outcomes (length of stay [LOS], hospital charges, intensive care unit [ICU] transfer during hospitalization, repeat emergency department [ED] visit within 30 days of discharge, readmission within 30 days of discharge, and poor outcome [transfer to the ICU, cardiac arrest, or death] within the first 24 hours of admission) at an institution that exclusively uses nocturnists (night‐shift based hospitalists) and a resident night float system for patients admitted at night to the general medicine service. A secondary aim was to determine the potential association between weekend admission and hospitalization‐relevant outcomes.

Methods

Study Sample and Selection

We conducted a retrospective medical record review at a large urban academic hospital. Using an administrative hospital data set, we assembled a list of approximately 9000 admissions to the general medicine service from the ED between January 2008 and October 2008. We sampled consecutive admissions from 3 distinct periods beginning in January, April, and July to capture outcomes at various points in the academic year. We attempted to review approximately 10% of all charts equally distributed among the 3 sampling periods (ie, 900 charts total with one‐third from each period) based on time available to the reviewers. We excluded patients not admitted to the general medicine service and patients without complete demographic or outcome information. We also excluded patients not admitted from the ED given that the vast majority of admissions to our hospital during the night (96%) or weekend (93%) are from the ED. Patients admitted to the general medicine service are cared for either by a hospitalist or by a teaching team comprised of 1 attending (about 40% of whom are hospitalists), 1 resident, 1 to 2 interns, and 1 to 3 medical students. From 7 am to 6:59 pm patients are admitted to the care of 1 of the primary daytime admitting teams. From 7 pm to 6:59 am patients are admitted by nocturnists (hospitalist service) or night float residents (teaching service). These patients are handed off to day teams at 7 am. Hospitalist teams change service on a weekly to biweekly basis and resident teams switch on a monthly basis; there is no difference in physician staffing between the weekend and weekdays. The Northwestern University Institutional Review Board approved this study.

Data Acquisition and Medical Records Reviews

We obtained demographic data including gender, age, race and ethnicity, patient insurance, admission day (weekday vs. weekend), admission time (defined as the time that a patient receives a hospital bed, which at our institution is also the time that admitting teams receive report and assume care for the patient), and the International Classification of Disease codes required to determine the Major Diagnostic Category (MDC) and calculate the Charlson Comorbidity Index8, 9 as part of an administrative data set. We divided the admission time into night admission (defined as 7 pm to 6:59 am) and day admission (defined as 7:00 am to 6:59 pm). We created a chart abstraction tool to allow manual recording of the additional fields of admitting team (hospitalist vs. resident), 30 day repeat ED visit, 30 day readmission, and poor outcomes within the first 24 hours of admission, directly from the electronic record.

Study Outcomes

We evaluated each admission for the following 6 primary outcomes which were specified a priori: LOS (defined as discharge date and time minus admission date and time), hospital charges (defined as charges billed as recorded in the administrative data set), ICU transfer during hospitalization (defined as 1 ICU day in the administrative data set), 30 day repeat ED visit (defined as a visit to our ED within 30 days of discharge as assessed by chart abstraction), 30 day readmission (defined as any planned or unplanned admission to any inpatient service at our institution within 30 days of discharge as assessed by chart abstraction), and poor outcome within 24 hours of admission (defined as transfer to the ICU, cardiac arrest, or death as assessed by chart abstraction). Each of these outcomes has been used in prior work to assess the quality of inpatient care.10, 11

Statistical Analysis

Interrater reliability between the 3 physician reviewers was assessed for 20 randomly selected admissions across the 4 separate review measures using interclass correlation coefficients. Comparisons between night admissions and day admissions, and between weekend and weekday admissions, for the continuous primary outcomes (LOS, hospital charges) were assessed using 2‐tailed t‐tests as well as Wilcoxon rank sum test. In the multivariable modeling, these outcomes were assessed by linear regression controlling for age, gender, race and ethnicity, Medicaid or self‐pay insurance, admission to the hospitalist or teaching service, most common MDC categories, and Charlson Comorbidity Index. Because both outcomes were right‐skewed, we separately assessed each after log‐transformation controlling for the same variables.

All comparisons of the dichotomous primary outcomes (ICU transfer during hospitalization, 30 day repeat ED visit, 30 day readmission, and poor outcome within the first 24 hours after admission) were assessed at the univariate level by chi‐squared test, and in the multivariable models using logistic regression, controlling for the same variables as the linear models above. All adjustments were specified a priori. All data analyses were conducted using Stata (College Station, TX; Version 11).

Results

We reviewed 857 records. After excluding 33 records lacking administrative data regarding gender, race and ethnicity, and other demographic variables, there were 824 medical records available for analysis. We reviewed a similar number of records from each time period: 274 from January 2008, 265 from April 2008, and 285 from July 2008. A total of 345 (42%) patients were admitted during the day, and 479 (58%) at night; 641 (78%) were admitted on weekdays, and 183 (22%) on weekends. The 33 excluded charts were similar to the included charts for both time of admission and outcomes. Results for parametric testing and nonparametric testing, as well as for log‐transformation and non‐log‐transformation of the continuous outcomes were similar in both magnitude and statistical significance, so we present the parametric and nonlog‐transformed results below for ease of interpretation.

Interrater reliability among the 3 reviewers was very high. There were no disagreements among the 20 multiple reviews for either poor outcomes within 24 hours of admission or admitting service; the interclass correlation coefficients for 30 day repeat ED visit and 30 day readmission were 0.97 and 0.87, respectively.

Patients admitted at night or on the weekend were similar to patients admitted during the day and week across age, gender, insurance class, MDC, and Charlson Comorbidity Index (Table 1). For unadjusted outcomes, patients admitted at night has a similar LOS, hospital charges, 30 day repeat ED visits, 30 day readmissions, and poor outcome within 24 hours of admission as those patients admitted during the day. They had a potentially lower chance of any ICU transfer during hospitalization though this did not reach statistical significance at P < 0.05 (night admission 6%, day admission 3%, P = 0.06) (Table 2).

Baseline Characteristics of Patients
CharacteristicsTime of DayDay of the Week
Day Admission (n = 345)Night Admission (n = 479)Weekday Admission (n = 641)Weekend Admission (n = 183)
  • NOTE: All P values > 0.05.

  • Abbreviation: ED, emergency department.

Age (years)60.859.760.658.7
Gender (% male)47434546
Race/Ethnicity (%)
White, Asian, other61545755
Black34383734
Hispanic58610
Medicaid or self pay (%)9101011
Major diagnostic category (%)
Respiratory disease14131413
Circulatory disease28232624
Digestive disease12121212
Other45524851
Charlson Comorbidity Index3.713.603.663.60
Outcomes, Unadjusted
OutcomesTime of DayDay of the Week
Day Admission (n = 345)Night Admission (n = 479)Weekday Admission (n = 641)Weekend Admission (n = 183)
  • Abbreviations: ED, emergency department; ICU, intensive care unit.

  • P < 0.05.

  • P= 0.06.

Length of stay4.34.14.33.8
Hospital charges$27,500$25,200$27,200*$22,700*
ICU transfer during hospitalization (%)635*1*
Repeat ED visit at 30 days (%)20222221
Readmission at 30 days (%)17202017
Poor outcome at 24 hours (ICU transfer, cardiac arrest, or death)(%)2121

Patients admitted to the hospital during the weekend were similar to patients admitted during the week for unadjusted LOS, 30 day repeat ED visit or readmission rate, and poor outcomes within 24 hours of admission as those admitted during the week; however, they had lower hospital charges (weekend admission $22,700, weekday admission $27,200; P = 0.02), and a lower chance of ICU transfer during hospitalization (weekend admission 1%, weekday admission 5%; P = 0.02) (Table 2).

In the multivariable linear and logistic regression models (Tables 3 and 4), we assessed the independent association between night admission or weekend admission and each hospitalization‐relevant outcome except for poor outcome within 24 hours of admission (poor outcome within 24 hours of admission was not modeled to avoid the risk of overfitting because there were only 13 total events). After adjustment for age, gender, race and ethnicity, admitting service (hospitalist or teaching), Medicaid or self‐pay insurance, MDC, and Charlson Comorbidity Index, there was no statistically significant association between night admission and worse outcomes for LOS, hospital charges, 30 day repeat ED visit, or 30 day readmission. Night admission was associated with a decreased chance of ICU transfer during hospitalization, but the difference was not statistically significant (odds ratio, 0.54; 95% confidence interval [CI], 0.26‐1.11, P = 0.09). Weekend admission was not associated with worse outcomes for LOS or 30 day repeat ED visit or readmission; however, weekend admission was associated with a decrease in overall charges ($4400; 95% CI, $8300 to $600) and a decreased chance of ICU transfer during hospitalization (odds ratio, 0.20; 95% CI, 0.050.88).

Linear Regressions for Continuous Outcomes (With Coefficients)
PredictorsLength of Stay (days), Coefficient (95% CI)Hospital Charges (dollars), Coefficient (95% CI)
  • Abbreviations: CI, confidence intervals; ICU, intensive care unit; MDC, major diagnostic category: comparison to other.

  • P < 0.05.

Night admission0.23 (0.77 to 0.32)2100 (5400 to 1100)
Weekend admission0.42 (1.07 to 0.23)4400 (8300 to 600)*
Age0.01 (0.01 to 0.03)0 (100 to 100)
Male gender0.15 (0.70 to 0.39)400 (3700 to 2800)
Race, Black0.18 (0.41 to 0.78)200 (3700 to 3400)
Ethnicity, Hispanic0.62 (1.73 to 0.49)2300 (8900 to 4300)
Medicaid or self‐pay insurance1.87 (0.93 to 2.82)*8900 (3300 to 14600)*
Hospitalist service0.26 (0.29 to 0.81)600 (3900 to 2700)
MDC: respiratory0.36 (1.18 to 0.46)700 (4200 to 5600)
MDC: circulatory1.36 (2.04 to 0.68)*600 (4600 to 3400)
MDC: digestive1.22 (2.08 to 0.35)*6800 (12000 to 1700)*
Charlson Comorbidity Index0.35 (0.22 to 0.49)*2200 (1400 to 3000)*
Logistic Regressions for Dichotomous Outcomes (With Odds Ratios)
PredictorsICU Transfer during Hospitalization, Odds Ratio (95% CI)Repeat ED Visit at 30 days, Odds Ratio (95% CI)Readmission at 30 days, Odds Ratio (95% CI)
  • Abbreviations: CI, confidence intervals; ICU, intensive care unit; MDC, major diagnostic category: comparison to other.

  • P < 0.05.

Night admission0.53 (0.26 to 1.11)1.13 (0.80 to 1.60)1.23 (0.86 to 1.78)
Weekend admission0.20 (0.05 to 0.88)*0.95 (0.63 to 1.44)0.80 (0.51 to 1.25)
Age1.00 (0.98 to 1.02)0.99 (0.98 to 1.002)1.00 (0.99 to 1.01)
Male gender0.98 (0.47 to 2.02)1.09 (0.78 to 1.54)0.91 (0.64 to 1.31)
Race, Black0.75 (0.33 to 1.70)1.48 (1.02 to 2.14)*1.12 (0.76 to 1.65)
Ethnicity, Hispanic0.76 (0.16 to 3.73)1.09 (0.55 to 2.17)1.11 (0.55 to 2.22)
Medicaid or self‐pay insurance0.75 (0.16 to 3.49)1.61 (0.95 to 2.72)2.14 (1.24 to 3.67)*
Hospitalist service0.68 (0.33 to 1.44)1.15 (0.81 to 1.63)0.99 (0.69 to 1.43)
MDC: respiratory1.18 (0.41 to 3.38)1.02 (0.61 to 1.69)1.16 (0.69 to 1.95)
MDC: circulatory1.22 (0.52 to 2.87)0.79 (0.51 to 1.22)0.80 (0.51 to 1.27)
MDC: digestive0.51 (0.11 to 2.32)0.83 (0.47 to 1.46)1.08 (0.62 to 1.91)
Charlson Comobrbidity Index1.25 (1.09 to 1.45)*1.09 (1.01 to 1.19)*1.11 (1.02 to 1.21)*

Our multivariate models explained very little of the variance in patient outcomes. For LOS and hospital charges, adjusted R2 values were 0.06 and 0.05, respectively. For ICU transfer during hospitalization, 30 day repeat ED visit, and 30 day readmission, the areas under the receiver operator curves were 0.75, 0.51, and 0.61 respectively.

To assess the robustness of our conclusions regarding night admission, we redefined night to include only patients admitted between the hours of 8 pm and 5:59 am. This did not change our conclusions. We also tested for interaction between night admission and weekend admission for all outcomes to assess whether night admissions on the weekend were in fact at increased risk of worse outcomes; we found no evidence of interaction (P > 0.3 for the interaction terms in each model).

Discussion

Among patients admitted to the medicine services at our academic medical center, night or weekend admission was not associated with worse hospitalization‐relevant outcomes. In some cases, night or weekend admission was associated with better outcomes, particularly in terms of ICU transfer during hospitalization and hospital charges. Prior research indicates worse outcomes during off‐hours,5 but we did not replicate this finding in our study.

The finding that admission at night was not associated with worse outcomes, particularly proximal outcomes such as LOS or ICU transfer during hospitalization, was surprising, though reassuring in view of the fact that more than half of our patients are admitted at night. We believe a few factors may be responsible. First, our general medicine service is staffed during the night (7 pm to 7 am) by in‐house nocturnists and night float residents. Second, our staffing ratio, while lower at night than during the day, remains the same on weekends and may be higher than in other settings. In continuously well‐staffed settings such as the ED12 and ICU,13 night and weekend admissions are only inconsistently associated with worse outcomes, which may be the same phenomena we observed in the current study. Third, the hospital used as the site of this study has received Nursing Magnet recognition and numerous quality awards such as the National Research Corporation's Consumer Choice Award and recognition as a Distinguished Hospital for Clinical Excellence by HealthGrades. Fourth, our integrated electronic medical record, computerized physician order entry system, and automatically generated sign out serve as complements to the morning hand off. Fifth, hospitalists and teaching teams rotate on a weekly, biweekly, or every 4 week basis, which may protect against discontinuity associated with the weekend. We believe that all of these factors may facilitate alert, comprehensive care during the night and weekend as well as safe and efficient transfer of patients from the night to the day providers.

We were also surprised by the association between weekend admission and lower charges and a lower chance of ICU transfer during hospitalization. We believe many of the same factors noted above may have played a role in these findings. In terms of hospital charges, it is possible that some workups were completed outside of the hospital rather than during the hospitalization, and that some tests were not ordered at all due to unavailability on weekends. The decreased chance of ICU transfer is unexplained. We hypothesize that there may have been a more conservative admission strategy within the ED, such that patients with high baseline severity were admitted directly to the ICU on the weekend rather than being admitted first to the general medicine floor. This hypothesis requires further study.

Our study had important limitations. It was a retrospective study from a single academic hospital. The sample size lacked sufficient power to detect differences in the low frequency of certain outcomes such as poor outcomes within 24 hours of admission (2% vs. 1%), and also for more frequent outcomes such as 30 day readmission; it is possible that with a larger sample there would have been statistically significant differences. Further, we recognize that the Charlson Comorbidity Index, which was developed to predict 1‐year mortality for medicine service patients, does not adjust for severity of illness at presentation, particularly for outcomes such as readmission. If patients admitted at night and during the weekend were less acutely ill despite having similar comorbidities and MDCs at admission, true associations between time of admission and worse outcomes could have been masked. Furthermore, the multivariable modeling explained very little of the variance in patient outcomes such that significant unmeasured confounding may still be present, and consequently our results cannot be interpreted in a causal way. Data was collected from electronic records, so it is possible that some adverse events were not recorded. However, it seems unlikely that major events such as death and transfer to an ICU would have been missed.

Several aspects of the study strengthen our confidence in the findings, including a large sample size, relevance of the outcomes, the adjustment for confounders, and an assessment for robustness of the conclusions based on restricting the definition of night and also testing for interaction between night and weekend admission. Our patient demographics and insurance mix resemble that of other academic hospitals,10 and perhaps our results may be generalizable to these settings, if not to non‐urban or community hospitals. Furthermore, the Charlson Comorbidity Index was associated with all 5 of the modeled outcomes we chose for our study, reaffirming their utility in assessing the quality of hospital care. Future directions for investigation may include examining the association of night admission with hospitalization‐relevant outcomes in nonacademic, nonurban settings, and examining whether the lack of association between night and weekend admission and worse outcomes persists with adjustment for initial severity of illness.

In summary, at a large, well‐staffed urban academic hospital, day or time of admission were not associated with worse hospitalization‐relevant outcomes. The use of nocturnists and night float teams for night admissions and continuity across weekends appears to be a safe approach to handling the increased volume of patients admitted at night, and a viable alternative to overnight call in the era of work hour restrictions.

The hospitalist movement and increasingly stringent resident work hour restrictions have led to the utilization of shift work in many hospitals.1 Use of nocturnist and night float systems, while often necessary, results in increased patient hand‐offs. Research suggests that hand‐offs in the inpatient setting can adversely affect patient outcomes as lack of continuity may increase the possibility of medical error.2, 3 In 2001, Bell et al.4 found that mortality was higher among patients admitted on weekends as compared to weekdays. Uneven staffing, lack of supervision, and fragmented care were cited as potential contributing factors.4 Similarly, Peberdy et al.5 in 2008 revealed that patients were less likely to survive a cardiac arrest if it occurred at night or on weekends, again attributed in part to fragmented patient care and understaffing.

The results of these studies raise concerns as to whether increased reliance on shift work and resulting handoffs compromises patient care.6, 7 The aim of this study was to evaluate the potential association between night admission and hospitalization‐relevant outcomes (length of stay [LOS], hospital charges, intensive care unit [ICU] transfer during hospitalization, repeat emergency department [ED] visit within 30 days of discharge, readmission within 30 days of discharge, and poor outcome [transfer to the ICU, cardiac arrest, or death] within the first 24 hours of admission) at an institution that exclusively uses nocturnists (night‐shift based hospitalists) and a resident night float system for patients admitted at night to the general medicine service. A secondary aim was to determine the potential association between weekend admission and hospitalization‐relevant outcomes.

Methods

Study Sample and Selection

We conducted a retrospective medical record review at a large urban academic hospital. Using an administrative hospital data set, we assembled a list of approximately 9000 admissions to the general medicine service from the ED between January 2008 and October 2008. We sampled consecutive admissions from 3 distinct periods beginning in January, April, and July to capture outcomes at various points in the academic year. We attempted to review approximately 10% of all charts equally distributed among the 3 sampling periods (ie, 900 charts total with one‐third from each period) based on time available to the reviewers. We excluded patients not admitted to the general medicine service and patients without complete demographic or outcome information. We also excluded patients not admitted from the ED given that the vast majority of admissions to our hospital during the night (96%) or weekend (93%) are from the ED. Patients admitted to the general medicine service are cared for either by a hospitalist or by a teaching team comprised of 1 attending (about 40% of whom are hospitalists), 1 resident, 1 to 2 interns, and 1 to 3 medical students. From 7 am to 6:59 pm patients are admitted to the care of 1 of the primary daytime admitting teams. From 7 pm to 6:59 am patients are admitted by nocturnists (hospitalist service) or night float residents (teaching service). These patients are handed off to day teams at 7 am. Hospitalist teams change service on a weekly to biweekly basis and resident teams switch on a monthly basis; there is no difference in physician staffing between the weekend and weekdays. The Northwestern University Institutional Review Board approved this study.

Data Acquisition and Medical Records Reviews

We obtained demographic data including gender, age, race and ethnicity, patient insurance, admission day (weekday vs. weekend), admission time (defined as the time that a patient receives a hospital bed, which at our institution is also the time that admitting teams receive report and assume care for the patient), and the International Classification of Disease codes required to determine the Major Diagnostic Category (MDC) and calculate the Charlson Comorbidity Index8, 9 as part of an administrative data set. We divided the admission time into night admission (defined as 7 pm to 6:59 am) and day admission (defined as 7:00 am to 6:59 pm). We created a chart abstraction tool to allow manual recording of the additional fields of admitting team (hospitalist vs. resident), 30 day repeat ED visit, 30 day readmission, and poor outcomes within the first 24 hours of admission, directly from the electronic record.

Study Outcomes

We evaluated each admission for the following 6 primary outcomes which were specified a priori: LOS (defined as discharge date and time minus admission date and time), hospital charges (defined as charges billed as recorded in the administrative data set), ICU transfer during hospitalization (defined as 1 ICU day in the administrative data set), 30 day repeat ED visit (defined as a visit to our ED within 30 days of discharge as assessed by chart abstraction), 30 day readmission (defined as any planned or unplanned admission to any inpatient service at our institution within 30 days of discharge as assessed by chart abstraction), and poor outcome within 24 hours of admission (defined as transfer to the ICU, cardiac arrest, or death as assessed by chart abstraction). Each of these outcomes has been used in prior work to assess the quality of inpatient care.10, 11

Statistical Analysis

Interrater reliability between the 3 physician reviewers was assessed for 20 randomly selected admissions across the 4 separate review measures using interclass correlation coefficients. Comparisons between night admissions and day admissions, and between weekend and weekday admissions, for the continuous primary outcomes (LOS, hospital charges) were assessed using 2‐tailed t‐tests as well as Wilcoxon rank sum test. In the multivariable modeling, these outcomes were assessed by linear regression controlling for age, gender, race and ethnicity, Medicaid or self‐pay insurance, admission to the hospitalist or teaching service, most common MDC categories, and Charlson Comorbidity Index. Because both outcomes were right‐skewed, we separately assessed each after log‐transformation controlling for the same variables.

All comparisons of the dichotomous primary outcomes (ICU transfer during hospitalization, 30 day repeat ED visit, 30 day readmission, and poor outcome within the first 24 hours after admission) were assessed at the univariate level by chi‐squared test, and in the multivariable models using logistic regression, controlling for the same variables as the linear models above. All adjustments were specified a priori. All data analyses were conducted using Stata (College Station, TX; Version 11).

Results

We reviewed 857 records. After excluding 33 records lacking administrative data regarding gender, race and ethnicity, and other demographic variables, there were 824 medical records available for analysis. We reviewed a similar number of records from each time period: 274 from January 2008, 265 from April 2008, and 285 from July 2008. A total of 345 (42%) patients were admitted during the day, and 479 (58%) at night; 641 (78%) were admitted on weekdays, and 183 (22%) on weekends. The 33 excluded charts were similar to the included charts for both time of admission and outcomes. Results for parametric testing and nonparametric testing, as well as for log‐transformation and non‐log‐transformation of the continuous outcomes were similar in both magnitude and statistical significance, so we present the parametric and nonlog‐transformed results below for ease of interpretation.

Interrater reliability among the 3 reviewers was very high. There were no disagreements among the 20 multiple reviews for either poor outcomes within 24 hours of admission or admitting service; the interclass correlation coefficients for 30 day repeat ED visit and 30 day readmission were 0.97 and 0.87, respectively.

Patients admitted at night or on the weekend were similar to patients admitted during the day and week across age, gender, insurance class, MDC, and Charlson Comorbidity Index (Table 1). For unadjusted outcomes, patients admitted at night has a similar LOS, hospital charges, 30 day repeat ED visits, 30 day readmissions, and poor outcome within 24 hours of admission as those patients admitted during the day. They had a potentially lower chance of any ICU transfer during hospitalization though this did not reach statistical significance at P < 0.05 (night admission 6%, day admission 3%, P = 0.06) (Table 2).

Baseline Characteristics of Patients
CharacteristicsTime of DayDay of the Week
Day Admission (n = 345)Night Admission (n = 479)Weekday Admission (n = 641)Weekend Admission (n = 183)
  • NOTE: All P values > 0.05.

  • Abbreviation: ED, emergency department.

Age (years)60.859.760.658.7
Gender (% male)47434546
Race/Ethnicity (%)
White, Asian, other61545755
Black34383734
Hispanic58610
Medicaid or self pay (%)9101011
Major diagnostic category (%)
Respiratory disease14131413
Circulatory disease28232624
Digestive disease12121212
Other45524851
Charlson Comorbidity Index3.713.603.663.60
Outcomes, Unadjusted
OutcomesTime of DayDay of the Week
Day Admission (n = 345)Night Admission (n = 479)Weekday Admission (n = 641)Weekend Admission (n = 183)
  • Abbreviations: ED, emergency department; ICU, intensive care unit.

  • P < 0.05.

  • P= 0.06.

Length of stay4.34.14.33.8
Hospital charges$27,500$25,200$27,200*$22,700*
ICU transfer during hospitalization (%)635*1*
Repeat ED visit at 30 days (%)20222221
Readmission at 30 days (%)17202017
Poor outcome at 24 hours (ICU transfer, cardiac arrest, or death)(%)2121

Patients admitted to the hospital during the weekend were similar to patients admitted during the week for unadjusted LOS, 30 day repeat ED visit or readmission rate, and poor outcomes within 24 hours of admission as those admitted during the week; however, they had lower hospital charges (weekend admission $22,700, weekday admission $27,200; P = 0.02), and a lower chance of ICU transfer during hospitalization (weekend admission 1%, weekday admission 5%; P = 0.02) (Table 2).

In the multivariable linear and logistic regression models (Tables 3 and 4), we assessed the independent association between night admission or weekend admission and each hospitalization‐relevant outcome except for poor outcome within 24 hours of admission (poor outcome within 24 hours of admission was not modeled to avoid the risk of overfitting because there were only 13 total events). After adjustment for age, gender, race and ethnicity, admitting service (hospitalist or teaching), Medicaid or self‐pay insurance, MDC, and Charlson Comorbidity Index, there was no statistically significant association between night admission and worse outcomes for LOS, hospital charges, 30 day repeat ED visit, or 30 day readmission. Night admission was associated with a decreased chance of ICU transfer during hospitalization, but the difference was not statistically significant (odds ratio, 0.54; 95% confidence interval [CI], 0.26‐1.11, P = 0.09). Weekend admission was not associated with worse outcomes for LOS or 30 day repeat ED visit or readmission; however, weekend admission was associated with a decrease in overall charges ($4400; 95% CI, $8300 to $600) and a decreased chance of ICU transfer during hospitalization (odds ratio, 0.20; 95% CI, 0.050.88).

Linear Regressions for Continuous Outcomes (With Coefficients)
PredictorsLength of Stay (days), Coefficient (95% CI)Hospital Charges (dollars), Coefficient (95% CI)
  • Abbreviations: CI, confidence intervals; ICU, intensive care unit; MDC, major diagnostic category: comparison to other.

  • P < 0.05.

Night admission0.23 (0.77 to 0.32)2100 (5400 to 1100)
Weekend admission0.42 (1.07 to 0.23)4400 (8300 to 600)*
Age0.01 (0.01 to 0.03)0 (100 to 100)
Male gender0.15 (0.70 to 0.39)400 (3700 to 2800)
Race, Black0.18 (0.41 to 0.78)200 (3700 to 3400)
Ethnicity, Hispanic0.62 (1.73 to 0.49)2300 (8900 to 4300)
Medicaid or self‐pay insurance1.87 (0.93 to 2.82)*8900 (3300 to 14600)*
Hospitalist service0.26 (0.29 to 0.81)600 (3900 to 2700)
MDC: respiratory0.36 (1.18 to 0.46)700 (4200 to 5600)
MDC: circulatory1.36 (2.04 to 0.68)*600 (4600 to 3400)
MDC: digestive1.22 (2.08 to 0.35)*6800 (12000 to 1700)*
Charlson Comorbidity Index0.35 (0.22 to 0.49)*2200 (1400 to 3000)*
Logistic Regressions for Dichotomous Outcomes (With Odds Ratios)
PredictorsICU Transfer during Hospitalization, Odds Ratio (95% CI)Repeat ED Visit at 30 days, Odds Ratio (95% CI)Readmission at 30 days, Odds Ratio (95% CI)
  • Abbreviations: CI, confidence intervals; ICU, intensive care unit; MDC, major diagnostic category: comparison to other.

  • P < 0.05.

Night admission0.53 (0.26 to 1.11)1.13 (0.80 to 1.60)1.23 (0.86 to 1.78)
Weekend admission0.20 (0.05 to 0.88)*0.95 (0.63 to 1.44)0.80 (0.51 to 1.25)
Age1.00 (0.98 to 1.02)0.99 (0.98 to 1.002)1.00 (0.99 to 1.01)
Male gender0.98 (0.47 to 2.02)1.09 (0.78 to 1.54)0.91 (0.64 to 1.31)
Race, Black0.75 (0.33 to 1.70)1.48 (1.02 to 2.14)*1.12 (0.76 to 1.65)
Ethnicity, Hispanic0.76 (0.16 to 3.73)1.09 (0.55 to 2.17)1.11 (0.55 to 2.22)
Medicaid or self‐pay insurance0.75 (0.16 to 3.49)1.61 (0.95 to 2.72)2.14 (1.24 to 3.67)*
Hospitalist service0.68 (0.33 to 1.44)1.15 (0.81 to 1.63)0.99 (0.69 to 1.43)
MDC: respiratory1.18 (0.41 to 3.38)1.02 (0.61 to 1.69)1.16 (0.69 to 1.95)
MDC: circulatory1.22 (0.52 to 2.87)0.79 (0.51 to 1.22)0.80 (0.51 to 1.27)
MDC: digestive0.51 (0.11 to 2.32)0.83 (0.47 to 1.46)1.08 (0.62 to 1.91)
Charlson Comobrbidity Index1.25 (1.09 to 1.45)*1.09 (1.01 to 1.19)*1.11 (1.02 to 1.21)*

Our multivariate models explained very little of the variance in patient outcomes. For LOS and hospital charges, adjusted R2 values were 0.06 and 0.05, respectively. For ICU transfer during hospitalization, 30 day repeat ED visit, and 30 day readmission, the areas under the receiver operator curves were 0.75, 0.51, and 0.61 respectively.

To assess the robustness of our conclusions regarding night admission, we redefined night to include only patients admitted between the hours of 8 pm and 5:59 am. This did not change our conclusions. We also tested for interaction between night admission and weekend admission for all outcomes to assess whether night admissions on the weekend were in fact at increased risk of worse outcomes; we found no evidence of interaction (P > 0.3 for the interaction terms in each model).

Discussion

Among patients admitted to the medicine services at our academic medical center, night or weekend admission was not associated with worse hospitalization‐relevant outcomes. In some cases, night or weekend admission was associated with better outcomes, particularly in terms of ICU transfer during hospitalization and hospital charges. Prior research indicates worse outcomes during off‐hours,5 but we did not replicate this finding in our study.

The finding that admission at night was not associated with worse outcomes, particularly proximal outcomes such as LOS or ICU transfer during hospitalization, was surprising, though reassuring in view of the fact that more than half of our patients are admitted at night. We believe a few factors may be responsible. First, our general medicine service is staffed during the night (7 pm to 7 am) by in‐house nocturnists and night float residents. Second, our staffing ratio, while lower at night than during the day, remains the same on weekends and may be higher than in other settings. In continuously well‐staffed settings such as the ED12 and ICU,13 night and weekend admissions are only inconsistently associated with worse outcomes, which may be the same phenomena we observed in the current study. Third, the hospital used as the site of this study has received Nursing Magnet recognition and numerous quality awards such as the National Research Corporation's Consumer Choice Award and recognition as a Distinguished Hospital for Clinical Excellence by HealthGrades. Fourth, our integrated electronic medical record, computerized physician order entry system, and automatically generated sign out serve as complements to the morning hand off. Fifth, hospitalists and teaching teams rotate on a weekly, biweekly, or every 4 week basis, which may protect against discontinuity associated with the weekend. We believe that all of these factors may facilitate alert, comprehensive care during the night and weekend as well as safe and efficient transfer of patients from the night to the day providers.

We were also surprised by the association between weekend admission and lower charges and a lower chance of ICU transfer during hospitalization. We believe many of the same factors noted above may have played a role in these findings. In terms of hospital charges, it is possible that some workups were completed outside of the hospital rather than during the hospitalization, and that some tests were not ordered at all due to unavailability on weekends. The decreased chance of ICU transfer is unexplained. We hypothesize that there may have been a more conservative admission strategy within the ED, such that patients with high baseline severity were admitted directly to the ICU on the weekend rather than being admitted first to the general medicine floor. This hypothesis requires further study.

Our study had important limitations. It was a retrospective study from a single academic hospital. The sample size lacked sufficient power to detect differences in the low frequency of certain outcomes such as poor outcomes within 24 hours of admission (2% vs. 1%), and also for more frequent outcomes such as 30 day readmission; it is possible that with a larger sample there would have been statistically significant differences. Further, we recognize that the Charlson Comorbidity Index, which was developed to predict 1‐year mortality for medicine service patients, does not adjust for severity of illness at presentation, particularly for outcomes such as readmission. If patients admitted at night and during the weekend were less acutely ill despite having similar comorbidities and MDCs at admission, true associations between time of admission and worse outcomes could have been masked. Furthermore, the multivariable modeling explained very little of the variance in patient outcomes such that significant unmeasured confounding may still be present, and consequently our results cannot be interpreted in a causal way. Data was collected from electronic records, so it is possible that some adverse events were not recorded. However, it seems unlikely that major events such as death and transfer to an ICU would have been missed.

Several aspects of the study strengthen our confidence in the findings, including a large sample size, relevance of the outcomes, the adjustment for confounders, and an assessment for robustness of the conclusions based on restricting the definition of night and also testing for interaction between night and weekend admission. Our patient demographics and insurance mix resemble that of other academic hospitals,10 and perhaps our results may be generalizable to these settings, if not to non‐urban or community hospitals. Furthermore, the Charlson Comorbidity Index was associated with all 5 of the modeled outcomes we chose for our study, reaffirming their utility in assessing the quality of hospital care. Future directions for investigation may include examining the association of night admission with hospitalization‐relevant outcomes in nonacademic, nonurban settings, and examining whether the lack of association between night and weekend admission and worse outcomes persists with adjustment for initial severity of illness.

In summary, at a large, well‐staffed urban academic hospital, day or time of admission were not associated with worse hospitalization‐relevant outcomes. The use of nocturnists and night float teams for night admissions and continuity across weekends appears to be a safe approach to handling the increased volume of patients admitted at night, and a viable alternative to overnight call in the era of work hour restrictions.

References
  1. Vaughn DM,Stout CL,McCampbell BL, et al.Three‐year results of mandated work hour restrictions: attending and resident perspectives and effects in a community hospital.Am Surg.2008;74(6):542546; discussion 546–547.
  2. Kitch BT,Cooper JB,Zapol WM, et al.Handoffs causing patient harm: a survey of medical and surgical house staff.Jt Comm J Qual Patient Saf.2008;34(10):563570.
  3. Petersen LA,Brennan TA,O'Neil AC,Cook EF,Lee TH.Does housestaff discontinuity of care increase the risk for preventable adverse events?Ann Intern Med.1994;121(11):866872.
  4. Bell CM,Redelmeier DA.Mortality among patients admitted to hospitals on weekends as compared with weekdays.N Engl J Med.2001;345(9):663668.
  5. Peberdy MA,Ornato JP,Larkin GL, et al.Survival from in‐hospital cardiac arrest during nights and weekends.JAMA.2008;299(7):785792.
  6. Sharma G,Freeman J,Zhang D,Goodwin JS.Continuity of care and intensive care unit use at the end of life.Arch Intern Med.2009;169(1):8186.
  7. Sharma G,Fletcher KE,Zhang D,Kuo YF,Freeman JL,Goodwin JS.Continuity of outpatient and inpatient care by primary care physicians for hospitalized older adults.JAMA.2009;301(16):16711680.
  8. Charlson ME,Ales KL,Simon R,MacKenzie CR.Why predictive indexes perform less well in validation studies: is it magic or methods?Arch Intern Med.1987;147:21552161.
  9. Deyo RA,Cherkin DC,Ciol MA.Adapting a clinical comorbidity index for use with ICD‐9‐CM administrative databases.J Clin Epidemiol.1992;45(6):613619.
  10. Roy CL,Liang CL,Lund M, et al.Implementation of a physician assistant/hospitalist service in an academic medical center: impact on efficiency and patient outcomes.J Hosp Med.2008;3(5):361368.
  11. Groarke JD,Gallagher J,Stack J, et al.Use of an admission early warning score to predict patient morbidity and mortality and treatment success.Emerg Med J.2008;25(12):803806.
  12. Schmulewitz L,Proudfoot A,Bell D.The impact of weekends on outcome for emergency patients.Clin Med.2005;5(6):621625.
  13. Meynaar IA,van der Spoel JI,Rommes JH,van Spreuwel‐Verheijen M,Bosman RJ,Spronk PE.Off hour admission to an intensivist‐led ICU is not associated with increased mortality.Crit Care.2009;13(3):R84.
References
  1. Vaughn DM,Stout CL,McCampbell BL, et al.Three‐year results of mandated work hour restrictions: attending and resident perspectives and effects in a community hospital.Am Surg.2008;74(6):542546; discussion 546–547.
  2. Kitch BT,Cooper JB,Zapol WM, et al.Handoffs causing patient harm: a survey of medical and surgical house staff.Jt Comm J Qual Patient Saf.2008;34(10):563570.
  3. Petersen LA,Brennan TA,O'Neil AC,Cook EF,Lee TH.Does housestaff discontinuity of care increase the risk for preventable adverse events?Ann Intern Med.1994;121(11):866872.
  4. Bell CM,Redelmeier DA.Mortality among patients admitted to hospitals on weekends as compared with weekdays.N Engl J Med.2001;345(9):663668.
  5. Peberdy MA,Ornato JP,Larkin GL, et al.Survival from in‐hospital cardiac arrest during nights and weekends.JAMA.2008;299(7):785792.
  6. Sharma G,Freeman J,Zhang D,Goodwin JS.Continuity of care and intensive care unit use at the end of life.Arch Intern Med.2009;169(1):8186.
  7. Sharma G,Fletcher KE,Zhang D,Kuo YF,Freeman JL,Goodwin JS.Continuity of outpatient and inpatient care by primary care physicians for hospitalized older adults.JAMA.2009;301(16):16711680.
  8. Charlson ME,Ales KL,Simon R,MacKenzie CR.Why predictive indexes perform less well in validation studies: is it magic or methods?Arch Intern Med.1987;147:21552161.
  9. Deyo RA,Cherkin DC,Ciol MA.Adapting a clinical comorbidity index for use with ICD‐9‐CM administrative databases.J Clin Epidemiol.1992;45(6):613619.
  10. Roy CL,Liang CL,Lund M, et al.Implementation of a physician assistant/hospitalist service in an academic medical center: impact on efficiency and patient outcomes.J Hosp Med.2008;3(5):361368.
  11. Groarke JD,Gallagher J,Stack J, et al.Use of an admission early warning score to predict patient morbidity and mortality and treatment success.Emerg Med J.2008;25(12):803806.
  12. Schmulewitz L,Proudfoot A,Bell D.The impact of weekends on outcome for emergency patients.Clin Med.2005;5(6):621625.
  13. Meynaar IA,van der Spoel JI,Rommes JH,van Spreuwel‐Verheijen M,Bosman RJ,Spronk PE.Off hour admission to an intensivist‐led ICU is not associated with increased mortality.Crit Care.2009;13(3):R84.
Issue
Journal of Hospital Medicine - 6(1)
Issue
Journal of Hospital Medicine - 6(1)
Page Number
10-14
Page Number
10-14
Publications
Publications
Article Type
Display Headline
The association between night or weekend admission and hospitalization‐relevant patient outcomes
Display Headline
The association between night or weekend admission and hospitalization‐relevant patient outcomes
Legacy Keywords
communication, continuity of care transition and discharge planning, education, outcomes measurement, patient safety, resident
Legacy Keywords
communication, continuity of care transition and discharge planning, education, outcomes measurement, patient safety, resident
Sections
Article Source

Copyright © 2010 Society of Hospital Medicine

Disallow All Ads
Correspondence Location
Division of General Internal Medicine, University of California, San Francisco, Box 1211, San Francisco, CA, 94143‐1211
Content Gating
No Gating (article Unlocked/Free)
Alternative CME
Article PDF Media

Discharge Summary Improvement

Article Type
Changed
Sun, 05/28/2017 - 21:47
Display Headline
Creating a better discharge summary: Improvement in quality and timeliness using an electronic discharge summary

Preventable or ameliorable adverse events have been reported to occur in 12% of patients in the period immediately following hospital discharge.1, 2 A potential contributor to this is the inadequate transfer of clinical information at hospital discharge. The discharge summary comprises a vital component of the information transfer between the inpatient and outpatient settings. Unfortunately, discharge summaries are often unavailable at the time of follow‐up care and often lack important content.37

A growing number of hospitals are implementing electronic medical records (EMR). This creates the opportunity to standardize the content of clinical documentation and creates the potential to assemble, immediately at the time of hospital discharge, major components of a discharge summary. With enhanced communication systems, this information can be delivered in a variety of ways with minimal delay. Previously, we reported the results of a survey of medicine faculty at an urban academic medical center evaluating the timeliness and quality of discharge summaries, the perceived incidence of preventable adverse events related to suboptimal information transfer at discharge, and a needs assessment for an electronically generated discharge summary that we planned to design.8 We now report the results of the follow‐up survey of outpatient physicians and an evaluation of the quality and timeliness of the electronic discharge summary we created.

Materials and Methods

Design

We conducted a pre‐post evaluation of the quality and timeliness of discharge summaries. In the initial phase of the study, we convened an advisory board comprised of 16 Department of Medicine physicians. The advisory board gave input on needs assessment and helped to create a survey to be administered to all medicine faculty with an outpatient practice. All respondents who had at least 1 patient admitted to the hospital within the 6 months prior to the survey were eligible. The results of the initial survey were reviewed with the advisory board and an electronic discharge summary was created with their input. To evaluate its impact, we conducted a repeat survey of all medicine faculty with an outpatient practice approximately 1 year after implementation of the electronic discharge summary.

To complement data received from the outpatient physician survey, a randomly selected sample of discharge summaries from general medical services during the same 3 month period before and after implementation of the electronic discharge summary were rated by 1 of 3 board‐certified internists (D.B.E., N.K., or M.P.L.).

Setting and Participants

The study was conducted at Northwestern Memorial Hospital, a 753‐bed hospital in Chicago, IL. The study was approved by the Institutional Review Board of the Northwestern University Feinberg School of Medicine. General medical patients were admitted to 1 of 2 primary physician services during the study period: a teaching service or a nonteaching hospitalist service. Discharge summaries had traditionally been dictated by inpatient physicians and delivered to outpatient physicians by both mail and facsimile via the medical record department. A recommended template for dictated discharge summaries was provided in the paper paging directory distributed yearly to inpatient physicians.

The hospital implemented an EMR and computerized physician order entry (CPOE) system (PowerChart Millennium; Cerner Corporation, Kansas City, MO) in August 2004. Although all history and physicals and progress notes were documented in the EMR, the system did not provide a method for delivering discharge summaries performed within the EMR to outpatient physician offices. Because of this, inpatient physicians were instructed to continue to dictate discharge summaries during the initial phase of the study.

Approximately 65% of outpatient physicians at the study site used an EMR in their offices during the study. Approximately 10% of outpatient physicians used the same EMR the hospital uses, while approximately 55% used a different EMR (EPIC Hyperspace; EPIC Systems Corporation, Verona, WI). The remaining physicians did not use an EMR in their offices.

Intervention: The Electronic Discharge Summary

A draft electronic discharge summary template was created by including elements ranked as highly important by outpatient physicians in our initial survey8 and elements required by The Joint Commission.9 The draft electronic discharge summary template was reviewed by the advisory board and modifications were made with their input. We automated the insertion of specific patient data elements, such as listed allergies and home medications, into the discharge summary template. We also created an electronic reminder system to inpatient physicians for summaries not completed 24 hours after discharge.

Because the majority of physicians in our initial survey preferred discharge summaries to be delivered either by facsimile or via an EMR, we concentrated our efforts on creating reliable systems for delivery by those routes. We created logic that queried the primary care physician field within the EMR at the time the discharge summary was electronically signed. An automated process then sent the discharge summary via electronic fax to the physician listed in the primary care physician field. Because a large number of outpatient physicians used an EMR different from the hospital's, we also created a process that sent discharge summaries from the hospital EMR into patient charts within this separate EMR.

The draft electronic discharge summary template was available for use in the EMR beginning in July 2005. The final electronic discharge summary, including automated content, physician reminder for incomplete summaries, and delivery systems as described above was implemented in June 2006. Upon implementation, inpatient physicians were instructed via email announcements and group meetings to begin completing electronic discharge summaries using the EMR. Beyond these announcements, inpatient physicians did not receive any specific training with regard to the new discharge summary process. An example of the final electronic discharge summary product is available in the Appendix.

Outpatient Physician Survey

Satisfaction with timeliness and quality of discharge summaries was assessed using a 5‐point Likert scale, where 5 represented very satisfied and 1 represented very dissatisfied. We also asked respondents to estimate the number of their patients who had sustained a preventable adverse event or near miss related to suboptimal transfer of information at discharge. We defined a preventable adverse event as a preventable medical problem or worsening of an existing problem and near miss as an error that did not result in patient harm but easily could have.

The preimplementation survey, accompanied by a cover letter signed by the hospital's chief of staff, was sent out in March 2005. A postcard reminder was sent approximately 2 weeks after the initial mail survey. A second survey was sent to nonresponders 6 weeks after the initial survey. Simultaneously, the survey was also sent in web‐based format to nonresponders via email. The postimplementation survey was sent out in February 2007 using a similar survey process.

Discharge Summary Review

A random sample of discharge summaries completed before and after the implementation of the electronic discharge summary was selected for review. The sample universe consisted of all general medicine service discharges between August and November 2005, before the electronic discharge summary was implemented, and August to November 2006, after implementation. To provide a balanced comparison, the sample was further limited to only the first chronological (index) discharge of a unique patient to home self‐care or home health nursing, with length of stay between 3 and 14 days. A total of 2232 discharges in 2005 and 2570 discharges in 2006 met these criteria. The discharge summary review sample was designed to randomly select approximately 100 discharge summaries meeting the criteria above within each study year, to produce an approximate 200‐record analysis sample. Each of the 3 physician reviewers was assigned to complete an approximately equal number of the 200 primary reviews.

Physician reviewers recorded whether the discharge summary was dictated versus done electronically, the length of the discharge summary (in words), the number of days from discharge to discharge summary completion, the type of service the patient was discharged from, and the author type (medical student, intern, resident, or attending). Physicians reviewers also assessed the overall clarity of discharge summaries using a 5‐point ordinal scale (1 = unintelligible; 2 = hard to read; 3 = neutral; 4 = understandable; and 5 = lucid).

Prior studies have evaluated the quality of discharge summaries using scoring tools created by the investigators.10, 11 We created our own discharge summary scoring tool based on these prior studies, recommendations from the literature,12 and the findings from our initial survey.8 We pilot‐tested the scoring tool and made minor revisions prior to the study. The final scoring tool consisted of 16 essential elements. Reviewers assessed whether each of the 16 essential elements was present, absent, or not applicable. A Discharge Summary Completeness Score was calculated by the number of the 16 essential elements that were rated as present divided by the number of applicable elements for each discharge summary, multiplied by 100 to produce a completeness percentage.

To assess interrater reliability, reviewers were assigned to independently complete second, duplicate reviews of approximately 90 summaries (30 per reviewer). The duplicate review sample was designed to produce approximately 45 paired re‐reviews in each year for reliability assessment. A final sample of 196 available summaries was completed for the main analysis and 174 primary and duplicate reviews were used to establish interrater reliability across 87 reviewer pairs.

Data Analysis

Physician characteristics, including specialty, faculty appointment type, and year of medical school graduation were provided by the hospital's medical staff office. Physician characteristics from before and after the implementation of the electronic discharge summary were compared using chi‐square tests. Likert scale ratings of physician satisfaction with the timeliness and quality of discharge summaries were compared using t‐tests. The proportion of physicians reporting 1 or more preventable adverse event or near miss before the implementation of the electronic discharge summary was compared to postimplementation proportions using chi‐square tests. In addition, we performed multivariate logistic regression to examine the likelihood of physicians reporting any preventable adverse event or near miss related to suboptimal information transfer. The regression models tested the likelihood of 1 or more preventable adverse event or near miss before versus after the implementation of the electronic discharge summary, controlling for physician characteristics and their number of hospitalized patients in the previous 6 months.

The proportions of discharge summary elements found to be present, the proportion of discharge summaries completed within 3 days, and discharge summary readability ratings before and after the implementation of the electronic discharge summary were compared using chi‐square tests; length in words was compared using t‐tests. Preimplementation and postimplementation Discharge Summary Completeness Scores were compared using the Mann‐Whitney U test. Discharge summary score interrater reliability was assessed using the Brennan‐Prediger Kappa for individual elements.13

Results

Outpatient Physician Survey

Physician Characteristics

Two hundred and twenty‐six of 416 (54%) eligible outpatient physicians completed the baseline survey and 256 of 397 (64%) completed the postimplementation survey. As shown in Table 1, there were no significant differences in specialty, faculty appointment type, or number of patients hospitalized between respondents to the survey before compared to respondents after the implementation of the electronic discharge summary. The number of respondents graduating medical school in 1990 or later was higher after implementation of the electronic discharge summary; however, this result was of borderline statistical significance.

Characteristics of Respondents to Outpatient Physician Discharge Summary Satisfaction Surveys
 Preelectronic Discharge Summary (n = 226)Postelectronic Discharge Summary (n = 256)P Value
  • Excludes 5 respondents with missing information on graduation year.

  • Excludes 91 respondents with missing data about the number of their hospitalized patients.

Practice Type  0.23
Generalist, n (%)127 (56.2)130 (50.8) 
Specialist, n (%)99 (43.8)126 (49.2) 
Faculty Appointment  0.38
Full‐time, n (%)104 (46.0)128 (50.0) 
Affiliated, n (%)122 (54.0)128 (50.0) 
Year of medical school graduation*  0.06
Before 1990, n (%)128 (57.4)124 (48.8) 
1990 or later, n (%)95 (42.6)130 (51.2) 
Number of patients hospitalized (last 6 months)  0.56
1‐4, n (%)15 (7.9)24 (12.0) 
5‐10, n (%)62 (32.5)66 (33.0) 
11‐19, n (%)35 (18.3)33 (16.5) 
20 or more, n (%)79 (41.4)77 (38.5) 

Timeliness and Content

Changes in outpatient physician satisfaction with the timeliness and quality of discharge summaries are summarized in Table 2. Satisfaction with the timeliness and quality of discharge summarizes improved significantly after the implementation of the electronic discharge summary (mean standard deviation [SD] timeliness rating, 2.59 1.02 versus 3.34 1.09; P < 0.001, mean quality rating 3.04 0.93 versus 3.64 0.99; P < 0.001).

Outpatient Physician Satisfaction with Timeliness, Quality and Communication
 Likert Scale Mean Score (SD)*
Preelectronic Discharge SummaryPostelectronic Discharge SummaryP Value
  • Outpatient physicians rated items using a 5‐point scale (1 = very dissatisfied; 2 = dissatisfied; 3 = somewhat satisfied; 4 = satisfied; and 5 = very satisfied). There were 14 and 19 missing values for each item, respectively.

Timeliness of the discharge summary2.59 (1.02)3.34 (1.09)<0.001
Quality of the discharge summary3.04 (0.93)3.64 (0.99)<0.001

Medical Error

The effect of the electronic discharge summary on perceived near misses and preventable adverse events is summarized in Table 3. Fewer outpatient physicians felt that 1 or more of their patients hospitalized in the preceding 6 months sustained a near miss due to suboptimal transfer of information after the implementation of the electronic discharge summary (65.7% vs. 52.9%, P = 0.008). Similarly, fewer outpatient physicians felt that 1 or more of their patients hospitalized in the preceding 6 months sustained a preventable adverse event due to suboptimal transfer of information after the implementation of the electronic discharge summary (40.7% vs. 30.2%, P = 0.02). In multivariate logistic regression analyses controlling for physician characteristics and their number of hospitalized patients in the previous 6 months, there was a statistically significant 40% reduction in the odds of a reported near miss (adjusted odds ratio [OR] = 0.60, P = 0.02). Although not quite statistically significant, there was a 33% reduction in the odds of a reported preventable adverse event (OR = 0.67, P = 0.08) after the implementation of the electronic discharge summary.

Reduction in Outpatient Physician Perception of Errors Related to Suboptimal Transfer of Information at Hospital Discharge
 Preelectronic Discharge SummaryPostelectronic Discharge SummaryP Value
  • Defined as an error that did not result in patient harm but easily could have. There were 23 missing responses.

  • Defined as a preventable medical problem or worsening of an existing problem. There were 22 missing responses.

Near miss*   
Number (%) reporting 1142 (65.7)108 (52.9) 
Crude odds ratioRef.0.570.008
Adjusted odds ratioRef.0.600.02
Preventable adverse event   
Number (%) reporting 188 (40.7)62 (30.2) 
Crude odds ratioRef.0.630.03
Adjusted odds ratioRef.0.670.08

Discharge Summary Review

Discharge Summary Characteristics

One hundred and one discharge summaries before implementation of the electronic discharge summary were compared to 95 discharge summaries produced the following year. Characteristics of discharge summaries before and after the implementation of the electronic discharge summary are summarized in Table 4. A large number of discharge summaries (52.5%) were already being typed into the EMR in 2005, prior to the implementation of our final electronic discharge summary product. The number of dictated discharge summaries decreased from 47.5% to 10.5% after implementation of the final electronic discharge summary product (P < 0.001). Discharge summaries were similar in length before and after the implementation of the electronic discharge summary. A higher percentage of discharge summaries were completed within 3 days of discharge after implementation of the electronic discharge summary; however, this result was of borderline statistical significance (59.4% vs. 72.6%; P = 0.05). The type of service from which patients were discharged and the distribution of author types were similar after the implementation of the electronic discharge summary.

Characteristics of Discharge Summaries
 Number (%) or MeanSDP Value
Preelectronic Discharge Summary (n = 101)Postelectronic Discharge Summary (n = 95)
Dictated, n (%)48 (47.5)10 (10.5)<0.001
Length in words, mean SD785 407830 3890.43
Completed within 3 days, n (%)60 (59.4)69 (72.6)0.05
Type of service, n (%)  0.29
Teaching service63 (62.4)66 (69.5) 
Nonteaching hospitalist service38 (37.6)29 (30.5) 
Author type, n (%)  0.62
Fourth year medical student13 (12.9)13 (13.7) 
Intern31 (30.7)37 (38.9) 
Resident19 (18.8)15 (15.8) 
Attending38 (37.6)30 (31.6) 

Because a large percentage of discharge summaries were already being done electronically in 2005, we evaluated the timeliness of dictated discharge summaries compared to electronic discharge summaries across both periods combined (preimplementation and postimplementation of the electronic discharge summary). A higher percentage of electronic discharge summaries were completed within 3 days of discharge as compared to dictated discharge summaries (44.8% versus 74.1%; P < 0.001).

Discharge Summary Completeness Score

The presence or absence of discharge summary elements before and after the implementation of the electronic discharge summary is shown in Table 5. Several elements of the discharge summary were present more often after the implementation of the electronic discharge summary. Specific improvements included discussion of follow‐up issues (52.0% versus 75.8%; P = 0.001, = 0.78), pending test results (13.9% vs. 46.3%; P < 0.001, = 0.92), and information provided to the patient and/or family (85.1% vs. 95.8%; P = 0.01, = 0.91). Significant laboratory findings were present less often after implementation of the electronic discharge summary (66.0% versus 51.1%; P = 0.04, = 0.84). The Discharge Summary Completeness Score was higher after the implementation of the electronic discharge summary (mean 74.1 versus 80.3, P = 0.007). Dictated discharge summaries had a significantly lower Discharge Summary Completeness Score compared to discharge summaries done electronically (71.3 vs. 79.6, P = 0.002) across both periods combined.

Improved Likelihood of Pertinent Content Items Present in Discharge Summary
 Number (%) of Content Items Present*P ValueBrennan‐Prediger Kappa
Preelectronic Discharge Summary (n = 101)Postelectronic Discharge Summary (n = 95)
  • n is less for certain elements as information was not applicable.

Dates of admission and discharge96 (95.0)94 (98.9)0.111.0
Reason for hospitalization100 (99.0)94 (100)0.331.0
Significant findings from history and exam78 (77.2)65 (68.4)0.160.26
Significant laboratory findings64 (66.0)47 (51.1)0.040.84
Significant radiological findings67 (75.3)71 (81.6)0.310.89
Significant findings from other tests41 (63.1)40 (71.4)0.330.88
List of procedures performed45 (81.8)35 (77.8)0.770.99
Procedure report findings49 (80.3)43 (78.2)0.610.92
Stress test report findings7 (100)3 (100)N/A1.0
Pathology report findings11 (39.3)3 (30.0)0.600.91
Discharge diagnosis89 (88.1)86 (93.5)0.200.86
Condition at discharge81 (81.0)80 (85.1)0.450.76
Discharge medications88 (87.1)88 (93.6)0.130.79
Follow‐up issues52 (52.0)72 (75.8)0.0010.78
Pending test results14 (13.9)44 (46.3)<0.0010.92
Information provided to patient and/or family, as appropriate86 (85.1)91 (95.8)0.010.91
Discharge Summary Completeness Score (percent present all applicable items)74.180.30.007 

Significantly more discharge summaries were rated as understandable or lucid after the implementation of the electronic discharge summary (41.6% vs. 59.0%; P = 0.02). In both periods combined, dictated discharge summaries were rated as understandable or lucid less often than electronic discharge summaries (34.5% vs. 56.5%; P < 0.001).

Discussion

Our study found that an electronic discharge summary was well accepted by inpatient physicians and significantly improved the quality and timeliness of discharge summaries. Prior studies have shown that the use of electronically entered discharge summaries improved the timeliness of discharge summaries.1416 However, the discharge summaries used in these studies required manual input of data into a computer system separate from the patient's medical record. To our knowledge, this is the first study to report the impact of discharge summaries generated from an EMR. Leveraging the EMR, we were able to automate the insertion of specific patient data elements, streamline delivery, and create an electronic reminder system to inpatient physicians for summaries not completed 24 hours after discharge.

Prior research has shown that the quality of discharges summaries is improved with the use of standardized content.10, 17 Using a standardized template for the electronic discharge summary, we likewise demonstrated improved quality of discharge summaries. Key discharge summary elements, specifically discussion of follow‐up issues, pending test results, and information provided to the patient and/or family, were present more reliably after the implementation of the electronic discharge summary. The importance of identifying pending test results is underscored by a recent study showing that many patients are discharged from hospitals with test results still pending, and that physicians are often unaware when results are abnormal.18 One discharge summary element, significant laboratory findings, was present less often after the implementation of the electronic discharge summary. Our template did not designate significant laboratory findings under a separate heading. Instead, we used a heading entitled Key Results (labs, imaging, pathology). Physicians completing the discharge summaries may have prioritized the report of imaging and pathology results in this section. A simple revision of our discharge summary template to include a separate heading for significant laboratory findings may result in improvement in this regard.

Timeliness of discharge summaries was improved in our study, but remained less than optimal. Although nearly three‐quarters of electronic discharge summaries were completed within 3 days of discharge, our ultimate goal is to have 100% of discharge summaries completed within 3 days. This is especially important for complicated patients requiring outpatient follow‐up soon after discharge. We are currently in the process of designing further modifications to the electronic discharge summary completion process. One modification that may be beneficial is the automation of additional patient specific data elements into the discharge summary. We also plan to link performance of medication reconciliation, completion of patient discharge instructions, and completion of the discharge summary into an integrated set of activities performed in the EMR prior to patient discharge.

We found that fewer outpatient physicians reported 1 or more of their patients having a preventable adverse event or near miss as a result of suboptimal transfer of information at discharge after the implementation of the electronic discharge summary. Although we did not measure preventable adverse events directly in our study, this is an important finding in light of the large number of patients who sustain preventable adverse events after hospital discharge1, 2 and prior research showing that the absence of discharge summaries at postdischarge follow‐up visits increased the risk for hospital readmission.19

We had wondered what effect the electronic discharge summary would have on the length and clarity of discharge summaries. A published commentary suggested that notes performed in EMRs were inordinately long and often difficult to read.20 We were pleased to discover that electronic discharge summaries were similar in length to previous discharge summaries and were rated higher with regard to clarity.

Our study has several limitations. First, many inpatient physicians began to use electronic discharge summaries prior to our creation of the final electronic discharge summary product. We had explicitly instructed physicians to continue to dictate discharge summaries in the first phase of our study. The fact that physicians quickly adopted the practice of completing discharge summaries electronically suggests that they preferred this method for completion and may help to explain the improvement in timeliness. A second limitation, as previously mentioned, is that our study did not measure adverse events directly. Instead, we asked outpatient physicians to estimate the number of their patients discharged in the last 6 months who had sustained a preventable adverse event or near miss related to suboptimal information transfer at discharge. We had limited space in the survey to define the meaning of a preventable adverse event; therefore, the description in the survey does not exactly match previous definitions.1, 2 Finally, the ordinal scale used to assess clarity of discharge summaries has not been previously validated.

In conclusion, the use of an electronic discharge summary significantly improved the quality and timeliness of discharge summaries. The discharge summary comprises a vital component of the information transfer between the inpatient and outpatient settings during the vulnerable period following hospital discharge. As hospitals expand their use of EMRs, they should take advantage of opportunities to leverage functionality to improve quality and timeliness of discharge summaries.

Files
References
  1. Forster AJ,Clark HD,Menard A, et al.Adverse events among medical patient after hospital discharge.CMAJ.2004;170:345349.
  2. Forster AJ,Harvey JF,Peterson JF,Gandhi TK,Bates DW.The incidence and severity of adverse events affecting patients after discharge from the hospital.Ann Intern Med.2003;138:161167.
  3. van Walraven C,Seth R,Laupacis A.Dissemination of discharge summaries. Not reaching follow‐up physicians.Can Fam Physician.2002;48:737742.
  4. van Walraven C,Seth R,Austin PC,Laupacis A.Effect of discharge summary availability during post‐discharge visits on hospital readmission.J Gen Intern Med.2002;17:186192.
  5. Wilson S,Warwick R,Chapman M,Miller R.General practitioner‐hospital communications: a review of discharge summaries.J Qual Clin Practice.2001;21:104108.
  6. Bertrand D,Rancois P,Bosson JL,Fauconnier J,Weil G.Quality assessment of discharge letters in a French university hospital.Int J Health Care Qual Assur.1998;11:9095.
  7. Kripalani S,LeFevre F,Phillips CO,Williams MV,Basaviah P,Baker DW.Deficits in communication and information transfer between hospital‐based and primary care physicians: implications for patient safety and continuity of care.JAMA.2007;297:831841.
  8. O'Leary KJ,Liebovitz DM,Feinglass J,Liss DT,Baker DW.Outpatient physicians' satisfaction with discharge summaries and perceived need for an electronic discharge summary.J Hosp Med.2006;1:317320.
  9. Standard IM.6.10: Hospital accreditation standards.Oakbrook Terrace, IL:Joint Commission on Accreditation of Healthcare Organizations;2006:338340.
  10. Rao P,Andrei A,Fried A,Gonzalez D,Shine D.Assessing quality and efficiency of discharge summaries.Am J Med Qual.2005;20:337343.
  11. Myers JS,Jaipaul K,Kogan JR,Krekun S,Bellini LM,Shea JA.Are discharge summaries teachable? The effects of a discharge summary curriculum on the quality of discharge summaries in an internal medicine residency program.Acad Med.2006;81(10 Suppl):S5S8.
  12. Halasyamani L,Kripalani S,Coleman E, et al.Transition of care for hospitalized elderly patients–development of a discharge checklist for hospitalists.J Hosp Med.2006;1:354360.
  13. Brennan RL,Prediger DJ.Coefficient kappa: some uses, misuses, and alternatives.Educ Psychol Meas.1981;41:687699.
  14. van Walraven C,Laupacis A,Seth R,Wells G.Dictated versus database‐generated discharge summaries: a randomized clinical trial.CMAJ.1999;160:319326.
  15. Lissauer T,Paterson CM,Simons A,Beard RW.Evaluation of computer generated neonatal discharge summaries.Arch Dis Child.1991;66:433436.
  16. Archbold RA,Laji K,Suliman A,Ranjadayalan K,Hemingway H,Timmis AD.Evaluation of a computer‐generated discharge summary for patients with acute coronary syndromes.Br J Gen Pract.1998;48:11631164.
  17. van Walraven C,Duke SM,Weinberg AL,Wells PS.Standardized or narrative discharge summaries: Which do family physicians prefer?Can Fam Phys.1998;44:6269.
  18. Roy CL,Poon EG,Karson AS, et al.Patient safety concerns arising from test results that return after hospital discharge.Ann Intern Med.2005;143:121128.
  19. van Walraven C,Seth R,Austin PC,Laupacis A.Effect of discharge summary availability during post‐discharge visits on hospital readmission.J Gen Intern Med.2002:17;186192.
  20. Hirschtick RE.A piece of my mind. Copy‐and‐paste.JAMA.2006;295:23352336.
Article PDF
Issue
Journal of Hospital Medicine - 4(4)
Publications
Page Number
219-225
Legacy Keywords
computerized physician order entry, discharge summary, electronic medical record, patient safety
Sections
Files
Files
Article PDF
Article PDF

Preventable or ameliorable adverse events have been reported to occur in 12% of patients in the period immediately following hospital discharge.1, 2 A potential contributor to this is the inadequate transfer of clinical information at hospital discharge. The discharge summary comprises a vital component of the information transfer between the inpatient and outpatient settings. Unfortunately, discharge summaries are often unavailable at the time of follow‐up care and often lack important content.37

A growing number of hospitals are implementing electronic medical records (EMR). This creates the opportunity to standardize the content of clinical documentation and creates the potential to assemble, immediately at the time of hospital discharge, major components of a discharge summary. With enhanced communication systems, this information can be delivered in a variety of ways with minimal delay. Previously, we reported the results of a survey of medicine faculty at an urban academic medical center evaluating the timeliness and quality of discharge summaries, the perceived incidence of preventable adverse events related to suboptimal information transfer at discharge, and a needs assessment for an electronically generated discharge summary that we planned to design.8 We now report the results of the follow‐up survey of outpatient physicians and an evaluation of the quality and timeliness of the electronic discharge summary we created.

Materials and Methods

Design

We conducted a pre‐post evaluation of the quality and timeliness of discharge summaries. In the initial phase of the study, we convened an advisory board comprised of 16 Department of Medicine physicians. The advisory board gave input on needs assessment and helped to create a survey to be administered to all medicine faculty with an outpatient practice. All respondents who had at least 1 patient admitted to the hospital within the 6 months prior to the survey were eligible. The results of the initial survey were reviewed with the advisory board and an electronic discharge summary was created with their input. To evaluate its impact, we conducted a repeat survey of all medicine faculty with an outpatient practice approximately 1 year after implementation of the electronic discharge summary.

To complement data received from the outpatient physician survey, a randomly selected sample of discharge summaries from general medical services during the same 3 month period before and after implementation of the electronic discharge summary were rated by 1 of 3 board‐certified internists (D.B.E., N.K., or M.P.L.).

Setting and Participants

The study was conducted at Northwestern Memorial Hospital, a 753‐bed hospital in Chicago, IL. The study was approved by the Institutional Review Board of the Northwestern University Feinberg School of Medicine. General medical patients were admitted to 1 of 2 primary physician services during the study period: a teaching service or a nonteaching hospitalist service. Discharge summaries had traditionally been dictated by inpatient physicians and delivered to outpatient physicians by both mail and facsimile via the medical record department. A recommended template for dictated discharge summaries was provided in the paper paging directory distributed yearly to inpatient physicians.

The hospital implemented an EMR and computerized physician order entry (CPOE) system (PowerChart Millennium; Cerner Corporation, Kansas City, MO) in August 2004. Although all history and physicals and progress notes were documented in the EMR, the system did not provide a method for delivering discharge summaries performed within the EMR to outpatient physician offices. Because of this, inpatient physicians were instructed to continue to dictate discharge summaries during the initial phase of the study.

Approximately 65% of outpatient physicians at the study site used an EMR in their offices during the study. Approximately 10% of outpatient physicians used the same EMR the hospital uses, while approximately 55% used a different EMR (EPIC Hyperspace; EPIC Systems Corporation, Verona, WI). The remaining physicians did not use an EMR in their offices.

Intervention: The Electronic Discharge Summary

A draft electronic discharge summary template was created by including elements ranked as highly important by outpatient physicians in our initial survey8 and elements required by The Joint Commission.9 The draft electronic discharge summary template was reviewed by the advisory board and modifications were made with their input. We automated the insertion of specific patient data elements, such as listed allergies and home medications, into the discharge summary template. We also created an electronic reminder system to inpatient physicians for summaries not completed 24 hours after discharge.

Because the majority of physicians in our initial survey preferred discharge summaries to be delivered either by facsimile or via an EMR, we concentrated our efforts on creating reliable systems for delivery by those routes. We created logic that queried the primary care physician field within the EMR at the time the discharge summary was electronically signed. An automated process then sent the discharge summary via electronic fax to the physician listed in the primary care physician field. Because a large number of outpatient physicians used an EMR different from the hospital's, we also created a process that sent discharge summaries from the hospital EMR into patient charts within this separate EMR.

The draft electronic discharge summary template was available for use in the EMR beginning in July 2005. The final electronic discharge summary, including automated content, physician reminder for incomplete summaries, and delivery systems as described above was implemented in June 2006. Upon implementation, inpatient physicians were instructed via email announcements and group meetings to begin completing electronic discharge summaries using the EMR. Beyond these announcements, inpatient physicians did not receive any specific training with regard to the new discharge summary process. An example of the final electronic discharge summary product is available in the Appendix.

Outpatient Physician Survey

Satisfaction with timeliness and quality of discharge summaries was assessed using a 5‐point Likert scale, where 5 represented very satisfied and 1 represented very dissatisfied. We also asked respondents to estimate the number of their patients who had sustained a preventable adverse event or near miss related to suboptimal transfer of information at discharge. We defined a preventable adverse event as a preventable medical problem or worsening of an existing problem and near miss as an error that did not result in patient harm but easily could have.

The preimplementation survey, accompanied by a cover letter signed by the hospital's chief of staff, was sent out in March 2005. A postcard reminder was sent approximately 2 weeks after the initial mail survey. A second survey was sent to nonresponders 6 weeks after the initial survey. Simultaneously, the survey was also sent in web‐based format to nonresponders via email. The postimplementation survey was sent out in February 2007 using a similar survey process.

Discharge Summary Review

A random sample of discharge summaries completed before and after the implementation of the electronic discharge summary was selected for review. The sample universe consisted of all general medicine service discharges between August and November 2005, before the electronic discharge summary was implemented, and August to November 2006, after implementation. To provide a balanced comparison, the sample was further limited to only the first chronological (index) discharge of a unique patient to home self‐care or home health nursing, with length of stay between 3 and 14 days. A total of 2232 discharges in 2005 and 2570 discharges in 2006 met these criteria. The discharge summary review sample was designed to randomly select approximately 100 discharge summaries meeting the criteria above within each study year, to produce an approximate 200‐record analysis sample. Each of the 3 physician reviewers was assigned to complete an approximately equal number of the 200 primary reviews.

Physician reviewers recorded whether the discharge summary was dictated versus done electronically, the length of the discharge summary (in words), the number of days from discharge to discharge summary completion, the type of service the patient was discharged from, and the author type (medical student, intern, resident, or attending). Physicians reviewers also assessed the overall clarity of discharge summaries using a 5‐point ordinal scale (1 = unintelligible; 2 = hard to read; 3 = neutral; 4 = understandable; and 5 = lucid).

Prior studies have evaluated the quality of discharge summaries using scoring tools created by the investigators.10, 11 We created our own discharge summary scoring tool based on these prior studies, recommendations from the literature,12 and the findings from our initial survey.8 We pilot‐tested the scoring tool and made minor revisions prior to the study. The final scoring tool consisted of 16 essential elements. Reviewers assessed whether each of the 16 essential elements was present, absent, or not applicable. A Discharge Summary Completeness Score was calculated by the number of the 16 essential elements that were rated as present divided by the number of applicable elements for each discharge summary, multiplied by 100 to produce a completeness percentage.

To assess interrater reliability, reviewers were assigned to independently complete second, duplicate reviews of approximately 90 summaries (30 per reviewer). The duplicate review sample was designed to produce approximately 45 paired re‐reviews in each year for reliability assessment. A final sample of 196 available summaries was completed for the main analysis and 174 primary and duplicate reviews were used to establish interrater reliability across 87 reviewer pairs.

Data Analysis

Physician characteristics, including specialty, faculty appointment type, and year of medical school graduation were provided by the hospital's medical staff office. Physician characteristics from before and after the implementation of the electronic discharge summary were compared using chi‐square tests. Likert scale ratings of physician satisfaction with the timeliness and quality of discharge summaries were compared using t‐tests. The proportion of physicians reporting 1 or more preventable adverse event or near miss before the implementation of the electronic discharge summary was compared to postimplementation proportions using chi‐square tests. In addition, we performed multivariate logistic regression to examine the likelihood of physicians reporting any preventable adverse event or near miss related to suboptimal information transfer. The regression models tested the likelihood of 1 or more preventable adverse event or near miss before versus after the implementation of the electronic discharge summary, controlling for physician characteristics and their number of hospitalized patients in the previous 6 months.

The proportions of discharge summary elements found to be present, the proportion of discharge summaries completed within 3 days, and discharge summary readability ratings before and after the implementation of the electronic discharge summary were compared using chi‐square tests; length in words was compared using t‐tests. Preimplementation and postimplementation Discharge Summary Completeness Scores were compared using the Mann‐Whitney U test. Discharge summary score interrater reliability was assessed using the Brennan‐Prediger Kappa for individual elements.13

Results

Outpatient Physician Survey

Physician Characteristics

Two hundred and twenty‐six of 416 (54%) eligible outpatient physicians completed the baseline survey and 256 of 397 (64%) completed the postimplementation survey. As shown in Table 1, there were no significant differences in specialty, faculty appointment type, or number of patients hospitalized between respondents to the survey before compared to respondents after the implementation of the electronic discharge summary. The number of respondents graduating medical school in 1990 or later was higher after implementation of the electronic discharge summary; however, this result was of borderline statistical significance.

Characteristics of Respondents to Outpatient Physician Discharge Summary Satisfaction Surveys
 Preelectronic Discharge Summary (n = 226)Postelectronic Discharge Summary (n = 256)P Value
  • Excludes 5 respondents with missing information on graduation year.

  • Excludes 91 respondents with missing data about the number of their hospitalized patients.

Practice Type  0.23
Generalist, n (%)127 (56.2)130 (50.8) 
Specialist, n (%)99 (43.8)126 (49.2) 
Faculty Appointment  0.38
Full‐time, n (%)104 (46.0)128 (50.0) 
Affiliated, n (%)122 (54.0)128 (50.0) 
Year of medical school graduation*  0.06
Before 1990, n (%)128 (57.4)124 (48.8) 
1990 or later, n (%)95 (42.6)130 (51.2) 
Number of patients hospitalized (last 6 months)  0.56
1‐4, n (%)15 (7.9)24 (12.0) 
5‐10, n (%)62 (32.5)66 (33.0) 
11‐19, n (%)35 (18.3)33 (16.5) 
20 or more, n (%)79 (41.4)77 (38.5) 

Timeliness and Content

Changes in outpatient physician satisfaction with the timeliness and quality of discharge summaries are summarized in Table 2. Satisfaction with the timeliness and quality of discharge summarizes improved significantly after the implementation of the electronic discharge summary (mean standard deviation [SD] timeliness rating, 2.59 1.02 versus 3.34 1.09; P < 0.001, mean quality rating 3.04 0.93 versus 3.64 0.99; P < 0.001).

Outpatient Physician Satisfaction with Timeliness, Quality and Communication
 Likert Scale Mean Score (SD)*
Preelectronic Discharge SummaryPostelectronic Discharge SummaryP Value
  • Outpatient physicians rated items using a 5‐point scale (1 = very dissatisfied; 2 = dissatisfied; 3 = somewhat satisfied; 4 = satisfied; and 5 = very satisfied). There were 14 and 19 missing values for each item, respectively.

Timeliness of the discharge summary2.59 (1.02)3.34 (1.09)<0.001
Quality of the discharge summary3.04 (0.93)3.64 (0.99)<0.001

Medical Error

The effect of the electronic discharge summary on perceived near misses and preventable adverse events is summarized in Table 3. Fewer outpatient physicians felt that 1 or more of their patients hospitalized in the preceding 6 months sustained a near miss due to suboptimal transfer of information after the implementation of the electronic discharge summary (65.7% vs. 52.9%, P = 0.008). Similarly, fewer outpatient physicians felt that 1 or more of their patients hospitalized in the preceding 6 months sustained a preventable adverse event due to suboptimal transfer of information after the implementation of the electronic discharge summary (40.7% vs. 30.2%, P = 0.02). In multivariate logistic regression analyses controlling for physician characteristics and their number of hospitalized patients in the previous 6 months, there was a statistically significant 40% reduction in the odds of a reported near miss (adjusted odds ratio [OR] = 0.60, P = 0.02). Although not quite statistically significant, there was a 33% reduction in the odds of a reported preventable adverse event (OR = 0.67, P = 0.08) after the implementation of the electronic discharge summary.

Reduction in Outpatient Physician Perception of Errors Related to Suboptimal Transfer of Information at Hospital Discharge
 Preelectronic Discharge SummaryPostelectronic Discharge SummaryP Value
  • Defined as an error that did not result in patient harm but easily could have. There were 23 missing responses.

  • Defined as a preventable medical problem or worsening of an existing problem. There were 22 missing responses.

Near miss*   
Number (%) reporting 1142 (65.7)108 (52.9) 
Crude odds ratioRef.0.570.008
Adjusted odds ratioRef.0.600.02
Preventable adverse event   
Number (%) reporting 188 (40.7)62 (30.2) 
Crude odds ratioRef.0.630.03
Adjusted odds ratioRef.0.670.08

Discharge Summary Review

Discharge Summary Characteristics

One hundred and one discharge summaries before implementation of the electronic discharge summary were compared to 95 discharge summaries produced the following year. Characteristics of discharge summaries before and after the implementation of the electronic discharge summary are summarized in Table 4. A large number of discharge summaries (52.5%) were already being typed into the EMR in 2005, prior to the implementation of our final electronic discharge summary product. The number of dictated discharge summaries decreased from 47.5% to 10.5% after implementation of the final electronic discharge summary product (P < 0.001). Discharge summaries were similar in length before and after the implementation of the electronic discharge summary. A higher percentage of discharge summaries were completed within 3 days of discharge after implementation of the electronic discharge summary; however, this result was of borderline statistical significance (59.4% vs. 72.6%; P = 0.05). The type of service from which patients were discharged and the distribution of author types were similar after the implementation of the electronic discharge summary.

Characteristics of Discharge Summaries
 Number (%) or MeanSDP Value
Preelectronic Discharge Summary (n = 101)Postelectronic Discharge Summary (n = 95)
Dictated, n (%)48 (47.5)10 (10.5)<0.001
Length in words, mean SD785 407830 3890.43
Completed within 3 days, n (%)60 (59.4)69 (72.6)0.05
Type of service, n (%)  0.29
Teaching service63 (62.4)66 (69.5) 
Nonteaching hospitalist service38 (37.6)29 (30.5) 
Author type, n (%)  0.62
Fourth year medical student13 (12.9)13 (13.7) 
Intern31 (30.7)37 (38.9) 
Resident19 (18.8)15 (15.8) 
Attending38 (37.6)30 (31.6) 

Because a large percentage of discharge summaries were already being done electronically in 2005, we evaluated the timeliness of dictated discharge summaries compared to electronic discharge summaries across both periods combined (preimplementation and postimplementation of the electronic discharge summary). A higher percentage of electronic discharge summaries were completed within 3 days of discharge as compared to dictated discharge summaries (44.8% versus 74.1%; P < 0.001).

Discharge Summary Completeness Score

The presence or absence of discharge summary elements before and after the implementation of the electronic discharge summary is shown in Table 5. Several elements of the discharge summary were present more often after the implementation of the electronic discharge summary. Specific improvements included discussion of follow‐up issues (52.0% versus 75.8%; P = 0.001, = 0.78), pending test results (13.9% vs. 46.3%; P < 0.001, = 0.92), and information provided to the patient and/or family (85.1% vs. 95.8%; P = 0.01, = 0.91). Significant laboratory findings were present less often after implementation of the electronic discharge summary (66.0% versus 51.1%; P = 0.04, = 0.84). The Discharge Summary Completeness Score was higher after the implementation of the electronic discharge summary (mean 74.1 versus 80.3, P = 0.007). Dictated discharge summaries had a significantly lower Discharge Summary Completeness Score compared to discharge summaries done electronically (71.3 vs. 79.6, P = 0.002) across both periods combined.

Improved Likelihood of Pertinent Content Items Present in Discharge Summary
 Number (%) of Content Items Present*P ValueBrennan‐Prediger Kappa
Preelectronic Discharge Summary (n = 101)Postelectronic Discharge Summary (n = 95)
  • n is less for certain elements as information was not applicable.

Dates of admission and discharge96 (95.0)94 (98.9)0.111.0
Reason for hospitalization100 (99.0)94 (100)0.331.0
Significant findings from history and exam78 (77.2)65 (68.4)0.160.26
Significant laboratory findings64 (66.0)47 (51.1)0.040.84
Significant radiological findings67 (75.3)71 (81.6)0.310.89
Significant findings from other tests41 (63.1)40 (71.4)0.330.88
List of procedures performed45 (81.8)35 (77.8)0.770.99
Procedure report findings49 (80.3)43 (78.2)0.610.92
Stress test report findings7 (100)3 (100)N/A1.0
Pathology report findings11 (39.3)3 (30.0)0.600.91
Discharge diagnosis89 (88.1)86 (93.5)0.200.86
Condition at discharge81 (81.0)80 (85.1)0.450.76
Discharge medications88 (87.1)88 (93.6)0.130.79
Follow‐up issues52 (52.0)72 (75.8)0.0010.78
Pending test results14 (13.9)44 (46.3)<0.0010.92
Information provided to patient and/or family, as appropriate86 (85.1)91 (95.8)0.010.91
Discharge Summary Completeness Score (percent present all applicable items)74.180.30.007 

Significantly more discharge summaries were rated as understandable or lucid after the implementation of the electronic discharge summary (41.6% vs. 59.0%; P = 0.02). In both periods combined, dictated discharge summaries were rated as understandable or lucid less often than electronic discharge summaries (34.5% vs. 56.5%; P < 0.001).

Discussion

Our study found that an electronic discharge summary was well accepted by inpatient physicians and significantly improved the quality and timeliness of discharge summaries. Prior studies have shown that the use of electronically entered discharge summaries improved the timeliness of discharge summaries.1416 However, the discharge summaries used in these studies required manual input of data into a computer system separate from the patient's medical record. To our knowledge, this is the first study to report the impact of discharge summaries generated from an EMR. Leveraging the EMR, we were able to automate the insertion of specific patient data elements, streamline delivery, and create an electronic reminder system to inpatient physicians for summaries not completed 24 hours after discharge.

Prior research has shown that the quality of discharges summaries is improved with the use of standardized content.10, 17 Using a standardized template for the electronic discharge summary, we likewise demonstrated improved quality of discharge summaries. Key discharge summary elements, specifically discussion of follow‐up issues, pending test results, and information provided to the patient and/or family, were present more reliably after the implementation of the electronic discharge summary. The importance of identifying pending test results is underscored by a recent study showing that many patients are discharged from hospitals with test results still pending, and that physicians are often unaware when results are abnormal.18 One discharge summary element, significant laboratory findings, was present less often after the implementation of the electronic discharge summary. Our template did not designate significant laboratory findings under a separate heading. Instead, we used a heading entitled Key Results (labs, imaging, pathology). Physicians completing the discharge summaries may have prioritized the report of imaging and pathology results in this section. A simple revision of our discharge summary template to include a separate heading for significant laboratory findings may result in improvement in this regard.

Timeliness of discharge summaries was improved in our study, but remained less than optimal. Although nearly three‐quarters of electronic discharge summaries were completed within 3 days of discharge, our ultimate goal is to have 100% of discharge summaries completed within 3 days. This is especially important for complicated patients requiring outpatient follow‐up soon after discharge. We are currently in the process of designing further modifications to the electronic discharge summary completion process. One modification that may be beneficial is the automation of additional patient specific data elements into the discharge summary. We also plan to link performance of medication reconciliation, completion of patient discharge instructions, and completion of the discharge summary into an integrated set of activities performed in the EMR prior to patient discharge.

We found that fewer outpatient physicians reported 1 or more of their patients having a preventable adverse event or near miss as a result of suboptimal transfer of information at discharge after the implementation of the electronic discharge summary. Although we did not measure preventable adverse events directly in our study, this is an important finding in light of the large number of patients who sustain preventable adverse events after hospital discharge1, 2 and prior research showing that the absence of discharge summaries at postdischarge follow‐up visits increased the risk for hospital readmission.19

We had wondered what effect the electronic discharge summary would have on the length and clarity of discharge summaries. A published commentary suggested that notes performed in EMRs were inordinately long and often difficult to read.20 We were pleased to discover that electronic discharge summaries were similar in length to previous discharge summaries and were rated higher with regard to clarity.

Our study has several limitations. First, many inpatient physicians began to use electronic discharge summaries prior to our creation of the final electronic discharge summary product. We had explicitly instructed physicians to continue to dictate discharge summaries in the first phase of our study. The fact that physicians quickly adopted the practice of completing discharge summaries electronically suggests that they preferred this method for completion and may help to explain the improvement in timeliness. A second limitation, as previously mentioned, is that our study did not measure adverse events directly. Instead, we asked outpatient physicians to estimate the number of their patients discharged in the last 6 months who had sustained a preventable adverse event or near miss related to suboptimal information transfer at discharge. We had limited space in the survey to define the meaning of a preventable adverse event; therefore, the description in the survey does not exactly match previous definitions.1, 2 Finally, the ordinal scale used to assess clarity of discharge summaries has not been previously validated.

In conclusion, the use of an electronic discharge summary significantly improved the quality and timeliness of discharge summaries. The discharge summary comprises a vital component of the information transfer between the inpatient and outpatient settings during the vulnerable period following hospital discharge. As hospitals expand their use of EMRs, they should take advantage of opportunities to leverage functionality to improve quality and timeliness of discharge summaries.

Preventable or ameliorable adverse events have been reported to occur in 12% of patients in the period immediately following hospital discharge.1, 2 A potential contributor to this is the inadequate transfer of clinical information at hospital discharge. The discharge summary comprises a vital component of the information transfer between the inpatient and outpatient settings. Unfortunately, discharge summaries are often unavailable at the time of follow‐up care and often lack important content.37

A growing number of hospitals are implementing electronic medical records (EMR). This creates the opportunity to standardize the content of clinical documentation and creates the potential to assemble, immediately at the time of hospital discharge, major components of a discharge summary. With enhanced communication systems, this information can be delivered in a variety of ways with minimal delay. Previously, we reported the results of a survey of medicine faculty at an urban academic medical center evaluating the timeliness and quality of discharge summaries, the perceived incidence of preventable adverse events related to suboptimal information transfer at discharge, and a needs assessment for an electronically generated discharge summary that we planned to design.8 We now report the results of the follow‐up survey of outpatient physicians and an evaluation of the quality and timeliness of the electronic discharge summary we created.

Materials and Methods

Design

We conducted a pre‐post evaluation of the quality and timeliness of discharge summaries. In the initial phase of the study, we convened an advisory board comprised of 16 Department of Medicine physicians. The advisory board gave input on needs assessment and helped to create a survey to be administered to all medicine faculty with an outpatient practice. All respondents who had at least 1 patient admitted to the hospital within the 6 months prior to the survey were eligible. The results of the initial survey were reviewed with the advisory board and an electronic discharge summary was created with their input. To evaluate its impact, we conducted a repeat survey of all medicine faculty with an outpatient practice approximately 1 year after implementation of the electronic discharge summary.

To complement data received from the outpatient physician survey, a randomly selected sample of discharge summaries from general medical services during the same 3 month period before and after implementation of the electronic discharge summary were rated by 1 of 3 board‐certified internists (D.B.E., N.K., or M.P.L.).

Setting and Participants

The study was conducted at Northwestern Memorial Hospital, a 753‐bed hospital in Chicago, IL. The study was approved by the Institutional Review Board of the Northwestern University Feinberg School of Medicine. General medical patients were admitted to 1 of 2 primary physician services during the study period: a teaching service or a nonteaching hospitalist service. Discharge summaries had traditionally been dictated by inpatient physicians and delivered to outpatient physicians by both mail and facsimile via the medical record department. A recommended template for dictated discharge summaries was provided in the paper paging directory distributed yearly to inpatient physicians.

The hospital implemented an EMR and computerized physician order entry (CPOE) system (PowerChart Millennium; Cerner Corporation, Kansas City, MO) in August 2004. Although all history and physicals and progress notes were documented in the EMR, the system did not provide a method for delivering discharge summaries performed within the EMR to outpatient physician offices. Because of this, inpatient physicians were instructed to continue to dictate discharge summaries during the initial phase of the study.

Approximately 65% of outpatient physicians at the study site used an EMR in their offices during the study. Approximately 10% of outpatient physicians used the same EMR the hospital uses, while approximately 55% used a different EMR (EPIC Hyperspace; EPIC Systems Corporation, Verona, WI). The remaining physicians did not use an EMR in their offices.

Intervention: The Electronic Discharge Summary

A draft electronic discharge summary template was created by including elements ranked as highly important by outpatient physicians in our initial survey8 and elements required by The Joint Commission.9 The draft electronic discharge summary template was reviewed by the advisory board and modifications were made with their input. We automated the insertion of specific patient data elements, such as listed allergies and home medications, into the discharge summary template. We also created an electronic reminder system to inpatient physicians for summaries not completed 24 hours after discharge.

Because the majority of physicians in our initial survey preferred discharge summaries to be delivered either by facsimile or via an EMR, we concentrated our efforts on creating reliable systems for delivery by those routes. We created logic that queried the primary care physician field within the EMR at the time the discharge summary was electronically signed. An automated process then sent the discharge summary via electronic fax to the physician listed in the primary care physician field. Because a large number of outpatient physicians used an EMR different from the hospital's, we also created a process that sent discharge summaries from the hospital EMR into patient charts within this separate EMR.

The draft electronic discharge summary template was available for use in the EMR beginning in July 2005. The final electronic discharge summary, including automated content, physician reminder for incomplete summaries, and delivery systems as described above was implemented in June 2006. Upon implementation, inpatient physicians were instructed via email announcements and group meetings to begin completing electronic discharge summaries using the EMR. Beyond these announcements, inpatient physicians did not receive any specific training with regard to the new discharge summary process. An example of the final electronic discharge summary product is available in the Appendix.

Outpatient Physician Survey

Satisfaction with timeliness and quality of discharge summaries was assessed using a 5‐point Likert scale, where 5 represented very satisfied and 1 represented very dissatisfied. We also asked respondents to estimate the number of their patients who had sustained a preventable adverse event or near miss related to suboptimal transfer of information at discharge. We defined a preventable adverse event as a preventable medical problem or worsening of an existing problem and near miss as an error that did not result in patient harm but easily could have.

The preimplementation survey, accompanied by a cover letter signed by the hospital's chief of staff, was sent out in March 2005. A postcard reminder was sent approximately 2 weeks after the initial mail survey. A second survey was sent to nonresponders 6 weeks after the initial survey. Simultaneously, the survey was also sent in web‐based format to nonresponders via email. The postimplementation survey was sent out in February 2007 using a similar survey process.

Discharge Summary Review

A random sample of discharge summaries completed before and after the implementation of the electronic discharge summary was selected for review. The sample universe consisted of all general medicine service discharges between August and November 2005, before the electronic discharge summary was implemented, and August to November 2006, after implementation. To provide a balanced comparison, the sample was further limited to only the first chronological (index) discharge of a unique patient to home self‐care or home health nursing, with length of stay between 3 and 14 days. A total of 2232 discharges in 2005 and 2570 discharges in 2006 met these criteria. The discharge summary review sample was designed to randomly select approximately 100 discharge summaries meeting the criteria above within each study year, to produce an approximate 200‐record analysis sample. Each of the 3 physician reviewers was assigned to complete an approximately equal number of the 200 primary reviews.

Physician reviewers recorded whether the discharge summary was dictated versus done electronically, the length of the discharge summary (in words), the number of days from discharge to discharge summary completion, the type of service the patient was discharged from, and the author type (medical student, intern, resident, or attending). Physicians reviewers also assessed the overall clarity of discharge summaries using a 5‐point ordinal scale (1 = unintelligible; 2 = hard to read; 3 = neutral; 4 = understandable; and 5 = lucid).

Prior studies have evaluated the quality of discharge summaries using scoring tools created by the investigators.10, 11 We created our own discharge summary scoring tool based on these prior studies, recommendations from the literature,12 and the findings from our initial survey.8 We pilot‐tested the scoring tool and made minor revisions prior to the study. The final scoring tool consisted of 16 essential elements. Reviewers assessed whether each of the 16 essential elements was present, absent, or not applicable. A Discharge Summary Completeness Score was calculated by the number of the 16 essential elements that were rated as present divided by the number of applicable elements for each discharge summary, multiplied by 100 to produce a completeness percentage.

To assess interrater reliability, reviewers were assigned to independently complete second, duplicate reviews of approximately 90 summaries (30 per reviewer). The duplicate review sample was designed to produce approximately 45 paired re‐reviews in each year for reliability assessment. A final sample of 196 available summaries was completed for the main analysis and 174 primary and duplicate reviews were used to establish interrater reliability across 87 reviewer pairs.

Data Analysis

Physician characteristics, including specialty, faculty appointment type, and year of medical school graduation were provided by the hospital's medical staff office. Physician characteristics from before and after the implementation of the electronic discharge summary were compared using chi‐square tests. Likert scale ratings of physician satisfaction with the timeliness and quality of discharge summaries were compared using t‐tests. The proportion of physicians reporting 1 or more preventable adverse event or near miss before the implementation of the electronic discharge summary was compared to postimplementation proportions using chi‐square tests. In addition, we performed multivariate logistic regression to examine the likelihood of physicians reporting any preventable adverse event or near miss related to suboptimal information transfer. The regression models tested the likelihood of 1 or more preventable adverse event or near miss before versus after the implementation of the electronic discharge summary, controlling for physician characteristics and their number of hospitalized patients in the previous 6 months.

The proportions of discharge summary elements found to be present, the proportion of discharge summaries completed within 3 days, and discharge summary readability ratings before and after the implementation of the electronic discharge summary were compared using chi‐square tests; length in words was compared using t‐tests. Preimplementation and postimplementation Discharge Summary Completeness Scores were compared using the Mann‐Whitney U test. Discharge summary score interrater reliability was assessed using the Brennan‐Prediger Kappa for individual elements.13

Results

Outpatient Physician Survey

Physician Characteristics

Two hundred and twenty‐six of 416 (54%) eligible outpatient physicians completed the baseline survey and 256 of 397 (64%) completed the postimplementation survey. As shown in Table 1, there were no significant differences in specialty, faculty appointment type, or number of patients hospitalized between respondents to the survey before compared to respondents after the implementation of the electronic discharge summary. The number of respondents graduating medical school in 1990 or later was higher after implementation of the electronic discharge summary; however, this result was of borderline statistical significance.

Characteristics of Respondents to Outpatient Physician Discharge Summary Satisfaction Surveys
 Preelectronic Discharge Summary (n = 226)Postelectronic Discharge Summary (n = 256)P Value
  • Excludes 5 respondents with missing information on graduation year.

  • Excludes 91 respondents with missing data about the number of their hospitalized patients.

Practice Type  0.23
Generalist, n (%)127 (56.2)130 (50.8) 
Specialist, n (%)99 (43.8)126 (49.2) 
Faculty Appointment  0.38
Full‐time, n (%)104 (46.0)128 (50.0) 
Affiliated, n (%)122 (54.0)128 (50.0) 
Year of medical school graduation*  0.06
Before 1990, n (%)128 (57.4)124 (48.8) 
1990 or later, n (%)95 (42.6)130 (51.2) 
Number of patients hospitalized (last 6 months)  0.56
1‐4, n (%)15 (7.9)24 (12.0) 
5‐10, n (%)62 (32.5)66 (33.0) 
11‐19, n (%)35 (18.3)33 (16.5) 
20 or more, n (%)79 (41.4)77 (38.5) 

Timeliness and Content

Changes in outpatient physician satisfaction with the timeliness and quality of discharge summaries are summarized in Table 2. Satisfaction with the timeliness and quality of discharge summarizes improved significantly after the implementation of the electronic discharge summary (mean standard deviation [SD] timeliness rating, 2.59 1.02 versus 3.34 1.09; P < 0.001, mean quality rating 3.04 0.93 versus 3.64 0.99; P < 0.001).

Outpatient Physician Satisfaction with Timeliness, Quality and Communication
 Likert Scale Mean Score (SD)*
Preelectronic Discharge SummaryPostelectronic Discharge SummaryP Value
  • Outpatient physicians rated items using a 5‐point scale (1 = very dissatisfied; 2 = dissatisfied; 3 = somewhat satisfied; 4 = satisfied; and 5 = very satisfied). There were 14 and 19 missing values for each item, respectively.

Timeliness of the discharge summary2.59 (1.02)3.34 (1.09)<0.001
Quality of the discharge summary3.04 (0.93)3.64 (0.99)<0.001

Medical Error

The effect of the electronic discharge summary on perceived near misses and preventable adverse events is summarized in Table 3. Fewer outpatient physicians felt that 1 or more of their patients hospitalized in the preceding 6 months sustained a near miss due to suboptimal transfer of information after the implementation of the electronic discharge summary (65.7% vs. 52.9%, P = 0.008). Similarly, fewer outpatient physicians felt that 1 or more of their patients hospitalized in the preceding 6 months sustained a preventable adverse event due to suboptimal transfer of information after the implementation of the electronic discharge summary (40.7% vs. 30.2%, P = 0.02). In multivariate logistic regression analyses controlling for physician characteristics and their number of hospitalized patients in the previous 6 months, there was a statistically significant 40% reduction in the odds of a reported near miss (adjusted odds ratio [OR] = 0.60, P = 0.02). Although not quite statistically significant, there was a 33% reduction in the odds of a reported preventable adverse event (OR = 0.67, P = 0.08) after the implementation of the electronic discharge summary.

Reduction in Outpatient Physician Perception of Errors Related to Suboptimal Transfer of Information at Hospital Discharge
 Preelectronic Discharge SummaryPostelectronic Discharge SummaryP Value
  • Defined as an error that did not result in patient harm but easily could have. There were 23 missing responses.

  • Defined as a preventable medical problem or worsening of an existing problem. There were 22 missing responses.

Near miss*   
Number (%) reporting 1142 (65.7)108 (52.9) 
Crude odds ratioRef.0.570.008
Adjusted odds ratioRef.0.600.02
Preventable adverse event   
Number (%) reporting 188 (40.7)62 (30.2) 
Crude odds ratioRef.0.630.03
Adjusted odds ratioRef.0.670.08

Discharge Summary Review

Discharge Summary Characteristics

One hundred and one discharge summaries before implementation of the electronic discharge summary were compared to 95 discharge summaries produced the following year. Characteristics of discharge summaries before and after the implementation of the electronic discharge summary are summarized in Table 4. A large number of discharge summaries (52.5%) were already being typed into the EMR in 2005, prior to the implementation of our final electronic discharge summary product. The number of dictated discharge summaries decreased from 47.5% to 10.5% after implementation of the final electronic discharge summary product (P < 0.001). Discharge summaries were similar in length before and after the implementation of the electronic discharge summary. A higher percentage of discharge summaries were completed within 3 days of discharge after implementation of the electronic discharge summary; however, this result was of borderline statistical significance (59.4% vs. 72.6%; P = 0.05). The type of service from which patients were discharged and the distribution of author types were similar after the implementation of the electronic discharge summary.

Characteristics of Discharge Summaries
 Number (%) or MeanSDP Value
Preelectronic Discharge Summary (n = 101)Postelectronic Discharge Summary (n = 95)
Dictated, n (%)48 (47.5)10 (10.5)<0.001
Length in words, mean SD785 407830 3890.43
Completed within 3 days, n (%)60 (59.4)69 (72.6)0.05
Type of service, n (%)  0.29
Teaching service63 (62.4)66 (69.5) 
Nonteaching hospitalist service38 (37.6)29 (30.5) 
Author type, n (%)  0.62
Fourth year medical student13 (12.9)13 (13.7) 
Intern31 (30.7)37 (38.9) 
Resident19 (18.8)15 (15.8) 
Attending38 (37.6)30 (31.6) 

Because a large percentage of discharge summaries were already being done electronically in 2005, we evaluated the timeliness of dictated discharge summaries compared to electronic discharge summaries across both periods combined (preimplementation and postimplementation of the electronic discharge summary). A higher percentage of electronic discharge summaries were completed within 3 days of discharge as compared to dictated discharge summaries (44.8% versus 74.1%; P < 0.001).

Discharge Summary Completeness Score

The presence or absence of discharge summary elements before and after the implementation of the electronic discharge summary is shown in Table 5. Several elements of the discharge summary were present more often after the implementation of the electronic discharge summary. Specific improvements included discussion of follow‐up issues (52.0% versus 75.8%; P = 0.001, = 0.78), pending test results (13.9% vs. 46.3%; P < 0.001, = 0.92), and information provided to the patient and/or family (85.1% vs. 95.8%; P = 0.01, = 0.91). Significant laboratory findings were present less often after implementation of the electronic discharge summary (66.0% versus 51.1%; P = 0.04, = 0.84). The Discharge Summary Completeness Score was higher after the implementation of the electronic discharge summary (mean 74.1 versus 80.3, P = 0.007). Dictated discharge summaries had a significantly lower Discharge Summary Completeness Score compared to discharge summaries done electronically (71.3 vs. 79.6, P = 0.002) across both periods combined.

Improved Likelihood of Pertinent Content Items Present in Discharge Summary
 Number (%) of Content Items Present*P ValueBrennan‐Prediger Kappa
Preelectronic Discharge Summary (n = 101)Postelectronic Discharge Summary (n = 95)
  • n is less for certain elements as information was not applicable.

Dates of admission and discharge96 (95.0)94 (98.9)0.111.0
Reason for hospitalization100 (99.0)94 (100)0.331.0
Significant findings from history and exam78 (77.2)65 (68.4)0.160.26
Significant laboratory findings64 (66.0)47 (51.1)0.040.84
Significant radiological findings67 (75.3)71 (81.6)0.310.89
Significant findings from other tests41 (63.1)40 (71.4)0.330.88
List of procedures performed45 (81.8)35 (77.8)0.770.99
Procedure report findings49 (80.3)43 (78.2)0.610.92
Stress test report findings7 (100)3 (100)N/A1.0
Pathology report findings11 (39.3)3 (30.0)0.600.91
Discharge diagnosis89 (88.1)86 (93.5)0.200.86
Condition at discharge81 (81.0)80 (85.1)0.450.76
Discharge medications88 (87.1)88 (93.6)0.130.79
Follow‐up issues52 (52.0)72 (75.8)0.0010.78
Pending test results14 (13.9)44 (46.3)<0.0010.92
Information provided to patient and/or family, as appropriate86 (85.1)91 (95.8)0.010.91
Discharge Summary Completeness Score (percent present all applicable items)74.180.30.007 

Significantly more discharge summaries were rated as understandable or lucid after the implementation of the electronic discharge summary (41.6% vs. 59.0%; P = 0.02). In both periods combined, dictated discharge summaries were rated as understandable or lucid less often than electronic discharge summaries (34.5% vs. 56.5%; P < 0.001).

Discussion

Our study found that an electronic discharge summary was well accepted by inpatient physicians and significantly improved the quality and timeliness of discharge summaries. Prior studies have shown that the use of electronically entered discharge summaries improved the timeliness of discharge summaries.1416 However, the discharge summaries used in these studies required manual input of data into a computer system separate from the patient's medical record. To our knowledge, this is the first study to report the impact of discharge summaries generated from an EMR. Leveraging the EMR, we were able to automate the insertion of specific patient data elements, streamline delivery, and create an electronic reminder system to inpatient physicians for summaries not completed 24 hours after discharge.

Prior research has shown that the quality of discharges summaries is improved with the use of standardized content.10, 17 Using a standardized template for the electronic discharge summary, we likewise demonstrated improved quality of discharge summaries. Key discharge summary elements, specifically discussion of follow‐up issues, pending test results, and information provided to the patient and/or family, were present more reliably after the implementation of the electronic discharge summary. The importance of identifying pending test results is underscored by a recent study showing that many patients are discharged from hospitals with test results still pending, and that physicians are often unaware when results are abnormal.18 One discharge summary element, significant laboratory findings, was present less often after the implementation of the electronic discharge summary. Our template did not designate significant laboratory findings under a separate heading. Instead, we used a heading entitled Key Results (labs, imaging, pathology). Physicians completing the discharge summaries may have prioritized the report of imaging and pathology results in this section. A simple revision of our discharge summary template to include a separate heading for significant laboratory findings may result in improvement in this regard.

Timeliness of discharge summaries was improved in our study, but remained less than optimal. Although nearly three‐quarters of electronic discharge summaries were completed within 3 days of discharge, our ultimate goal is to have 100% of discharge summaries completed within 3 days. This is especially important for complicated patients requiring outpatient follow‐up soon after discharge. We are currently in the process of designing further modifications to the electronic discharge summary completion process. One modification that may be beneficial is the automation of additional patient specific data elements into the discharge summary. We also plan to link performance of medication reconciliation, completion of patient discharge instructions, and completion of the discharge summary into an integrated set of activities performed in the EMR prior to patient discharge.

We found that fewer outpatient physicians reported 1 or more of their patients having a preventable adverse event or near miss as a result of suboptimal transfer of information at discharge after the implementation of the electronic discharge summary. Although we did not measure preventable adverse events directly in our study, this is an important finding in light of the large number of patients who sustain preventable adverse events after hospital discharge1, 2 and prior research showing that the absence of discharge summaries at postdischarge follow‐up visits increased the risk for hospital readmission.19

We had wondered what effect the electronic discharge summary would have on the length and clarity of discharge summaries. A published commentary suggested that notes performed in EMRs were inordinately long and often difficult to read.20 We were pleased to discover that electronic discharge summaries were similar in length to previous discharge summaries and were rated higher with regard to clarity.

Our study has several limitations. First, many inpatient physicians began to use electronic discharge summaries prior to our creation of the final electronic discharge summary product. We had explicitly instructed physicians to continue to dictate discharge summaries in the first phase of our study. The fact that physicians quickly adopted the practice of completing discharge summaries electronically suggests that they preferred this method for completion and may help to explain the improvement in timeliness. A second limitation, as previously mentioned, is that our study did not measure adverse events directly. Instead, we asked outpatient physicians to estimate the number of their patients discharged in the last 6 months who had sustained a preventable adverse event or near miss related to suboptimal information transfer at discharge. We had limited space in the survey to define the meaning of a preventable adverse event; therefore, the description in the survey does not exactly match previous definitions.1, 2 Finally, the ordinal scale used to assess clarity of discharge summaries has not been previously validated.

In conclusion, the use of an electronic discharge summary significantly improved the quality and timeliness of discharge summaries. The discharge summary comprises a vital component of the information transfer between the inpatient and outpatient settings during the vulnerable period following hospital discharge. As hospitals expand their use of EMRs, they should take advantage of opportunities to leverage functionality to improve quality and timeliness of discharge summaries.

References
  1. Forster AJ,Clark HD,Menard A, et al.Adverse events among medical patient after hospital discharge.CMAJ.2004;170:345349.
  2. Forster AJ,Harvey JF,Peterson JF,Gandhi TK,Bates DW.The incidence and severity of adverse events affecting patients after discharge from the hospital.Ann Intern Med.2003;138:161167.
  3. van Walraven C,Seth R,Laupacis A.Dissemination of discharge summaries. Not reaching follow‐up physicians.Can Fam Physician.2002;48:737742.
  4. van Walraven C,Seth R,Austin PC,Laupacis A.Effect of discharge summary availability during post‐discharge visits on hospital readmission.J Gen Intern Med.2002;17:186192.
  5. Wilson S,Warwick R,Chapman M,Miller R.General practitioner‐hospital communications: a review of discharge summaries.J Qual Clin Practice.2001;21:104108.
  6. Bertrand D,Rancois P,Bosson JL,Fauconnier J,Weil G.Quality assessment of discharge letters in a French university hospital.Int J Health Care Qual Assur.1998;11:9095.
  7. Kripalani S,LeFevre F,Phillips CO,Williams MV,Basaviah P,Baker DW.Deficits in communication and information transfer between hospital‐based and primary care physicians: implications for patient safety and continuity of care.JAMA.2007;297:831841.
  8. O'Leary KJ,Liebovitz DM,Feinglass J,Liss DT,Baker DW.Outpatient physicians' satisfaction with discharge summaries and perceived need for an electronic discharge summary.J Hosp Med.2006;1:317320.
  9. Standard IM.6.10: Hospital accreditation standards.Oakbrook Terrace, IL:Joint Commission on Accreditation of Healthcare Organizations;2006:338340.
  10. Rao P,Andrei A,Fried A,Gonzalez D,Shine D.Assessing quality and efficiency of discharge summaries.Am J Med Qual.2005;20:337343.
  11. Myers JS,Jaipaul K,Kogan JR,Krekun S,Bellini LM,Shea JA.Are discharge summaries teachable? The effects of a discharge summary curriculum on the quality of discharge summaries in an internal medicine residency program.Acad Med.2006;81(10 Suppl):S5S8.
  12. Halasyamani L,Kripalani S,Coleman E, et al.Transition of care for hospitalized elderly patients–development of a discharge checklist for hospitalists.J Hosp Med.2006;1:354360.
  13. Brennan RL,Prediger DJ.Coefficient kappa: some uses, misuses, and alternatives.Educ Psychol Meas.1981;41:687699.
  14. van Walraven C,Laupacis A,Seth R,Wells G.Dictated versus database‐generated discharge summaries: a randomized clinical trial.CMAJ.1999;160:319326.
  15. Lissauer T,Paterson CM,Simons A,Beard RW.Evaluation of computer generated neonatal discharge summaries.Arch Dis Child.1991;66:433436.
  16. Archbold RA,Laji K,Suliman A,Ranjadayalan K,Hemingway H,Timmis AD.Evaluation of a computer‐generated discharge summary for patients with acute coronary syndromes.Br J Gen Pract.1998;48:11631164.
  17. van Walraven C,Duke SM,Weinberg AL,Wells PS.Standardized or narrative discharge summaries: Which do family physicians prefer?Can Fam Phys.1998;44:6269.
  18. Roy CL,Poon EG,Karson AS, et al.Patient safety concerns arising from test results that return after hospital discharge.Ann Intern Med.2005;143:121128.
  19. van Walraven C,Seth R,Austin PC,Laupacis A.Effect of discharge summary availability during post‐discharge visits on hospital readmission.J Gen Intern Med.2002:17;186192.
  20. Hirschtick RE.A piece of my mind. Copy‐and‐paste.JAMA.2006;295:23352336.
References
  1. Forster AJ,Clark HD,Menard A, et al.Adverse events among medical patient after hospital discharge.CMAJ.2004;170:345349.
  2. Forster AJ,Harvey JF,Peterson JF,Gandhi TK,Bates DW.The incidence and severity of adverse events affecting patients after discharge from the hospital.Ann Intern Med.2003;138:161167.
  3. van Walraven C,Seth R,Laupacis A.Dissemination of discharge summaries. Not reaching follow‐up physicians.Can Fam Physician.2002;48:737742.
  4. van Walraven C,Seth R,Austin PC,Laupacis A.Effect of discharge summary availability during post‐discharge visits on hospital readmission.J Gen Intern Med.2002;17:186192.
  5. Wilson S,Warwick R,Chapman M,Miller R.General practitioner‐hospital communications: a review of discharge summaries.J Qual Clin Practice.2001;21:104108.
  6. Bertrand D,Rancois P,Bosson JL,Fauconnier J,Weil G.Quality assessment of discharge letters in a French university hospital.Int J Health Care Qual Assur.1998;11:9095.
  7. Kripalani S,LeFevre F,Phillips CO,Williams MV,Basaviah P,Baker DW.Deficits in communication and information transfer between hospital‐based and primary care physicians: implications for patient safety and continuity of care.JAMA.2007;297:831841.
  8. O'Leary KJ,Liebovitz DM,Feinglass J,Liss DT,Baker DW.Outpatient physicians' satisfaction with discharge summaries and perceived need for an electronic discharge summary.J Hosp Med.2006;1:317320.
  9. Standard IM.6.10: Hospital accreditation standards.Oakbrook Terrace, IL:Joint Commission on Accreditation of Healthcare Organizations;2006:338340.
  10. Rao P,Andrei A,Fried A,Gonzalez D,Shine D.Assessing quality and efficiency of discharge summaries.Am J Med Qual.2005;20:337343.
  11. Myers JS,Jaipaul K,Kogan JR,Krekun S,Bellini LM,Shea JA.Are discharge summaries teachable? The effects of a discharge summary curriculum on the quality of discharge summaries in an internal medicine residency program.Acad Med.2006;81(10 Suppl):S5S8.
  12. Halasyamani L,Kripalani S,Coleman E, et al.Transition of care for hospitalized elderly patients–development of a discharge checklist for hospitalists.J Hosp Med.2006;1:354360.
  13. Brennan RL,Prediger DJ.Coefficient kappa: some uses, misuses, and alternatives.Educ Psychol Meas.1981;41:687699.
  14. van Walraven C,Laupacis A,Seth R,Wells G.Dictated versus database‐generated discharge summaries: a randomized clinical trial.CMAJ.1999;160:319326.
  15. Lissauer T,Paterson CM,Simons A,Beard RW.Evaluation of computer generated neonatal discharge summaries.Arch Dis Child.1991;66:433436.
  16. Archbold RA,Laji K,Suliman A,Ranjadayalan K,Hemingway H,Timmis AD.Evaluation of a computer‐generated discharge summary for patients with acute coronary syndromes.Br J Gen Pract.1998;48:11631164.
  17. van Walraven C,Duke SM,Weinberg AL,Wells PS.Standardized or narrative discharge summaries: Which do family physicians prefer?Can Fam Phys.1998;44:6269.
  18. Roy CL,Poon EG,Karson AS, et al.Patient safety concerns arising from test results that return after hospital discharge.Ann Intern Med.2005;143:121128.
  19. van Walraven C,Seth R,Austin PC,Laupacis A.Effect of discharge summary availability during post‐discharge visits on hospital readmission.J Gen Intern Med.2002:17;186192.
  20. Hirschtick RE.A piece of my mind. Copy‐and‐paste.JAMA.2006;295:23352336.
Issue
Journal of Hospital Medicine - 4(4)
Issue
Journal of Hospital Medicine - 4(4)
Page Number
219-225
Page Number
219-225
Publications
Publications
Article Type
Display Headline
Creating a better discharge summary: Improvement in quality and timeliness using an electronic discharge summary
Display Headline
Creating a better discharge summary: Improvement in quality and timeliness using an electronic discharge summary
Legacy Keywords
computerized physician order entry, discharge summary, electronic medical record, patient safety
Legacy Keywords
computerized physician order entry, discharge summary, electronic medical record, patient safety
Sections
Article Source

Copyright © 2009 Society of Hospital Medicine

Disallow All Ads
Correspondence Location
Division of Hospital Medicine, 259 East Erie Street, Suite 475, Chicago, IL, 60611
Content Gating
No Gating (article Unlocked/Free)
Alternative CME
Article PDF Media
Media Files

Discharge Summary Survey

Article Type
Changed
Mon, 01/02/2017 - 19:34
Display Headline
Outpatient physicians' satisfaction with discharge summaries and perceived need for an electronic discharge summary

Twelve percent of patients have been reported to have preventable or ameliorable adverse events in the period immediately following hospital discharge.1, 2 A potential contributor to the number of adverse events is inadequate transfer of clinical information at hospital discharge. The discharge summary is a vital component of the transfer of information from the inpatient to the outpatient setting. Unfortunately, discharge summaries are often unavailable when follow‐up care occurs and often lack important content.36

Many hospitals are implementing an electronic medical record systems. This creates the opportunity at hospital discharge to immediately assemble the major components of a discharge summary. With enhanced communication systems, this information can be delivered in a variety of ways with minimal delay. We report the results and evaluation of a survey of medicine faculty at an urban academic medical center about the timeliness and quality of discharge summaries, the perceived incidence of adverse events related to suboptimal information transfer at discharge, and the need for the electronically generated discharge summary we plan to design.

METHODS

Study Site

The study was conducted at a 753‐bed academic hospital in Chicago, Illinois. Discharge summaries have traditionally been dictated by inpatient physicians and delivered to outpatient physicians by both mail and facsimile via the medical records department. The hospital has used an electronic medical record and computerized physician order entry system (PowerChart Millennium from Cerner Corporation) since August 2004. Although all history and physicals and progress notes were documented in the electronic medical record, the system did not provide a method for delivering the discharge summaries contained in the electronic medical record to outpatient physician offices. Because of this, inpatient physicians continued to dictate discharge summaries during this study.

Participants

An advisory board consisting of 16 physicians from the Department of Medicine was convened. The advisory board gave input on needs assessment and helped to create a survey to be administered to all 425 medicine faculty who have an outpatient practice. All respondents who had at least 1 patient admitted to the hospital within the 6 months prior to the survey were eligible.

Survey Content

Our survey consisted of 2 parts. In the first part, we asked respondents to estimate how many of their patients had been discharged from the hospital in the past 6 months and to reflect on these patients as they completed the survey. Satisfaction with the timeliness and quality of discharge summaries was assessed using a 5‐point Likert scale, from 5, very satisfied, to 1, very unsatisfied. The frequency of hospital follow‐up of a patient occurring prior to arrival of the discharge summary was assessed as the percentage of times this occurred in 20% increments (0%‐19%, 20%‐39%, 40%‐59%, 60%‐79%, and 80%‐100%). The number of discharge summaries missing critical information and the number of summaries containing unnecessary information were similarly assessed using 20% increments. We then asked respondents to estimate the number of patients who had sustained a preventable adverse event related to suboptimal transfer of information at discharge. We defined a preventable adverse event as a preventable medical problem or worsening of an existing problem.

In the second part of the survey, we elicited preferences for discharge summary content and method of delivery. We assessed preferences for discharge summary content by asking respondents to rank items on a scale from 1 to 10, from 10, most important, to 1, least important. Preferences for delivery of discharge summaries were assessed by asking respondents to indicate one or more delivery methods, including facsimile, mail, the electronic medical record, and E‐mail.

Survey Process

The survey was sent out in March 2005. A postcard reminder was sent out approximately 2 weeks after the initial survey was mailed. A second survey was sent to nonresponders 6 weeks after the initial survey. Simultaneously, the survey was also sent in Web‐based format to nonresponders via email.

Data Analysis

Physician characteristics, including practice type, faculty appointment type, and year of medical school graduation, were provided by the hospital's medical staff office. Physician respondents and nonrespondents were compared using the chi‐square test and logistic regression to determine potential response biases. We calculated means and standard deviations and percentages for categorical variables. Logistic regression was used to examine the likelihood of participants reporting any preventable adverse event related to suboptimal transfer of information. The regression model tested the likelihood of one or more preventable adverse events reported with the frequency of seeing patients for follow‐up prior to the arrival of discharge summaries, controlling for participant characteristics and the number of hospitalized patients each physician had in the previous 6 months.

RESULTS

Physician Characteristics

The survey was sent to 425 physicians, 9 of whom were excluded because they had had no patients admitted within the past 6 months. Of the 416 eligible respondents, 2 returned a survey that was incomplete and not usable, and 226 returned a completed survey (response rate of 54%). The characteristics of responders and nonresponders are shown in Table 1. General medicine physicians completed the survey more often than specialist physicians (56% vs. 44%, P < .001). Affiliated faculty were also more likely to complete the survey than full‐time faculty; multivariate logistic regression revealed this was purely a function of the larger number of specialists among the full‐time faculty.

Participant Characteristics
Responders (N = 226) Nonresponders (N = 188) P value
  • Excludes 35 participants with missing data about the number of their hospitalized patients

Practice type
Generalist, N (%) 127 (56.2) 65 (34.6) < .001
Specialist, N (%) 99 (43.8) 123 (65.4)
Faculty appointment
Full‐time, N (%) 104 (46.0 106 (56.4) .04
Affiliated, N (%) 122 (54.0) 82 (43.6)
Year of medical school graduation
Before 1990, N (%) 131 (58.0) 127 (67.6) .04
1990 or later, N (%) 95 (42.0) 61 (32.4)
Number of patients hospitalized in last 6 monthsa
1‐4, N (%) 15 (7.9)
5‐10, N (%) 62 (32.5)
11‐19, N (%) 35 (18.3)
20 or more, N (%) 79 (41.4)

Timeliness and Content

Only 19% of the participants were satisfied or very satisfied with the timeliness of discharge summaries. Among all participants, 33% indicated that 60% or more of their patients were seen for their follow‐up outpatient visit prior to the arrival of the discharge summary, and 22% indicated that for 60% or more of their patients they never received a discharge summary at all.

Only 32% of the participants were satisfied or very satisfied with the quality of discharge summaries. Among all participants, 17% believed that 60% or more of discharge summaries missed critical information. Unnecessary information in the discharge summary was less problematic; only 9% of participants indicated that 60% or more of discharge summaries contained unnecessary information.

Preventable Adverse Events

Overall, 41% of participants believed that in the previous 6 months at least one of their patients had sustained a preventable adverse event related to poor transfer of information at hospital discharge. Reporting one or more preventable adverse events was positively associated with physicians' reports of how often they saw patients for a first postdischarge follow‐up without having a discharge summary available. After adjusting for participant characteristics and the number of patients hospitalized by each physician, logistic regression results indicated that each 20% increase in the frequency of discharge summaries not arriving prior to patient follow‐up appointments was associated with a 28% increase in the odds of a reported preventable adverse event (adjusted OR = 1.28, P = .04).

Preferences for Content and Delivery

The mean rating for importance of discharge summary elements is shown in Table 2. No discharge summary element had a mean rating of less than 5. Participants preferred discharge summaries be delivered via the following methods: facsimile, 48%; mail, 30%; electronic medical record, 41%; and E‐mail, 30%.

Preferred Content of Discharge Summary Ranked by Importance
Mean rating (scale of 1‐10)
Medications at discharge 9.69
Follow‐up issues 9.09
Discharge diagnosis 9.02
List of procedures performed 8.79
Pathology reports 8.78
Pending test results 8.68
Procedure reports 8.16
Stress test reports 8.07
Dates of admission and discharge 8.01
Problem list 7.99
List of radiology tests performed 7.84
Echocardiogram reports 7.79
Follow‐up appointments 7.79
Radiology reports 7.76
Names of consulting attendings 7.64
Name of inpatient attending 7.28
Labs from last hospital day 7.08
Medications at admission 6.91
Allergies 6.56
All lab results 6.22
Code status 6.09
Names of inpatient house officers 5.64

DISCUSSION

Our study found that outpatient physicians were not satisfied with the timeliness or the quality of current discharge summaries. Our findings are in agreement with previous studies demonstrating that discharge summaries were often not available to outpatient physicians3,4 and were often of poor quality.5, 6

Preventable or ameliorable adverse events have been reported to occur in 12% of patients in the period immediately following hospital discharge.1, 2 No studies have evaluated the relationship between discharge summaries and preventable adverse events following discharge. Our study found that 41% of outpatient physicians believed that at least one of their patients in the 6 months prior to the survey had sustained a preventable adverse event related to the suboptimal transfer of information at hospital discharge. In addition, the likelihood of physicians reporting one or more preventable adverse events increased with the frequency of seeing patients for follow‐up prior to discharge summary arrival.

In preparation for revising the discharge summary, we asked outpatient physicians to rate the importance of discharge summary content and their preference for method of delivery of discharge summaries. As in previous studies, the outpatient physicians rated discharge medications, discharge diagnosis, test results, and follow‐up plans as highly important.7, 8 Much of this clinical data is now available in the electronic medical record. Therefore, it is possible to electronically assemble much, if not all, of discharge summary content. One recent study demonstrated that database‐generated discharge summaries significantly increased the likelihood that a discharge summary was generated within 4 weeks of hospital discharge.9 The database used in that study required manual data input from a handwritten form. To our knowledge, no study has reported the experience of discharge summaries generated from an electronic medical record.

Our study had several limitations. First, our study used physician survey to assess the timeliness of receiving discharge summaries. Measuring the time to actual receipt of discharge summaries by physicians was beyond the scope of our study. Second, our study did not measure adverse events directly. Instead, we asked outpatient physicians to estimate how many of their patients discharged in the last 6 months had sustained a preventable adverse event related to suboptimal information transfer at discharge. We had limited space in the questionnaire to define the meaning of a preventable adverse event; therefore, the description in the survey does not exactly match previous definitions.1, 2 Our study had a response rate of 54%. It is possible that nonresponders may have been more satisfied with the quality and timeliness of discharge summaries and may have believed fewer patients experienced preventable adverse events related to suboptimal information transfer at discharge.

The results of our study suggest that the use of systems to improve the quality and delivery of discharge summaries has the potential to improve outpatient physician satisfaction and to reduce the number of preventable adverse events that occur during the vulnerable period following hospital discharge. With the use of electronic medical records, we now have the potential to automate the process of assembling and delivering clinical information with minimal delay. We are now using the information from this study to design a partially automated, high‐quality discharge summary that can be delivered to outpatient physicians immediately on discharge.

References
  1. Forster AJ,Clark HD,Menard A, et al.Adverse events among medical patient after hospital discharge.CMAJ.2004;170:345349.
  2. Forster AJ,Harvey JF,Peterson JF,Gandhi TK,Bates DW.The incidence and severity of adverse events affecting patients after discharge from the hospital.Ann Intern Med.2003;138:161167.
  3. van Walraven C,Seth R,Laupacis A.Dissemination of discharge summaries. Not reaching follow‐up physicians.Can Fam Physician.2002;48:737742.
  4. van Walraven C,Seth R,Austin PC,Laupacis A.Effect of discharge summary availability during post‐discharge visits on hospital readmission.J Gen Intern Med.2002;17:186192.
  5. Wilson S,Warwick R,Chapman M,Miller R.General practitioner‐hospital communications: a review of discharge summaries.J Qual Clin Pract.2001;21:104108.
  6. Bertrand D,Rancois P,Bosson JL,Fauconnier J,Weil G.Quality assessment of discharge letters in a French university hospital.Int J Health Care Qual Assur.1998;11:9095.
  7. Solomon JK,Maxwell RB,Hopkins AP.Content of a discharge summary from a medical ward: views of general practitioners and hospital doctors.J R Coll Physicians Lond.1995;29:307310.
  8. van Walraven C,Rokosh E.What is necessary for high‐quality discharge summaries?Am J Med Qual.1999;14:160169.
  9. van Walraven C,Laupacis A,Seth R,Wells G.Dictated versus database‐generated discharge summaries: a randomized clinical trial.CMAJ.1999;160:319326.
Article PDF
Issue
Journal of Hospital Medicine - 1(5)
Publications
Page Number
317-320
Legacy Keywords
patient discharge, medical errors, medical record, medical informatics
Sections
Article PDF
Article PDF

Twelve percent of patients have been reported to have preventable or ameliorable adverse events in the period immediately following hospital discharge.1, 2 A potential contributor to the number of adverse events is inadequate transfer of clinical information at hospital discharge. The discharge summary is a vital component of the transfer of information from the inpatient to the outpatient setting. Unfortunately, discharge summaries are often unavailable when follow‐up care occurs and often lack important content.36

Many hospitals are implementing an electronic medical record systems. This creates the opportunity at hospital discharge to immediately assemble the major components of a discharge summary. With enhanced communication systems, this information can be delivered in a variety of ways with minimal delay. We report the results and evaluation of a survey of medicine faculty at an urban academic medical center about the timeliness and quality of discharge summaries, the perceived incidence of adverse events related to suboptimal information transfer at discharge, and the need for the electronically generated discharge summary we plan to design.

METHODS

Study Site

The study was conducted at a 753‐bed academic hospital in Chicago, Illinois. Discharge summaries have traditionally been dictated by inpatient physicians and delivered to outpatient physicians by both mail and facsimile via the medical records department. The hospital has used an electronic medical record and computerized physician order entry system (PowerChart Millennium from Cerner Corporation) since August 2004. Although all history and physicals and progress notes were documented in the electronic medical record, the system did not provide a method for delivering the discharge summaries contained in the electronic medical record to outpatient physician offices. Because of this, inpatient physicians continued to dictate discharge summaries during this study.

Participants

An advisory board consisting of 16 physicians from the Department of Medicine was convened. The advisory board gave input on needs assessment and helped to create a survey to be administered to all 425 medicine faculty who have an outpatient practice. All respondents who had at least 1 patient admitted to the hospital within the 6 months prior to the survey were eligible.

Survey Content

Our survey consisted of 2 parts. In the first part, we asked respondents to estimate how many of their patients had been discharged from the hospital in the past 6 months and to reflect on these patients as they completed the survey. Satisfaction with the timeliness and quality of discharge summaries was assessed using a 5‐point Likert scale, from 5, very satisfied, to 1, very unsatisfied. The frequency of hospital follow‐up of a patient occurring prior to arrival of the discharge summary was assessed as the percentage of times this occurred in 20% increments (0%‐19%, 20%‐39%, 40%‐59%, 60%‐79%, and 80%‐100%). The number of discharge summaries missing critical information and the number of summaries containing unnecessary information were similarly assessed using 20% increments. We then asked respondents to estimate the number of patients who had sustained a preventable adverse event related to suboptimal transfer of information at discharge. We defined a preventable adverse event as a preventable medical problem or worsening of an existing problem.

In the second part of the survey, we elicited preferences for discharge summary content and method of delivery. We assessed preferences for discharge summary content by asking respondents to rank items on a scale from 1 to 10, from 10, most important, to 1, least important. Preferences for delivery of discharge summaries were assessed by asking respondents to indicate one or more delivery methods, including facsimile, mail, the electronic medical record, and E‐mail.

Survey Process

The survey was sent out in March 2005. A postcard reminder was sent out approximately 2 weeks after the initial survey was mailed. A second survey was sent to nonresponders 6 weeks after the initial survey. Simultaneously, the survey was also sent in Web‐based format to nonresponders via email.

Data Analysis

Physician characteristics, including practice type, faculty appointment type, and year of medical school graduation, were provided by the hospital's medical staff office. Physician respondents and nonrespondents were compared using the chi‐square test and logistic regression to determine potential response biases. We calculated means and standard deviations and percentages for categorical variables. Logistic regression was used to examine the likelihood of participants reporting any preventable adverse event related to suboptimal transfer of information. The regression model tested the likelihood of one or more preventable adverse events reported with the frequency of seeing patients for follow‐up prior to the arrival of discharge summaries, controlling for participant characteristics and the number of hospitalized patients each physician had in the previous 6 months.

RESULTS

Physician Characteristics

The survey was sent to 425 physicians, 9 of whom were excluded because they had had no patients admitted within the past 6 months. Of the 416 eligible respondents, 2 returned a survey that was incomplete and not usable, and 226 returned a completed survey (response rate of 54%). The characteristics of responders and nonresponders are shown in Table 1. General medicine physicians completed the survey more often than specialist physicians (56% vs. 44%, P < .001). Affiliated faculty were also more likely to complete the survey than full‐time faculty; multivariate logistic regression revealed this was purely a function of the larger number of specialists among the full‐time faculty.

Participant Characteristics
Responders (N = 226) Nonresponders (N = 188) P value
  • Excludes 35 participants with missing data about the number of their hospitalized patients

Practice type
Generalist, N (%) 127 (56.2) 65 (34.6) < .001
Specialist, N (%) 99 (43.8) 123 (65.4)
Faculty appointment
Full‐time, N (%) 104 (46.0 106 (56.4) .04
Affiliated, N (%) 122 (54.0) 82 (43.6)
Year of medical school graduation
Before 1990, N (%) 131 (58.0) 127 (67.6) .04
1990 or later, N (%) 95 (42.0) 61 (32.4)
Number of patients hospitalized in last 6 monthsa
1‐4, N (%) 15 (7.9)
5‐10, N (%) 62 (32.5)
11‐19, N (%) 35 (18.3)
20 or more, N (%) 79 (41.4)

Timeliness and Content

Only 19% of the participants were satisfied or very satisfied with the timeliness of discharge summaries. Among all participants, 33% indicated that 60% or more of their patients were seen for their follow‐up outpatient visit prior to the arrival of the discharge summary, and 22% indicated that for 60% or more of their patients they never received a discharge summary at all.

Only 32% of the participants were satisfied or very satisfied with the quality of discharge summaries. Among all participants, 17% believed that 60% or more of discharge summaries missed critical information. Unnecessary information in the discharge summary was less problematic; only 9% of participants indicated that 60% or more of discharge summaries contained unnecessary information.

Preventable Adverse Events

Overall, 41% of participants believed that in the previous 6 months at least one of their patients had sustained a preventable adverse event related to poor transfer of information at hospital discharge. Reporting one or more preventable adverse events was positively associated with physicians' reports of how often they saw patients for a first postdischarge follow‐up without having a discharge summary available. After adjusting for participant characteristics and the number of patients hospitalized by each physician, logistic regression results indicated that each 20% increase in the frequency of discharge summaries not arriving prior to patient follow‐up appointments was associated with a 28% increase in the odds of a reported preventable adverse event (adjusted OR = 1.28, P = .04).

Preferences for Content and Delivery

The mean rating for importance of discharge summary elements is shown in Table 2. No discharge summary element had a mean rating of less than 5. Participants preferred discharge summaries be delivered via the following methods: facsimile, 48%; mail, 30%; electronic medical record, 41%; and E‐mail, 30%.

Preferred Content of Discharge Summary Ranked by Importance
Mean rating (scale of 1‐10)
Medications at discharge 9.69
Follow‐up issues 9.09
Discharge diagnosis 9.02
List of procedures performed 8.79
Pathology reports 8.78
Pending test results 8.68
Procedure reports 8.16
Stress test reports 8.07
Dates of admission and discharge 8.01
Problem list 7.99
List of radiology tests performed 7.84
Echocardiogram reports 7.79
Follow‐up appointments 7.79
Radiology reports 7.76
Names of consulting attendings 7.64
Name of inpatient attending 7.28
Labs from last hospital day 7.08
Medications at admission 6.91
Allergies 6.56
All lab results 6.22
Code status 6.09
Names of inpatient house officers 5.64

DISCUSSION

Our study found that outpatient physicians were not satisfied with the timeliness or the quality of current discharge summaries. Our findings are in agreement with previous studies demonstrating that discharge summaries were often not available to outpatient physicians3,4 and were often of poor quality.5, 6

Preventable or ameliorable adverse events have been reported to occur in 12% of patients in the period immediately following hospital discharge.1, 2 No studies have evaluated the relationship between discharge summaries and preventable adverse events following discharge. Our study found that 41% of outpatient physicians believed that at least one of their patients in the 6 months prior to the survey had sustained a preventable adverse event related to the suboptimal transfer of information at hospital discharge. In addition, the likelihood of physicians reporting one or more preventable adverse events increased with the frequency of seeing patients for follow‐up prior to discharge summary arrival.

In preparation for revising the discharge summary, we asked outpatient physicians to rate the importance of discharge summary content and their preference for method of delivery of discharge summaries. As in previous studies, the outpatient physicians rated discharge medications, discharge diagnosis, test results, and follow‐up plans as highly important.7, 8 Much of this clinical data is now available in the electronic medical record. Therefore, it is possible to electronically assemble much, if not all, of discharge summary content. One recent study demonstrated that database‐generated discharge summaries significantly increased the likelihood that a discharge summary was generated within 4 weeks of hospital discharge.9 The database used in that study required manual data input from a handwritten form. To our knowledge, no study has reported the experience of discharge summaries generated from an electronic medical record.

Our study had several limitations. First, our study used physician survey to assess the timeliness of receiving discharge summaries. Measuring the time to actual receipt of discharge summaries by physicians was beyond the scope of our study. Second, our study did not measure adverse events directly. Instead, we asked outpatient physicians to estimate how many of their patients discharged in the last 6 months had sustained a preventable adverse event related to suboptimal information transfer at discharge. We had limited space in the questionnaire to define the meaning of a preventable adverse event; therefore, the description in the survey does not exactly match previous definitions.1, 2 Our study had a response rate of 54%. It is possible that nonresponders may have been more satisfied with the quality and timeliness of discharge summaries and may have believed fewer patients experienced preventable adverse events related to suboptimal information transfer at discharge.

The results of our study suggest that the use of systems to improve the quality and delivery of discharge summaries has the potential to improve outpatient physician satisfaction and to reduce the number of preventable adverse events that occur during the vulnerable period following hospital discharge. With the use of electronic medical records, we now have the potential to automate the process of assembling and delivering clinical information with minimal delay. We are now using the information from this study to design a partially automated, high‐quality discharge summary that can be delivered to outpatient physicians immediately on discharge.

Twelve percent of patients have been reported to have preventable or ameliorable adverse events in the period immediately following hospital discharge.1, 2 A potential contributor to the number of adverse events is inadequate transfer of clinical information at hospital discharge. The discharge summary is a vital component of the transfer of information from the inpatient to the outpatient setting. Unfortunately, discharge summaries are often unavailable when follow‐up care occurs and often lack important content.36

Many hospitals are implementing an electronic medical record systems. This creates the opportunity at hospital discharge to immediately assemble the major components of a discharge summary. With enhanced communication systems, this information can be delivered in a variety of ways with minimal delay. We report the results and evaluation of a survey of medicine faculty at an urban academic medical center about the timeliness and quality of discharge summaries, the perceived incidence of adverse events related to suboptimal information transfer at discharge, and the need for the electronically generated discharge summary we plan to design.

METHODS

Study Site

The study was conducted at a 753‐bed academic hospital in Chicago, Illinois. Discharge summaries have traditionally been dictated by inpatient physicians and delivered to outpatient physicians by both mail and facsimile via the medical records department. The hospital has used an electronic medical record and computerized physician order entry system (PowerChart Millennium from Cerner Corporation) since August 2004. Although all history and physicals and progress notes were documented in the electronic medical record, the system did not provide a method for delivering the discharge summaries contained in the electronic medical record to outpatient physician offices. Because of this, inpatient physicians continued to dictate discharge summaries during this study.

Participants

An advisory board consisting of 16 physicians from the Department of Medicine was convened. The advisory board gave input on needs assessment and helped to create a survey to be administered to all 425 medicine faculty who have an outpatient practice. All respondents who had at least 1 patient admitted to the hospital within the 6 months prior to the survey were eligible.

Survey Content

Our survey consisted of 2 parts. In the first part, we asked respondents to estimate how many of their patients had been discharged from the hospital in the past 6 months and to reflect on these patients as they completed the survey. Satisfaction with the timeliness and quality of discharge summaries was assessed using a 5‐point Likert scale, from 5, very satisfied, to 1, very unsatisfied. The frequency of hospital follow‐up of a patient occurring prior to arrival of the discharge summary was assessed as the percentage of times this occurred in 20% increments (0%‐19%, 20%‐39%, 40%‐59%, 60%‐79%, and 80%‐100%). The number of discharge summaries missing critical information and the number of summaries containing unnecessary information were similarly assessed using 20% increments. We then asked respondents to estimate the number of patients who had sustained a preventable adverse event related to suboptimal transfer of information at discharge. We defined a preventable adverse event as a preventable medical problem or worsening of an existing problem.

In the second part of the survey, we elicited preferences for discharge summary content and method of delivery. We assessed preferences for discharge summary content by asking respondents to rank items on a scale from 1 to 10, from 10, most important, to 1, least important. Preferences for delivery of discharge summaries were assessed by asking respondents to indicate one or more delivery methods, including facsimile, mail, the electronic medical record, and E‐mail.

Survey Process

The survey was sent out in March 2005. A postcard reminder was sent out approximately 2 weeks after the initial survey was mailed. A second survey was sent to nonresponders 6 weeks after the initial survey. Simultaneously, the survey was also sent in Web‐based format to nonresponders via email.

Data Analysis

Physician characteristics, including practice type, faculty appointment type, and year of medical school graduation, were provided by the hospital's medical staff office. Physician respondents and nonrespondents were compared using the chi‐square test and logistic regression to determine potential response biases. We calculated means and standard deviations and percentages for categorical variables. Logistic regression was used to examine the likelihood of participants reporting any preventable adverse event related to suboptimal transfer of information. The regression model tested the likelihood of one or more preventable adverse events reported with the frequency of seeing patients for follow‐up prior to the arrival of discharge summaries, controlling for participant characteristics and the number of hospitalized patients each physician had in the previous 6 months.

RESULTS

Physician Characteristics

The survey was sent to 425 physicians, 9 of whom were excluded because they had had no patients admitted within the past 6 months. Of the 416 eligible respondents, 2 returned a survey that was incomplete and not usable, and 226 returned a completed survey (response rate of 54%). The characteristics of responders and nonresponders are shown in Table 1. General medicine physicians completed the survey more often than specialist physicians (56% vs. 44%, P < .001). Affiliated faculty were also more likely to complete the survey than full‐time faculty; multivariate logistic regression revealed this was purely a function of the larger number of specialists among the full‐time faculty.

Participant Characteristics
Responders (N = 226) Nonresponders (N = 188) P value
  • Excludes 35 participants with missing data about the number of their hospitalized patients

Practice type
Generalist, N (%) 127 (56.2) 65 (34.6) < .001
Specialist, N (%) 99 (43.8) 123 (65.4)
Faculty appointment
Full‐time, N (%) 104 (46.0 106 (56.4) .04
Affiliated, N (%) 122 (54.0) 82 (43.6)
Year of medical school graduation
Before 1990, N (%) 131 (58.0) 127 (67.6) .04
1990 or later, N (%) 95 (42.0) 61 (32.4)
Number of patients hospitalized in last 6 monthsa
1‐4, N (%) 15 (7.9)
5‐10, N (%) 62 (32.5)
11‐19, N (%) 35 (18.3)
20 or more, N (%) 79 (41.4)

Timeliness and Content

Only 19% of the participants were satisfied or very satisfied with the timeliness of discharge summaries. Among all participants, 33% indicated that 60% or more of their patients were seen for their follow‐up outpatient visit prior to the arrival of the discharge summary, and 22% indicated that for 60% or more of their patients they never received a discharge summary at all.

Only 32% of the participants were satisfied or very satisfied with the quality of discharge summaries. Among all participants, 17% believed that 60% or more of discharge summaries missed critical information. Unnecessary information in the discharge summary was less problematic; only 9% of participants indicated that 60% or more of discharge summaries contained unnecessary information.

Preventable Adverse Events

Overall, 41% of participants believed that in the previous 6 months at least one of their patients had sustained a preventable adverse event related to poor transfer of information at hospital discharge. Reporting one or more preventable adverse events was positively associated with physicians' reports of how often they saw patients for a first postdischarge follow‐up without having a discharge summary available. After adjusting for participant characteristics and the number of patients hospitalized by each physician, logistic regression results indicated that each 20% increase in the frequency of discharge summaries not arriving prior to patient follow‐up appointments was associated with a 28% increase in the odds of a reported preventable adverse event (adjusted OR = 1.28, P = .04).

Preferences for Content and Delivery

The mean rating for importance of discharge summary elements is shown in Table 2. No discharge summary element had a mean rating of less than 5. Participants preferred discharge summaries be delivered via the following methods: facsimile, 48%; mail, 30%; electronic medical record, 41%; and E‐mail, 30%.

Preferred Content of Discharge Summary Ranked by Importance
Mean rating (scale of 1‐10)
Medications at discharge 9.69
Follow‐up issues 9.09
Discharge diagnosis 9.02
List of procedures performed 8.79
Pathology reports 8.78
Pending test results 8.68
Procedure reports 8.16
Stress test reports 8.07
Dates of admission and discharge 8.01
Problem list 7.99
List of radiology tests performed 7.84
Echocardiogram reports 7.79
Follow‐up appointments 7.79
Radiology reports 7.76
Names of consulting attendings 7.64
Name of inpatient attending 7.28
Labs from last hospital day 7.08
Medications at admission 6.91
Allergies 6.56
All lab results 6.22
Code status 6.09
Names of inpatient house officers 5.64

DISCUSSION

Our study found that outpatient physicians were not satisfied with the timeliness or the quality of current discharge summaries. Our findings are in agreement with previous studies demonstrating that discharge summaries were often not available to outpatient physicians3,4 and were often of poor quality.5, 6

Preventable or ameliorable adverse events have been reported to occur in 12% of patients in the period immediately following hospital discharge.1, 2 No studies have evaluated the relationship between discharge summaries and preventable adverse events following discharge. Our study found that 41% of outpatient physicians believed that at least one of their patients in the 6 months prior to the survey had sustained a preventable adverse event related to the suboptimal transfer of information at hospital discharge. In addition, the likelihood of physicians reporting one or more preventable adverse events increased with the frequency of seeing patients for follow‐up prior to discharge summary arrival.

In preparation for revising the discharge summary, we asked outpatient physicians to rate the importance of discharge summary content and their preference for method of delivery of discharge summaries. As in previous studies, the outpatient physicians rated discharge medications, discharge diagnosis, test results, and follow‐up plans as highly important.7, 8 Much of this clinical data is now available in the electronic medical record. Therefore, it is possible to electronically assemble much, if not all, of discharge summary content. One recent study demonstrated that database‐generated discharge summaries significantly increased the likelihood that a discharge summary was generated within 4 weeks of hospital discharge.9 The database used in that study required manual data input from a handwritten form. To our knowledge, no study has reported the experience of discharge summaries generated from an electronic medical record.

Our study had several limitations. First, our study used physician survey to assess the timeliness of receiving discharge summaries. Measuring the time to actual receipt of discharge summaries by physicians was beyond the scope of our study. Second, our study did not measure adverse events directly. Instead, we asked outpatient physicians to estimate how many of their patients discharged in the last 6 months had sustained a preventable adverse event related to suboptimal information transfer at discharge. We had limited space in the questionnaire to define the meaning of a preventable adverse event; therefore, the description in the survey does not exactly match previous definitions.1, 2 Our study had a response rate of 54%. It is possible that nonresponders may have been more satisfied with the quality and timeliness of discharge summaries and may have believed fewer patients experienced preventable adverse events related to suboptimal information transfer at discharge.

The results of our study suggest that the use of systems to improve the quality and delivery of discharge summaries has the potential to improve outpatient physician satisfaction and to reduce the number of preventable adverse events that occur during the vulnerable period following hospital discharge. With the use of electronic medical records, we now have the potential to automate the process of assembling and delivering clinical information with minimal delay. We are now using the information from this study to design a partially automated, high‐quality discharge summary that can be delivered to outpatient physicians immediately on discharge.

References
  1. Forster AJ,Clark HD,Menard A, et al.Adverse events among medical patient after hospital discharge.CMAJ.2004;170:345349.
  2. Forster AJ,Harvey JF,Peterson JF,Gandhi TK,Bates DW.The incidence and severity of adverse events affecting patients after discharge from the hospital.Ann Intern Med.2003;138:161167.
  3. van Walraven C,Seth R,Laupacis A.Dissemination of discharge summaries. Not reaching follow‐up physicians.Can Fam Physician.2002;48:737742.
  4. van Walraven C,Seth R,Austin PC,Laupacis A.Effect of discharge summary availability during post‐discharge visits on hospital readmission.J Gen Intern Med.2002;17:186192.
  5. Wilson S,Warwick R,Chapman M,Miller R.General practitioner‐hospital communications: a review of discharge summaries.J Qual Clin Pract.2001;21:104108.
  6. Bertrand D,Rancois P,Bosson JL,Fauconnier J,Weil G.Quality assessment of discharge letters in a French university hospital.Int J Health Care Qual Assur.1998;11:9095.
  7. Solomon JK,Maxwell RB,Hopkins AP.Content of a discharge summary from a medical ward: views of general practitioners and hospital doctors.J R Coll Physicians Lond.1995;29:307310.
  8. van Walraven C,Rokosh E.What is necessary for high‐quality discharge summaries?Am J Med Qual.1999;14:160169.
  9. van Walraven C,Laupacis A,Seth R,Wells G.Dictated versus database‐generated discharge summaries: a randomized clinical trial.CMAJ.1999;160:319326.
References
  1. Forster AJ,Clark HD,Menard A, et al.Adverse events among medical patient after hospital discharge.CMAJ.2004;170:345349.
  2. Forster AJ,Harvey JF,Peterson JF,Gandhi TK,Bates DW.The incidence and severity of adverse events affecting patients after discharge from the hospital.Ann Intern Med.2003;138:161167.
  3. van Walraven C,Seth R,Laupacis A.Dissemination of discharge summaries. Not reaching follow‐up physicians.Can Fam Physician.2002;48:737742.
  4. van Walraven C,Seth R,Austin PC,Laupacis A.Effect of discharge summary availability during post‐discharge visits on hospital readmission.J Gen Intern Med.2002;17:186192.
  5. Wilson S,Warwick R,Chapman M,Miller R.General practitioner‐hospital communications: a review of discharge summaries.J Qual Clin Pract.2001;21:104108.
  6. Bertrand D,Rancois P,Bosson JL,Fauconnier J,Weil G.Quality assessment of discharge letters in a French university hospital.Int J Health Care Qual Assur.1998;11:9095.
  7. Solomon JK,Maxwell RB,Hopkins AP.Content of a discharge summary from a medical ward: views of general practitioners and hospital doctors.J R Coll Physicians Lond.1995;29:307310.
  8. van Walraven C,Rokosh E.What is necessary for high‐quality discharge summaries?Am J Med Qual.1999;14:160169.
  9. van Walraven C,Laupacis A,Seth R,Wells G.Dictated versus database‐generated discharge summaries: a randomized clinical trial.CMAJ.1999;160:319326.
Issue
Journal of Hospital Medicine - 1(5)
Issue
Journal of Hospital Medicine - 1(5)
Page Number
317-320
Page Number
317-320
Publications
Publications
Article Type
Display Headline
Outpatient physicians' satisfaction with discharge summaries and perceived need for an electronic discharge summary
Display Headline
Outpatient physicians' satisfaction with discharge summaries and perceived need for an electronic discharge summary
Legacy Keywords
patient discharge, medical errors, medical record, medical informatics
Legacy Keywords
patient discharge, medical errors, medical record, medical informatics
Sections
Article Source
Copyright © 2006 Society of Hospital Medicine
Disallow All Ads
Correspondence Location
Associate Division Chief for Hospital Medicine, Division of General Internal Medicine, 675 North St. Clair, Suite 18‐200, Chicago, IL 60611; Fax: (312) 695‐2857
Content Gating
Gated (full article locked unless allowed per User)
Gating Strategy
First Peek Free
Article PDF Media