Brief Reports

Today more than one‐third of emergency departments (EDs) in the United States have affiliated observation units, where patients can stay 24 to 48 hours without being admitted to the hospital.[1] Observation units experienced significant growth in the United States from 2005 to 2007, secondary to policy changes involving the Centers for Medicare and Medicaid Services (CMS), which expanded reimbursement for observation services to include any clinical condition. Furthermore, CMS implemented the Recovery Audit Contractor process, which was able to fine providers and facilities for inappropriate claims, with the principle method for charge recovery being inappropriate charges for short inpatient stays.

ED observation units (EDOUs) vary in the number of beds, but are often located adjacent to the emergency department.[2] It is estimated that EDOUs have the capacity for caring for 5% to 10% of any given ED volume.[2] Almost half of EDOUs are protocol driven, allowing these units to discharge up to 80% of all patients within 24 hours.[1, 2] Some studies have suggested that EDOUs are associated with a decrease in overall hospitalization rates, leading to cost savings.[1] However, these studies were limited by their single‐center design or simulated in nature. In addition, other studies show that EDOUs decrease inpatient admissions, length of stay, and costs related to specific clinical conditions such as chest pain, transient ischemic attack, and syncope.[3]

To further evaluate the association of observation units on ED hospital admission rates nationally, we analyzed the largest ED‐based survey, the 2010 National Hospital Ambulatory Medical Care Survey (NHAMCS), to assess the impact of observation units on hospital admissions from the ED. We hypothesized that observation units decrease overall hospital admissions from the ED.

METHODS

RESULTS

There were 24,232 ED visits from 315 hospitals in the United States in our study. Of these, 82 (20.6%) hospitals had an observation unit physically separate from the ED. Hospitals with and without observation units did not have different hospital patient level characteristics. There was no association between hospital ownership, teaching status, region location, urban or rural location, and hospitals with observation units when compared with hospitals without observation units (Table 1).

In addition, there was no association between patient characteristics at the ED visit level in hospitals with observation units when compared with patient characteristics at the ED visit level in hospitals without observation units (Table 2). The average ED risk‐standardized hospital admission rate for hospitals with observation units was 13.7% (95% confidence interval [CI]: 11.3 to 16.0) compared to 16.0% (95% CI: 14.1 to 17.7) for hospitals without observation units (Figure 1). This difference of 2.3% (95% CI: 0.1 to 4.7) was not statistically significant.

Figure 1

DISCUSSION

In this national study of hospital admissions from the ED, we did not find that hospitals with observation units had a statistically significant lower ED risk‐standardized admission rate when compared with hospitals that did not have observation units. However, the difference of ED risk‐standardized hospital admission rates between hospitals with observation units and those without observation units was very small, and we were likely underpowered to detect a statistically significant difference.

Recently, EDOUs have received much attention, in part because of increases in their numbers and frequency of use.[7] Prior studies, which did not report admission rates that were risk standardized, have also demonstrated no difference in the admission rates among hospitals with and without observation units.[6, 8] Although this result seems counterintuitive, several possible explanations exist.

One reason that there may not be a relation between the rate of inpatient admission and the presence of an observation unit is that the introduction of an EDOU appears to change physician behavior. When the option to admit to an observation unit is present, ED physicians are 2 times more likely to disposition patients to observation status without a statistically significant change in the rate of inpatient admission.[6] Studies have demonstrated that after the introduction of an observation unit, ED physicians tend to overutilize observation among patients who previously would have been discharged, while continuing to admit patients as inpatients who meet observation criteria, which could result in an increase in cost for payers and patients.[7, 9]

Observation units that are protocol driven have been associated with the best patient outcomes including shorter length of stay, lower likelihood of subsequent inpatient admission, and decreased cost.[10] Furthermore, studies evaluating EDOUs suggest increased patient satisfaction and improved patient safety, especially for protocol‐driven EDOUs.[2] However, currently, only half of dedicated observation units are protocol driven. It is also possible that the ED inpatient admission rate does not capture the full impact of an observation unit on care delivery and quality. Observation units are more likely to be present in EDs with a higher overall patient census, longer patient lengths of stay, and higher rates of ambulance diversion.[6, 8] Unfortunately, NHAMCS does not distinguish protocol‐driven versus nonprotocol‐driven observation units. From a policy standpoint, as EDOUs continue to emerge, there is an opportunity to standardize how EDOUs function by using best practices.

This study should be evaluated in the context of limitations such as heterogeneity in the management of EDOUs, limited hospital factor variables that may influence hospital admissions, and small sample size associated with each hospital. Because we were not able to determine which EDs used protocol‐driven observation units, we were not able to determine the impact of having a protocol‐driven observation unit on inpatient hospital admission rates. Additionally, the study may suffer from a selection bias, as EDs with observation units have been shown to have higher patient volume, longer patient lengths of stay, and greater rates of ED diversion. Despite the small sample size, our risk‐standardized model accounted for case mix and hospital factors associated with hospital admission rates and had a high C statistic value, which indicates that the predicted probability of being admitted from the ED highly correlates with the actual outcome of being admitted from the ED. We were unable to track hospitals longitudinally to determine if a hospital's high volume is associated with the creation of EDOUs as a means to offset its demand. However, in our analysis, we did control for overall patient volume when calculating the RHSAR. Finally, we were not able to limit the dataset to observation unit admission conditions because of the limited number of visits provided per hospital by NHAMCS. We conducted an analysis using 80% power and a P value of 0.05 to determine the sample size needed to have statistically significant results. We would require 920 hospitals to have statistically significant results, which suggests we were underpowered to detect a statistically significant difference.

In this preliminary study, we did not find an association between the presence of EDOUs and ED hospital admissions. Our study was limited by an inability to analyze administrative differences and to adjust for certain hospital factors that are likely to influence inpatient admissions via the ED. Nonetheless, our findings suggest that EDOUs merit further evaluation of their potential cost savings and the quality of the care they provide. An evaluation of ED observation departmental management is also needed to assess differences in care at observation units managed by emergency physicians versus nonemergency physicians.

Today more than one‐third of emergency departments (EDs) in the United States have affiliated observation units, where patients can stay 24 to 48 hours without being admitted to the hospital.[1] Observation units experienced significant growth in the United States from 2005 to 2007, secondary to policy changes involving the Centers for Medicare and Medicaid Services (CMS), which expanded reimbursement for observation services to include any clinical condition. Furthermore, CMS implemented the Recovery Audit Contractor process, which was able to fine providers and facilities for inappropriate claims, with the principle method for charge recovery being inappropriate charges for short inpatient stays.

ED observation units (EDOUs) vary in the number of beds, but are often located adjacent to the emergency department.[2] It is estimated that EDOUs have the capacity for caring for 5% to 10% of any given ED volume.[2] Almost half of EDOUs are protocol driven, allowing these units to discharge up to 80% of all patients within 24 hours.[1, 2] Some studies have suggested that EDOUs are associated with a decrease in overall hospitalization rates, leading to cost savings.[1] However, these studies were limited by their single‐center design or simulated in nature. In addition, other studies show that EDOUs decrease inpatient admissions, length of stay, and costs related to specific clinical conditions such as chest pain, transient ischemic attack, and syncope.[3]

To further evaluate the association of observation units on ED hospital admission rates nationally, we analyzed the largest ED‐based survey, the 2010 National Hospital Ambulatory Medical Care Survey (NHAMCS), to assess the impact of observation units on hospital admissions from the ED. We hypothesized that observation units decrease overall hospital admissions from the ED.

METHODS

RESULTS

There were 24,232 ED visits from 315 hospitals in the United States in our study. Of these, 82 (20.6%) hospitals had an observation unit physically separate from the ED. Hospitals with and without observation units did not have different hospital patient level characteristics. There was no association between hospital ownership, teaching status, region location, urban or rural location, and hospitals with observation units when compared with hospitals without observation units (Table 1).

In addition, there was no association between patient characteristics at the ED visit level in hospitals with observation units when compared with patient characteristics at the ED visit level in hospitals without observation units (Table 2). The average ED risk‐standardized hospital admission rate for hospitals with observation units was 13.7% (95% confidence interval [CI]: 11.3 to 16.0) compared to 16.0% (95% CI: 14.1 to 17.7) for hospitals without observation units (Figure 1). This difference of 2.3% (95% CI: 0.1 to 4.7) was not statistically significant.

Figure 1

DISCUSSION

In this national study of hospital admissions from the ED, we did not find that hospitals with observation units had a statistically significant lower ED risk‐standardized admission rate when compared with hospitals that did not have observation units. However, the difference of ED risk‐standardized hospital admission rates between hospitals with observation units and those without observation units was very small, and we were likely underpowered to detect a statistically significant difference.

Recently, EDOUs have received much attention, in part because of increases in their numbers and frequency of use.[7] Prior studies, which did not report admission rates that were risk standardized, have also demonstrated no difference in the admission rates among hospitals with and without observation units.[6, 8] Although this result seems counterintuitive, several possible explanations exist.

One reason that there may not be a relation between the rate of inpatient admission and the presence of an observation unit is that the introduction of an EDOU appears to change physician behavior. When the option to admit to an observation unit is present, ED physicians are 2 times more likely to disposition patients to observation status without a statistically significant change in the rate of inpatient admission.[6] Studies have demonstrated that after the introduction of an observation unit, ED physicians tend to overutilize observation among patients who previously would have been discharged, while continuing to admit patients as inpatients who meet observation criteria, which could result in an increase in cost for payers and patients.[7, 9]

Observation units that are protocol driven have been associated with the best patient outcomes including shorter length of stay, lower likelihood of subsequent inpatient admission, and decreased cost.[10] Furthermore, studies evaluating EDOUs suggest increased patient satisfaction and improved patient safety, especially for protocol‐driven EDOUs.[2] However, currently, only half of dedicated observation units are protocol driven. It is also possible that the ED inpatient admission rate does not capture the full impact of an observation unit on care delivery and quality. Observation units are more likely to be present in EDs with a higher overall patient census, longer patient lengths of stay, and higher rates of ambulance diversion.[6, 8] Unfortunately, NHAMCS does not distinguish protocol‐driven versus nonprotocol‐driven observation units. From a policy standpoint, as EDOUs continue to emerge, there is an opportunity to standardize how EDOUs function by using best practices.

This study should be evaluated in the context of limitations such as heterogeneity in the management of EDOUs, limited hospital factor variables that may influence hospital admissions, and small sample size associated with each hospital. Because we were not able to determine which EDs used protocol‐driven observation units, we were not able to determine the impact of having a protocol‐driven observation unit on inpatient hospital admission rates. Additionally, the study may suffer from a selection bias, as EDs with observation units have been shown to have higher patient volume, longer patient lengths of stay, and greater rates of ED diversion. Despite the small sample size, our risk‐standardized model accounted for case mix and hospital factors associated with hospital admission rates and had a high C statistic value, which indicates that the predicted probability of being admitted from the ED highly correlates with the actual outcome of being admitted from the ED. We were unable to track hospitals longitudinally to determine if a hospital's high volume is associated with the creation of EDOUs as a means to offset its demand. However, in our analysis, we did control for overall patient volume when calculating the RHSAR. Finally, we were not able to limit the dataset to observation unit admission conditions because of the limited number of visits provided per hospital by NHAMCS. We conducted an analysis using 80% power and a P value of 0.05 to determine the sample size needed to have statistically significant results. We would require 920 hospitals to have statistically significant results, which suggests we were underpowered to detect a statistically significant difference.

In this preliminary study, we did not find an association between the presence of EDOUs and ED hospital admissions. Our study was limited by an inability to analyze administrative differences and to adjust for certain hospital factors that are likely to influence inpatient admissions via the ED. Nonetheless, our findings suggest that EDOUs merit further evaluation of their potential cost savings and the quality of the care they provide. An evaluation of ED observation departmental management is also needed to assess differences in care at observation units managed by emergency physicians versus nonemergency physicians.

Today more than one‐third of emergency departments (EDs) in the United States have affiliated observation units, where patients can stay 24 to 48 hours without being admitted to the hospital.[1] Observation units experienced significant growth in the United States from 2005 to 2007, secondary to policy changes involving the Centers for Medicare and Medicaid Services (CMS), which expanded reimbursement for observation services to include any clinical condition. Furthermore, CMS implemented the Recovery Audit Contractor process, which was able to fine providers and facilities for inappropriate claims, with the principle method for charge recovery being inappropriate charges for short inpatient stays.

ED observation units (EDOUs) vary in the number of beds, but are often located adjacent to the emergency department.[2] It is estimated that EDOUs have the capacity for caring for 5% to 10% of any given ED volume.[2] Almost half of EDOUs are protocol driven, allowing these units to discharge up to 80% of all patients within 24 hours.[1, 2] Some studies have suggested that EDOUs are associated with a decrease in overall hospitalization rates, leading to cost savings.[1] However, these studies were limited by their single‐center design or simulated in nature. In addition, other studies show that EDOUs decrease inpatient admissions, length of stay, and costs related to specific clinical conditions such as chest pain, transient ischemic attack, and syncope.[3]

To further evaluate the association of observation units on ED hospital admission rates nationally, we analyzed the largest ED‐based survey, the 2010 National Hospital Ambulatory Medical Care Survey (NHAMCS), to assess the impact of observation units on hospital admissions from the ED. We hypothesized that observation units decrease overall hospital admissions from the ED.

METHODS

RESULTS

There were 24,232 ED visits from 315 hospitals in the United States in our study. Of these, 82 (20.6%) hospitals had an observation unit physically separate from the ED. Hospitals with and without observation units did not have different hospital patient level characteristics. There was no association between hospital ownership, teaching status, region location, urban or rural location, and hospitals with observation units when compared with hospitals without observation units (Table 1).

In addition, there was no association between patient characteristics at the ED visit level in hospitals with observation units when compared with patient characteristics at the ED visit level in hospitals without observation units (Table 2). The average ED risk‐standardized hospital admission rate for hospitals with observation units was 13.7% (95% confidence interval [CI]: 11.3 to 16.0) compared to 16.0% (95% CI: 14.1 to 17.7) for hospitals without observation units (Figure 1). This difference of 2.3% (95% CI: 0.1 to 4.7) was not statistically significant.

Figure 1

DISCUSSION

In this national study of hospital admissions from the ED, we did not find that hospitals with observation units had a statistically significant lower ED risk‐standardized admission rate when compared with hospitals that did not have observation units. However, the difference of ED risk‐standardized hospital admission rates between hospitals with observation units and those without observation units was very small, and we were likely underpowered to detect a statistically significant difference.

Recently, EDOUs have received much attention, in part because of increases in their numbers and frequency of use.[7] Prior studies, which did not report admission rates that were risk standardized, have also demonstrated no difference in the admission rates among hospitals with and without observation units.[6, 8] Although this result seems counterintuitive, several possible explanations exist.

One reason that there may not be a relation between the rate of inpatient admission and the presence of an observation unit is that the introduction of an EDOU appears to change physician behavior. When the option to admit to an observation unit is present, ED physicians are 2 times more likely to disposition patients to observation status without a statistically significant change in the rate of inpatient admission.[6] Studies have demonstrated that after the introduction of an observation unit, ED physicians tend to overutilize observation among patients who previously would have been discharged, while continuing to admit patients as inpatients who meet observation criteria, which could result in an increase in cost for payers and patients.[7, 9]

Observation units that are protocol driven have been associated with the best patient outcomes including shorter length of stay, lower likelihood of subsequent inpatient admission, and decreased cost.[10] Furthermore, studies evaluating EDOUs suggest increased patient satisfaction and improved patient safety, especially for protocol‐driven EDOUs.[2] However, currently, only half of dedicated observation units are protocol driven. It is also possible that the ED inpatient admission rate does not capture the full impact of an observation unit on care delivery and quality. Observation units are more likely to be present in EDs with a higher overall patient census, longer patient lengths of stay, and higher rates of ambulance diversion.[6, 8] Unfortunately, NHAMCS does not distinguish protocol‐driven versus nonprotocol‐driven observation units. From a policy standpoint, as EDOUs continue to emerge, there is an opportunity to standardize how EDOUs function by using best practices.

This study should be evaluated in the context of limitations such as heterogeneity in the management of EDOUs, limited hospital factor variables that may influence hospital admissions, and small sample size associated with each hospital. Because we were not able to determine which EDs used protocol‐driven observation units, we were not able to determine the impact of having a protocol‐driven observation unit on inpatient hospital admission rates. Additionally, the study may suffer from a selection bias, as EDs with observation units have been shown to have higher patient volume, longer patient lengths of stay, and greater rates of ED diversion. Despite the small sample size, our risk‐standardized model accounted for case mix and hospital factors associated with hospital admission rates and had a high C statistic value, which indicates that the predicted probability of being admitted from the ED highly correlates with the actual outcome of being admitted from the ED. We were unable to track hospitals longitudinally to determine if a hospital's high volume is associated with the creation of EDOUs as a means to offset its demand. However, in our analysis, we did control for overall patient volume when calculating the RHSAR. Finally, we were not able to limit the dataset to observation unit admission conditions because of the limited number of visits provided per hospital by NHAMCS. We conducted an analysis using 80% power and a P value of 0.05 to determine the sample size needed to have statistically significant results. We would require 920 hospitals to have statistically significant results, which suggests we were underpowered to detect a statistically significant difference.

In this preliminary study, we did not find an association between the presence of EDOUs and ED hospital admissions. Our study was limited by an inability to analyze administrative differences and to adjust for certain hospital factors that are likely to influence inpatient admissions via the ED. Nonetheless, our findings suggest that EDOUs merit further evaluation of their potential cost savings and the quality of the care they provide. An evaluation of ED observation departmental management is also needed to assess differences in care at observation units managed by emergency physicians versus nonemergency physicians.

Hospital discharge planning is a complex process requiring efficient coordination of many different medical and social support services. For this reason, multidisciplinary teams work together to develop individualized discharge plans in an attempt to reduce preventable adverse events related to hospital discharge.[1, 2, 3, 4, 5] Despite these ongoing efforts, optimal discharge strategies have yet to be realized.[1, 4, 5, 6, 7, 8, 9]

One factor that may improve the discharge process is the early identification of patients who are approaching discharge.[10] Multidisciplinary teams cannot fully deploy comprehensive discharge plans until a physician deems a patient to be approaching discharge readiness.[8]

To our knowledge, no studies have examined the performance of physician predictions of upcoming discharge. Instead, prior studies have found that physicians have difficulty predicting the length of stay for patients seen in the emergency room and for elderly patients newly admitted to general medicine floor.[11, 12] The purpose of this study was to evaluate the ability of inpatient general medicine physicians to predict next or same‐day hospital discharges to help inform the timing of discharge planning.

METHODS

We collected daily in‐person predictions from all senior residents and attendings separately on the inpatient general medicine teams (5 resident/attending services and 4 attending‐only services) at a single 950‐bed academic medical center. We asked these physicians to predict whether each patient under their care had a greater than or equal to 80% chance of being discharged the next day, the same day, or neither (ie, no discharge on the next or same day).

Physician predictions of discharge occurred Monday through Friday at 1 of 3 time points: morning (79 _am), midday (122 _pm), or afternoon (57 _pm). Data collection focused on 1 time point per week during 2 different weeks in November 2013 and 1 week in February 2014. Predictions of same‐day discharge could only be made at the morning and midday time points. Each patient could have multiple predictions if they remained hospitalized during subsequent assessments. For each physician making a prediction, we recorded the physician training level (resident or attending).

This protocol was deemed exempt by our university's institutional review board.

RESULTS

A total of 2660 predictions were made by 24 attendings and 15 residents. Nineteen predictions were excluded because of missing prediction type or date of discharge, leaving 2641 predictions for analysis. Table 1 summarizes the total number of predictions within subgroups.

The overall daily discharge rate in our population was 22.3% (see Supporting Table 1 in the online version of this article for the raw values). The SN and PPV of physician predictions of next‐day discharge were 48% (95% confidence interval [CI]: 43%‐52%) and 51% (95% CI: 46%‐56%), respectively. The SN and PPV for same‐day discharge predictions were 73% (95% CI: 68%‐78%) and 69% (95% CI: 64%‐73%), respectively. The SP for next and same‐day discharge predictions was 90% (95% CI: 89%‐91%) and 95% (95% CI: 94%‐96%), whereas the NPV was 89% (95% CI: 88%‐90%) and 96% (95% CI: 95%‐97%), respectively.

Outcome measures for each prediction type are stratified by time of day and summarized in Table 2. For next‐day discharge predictions, the SN and PPV were lowest in the morning (SN 27%, PPV 33%) and peaked by the afternoon (SN 67%, PPV 69%). Similarly, for same‐day discharges, SN and PPV were highest later in the day (midday SN 88%, PPV 79%). This trend is also demonstrated in the SP and NPV, which increased as time to actual discharge approached, although the trends are not as pronounced as for SN and PPV.

The overall agreement between resident and attending predictions was measured and found to have kappa values of 0.51 (P < 0.001) for next‐day predictions and 0.73 (P < 0.001) for same‐day predictions, indicating moderate and substantial agreement, respectively (see Supporting Table 2 in the online version of this article).[13]

DISCUSSION

This is the first study, to our knowledge, to examine the ability of physicians to predict upcoming discharge during the course of routine general medicine inpatient care. We found that although physicians are poor predictors of discharge in the morning prior to the day of expected discharge, their ability to correctly predict inpatient discharges showed continual improvement as the difference between the prediction time and time of actual discharge shortened.

For next‐day predictions, the most accurate time point was the afternoon, when physicians correctly predicted more than two‐thirds of actual next‐day discharges. This finding suggests that physicians can provide meaningful discharge estimates as early as the afternoon prior to expected discharge. This may be an optimal time for physicians to meet with the multidisciplinary discharge teams, as many preparations hinge on timely and accurate predictions of discharge (eg, arranging patient transportation, postdischarge visits by a home health company). Multidisciplinary teams will also be reassured that an afternoon prediction of next‐day discharge would only prematurely activate discharge resources in roughly 3 out of every 10 occurrences. Even in these instances, patients may benefit from the extra time for disease counselling, medication teaching, and arrangement of home services.[4, 5, 6, 7, 8, 9]

This investigation has several limitations. Our study was conducted at a large tertiary care center over brief time periods, with an overall discharge rate of about 1 in 5 patients per day. Thus, the results may not be generalizable to hospitals with different patient populations, volume, or turnover, or when predictions are made at different times throughout the year. Furthermore, we were unable to determine if the outcome measures were affected by prolonged lengths of stay or excessive predictions on relatively few patients. However, we sought to mitigate these constraints by surveying many different respondents with varying experience levels, caring for a heterogeneous patient population at nonconsecutive time points during the year. A review of our hospital's administrative data suggests that the bed occupancy and average length of stay during our surveys were similar with most other time points during the year, and therefore representative of a typical inpatient general medicine service.

Our investigation was a novel investigation into the performance of physician discharge predictions, which are daily predictions made either explicitly or implicitly by physicians caring for patients on a general medicine ward. By utilizing a simple, subjective survey without bulky calculations, this approach closely mirrors real‐world practice patterns, and if further validated, could be easily assimilated into the normal workflow of a wide range of busy clinicians to more effectively activate individualized discharge plans.[1, 2, 3, 4, 5]

Future work could capture additional patient information, such as functional status, diagnosis, and current length of stay, which would allow identification of certain subsets of patients in which physicians are more or less accurate in predicting hospital discharge. Additionally, the outcomes of incorrect predictions, particularly the surprise discharges that left even though they were predicted to stay, could be assessed. If patients were discharged prematurely, this may be reflected by a higher 30‐day readmission rate, lower clinic follow‐up rate, and/or lower patient satisfaction scores.

CONCLUSION

Although physicians are poor predictors of discharge in the morning prior to the day of expected discharge, their ability to correctly predict inpatient discharges steadily improves as the difference between the prediction time and time of actual discharge shortened. It remains to be determined if systematic incorporation of physician discharge predictions into standard workflows will improve the effectiveness of transition of care interventions.

Disclosure: Nothing to report.

Hospital discharge planning is a complex process requiring efficient coordination of many different medical and social support services. For this reason, multidisciplinary teams work together to develop individualized discharge plans in an attempt to reduce preventable adverse events related to hospital discharge.[1, 2, 3, 4, 5] Despite these ongoing efforts, optimal discharge strategies have yet to be realized.[1, 4, 5, 6, 7, 8, 9]

One factor that may improve the discharge process is the early identification of patients who are approaching discharge.[10] Multidisciplinary teams cannot fully deploy comprehensive discharge plans until a physician deems a patient to be approaching discharge readiness.[8]

To our knowledge, no studies have examined the performance of physician predictions of upcoming discharge. Instead, prior studies have found that physicians have difficulty predicting the length of stay for patients seen in the emergency room and for elderly patients newly admitted to general medicine floor.[11, 12] The purpose of this study was to evaluate the ability of inpatient general medicine physicians to predict next or same‐day hospital discharges to help inform the timing of discharge planning.

METHODS

We collected daily in‐person predictions from all senior residents and attendings separately on the inpatient general medicine teams (5 resident/attending services and 4 attending‐only services) at a single 950‐bed academic medical center. We asked these physicians to predict whether each patient under their care had a greater than or equal to 80% chance of being discharged the next day, the same day, or neither (ie, no discharge on the next or same day).

Physician predictions of discharge occurred Monday through Friday at 1 of 3 time points: morning (79 _am), midday (122 _pm), or afternoon (57 _pm). Data collection focused on 1 time point per week during 2 different weeks in November 2013 and 1 week in February 2014. Predictions of same‐day discharge could only be made at the morning and midday time points. Each patient could have multiple predictions if they remained hospitalized during subsequent assessments. For each physician making a prediction, we recorded the physician training level (resident or attending).

This protocol was deemed exempt by our university's institutional review board.

RESULTS

A total of 2660 predictions were made by 24 attendings and 15 residents. Nineteen predictions were excluded because of missing prediction type or date of discharge, leaving 2641 predictions for analysis. Table 1 summarizes the total number of predictions within subgroups.

The overall daily discharge rate in our population was 22.3% (see Supporting Table 1 in the online version of this article for the raw values). The SN and PPV of physician predictions of next‐day discharge were 48% (95% confidence interval [CI]: 43%‐52%) and 51% (95% CI: 46%‐56%), respectively. The SN and PPV for same‐day discharge predictions were 73% (95% CI: 68%‐78%) and 69% (95% CI: 64%‐73%), respectively. The SP for next and same‐day discharge predictions was 90% (95% CI: 89%‐91%) and 95% (95% CI: 94%‐96%), whereas the NPV was 89% (95% CI: 88%‐90%) and 96% (95% CI: 95%‐97%), respectively.

Outcome measures for each prediction type are stratified by time of day and summarized in Table 2. For next‐day discharge predictions, the SN and PPV were lowest in the morning (SN 27%, PPV 33%) and peaked by the afternoon (SN 67%, PPV 69%). Similarly, for same‐day discharges, SN and PPV were highest later in the day (midday SN 88%, PPV 79%). This trend is also demonstrated in the SP and NPV, which increased as time to actual discharge approached, although the trends are not as pronounced as for SN and PPV.

The overall agreement between resident and attending predictions was measured and found to have kappa values of 0.51 (P < 0.001) for next‐day predictions and 0.73 (P < 0.001) for same‐day predictions, indicating moderate and substantial agreement, respectively (see Supporting Table 2 in the online version of this article).[13]

DISCUSSION

This is the first study, to our knowledge, to examine the ability of physicians to predict upcoming discharge during the course of routine general medicine inpatient care. We found that although physicians are poor predictors of discharge in the morning prior to the day of expected discharge, their ability to correctly predict inpatient discharges showed continual improvement as the difference between the prediction time and time of actual discharge shortened.

For next‐day predictions, the most accurate time point was the afternoon, when physicians correctly predicted more than two‐thirds of actual next‐day discharges. This finding suggests that physicians can provide meaningful discharge estimates as early as the afternoon prior to expected discharge. This may be an optimal time for physicians to meet with the multidisciplinary discharge teams, as many preparations hinge on timely and accurate predictions of discharge (eg, arranging patient transportation, postdischarge visits by a home health company). Multidisciplinary teams will also be reassured that an afternoon prediction of next‐day discharge would only prematurely activate discharge resources in roughly 3 out of every 10 occurrences. Even in these instances, patients may benefit from the extra time for disease counselling, medication teaching, and arrangement of home services.[4, 5, 6, 7, 8, 9]

This investigation has several limitations. Our study was conducted at a large tertiary care center over brief time periods, with an overall discharge rate of about 1 in 5 patients per day. Thus, the results may not be generalizable to hospitals with different patient populations, volume, or turnover, or when predictions are made at different times throughout the year. Furthermore, we were unable to determine if the outcome measures were affected by prolonged lengths of stay or excessive predictions on relatively few patients. However, we sought to mitigate these constraints by surveying many different respondents with varying experience levels, caring for a heterogeneous patient population at nonconsecutive time points during the year. A review of our hospital's administrative data suggests that the bed occupancy and average length of stay during our surveys were similar with most other time points during the year, and therefore representative of a typical inpatient general medicine service.

Our investigation was a novel investigation into the performance of physician discharge predictions, which are daily predictions made either explicitly or implicitly by physicians caring for patients on a general medicine ward. By utilizing a simple, subjective survey without bulky calculations, this approach closely mirrors real‐world practice patterns, and if further validated, could be easily assimilated into the normal workflow of a wide range of busy clinicians to more effectively activate individualized discharge plans.[1, 2, 3, 4, 5]

Future work could capture additional patient information, such as functional status, diagnosis, and current length of stay, which would allow identification of certain subsets of patients in which physicians are more or less accurate in predicting hospital discharge. Additionally, the outcomes of incorrect predictions, particularly the surprise discharges that left even though they were predicted to stay, could be assessed. If patients were discharged prematurely, this may be reflected by a higher 30‐day readmission rate, lower clinic follow‐up rate, and/or lower patient satisfaction scores.

CONCLUSION

Although physicians are poor predictors of discharge in the morning prior to the day of expected discharge, their ability to correctly predict inpatient discharges steadily improves as the difference between the prediction time and time of actual discharge shortened. It remains to be determined if systematic incorporation of physician discharge predictions into standard workflows will improve the effectiveness of transition of care interventions.

Disclosure: Nothing to report.

Hospital discharge planning is a complex process requiring efficient coordination of many different medical and social support services. For this reason, multidisciplinary teams work together to develop individualized discharge plans in an attempt to reduce preventable adverse events related to hospital discharge.[1, 2, 3, 4, 5] Despite these ongoing efforts, optimal discharge strategies have yet to be realized.[1, 4, 5, 6, 7, 8, 9]

One factor that may improve the discharge process is the early identification of patients who are approaching discharge.[10] Multidisciplinary teams cannot fully deploy comprehensive discharge plans until a physician deems a patient to be approaching discharge readiness.[8]

To our knowledge, no studies have examined the performance of physician predictions of upcoming discharge. Instead, prior studies have found that physicians have difficulty predicting the length of stay for patients seen in the emergency room and for elderly patients newly admitted to general medicine floor.[11, 12] The purpose of this study was to evaluate the ability of inpatient general medicine physicians to predict next or same‐day hospital discharges to help inform the timing of discharge planning.

METHODS

We collected daily in‐person predictions from all senior residents and attendings separately on the inpatient general medicine teams (5 resident/attending services and 4 attending‐only services) at a single 950‐bed academic medical center. We asked these physicians to predict whether each patient under their care had a greater than or equal to 80% chance of being discharged the next day, the same day, or neither (ie, no discharge on the next or same day).

Physician predictions of discharge occurred Monday through Friday at 1 of 3 time points: morning (79 _am), midday (122 _pm), or afternoon (57 _pm). Data collection focused on 1 time point per week during 2 different weeks in November 2013 and 1 week in February 2014. Predictions of same‐day discharge could only be made at the morning and midday time points. Each patient could have multiple predictions if they remained hospitalized during subsequent assessments. For each physician making a prediction, we recorded the physician training level (resident or attending).

This protocol was deemed exempt by our university's institutional review board.

RESULTS

A total of 2660 predictions were made by 24 attendings and 15 residents. Nineteen predictions were excluded because of missing prediction type or date of discharge, leaving 2641 predictions for analysis. Table 1 summarizes the total number of predictions within subgroups.

The overall daily discharge rate in our population was 22.3% (see Supporting Table 1 in the online version of this article for the raw values). The SN and PPV of physician predictions of next‐day discharge were 48% (95% confidence interval [CI]: 43%‐52%) and 51% (95% CI: 46%‐56%), respectively. The SN and PPV for same‐day discharge predictions were 73% (95% CI: 68%‐78%) and 69% (95% CI: 64%‐73%), respectively. The SP for next and same‐day discharge predictions was 90% (95% CI: 89%‐91%) and 95% (95% CI: 94%‐96%), whereas the NPV was 89% (95% CI: 88%‐90%) and 96% (95% CI: 95%‐97%), respectively.

Outcome measures for each prediction type are stratified by time of day and summarized in Table 2. For next‐day discharge predictions, the SN and PPV were lowest in the morning (SN 27%, PPV 33%) and peaked by the afternoon (SN 67%, PPV 69%). Similarly, for same‐day discharges, SN and PPV were highest later in the day (midday SN 88%, PPV 79%). This trend is also demonstrated in the SP and NPV, which increased as time to actual discharge approached, although the trends are not as pronounced as for SN and PPV.

The overall agreement between resident and attending predictions was measured and found to have kappa values of 0.51 (P < 0.001) for next‐day predictions and 0.73 (P < 0.001) for same‐day predictions, indicating moderate and substantial agreement, respectively (see Supporting Table 2 in the online version of this article).[13]

DISCUSSION

This is the first study, to our knowledge, to examine the ability of physicians to predict upcoming discharge during the course of routine general medicine inpatient care. We found that although physicians are poor predictors of discharge in the morning prior to the day of expected discharge, their ability to correctly predict inpatient discharges showed continual improvement as the difference between the prediction time and time of actual discharge shortened.

For next‐day predictions, the most accurate time point was the afternoon, when physicians correctly predicted more than two‐thirds of actual next‐day discharges. This finding suggests that physicians can provide meaningful discharge estimates as early as the afternoon prior to expected discharge. This may be an optimal time for physicians to meet with the multidisciplinary discharge teams, as many preparations hinge on timely and accurate predictions of discharge (eg, arranging patient transportation, postdischarge visits by a home health company). Multidisciplinary teams will also be reassured that an afternoon prediction of next‐day discharge would only prematurely activate discharge resources in roughly 3 out of every 10 occurrences. Even in these instances, patients may benefit from the extra time for disease counselling, medication teaching, and arrangement of home services.[4, 5, 6, 7, 8, 9]

This investigation has several limitations. Our study was conducted at a large tertiary care center over brief time periods, with an overall discharge rate of about 1 in 5 patients per day. Thus, the results may not be generalizable to hospitals with different patient populations, volume, or turnover, or when predictions are made at different times throughout the year. Furthermore, we were unable to determine if the outcome measures were affected by prolonged lengths of stay or excessive predictions on relatively few patients. However, we sought to mitigate these constraints by surveying many different respondents with varying experience levels, caring for a heterogeneous patient population at nonconsecutive time points during the year. A review of our hospital's administrative data suggests that the bed occupancy and average length of stay during our surveys were similar with most other time points during the year, and therefore representative of a typical inpatient general medicine service.

Our investigation was a novel investigation into the performance of physician discharge predictions, which are daily predictions made either explicitly or implicitly by physicians caring for patients on a general medicine ward. By utilizing a simple, subjective survey without bulky calculations, this approach closely mirrors real‐world practice patterns, and if further validated, could be easily assimilated into the normal workflow of a wide range of busy clinicians to more effectively activate individualized discharge plans.[1, 2, 3, 4, 5]

Future work could capture additional patient information, such as functional status, diagnosis, and current length of stay, which would allow identification of certain subsets of patients in which physicians are more or less accurate in predicting hospital discharge. Additionally, the outcomes of incorrect predictions, particularly the surprise discharges that left even though they were predicted to stay, could be assessed. If patients were discharged prematurely, this may be reflected by a higher 30‐day readmission rate, lower clinic follow‐up rate, and/or lower patient satisfaction scores.

CONCLUSION

Although physicians are poor predictors of discharge in the morning prior to the day of expected discharge, their ability to correctly predict inpatient discharges steadily improves as the difference between the prediction time and time of actual discharge shortened. It remains to be determined if systematic incorporation of physician discharge predictions into standard workflows will improve the effectiveness of transition of care interventions.

Disclosure: Nothing to report.

The period following discharge from the hospital is a vulnerable time for patients that can result in adverse events including avoidable emergency room visits and rehospitalizations.[1] Approximately 8.5% of all visits to the hospital result in readmissions within 30 days.[2] Poor communication of discharge information is even more pronounced for patients with language barriers or limited health literacy, particularly in ethnically diverse communities where up to 60% may speak languages other than English or French at home.[3] Defined as the degree to which individuals can obtain, process, and understand basic health information and services needed to make appropriate health decisions,[4] an estimated 55% of Canadians between the ages of 16 and 65 years have limited health literacy, and only 12% of those above the age of 65 years have adequate health literacy skills.[5]

Previous authors have demonstrated the benefits of using multiple interventions, including nonverbal communication, when designing for individuals with limited literacy.[6] Visual aids have been shown to be particularly useful to non‐English speakers and patients with limited health literacy.[7] In particular, research on medication tools for patients with limited health literacy has shown that illustrated schedules can be helpful.[8]

Typical discharge summaries are documents that are transmitted from the hospital to outpatient physicians to coordinate clinical care. The form codesigned by our team is intended to complement the summary and facilitate patient education and to provide instructions for patients to refer to after discharge.

PURPOSE

The objective of this work was to design instructions for patients going home from the hospital with relevant and actionable information, presented in an easily understandable and usable form.

METHODS

We used participatory action methodology,[9] an approach to research that encourages researchers and those who will benefit from the research to work together across all phases of research, by engaging end‐users of patient instructions from the beginning of the project. Mixed methods were used to understand needs, develop content and design, and iteratively evaluate and refine the instructions. An advisory team of patients, physicians, pharmacists, designers, researchers, and patient‐education professionals gave input into study design and execution.

Although formal inclusion and exclusion criteria were not used, care was taken to engage patients with language barriers, limited health literacy, and mental health issues.

Key methods used are listed below. See Figure 1 for a timeline of the process used to develop the instructions.

Figure 1

RESULTS

Forty‐four patients, 12 caregivers, 30 healthcare personnel, 7 patient‐education professionals, and 8 designers were involved in the design (see Figure 2 for an image of the template) based on best practices in information design, graphic design, and patient education.

Figure 2

DISCUSSION

This initiative is an example of engaging patients and caregivers as active partners in the healthcare system. Patients and caregivers were engaged as codesigners of the form from the outset and continuously throughout.

The instructions can be given to patients and caregivers at discharge as both a teaching tool and a reference that can be reviewed when at home. Process considerations are very important. As family and caregivers play an instrumental role in postdischarge care, the instructions should be given whenever possible in the presence of family. The form is a simple addition to any discharge process. It can be filled out by a single provider, a multidisciplinary team, or even the patient while undergoing discharge teaching. The time and resources to fill out the instructions will vary depending on the discharge process in place. Good discharge practices,[15] such as engaging the patient in the conversation and teach back, should be followed.

The form has been licensed as creative commons, so that any healthcare organization can use and adapt the materials to meet the needs of their patients.

The development of the form is only the first step in a larger project. Almost all of the study participants involved in the initiative were from the general internal medicine wards in downtown Toronto. We do not know yet if the results can be generalized to different patient and provider populations.

The instructions are currently being implemented in 8 hospitals throughout Toronto, spanning rehabilitation, acute care, surgery, and pediatrics. The form appears to have been appropriate and generalizable to all of these settings, but results from this multisite implementation on patient and provider experience or health outcomes are not available yet. Anticipated barriers include determining who has the responsibility for filling out the instructions and validating the accuracy of the medication list.

Discharge instructions serve many purposes. Though previous authors have developed checklists to ensure critical discharge information is included in discharge teaching, the creation of a patient‐oriented form, codesigned with patients and caregivers to provide the information that patients explicitly want at discharge, has been lacking. Using participatory action research, mixed methods, and codesign methodology, and including hard‐to‐reach patient groups was helpful in creating a design that will provide patients with key information at discharge in an easy‐to‐understand format.

The period following discharge from the hospital is a vulnerable time for patients that can result in adverse events including avoidable emergency room visits and rehospitalizations.[1] Approximately 8.5% of all visits to the hospital result in readmissions within 30 days.[2] Poor communication of discharge information is even more pronounced for patients with language barriers or limited health literacy, particularly in ethnically diverse communities where up to 60% may speak languages other than English or French at home.[3] Defined as the degree to which individuals can obtain, process, and understand basic health information and services needed to make appropriate health decisions,[4] an estimated 55% of Canadians between the ages of 16 and 65 years have limited health literacy, and only 12% of those above the age of 65 years have adequate health literacy skills.[5]

Previous authors have demonstrated the benefits of using multiple interventions, including nonverbal communication, when designing for individuals with limited literacy.[6] Visual aids have been shown to be particularly useful to non‐English speakers and patients with limited health literacy.[7] In particular, research on medication tools for patients with limited health literacy has shown that illustrated schedules can be helpful.[8]

Typical discharge summaries are documents that are transmitted from the hospital to outpatient physicians to coordinate clinical care. The form codesigned by our team is intended to complement the summary and facilitate patient education and to provide instructions for patients to refer to after discharge.

PURPOSE

The objective of this work was to design instructions for patients going home from the hospital with relevant and actionable information, presented in an easily understandable and usable form.

METHODS

We used participatory action methodology,[9] an approach to research that encourages researchers and those who will benefit from the research to work together across all phases of research, by engaging end‐users of patient instructions from the beginning of the project. Mixed methods were used to understand needs, develop content and design, and iteratively evaluate and refine the instructions. An advisory team of patients, physicians, pharmacists, designers, researchers, and patient‐education professionals gave input into study design and execution.

Although formal inclusion and exclusion criteria were not used, care was taken to engage patients with language barriers, limited health literacy, and mental health issues.

Key methods used are listed below. See Figure 1 for a timeline of the process used to develop the instructions.

Figure 1

RESULTS

Forty‐four patients, 12 caregivers, 30 healthcare personnel, 7 patient‐education professionals, and 8 designers were involved in the design (see Figure 2 for an image of the template) based on best practices in information design, graphic design, and patient education.

Figure 2

DISCUSSION

This initiative is an example of engaging patients and caregivers as active partners in the healthcare system. Patients and caregivers were engaged as codesigners of the form from the outset and continuously throughout.

The instructions can be given to patients and caregivers at discharge as both a teaching tool and a reference that can be reviewed when at home. Process considerations are very important. As family and caregivers play an instrumental role in postdischarge care, the instructions should be given whenever possible in the presence of family. The form is a simple addition to any discharge process. It can be filled out by a single provider, a multidisciplinary team, or even the patient while undergoing discharge teaching. The time and resources to fill out the instructions will vary depending on the discharge process in place. Good discharge practices,[15] such as engaging the patient in the conversation and teach back, should be followed.

The form has been licensed as creative commons, so that any healthcare organization can use and adapt the materials to meet the needs of their patients.

The development of the form is only the first step in a larger project. Almost all of the study participants involved in the initiative were from the general internal medicine wards in downtown Toronto. We do not know yet if the results can be generalized to different patient and provider populations.

The instructions are currently being implemented in 8 hospitals throughout Toronto, spanning rehabilitation, acute care, surgery, and pediatrics. The form appears to have been appropriate and generalizable to all of these settings, but results from this multisite implementation on patient and provider experience or health outcomes are not available yet. Anticipated barriers include determining who has the responsibility for filling out the instructions and validating the accuracy of the medication list.

Discharge instructions serve many purposes. Though previous authors have developed checklists to ensure critical discharge information is included in discharge teaching, the creation of a patient‐oriented form, codesigned with patients and caregivers to provide the information that patients explicitly want at discharge, has been lacking. Using participatory action research, mixed methods, and codesign methodology, and including hard‐to‐reach patient groups was helpful in creating a design that will provide patients with key information at discharge in an easy‐to‐understand format.

The period following discharge from the hospital is a vulnerable time for patients that can result in adverse events including avoidable emergency room visits and rehospitalizations.[1] Approximately 8.5% of all visits to the hospital result in readmissions within 30 days.[2] Poor communication of discharge information is even more pronounced for patients with language barriers or limited health literacy, particularly in ethnically diverse communities where up to 60% may speak languages other than English or French at home.[3] Defined as the degree to which individuals can obtain, process, and understand basic health information and services needed to make appropriate health decisions,[4] an estimated 55% of Canadians between the ages of 16 and 65 years have limited health literacy, and only 12% of those above the age of 65 years have adequate health literacy skills.[5]

Previous authors have demonstrated the benefits of using multiple interventions, including nonverbal communication, when designing for individuals with limited literacy.[6] Visual aids have been shown to be particularly useful to non‐English speakers and patients with limited health literacy.[7] In particular, research on medication tools for patients with limited health literacy has shown that illustrated schedules can be helpful.[8]

Typical discharge summaries are documents that are transmitted from the hospital to outpatient physicians to coordinate clinical care. The form codesigned by our team is intended to complement the summary and facilitate patient education and to provide instructions for patients to refer to after discharge.

PURPOSE

The objective of this work was to design instructions for patients going home from the hospital with relevant and actionable information, presented in an easily understandable and usable form.

METHODS

We used participatory action methodology,[9] an approach to research that encourages researchers and those who will benefit from the research to work together across all phases of research, by engaging end‐users of patient instructions from the beginning of the project. Mixed methods were used to understand needs, develop content and design, and iteratively evaluate and refine the instructions. An advisory team of patients, physicians, pharmacists, designers, researchers, and patient‐education professionals gave input into study design and execution.

Although formal inclusion and exclusion criteria were not used, care was taken to engage patients with language barriers, limited health literacy, and mental health issues.

Key methods used are listed below. See Figure 1 for a timeline of the process used to develop the instructions.

Figure 1

RESULTS

Forty‐four patients, 12 caregivers, 30 healthcare personnel, 7 patient‐education professionals, and 8 designers were involved in the design (see Figure 2 for an image of the template) based on best practices in information design, graphic design, and patient education.

Figure 2

DISCUSSION

This initiative is an example of engaging patients and caregivers as active partners in the healthcare system. Patients and caregivers were engaged as codesigners of the form from the outset and continuously throughout.

The instructions can be given to patients and caregivers at discharge as both a teaching tool and a reference that can be reviewed when at home. Process considerations are very important. As family and caregivers play an instrumental role in postdischarge care, the instructions should be given whenever possible in the presence of family. The form is a simple addition to any discharge process. It can be filled out by a single provider, a multidisciplinary team, or even the patient while undergoing discharge teaching. The time and resources to fill out the instructions will vary depending on the discharge process in place. Good discharge practices,[15] such as engaging the patient in the conversation and teach back, should be followed.

The form has been licensed as creative commons, so that any healthcare organization can use and adapt the materials to meet the needs of their patients.

The development of the form is only the first step in a larger project. Almost all of the study participants involved in the initiative were from the general internal medicine wards in downtown Toronto. We do not know yet if the results can be generalized to different patient and provider populations.

The instructions are currently being implemented in 8 hospitals throughout Toronto, spanning rehabilitation, acute care, surgery, and pediatrics. The form appears to have been appropriate and generalizable to all of these settings, but results from this multisite implementation on patient and provider experience or health outcomes are not available yet. Anticipated barriers include determining who has the responsibility for filling out the instructions and validating the accuracy of the medication list.

Discharge instructions serve many purposes. Though previous authors have developed checklists to ensure critical discharge information is included in discharge teaching, the creation of a patient‐oriented form, codesigned with patients and caregivers to provide the information that patients explicitly want at discharge, has been lacking. Using participatory action research, mixed methods, and codesign methodology, and including hard‐to‐reach patient groups was helpful in creating a design that will provide patients with key information at discharge in an easy‐to‐understand format.

Patient experience has become a major component of the Center for Medicare and Medicaid Services Value‐Based Purchasing initiative.[1] Hospitals have therefore focused quality improvement (QI) efforts on this area.[2] Hospital performance in the realm of patient experience is generally determined using systematic surveys with closed‐ended questions, but patient‐generated narrative feedback can help hospitals identify the components of care that contribute to patient satisfaction and or are in need of improvement.[3] Online narrative responses posted by patients on rating websites or social media have been criticized because they may not be representative of the population,[4] but they also have some advantages.[5] Any patient may leave a comment, not just those who are selected for a survey. Patients may also experience benefits through the act of sharing their story with others. Moreover, most US hospitals use some form of social media,[6] which they can theoretically use to self‐collect narrative data online. To realize the full potential of patient‐generated online narratives, we need a clearer understanding of the best practices for collecting and using these narratives. We therefore solicited patient feedback on the Facebook page of a large tertiary academic medical center to determine whether it is feasible to use social media platforms for learning about and improving hospital quality.

METHODS

Baystate Medical Center (BMC) is a tertiary care medical center in western Massachusetts. We identified key BMC stakeholders in the areas of QI and public affairs. Noting that patients have expressed interest in leaving comments via social media,[7] the group opted to perform a pilot study to obtain patient narratives via a Facebook prompt (Facebook is a social media site used by an estimated 58% of US adults[8]). The BMC public affairs department delivered a press release to the local media describing a 3‐week period during which patients were invited to leave narrative feedback on the BMC Facebook wall. The BMC Institutional Review Board deemed that this study did not constitute human subjects research.

During March 2014 (March 10, 2014March 24, 2014), we posted (once a week) an open‐ended prompt on BMC's Facebook wall. The prompt was designed to elicit novel descriptions of patient experience that could help to drive QI. It read: We want to hear about your experiences. In the comment section below, please tell us what we do well and how we can improve your care. Because of concerns about the potential reputational risks of allowing open feedback on a public social media page, the prompt also reminded patients of the social media ground rules: there should be no mention of specific physicians, nurses, or other caregivers by name (for liability reasons); and patients should not include details about their medical history (for privacy reasons).

We collected all posts to preserve comments and used directed qualitative content analysis to examine them.[9] Two research team members[3, 10, 11] independently coded the responses. Starting with an a priori codebook that was developed during a previous study,[3] they amended the codebook through an iterative process to incorporate new concepts. After independently coding all blocks of text, the coders reviewed their coding selections and resolved discrepancies through discussion. We then performed second‐level coding, in which codes were organized into major pertinent themes. We reviewed the coded text after applying secondary codes in order to check for accuracy of coding and theme assignment as well as completeness of second‐level coding. We calculated percent agreement, defined as both raters scoring a block of text with the same code divided by total number of codes. We also calculated the Spearman correlation between the 2 reviewers. We used descriptive statistics to assess the frequency of select codes and themes (see Supporting Information, Appendix 1 and Appendix 2, in the online version of this article).[9, 12, 13]

RESULTS

Over a 3‐week study period, 47 comments were submitted by 37 respondents. This yielded 148 codable statements (Table 1). Despite limited information on respondents, we ascertained from Facebook that 32 (86%) were women and 5 (14%) were men.

From coded text, several broad themes were identified (see Table 1 for representative quotes): (1) comments about staff (17/37 respondents, 45.9%). These included positive descriptions of efficiency, caring behavior, good training, and good communication, whereas negative comments included descriptions of unfriendliness, apparent lack of caring, inattentiveness, poor training, unprofessional behavior, and poor communication; (2) comments about specific departments (22/37, 59.5%); (3) comments on technical aspects of care, including perceived errors, incorrect diagnoses, and inattention to pain control (9/37, 24.3%); and (4) comments describing the hospital physical plant, parking, and amenities (9/37, 24.3%). There were a few miscellaneous comments that did not fit into these broad themes, such as expressions of gratitude for our solicitation of narratives. Percent agreement between coders was 80% and Spearman's Rho was 0.82 (p<0.001).

A small number (n=3) of respondents repeatedly made comments over the 3‐week period, accounting for 30% (45/148) of codes. These repetitive commenters tended to dominate the Facebook conversation, at times describing the same experience more than once.

DISCUSSION

In this study evaluating the potential utility of social media as a hospital QI tool, several broad themes emerged. From these themes, we identified several areas that could be deemed as QI targets, including: training staff to be more responsive and sensitive to patients needs and concerns, improving patient and visitor parking, and reducing emergency department waiting times. However, the insight gained from solicited Facebook comments was similar to feedback gained from more traditional approaches of soliciting patient perspectives on care, such as patient experience surveys.[14]

Our findings should be viewed in the context of prior work focused on patient narratives in healthcare. Greaves et al. used sentiment analysis to describe the content of nearly 200,000 tweets (comments posted on the social networking website Twitter) sent to National Health Service (NHS) hospitals.[15] Themes were similar to those found in our study: (1) interaction with staff, (2) environment and facilities, and (3) issues of access and timeliness of service. Notably, these themes mirrored prior work examining narratives at NHS hospitals[3] and were similar to domains of commonly used surveys of patient experience.[14] The authors noted that there were issues with the signal to noise ratio (only about 10% of tweets were about quality) and the enforced brevity of Twitter (tweets must be 140 characters or less). These limitations suggest that using Twitter to identify QI targets would be difficult.

In contrast to Greaves et al., we chose to solicit feedback on our hospital's Facebook page. Facebook does not have Twitter's enforced brevity, allowing for more detailed narratives. In addition, we did not encounter the signal‐to‐noise problem, because our prompt was designed to request feedback that was relevant to recent experiences of care. However, a few respondents dominated the conversation, supporting the hypothesis that those most likely to comment may be the patients or families who have had the best or worst experiences. In the future, we will attempt to address this limitation and reduce the influence of repeat commenters by changing our prompt (eg, Please tell us about your experience, but please do not post descriptions of the same experience more than once.).

This pilot demonstrated some of the previously described benefits of online narratives.[5] First, there appears to be value in allowing patients to share their experiences and to read the experiences of others (as indicated in a few grateful patients comments). Second, soliciting online narratives offers a way for hospitals to demonstrate a commitment to transparency. Third, in contrast to closed‐ended survey questions, narrative comments help to identify why patients were satisfied or unsatisfied with their care. Although some surveys with closed‐ended questions also allow for narratives, these comments may or may not be carefully reviewed by the hospital. Using social media to solicit and respond to comments enhances existing methods for evaluating patient experience by engaging patients in a public space, which increases the likelihood that hospitals will attempt to improve care in response.

Notably, none of the identified areas for improvement could be considered novel QI targets for BMC. For example, our hospital has been very focused on training staff around patient experience, and emergency department wait times are the focus of a system‐wide improvement effort called Patient Progress.

This study has other limitations. We conducted this study over a 3‐week time period in a single center and on a single social media site whose members may not be representative of the overall patient population at BMC. Although we do not know how generalizable our findings are (in terms of identifying QI targets), we feel that we have demonstrated how using social media to collect data on patient experience is feasible and could be informative for other hospitals in other locations. It is possible that we did not allow the experiment to run long enough; a longer time or broader outreach (eg, a handout given to every discharged patient over a longer period) may be needed to allow patients adequate opportunity to comment. Of note, we did not specifically examine responses by time period, but it does seem, in post hoc analysis, that after 2 weeks of data collection we reached theoretical saturation with no new themes emerging in the third week (eg, third‐week comments included I heart your nurses. and Love Baystate but hate the ER.). More work is also needed that includes a broader range of social media platforms and more participating hospitals.

In conclusion, the opportunity to provide feedback on Facebook has the potential to engage and empower patients, and hospitals can use these online narratives to help to drive improvement efforts. Yet potential benefits must be weighed against reputational risks, a lack of representative respondents, and the paucity of novel QI targets obtained in this study.

Disclosures: Dr. Lagu is supported by the National Heart, Lung, and Blood Institute of the National Institutes of Health under award number K01HL114745. The authors report no conflicts of interest.

Patient experience has become a major component of the Center for Medicare and Medicaid Services Value‐Based Purchasing initiative.[1] Hospitals have therefore focused quality improvement (QI) efforts on this area.[2] Hospital performance in the realm of patient experience is generally determined using systematic surveys with closed‐ended questions, but patient‐generated narrative feedback can help hospitals identify the components of care that contribute to patient satisfaction and or are in need of improvement.[3] Online narrative responses posted by patients on rating websites or social media have been criticized because they may not be representative of the population,[4] but they also have some advantages.[5] Any patient may leave a comment, not just those who are selected for a survey. Patients may also experience benefits through the act of sharing their story with others. Moreover, most US hospitals use some form of social media,[6] which they can theoretically use to self‐collect narrative data online. To realize the full potential of patient‐generated online narratives, we need a clearer understanding of the best practices for collecting and using these narratives. We therefore solicited patient feedback on the Facebook page of a large tertiary academic medical center to determine whether it is feasible to use social media platforms for learning about and improving hospital quality.

METHODS

Baystate Medical Center (BMC) is a tertiary care medical center in western Massachusetts. We identified key BMC stakeholders in the areas of QI and public affairs. Noting that patients have expressed interest in leaving comments via social media,[7] the group opted to perform a pilot study to obtain patient narratives via a Facebook prompt (Facebook is a social media site used by an estimated 58% of US adults[8]). The BMC public affairs department delivered a press release to the local media describing a 3‐week period during which patients were invited to leave narrative feedback on the BMC Facebook wall. The BMC Institutional Review Board deemed that this study did not constitute human subjects research.

During March 2014 (March 10, 2014March 24, 2014), we posted (once a week) an open‐ended prompt on BMC's Facebook wall. The prompt was designed to elicit novel descriptions of patient experience that could help to drive QI. It read: We want to hear about your experiences. In the comment section below, please tell us what we do well and how we can improve your care. Because of concerns about the potential reputational risks of allowing open feedback on a public social media page, the prompt also reminded patients of the social media ground rules: there should be no mention of specific physicians, nurses, or other caregivers by name (for liability reasons); and patients should not include details about their medical history (for privacy reasons).

We collected all posts to preserve comments and used directed qualitative content analysis to examine them.[9] Two research team members[3, 10, 11] independently coded the responses. Starting with an a priori codebook that was developed during a previous study,[3] they amended the codebook through an iterative process to incorporate new concepts. After independently coding all blocks of text, the coders reviewed their coding selections and resolved discrepancies through discussion. We then performed second‐level coding, in which codes were organized into major pertinent themes. We reviewed the coded text after applying secondary codes in order to check for accuracy of coding and theme assignment as well as completeness of second‐level coding. We calculated percent agreement, defined as both raters scoring a block of text with the same code divided by total number of codes. We also calculated the Spearman correlation between the 2 reviewers. We used descriptive statistics to assess the frequency of select codes and themes (see Supporting Information, Appendix 1 and Appendix 2, in the online version of this article).[9, 12, 13]

RESULTS

Over a 3‐week study period, 47 comments were submitted by 37 respondents. This yielded 148 codable statements (Table 1). Despite limited information on respondents, we ascertained from Facebook that 32 (86%) were women and 5 (14%) were men.

From coded text, several broad themes were identified (see Table 1 for representative quotes): (1) comments about staff (17/37 respondents, 45.9%). These included positive descriptions of efficiency, caring behavior, good training, and good communication, whereas negative comments included descriptions of unfriendliness, apparent lack of caring, inattentiveness, poor training, unprofessional behavior, and poor communication; (2) comments about specific departments (22/37, 59.5%); (3) comments on technical aspects of care, including perceived errors, incorrect diagnoses, and inattention to pain control (9/37, 24.3%); and (4) comments describing the hospital physical plant, parking, and amenities (9/37, 24.3%). There were a few miscellaneous comments that did not fit into these broad themes, such as expressions of gratitude for our solicitation of narratives. Percent agreement between coders was 80% and Spearman's Rho was 0.82 (p<0.001).

A small number (n=3) of respondents repeatedly made comments over the 3‐week period, accounting for 30% (45/148) of codes. These repetitive commenters tended to dominate the Facebook conversation, at times describing the same experience more than once.

DISCUSSION

In this study evaluating the potential utility of social media as a hospital QI tool, several broad themes emerged. From these themes, we identified several areas that could be deemed as QI targets, including: training staff to be more responsive and sensitive to patients needs and concerns, improving patient and visitor parking, and reducing emergency department waiting times. However, the insight gained from solicited Facebook comments was similar to feedback gained from more traditional approaches of soliciting patient perspectives on care, such as patient experience surveys.[14]

Our findings should be viewed in the context of prior work focused on patient narratives in healthcare. Greaves et al. used sentiment analysis to describe the content of nearly 200,000 tweets (comments posted on the social networking website Twitter) sent to National Health Service (NHS) hospitals.[15] Themes were similar to those found in our study: (1) interaction with staff, (2) environment and facilities, and (3) issues of access and timeliness of service. Notably, these themes mirrored prior work examining narratives at NHS hospitals[3] and were similar to domains of commonly used surveys of patient experience.[14] The authors noted that there were issues with the signal to noise ratio (only about 10% of tweets were about quality) and the enforced brevity of Twitter (tweets must be 140 characters or less). These limitations suggest that using Twitter to identify QI targets would be difficult.

In contrast to Greaves et al., we chose to solicit feedback on our hospital's Facebook page. Facebook does not have Twitter's enforced brevity, allowing for more detailed narratives. In addition, we did not encounter the signal‐to‐noise problem, because our prompt was designed to request feedback that was relevant to recent experiences of care. However, a few respondents dominated the conversation, supporting the hypothesis that those most likely to comment may be the patients or families who have had the best or worst experiences. In the future, we will attempt to address this limitation and reduce the influence of repeat commenters by changing our prompt (eg, Please tell us about your experience, but please do not post descriptions of the same experience more than once.).

This pilot demonstrated some of the previously described benefits of online narratives.[5] First, there appears to be value in allowing patients to share their experiences and to read the experiences of others (as indicated in a few grateful patients comments). Second, soliciting online narratives offers a way for hospitals to demonstrate a commitment to transparency. Third, in contrast to closed‐ended survey questions, narrative comments help to identify why patients were satisfied or unsatisfied with their care. Although some surveys with closed‐ended questions also allow for narratives, these comments may or may not be carefully reviewed by the hospital. Using social media to solicit and respond to comments enhances existing methods for evaluating patient experience by engaging patients in a public space, which increases the likelihood that hospitals will attempt to improve care in response.

Notably, none of the identified areas for improvement could be considered novel QI targets for BMC. For example, our hospital has been very focused on training staff around patient experience, and emergency department wait times are the focus of a system‐wide improvement effort called Patient Progress.

This study has other limitations. We conducted this study over a 3‐week time period in a single center and on a single social media site whose members may not be representative of the overall patient population at BMC. Although we do not know how generalizable our findings are (in terms of identifying QI targets), we feel that we have demonstrated how using social media to collect data on patient experience is feasible and could be informative for other hospitals in other locations. It is possible that we did not allow the experiment to run long enough; a longer time or broader outreach (eg, a handout given to every discharged patient over a longer period) may be needed to allow patients adequate opportunity to comment. Of note, we did not specifically examine responses by time period, but it does seem, in post hoc analysis, that after 2 weeks of data collection we reached theoretical saturation with no new themes emerging in the third week (eg, third‐week comments included I heart your nurses. and Love Baystate but hate the ER.). More work is also needed that includes a broader range of social media platforms and more participating hospitals.

In conclusion, the opportunity to provide feedback on Facebook has the potential to engage and empower patients, and hospitals can use these online narratives to help to drive improvement efforts. Yet potential benefits must be weighed against reputational risks, a lack of representative respondents, and the paucity of novel QI targets obtained in this study.

Disclosures: Dr. Lagu is supported by the National Heart, Lung, and Blood Institute of the National Institutes of Health under award number K01HL114745. The authors report no conflicts of interest.

Patient experience has become a major component of the Center for Medicare and Medicaid Services Value‐Based Purchasing initiative.[1] Hospitals have therefore focused quality improvement (QI) efforts on this area.[2] Hospital performance in the realm of patient experience is generally determined using systematic surveys with closed‐ended questions, but patient‐generated narrative feedback can help hospitals identify the components of care that contribute to patient satisfaction and or are in need of improvement.[3] Online narrative responses posted by patients on rating websites or social media have been criticized because they may not be representative of the population,[4] but they also have some advantages.[5] Any patient may leave a comment, not just those who are selected for a survey. Patients may also experience benefits through the act of sharing their story with others. Moreover, most US hospitals use some form of social media,[6] which they can theoretically use to self‐collect narrative data online. To realize the full potential of patient‐generated online narratives, we need a clearer understanding of the best practices for collecting and using these narratives. We therefore solicited patient feedback on the Facebook page of a large tertiary academic medical center to determine whether it is feasible to use social media platforms for learning about and improving hospital quality.

METHODS

Baystate Medical Center (BMC) is a tertiary care medical center in western Massachusetts. We identified key BMC stakeholders in the areas of QI and public affairs. Noting that patients have expressed interest in leaving comments via social media,[7] the group opted to perform a pilot study to obtain patient narratives via a Facebook prompt (Facebook is a social media site used by an estimated 58% of US adults[8]). The BMC public affairs department delivered a press release to the local media describing a 3‐week period during which patients were invited to leave narrative feedback on the BMC Facebook wall. The BMC Institutional Review Board deemed that this study did not constitute human subjects research.

During March 2014 (March 10, 2014March 24, 2014), we posted (once a week) an open‐ended prompt on BMC's Facebook wall. The prompt was designed to elicit novel descriptions of patient experience that could help to drive QI. It read: We want to hear about your experiences. In the comment section below, please tell us what we do well and how we can improve your care. Because of concerns about the potential reputational risks of allowing open feedback on a public social media page, the prompt also reminded patients of the social media ground rules: there should be no mention of specific physicians, nurses, or other caregivers by name (for liability reasons); and patients should not include details about their medical history (for privacy reasons).

We collected all posts to preserve comments and used directed qualitative content analysis to examine them.[9] Two research team members[3, 10, 11] independently coded the responses. Starting with an a priori codebook that was developed during a previous study,[3] they amended the codebook through an iterative process to incorporate new concepts. After independently coding all blocks of text, the coders reviewed their coding selections and resolved discrepancies through discussion. We then performed second‐level coding, in which codes were organized into major pertinent themes. We reviewed the coded text after applying secondary codes in order to check for accuracy of coding and theme assignment as well as completeness of second‐level coding. We calculated percent agreement, defined as both raters scoring a block of text with the same code divided by total number of codes. We also calculated the Spearman correlation between the 2 reviewers. We used descriptive statistics to assess the frequency of select codes and themes (see Supporting Information, Appendix 1 and Appendix 2, in the online version of this article).[9, 12, 13]

RESULTS

Over a 3‐week study period, 47 comments were submitted by 37 respondents. This yielded 148 codable statements (Table 1). Despite limited information on respondents, we ascertained from Facebook that 32 (86%) were women and 5 (14%) were men.

From coded text, several broad themes were identified (see Table 1 for representative quotes): (1) comments about staff (17/37 respondents, 45.9%). These included positive descriptions of efficiency, caring behavior, good training, and good communication, whereas negative comments included descriptions of unfriendliness, apparent lack of caring, inattentiveness, poor training, unprofessional behavior, and poor communication; (2) comments about specific departments (22/37, 59.5%); (3) comments on technical aspects of care, including perceived errors, incorrect diagnoses, and inattention to pain control (9/37, 24.3%); and (4) comments describing the hospital physical plant, parking, and amenities (9/37, 24.3%). There were a few miscellaneous comments that did not fit into these broad themes, such as expressions of gratitude for our solicitation of narratives. Percent agreement between coders was 80% and Spearman's Rho was 0.82 (p<0.001).

A small number (n=3) of respondents repeatedly made comments over the 3‐week period, accounting for 30% (45/148) of codes. These repetitive commenters tended to dominate the Facebook conversation, at times describing the same experience more than once.

DISCUSSION

In this study evaluating the potential utility of social media as a hospital QI tool, several broad themes emerged. From these themes, we identified several areas that could be deemed as QI targets, including: training staff to be more responsive and sensitive to patients needs and concerns, improving patient and visitor parking, and reducing emergency department waiting times. However, the insight gained from solicited Facebook comments was similar to feedback gained from more traditional approaches of soliciting patient perspectives on care, such as patient experience surveys.[14]

Our findings should be viewed in the context of prior work focused on patient narratives in healthcare. Greaves et al. used sentiment analysis to describe the content of nearly 200,000 tweets (comments posted on the social networking website Twitter) sent to National Health Service (NHS) hospitals.[15] Themes were similar to those found in our study: (1) interaction with staff, (2) environment and facilities, and (3) issues of access and timeliness of service. Notably, these themes mirrored prior work examining narratives at NHS hospitals[3] and were similar to domains of commonly used surveys of patient experience.[14] The authors noted that there were issues with the signal to noise ratio (only about 10% of tweets were about quality) and the enforced brevity of Twitter (tweets must be 140 characters or less). These limitations suggest that using Twitter to identify QI targets would be difficult.

In contrast to Greaves et al., we chose to solicit feedback on our hospital's Facebook page. Facebook does not have Twitter's enforced brevity, allowing for more detailed narratives. In addition, we did not encounter the signal‐to‐noise problem, because our prompt was designed to request feedback that was relevant to recent experiences of care. However, a few respondents dominated the conversation, supporting the hypothesis that those most likely to comment may be the patients or families who have had the best or worst experiences. In the future, we will attempt to address this limitation and reduce the influence of repeat commenters by changing our prompt (eg, Please tell us about your experience, but please do not post descriptions of the same experience more than once.).

This pilot demonstrated some of the previously described benefits of online narratives.[5] First, there appears to be value in allowing patients to share their experiences and to read the experiences of others (as indicated in a few grateful patients comments). Second, soliciting online narratives offers a way for hospitals to demonstrate a commitment to transparency. Third, in contrast to closed‐ended survey questions, narrative comments help to identify why patients were satisfied or unsatisfied with their care. Although some surveys with closed‐ended questions also allow for narratives, these comments may or may not be carefully reviewed by the hospital. Using social media to solicit and respond to comments enhances existing methods for evaluating patient experience by engaging patients in a public space, which increases the likelihood that hospitals will attempt to improve care in response.

Notably, none of the identified areas for improvement could be considered novel QI targets for BMC. For example, our hospital has been very focused on training staff around patient experience, and emergency department wait times are the focus of a system‐wide improvement effort called Patient Progress.

This study has other limitations. We conducted this study over a 3‐week time period in a single center and on a single social media site whose members may not be representative of the overall patient population at BMC. Although we do not know how generalizable our findings are (in terms of identifying QI targets), we feel that we have demonstrated how using social media to collect data on patient experience is feasible and could be informative for other hospitals in other locations. It is possible that we did not allow the experiment to run long enough; a longer time or broader outreach (eg, a handout given to every discharged patient over a longer period) may be needed to allow patients adequate opportunity to comment. Of note, we did not specifically examine responses by time period, but it does seem, in post hoc analysis, that after 2 weeks of data collection we reached theoretical saturation with no new themes emerging in the third week (eg, third‐week comments included I heart your nurses. and Love Baystate but hate the ER.). More work is also needed that includes a broader range of social media platforms and more participating hospitals.

In conclusion, the opportunity to provide feedback on Facebook has the potential to engage and empower patients, and hospitals can use these online narratives to help to drive improvement efforts. Yet potential benefits must be weighed against reputational risks, a lack of representative respondents, and the paucity of novel QI targets obtained in this study.

Disclosures: Dr. Lagu is supported by the National Heart, Lung, and Blood Institute of the National Institutes of Health under award number K01HL114745. The authors report no conflicts of interest.

A seminal 1996 New England Journal of Medicine article introduced the term hospitalist to describe the emerging trend of primary care physicians practicing in inpatient hospital settings.[1] Although physicians had practice patterns akin to hospitalists prior to the introduction of the term,[2] the field continues to grow and formalize as a unique specialty in medicine.

There is currently no board certification or specialty billing code associated with hospitalists. In 2009, the American Board of Internal Medicine and American Board of Family Medicine introduced a Focused Practice in Hospital Medicine optional recertification pathway.[3] However, absent a unique identifier, it remains difficult to identify the number of hospitalists practicing today. Issues with identification notwithstanding, published data consistently suggest that the number of hospitalists has grown dramatically over the last 2 decades.[4, 5, 6]

The Centers for Medicare and Medicaid Services (CMS), along with other payers, classify hospitalists based on their board certificationmost commonly internal medicine or family practice. Other approaches for more precise assessment utilized billing data or hospital designation. Saint et al. identified hospital‐based providers practicing in Washington State in 1994 using variable thresholds of billing for inpatient services.[2] In 2011, Welch et al. identified 25,787 hospitalists nationwide, using a 90% threshold of billing inpatient services in Medicare data.[6] That same year, an American Hospital Association survey identified 34,411 hospitalists based on self‐reporting.[4]

Building on the work of previous researchers, we applied an updated threshold of inpatient services in publicly available 2012 Medicare Provider Utilization and Payment Data to identify a range of hospitalists practicing in the United States. We also examine the codes billed by providers identified in different decile billing thresholds to assess the validity of using lower thresholds to identify hospitalists.

METHODS

RESULTS

The 2012 Medicare pay data included 664,253 physicians with unique NPIs. Of these, 169,317 had IM/FP specialty designations, whereas just under half (46.25%) of those physicians billed any of the inpatient HCPCS codes associated with our threshold.

Table 1 describes the range of number of hospitalists identified by varying the threshold of inpatient services. A total of 28,473 providers bill the threshold‐associated inpatient codes almost exclusively, whereas each descending decile increases in size by an average of 7.29%.

We also analyzed billing patterns of a subset of physicians who billed our threshold codes between 60% and 70% of the time to better characterize the remainder of clinical work they perform. This group included 2282 physicians and only 56 unique HCPCS codes with frequencies greater than 0.1%. After clustering related codes, we identified 4 common code groups that account for the majority of the remaining billing beyond inpatient threshold codes (Table 2).

DISCUSSION

Hospitalists make up approximately 5% of the practicing physicians nationwide, performing a critical role caring for hospitalized patients. Saint et al. defined a pure hospitalist as a physician who meets a 90% threshold of inpatient services.[2] This approach has been replicated in subsequent studies that used a 90% threshold to identify hospitalists.[5, 6] Our results with the same threshold reveal more than 28,000 hospitalists with nearly uniform practice patterns, a 10% growth in the number of hospitalists from the Welch et al. analysis in 2011.[6]

A threshold is not a perfect tool for identifying groups of practicing physicians, as it creates an arbitrary cutoff within a dataset. Undoubtedly our analysis could include providers who would not consider themselves hospitalists, or alternatively, appear to have a hospital‐based practice when they do not. Our results suggest that a 90% threshold may identify a majority of practicing hospitalists, but excludes providers who likely identify as hospitalists albeit with divergent practice and billing patterns.

A lower threshold may be more inclusive of the current realities of hospitalist practice, accounting for the myriad other services provided during, immediately prior to, or following a hospitalization. With hospitalists commonly practicing in diverse facility settings, rotating through rehabilitation or nursing home facilities, discharge clinics, and preoperative medicine practices, the continued use of a 90% threshold appears to exclude a sizable number of practicing hospitalists.

In the absence of a formal identifier, developing identification methodologies that account for the diversity of hospitalist practice is crucial. As physician payment transitions to value‐based reimbursement, systems must have the ability to account for and allocate the most efficient mix of providers for their patient populations. Because provider alignment and coordination are structural features of these programs, these systems‐based changes in effect require accurate identification of hospitalists, yet currently lack the tools to do so.

A seminal 1996 New England Journal of Medicine article introduced the term hospitalist to describe the emerging trend of primary care physicians practicing in inpatient hospital settings.[1] Although physicians had practice patterns akin to hospitalists prior to the introduction of the term,[2] the field continues to grow and formalize as a unique specialty in medicine.

There is currently no board certification or specialty billing code associated with hospitalists. In 2009, the American Board of Internal Medicine and American Board of Family Medicine introduced a Focused Practice in Hospital Medicine optional recertification pathway.[3] However, absent a unique identifier, it remains difficult to identify the number of hospitalists practicing today. Issues with identification notwithstanding, published data consistently suggest that the number of hospitalists has grown dramatically over the last 2 decades.[4, 5, 6]

The Centers for Medicare and Medicaid Services (CMS), along with other payers, classify hospitalists based on their board certificationmost commonly internal medicine or family practice. Other approaches for more precise assessment utilized billing data or hospital designation. Saint et al. identified hospital‐based providers practicing in Washington State in 1994 using variable thresholds of billing for inpatient services.[2] In 2011, Welch et al. identified 25,787 hospitalists nationwide, using a 90% threshold of billing inpatient services in Medicare data.[6] That same year, an American Hospital Association survey identified 34,411 hospitalists based on self‐reporting.[4]

Building on the work of previous researchers, we applied an updated threshold of inpatient services in publicly available 2012 Medicare Provider Utilization and Payment Data to identify a range of hospitalists practicing in the United States. We also examine the codes billed by providers identified in different decile billing thresholds to assess the validity of using lower thresholds to identify hospitalists.

METHODS

RESULTS

The 2012 Medicare pay data included 664,253 physicians with unique NPIs. Of these, 169,317 had IM/FP specialty designations, whereas just under half (46.25%) of those physicians billed any of the inpatient HCPCS codes associated with our threshold.

Table 1 describes the range of number of hospitalists identified by varying the threshold of inpatient services. A total of 28,473 providers bill the threshold‐associated inpatient codes almost exclusively, whereas each descending decile increases in size by an average of 7.29%.

We also analyzed billing patterns of a subset of physicians who billed our threshold codes between 60% and 70% of the time to better characterize the remainder of clinical work they perform. This group included 2282 physicians and only 56 unique HCPCS codes with frequencies greater than 0.1%. After clustering related codes, we identified 4 common code groups that account for the majority of the remaining billing beyond inpatient threshold codes (Table 2).

DISCUSSION

Hospitalists make up approximately 5% of the practicing physicians nationwide, performing a critical role caring for hospitalized patients. Saint et al. defined a pure hospitalist as a physician who meets a 90% threshold of inpatient services.[2] This approach has been replicated in subsequent studies that used a 90% threshold to identify hospitalists.[5, 6] Our results with the same threshold reveal more than 28,000 hospitalists with nearly uniform practice patterns, a 10% growth in the number of hospitalists from the Welch et al. analysis in 2011.[6]

A threshold is not a perfect tool for identifying groups of practicing physicians, as it creates an arbitrary cutoff within a dataset. Undoubtedly our analysis could include providers who would not consider themselves hospitalists, or alternatively, appear to have a hospital‐based practice when they do not. Our results suggest that a 90% threshold may identify a majority of practicing hospitalists, but excludes providers who likely identify as hospitalists albeit with divergent practice and billing patterns.

A lower threshold may be more inclusive of the current realities of hospitalist practice, accounting for the myriad other services provided during, immediately prior to, or following a hospitalization. With hospitalists commonly practicing in diverse facility settings, rotating through rehabilitation or nursing home facilities, discharge clinics, and preoperative medicine practices, the continued use of a 90% threshold appears to exclude a sizable number of practicing hospitalists.

In the absence of a formal identifier, developing identification methodologies that account for the diversity of hospitalist practice is crucial. As physician payment transitions to value‐based reimbursement, systems must have the ability to account for and allocate the most efficient mix of providers for their patient populations. Because provider alignment and coordination are structural features of these programs, these systems‐based changes in effect require accurate identification of hospitalists, yet currently lack the tools to do so.

A seminal 1996 New England Journal of Medicine article introduced the term hospitalist to describe the emerging trend of primary care physicians practicing in inpatient hospital settings.[1] Although physicians had practice patterns akin to hospitalists prior to the introduction of the term,[2] the field continues to grow and formalize as a unique specialty in medicine.

There is currently no board certification or specialty billing code associated with hospitalists. In 2009, the American Board of Internal Medicine and American Board of Family Medicine introduced a Focused Practice in Hospital Medicine optional recertification pathway.[3] However, absent a unique identifier, it remains difficult to identify the number of hospitalists practicing today. Issues with identification notwithstanding, published data consistently suggest that the number of hospitalists has grown dramatically over the last 2 decades.[4, 5, 6]

The Centers for Medicare and Medicaid Services (CMS), along with other payers, classify hospitalists based on their board certificationmost commonly internal medicine or family practice. Other approaches for more precise assessment utilized billing data or hospital designation. Saint et al. identified hospital‐based providers practicing in Washington State in 1994 using variable thresholds of billing for inpatient services.[2] In 2011, Welch et al. identified 25,787 hospitalists nationwide, using a 90% threshold of billing inpatient services in Medicare data.[6] That same year, an American Hospital Association survey identified 34,411 hospitalists based on self‐reporting.[4]

Building on the work of previous researchers, we applied an updated threshold of inpatient services in publicly available 2012 Medicare Provider Utilization and Payment Data to identify a range of hospitalists practicing in the United States. We also examine the codes billed by providers identified in different decile billing thresholds to assess the validity of using lower thresholds to identify hospitalists.

METHODS

RESULTS

The 2012 Medicare pay data included 664,253 physicians with unique NPIs. Of these, 169,317 had IM/FP specialty designations, whereas just under half (46.25%) of those physicians billed any of the inpatient HCPCS codes associated with our threshold.

Table 1 describes the range of number of hospitalists identified by varying the threshold of inpatient services. A total of 28,473 providers bill the threshold‐associated inpatient codes almost exclusively, whereas each descending decile increases in size by an average of 7.29%.

We also analyzed billing patterns of a subset of physicians who billed our threshold codes between 60% and 70% of the time to better characterize the remainder of clinical work they perform. This group included 2282 physicians and only 56 unique HCPCS codes with frequencies greater than 0.1%. After clustering related codes, we identified 4 common code groups that account for the majority of the remaining billing beyond inpatient threshold codes (Table 2).

DISCUSSION

Hospitalists make up approximately 5% of the practicing physicians nationwide, performing a critical role caring for hospitalized patients. Saint et al. defined a pure hospitalist as a physician who meets a 90% threshold of inpatient services.[2] This approach has been replicated in subsequent studies that used a 90% threshold to identify hospitalists.[5, 6] Our results with the same threshold reveal more than 28,000 hospitalists with nearly uniform practice patterns, a 10% growth in the number of hospitalists from the Welch et al. analysis in 2011.[6]

A threshold is not a perfect tool for identifying groups of practicing physicians, as it creates an arbitrary cutoff within a dataset. Undoubtedly our analysis could include providers who would not consider themselves hospitalists, or alternatively, appear to have a hospital‐based practice when they do not. Our results suggest that a 90% threshold may identify a majority of practicing hospitalists, but excludes providers who likely identify as hospitalists albeit with divergent practice and billing patterns.

A lower threshold may be more inclusive of the current realities of hospitalist practice, accounting for the myriad other services provided during, immediately prior to, or following a hospitalization. With hospitalists commonly practicing in diverse facility settings, rotating through rehabilitation or nursing home facilities, discharge clinics, and preoperative medicine practices, the continued use of a 90% threshold appears to exclude a sizable number of practicing hospitalists.

In the absence of a formal identifier, developing identification methodologies that account for the diversity of hospitalist practice is crucial. As physician payment transitions to value‐based reimbursement, systems must have the ability to account for and allocate the most efficient mix of providers for their patient populations. Because provider alignment and coordination are structural features of these programs, these systems‐based changes in effect require accurate identification of hospitalists, yet currently lack the tools to do so.

Patient outcomes tend to be worse for adults admitted on the weekend compared to the weekday.[1, 2, 3, 4] In pediatric populations, urgent surgeries on weekends are associated with increased morbidity and mortality[5]; however, studies of mortality and admission timing in the pediatric critical care setting are mixed.[6, 7] Hospital readmission is considered a potential marker of hospital quality. We hypothesized that (1) being admitted and (2) being discharged on the weekend would adversely affect 30‐day unplanned readmission for pediatric patients.

METHODS

RESULTS

We included a total of 55,383 hospitalizations from 32,112 patients (see Supporting Appendix Figure in the online version of this article for cohort derivation). All‐cause 30‐day readmissions occurred in 14.9% of hospital discharges; the 30‐day unplanned readmission rate was 10.3% (see the Supporting Appendix Table in the online version of this article for demographic characteristics).

Weekend Admission

Overall, 82% of admissions occurred during the week, with Tuesday as the highest admitting volume day (Figure 1). Children admitted on the weekend had higher odds of unplanned readmission compared to children admitted on weekdays (unadjusted odds ratio [OR]=1.15 [95% confidence interval {CI}: 1.07‐1.24]). Adjusting the analysis for age, gender, race/ethnicity, insurance, length of stay, CCCs, and technology dependency, higher odds of readmission remains significantly higher than weekday admission (adjusted OR=1.09 [95% CI: 1.004‐1.18]) (Table 1). Age, admission source, payer, length of stay, number of complex chronic conditions, and technology dependency were also significantly associated with readmission in the weekend admission model (see the Supporting Appendix Table in the online version of this article). A sensitivity analysis examining the association of weekend admission and readmission within different subpopulations of children with varying numbers of CCCs (ie, among children without CCCs, with 1 CCC, 2 CCCs, and 3+ CCCs) showed that the association remains the same in each subgroup. Further, a sensitivity analysis examining odds of any 30‐day readmission was similar to the primary analysis with higher odds of readmission in adjusted analysis (adjusted OR=1.09 [95% CI: 1.02‐1.18]).

Figure 1

Table 1.Patient Characteristics During Hospitalizations

	30‐Day Unplanned Readmission Rate	Unadjusted Odds of Unplanned Readmission (95% CI)	Weekend Admission Model: Adjusted Odds of Unplanned Readmission (95% CI)	Weekend Discharge Model: Adjusted Odds of Unplanned Readmission (95% CI)
NOTE: Abbreviations: CI, confidence interval. *P<0.05. Model adjusted for age category, gender, admission source, race/ethnicity, primary payer type, length of stay, number of complex chronic conditions, and technology dependency.
Weekend admission, n=7,533	11.4%, n=973	1.15 (1.07‐1.24)*	1.09 (1.004‐1.18)*
Weekend discharge, n=13,911	9.7%, n=1,344	0.91 (0.85‐0.97)*		0.97 (0.91‐1.04)

DISCUSSION

Although the so‐called weekend effect has been established in adults,[1, 2, 3, 4] evidence is mixed for children. In this sample, where the 30‐day pediatric readmission rate is consistent with national pediatric rates,[10] pediatric patients admitted on the weekend have higher odds of readmission compared to children admitted during the week, even when accounting for patient characteristics and hospital length of stay. In contrast, weekend discharge was not associated with readmission.

The association of weekend admission and subsequent readmission is intriguing and may be interpreted in 1 of 2 ways: either patients admitted on the weekend are fundamentally different and thus have higher readmission rates, or care on the weekend is different. It is important to note that we adjusted the analysis for patient characteristics including number of CCCs and technology dependency to account for differences in chronic illness. We also accounted for length of stay as a marker of severity of illness in the hospital. Yet even accounting for these known differences, we cannot discern from these data if the different outcomes for children admitted on the weekend are related to residual population differences (eg, lack of access to primary care or walk‐in clinics) or differences in initial clinical management on the weekend.

Initial clinical management on weekend may be different due to differences in physician, nursing, and other ancillary staffing affecting availability of diagnostic and therapeutic interventions. Additionally, smaller staff size on the weekend may lead to increased workload. Although we are unable to directly measure resident workload in our study, prior studies suggest higher workload is associated with worse outcomes for adult patients,[11] including readmission.[12] Additionally, nurse staffing, which may vary based on day of week, has been associated with pediatric readmission.[13]

Discharge timing in our population is consistent with prior literature, with Friday being the most common discharge day of week.[14] Prior literature has shown no difference in readmission rates between Friday discharge and midweek discharge for pediatric patients.[14] Our work builds on this existing literature, demonstrating no association with weekend discharge and readmission. There were lower discharge volumes on the weekends, particularly in patients with more CCCs, suggesting that physicians avoid complicated discharges on Saturday and Sunday.

This study should be interpreted in the context of several limitations. First, this study was conducted at a single tertiary care pediatric institution. Our patient population had a high rate of children with CCCs, potentially limiting generalizability to other pediatric institutions. Ideally, we would adjust our model for clusters at the clinical service or attending physician level; however, the heterogeneity of our services and data limits prohibited these analyses. Readmissions that may have occurred at other institutions are not observable in this dataset; however, there is no reason to believe patients admitted or discharged on the weekend would have different rates of other hospital readmissions than patients admitted or discharged on weekdays. Additionally, early readmissions may be particularly affected by in‐hospital and discharge factors.[15] However, the very low rate of early readmission prohibited limiting the analyses to early readmission. Finally, we relied on administrative data to adjust for patient severity using typical methods such as CCCs; however, other patient differences may have existed beyond those that could be captured with administrative data.

CONCLUSION

Children admitted to the hospital on the weekend have higher rates of 30‐day unplanned readmission than children admitted during the week, suggesting differences of care in initial management on the weekend. Understanding this difference from the perspectives of multiple stakeholders may illuminate potential reasons for this disparity.

Patient outcomes tend to be worse for adults admitted on the weekend compared to the weekday.[1, 2, 3, 4] In pediatric populations, urgent surgeries on weekends are associated with increased morbidity and mortality[5]; however, studies of mortality and admission timing in the pediatric critical care setting are mixed.[6, 7] Hospital readmission is considered a potential marker of hospital quality. We hypothesized that (1) being admitted and (2) being discharged on the weekend would adversely affect 30‐day unplanned readmission for pediatric patients.

METHODS

RESULTS

We included a total of 55,383 hospitalizations from 32,112 patients (see Supporting Appendix Figure in the online version of this article for cohort derivation). All‐cause 30‐day readmissions occurred in 14.9% of hospital discharges; the 30‐day unplanned readmission rate was 10.3% (see the Supporting Appendix Table in the online version of this article for demographic characteristics).

Weekend Admission

Overall, 82% of admissions occurred during the week, with Tuesday as the highest admitting volume day (Figure 1). Children admitted on the weekend had higher odds of unplanned readmission compared to children admitted on weekdays (unadjusted odds ratio [OR]=1.15 [95% confidence interval {CI}: 1.07‐1.24]). Adjusting the analysis for age, gender, race/ethnicity, insurance, length of stay, CCCs, and technology dependency, higher odds of readmission remains significantly higher than weekday admission (adjusted OR=1.09 [95% CI: 1.004‐1.18]) (Table 1). Age, admission source, payer, length of stay, number of complex chronic conditions, and technology dependency were also significantly associated with readmission in the weekend admission model (see the Supporting Appendix Table in the online version of this article). A sensitivity analysis examining the association of weekend admission and readmission within different subpopulations of children with varying numbers of CCCs (ie, among children without CCCs, with 1 CCC, 2 CCCs, and 3+ CCCs) showed that the association remains the same in each subgroup. Further, a sensitivity analysis examining odds of any 30‐day readmission was similar to the primary analysis with higher odds of readmission in adjusted analysis (adjusted OR=1.09 [95% CI: 1.02‐1.18]).

Figure 1

Table 1.Patient Characteristics During Hospitalizations

	30‐Day Unplanned Readmission Rate	Unadjusted Odds of Unplanned Readmission (95% CI)	Weekend Admission Model: Adjusted Odds of Unplanned Readmission (95% CI)	Weekend Discharge Model: Adjusted Odds of Unplanned Readmission (95% CI)
NOTE: Abbreviations: CI, confidence interval. *P<0.05. Model adjusted for age category, gender, admission source, race/ethnicity, primary payer type, length of stay, number of complex chronic conditions, and technology dependency.
Weekend admission, n=7,533	11.4%, n=973	1.15 (1.07‐1.24)*	1.09 (1.004‐1.18)*
Weekend discharge, n=13,911	9.7%, n=1,344	0.91 (0.85‐0.97)*		0.97 (0.91‐1.04)

DISCUSSION

Although the so‐called weekend effect has been established in adults,[1, 2, 3, 4] evidence is mixed for children. In this sample, where the 30‐day pediatric readmission rate is consistent with national pediatric rates,[10] pediatric patients admitted on the weekend have higher odds of readmission compared to children admitted during the week, even when accounting for patient characteristics and hospital length of stay. In contrast, weekend discharge was not associated with readmission.

The association of weekend admission and subsequent readmission is intriguing and may be interpreted in 1 of 2 ways: either patients admitted on the weekend are fundamentally different and thus have higher readmission rates, or care on the weekend is different. It is important to note that we adjusted the analysis for patient characteristics including number of CCCs and technology dependency to account for differences in chronic illness. We also accounted for length of stay as a marker of severity of illness in the hospital. Yet even accounting for these known differences, we cannot discern from these data if the different outcomes for children admitted on the weekend are related to residual population differences (eg, lack of access to primary care or walk‐in clinics) or differences in initial clinical management on the weekend.

Initial clinical management on weekend may be different due to differences in physician, nursing, and other ancillary staffing affecting availability of diagnostic and therapeutic interventions. Additionally, smaller staff size on the weekend may lead to increased workload. Although we are unable to directly measure resident workload in our study, prior studies suggest higher workload is associated with worse outcomes for adult patients,[11] including readmission.[12] Additionally, nurse staffing, which may vary based on day of week, has been associated with pediatric readmission.[13]

Discharge timing in our population is consistent with prior literature, with Friday being the most common discharge day of week.[14] Prior literature has shown no difference in readmission rates between Friday discharge and midweek discharge for pediatric patients.[14] Our work builds on this existing literature, demonstrating no association with weekend discharge and readmission. There were lower discharge volumes on the weekends, particularly in patients with more CCCs, suggesting that physicians avoid complicated discharges on Saturday and Sunday.

This study should be interpreted in the context of several limitations. First, this study was conducted at a single tertiary care pediatric institution. Our patient population had a high rate of children with CCCs, potentially limiting generalizability to other pediatric institutions. Ideally, we would adjust our model for clusters at the clinical service or attending physician level; however, the heterogeneity of our services and data limits prohibited these analyses. Readmissions that may have occurred at other institutions are not observable in this dataset; however, there is no reason to believe patients admitted or discharged on the weekend would have different rates of other hospital readmissions than patients admitted or discharged on weekdays. Additionally, early readmissions may be particularly affected by in‐hospital and discharge factors.[15] However, the very low rate of early readmission prohibited limiting the analyses to early readmission. Finally, we relied on administrative data to adjust for patient severity using typical methods such as CCCs; however, other patient differences may have existed beyond those that could be captured with administrative data.

CONCLUSION

Children admitted to the hospital on the weekend have higher rates of 30‐day unplanned readmission than children admitted during the week, suggesting differences of care in initial management on the weekend. Understanding this difference from the perspectives of multiple stakeholders may illuminate potential reasons for this disparity.

Patient outcomes tend to be worse for adults admitted on the weekend compared to the weekday.[1, 2, 3, 4] In pediatric populations, urgent surgeries on weekends are associated with increased morbidity and mortality[5]; however, studies of mortality and admission timing in the pediatric critical care setting are mixed.[6, 7] Hospital readmission is considered a potential marker of hospital quality. We hypothesized that (1) being admitted and (2) being discharged on the weekend would adversely affect 30‐day unplanned readmission for pediatric patients.

METHODS

RESULTS

We included a total of 55,383 hospitalizations from 32,112 patients (see Supporting Appendix Figure in the online version of this article for cohort derivation). All‐cause 30‐day readmissions occurred in 14.9% of hospital discharges; the 30‐day unplanned readmission rate was 10.3% (see the Supporting Appendix Table in the online version of this article for demographic characteristics).

Weekend Admission

Overall, 82% of admissions occurred during the week, with Tuesday as the highest admitting volume day (Figure 1). Children admitted on the weekend had higher odds of unplanned readmission compared to children admitted on weekdays (unadjusted odds ratio [OR]=1.15 [95% confidence interval {CI}: 1.07‐1.24]). Adjusting the analysis for age, gender, race/ethnicity, insurance, length of stay, CCCs, and technology dependency, higher odds of readmission remains significantly higher than weekday admission (adjusted OR=1.09 [95% CI: 1.004‐1.18]) (Table 1). Age, admission source, payer, length of stay, number of complex chronic conditions, and technology dependency were also significantly associated with readmission in the weekend admission model (see the Supporting Appendix Table in the online version of this article). A sensitivity analysis examining the association of weekend admission and readmission within different subpopulations of children with varying numbers of CCCs (ie, among children without CCCs, with 1 CCC, 2 CCCs, and 3+ CCCs) showed that the association remains the same in each subgroup. Further, a sensitivity analysis examining odds of any 30‐day readmission was similar to the primary analysis with higher odds of readmission in adjusted analysis (adjusted OR=1.09 [95% CI: 1.02‐1.18]).

Figure 1

Table 1.Patient Characteristics During Hospitalizations

	30‐Day Unplanned Readmission Rate	Unadjusted Odds of Unplanned Readmission (95% CI)	Weekend Admission Model: Adjusted Odds of Unplanned Readmission (95% CI)	Weekend Discharge Model: Adjusted Odds of Unplanned Readmission (95% CI)
NOTE: Abbreviations: CI, confidence interval. *P<0.05. Model adjusted for age category, gender, admission source, race/ethnicity, primary payer type, length of stay, number of complex chronic conditions, and technology dependency.
Weekend admission, n=7,533	11.4%, n=973	1.15 (1.07‐1.24)*	1.09 (1.004‐1.18)*
Weekend discharge, n=13,911	9.7%, n=1,344	0.91 (0.85‐0.97)*		0.97 (0.91‐1.04)

DISCUSSION

Although the so‐called weekend effect has been established in adults,[1, 2, 3, 4] evidence is mixed for children. In this sample, where the 30‐day pediatric readmission rate is consistent with national pediatric rates,[10] pediatric patients admitted on the weekend have higher odds of readmission compared to children admitted during the week, even when accounting for patient characteristics and hospital length of stay. In contrast, weekend discharge was not associated with readmission.

The association of weekend admission and subsequent readmission is intriguing and may be interpreted in 1 of 2 ways: either patients admitted on the weekend are fundamentally different and thus have higher readmission rates, or care on the weekend is different. It is important to note that we adjusted the analysis for patient characteristics including number of CCCs and technology dependency to account for differences in chronic illness. We also accounted for length of stay as a marker of severity of illness in the hospital. Yet even accounting for these known differences, we cannot discern from these data if the different outcomes for children admitted on the weekend are related to residual population differences (eg, lack of access to primary care or walk‐in clinics) or differences in initial clinical management on the weekend.

Initial clinical management on weekend may be different due to differences in physician, nursing, and other ancillary staffing affecting availability of diagnostic and therapeutic interventions. Additionally, smaller staff size on the weekend may lead to increased workload. Although we are unable to directly measure resident workload in our study, prior studies suggest higher workload is associated with worse outcomes for adult patients,[11] including readmission.[12] Additionally, nurse staffing, which may vary based on day of week, has been associated with pediatric readmission.[13]

Discharge timing in our population is consistent with prior literature, with Friday being the most common discharge day of week.[14] Prior literature has shown no difference in readmission rates between Friday discharge and midweek discharge for pediatric patients.[14] Our work builds on this existing literature, demonstrating no association with weekend discharge and readmission. There were lower discharge volumes on the weekends, particularly in patients with more CCCs, suggesting that physicians avoid complicated discharges on Saturday and Sunday.

This study should be interpreted in the context of several limitations. First, this study was conducted at a single tertiary care pediatric institution. Our patient population had a high rate of children with CCCs, potentially limiting generalizability to other pediatric institutions. Ideally, we would adjust our model for clusters at the clinical service or attending physician level; however, the heterogeneity of our services and data limits prohibited these analyses. Readmissions that may have occurred at other institutions are not observable in this dataset; however, there is no reason to believe patients admitted or discharged on the weekend would have different rates of other hospital readmissions than patients admitted or discharged on weekdays. Additionally, early readmissions may be particularly affected by in‐hospital and discharge factors.[15] However, the very low rate of early readmission prohibited limiting the analyses to early readmission. Finally, we relied on administrative data to adjust for patient severity using typical methods such as CCCs; however, other patient differences may have existed beyond those that could be captured with administrative data.

CONCLUSION

Children admitted to the hospital on the weekend have higher rates of 30‐day unplanned readmission than children admitted during the week, suggesting differences of care in initial management on the weekend. Understanding this difference from the perspectives of multiple stakeholders may illuminate potential reasons for this disparity.

Anticoagulants are among the prescriptions with the highest risk of an adverse drug event (ADE) after discharge, and many of these events are preventable.[1, 2, 3] In recognition of the high risk for adverse events, the Institute for Healthcare Improvement Map details several key features of a safe anticoagulation management program, including the recommendation during the transition period that clinicians ensure proper lab monitoring and establish capacity for follow‐up testing.[4]

Despite the potential for harm, most hospitals do not have a structured process for the transmission of vital information related to warfarin management from the inpatient to the ambulatory setting. Our aim was to develop a concise report containing the essential information regarding the patient's anticoagulation regimen, the Safe Transitions Anticoagulation Report (STAR), and a process to ensure the report can be readily accessed and utilized by ambulatory clinicians.

METHODS

We assembled an interdisciplinary team to develop a new report and workflow to ensure that information on inpatient warfarin management was transmitted to outpatient providers in a reliable and structured manner. Explicit goals were to maximize use of the electronic medical record (EMR) to autopopulate aspects of the new report and create a process that worked seamlessly into the workflow. The final items were selected based on the risk of harm if not conveyed and feasibility of incorporation through the EMR:

• Warfarin doses: the 7 warfarin doses immediately prior to discharge
• International normalized ratio (INR) values: the 7 INR values immediately prior to discharge
• Bridging anticoagulation: the low‐molecular‐weight heparin (LMWH) prescribed as a bridging anticoagulant, if any

The STAR resides in both the discharge summary and the after visit summary (AVS) for patients discharged on warfarin. At our institution, the AVS contains a medication list, discharge instructions, and appointments, and is automatically produced through our EMR. Our institution utilizes the Epic EMR (Epic Systems Corp., Verona, WI) in the hospital, ambulatory clinics, and faculty practices.

A process was developed where a structured table is automatically created (Figure 1). A field was added to our EMR's discharge summary template asking whether the patient is being discharged on warfarin. Answering yes produces a second question asking whether the patient is also being discharged on bridging anticoagulation with LMWH. The STAR is not inserted into the discharge summary if the clinician completing the discharge summary deletes the anticoagulation question. The STAR is automatically created for patients discharged on warfarin and inserted into the AVS by the EMR regardless of whether the discharge summary has been completed. Patients are instructed by their nurse to bring their AVS to their follow‐up appointments.

Figure 1

The STAR project team utilized plan‐do‐study‐act cycles to test small changes and make revisions. The workflow was piloted on 2 medical/surgical units from January 2014 through March 2014, and revised based on feedback from clinicians and nursing staff. The STAR initiative was fully implemented across our institution in April 2014.

The study was evaluated by the institutional review board of the Icahn School of Medicine at Mount Sinai, and full review was waived.

RESULTS

The STAR was embedded in the discharge summary for 1370 (78.6%) discharges during the intervention period. A total of 505 patients in the preintervention period and 292 patients in the intervention period were established patients at an affiliated practice and comprised the study population. Demographics and indications for warfarin for the preintervention and intervention groups are listed in the Table 1.

The frequency of INR testing within 10 days of discharge was similar for the preintervention and intervention periods (41.4% and 47.6%, respectively, P=0.09). Similarly, the likelihood of having at least 1 therapeutic INR value within 10 days of discharge was not statistically different for the groups (17.0% and 21.2%, P=0.14). The pattern was similar for the 3‐, 7‐, and 30‐day periods; a higher percentage of the intervention group had INR testing and attained a therapeutic INR value, though for no time period did this reach statistical significance. This pattern was also found when limiting the analysis to patients discharged home rather than to a facility, patients on warfarin, prior to admission, and patients started on warfarin during the hospitalization.

A total of 87of 207 (42.0%) clinicians responded to the survey. Of respondents, 75% reported that they had seen 1 patient who had been discharged on warfarin since the STAR initiative had begun, 58% of whom reported having viewed the STAR. Most respondents who viewed the STAR found it to be helpful or very helpful in guiding warfarin management (67%), improving their workflow and efficiency (58%), and improving patient safety (77%). Approximately one‐third of respondents who had viewed the STAR (34%) reported that they selected a different dose than they would have chosen had the STAR not been available.

DISCUSSION

We developed a concise report that is seamlessly created and inserted into the discharge summary. Though the STAR was perceived as improving patient safety by ambulatory care providers, there was no impact on attaining a therapeutic INR after discharge. There are several possible explanations for a lack of benefit. Most notably, our intervention was comprised of a stand‐alone EMR‐based tool and focused on 1 component of the transitions process. Given the complexity of healthcare delivery and anticoagulant management, it is likely that broader interventions are required to improve clinical outcomes over the transition period. Potential targets of multifaceted approaches may include improving access to care, providing greater access to anticoagulation clinics, enhancing patient education, and promoting direct physician‐physician communication. Bundled interventions will likely need to include involvement of an interdisciplinary team, such as pharmacist involvement in the medication reconciliation process.

The transition period from the hospital to the outpatient setting has the potential to jeopardize patient safety if vital information is not reliably transmitted across venues.[5, 6, 7, 8] Forster and colleagues noted an 11% incidence of ADEs in the posthospitalization period, of which 60% were either preventable or ameliorable.[3] To decrease the risk to patient safety during the transitions period, the Transitions of Care Consensus policy statement by the Society of Hospital Medicine and other medical organizations called for incorporation of standard data transfer forms (templates and transmission protocols).[9] Despite the high risk and preventable nature of many of the events, few specific tools have been developed. As part of broader initiatives to improve the transitions process, the STAR may have the potential to be a means for health systems to improve the quality of the transition of care for patients on anticoagulants.

Our study has several limitations. First, it was performed at a single health system. It is unknown whether the EMR‐based report could be similarly employed at other systems. Second, our study was unable to assess clinical endpoints. Given the lack of effect on attaining a therapeutic INR, it is unlikely that downstream outcomes, such as thromboembolism, were impacted. Lastly, we were unable to examine whether our intervention improved the care of patients whose outpatient provider was external to our system.

The STAR is a concise tool developed to provide essential anticoagulant‐related information to ambulatory providers. Though the report was perceived as improving patient safety, our finding of a lack of impact on attaining a therapeutic INR after discharge suggests that the tool would need to be a component of a broader multifaceted intervention to impact clinical outcomes.

Anticoagulants are among the prescriptions with the highest risk of an adverse drug event (ADE) after discharge, and many of these events are preventable.[1, 2, 3] In recognition of the high risk for adverse events, the Institute for Healthcare Improvement Map details several key features of a safe anticoagulation management program, including the recommendation during the transition period that clinicians ensure proper lab monitoring and establish capacity for follow‐up testing.[4]

Despite the potential for harm, most hospitals do not have a structured process for the transmission of vital information related to warfarin management from the inpatient to the ambulatory setting. Our aim was to develop a concise report containing the essential information regarding the patient's anticoagulation regimen, the Safe Transitions Anticoagulation Report (STAR), and a process to ensure the report can be readily accessed and utilized by ambulatory clinicians.

METHODS

We assembled an interdisciplinary team to develop a new report and workflow to ensure that information on inpatient warfarin management was transmitted to outpatient providers in a reliable and structured manner. Explicit goals were to maximize use of the electronic medical record (EMR) to autopopulate aspects of the new report and create a process that worked seamlessly into the workflow. The final items were selected based on the risk of harm if not conveyed and feasibility of incorporation through the EMR:

• Warfarin doses: the 7 warfarin doses immediately prior to discharge
• International normalized ratio (INR) values: the 7 INR values immediately prior to discharge
• Bridging anticoagulation: the low‐molecular‐weight heparin (LMWH) prescribed as a bridging anticoagulant, if any

The STAR resides in both the discharge summary and the after visit summary (AVS) for patients discharged on warfarin. At our institution, the AVS contains a medication list, discharge instructions, and appointments, and is automatically produced through our EMR. Our institution utilizes the Epic EMR (Epic Systems Corp., Verona, WI) in the hospital, ambulatory clinics, and faculty practices.

A process was developed where a structured table is automatically created (Figure 1). A field was added to our EMR's discharge summary template asking whether the patient is being discharged on warfarin. Answering yes produces a second question asking whether the patient is also being discharged on bridging anticoagulation with LMWH. The STAR is not inserted into the discharge summary if the clinician completing the discharge summary deletes the anticoagulation question. The STAR is automatically created for patients discharged on warfarin and inserted into the AVS by the EMR regardless of whether the discharge summary has been completed. Patients are instructed by their nurse to bring their AVS to their follow‐up appointments.

Figure 1

The STAR project team utilized plan‐do‐study‐act cycles to test small changes and make revisions. The workflow was piloted on 2 medical/surgical units from January 2014 through March 2014, and revised based on feedback from clinicians and nursing staff. The STAR initiative was fully implemented across our institution in April 2014.

The study was evaluated by the institutional review board of the Icahn School of Medicine at Mount Sinai, and full review was waived.