Alan J. Forster, MD, MSc, FRCPC

Study Setting

TOH is a tertiary‐care teaching hospital with two inpatient campuses. The hospital operates within a publicly funded health care system, serves a population of approximately 1.5 million people in Ottawa and Eastern Ontario, treats all major trauma patients for the region, and provides most of the oncological care in the region.

TOH is the primary medical teaching hospital at the University of Ottawa. In 2010, there were 197 residents starting their first year of postgraduate training in one of 29 programs.

Inclusion Criteria

The study period extended from April 15, 2004 to December 31, 2008. We used this start time because our hospital switched to new coding systems for procedures and diagnoses in April 2002. Since these new coding systems contributed to our outcome statistic, we used a very long period (ie, two years) for coding patterns to stabilize to ensure that any changes seen were not a function of coding patterns. We ended our study in December 2008 because this was the last date of complete data at the time we started the analysis.

We included all medical, surgical, and obstetrical patients admitted to TOH during this time except those who were: younger than 15 years old; transferred to or from another acute care hospital; or obstetrical patients hospitalized for routine childbirth. These patients were excluded because they were not part of the multivariate model that we used to calculate risk of death in hospital (discussed below).24 These exclusions accounted for 25.4% of all admissions during the study period (36,820less than 15 years old; 12,931transferred to or from the hospital; and 44,220uncomplicated admission for childbirth).

All data used in this study came from The Ottawa Hospital Data Warehouse (TOHDW). This is a repository of clinical, laboratory, and administrative data originating from the hospital's major operational information systems. TOHDW contains information on patient demographics and diagnoses, as well as procedures and patient transfers between different units or hospital services during the admission.

Primary OutcomeRatio of Observed to Expected Number of Deaths per Week

For each study day, we measured the number of hospital deaths from the patient registration table in TOHDW. This statistic was collated for each week to ensure numeric stability, especially in our subgroup analyses.

We calculated the weekly expected number of hospital deaths using an extension of the Escobar model.24 The Escobar is a logistic regression model that estimated the probability of death in hospital that was derived and internally validated on almost 260,000 hospitalizations at 17 hospitals in the Kaiser Permanente Health Plan. It included six covariates that were measurable at admission including: patient age; patient sex; admission urgency (ie, elective or emergent) and service (ie, medical or surgical); admission diagnosis; severity of acute illness as measured by the Laboratory‐based Acute Physiology Score (LAPS); and chronic comorbidities as measured by the COmorbidity Point Score (COPS). Hospitalizations were grouped by admission diagnosis. The final model had excellent discrimination (c‐statistic 0.88) and calibration (P value of Hosmer Lemeshow statistic for entire cohort 0.66). This model was externally validated in our center with a c‐statistic of 0.901.25

We extended the Escobar model in several ways (Wong et al., Derivation and validation of a model to predict the daily risk of death in hospital, 2010, unpublished work). First, we modified it into a survival (rather than a logistic) model so it could estimate a daily probability of death in hospital. Second, we included the same covariates as Escobar except that we expressed LAPS as a time‐dependent covariate (meaning that the model accounted for changes in its value during the hospitalization). Finally, we included other time‐dependent covariates including: admission to intensive care unit; undergoing significant procedures; and awaiting long‐term care. This model had excellent discrimination (concordance probability of 0.895, 95% confidence interval [CI] 0.8890.902) and calibration.

We used this survival model to estimate the daily risk of death for all patients in the hospital each day. Summing these risks over hospital patients on each day returned the daily number of expected hospital deaths. This was collated per week.

The outcome statistic for this study was the ratio of the observed to expected weekly number of hospital deaths. Ratios exceeding 1 indicate that more deaths were observed than were expected (given the distribution of important covariates in those people during that week). This outcome statistic has several advantages. First, it accounts for the number of patients in the hospital each day. This is important because the number of hospital deaths will increase as the number of people in hospital increase. Second, it accounts for the severity of illness in each patient on each hospital day. This accounts for daily changes in risk of patient death, because calculation of the expected number of deaths per day was done using a multivariate survival model that included time‐dependent covariates. Therefore, each individual's predicted hazard of death (which was summed over the entire hospital to calculate the total expected number of deaths in hospital each day) took into account the latest values of these covariates. Previous analyses only accounted for risk of death at admission.

Expressing Physician Experience

The latent measure26 in all July Phenomenon studies is collective house‐staff physician experience. This is quantified by a surrogate date variable in which July 1the date that new house‐staff start their training in North Americarepresents minimal experience and June 30 represents maximal experience. We expressed collective physician experience on a scale from 0 (minimum experience) on July 1 to 1 (maximum experience) on June 30. A similar approach has been used previously13 and has advantages over the other methods used to capture collective house‐staff experience. In the stratified, incomplete approach,47, 911, 13, 1517 periods with inexperienced house‐staff (eg, July and August) are grouped together and compared to times with experienced house‐staff (eg, May and June), while ignoring all other data. The specification of cut‐points for this stratification is arbitrary and the method ignores large amounts of data. In the stratified, complete approach, periods with inexperienced house‐staff (eg, July and August) are grouped together and compared to all other times of the year.8, 12, 14, 1820, 22 This is potentially less biased because there are no lost data. However, the cut‐point for determining when house‐staff transition from inexperienced to experienced is arbitrary, and the model assumes that the transition is sudden. This is suboptimal because acquisition of experience is a gradual, constant process.

The pattern by which collective physician experience changes between July 1st and June 30th is unknown. We therefore expressed this evolution using five different patterns varying from a linear change to a natural logarithmic change (see Supporting Appendix B in the online version of this article).

Analysis

We first examined for autocorrelation in our outcome variable using Ljung‐Box statistics at lag 6 and 12 in PROC ARIMA (SAS 9.2, Cary, NC). If significant autocorrelation was absent in our data, linear regression modeling was used to associate the ratio of the observed to expected number of weekly deaths (the outcome variable) with the collective first year physician experience (the predictor variable). Time‐series methodology was to be used if significant autocorrelation was present.

In our baseline analysis, we included all hospitalizations together. In stratified analyses, we categorized hospitalizations by admission status (emergent vs elective) and admission service (medicine vs surgery).

Weekly Deaths: Observed, Expected, and Ratio

Figure 1A presents the observed weekly number of deaths during the study period. There was an average of 31 deaths per week (range 1551). Some large fluctuations in the weekly number of deaths were seen; in 2007, for example, the number of observed deaths went from 21 in week 13 up to 46 in week 15. However, no obvious seasonal trends in the observed weekly number of deaths were seen (Figure 1A, heavy line) nor were trends between years obvious.

Figure 1

Figure 1B presents the expected weekly number of deaths during the study period. The expected weekly number of deaths averaged 29.6 (range 22.238.7). The expected weekly number of deaths was notably less variable than the observed number of deaths. However, important variations in the expected number of deaths were seen; for example, in 2005, the expected number of deaths increased from 24.1 in week 41 to 29.6 in week 44. Again, we saw no obvious seasonal trends in the expected weekly number of deaths (Figure 1B, heavy line).

Figure 1C illustrates the ratio of observed to the expected weekly number of deaths. The average observed to expected ratio slightly exceeded unity (1.05) and ranged from 0.488 (week 24, in 2008) to 1.821 (week 51, in 2008). We saw no obvious seasonal trends in the ratio of the observed to expected number of weekly deaths. In addition, obvious trends in this ratio were absent over the study period.

Association Between House‐Staff Experience and Death in Hospital

We found no evidence of autocorrelation in the ratio of observed to expected weekly number of deaths. The ratio of observed to expected number of hospital deaths was not significantly associated with house‐staff physician experience (Table 2). This conclusion did not change regardless of which house‐staff physician experience pattern was used in the linear model (Table 2). In addition, our analysis found no significant association between physician experience and patient mortality when analyses were stratified by admission service or admission status (Table 2).

Study Setting

TOH is a tertiary‐care teaching hospital with two inpatient campuses. The hospital operates within a publicly funded health care system, serves a population of approximately 1.5 million people in Ottawa and Eastern Ontario, treats all major trauma patients for the region, and provides most of the oncological care in the region.

TOH is the primary medical teaching hospital at the University of Ottawa. In 2010, there were 197 residents starting their first year of postgraduate training in one of 29 programs.

Inclusion Criteria

The study period extended from April 15, 2004 to December 31, 2008. We used this start time because our hospital switched to new coding systems for procedures and diagnoses in April 2002. Since these new coding systems contributed to our outcome statistic, we used a very long period (ie, two years) for coding patterns to stabilize to ensure that any changes seen were not a function of coding patterns. We ended our study in December 2008 because this was the last date of complete data at the time we started the analysis.

We included all medical, surgical, and obstetrical patients admitted to TOH during this time except those who were: younger than 15 years old; transferred to or from another acute care hospital; or obstetrical patients hospitalized for routine childbirth. These patients were excluded because they were not part of the multivariate model that we used to calculate risk of death in hospital (discussed below).24 These exclusions accounted for 25.4% of all admissions during the study period (36,820less than 15 years old; 12,931transferred to or from the hospital; and 44,220uncomplicated admission for childbirth).

All data used in this study came from The Ottawa Hospital Data Warehouse (TOHDW). This is a repository of clinical, laboratory, and administrative data originating from the hospital's major operational information systems. TOHDW contains information on patient demographics and diagnoses, as well as procedures and patient transfers between different units or hospital services during the admission.

Primary OutcomeRatio of Observed to Expected Number of Deaths per Week

For each study day, we measured the number of hospital deaths from the patient registration table in TOHDW. This statistic was collated for each week to ensure numeric stability, especially in our subgroup analyses.

We calculated the weekly expected number of hospital deaths using an extension of the Escobar model.24 The Escobar is a logistic regression model that estimated the probability of death in hospital that was derived and internally validated on almost 260,000 hospitalizations at 17 hospitals in the Kaiser Permanente Health Plan. It included six covariates that were measurable at admission including: patient age; patient sex; admission urgency (ie, elective or emergent) and service (ie, medical or surgical); admission diagnosis; severity of acute illness as measured by the Laboratory‐based Acute Physiology Score (LAPS); and chronic comorbidities as measured by the COmorbidity Point Score (COPS). Hospitalizations were grouped by admission diagnosis. The final model had excellent discrimination (c‐statistic 0.88) and calibration (P value of Hosmer Lemeshow statistic for entire cohort 0.66). This model was externally validated in our center with a c‐statistic of 0.901.25

We extended the Escobar model in several ways (Wong et al., Derivation and validation of a model to predict the daily risk of death in hospital, 2010, unpublished work). First, we modified it into a survival (rather than a logistic) model so it could estimate a daily probability of death in hospital. Second, we included the same covariates as Escobar except that we expressed LAPS as a time‐dependent covariate (meaning that the model accounted for changes in its value during the hospitalization). Finally, we included other time‐dependent covariates including: admission to intensive care unit; undergoing significant procedures; and awaiting long‐term care. This model had excellent discrimination (concordance probability of 0.895, 95% confidence interval [CI] 0.8890.902) and calibration.

We used this survival model to estimate the daily risk of death for all patients in the hospital each day. Summing these risks over hospital patients on each day returned the daily number of expected hospital deaths. This was collated per week.

The outcome statistic for this study was the ratio of the observed to expected weekly number of hospital deaths. Ratios exceeding 1 indicate that more deaths were observed than were expected (given the distribution of important covariates in those people during that week). This outcome statistic has several advantages. First, it accounts for the number of patients in the hospital each day. This is important because the number of hospital deaths will increase as the number of people in hospital increase. Second, it accounts for the severity of illness in each patient on each hospital day. This accounts for daily changes in risk of patient death, because calculation of the expected number of deaths per day was done using a multivariate survival model that included time‐dependent covariates. Therefore, each individual's predicted hazard of death (which was summed over the entire hospital to calculate the total expected number of deaths in hospital each day) took into account the latest values of these covariates. Previous analyses only accounted for risk of death at admission.

Expressing Physician Experience

The latent measure26 in all July Phenomenon studies is collective house‐staff physician experience. This is quantified by a surrogate date variable in which July 1the date that new house‐staff start their training in North Americarepresents minimal experience and June 30 represents maximal experience. We expressed collective physician experience on a scale from 0 (minimum experience) on July 1 to 1 (maximum experience) on June 30. A similar approach has been used previously13 and has advantages over the other methods used to capture collective house‐staff experience. In the stratified, incomplete approach,47, 911, 13, 1517 periods with inexperienced house‐staff (eg, July and August) are grouped together and compared to times with experienced house‐staff (eg, May and June), while ignoring all other data. The specification of cut‐points for this stratification is arbitrary and the method ignores large amounts of data. In the stratified, complete approach, periods with inexperienced house‐staff (eg, July and August) are grouped together and compared to all other times of the year.8, 12, 14, 1820, 22 This is potentially less biased because there are no lost data. However, the cut‐point for determining when house‐staff transition from inexperienced to experienced is arbitrary, and the model assumes that the transition is sudden. This is suboptimal because acquisition of experience is a gradual, constant process.

The pattern by which collective physician experience changes between July 1st and June 30th is unknown. We therefore expressed this evolution using five different patterns varying from a linear change to a natural logarithmic change (see Supporting Appendix B in the online version of this article).

Analysis

We first examined for autocorrelation in our outcome variable using Ljung‐Box statistics at lag 6 and 12 in PROC ARIMA (SAS 9.2, Cary, NC). If significant autocorrelation was absent in our data, linear regression modeling was used to associate the ratio of the observed to expected number of weekly deaths (the outcome variable) with the collective first year physician experience (the predictor variable). Time‐series methodology was to be used if significant autocorrelation was present.

In our baseline analysis, we included all hospitalizations together. In stratified analyses, we categorized hospitalizations by admission status (emergent vs elective) and admission service (medicine vs surgery).

Weekly Deaths: Observed, Expected, and Ratio

Figure 1A presents the observed weekly number of deaths during the study period. There was an average of 31 deaths per week (range 1551). Some large fluctuations in the weekly number of deaths were seen; in 2007, for example, the number of observed deaths went from 21 in week 13 up to 46 in week 15. However, no obvious seasonal trends in the observed weekly number of deaths were seen (Figure 1A, heavy line) nor were trends between years obvious.

Figure 1

Figure 1B presents the expected weekly number of deaths during the study period. The expected weekly number of deaths averaged 29.6 (range 22.238.7). The expected weekly number of deaths was notably less variable than the observed number of deaths. However, important variations in the expected number of deaths were seen; for example, in 2005, the expected number of deaths increased from 24.1 in week 41 to 29.6 in week 44. Again, we saw no obvious seasonal trends in the expected weekly number of deaths (Figure 1B, heavy line).

Figure 1C illustrates the ratio of observed to the expected weekly number of deaths. The average observed to expected ratio slightly exceeded unity (1.05) and ranged from 0.488 (week 24, in 2008) to 1.821 (week 51, in 2008). We saw no obvious seasonal trends in the ratio of the observed to expected number of weekly deaths. In addition, obvious trends in this ratio were absent over the study period.

Association Between House‐Staff Experience and Death in Hospital

We found no evidence of autocorrelation in the ratio of observed to expected weekly number of deaths. The ratio of observed to expected number of hospital deaths was not significantly associated with house‐staff physician experience (Table 2). This conclusion did not change regardless of which house‐staff physician experience pattern was used in the linear model (Table 2). In addition, our analysis found no significant association between physician experience and patient mortality when analyses were stratified by admission service or admission status (Table 2).

Study Setting

TOH is a tertiary‐care teaching hospital with two inpatient campuses. The hospital operates within a publicly funded health care system, serves a population of approximately 1.5 million people in Ottawa and Eastern Ontario, treats all major trauma patients for the region, and provides most of the oncological care in the region.

TOH is the primary medical teaching hospital at the University of Ottawa. In 2010, there were 197 residents starting their first year of postgraduate training in one of 29 programs.

Inclusion Criteria

The study period extended from April 15, 2004 to December 31, 2008. We used this start time because our hospital switched to new coding systems for procedures and diagnoses in April 2002. Since these new coding systems contributed to our outcome statistic, we used a very long period (ie, two years) for coding patterns to stabilize to ensure that any changes seen were not a function of coding patterns. We ended our study in December 2008 because this was the last date of complete data at the time we started the analysis.

We included all medical, surgical, and obstetrical patients admitted to TOH during this time except those who were: younger than 15 years old; transferred to or from another acute care hospital; or obstetrical patients hospitalized for routine childbirth. These patients were excluded because they were not part of the multivariate model that we used to calculate risk of death in hospital (discussed below).24 These exclusions accounted for 25.4% of all admissions during the study period (36,820less than 15 years old; 12,931transferred to or from the hospital; and 44,220uncomplicated admission for childbirth).

All data used in this study came from The Ottawa Hospital Data Warehouse (TOHDW). This is a repository of clinical, laboratory, and administrative data originating from the hospital's major operational information systems. TOHDW contains information on patient demographics and diagnoses, as well as procedures and patient transfers between different units or hospital services during the admission.

Primary OutcomeRatio of Observed to Expected Number of Deaths per Week

For each study day, we measured the number of hospital deaths from the patient registration table in TOHDW. This statistic was collated for each week to ensure numeric stability, especially in our subgroup analyses.

We calculated the weekly expected number of hospital deaths using an extension of the Escobar model.24 The Escobar is a logistic regression model that estimated the probability of death in hospital that was derived and internally validated on almost 260,000 hospitalizations at 17 hospitals in the Kaiser Permanente Health Plan. It included six covariates that were measurable at admission including: patient age; patient sex; admission urgency (ie, elective or emergent) and service (ie, medical or surgical); admission diagnosis; severity of acute illness as measured by the Laboratory‐based Acute Physiology Score (LAPS); and chronic comorbidities as measured by the COmorbidity Point Score (COPS). Hospitalizations were grouped by admission diagnosis. The final model had excellent discrimination (c‐statistic 0.88) and calibration (P value of Hosmer Lemeshow statistic for entire cohort 0.66). This model was externally validated in our center with a c‐statistic of 0.901.25

We extended the Escobar model in several ways (Wong et al., Derivation and validation of a model to predict the daily risk of death in hospital, 2010, unpublished work). First, we modified it into a survival (rather than a logistic) model so it could estimate a daily probability of death in hospital. Second, we included the same covariates as Escobar except that we expressed LAPS as a time‐dependent covariate (meaning that the model accounted for changes in its value during the hospitalization). Finally, we included other time‐dependent covariates including: admission to intensive care unit; undergoing significant procedures; and awaiting long‐term care. This model had excellent discrimination (concordance probability of 0.895, 95% confidence interval [CI] 0.8890.902) and calibration.

We used this survival model to estimate the daily risk of death for all patients in the hospital each day. Summing these risks over hospital patients on each day returned the daily number of expected hospital deaths. This was collated per week.

The outcome statistic for this study was the ratio of the observed to expected weekly number of hospital deaths. Ratios exceeding 1 indicate that more deaths were observed than were expected (given the distribution of important covariates in those people during that week). This outcome statistic has several advantages. First, it accounts for the number of patients in the hospital each day. This is important because the number of hospital deaths will increase as the number of people in hospital increase. Second, it accounts for the severity of illness in each patient on each hospital day. This accounts for daily changes in risk of patient death, because calculation of the expected number of deaths per day was done using a multivariate survival model that included time‐dependent covariates. Therefore, each individual's predicted hazard of death (which was summed over the entire hospital to calculate the total expected number of deaths in hospital each day) took into account the latest values of these covariates. Previous analyses only accounted for risk of death at admission.

Expressing Physician Experience

The latent measure26 in all July Phenomenon studies is collective house‐staff physician experience. This is quantified by a surrogate date variable in which July 1the date that new house‐staff start their training in North Americarepresents minimal experience and June 30 represents maximal experience. We expressed collective physician experience on a scale from 0 (minimum experience) on July 1 to 1 (maximum experience) on June 30. A similar approach has been used previously13 and has advantages over the other methods used to capture collective house‐staff experience. In the stratified, incomplete approach,47, 911, 13, 1517 periods with inexperienced house‐staff (eg, July and August) are grouped together and compared to times with experienced house‐staff (eg, May and June), while ignoring all other data. The specification of cut‐points for this stratification is arbitrary and the method ignores large amounts of data. In the stratified, complete approach, periods with inexperienced house‐staff (eg, July and August) are grouped together and compared to all other times of the year.8, 12, 14, 1820, 22 This is potentially less biased because there are no lost data. However, the cut‐point for determining when house‐staff transition from inexperienced to experienced is arbitrary, and the model assumes that the transition is sudden. This is suboptimal because acquisition of experience is a gradual, constant process.

The pattern by which collective physician experience changes between July 1st and June 30th is unknown. We therefore expressed this evolution using five different patterns varying from a linear change to a natural logarithmic change (see Supporting Appendix B in the online version of this article).

Analysis

We first examined for autocorrelation in our outcome variable using Ljung‐Box statistics at lag 6 and 12 in PROC ARIMA (SAS 9.2, Cary, NC). If significant autocorrelation was absent in our data, linear regression modeling was used to associate the ratio of the observed to expected number of weekly deaths (the outcome variable) with the collective first year physician experience (the predictor variable). Time‐series methodology was to be used if significant autocorrelation was present.

In our baseline analysis, we included all hospitalizations together. In stratified analyses, we categorized hospitalizations by admission status (emergent vs elective) and admission service (medicine vs surgery).

Weekly Deaths: Observed, Expected, and Ratio

Figure 1A presents the observed weekly number of deaths during the study period. There was an average of 31 deaths per week (range 1551). Some large fluctuations in the weekly number of deaths were seen; in 2007, for example, the number of observed deaths went from 21 in week 13 up to 46 in week 15. However, no obvious seasonal trends in the observed weekly number of deaths were seen (Figure 1A, heavy line) nor were trends between years obvious.

Figure 1

Figure 1B presents the expected weekly number of deaths during the study period. The expected weekly number of deaths averaged 29.6 (range 22.238.7). The expected weekly number of deaths was notably less variable than the observed number of deaths. However, important variations in the expected number of deaths were seen; for example, in 2005, the expected number of deaths increased from 24.1 in week 41 to 29.6 in week 44. Again, we saw no obvious seasonal trends in the expected weekly number of deaths (Figure 1B, heavy line).

Figure 1C illustrates the ratio of observed to the expected weekly number of deaths. The average observed to expected ratio slightly exceeded unity (1.05) and ranged from 0.488 (week 24, in 2008) to 1.821 (week 51, in 2008). We saw no obvious seasonal trends in the ratio of the observed to expected number of weekly deaths. In addition, obvious trends in this ratio were absent over the study period.

Association Between House‐Staff Experience and Death in Hospital

We found no evidence of autocorrelation in the ratio of observed to expected weekly number of deaths. The ratio of observed to expected number of hospital deaths was not significantly associated with house‐staff physician experience (Table 2). This conclusion did not change regardless of which house‐staff physician experience pattern was used in the linear model (Table 2). In addition, our analysis found no significant association between physician experience and patient mortality when analyses were stratified by admission service or admission status (Table 2).

Study Design

This was a multicenter prospective cohort study of consecutive patients discharged to the community from the medical or surgical services of 11 Ontario hospitals (6 university‐affiliated hospitals and 5 community hospitals) in 5 cities after an elective or emergency hospitalization. Patients were invited to participate in the study if they were cognitively intact, had a telephone, and provided written informed consent. Patients were excluded if they were less than 18 years old, were discharged to nursing homes, or were not proficient in English and did not have someone to help communicate with study staff. Enrolled patients were excluded from the analysis if they had less than 2 physician visits prior to one of the study's outcomes or the end of patient observation (which was 6 months postdischarge). This final exclusion criterion was necessary since 2 continuity measures (including postdischarge physician continuity and postdischarge information continuity) were incalculable with less than 2 physician visits during follow‐up (Supporting information). The study was approved by the research ethics board of each participating hospital.

Data Collection

Prior to hospital discharge, patients were interviewed by study personnel to identify their baseline functional status, their living conditions, all physicians who regularly treated the patient prior to admission (including both family physicians and consultants), and chronic medical conditions. The latter were confirmed by a review of the patient's chart and hospital discharge summary, when available. Patients also provided principal contacts whom we could contact in the event patients could not be reached. The chart and discharge summary were also used to identify diagnoses in hospitalincluding complications (diagnoses arising in the hospital)and medications at discharge.

Patients or their designated contacts were telephoned 1, 3, and 6 months after hospital discharge to identify the date and the physician of all postdischarge physician visits. For each postdischarge physician visit, we determined whether the physician had access to a discharge summary for the index hospitalization. We also determined the availability of information from all previous postdischarge visits that the patient had with other physicians. The methods used to collect these data were previously detailed.26 Briefly, we used three complementary methods to elicit this information from each follow‐up physician. First, patients gave the physician a survey on which the physician listed all prior visits with other doctors for which they had information. If this survey was not returned, we faxed the survey to the physician. If the faxed survey was not returned, we telephoned the physician or their office staff and administered the survey over the telephone.

Continuity Measures

We measured components of both provider and information continuity. For the posthospitalization period, we measured provider continuity for physicians who had provided patient care during three distinct phases: the prehospital period; the hospital period; and the postdischarge period. Prehospital physicians were those classified by the patient as their regular physician(s) (defined as physiciansboth family physicians and consultantsthat they had seen in the past and were likely to see again in the future). Hospital provider continuity was divided into 2 components: hospital physician continuity (ie, the most responsible physician in the hospital); and hospital consultant continuity (ie, another physician who consulted on the patient during admission). Information continuity was divided into discharge summary continuity and postdischarge visit information continuity.

We quantified provider and information continuity using Breslau's Usual Provider of Continuity (UPC)27 measure. It is a widely used and validated continuity measure whose values are meaningful and interpretable.6 The UPC measures the proportion of visits with the physician of interest (for provider continuity) or the proportion of visits having the information of interest (for information continuity). The UPC was calculated as: $${\rm UPC} = {\rm n}_{\rm i} / {\rm N}$$where UPC is the Usual Provider of Continuity; n_i is the number of postdischarge visits to the physician type of interest (eg, prehospital; hospital; postdischarge) or the number of visits at which the information of interest (eg, discharge summary) was available; and N is the total number of postdischarge visits. The UPC ranges from 0 to 1 where 0 is perfect discontinuity and 1 is perfect continuity. Details regarding the provider and information continuity measures are given in the supporting information and were discussed in greater detail in a previous study.28

As the formulae in the supporting information suggest, all continuity measures were incalculable prior to the first postdischarge visit and all continuity measures changed value at each visit during patient observation. In addition, a particular physician visit could increase multiple continuity measures simultaneously. For example, a visit with a physician who was the hospital physician and who regularly treated the patient prior to the hospitalization would increase both hospital and prehospital provider continuity. If the patient had previously seen the physician after discharge, the visit would also increase postdischarge physician continuity.

Study Outcomes

Outcomes for the study included time to all‐cause death and time to all‐cause, urgent readmission. To be classified as urgent, readmissions could not be arranged when the patient was originally discharged from hospital or more than 4 weeks prior to the readmission. All hospital admissions meeting these criteria during the 6 month study period were labeled in this study as urgent readmissions even if they were unrelated to the index admission.

Principal contacts were called if we were unable to reach the patient to determine their outcomes. If the patient's vital status remained unclear, we contacted the Office of the Provincial Registrar to determine if and when the patient died during the 6 months after discharge from hospital.

Analysis

Outcome incidence densities and 95% confidence intervals [CIs] were calculated using PROC GENMOD in SAS to account for clustering of patients in hospitals. We used multivariate proportional hazards modeling to determine the independent association of provider and information continuity measures with time to death and time to urgent readmission. Patient observation started when patients were discharged from the hospital. Patient observation ended at the earliest of the following: death; urgent readmission to the hospital; end of follow‐up (which was 6 months after discharge from the hospital) or loss to follow‐up. Because hospital consultant continuity was very highly skewed (95.6% of patients had a value of 0; mean value of 0.016; skewness 6.9), it was not included in the primary regression models but was included in a sensitivity analysis.

To adjust for potential confounders in the association between continuity and the outcomes, our model included all factors that were independently associated with either the outcome or any continuity measure. Factors associated with death or urgent readmission were summarized using the LACE index.29 This index combines a patient's hospital length of stay, admission acuity, patient comorbidity (measured with the Charlson Score30 using updated disease category weights by Schneeweiss et al.),31 and emergency room utilization (measured as the number of visits in the 6 months prior to admission) into a single number ranging from 0 to 19. The LACE index was moderately discriminative and highly accurate at predicting 30‐day death or urgent readmission.29 In a separate study,28 we found that the following factors were independently associated with at least one of the continuity measures: patient age; patient sex; number of admissions in previous 6 months; number of regular treating physicians prior to admission; hospital service (medicine vs. surgery); and number of complications in the hospital (defined as new problems arising after admission to hospital). By including all factors that were independently associated with either the outcome or continuity, we controlled for all measured factors that could act as confounders in the association between continuity and outcomes. We accounted for the clustered study design by using conditional proportional hazards models that stratified by hospitals.32 Analytical details are given in the supporting information.

Continuity Measures

Table 2 summarizes all continuity scores. Since continuity scores varied significantly over time,28 Table 2 provides continuity scores on the last day of patient observation. Preadmission provider, postdischarge provider, and discharge summary continuity all had similar values and distributions with median values ranging between 0.444 and 0.571. 1797 (46.4%) patients had a hospital physician provider continuity scorae of 0.

Study Outcomes

During a median of 175 days of observation, 45 patients died (event rate 2.6 events per 100 patient‐years observation [95% CI 2.0‐3.4]) and 340 patients were urgently readmitted (event rate 19.6 events per 100 patient‐years observation [95% CI 15.9‐24.3]). Figure 2 presents the survival curves for time to death and time to urgent readmission. The hazard of death was consistent through the observation period but the risk of urgent readmission decreased slightly after 90 days postdischarge.

Association Between Continuity and Outcomes

Table 3 summarizes the association between provider and information continuity with study outcomes. No continuity measure was associated with time to death by itself (Table 3, column A) or with the other continuity measures (Table 3, column B). Preadmission physician continuity was associated with a significantly decreased risk of urgent readmission. When the proportion of postdischarge visits with a prehospital physician increased by 10%, the adjusted risk of urgent readmission decreased by 6% (adjusted hazards ratio (adj‐HR)) of 0.94 (95% CI, 0.91‐0.98). None of the other continuity measuresincluding hospital physicianwere significantly associated with urgent readmission either by themselves (Table 3, column A) or after adjusting for other continuity measures (Table 3, column B).

Increased patient age and increased LACE index score were both strongly associated with an increased risk of death (adj‐HR 1.43 [1.13‐1.82] and 1.16 [1.06‐1.26], respectively) and urgent readmission (adj‐HR 1.18 [1.10‐1.28] and 1.10 [1.07‐1.14], respectively). Hospitalization in the 6 months prior to admission significantly increased the risk of urgent readmission but not death. The risk of urgent readmission increased significantly as the number of regular prehospital physicians increased.

Sensitivity Analyses

Our study conclusions did not change in the sensitivity analyses. The number of postdischarge physician visits (expressed as a time‐dependent covariate) was not associated with either death or with urgent readmission and preadmission physician continuity remained significantly associated with time to urgent readmission (supporting information). Adding consultant continuity to the model also did not change our results (supporting information). In‐hospital consultant continuity was associated with an increased risk of urgent readmission (adj‐HR 1.10, 95% CI, 1.01‐1.20). The association between pre‐admission physician continuity and time to urgent readmission did not interact significantly with patient age, LACE index score, or number of previous admissions.

Study Design

This was a multicenter prospective cohort study of consecutive patients discharged to the community from the medical or surgical services of 11 Ontario hospitals (6 university‐affiliated hospitals and 5 community hospitals) in 5 cities after an elective or emergency hospitalization. Patients were invited to participate in the study if they were cognitively intact, had a telephone, and provided written informed consent. Patients were excluded if they were less than 18 years old, were discharged to nursing homes, or were not proficient in English and did not have someone to help communicate with study staff. Enrolled patients were excluded from the analysis if they had less than 2 physician visits prior to one of the study's outcomes or the end of patient observation (which was 6 months postdischarge). This final exclusion criterion was necessary since 2 continuity measures (including postdischarge physician continuity and postdischarge information continuity) were incalculable with less than 2 physician visits during follow‐up (Supporting information). The study was approved by the research ethics board of each participating hospital.

Data Collection

Prior to hospital discharge, patients were interviewed by study personnel to identify their baseline functional status, their living conditions, all physicians who regularly treated the patient prior to admission (including both family physicians and consultants), and chronic medical conditions. The latter were confirmed by a review of the patient's chart and hospital discharge summary, when available. Patients also provided principal contacts whom we could contact in the event patients could not be reached. The chart and discharge summary were also used to identify diagnoses in hospitalincluding complications (diagnoses arising in the hospital)and medications at discharge.

Patients or their designated contacts were telephoned 1, 3, and 6 months after hospital discharge to identify the date and the physician of all postdischarge physician visits. For each postdischarge physician visit, we determined whether the physician had access to a discharge summary for the index hospitalization. We also determined the availability of information from all previous postdischarge visits that the patient had with other physicians. The methods used to collect these data were previously detailed.26 Briefly, we used three complementary methods to elicit this information from each follow‐up physician. First, patients gave the physician a survey on which the physician listed all prior visits with other doctors for which they had information. If this survey was not returned, we faxed the survey to the physician. If the faxed survey was not returned, we telephoned the physician or their office staff and administered the survey over the telephone.

Continuity Measures

We measured components of both provider and information continuity. For the posthospitalization period, we measured provider continuity for physicians who had provided patient care during three distinct phases: the prehospital period; the hospital period; and the postdischarge period. Prehospital physicians were those classified by the patient as their regular physician(s) (defined as physiciansboth family physicians and consultantsthat they had seen in the past and were likely to see again in the future). Hospital provider continuity was divided into 2 components: hospital physician continuity (ie, the most responsible physician in the hospital); and hospital consultant continuity (ie, another physician who consulted on the patient during admission). Information continuity was divided into discharge summary continuity and postdischarge visit information continuity.

We quantified provider and information continuity using Breslau's Usual Provider of Continuity (UPC)27 measure. It is a widely used and validated continuity measure whose values are meaningful and interpretable.6 The UPC measures the proportion of visits with the physician of interest (for provider continuity) or the proportion of visits having the information of interest (for information continuity). The UPC was calculated as: $${\rm UPC} = {\rm n}_{\rm i} / {\rm N}$$where UPC is the Usual Provider of Continuity; n_i is the number of postdischarge visits to the physician type of interest (eg, prehospital; hospital; postdischarge) or the number of visits at which the information of interest (eg, discharge summary) was available; and N is the total number of postdischarge visits. The UPC ranges from 0 to 1 where 0 is perfect discontinuity and 1 is perfect continuity. Details regarding the provider and information continuity measures are given in the supporting information and were discussed in greater detail in a previous study.28

As the formulae in the supporting information suggest, all continuity measures were incalculable prior to the first postdischarge visit and all continuity measures changed value at each visit during patient observation. In addition, a particular physician visit could increase multiple continuity measures simultaneously. For example, a visit with a physician who was the hospital physician and who regularly treated the patient prior to the hospitalization would increase both hospital and prehospital provider continuity. If the patient had previously seen the physician after discharge, the visit would also increase postdischarge physician continuity.

Study Outcomes

Outcomes for the study included time to all‐cause death and time to all‐cause, urgent readmission. To be classified as urgent, readmissions could not be arranged when the patient was originally discharged from hospital or more than 4 weeks prior to the readmission. All hospital admissions meeting these criteria during the 6 month study period were labeled in this study as urgent readmissions even if they were unrelated to the index admission.

Principal contacts were called if we were unable to reach the patient to determine their outcomes. If the patient's vital status remained unclear, we contacted the Office of the Provincial Registrar to determine if and when the patient died during the 6 months after discharge from hospital.

Analysis

Outcome incidence densities and 95% confidence intervals [CIs] were calculated using PROC GENMOD in SAS to account for clustering of patients in hospitals. We used multivariate proportional hazards modeling to determine the independent association of provider and information continuity measures with time to death and time to urgent readmission. Patient observation started when patients were discharged from the hospital. Patient observation ended at the earliest of the following: death; urgent readmission to the hospital; end of follow‐up (which was 6 months after discharge from the hospital) or loss to follow‐up. Because hospital consultant continuity was very highly skewed (95.6% of patients had a value of 0; mean value of 0.016; skewness 6.9), it was not included in the primary regression models but was included in a sensitivity analysis.

To adjust for potential confounders in the association between continuity and the outcomes, our model included all factors that were independently associated with either the outcome or any continuity measure. Factors associated with death or urgent readmission were summarized using the LACE index.29 This index combines a patient's hospital length of stay, admission acuity, patient comorbidity (measured with the Charlson Score30 using updated disease category weights by Schneeweiss et al.),31 and emergency room utilization (measured as the number of visits in the 6 months prior to admission) into a single number ranging from 0 to 19. The LACE index was moderately discriminative and highly accurate at predicting 30‐day death or urgent readmission.29 In a separate study,28 we found that the following factors were independently associated with at least one of the continuity measures: patient age; patient sex; number of admissions in previous 6 months; number of regular treating physicians prior to admission; hospital service (medicine vs. surgery); and number of complications in the hospital (defined as new problems arising after admission to hospital). By including all factors that were independently associated with either the outcome or continuity, we controlled for all measured factors that could act as confounders in the association between continuity and outcomes. We accounted for the clustered study design by using conditional proportional hazards models that stratified by hospitals.32 Analytical details are given in the supporting information.

Continuity Measures

Table 2 summarizes all continuity scores. Since continuity scores varied significantly over time,28 Table 2 provides continuity scores on the last day of patient observation. Preadmission provider, postdischarge provider, and discharge summary continuity all had similar values and distributions with median values ranging between 0.444 and 0.571. 1797 (46.4%) patients had a hospital physician provider continuity scorae of 0.

Study Outcomes

During a median of 175 days of observation, 45 patients died (event rate 2.6 events per 100 patient‐years observation [95% CI 2.0‐3.4]) and 340 patients were urgently readmitted (event rate 19.6 events per 100 patient‐years observation [95% CI 15.9‐24.3]). Figure 2 presents the survival curves for time to death and time to urgent readmission. The hazard of death was consistent through the observation period but the risk of urgent readmission decreased slightly after 90 days postdischarge.

Association Between Continuity and Outcomes

Table 3 summarizes the association between provider and information continuity with study outcomes. No continuity measure was associated with time to death by itself (Table 3, column A) or with the other continuity measures (Table 3, column B). Preadmission physician continuity was associated with a significantly decreased risk of urgent readmission. When the proportion of postdischarge visits with a prehospital physician increased by 10%, the adjusted risk of urgent readmission decreased by 6% (adjusted hazards ratio (adj‐HR)) of 0.94 (95% CI, 0.91‐0.98). None of the other continuity measuresincluding hospital physicianwere significantly associated with urgent readmission either by themselves (Table 3, column A) or after adjusting for other continuity measures (Table 3, column B).

Increased patient age and increased LACE index score were both strongly associated with an increased risk of death (adj‐HR 1.43 [1.13‐1.82] and 1.16 [1.06‐1.26], respectively) and urgent readmission (adj‐HR 1.18 [1.10‐1.28] and 1.10 [1.07‐1.14], respectively). Hospitalization in the 6 months prior to admission significantly increased the risk of urgent readmission but not death. The risk of urgent readmission increased significantly as the number of regular prehospital physicians increased.

Sensitivity Analyses

Our study conclusions did not change in the sensitivity analyses. The number of postdischarge physician visits (expressed as a time‐dependent covariate) was not associated with either death or with urgent readmission and preadmission physician continuity remained significantly associated with time to urgent readmission (supporting information). Adding consultant continuity to the model also did not change our results (supporting information). In‐hospital consultant continuity was associated with an increased risk of urgent readmission (adj‐HR 1.10, 95% CI, 1.01‐1.20). The association between pre‐admission physician continuity and time to urgent readmission did not interact significantly with patient age, LACE index score, or number of previous admissions.

Study Design

This was a multicenter prospective cohort study of consecutive patients discharged to the community from the medical or surgical services of 11 Ontario hospitals (6 university‐affiliated hospitals and 5 community hospitals) in 5 cities after an elective or emergency hospitalization. Patients were invited to participate in the study if they were cognitively intact, had a telephone, and provided written informed consent. Patients were excluded if they were less than 18 years old, were discharged to nursing homes, or were not proficient in English and did not have someone to help communicate with study staff. Enrolled patients were excluded from the analysis if they had less than 2 physician visits prior to one of the study's outcomes or the end of patient observation (which was 6 months postdischarge). This final exclusion criterion was necessary since 2 continuity measures (including postdischarge physician continuity and postdischarge information continuity) were incalculable with less than 2 physician visits during follow‐up (Supporting information). The study was approved by the research ethics board of each participating hospital.

Data Collection

Prior to hospital discharge, patients were interviewed by study personnel to identify their baseline functional status, their living conditions, all physicians who regularly treated the patient prior to admission (including both family physicians and consultants), and chronic medical conditions. The latter were confirmed by a review of the patient's chart and hospital discharge summary, when available. Patients also provided principal contacts whom we could contact in the event patients could not be reached. The chart and discharge summary were also used to identify diagnoses in hospitalincluding complications (diagnoses arising in the hospital)and medications at discharge.

Patients or their designated contacts were telephoned 1, 3, and 6 months after hospital discharge to identify the date and the physician of all postdischarge physician visits. For each postdischarge physician visit, we determined whether the physician had access to a discharge summary for the index hospitalization. We also determined the availability of information from all previous postdischarge visits that the patient had with other physicians. The methods used to collect these data were previously detailed.26 Briefly, we used three complementary methods to elicit this information from each follow‐up physician. First, patients gave the physician a survey on which the physician listed all prior visits with other doctors for which they had information. If this survey was not returned, we faxed the survey to the physician. If the faxed survey was not returned, we telephoned the physician or their office staff and administered the survey over the telephone.

Continuity Measures

We measured components of both provider and information continuity. For the posthospitalization period, we measured provider continuity for physicians who had provided patient care during three distinct phases: the prehospital period; the hospital period; and the postdischarge period. Prehospital physicians were those classified by the patient as their regular physician(s) (defined as physiciansboth family physicians and consultantsthat they had seen in the past and were likely to see again in the future). Hospital provider continuity was divided into 2 components: hospital physician continuity (ie, the most responsible physician in the hospital); and hospital consultant continuity (ie, another physician who consulted on the patient during admission). Information continuity was divided into discharge summary continuity and postdischarge visit information continuity.

We quantified provider and information continuity using Breslau's Usual Provider of Continuity (UPC)27 measure. It is a widely used and validated continuity measure whose values are meaningful and interpretable.6 The UPC measures the proportion of visits with the physician of interest (for provider continuity) or the proportion of visits having the information of interest (for information continuity). The UPC was calculated as: $${\rm UPC} = {\rm n}_{\rm i} / {\rm N}$$where UPC is the Usual Provider of Continuity; n_i is the number of postdischarge visits to the physician type of interest (eg, prehospital; hospital; postdischarge) or the number of visits at which the information of interest (eg, discharge summary) was available; and N is the total number of postdischarge visits. The UPC ranges from 0 to 1 where 0 is perfect discontinuity and 1 is perfect continuity. Details regarding the provider and information continuity measures are given in the supporting information and were discussed in greater detail in a previous study.28

As the formulae in the supporting information suggest, all continuity measures were incalculable prior to the first postdischarge visit and all continuity measures changed value at each visit during patient observation. In addition, a particular physician visit could increase multiple continuity measures simultaneously. For example, a visit with a physician who was the hospital physician and who regularly treated the patient prior to the hospitalization would increase both hospital and prehospital provider continuity. If the patient had previously seen the physician after discharge, the visit would also increase postdischarge physician continuity.

Study Outcomes

Outcomes for the study included time to all‐cause death and time to all‐cause, urgent readmission. To be classified as urgent, readmissions could not be arranged when the patient was originally discharged from hospital or more than 4 weeks prior to the readmission. All hospital admissions meeting these criteria during the 6 month study period were labeled in this study as urgent readmissions even if they were unrelated to the index admission.

Principal contacts were called if we were unable to reach the patient to determine their outcomes. If the patient's vital status remained unclear, we contacted the Office of the Provincial Registrar to determine if and when the patient died during the 6 months after discharge from hospital.

Analysis

Outcome incidence densities and 95% confidence intervals [CIs] were calculated using PROC GENMOD in SAS to account for clustering of patients in hospitals. We used multivariate proportional hazards modeling to determine the independent association of provider and information continuity measures with time to death and time to urgent readmission. Patient observation started when patients were discharged from the hospital. Patient observation ended at the earliest of the following: death; urgent readmission to the hospital; end of follow‐up (which was 6 months after discharge from the hospital) or loss to follow‐up. Because hospital consultant continuity was very highly skewed (95.6% of patients had a value of 0; mean value of 0.016; skewness 6.9), it was not included in the primary regression models but was included in a sensitivity analysis.

To adjust for potential confounders in the association between continuity and the outcomes, our model included all factors that were independently associated with either the outcome or any continuity measure. Factors associated with death or urgent readmission were summarized using the LACE index.29 This index combines a patient's hospital length of stay, admission acuity, patient comorbidity (measured with the Charlson Score30 using updated disease category weights by Schneeweiss et al.),31 and emergency room utilization (measured as the number of visits in the 6 months prior to admission) into a single number ranging from 0 to 19. The LACE index was moderately discriminative and highly accurate at predicting 30‐day death or urgent readmission.29 In a separate study,28 we found that the following factors were independently associated with at least one of the continuity measures: patient age; patient sex; number of admissions in previous 6 months; number of regular treating physicians prior to admission; hospital service (medicine vs. surgery); and number of complications in the hospital (defined as new problems arising after admission to hospital). By including all factors that were independently associated with either the outcome or continuity, we controlled for all measured factors that could act as confounders in the association between continuity and outcomes. We accounted for the clustered study design by using conditional proportional hazards models that stratified by hospitals.32 Analytical details are given in the supporting information.

Continuity Measures

Table 2 summarizes all continuity scores. Since continuity scores varied significantly over time,28 Table 2 provides continuity scores on the last day of patient observation. Preadmission provider, postdischarge provider, and discharge summary continuity all had similar values and distributions with median values ranging between 0.444 and 0.571. 1797 (46.4%) patients had a hospital physician provider continuity scorae of 0.

Study Outcomes

During a median of 175 days of observation, 45 patients died (event rate 2.6 events per 100 patient‐years observation [95% CI 2.0‐3.4]) and 340 patients were urgently readmitted (event rate 19.6 events per 100 patient‐years observation [95% CI 15.9‐24.3]). Figure 2 presents the survival curves for time to death and time to urgent readmission. The hazard of death was consistent through the observation period but the risk of urgent readmission decreased slightly after 90 days postdischarge.

Association Between Continuity and Outcomes

Table 3 summarizes the association between provider and information continuity with study outcomes. No continuity measure was associated with time to death by itself (Table 3, column A) or with the other continuity measures (Table 3, column B). Preadmission physician continuity was associated with a significantly decreased risk of urgent readmission. When the proportion of postdischarge visits with a prehospital physician increased by 10%, the adjusted risk of urgent readmission decreased by 6% (adjusted hazards ratio (adj‐HR)) of 0.94 (95% CI, 0.91‐0.98). None of the other continuity measuresincluding hospital physicianwere significantly associated with urgent readmission either by themselves (Table 3, column A) or after adjusting for other continuity measures (Table 3, column B).

Increased patient age and increased LACE index score were both strongly associated with an increased risk of death (adj‐HR 1.43 [1.13‐1.82] and 1.16 [1.06‐1.26], respectively) and urgent readmission (adj‐HR 1.18 [1.10‐1.28] and 1.10 [1.07‐1.14], respectively). Hospitalization in the 6 months prior to admission significantly increased the risk of urgent readmission but not death. The risk of urgent readmission increased significantly as the number of regular prehospital physicians increased.

Sensitivity Analyses

Our study conclusions did not change in the sensitivity analyses. The number of postdischarge physician visits (expressed as a time‐dependent covariate) was not associated with either death or with urgent readmission and preadmission physician continuity remained significantly associated with time to urgent readmission (supporting information). Adding consultant continuity to the model also did not change our results (supporting information). In‐hospital consultant continuity was associated with an increased risk of urgent readmission (adj‐HR 1.10, 95% CI, 1.01‐1.20). The association between pre‐admission physician continuity and time to urgent readmission did not interact significantly with patient age, LACE index score, or number of previous admissions.