Pending Tests at Discharge

The period following discharge is a vulnerable time for patientsthe prevalence of medical errors related to this transition is high and has important patient safety and medico‐legal ramifications.13 Factors contributing to this vulnerability include complexity of hospitalized patients, shorter lengths of stay, and increased discontinuity of care. Hospitalists have recognized this threat to patient safety and have worked toward improving information exchange between inpatient and outpatient providers at hospital discharge.46 Nonetheless, the evidence suggests that more work is necessary. A recent study found that discharge summaries are often incomplete, and do not contain important information requiring follow‐up, such as pending tests.7 Additionally, a review by Kripalani et al. characterizing information deficits at hospital discharge found few interventions which specifically improve communication of pending tests at hospital discharge.8

In a prior study we determined that 41% of patients left the hospital before all laboratory and radiology test results were finalized. Of these results, 9.4% were potentially actionable and could have altered management. Physicians were aware of only 38% of post‐discharge test results.9 This awareness gap is a consequence of several factors including the lack of systems to track and alert providers of test results finalized post discharge. Also, it is unclear who is responsible for pending tests at discharge, since these tests are ordered by the inpatient physicians but often reported in the time period between hospital discharge and the patient's first follow‐up appointment with the primary care physician (PCP). Because responsibility is not explicitly made in the final communication between physicians at discharge, such test results may not be reviewed in a timely manner, potentially resulting in delays in treatment, a need for readmission, or other unfavorable outcomes.

Even in integrated health systems with advanced electronic health records, missed test results which result in treatment delays remain prevalent.10, 11 Test result management applications aid clinicians in reviewing and acting upon results as they become available and such systems may provide solutions to this problem. At Partners Healthcare in Boston, the Results Manager (RM) application was developed to help clinicians in the ambulatory setting safely, reliably, and efficiently review and act upon test results. The application enables clinicians to prioritize test results, utilize guidelines, and generate letters to patients. This system also prompts physicians to set reminders for future testing.12 In a 2.5‐year study evaluating the impact of this intervention, PCPs at 26 adult primary care practices were able to expedite communication of outpatient laboratory and imaging test results to patients with the help of RM. Patients of physicians who participated in the project reported greater satisfaction with test result communication and with information provided about their condition than did a control group of similar patients.13 RM has not yet been studied in the inpatient setting or at care transitions. We describe an attempt at modifying the Partners RM application to help inpatient physicians manage pending tests at hospital discharge.

Methods

Test Result Management Application

RM was originally developed by Partners Healthcare to improve timely review and appropriate management of test results in the ambulatory setting. RM was developed for and vetted by primarily ambulatory physicians. The application is browser‐based, provider‐centric, and embedded in the LMR to help ambulatory clinicians review and act upon test results in a safe, reliable, and efficient manner. Although RM has access to all inpatient and outpatient data in the Partners Clinical Data Repository (CDR), given the volume of inpatient tests ordered, hospital‐based results are suppressed by default to limit inundating ambulatory clinicians' queues. Therefore, users of RM only receive results of laboratory and radiology tests ordered in the ambulatory setting. They can track these tests for specific patients for a designated period of time by placing the patient on a watch list. Finally, RM incorporates extensive decision support features to classify the degree of abnormality for each result, presents guidelines to help clinicians manage abnormal results, allows clinicians to generate result letters to patients using predefined, context‐sensitive templates, and prompts physicians to set reminders for future testing. Because RM was developed from the ambulatory perspective, there was limited input from hospitalist physicians with regard to inpatient workflow in the original design of the module.12 See Figure 1 for a screen shot of RM and a description of its features.

Figure 1

For purposes of this pilot, we modified RM to allow results of tests ordered in the inpatient setting to be available for viewing (Hospitalist Results Manager, HRM). This feature was turned on only for inpatient physicians as previously defined. Inpatient tests, including pending tests at discharge, continued to be suppressed from PCP's RM queue (however, any physician could access a patient's test result(s) directly from the Partners CDR). Inpatient physicians could track laboratory and radiology results finalized after discharge by keeping discharged patients on their HRM watch list for a designated period of time. The finalized results would become available for review in their HRM queue and abnormal results were displayed prominently at the top of this queue. Inpatient physicians were trained to use HRM in a series of meetings and demonstrations. Although HRM could be accessed from inpatient clinical workstations, it was not part of the inpatient clinical information system.

Results

A total of 35 inpatient physicians participated in the pilot. Among 649 patients discharged during the study period, there were 1075 tests pending of which 555 were subsequently flagged as abnormal in HRM. Study surveys were sent to the 35 inpatient physician participants and 29 were completed, including partial responses (83% survey response rate). The 35 inpatient physician participants had the following characteristics: 22 were male, 13 were female; 21 were trainees and 14 were nontrainees/faculty. All 21 trainees were PGY2s. Nontrainees and faculty varied in experience level (PGY 15: 5, PGY 610: 7, PGY 1120: 1, PGY 21+: 1). Of 29 survey respondents, 7 were from hospital A and 22 were from hospital B; 19 were trainees and 10 were nontrainees/faculty. Of the 6 nonrespondents, 2 were from hospital A and 4 were from hospital B; 2 were trainees and 4 were nontrainees/faculty.

Table 1 shows the results of our survey of inpatient physicians regarding usage of HRM. Of 29 survey respondents, 14 (48%) reported never using HRM. Thirteen (45%) reported using HRM 1 to 2 times per week. None of the respondents used it more than 4 times per week. The frequency of usage was similar for hospitals A and B. Table 2 details barriers to using HRM. Twenty‐three inpatient physicians (79%) reported barriers. Seventeen (59%) thought that results in their HRM queue were not clinically relevant, 16 (55%) felt that HRM did not fit into their daily workflow, 14 (48%) had limited time to use HRM, and 12 (41%) noted that too many results in their HRM queue were on other physician's patients. Seven (24%) reported operational issues and 3 (10%) reported technical issues prohibiting use of HRM. With regard to preferred elements of an ideal results manager system, 21 (72%) inpatient physician respondents wanted to receive notification of abnormal and clinician‐designated pending test results. Four (14%) wanted to receive only abnormal results and 1 (3%) wanted to receive all results. Twenty‐seven (93%) physicians agreed that an ideally designed computerized test result management application would be valuable for managing pending tests at discharge.

Table 3 provides comments from inpatient physician respondents regarding obstacles prohibiting use of HRM and suggestions for future systems.

Discussion

We describe a pilot implementation of a computerized application for the management of pending tests at hospital discharge. From responses to post‐implementation surveys, we were able to identify multiple factors prohibiting successful implementation of the application. These observations may help inform future interventions and evaluations.

Almost half of inpatient physicians reported never using HRM despite training and reminders. The feedback provided by physicians in our study suggested that HRM was not ideally designed from an inpatient physician perspective. We discovered several barriers to its use: (1) HRM overburdened physicians with clinically irrelevant test results, suggesting that more robust filtering of abnormal but low importance test results may be required (eg, a borderline electrolyte abnormality or low but stable hematocrit); (2) HRM did not integrate well into inpatient workflowthe system was not integrated into the inpatient results viewing and computerized physician order entry (CPOE) applications, and therefore required an extra step to access; (3) there was no mechanism of alerting inpatient physicians that finalized test results were available for viewing in their HRM queues (eg, by email or by an alert in the inpatient computer system); (4) because responsibility for these results was unclear, most inpatient physicians had no formal method of managing them, and for many, using HRM represented an additional task; and finally (5) several physicians commented on finding results in their HRM queue that belonged to other physician's patients, implying that the hospital databases were inaccurate in identifying the discharging physician or that rotation schedules, and therefore patient responsibility, had changed in the intervening period. Table 4 summarizes the advantages and respective limitations of features of HRM available to inpatient physicians.

In the literature, there is little information regarding optimal features of a test result management system for transitions from the inpatient to ambulatory care setting. Prior studies outline important functions for results management systems developed for noninpatient sites of care, including the ambulatory and emergency room setting.12, 14, 15 These include a method of prioritizing by degree of abnormality, the ability to reliably and efficiently act upon results, and an automated alerting system for abnormal results. Findings from our study provide insight in defining core functions for result management systems which focus on transitions from the inpatient to ambulatory care setting. These functions include tight integration with applications used by inpatient physicians, clear assignment of responsibility for test results finalized after hospital discharge (as well as a mechanism to reassign responsibility), automated alerts to responsible providers of test results finalized post‐discharge, and ways to automatically filter test results to avoid over‐burdening physicians with clinically irrelevant results.

Almost all surveyed inpatient physicians agreed that an ideally designed electronic post‐discharge results management system would be valuable. For such systems to be successfully adopted, we offer several principles to help guide future work. These include: (1) clarifying responsibility at the time a test is ordered and again at discharge, (2) understanding workflow and communication patterns among inpatient and outpatient clinicians, and (3) integrating technological solutions into existing systems to minimize workflow disruptions. For example, if the primary responsibility for post‐discharge result follow‐up lies with the ordering physician, the system should be integrated within the EMR most often used by inpatient physicians and become part of inpatient physician workflow. If the system depends on administrative databases to identify the responsible providers, these must be accurate. Alternatively, in organizations with computerized provider order entry, responsibility for the result could be assigned when the test is ordered and confirmed at discharge (ie, the results management system would be integrated into the discharge order such that pending tests are reviewed at the time of discharge). The discharging physician should have the ability to assign responsibility for each pending test and select preferred mode(s) of notification once its result is finalized (eg, e‐mail, alphanumeric page, etc.). The system should have the ability to generate an automatic notification to the inpatient and PCP (and perhaps other designated providers involved in the patient's inpatient care), but it should not burden busy clinicians with unnecessary alerts and warnings. Finally, the rules by which results are prioritized must be robust enough to filter out less urgent results, and should be modified to reflect the severity of illness of recently discharged patients. In essence, in consideration of the time constraints of busy clinicians, an ideal results management system should achieve automated notification of test results while minimizing the risk of alert fatigue from the potentially large volume of alerts generated.

Our study has several important limitations. First, although our survey response rate was high, the sample size of actual participants was small. Second, because the study was conducted in 2 similar, tertiary care academic centers, it may not generalize to other settings (we note that hospital B included a nonteaching service similar to those in nonacademic medical centers). This may be particularly true in assessing the importance of specific barriers to use of results management systems, which may vary at different institutions. Third, the representation of survey respondents were skeweda majority of the responses were from trainees (all post‐graduate level [PGY] level 2) and from hospital B. Fourth, we did not actively monitor physician interaction with the test result management application, and therefore, we depended heavily on physician recollection of use of the system when responding to surveys. Finally, we did not convene focus groups of key individuals with regard to the factors facilitating or prohibiting adoption of the system. Use of semi‐structured, key informant interviews (ie, focus groups) before and after implementation of an electronic results management application, have been shown to be effective in evaluating potential barriers and facilitators of adoption.16 Focus groups of and/or interviews with inpatient and PCPs, physician extenders, and housestaff could have been useful to better characterize the potential barriers and facilitators of adoption noted by survey respondents in our study.

In summary, we offer several lessons from our attempt to implement a system to manage pending tests at hospital discharge. The success of implementing future systems to address this patient safety concern will rely on accurately assigning responsibility for these test results, integrating the system within clinical information systems commonly used by the inpatient physician, addressing workflow issues and time constraints, maximizing appropriateness of alerting, and minimizing alert fatigue.

The period following discharge is a vulnerable time for patientsthe prevalence of medical errors related to this transition is high and has important patient safety and medico‐legal ramifications.13 Factors contributing to this vulnerability include complexity of hospitalized patients, shorter lengths of stay, and increased discontinuity of care. Hospitalists have recognized this threat to patient safety and have worked toward improving information exchange between inpatient and outpatient providers at hospital discharge.46 Nonetheless, the evidence suggests that more work is necessary. A recent study found that discharge summaries are often incomplete, and do not contain important information requiring follow‐up, such as pending tests.7 Additionally, a review by Kripalani et al. characterizing information deficits at hospital discharge found few interventions which specifically improve communication of pending tests at hospital discharge.8

In a prior study we determined that 41% of patients left the hospital before all laboratory and radiology test results were finalized. Of these results, 9.4% were potentially actionable and could have altered management. Physicians were aware of only 38% of post‐discharge test results.9 This awareness gap is a consequence of several factors including the lack of systems to track and alert providers of test results finalized post discharge. Also, it is unclear who is responsible for pending tests at discharge, since these tests are ordered by the inpatient physicians but often reported in the time period between hospital discharge and the patient's first follow‐up appointment with the primary care physician (PCP). Because responsibility is not explicitly made in the final communication between physicians at discharge, such test results may not be reviewed in a timely manner, potentially resulting in delays in treatment, a need for readmission, or other unfavorable outcomes.

Even in integrated health systems with advanced electronic health records, missed test results which result in treatment delays remain prevalent.10, 11 Test result management applications aid clinicians in reviewing and acting upon results as they become available and such systems may provide solutions to this problem. At Partners Healthcare in Boston, the Results Manager (RM) application was developed to help clinicians in the ambulatory setting safely, reliably, and efficiently review and act upon test results. The application enables clinicians to prioritize test results, utilize guidelines, and generate letters to patients. This system also prompts physicians to set reminders for future testing.12 In a 2.5‐year study evaluating the impact of this intervention, PCPs at 26 adult primary care practices were able to expedite communication of outpatient laboratory and imaging test results to patients with the help of RM. Patients of physicians who participated in the project reported greater satisfaction with test result communication and with information provided about their condition than did a control group of similar patients.13 RM has not yet been studied in the inpatient setting or at care transitions. We describe an attempt at modifying the Partners RM application to help inpatient physicians manage pending tests at hospital discharge.

Methods

Test Result Management Application

RM was originally developed by Partners Healthcare to improve timely review and appropriate management of test results in the ambulatory setting. RM was developed for and vetted by primarily ambulatory physicians. The application is browser‐based, provider‐centric, and embedded in the LMR to help ambulatory clinicians review and act upon test results in a safe, reliable, and efficient manner. Although RM has access to all inpatient and outpatient data in the Partners Clinical Data Repository (CDR), given the volume of inpatient tests ordered, hospital‐based results are suppressed by default to limit inundating ambulatory clinicians' queues. Therefore, users of RM only receive results of laboratory and radiology tests ordered in the ambulatory setting. They can track these tests for specific patients for a designated period of time by placing the patient on a watch list. Finally, RM incorporates extensive decision support features to classify the degree of abnormality for each result, presents guidelines to help clinicians manage abnormal results, allows clinicians to generate result letters to patients using predefined, context‐sensitive templates, and prompts physicians to set reminders for future testing. Because RM was developed from the ambulatory perspective, there was limited input from hospitalist physicians with regard to inpatient workflow in the original design of the module.12 See Figure 1 for a screen shot of RM and a description of its features.

Figure 1

For purposes of this pilot, we modified RM to allow results of tests ordered in the inpatient setting to be available for viewing (Hospitalist Results Manager, HRM). This feature was turned on only for inpatient physicians as previously defined. Inpatient tests, including pending tests at discharge, continued to be suppressed from PCP's RM queue (however, any physician could access a patient's test result(s) directly from the Partners CDR). Inpatient physicians could track laboratory and radiology results finalized after discharge by keeping discharged patients on their HRM watch list for a designated period of time. The finalized results would become available for review in their HRM queue and abnormal results were displayed prominently at the top of this queue. Inpatient physicians were trained to use HRM in a series of meetings and demonstrations. Although HRM could be accessed from inpatient clinical workstations, it was not part of the inpatient clinical information system.

Results

A total of 35 inpatient physicians participated in the pilot. Among 649 patients discharged during the study period, there were 1075 tests pending of which 555 were subsequently flagged as abnormal in HRM. Study surveys were sent to the 35 inpatient physician participants and 29 were completed, including partial responses (83% survey response rate). The 35 inpatient physician participants had the following characteristics: 22 were male, 13 were female; 21 were trainees and 14 were nontrainees/faculty. All 21 trainees were PGY2s. Nontrainees and faculty varied in experience level (PGY 15: 5, PGY 610: 7, PGY 1120: 1, PGY 21+: 1). Of 29 survey respondents, 7 were from hospital A and 22 were from hospital B; 19 were trainees and 10 were nontrainees/faculty. Of the 6 nonrespondents, 2 were from hospital A and 4 were from hospital B; 2 were trainees and 4 were nontrainees/faculty.

Table 1 shows the results of our survey of inpatient physicians regarding usage of HRM. Of 29 survey respondents, 14 (48%) reported never using HRM. Thirteen (45%) reported using HRM 1 to 2 times per week. None of the respondents used it more than 4 times per week. The frequency of usage was similar for hospitals A and B. Table 2 details barriers to using HRM. Twenty‐three inpatient physicians (79%) reported barriers. Seventeen (59%) thought that results in their HRM queue were not clinically relevant, 16 (55%) felt that HRM did not fit into their daily workflow, 14 (48%) had limited time to use HRM, and 12 (41%) noted that too many results in their HRM queue were on other physician's patients. Seven (24%) reported operational issues and 3 (10%) reported technical issues prohibiting use of HRM. With regard to preferred elements of an ideal results manager system, 21 (72%) inpatient physician respondents wanted to receive notification of abnormal and clinician‐designated pending test results. Four (14%) wanted to receive only abnormal results and 1 (3%) wanted to receive all results. Twenty‐seven (93%) physicians agreed that an ideally designed computerized test result management application would be valuable for managing pending tests at discharge.

Table 3 provides comments from inpatient physician respondents regarding obstacles prohibiting use of HRM and suggestions for future systems.

Discussion

We describe a pilot implementation of a computerized application for the management of pending tests at hospital discharge. From responses to post‐implementation surveys, we were able to identify multiple factors prohibiting successful implementation of the application. These observations may help inform future interventions and evaluations.

Almost half of inpatient physicians reported never using HRM despite training and reminders. The feedback provided by physicians in our study suggested that HRM was not ideally designed from an inpatient physician perspective. We discovered several barriers to its use: (1) HRM overburdened physicians with clinically irrelevant test results, suggesting that more robust filtering of abnormal but low importance test results may be required (eg, a borderline electrolyte abnormality or low but stable hematocrit); (2) HRM did not integrate well into inpatient workflowthe system was not integrated into the inpatient results viewing and computerized physician order entry (CPOE) applications, and therefore required an extra step to access; (3) there was no mechanism of alerting inpatient physicians that finalized test results were available for viewing in their HRM queues (eg, by email or by an alert in the inpatient computer system); (4) because responsibility for these results was unclear, most inpatient physicians had no formal method of managing them, and for many, using HRM represented an additional task; and finally (5) several physicians commented on finding results in their HRM queue that belonged to other physician's patients, implying that the hospital databases were inaccurate in identifying the discharging physician or that rotation schedules, and therefore patient responsibility, had changed in the intervening period. Table 4 summarizes the advantages and respective limitations of features of HRM available to inpatient physicians.

In the literature, there is little information regarding optimal features of a test result management system for transitions from the inpatient to ambulatory care setting. Prior studies outline important functions for results management systems developed for noninpatient sites of care, including the ambulatory and emergency room setting.12, 14, 15 These include a method of prioritizing by degree of abnormality, the ability to reliably and efficiently act upon results, and an automated alerting system for abnormal results. Findings from our study provide insight in defining core functions for result management systems which focus on transitions from the inpatient to ambulatory care setting. These functions include tight integration with applications used by inpatient physicians, clear assignment of responsibility for test results finalized after hospital discharge (as well as a mechanism to reassign responsibility), automated alerts to responsible providers of test results finalized post‐discharge, and ways to automatically filter test results to avoid over‐burdening physicians with clinically irrelevant results.

Almost all surveyed inpatient physicians agreed that an ideally designed electronic post‐discharge results management system would be valuable. For such systems to be successfully adopted, we offer several principles to help guide future work. These include: (1) clarifying responsibility at the time a test is ordered and again at discharge, (2) understanding workflow and communication patterns among inpatient and outpatient clinicians, and (3) integrating technological solutions into existing systems to minimize workflow disruptions. For example, if the primary responsibility for post‐discharge result follow‐up lies with the ordering physician, the system should be integrated within the EMR most often used by inpatient physicians and become part of inpatient physician workflow. If the system depends on administrative databases to identify the responsible providers, these must be accurate. Alternatively, in organizations with computerized provider order entry, responsibility for the result could be assigned when the test is ordered and confirmed at discharge (ie, the results management system would be integrated into the discharge order such that pending tests are reviewed at the time of discharge). The discharging physician should have the ability to assign responsibility for each pending test and select preferred mode(s) of notification once its result is finalized (eg, e‐mail, alphanumeric page, etc.). The system should have the ability to generate an automatic notification to the inpatient and PCP (and perhaps other designated providers involved in the patient's inpatient care), but it should not burden busy clinicians with unnecessary alerts and warnings. Finally, the rules by which results are prioritized must be robust enough to filter out less urgent results, and should be modified to reflect the severity of illness of recently discharged patients. In essence, in consideration of the time constraints of busy clinicians, an ideal results management system should achieve automated notification of test results while minimizing the risk of alert fatigue from the potentially large volume of alerts generated.

Our study has several important limitations. First, although our survey response rate was high, the sample size of actual participants was small. Second, because the study was conducted in 2 similar, tertiary care academic centers, it may not generalize to other settings (we note that hospital B included a nonteaching service similar to those in nonacademic medical centers). This may be particularly true in assessing the importance of specific barriers to use of results management systems, which may vary at different institutions. Third, the representation of survey respondents were skeweda majority of the responses were from trainees (all post‐graduate level [PGY] level 2) and from hospital B. Fourth, we did not actively monitor physician interaction with the test result management application, and therefore, we depended heavily on physician recollection of use of the system when responding to surveys. Finally, we did not convene focus groups of key individuals with regard to the factors facilitating or prohibiting adoption of the system. Use of semi‐structured, key informant interviews (ie, focus groups) before and after implementation of an electronic results management application, have been shown to be effective in evaluating potential barriers and facilitators of adoption.16 Focus groups of and/or interviews with inpatient and PCPs, physician extenders, and housestaff could have been useful to better characterize the potential barriers and facilitators of adoption noted by survey respondents in our study.

In summary, we offer several lessons from our attempt to implement a system to manage pending tests at hospital discharge. The success of implementing future systems to address this patient safety concern will rely on accurately assigning responsibility for these test results, integrating the system within clinical information systems commonly used by the inpatient physician, addressing workflow issues and time constraints, maximizing appropriateness of alerting, and minimizing alert fatigue.

The period following discharge is a vulnerable time for patientsthe prevalence of medical errors related to this transition is high and has important patient safety and medico‐legal ramifications.13 Factors contributing to this vulnerability include complexity of hospitalized patients, shorter lengths of stay, and increased discontinuity of care. Hospitalists have recognized this threat to patient safety and have worked toward improving information exchange between inpatient and outpatient providers at hospital discharge.46 Nonetheless, the evidence suggests that more work is necessary. A recent study found that discharge summaries are often incomplete, and do not contain important information requiring follow‐up, such as pending tests.7 Additionally, a review by Kripalani et al. characterizing information deficits at hospital discharge found few interventions which specifically improve communication of pending tests at hospital discharge.8

In a prior study we determined that 41% of patients left the hospital before all laboratory and radiology test results were finalized. Of these results, 9.4% were potentially actionable and could have altered management. Physicians were aware of only 38% of post‐discharge test results.9 This awareness gap is a consequence of several factors including the lack of systems to track and alert providers of test results finalized post discharge. Also, it is unclear who is responsible for pending tests at discharge, since these tests are ordered by the inpatient physicians but often reported in the time period between hospital discharge and the patient's first follow‐up appointment with the primary care physician (PCP). Because responsibility is not explicitly made in the final communication between physicians at discharge, such test results may not be reviewed in a timely manner, potentially resulting in delays in treatment, a need for readmission, or other unfavorable outcomes.

Even in integrated health systems with advanced electronic health records, missed test results which result in treatment delays remain prevalent.10, 11 Test result management applications aid clinicians in reviewing and acting upon results as they become available and such systems may provide solutions to this problem. At Partners Healthcare in Boston, the Results Manager (RM) application was developed to help clinicians in the ambulatory setting safely, reliably, and efficiently review and act upon test results. The application enables clinicians to prioritize test results, utilize guidelines, and generate letters to patients. This system also prompts physicians to set reminders for future testing.12 In a 2.5‐year study evaluating the impact of this intervention, PCPs at 26 adult primary care practices were able to expedite communication of outpatient laboratory and imaging test results to patients with the help of RM. Patients of physicians who participated in the project reported greater satisfaction with test result communication and with information provided about their condition than did a control group of similar patients.13 RM has not yet been studied in the inpatient setting or at care transitions. We describe an attempt at modifying the Partners RM application to help inpatient physicians manage pending tests at hospital discharge.

Methods

Test Result Management Application

RM was originally developed by Partners Healthcare to improve timely review and appropriate management of test results in the ambulatory setting. RM was developed for and vetted by primarily ambulatory physicians. The application is browser‐based, provider‐centric, and embedded in the LMR to help ambulatory clinicians review and act upon test results in a safe, reliable, and efficient manner. Although RM has access to all inpatient and outpatient data in the Partners Clinical Data Repository (CDR), given the volume of inpatient tests ordered, hospital‐based results are suppressed by default to limit inundating ambulatory clinicians' queues. Therefore, users of RM only receive results of laboratory and radiology tests ordered in the ambulatory setting. They can track these tests for specific patients for a designated period of time by placing the patient on a watch list. Finally, RM incorporates extensive decision support features to classify the degree of abnormality for each result, presents guidelines to help clinicians manage abnormal results, allows clinicians to generate result letters to patients using predefined, context‐sensitive templates, and prompts physicians to set reminders for future testing. Because RM was developed from the ambulatory perspective, there was limited input from hospitalist physicians with regard to inpatient workflow in the original design of the module.12 See Figure 1 for a screen shot of RM and a description of its features.

Figure 1

For purposes of this pilot, we modified RM to allow results of tests ordered in the inpatient setting to be available for viewing (Hospitalist Results Manager, HRM). This feature was turned on only for inpatient physicians as previously defined. Inpatient tests, including pending tests at discharge, continued to be suppressed from PCP's RM queue (however, any physician could access a patient's test result(s) directly from the Partners CDR). Inpatient physicians could track laboratory and radiology results finalized after discharge by keeping discharged patients on their HRM watch list for a designated period of time. The finalized results would become available for review in their HRM queue and abnormal results were displayed prominently at the top of this queue. Inpatient physicians were trained to use HRM in a series of meetings and demonstrations. Although HRM could be accessed from inpatient clinical workstations, it was not part of the inpatient clinical information system.

Results

A total of 35 inpatient physicians participated in the pilot. Among 649 patients discharged during the study period, there were 1075 tests pending of which 555 were subsequently flagged as abnormal in HRM. Study surveys were sent to the 35 inpatient physician participants and 29 were completed, including partial responses (83% survey response rate). The 35 inpatient physician participants had the following characteristics: 22 were male, 13 were female; 21 were trainees and 14 were nontrainees/faculty. All 21 trainees were PGY2s. Nontrainees and faculty varied in experience level (PGY 15: 5, PGY 610: 7, PGY 1120: 1, PGY 21+: 1). Of 29 survey respondents, 7 were from hospital A and 22 were from hospital B; 19 were trainees and 10 were nontrainees/faculty. Of the 6 nonrespondents, 2 were from hospital A and 4 were from hospital B; 2 were trainees and 4 were nontrainees/faculty.

Table 1 shows the results of our survey of inpatient physicians regarding usage of HRM. Of 29 survey respondents, 14 (48%) reported never using HRM. Thirteen (45%) reported using HRM 1 to 2 times per week. None of the respondents used it more than 4 times per week. The frequency of usage was similar for hospitals A and B. Table 2 details barriers to using HRM. Twenty‐three inpatient physicians (79%) reported barriers. Seventeen (59%) thought that results in their HRM queue were not clinically relevant, 16 (55%) felt that HRM did not fit into their daily workflow, 14 (48%) had limited time to use HRM, and 12 (41%) noted that too many results in their HRM queue were on other physician's patients. Seven (24%) reported operational issues and 3 (10%) reported technical issues prohibiting use of HRM. With regard to preferred elements of an ideal results manager system, 21 (72%) inpatient physician respondents wanted to receive notification of abnormal and clinician‐designated pending test results. Four (14%) wanted to receive only abnormal results and 1 (3%) wanted to receive all results. Twenty‐seven (93%) physicians agreed that an ideally designed computerized test result management application would be valuable for managing pending tests at discharge.

Table 3 provides comments from inpatient physician respondents regarding obstacles prohibiting use of HRM and suggestions for future systems.

Discussion

We describe a pilot implementation of a computerized application for the management of pending tests at hospital discharge. From responses to post‐implementation surveys, we were able to identify multiple factors prohibiting successful implementation of the application. These observations may help inform future interventions and evaluations.

Almost half of inpatient physicians reported never using HRM despite training and reminders. The feedback provided by physicians in our study suggested that HRM was not ideally designed from an inpatient physician perspective. We discovered several barriers to its use: (1) HRM overburdened physicians with clinically irrelevant test results, suggesting that more robust filtering of abnormal but low importance test results may be required (eg, a borderline electrolyte abnormality or low but stable hematocrit); (2) HRM did not integrate well into inpatient workflowthe system was not integrated into the inpatient results viewing and computerized physician order entry (CPOE) applications, and therefore required an extra step to access; (3) there was no mechanism of alerting inpatient physicians that finalized test results were available for viewing in their HRM queues (eg, by email or by an alert in the inpatient computer system); (4) because responsibility for these results was unclear, most inpatient physicians had no formal method of managing them, and for many, using HRM represented an additional task; and finally (5) several physicians commented on finding results in their HRM queue that belonged to other physician's patients, implying that the hospital databases were inaccurate in identifying the discharging physician or that rotation schedules, and therefore patient responsibility, had changed in the intervening period. Table 4 summarizes the advantages and respective limitations of features of HRM available to inpatient physicians.

In the literature, there is little information regarding optimal features of a test result management system for transitions from the inpatient to ambulatory care setting. Prior studies outline important functions for results management systems developed for noninpatient sites of care, including the ambulatory and emergency room setting.12, 14, 15 These include a method of prioritizing by degree of abnormality, the ability to reliably and efficiently act upon results, and an automated alerting system for abnormal results. Findings from our study provide insight in defining core functions for result management systems which focus on transitions from the inpatient to ambulatory care setting. These functions include tight integration with applications used by inpatient physicians, clear assignment of responsibility for test results finalized after hospital discharge (as well as a mechanism to reassign responsibility), automated alerts to responsible providers of test results finalized post‐discharge, and ways to automatically filter test results to avoid over‐burdening physicians with clinically irrelevant results.

Almost all surveyed inpatient physicians agreed that an ideally designed electronic post‐discharge results management system would be valuable. For such systems to be successfully adopted, we offer several principles to help guide future work. These include: (1) clarifying responsibility at the time a test is ordered and again at discharge, (2) understanding workflow and communication patterns among inpatient and outpatient clinicians, and (3) integrating technological solutions into existing systems to minimize workflow disruptions. For example, if the primary responsibility for post‐discharge result follow‐up lies with the ordering physician, the system should be integrated within the EMR most often used by inpatient physicians and become part of inpatient physician workflow. If the system depends on administrative databases to identify the responsible providers, these must be accurate. Alternatively, in organizations with computerized provider order entry, responsibility for the result could be assigned when the test is ordered and confirmed at discharge (ie, the results management system would be integrated into the discharge order such that pending tests are reviewed at the time of discharge). The discharging physician should have the ability to assign responsibility for each pending test and select preferred mode(s) of notification once its result is finalized (eg, e‐mail, alphanumeric page, etc.). The system should have the ability to generate an automatic notification to the inpatient and PCP (and perhaps other designated providers involved in the patient's inpatient care), but it should not burden busy clinicians with unnecessary alerts and warnings. Finally, the rules by which results are prioritized must be robust enough to filter out less urgent results, and should be modified to reflect the severity of illness of recently discharged patients. In essence, in consideration of the time constraints of busy clinicians, an ideal results management system should achieve automated notification of test results while minimizing the risk of alert fatigue from the potentially large volume of alerts generated.

Our study has several important limitations. First, although our survey response rate was high, the sample size of actual participants was small. Second, because the study was conducted in 2 similar, tertiary care academic centers, it may not generalize to other settings (we note that hospital B included a nonteaching service similar to those in nonacademic medical centers). This may be particularly true in assessing the importance of specific barriers to use of results management systems, which may vary at different institutions. Third, the representation of survey respondents were skeweda majority of the responses were from trainees (all post‐graduate level [PGY] level 2) and from hospital B. Fourth, we did not actively monitor physician interaction with the test result management application, and therefore, we depended heavily on physician recollection of use of the system when responding to surveys. Finally, we did not convene focus groups of key individuals with regard to the factors facilitating or prohibiting adoption of the system. Use of semi‐structured, key informant interviews (ie, focus groups) before and after implementation of an electronic results management application, have been shown to be effective in evaluating potential barriers and facilitators of adoption.16 Focus groups of and/or interviews with inpatient and PCPs, physician extenders, and housestaff could have been useful to better characterize the potential barriers and facilitators of adoption noted by survey respondents in our study.

In summary, we offer several lessons from our attempt to implement a system to manage pending tests at hospital discharge. The success of implementing future systems to address this patient safety concern will rely on accurately assigning responsibility for these test results, integrating the system within clinical information systems commonly used by the inpatient physician, addressing workflow issues and time constraints, maximizing appropriateness of alerting, and minimizing alert fatigue.