Program Profile

Increasing complexities within health care systems are significant impediments to the consistent delivery of safe and effective patient care. These impediments include an increase in specialization of care, staff shortages, burnout, poor coordination of services and access to care, as well as rising costs.¹ High reliability organizations (HROs) provide safe, high-quality, and effective care in highly complex and risk-prone environments without causing harm or experiencing catastrophic events.²

Within the US Department of Veterans Affairs (VA), the Veterans Health Administration (VHA) operates the nation’s largest integrated health care system, providing care to > 9 million veterans. The VHA formally launched plans for an enterprise-wide HRO in February 2019. During the first year, 18 medical facilities comprised cohort1 of the journey to high reliability. Cohort 2 began in October 2020 and consisted of 54 facilities. Cohort 3 started in October 2021 with 67 facilities.³

Health care organizations seeking high reliability exercise a philosophy aimed at learning from errors and addressing system failures. High reliability is accomplished by implementing 5 principles: (1) sensitivity to operations (a heightened understanding of the current state of systems); (2) preoccupation with failure (striving to anticipate risks that might suggest a much larger system problem); (3) reluctance to simplify (avoiding making any assumptions regarding the causes of failures); (4) commitment to resilience (preparing for potential failures and bouncing back when they occur); and (5) deference to expertise (deferring to individuals with the skills and proficiency to make the best decisions).² The VHA also recognized that a successful journey to high reliability—in addition to achieving a culture of safety—relies on the implementation of foundational HRO practices: leader rounding, visual management systems, safety forums, and safety huddles. This article describes an initiative for how these foundational practices were implemented in a large integrated health care system.

BACKGROUND

The VHA has focused on 4 foundational components as part of its enterprise activities and support structure to implement HRO principles and practices. These components were selected based on pilot activities that preceded the enterprise-wide effort, reviews of the literature, and expert consultation with both government and private sector health systems. To support the implementation of these practices, the VHA provided training, toolkits, HRO executive leader coaching, and peer-to-peer mentoring. As the VHA enters its fifth year seeking high reliability, we undertook an initiative to reflect on our own experiences and refine our practices based on an updated literature review.

As part of this enterprise-wide initiative, we conducted a literature review from 2018 to March 2023 seeking recent evidence describing the value of implementing the 4 foundational HRO practices to advance high reliability and improve patient safety. A 5-year period was used to ensure recency and value of evidence.

Eligible literature was identified in PubMed, PsycINFO, the Cumulative Index to Nursing and Allied Health Literature, ScienceDirect, Scopus, the Cochrane Library, and ProQuest Dissertations & Theses Global. Inclusion and exclusion criteria were peer-reviewed interdisciplinary documents(eg, publications, dissertations, conference proceedings, and grey literature) written in English. Search terms included high reliability organizations, foundational practices, and patient safety. Boolean operators (AND, OR) were also used in the search. The search resulted in a dearth of evidence that addressed implementation of all 4 foundational practices across a health care system. Retrieved evidence focused on the implementation of only 1 particular foundational practice in a specific health care setting. In addition to describing the formal processes for the implementation of each foundational HRO practice, a brief description of representative examples of strong practices within the VHA is provided.

To support the implementation of HROs, the VHA paired HRO executive leader coaches with select medical center directors and their leadership teams. Executive leader coaches also support an organization’s HRO Lead and HRO Champion. The HRO Lead coordinates and facilitates the implementation of HRO principles and practices in pursuit of no harm across an organization. The HRO Champion supports the same as the HRO Lead, but typically has a different specialty background. For example, if the HRO Lead has an administrative background, the HRO Champion would have a clinical background.

Coaching focuses heavily on supporting site-specific implementation and sustainment of the 4 HRO foundational practices. The aim is to accelerate change, build enduring capacity, foster a safety culture, and accelerate HRO maturity. To measure change, HRO executive leader coaches track the progress of their aligned VA medical centers (VAMCs) using the Organizational Learning Tool (OLT). This tool was developed to provide information such as a facility summary and relationships between a medical center director, HRO Lead, HRO Champion, and the executive leader coach (Figure 1). The OLT also serves as a structured process to measure leader coaching performance against mutually agreed upon objectives that ultimately contribute to enterprise outcomes. It also collects data on the progress in implementing foundational practices, strong practices, needs and gaps, and more (Figure 2). Data collected from facilities supported by HRO executive leader coaches on whether foundational practices are in place are briefly described.

Leader Rounding

Leader rounding for high reliability ensures effective, bidirectional communication and collaboration among all disciplines to improve patient safety. It is an essential feature of a robust patient safety culture and an important method for demonstrating leadership engagement with high reliability.^4,5 These rounds are conducted by organizational leadership (eg, executive teams, department/service chiefs, or unit managers) and frontline staff from different areas. They are specifically focused on high reliability, patient and staff safety, and improvement efforts. The aim is to learn about daily challenges that may contribute to patient harm.⁴

Leader rounding has been found to be highly effective at improving leadership visibility across the organization. It enhances interaction and open communication with frontline staff, fostering leader-staff collaboration and shared decision-making,as well as promoting leadership understanding of operational, clinical, nonclinical (eg, administrative, nutrition services, or facilities management), and patient/family experience issues.⁴ Collaboration among team members fosters the delivery of more effective and efficient care, increases staff satisfaction, and improves employee retention.⁶ Leader rounding for high reliability significantly contributes to the breakdown of power barriers by giving team members voice and agency, ultimately leading to deeper engagement.⁷

It is important that leader rounding for high reliability occurs as planned and when possible, scheduled in advance. This helps to avoid rounding at peak times when care activities are being performed.^4,6 When scheduling conflicts arise, another leader should be sent to participate in rounds.⁴ Developing a list of questions in advance allows leadership to prepare messaging to share with staff as it relates to high reliability and patient safety (Table).^4,6,8

Closing the loop improves bidirectional communication and is critical to leader rounding for high reliability. Closed-loop communication and following up on and/or closing out issues raised during rounding empowers the sharing of information, which is critical for advancing a culture of safety.^4,8 Enhanced feedback is also associated with greater workforce engagement, staff feeling more connected to quality improvement activities, and lower rates of employee burnout.⁷ It is important to recognize that senior leaders are not responsible for resolving all issues. If a team or manager can resolve concerns that are raised, this should be encouraged and supported. Maintaining accountability at the lowest level of the organization promotes principles and practices of high reliability (Figure 3).^4,8

The VA Bedford Healthcare System created and implemented a strong practice for leader rounding for high reliability. This phased implementation involved creating an evidence-based process, deciding on an appropriate cadence, developing a tracking tool, and measuring impact to determine the overall effectiveness of leader rounding for high reliability.⁴

A visual management system (VMS) displays clinical and operational performance aligned with HRO goals and practices. It is used to view and guide discussions between interdisciplinary teams during tiered safety huddles, leader rounds for high reliability, and frontline staff on the current status and safety trends in a particular area.^8,9 A VMS is highly effective in creating an environment where all staff members, especially frontline workers, feel empowered to voice their concerns related to safety or to identify improvement opportunities.^8,10 Increased leader engagement in patient safety and heightened transparency of information associated with the use of a VMS improves staff morale and professional satisfaction.¹⁰

A VMS may be a dry-erase or whiteboard display, paper-based display, or electronic status board.⁸ VMSs are usually located in or near work settings (eg, nurses’ station, staff break room, or conference room).⁸ Although they can take different forms and display several types of information, a VMS should be easy to update and meet the specific needs of a work area. In the VHA, a VMS displays: (1) essential information for staff members to effectively perform their work; (2) improvement project ideas; (3) current work in progress; (4) tracking of implemented improvement activities; (5) strong practices that have been effective; and (6) staff recognition for those who have enhanced patient safety, including the reporting of close calls and near misses.

The VHA uses the MESS (methods, equipment, staffing, and supplies) VMS format. This format empowers staff to identify whether proper procedures and practices are in place, essential equipment and supplies are readily available in the quantity needed, and appropriate staffing is on hand to provide safe, high-quality patient care.⁸ Colored magnets are used as visual cues in a stoplight classification system to identify low or no safety risks (green), at risk (yellow), or high risk (red). Green coded issues are addressed locally by a manager or supervisor. Yellow coded concerns require increased staff and leadership vigilance. Red coded issues indicate that patient care would be impacted that day and therefore need to be immediately escalated and addressed with senior leaders to mitigate the threat.^4,11 Dayton VAMC successfully implemented a VMS, using both physical and electronic visual management boards. The Dayton VAMC VMS boards are closely tied to tiered safety huddles and leader rounding for high reliability.   

Safety Forums

Safety forums are another foundational practice of VHA health care organizations seeking high reliability. Recurring monthly, safety forums focus on reinforcing HRO principles and practices, safety programs, the importance and appreciation of reporting, and just culture. The emphasis on just culture reminds staff that adverse events in the organization are viewed as valuable learning opportunities to understand the factors leading to the situation as opposed to immediately assigning blame.¹²

Psychological safety is another important focus. When individuals feel psychologically safe, they are more likely to voice concerns and act without fear of reprisal, which supports a culture of safety.¹³ Safety forums are open to all members of the health care organization, including both clinical and nonclinical staff. Forums can be conducted by an HRO Lead, HRO Champion, Patient Safety Manager, or even executive leadership. Rotating the responsibility of leading these forums demonstrates that high reliability and safety are everyone’s responsibility.

Safety forums publicly review and discuss errors, adverse events, close calls, and near misses. Time is also spent discussing root cause analysis trends and highlighting continuous process improvement principles and current projects. During safety forums, leaders should recognize individuals for safety behaviors and reward reporting through a safety awards program.¹⁴ All forums should conclude with a question-and-answer session. Forums typically occur in virtual 30-minute sessionsbut can last up to 60 minutes when guest speakers attend and continuing education credit is offered.

The Jesse Brown VAMC in Chicago developed an interactive monthly safety forum appealing to a broad audience. Each forum is attended by about 200 staff members and includes leader engagement and panel discussions led by the chief medical officer, with topics on both patient and team safety connecting with HRO principles. A planning committee prepares guest speakers and offers continuing education credits.

Based on the processes of high reliability industries like aviation and nuclear power, tiered safety huddles have been increasingly adopted in health care. Huddles (health care, utilizing, deliberate, discussion,linking, and events) are department-level interdisciplinary meetings that last no more than 15 minutes.¹⁵ Their purpose is to improve communication by sharing day-to-day information across multiple disciplines, identify issues that may impact the delivery of care (eg, patient and staff safety concerns, staffing issues, or inadequate supplies) and resolve problems.

Tiered safety huddles are gaining popularity, especially in organizations seeking high reliability. They are more complex than traditional huddles because of the mechanics of elevating safety issues (eg, bedside to executive leadership teams), feedback loops, and sequencing, among other factors.^15,16

Tiered safety huddles are focused, transparent forums with multidisciplinary staff, including frontline workers, along with senior leadership.^15,16 When initially implemented, tiered safety huddles may take longer than the suggested 15 minutes; however, as teams become more experienced, huddles become more efficient.¹⁵ The goal of tiered safety huddles is to proactively identify, share, address, and resolve problems that have the potential to impact the delivery of safe and quality patient care. This may include addressing staffing shortfalls, inadequate allocation of supplies and equipment, operational issues, etc.^8,15 Critical to theeffective utilization of tiered safety huddles is the appropriate escalation of issues between tiers. The most critical issues are elevated to higher tiers so they are addressed by the most qualified person in the organization.

Deciding on the number of tiers typically depends on the size and scope of services provided by the health care organization or integrated system.For example, tiered huddles in the VHA originate at the point of service (eg, critical care unit). Tier 1 includes staff members at the unit/team level along with immediate supervisors/managers. Tier 2 involves departments and service lines (eg, pharmacy, podiatry, or internal medicine) including their respective leadership. Tier 3 is the executive leadership team. This process allows for bidirectional communication instead of the traditional hierarchical communication pathway (Figure 4). Issues identified that cannot be addressed at a particular tier are elevated to the next tier. Elevated issues typically involve systems or processes requiring attention and resolution by senior leadership.¹⁵ Tier 4 huddles at the Veterans Integrated Services Network level and Tier 5 huddles at the VHA Central Office level are being initiated. These additional levels will more effectively identify system-level risks and issues that may impact multiple VHA facilities and may be addressed through centralized functions and resources.

Tiered safety huddles have been found to be instrumental to ensuring the flow of information across organizations, improving multidisciplinary and leadership engagement and collaboration, as well as increasing accountability for safety.Tiered safety huddles increase situational awareness, which improves an organization’s ability to appropriately respond to safety concerns.Furthermore, tiered safety huddles enhance teamwork and interprofessional collaboration, and have been found to significantly increase the reporting of patient safety events.^15-19

The VA Connecticut Healthcare System tiered huddles followed a pilot testing implementation process. After receiving executive-level commitment, an evidence-based process was enacted, including staff education, selecting a VMS, determining tier interaction, and deciding on metrics to track.¹⁵

Implementing Foundational Practices

To examine the progress of the implementation of the 4 foundational HRO practices, quarterly metrics derived from the OLT are reviewed to determine whether each is being implemented and sustained. The OLT also tracks progress over time. For example, at the 27 cohort 2 and lead sites that initiated leader coaching in 2021 and continued through 2022, coaches observed a 27% increase in leader rounding for high reliability and a 46% increase in the use of VMSs. For the 66 cohort 3 sites that began leader coaching in 2022, coaches documented similar changes, ranging from a 40% increase in leader rounding for high reliability to a 66% increase in the use of safety forums. Additional data continue to be collected and analyzed to publish more comprehensive findings.

DISCUSSION

Incorporating leader rounding for high reliability, VMSs, safety forums, and tiered safety huddles into daily operations is critical to building and sustaining a robust culture of safety.⁸ The 4 foundational HRO practices are instrumental in providing psychologically safe forums for staff to share concerns and actively participate. These practices also promote continual, efficient bidirectional communication throughout organizational lines and across services. The increased visibility and transparency of leaders demonstrate the importance of fostering trust, enhancing closed-loop communication with issues that arise, and building momentum to achieve high reliability. The interconnectedness of the foundational HRO practices identified and implemented by the VHA helps foster teamwork and collaboration built on trust, respect, enthusiasm for improvement, and the delivery of exceptional patient care.

CONCLUSIONS

Incorporating the 4 foundational practices into daily operations is beneficial to the delivery of safe, high-quality health care. This effective and sustained application can strengthen a health care organization on its journey to high reliability and establishing a culture of safety. To be effective, these foundational practices should be personalized to support the unique circumstances of every health care environment. While the exact methodology by which organizations implement these practices may differ, they will help organizations approach patient safety in a more transparent and thoughtful manner.

Acknowledgments

The authors thank Aaron M. Sawyer, PhD, PMP, and Jessica Fankhauser, MA, for their unwavering administrative support, and Jeff Wright for exceptional graphic design support.

Increasing complexities within health care systems are significant impediments to the consistent delivery of safe and effective patient care. These impediments include an increase in specialization of care, staff shortages, burnout, poor coordination of services and access to care, as well as rising costs.¹ High reliability organizations (HROs) provide safe, high-quality, and effective care in highly complex and risk-prone environments without causing harm or experiencing catastrophic events.²

Within the US Department of Veterans Affairs (VA), the Veterans Health Administration (VHA) operates the nation’s largest integrated health care system, providing care to > 9 million veterans. The VHA formally launched plans for an enterprise-wide HRO in February 2019. During the first year, 18 medical facilities comprised cohort1 of the journey to high reliability. Cohort 2 began in October 2020 and consisted of 54 facilities. Cohort 3 started in October 2021 with 67 facilities.³

Health care organizations seeking high reliability exercise a philosophy aimed at learning from errors and addressing system failures. High reliability is accomplished by implementing 5 principles: (1) sensitivity to operations (a heightened understanding of the current state of systems); (2) preoccupation with failure (striving to anticipate risks that might suggest a much larger system problem); (3) reluctance to simplify (avoiding making any assumptions regarding the causes of failures); (4) commitment to resilience (preparing for potential failures and bouncing back when they occur); and (5) deference to expertise (deferring to individuals with the skills and proficiency to make the best decisions).² The VHA also recognized that a successful journey to high reliability—in addition to achieving a culture of safety—relies on the implementation of foundational HRO practices: leader rounding, visual management systems, safety forums, and safety huddles. This article describes an initiative for how these foundational practices were implemented in a large integrated health care system.

BACKGROUND

The VHA has focused on 4 foundational components as part of its enterprise activities and support structure to implement HRO principles and practices. These components were selected based on pilot activities that preceded the enterprise-wide effort, reviews of the literature, and expert consultation with both government and private sector health systems. To support the implementation of these practices, the VHA provided training, toolkits, HRO executive leader coaching, and peer-to-peer mentoring. As the VHA enters its fifth year seeking high reliability, we undertook an initiative to reflect on our own experiences and refine our practices based on an updated literature review.

As part of this enterprise-wide initiative, we conducted a literature review from 2018 to March 2023 seeking recent evidence describing the value of implementing the 4 foundational HRO practices to advance high reliability and improve patient safety. A 5-year period was used to ensure recency and value of evidence.

Eligible literature was identified in PubMed, PsycINFO, the Cumulative Index to Nursing and Allied Health Literature, ScienceDirect, Scopus, the Cochrane Library, and ProQuest Dissertations & Theses Global. Inclusion and exclusion criteria were peer-reviewed interdisciplinary documents(eg, publications, dissertations, conference proceedings, and grey literature) written in English. Search terms included high reliability organizations, foundational practices, and patient safety. Boolean operators (AND, OR) were also used in the search. The search resulted in a dearth of evidence that addressed implementation of all 4 foundational practices across a health care system. Retrieved evidence focused on the implementation of only 1 particular foundational practice in a specific health care setting. In addition to describing the formal processes for the implementation of each foundational HRO practice, a brief description of representative examples of strong practices within the VHA is provided.

To support the implementation of HROs, the VHA paired HRO executive leader coaches with select medical center directors and their leadership teams. Executive leader coaches also support an organization’s HRO Lead and HRO Champion. The HRO Lead coordinates and facilitates the implementation of HRO principles and practices in pursuit of no harm across an organization. The HRO Champion supports the same as the HRO Lead, but typically has a different specialty background. For example, if the HRO Lead has an administrative background, the HRO Champion would have a clinical background.

Coaching focuses heavily on supporting site-specific implementation and sustainment of the 4 HRO foundational practices. The aim is to accelerate change, build enduring capacity, foster a safety culture, and accelerate HRO maturity. To measure change, HRO executive leader coaches track the progress of their aligned VA medical centers (VAMCs) using the Organizational Learning Tool (OLT). This tool was developed to provide information such as a facility summary and relationships between a medical center director, HRO Lead, HRO Champion, and the executive leader coach (Figure 1). The OLT also serves as a structured process to measure leader coaching performance against mutually agreed upon objectives that ultimately contribute to enterprise outcomes. It also collects data on the progress in implementing foundational practices, strong practices, needs and gaps, and more (Figure 2). Data collected from facilities supported by HRO executive leader coaches on whether foundational practices are in place are briefly described.

Leader Rounding

Leader rounding for high reliability ensures effective, bidirectional communication and collaboration among all disciplines to improve patient safety. It is an essential feature of a robust patient safety culture and an important method for demonstrating leadership engagement with high reliability.^4,5 These rounds are conducted by organizational leadership (eg, executive teams, department/service chiefs, or unit managers) and frontline staff from different areas. They are specifically focused on high reliability, patient and staff safety, and improvement efforts. The aim is to learn about daily challenges that may contribute to patient harm.⁴

Leader rounding has been found to be highly effective at improving leadership visibility across the organization. It enhances interaction and open communication with frontline staff, fostering leader-staff collaboration and shared decision-making,as well as promoting leadership understanding of operational, clinical, nonclinical (eg, administrative, nutrition services, or facilities management), and patient/family experience issues.⁴ Collaboration among team members fosters the delivery of more effective and efficient care, increases staff satisfaction, and improves employee retention.⁶ Leader rounding for high reliability significantly contributes to the breakdown of power barriers by giving team members voice and agency, ultimately leading to deeper engagement.⁷

It is important that leader rounding for high reliability occurs as planned and when possible, scheduled in advance. This helps to avoid rounding at peak times when care activities are being performed.^4,6 When scheduling conflicts arise, another leader should be sent to participate in rounds.⁴ Developing a list of questions in advance allows leadership to prepare messaging to share with staff as it relates to high reliability and patient safety (Table).^4,6,8

Closing the loop improves bidirectional communication and is critical to leader rounding for high reliability. Closed-loop communication and following up on and/or closing out issues raised during rounding empowers the sharing of information, which is critical for advancing a culture of safety.^4,8 Enhanced feedback is also associated with greater workforce engagement, staff feeling more connected to quality improvement activities, and lower rates of employee burnout.⁷ It is important to recognize that senior leaders are not responsible for resolving all issues. If a team or manager can resolve concerns that are raised, this should be encouraged and supported. Maintaining accountability at the lowest level of the organization promotes principles and practices of high reliability (Figure 3).^4,8

The VA Bedford Healthcare System created and implemented a strong practice for leader rounding for high reliability. This phased implementation involved creating an evidence-based process, deciding on an appropriate cadence, developing a tracking tool, and measuring impact to determine the overall effectiveness of leader rounding for high reliability.⁴

A visual management system (VMS) displays clinical and operational performance aligned with HRO goals and practices. It is used to view and guide discussions between interdisciplinary teams during tiered safety huddles, leader rounds for high reliability, and frontline staff on the current status and safety trends in a particular area.^8,9 A VMS is highly effective in creating an environment where all staff members, especially frontline workers, feel empowered to voice their concerns related to safety or to identify improvement opportunities.^8,10 Increased leader engagement in patient safety and heightened transparency of information associated with the use of a VMS improves staff morale and professional satisfaction.¹⁰

A VMS may be a dry-erase or whiteboard display, paper-based display, or electronic status board.⁸ VMSs are usually located in or near work settings (eg, nurses’ station, staff break room, or conference room).⁸ Although they can take different forms and display several types of information, a VMS should be easy to update and meet the specific needs of a work area. In the VHA, a VMS displays: (1) essential information for staff members to effectively perform their work; (2) improvement project ideas; (3) current work in progress; (4) tracking of implemented improvement activities; (5) strong practices that have been effective; and (6) staff recognition for those who have enhanced patient safety, including the reporting of close calls and near misses.

The VHA uses the MESS (methods, equipment, staffing, and supplies) VMS format. This format empowers staff to identify whether proper procedures and practices are in place, essential equipment and supplies are readily available in the quantity needed, and appropriate staffing is on hand to provide safe, high-quality patient care.⁸ Colored magnets are used as visual cues in a stoplight classification system to identify low or no safety risks (green), at risk (yellow), or high risk (red). Green coded issues are addressed locally by a manager or supervisor. Yellow coded concerns require increased staff and leadership vigilance. Red coded issues indicate that patient care would be impacted that day and therefore need to be immediately escalated and addressed with senior leaders to mitigate the threat.^4,11 Dayton VAMC successfully implemented a VMS, using both physical and electronic visual management boards. The Dayton VAMC VMS boards are closely tied to tiered safety huddles and leader rounding for high reliability.   

Safety Forums

Safety forums are another foundational practice of VHA health care organizations seeking high reliability. Recurring monthly, safety forums focus on reinforcing HRO principles and practices, safety programs, the importance and appreciation of reporting, and just culture. The emphasis on just culture reminds staff that adverse events in the organization are viewed as valuable learning opportunities to understand the factors leading to the situation as opposed to immediately assigning blame.¹²

Psychological safety is another important focus. When individuals feel psychologically safe, they are more likely to voice concerns and act without fear of reprisal, which supports a culture of safety.¹³ Safety forums are open to all members of the health care organization, including both clinical and nonclinical staff. Forums can be conducted by an HRO Lead, HRO Champion, Patient Safety Manager, or even executive leadership. Rotating the responsibility of leading these forums demonstrates that high reliability and safety are everyone’s responsibility.

Safety forums publicly review and discuss errors, adverse events, close calls, and near misses. Time is also spent discussing root cause analysis trends and highlighting continuous process improvement principles and current projects. During safety forums, leaders should recognize individuals for safety behaviors and reward reporting through a safety awards program.¹⁴ All forums should conclude with a question-and-answer session. Forums typically occur in virtual 30-minute sessionsbut can last up to 60 minutes when guest speakers attend and continuing education credit is offered.

The Jesse Brown VAMC in Chicago developed an interactive monthly safety forum appealing to a broad audience. Each forum is attended by about 200 staff members and includes leader engagement and panel discussions led by the chief medical officer, with topics on both patient and team safety connecting with HRO principles. A planning committee prepares guest speakers and offers continuing education credits.

Based on the processes of high reliability industries like aviation and nuclear power, tiered safety huddles have been increasingly adopted in health care. Huddles (health care, utilizing, deliberate, discussion,linking, and events) are department-level interdisciplinary meetings that last no more than 15 minutes.¹⁵ Their purpose is to improve communication by sharing day-to-day information across multiple disciplines, identify issues that may impact the delivery of care (eg, patient and staff safety concerns, staffing issues, or inadequate supplies) and resolve problems.

Tiered safety huddles are gaining popularity, especially in organizations seeking high reliability. They are more complex than traditional huddles because of the mechanics of elevating safety issues (eg, bedside to executive leadership teams), feedback loops, and sequencing, among other factors.^15,16

Tiered safety huddles are focused, transparent forums with multidisciplinary staff, including frontline workers, along with senior leadership.^15,16 When initially implemented, tiered safety huddles may take longer than the suggested 15 minutes; however, as teams become more experienced, huddles become more efficient.¹⁵ The goal of tiered safety huddles is to proactively identify, share, address, and resolve problems that have the potential to impact the delivery of safe and quality patient care. This may include addressing staffing shortfalls, inadequate allocation of supplies and equipment, operational issues, etc.^8,15 Critical to theeffective utilization of tiered safety huddles is the appropriate escalation of issues between tiers. The most critical issues are elevated to higher tiers so they are addressed by the most qualified person in the organization.

Deciding on the number of tiers typically depends on the size and scope of services provided by the health care organization or integrated system.For example, tiered huddles in the VHA originate at the point of service (eg, critical care unit). Tier 1 includes staff members at the unit/team level along with immediate supervisors/managers. Tier 2 involves departments and service lines (eg, pharmacy, podiatry, or internal medicine) including their respective leadership. Tier 3 is the executive leadership team. This process allows for bidirectional communication instead of the traditional hierarchical communication pathway (Figure 4). Issues identified that cannot be addressed at a particular tier are elevated to the next tier. Elevated issues typically involve systems or processes requiring attention and resolution by senior leadership.¹⁵ Tier 4 huddles at the Veterans Integrated Services Network level and Tier 5 huddles at the VHA Central Office level are being initiated. These additional levels will more effectively identify system-level risks and issues that may impact multiple VHA facilities and may be addressed through centralized functions and resources.

Tiered safety huddles have been found to be instrumental to ensuring the flow of information across organizations, improving multidisciplinary and leadership engagement and collaboration, as well as increasing accountability for safety.Tiered safety huddles increase situational awareness, which improves an organization’s ability to appropriately respond to safety concerns.Furthermore, tiered safety huddles enhance teamwork and interprofessional collaboration, and have been found to significantly increase the reporting of patient safety events.^15-19

The VA Connecticut Healthcare System tiered huddles followed a pilot testing implementation process. After receiving executive-level commitment, an evidence-based process was enacted, including staff education, selecting a VMS, determining tier interaction, and deciding on metrics to track.¹⁵

Implementing Foundational Practices

To examine the progress of the implementation of the 4 foundational HRO practices, quarterly metrics derived from the OLT are reviewed to determine whether each is being implemented and sustained. The OLT also tracks progress over time. For example, at the 27 cohort 2 and lead sites that initiated leader coaching in 2021 and continued through 2022, coaches observed a 27% increase in leader rounding for high reliability and a 46% increase in the use of VMSs. For the 66 cohort 3 sites that began leader coaching in 2022, coaches documented similar changes, ranging from a 40% increase in leader rounding for high reliability to a 66% increase in the use of safety forums. Additional data continue to be collected and analyzed to publish more comprehensive findings.

DISCUSSION

Incorporating leader rounding for high reliability, VMSs, safety forums, and tiered safety huddles into daily operations is critical to building and sustaining a robust culture of safety.⁸ The 4 foundational HRO practices are instrumental in providing psychologically safe forums for staff to share concerns and actively participate. These practices also promote continual, efficient bidirectional communication throughout organizational lines and across services. The increased visibility and transparency of leaders demonstrate the importance of fostering trust, enhancing closed-loop communication with issues that arise, and building momentum to achieve high reliability. The interconnectedness of the foundational HRO practices identified and implemented by the VHA helps foster teamwork and collaboration built on trust, respect, enthusiasm for improvement, and the delivery of exceptional patient care.

CONCLUSIONS

Incorporating the 4 foundational practices into daily operations is beneficial to the delivery of safe, high-quality health care. This effective and sustained application can strengthen a health care organization on its journey to high reliability and establishing a culture of safety. To be effective, these foundational practices should be personalized to support the unique circumstances of every health care environment. While the exact methodology by which organizations implement these practices may differ, they will help organizations approach patient safety in a more transparent and thoughtful manner.

Acknowledgments

The authors thank Aaron M. Sawyer, PhD, PMP, and Jessica Fankhauser, MA, for their unwavering administrative support, and Jeff Wright for exceptional graphic design support.

Increasing complexities within health care systems are significant impediments to the consistent delivery of safe and effective patient care. These impediments include an increase in specialization of care, staff shortages, burnout, poor coordination of services and access to care, as well as rising costs.¹ High reliability organizations (HROs) provide safe, high-quality, and effective care in highly complex and risk-prone environments without causing harm or experiencing catastrophic events.²

Within the US Department of Veterans Affairs (VA), the Veterans Health Administration (VHA) operates the nation’s largest integrated health care system, providing care to > 9 million veterans. The VHA formally launched plans for an enterprise-wide HRO in February 2019. During the first year, 18 medical facilities comprised cohort1 of the journey to high reliability. Cohort 2 began in October 2020 and consisted of 54 facilities. Cohort 3 started in October 2021 with 67 facilities.³

Health care organizations seeking high reliability exercise a philosophy aimed at learning from errors and addressing system failures. High reliability is accomplished by implementing 5 principles: (1) sensitivity to operations (a heightened understanding of the current state of systems); (2) preoccupation with failure (striving to anticipate risks that might suggest a much larger system problem); (3) reluctance to simplify (avoiding making any assumptions regarding the causes of failures); (4) commitment to resilience (preparing for potential failures and bouncing back when they occur); and (5) deference to expertise (deferring to individuals with the skills and proficiency to make the best decisions).² The VHA also recognized that a successful journey to high reliability—in addition to achieving a culture of safety—relies on the implementation of foundational HRO practices: leader rounding, visual management systems, safety forums, and safety huddles. This article describes an initiative for how these foundational practices were implemented in a large integrated health care system.

BACKGROUND

The VHA has focused on 4 foundational components as part of its enterprise activities and support structure to implement HRO principles and practices. These components were selected based on pilot activities that preceded the enterprise-wide effort, reviews of the literature, and expert consultation with both government and private sector health systems. To support the implementation of these practices, the VHA provided training, toolkits, HRO executive leader coaching, and peer-to-peer mentoring. As the VHA enters its fifth year seeking high reliability, we undertook an initiative to reflect on our own experiences and refine our practices based on an updated literature review.

As part of this enterprise-wide initiative, we conducted a literature review from 2018 to March 2023 seeking recent evidence describing the value of implementing the 4 foundational HRO practices to advance high reliability and improve patient safety. A 5-year period was used to ensure recency and value of evidence.

Eligible literature was identified in PubMed, PsycINFO, the Cumulative Index to Nursing and Allied Health Literature, ScienceDirect, Scopus, the Cochrane Library, and ProQuest Dissertations & Theses Global. Inclusion and exclusion criteria were peer-reviewed interdisciplinary documents(eg, publications, dissertations, conference proceedings, and grey literature) written in English. Search terms included high reliability organizations, foundational practices, and patient safety. Boolean operators (AND, OR) were also used in the search. The search resulted in a dearth of evidence that addressed implementation of all 4 foundational practices across a health care system. Retrieved evidence focused on the implementation of only 1 particular foundational practice in a specific health care setting. In addition to describing the formal processes for the implementation of each foundational HRO practice, a brief description of representative examples of strong practices within the VHA is provided.

To support the implementation of HROs, the VHA paired HRO executive leader coaches with select medical center directors and their leadership teams. Executive leader coaches also support an organization’s HRO Lead and HRO Champion. The HRO Lead coordinates and facilitates the implementation of HRO principles and practices in pursuit of no harm across an organization. The HRO Champion supports the same as the HRO Lead, but typically has a different specialty background. For example, if the HRO Lead has an administrative background, the HRO Champion would have a clinical background.

Coaching focuses heavily on supporting site-specific implementation and sustainment of the 4 HRO foundational practices. The aim is to accelerate change, build enduring capacity, foster a safety culture, and accelerate HRO maturity. To measure change, HRO executive leader coaches track the progress of their aligned VA medical centers (VAMCs) using the Organizational Learning Tool (OLT). This tool was developed to provide information such as a facility summary and relationships between a medical center director, HRO Lead, HRO Champion, and the executive leader coach (Figure 1). The OLT also serves as a structured process to measure leader coaching performance against mutually agreed upon objectives that ultimately contribute to enterprise outcomes. It also collects data on the progress in implementing foundational practices, strong practices, needs and gaps, and more (Figure 2). Data collected from facilities supported by HRO executive leader coaches on whether foundational practices are in place are briefly described.

Leader Rounding

Leader rounding for high reliability ensures effective, bidirectional communication and collaboration among all disciplines to improve patient safety. It is an essential feature of a robust patient safety culture and an important method for demonstrating leadership engagement with high reliability.^4,5 These rounds are conducted by organizational leadership (eg, executive teams, department/service chiefs, or unit managers) and frontline staff from different areas. They are specifically focused on high reliability, patient and staff safety, and improvement efforts. The aim is to learn about daily challenges that may contribute to patient harm.⁴

Leader rounding has been found to be highly effective at improving leadership visibility across the organization. It enhances interaction and open communication with frontline staff, fostering leader-staff collaboration and shared decision-making,as well as promoting leadership understanding of operational, clinical, nonclinical (eg, administrative, nutrition services, or facilities management), and patient/family experience issues.⁴ Collaboration among team members fosters the delivery of more effective and efficient care, increases staff satisfaction, and improves employee retention.⁶ Leader rounding for high reliability significantly contributes to the breakdown of power barriers by giving team members voice and agency, ultimately leading to deeper engagement.⁷

It is important that leader rounding for high reliability occurs as planned and when possible, scheduled in advance. This helps to avoid rounding at peak times when care activities are being performed.^4,6 When scheduling conflicts arise, another leader should be sent to participate in rounds.⁴ Developing a list of questions in advance allows leadership to prepare messaging to share with staff as it relates to high reliability and patient safety (Table).^4,6,8

Closing the loop improves bidirectional communication and is critical to leader rounding for high reliability. Closed-loop communication and following up on and/or closing out issues raised during rounding empowers the sharing of information, which is critical for advancing a culture of safety.^4,8 Enhanced feedback is also associated with greater workforce engagement, staff feeling more connected to quality improvement activities, and lower rates of employee burnout.⁷ It is important to recognize that senior leaders are not responsible for resolving all issues. If a team or manager can resolve concerns that are raised, this should be encouraged and supported. Maintaining accountability at the lowest level of the organization promotes principles and practices of high reliability (Figure 3).^4,8

The VA Bedford Healthcare System created and implemented a strong practice for leader rounding for high reliability. This phased implementation involved creating an evidence-based process, deciding on an appropriate cadence, developing a tracking tool, and measuring impact to determine the overall effectiveness of leader rounding for high reliability.⁴

A visual management system (VMS) displays clinical and operational performance aligned with HRO goals and practices. It is used to view and guide discussions between interdisciplinary teams during tiered safety huddles, leader rounds for high reliability, and frontline staff on the current status and safety trends in a particular area.^8,9 A VMS is highly effective in creating an environment where all staff members, especially frontline workers, feel empowered to voice their concerns related to safety or to identify improvement opportunities.^8,10 Increased leader engagement in patient safety and heightened transparency of information associated with the use of a VMS improves staff morale and professional satisfaction.¹⁰

A VMS may be a dry-erase or whiteboard display, paper-based display, or electronic status board.⁸ VMSs are usually located in or near work settings (eg, nurses’ station, staff break room, or conference room).⁸ Although they can take different forms and display several types of information, a VMS should be easy to update and meet the specific needs of a work area. In the VHA, a VMS displays: (1) essential information for staff members to effectively perform their work; (2) improvement project ideas; (3) current work in progress; (4) tracking of implemented improvement activities; (5) strong practices that have been effective; and (6) staff recognition for those who have enhanced patient safety, including the reporting of close calls and near misses.

The VHA uses the MESS (methods, equipment, staffing, and supplies) VMS format. This format empowers staff to identify whether proper procedures and practices are in place, essential equipment and supplies are readily available in the quantity needed, and appropriate staffing is on hand to provide safe, high-quality patient care.⁸ Colored magnets are used as visual cues in a stoplight classification system to identify low or no safety risks (green), at risk (yellow), or high risk (red). Green coded issues are addressed locally by a manager or supervisor. Yellow coded concerns require increased staff and leadership vigilance. Red coded issues indicate that patient care would be impacted that day and therefore need to be immediately escalated and addressed with senior leaders to mitigate the threat.^4,11 Dayton VAMC successfully implemented a VMS, using both physical and electronic visual management boards. The Dayton VAMC VMS boards are closely tied to tiered safety huddles and leader rounding for high reliability.   

Safety Forums

Safety forums are another foundational practice of VHA health care organizations seeking high reliability. Recurring monthly, safety forums focus on reinforcing HRO principles and practices, safety programs, the importance and appreciation of reporting, and just culture. The emphasis on just culture reminds staff that adverse events in the organization are viewed as valuable learning opportunities to understand the factors leading to the situation as opposed to immediately assigning blame.¹²

Psychological safety is another important focus. When individuals feel psychologically safe, they are more likely to voice concerns and act without fear of reprisal, which supports a culture of safety.¹³ Safety forums are open to all members of the health care organization, including both clinical and nonclinical staff. Forums can be conducted by an HRO Lead, HRO Champion, Patient Safety Manager, or even executive leadership. Rotating the responsibility of leading these forums demonstrates that high reliability and safety are everyone’s responsibility.

Safety forums publicly review and discuss errors, adverse events, close calls, and near misses. Time is also spent discussing root cause analysis trends and highlighting continuous process improvement principles and current projects. During safety forums, leaders should recognize individuals for safety behaviors and reward reporting through a safety awards program.¹⁴ All forums should conclude with a question-and-answer session. Forums typically occur in virtual 30-minute sessionsbut can last up to 60 minutes when guest speakers attend and continuing education credit is offered.

The Jesse Brown VAMC in Chicago developed an interactive monthly safety forum appealing to a broad audience. Each forum is attended by about 200 staff members and includes leader engagement and panel discussions led by the chief medical officer, with topics on both patient and team safety connecting with HRO principles. A planning committee prepares guest speakers and offers continuing education credits.

Based on the processes of high reliability industries like aviation and nuclear power, tiered safety huddles have been increasingly adopted in health care. Huddles (health care, utilizing, deliberate, discussion,linking, and events) are department-level interdisciplinary meetings that last no more than 15 minutes.¹⁵ Their purpose is to improve communication by sharing day-to-day information across multiple disciplines, identify issues that may impact the delivery of care (eg, patient and staff safety concerns, staffing issues, or inadequate supplies) and resolve problems.

Tiered safety huddles are gaining popularity, especially in organizations seeking high reliability. They are more complex than traditional huddles because of the mechanics of elevating safety issues (eg, bedside to executive leadership teams), feedback loops, and sequencing, among other factors.^15,16

Tiered safety huddles are focused, transparent forums with multidisciplinary staff, including frontline workers, along with senior leadership.^15,16 When initially implemented, tiered safety huddles may take longer than the suggested 15 minutes; however, as teams become more experienced, huddles become more efficient.¹⁵ The goal of tiered safety huddles is to proactively identify, share, address, and resolve problems that have the potential to impact the delivery of safe and quality patient care. This may include addressing staffing shortfalls, inadequate allocation of supplies and equipment, operational issues, etc.^8,15 Critical to theeffective utilization of tiered safety huddles is the appropriate escalation of issues between tiers. The most critical issues are elevated to higher tiers so they are addressed by the most qualified person in the organization.

Deciding on the number of tiers typically depends on the size and scope of services provided by the health care organization or integrated system.For example, tiered huddles in the VHA originate at the point of service (eg, critical care unit). Tier 1 includes staff members at the unit/team level along with immediate supervisors/managers. Tier 2 involves departments and service lines (eg, pharmacy, podiatry, or internal medicine) including their respective leadership. Tier 3 is the executive leadership team. This process allows for bidirectional communication instead of the traditional hierarchical communication pathway (Figure 4). Issues identified that cannot be addressed at a particular tier are elevated to the next tier. Elevated issues typically involve systems or processes requiring attention and resolution by senior leadership.¹⁵ Tier 4 huddles at the Veterans Integrated Services Network level and Tier 5 huddles at the VHA Central Office level are being initiated. These additional levels will more effectively identify system-level risks and issues that may impact multiple VHA facilities and may be addressed through centralized functions and resources.

Tiered safety huddles have been found to be instrumental to ensuring the flow of information across organizations, improving multidisciplinary and leadership engagement and collaboration, as well as increasing accountability for safety.Tiered safety huddles increase situational awareness, which improves an organization’s ability to appropriately respond to safety concerns.Furthermore, tiered safety huddles enhance teamwork and interprofessional collaboration, and have been found to significantly increase the reporting of patient safety events.^15-19

The VA Connecticut Healthcare System tiered huddles followed a pilot testing implementation process. After receiving executive-level commitment, an evidence-based process was enacted, including staff education, selecting a VMS, determining tier interaction, and deciding on metrics to track.¹⁵

Implementing Foundational Practices

To examine the progress of the implementation of the 4 foundational HRO practices, quarterly metrics derived from the OLT are reviewed to determine whether each is being implemented and sustained. The OLT also tracks progress over time. For example, at the 27 cohort 2 and lead sites that initiated leader coaching in 2021 and continued through 2022, coaches observed a 27% increase in leader rounding for high reliability and a 46% increase in the use of VMSs. For the 66 cohort 3 sites that began leader coaching in 2022, coaches documented similar changes, ranging from a 40% increase in leader rounding for high reliability to a 66% increase in the use of safety forums. Additional data continue to be collected and analyzed to publish more comprehensive findings.

DISCUSSION

Incorporating leader rounding for high reliability, VMSs, safety forums, and tiered safety huddles into daily operations is critical to building and sustaining a robust culture of safety.⁸ The 4 foundational HRO practices are instrumental in providing psychologically safe forums for staff to share concerns and actively participate. These practices also promote continual, efficient bidirectional communication throughout organizational lines and across services. The increased visibility and transparency of leaders demonstrate the importance of fostering trust, enhancing closed-loop communication with issues that arise, and building momentum to achieve high reliability. The interconnectedness of the foundational HRO practices identified and implemented by the VHA helps foster teamwork and collaboration built on trust, respect, enthusiasm for improvement, and the delivery of exceptional patient care.

CONCLUSIONS

Incorporating the 4 foundational practices into daily operations is beneficial to the delivery of safe, high-quality health care. This effective and sustained application can strengthen a health care organization on its journey to high reliability and establishing a culture of safety. To be effective, these foundational practices should be personalized to support the unique circumstances of every health care environment. While the exact methodology by which organizations implement these practices may differ, they will help organizations approach patient safety in a more transparent and thoughtful manner.

Acknowledgments

The authors thank Aaron M. Sawyer, PhD, PMP, and Jessica Fankhauser, MA, for their unwavering administrative support, and Jeff Wright for exceptional graphic design support.

The Olin E. Teague Veterans’ Center (OETVC) in Temple, Texas, is a teaching hospital with 189 beds that provides patients access to medical, surgical, and specialty care. In 2022, 116,359 veterans received care at OETVC and 5393 inpatient admissions were noted. The inpatient ward consists of 3 teaching teams staffed by an attending physician, a second-year internal medicine resident, and 2 to 3 interns while hospitalists staff the 3 nonteaching teams. OETVC residents receive training on both routine and complex medical problems.

Each day, teaching teams discharge patients. With the complexity of discharges, there is always a risk of patients not following up with their primary care physicians, potential issues with filling medications, confusion about new medication regiments, and even potential postdischarge questions. In 1990, Holloway and colleagues evaluated potential risk factors for readmission among veterans. This study found that discharge from a geriatrics or intermediate care bed, chronic disease diagnosis, ≥ 2 procedures performed, increasing age, and distance from a veterans affairs medical center were risk factors.¹

Several community hospital studies have evaluated readmission risk factors. One from 2000 noted that patients with more hospitalizations, lower mental health function, a diagnosis of chronic obstructive pulmonary disorder, and increased satisfaction with access to emergency care were associated with increased readmission in 90 days.² Due to the readmission risks, OETVC decided to construct a program that would help these patients successfully transition from inpatient to outpatient care while establishing means to discuss their care with a physician for reassurance and guidance.

TRANSITION OF CARE PROGRAM

Transition of care programs have been implemented and evaluated in many institutions. A 2017 systematic review of transition of care programs supported the use of tailored discharge planning and postdischarge phone calls to reduce hospital readmission, noting that 6 studies demonstrated a statistically significant reduction in 30-day readmission rate.³ Another study found that pharmacy involvement in the transition of care reduced medication-related problems following discharge.⁴

Program Goals

The foundational goal of our program was to bridge the gap between inpatient and outpatient medicine. We hoped to improve patient adherence with their discharge regimens, improve access to primary care physicians, and improve discharge follow-up. Since hospitalization can be overwhelming, we hoped to capture potential barriers to medical care postdischarge when patients return home while decreasing hospital readmissions. Our second- and third-year resident physicians spend as much time as needed going through the patient’s course of illness throughout their hospitalization and treatment plans to ensure their understanding and potential success.

This program benefits residents by providing medical education and patient communication opportunities. Residents must review the patient’s clinical trajectory before calling them. In this process, residents develop an understanding of routine and complex illness scripts, or pathways of common illnesses. They also prepare for potential questions about the hospitalization, new medications, and follow-up care. Lastly, residents can focus on communication skills. Without the time pressures of returning to a busy rotation, the residents spend as much time discussing the hospital course and ensuring patient understanding as needed.

At the beginning of each week, second- and third-year residents review the list of discharges from the 3 teaching teams. The list is generated by a medical service management analyst. The residents review patient records for inpatient services, laboratory results, medication changes, and proposed follow-up plans designed by the admission team prior to their phone call. The resident is also responsible for reviewing and reconciling discharge instructions and orders. Then, the resident calls the patient and reviews their hospitalization. If a patient does not answer, the resident leaves a voicemail that complies with the Health Insurance Portability and Accountability Act.

When patients answer the call, the resident follows a script (Appendix). Residents are encouraged to ask patients open-ended questions and address any new needs. They also discuss changes in symptoms, medications, functional status, and remind the patient about follow-up appointments. If imaging or specific orders were missed at discharge, the residents notify the chief resident, lead hospitalist, or deputy associate chief of staff for medical service. If additional laboratory tests need to be ordered, the resident devises a follow-up plan. If needed, specialty referrals can be placed. When residents feel there are multiple items that need to be addressed or if they notice any major concerns, they can recommend the patient present to the emergency department for evaluation. The chief resident, lead hospitalist, and deputy associate chair for medical service are available to assist with discussions about complex medical situations or new concerning symptoms. Residents document their encounters in the Computerized Patient Record System health record and any tests that need follow-up. This differs from the standard of care follow-up programs, which are conducted by primary care medicine nurses and do not fully discuss the hospitalization.

Implementation

This program was implemented as a 1-week elective for interested residents and part of the clinic rotation. The internal medicine medical service analyst pulls all discharges on Friday, which are then provided to the residents. The residents on rotation work through the discharges and find teaching team patients to follow up with and call.

Findings

Implementation of this program has yielded many benefits. The reminder of the importance of a primary care appointment has motivated patients to continue following up on an outpatient basis. Residents were also able to capture lapses in patient understanding. Residents could answer forgotten questions and help patients understand their admission pathology without time pressures. Residents have identified patients with hypoglycemia due to changed insulin regimens, set up specialist follow-up appointments, and provided additional education facilitating adherence. Additionally, several residents have expressed satisfaction with the ability to practice their communication skills. Others appreciated contributing to future patient successes.

While the focus on this article has been to share the program description, we have tabulated preliminary data. In January 2023, there were 239 internal medicine admissions; 158 admissions (66%) were teaching team patients, and 97 patients (61%) were called by a resident and spoken to regarding their care. There were 24 teaching team readmissions within 30 days, and 10 (42%) received a follow-up phone call. Eighty-one admitted patients were treated by nonteaching teams, 10 (12%) of whom were readmissions. Comparing 30-day readmission rates, 10 nonteaching team patients (12%), 10 teaching team patients (6.3%) who talk to a resident in the transition of care program were readmitted, and 24 teaching team patients who did not talk to a resident (10%) were readmitted.

The OETVC transition of care program was planned, formulated, and implemented without modeling after any other projects or institutions. This program aimed to utilize our residents as resources for patients.

Transition of care is defined as steps taken in a clinical encounter to assist with the coordination and continuity of patient care transferring between locations or levels of care.⁵ A 2018 study evaluating the utility of transition of care programs on adults aged ≥ 60 years found a reduction in rehospitalization rates, increased use of primary care services, and potential reduction in home health usage.⁶

In 2021, Johns Hopkins University School of Medicine implemented a program after polling residents and discovering their awareness of gaps in the transition of care.⁷ In 2002, pharmacists evaluated the impact of follow-up telephone calls to recently hospitalized patients. This group of pharmacists found that these calls were associated with increased patient satisfaction, resolution of medication-related problems and fewer emergency department returns.⁸

Our program differs from other transition of care programs in that resident physicians made the follow-up calls to patients. Residents could address all aspects of medical care, including new symptoms, new prescriptions, adverse events, and risk factors for readmission, or order new imaging and medications when appropriate. In the program, residents called all patients discharged after receiving care within their team. Calls were not based on risk assessments. The residents were able to speak with 61% of discharged patients. When readmission rates were compared between patients who received a resident follow-up phone call and those who did not, patients receiving the resident phone call were readmitted at a lower rate: 6.3% vs 10%, respectively.

While our data suggest a potential trend of decreased readmission, more follow-up over a longer period may be needed. We believe this program can benefit patients and our model can act as a template for other institutions interested in starting their own programs.

Challenges

Although our process is efficient, there have been some challenges. The discharge is created by the medical service management analyst and then sent to the chief resident, but there was concern that the list could be missed if either individual was unavailable. The chairperson for the department of medicine and their secretary are now involved in the process. To reduce unanswered telephone calls, residents use OETVC phones. Health Insurance Portability and Accountability Act-compliant voicemails providing a time for a follow-up call were implemented. As a result, veterans have answered their phones more regularly and are more aware of calls. Orders are generally placed by the chief resident, lead hospitalist, or chair of the medical service to ensure follow-up because residents are on rotation for 1 week at a time. Access to a physician also allows patients to discuss items unrelated to their hospitalization, introducing new symptoms, or situations requiring a resident to act with limited data.

CONCLUSIONS

The transition of care follow-up program described in this article may be beneficial for both internal medicine residents and patients. Second- and third-year residents are developing a better understanding of the trajectory of many illnesses and are given the opportunity to retrospectively analyze what they would do differently based on knowledge gained from their chart reviews. They are also given the opportunity to work on communication skills and explain courses of illnesses to patients in an easy-to-understand format without time constraints. Patients now have access to a physician following discharge to discuss any concerns with their hospitalization, condition, and follow-up. This program will continue to address barriers to care and adapt to improve the success of care transitions.

The Olin E. Teague Veterans’ Center (OETVC) in Temple, Texas, is a teaching hospital with 189 beds that provides patients access to medical, surgical, and specialty care. In 2022, 116,359 veterans received care at OETVC and 5393 inpatient admissions were noted. The inpatient ward consists of 3 teaching teams staffed by an attending physician, a second-year internal medicine resident, and 2 to 3 interns while hospitalists staff the 3 nonteaching teams. OETVC residents receive training on both routine and complex medical problems.

Each day, teaching teams discharge patients. With the complexity of discharges, there is always a risk of patients not following up with their primary care physicians, potential issues with filling medications, confusion about new medication regiments, and even potential postdischarge questions. In 1990, Holloway and colleagues evaluated potential risk factors for readmission among veterans. This study found that discharge from a geriatrics or intermediate care bed, chronic disease diagnosis, ≥ 2 procedures performed, increasing age, and distance from a veterans affairs medical center were risk factors.¹

Several community hospital studies have evaluated readmission risk factors. One from 2000 noted that patients with more hospitalizations, lower mental health function, a diagnosis of chronic obstructive pulmonary disorder, and increased satisfaction with access to emergency care were associated with increased readmission in 90 days.² Due to the readmission risks, OETVC decided to construct a program that would help these patients successfully transition from inpatient to outpatient care while establishing means to discuss their care with a physician for reassurance and guidance.

TRANSITION OF CARE PROGRAM

Transition of care programs have been implemented and evaluated in many institutions. A 2017 systematic review of transition of care programs supported the use of tailored discharge planning and postdischarge phone calls to reduce hospital readmission, noting that 6 studies demonstrated a statistically significant reduction in 30-day readmission rate.³ Another study found that pharmacy involvement in the transition of care reduced medication-related problems following discharge.⁴

Program Goals

The foundational goal of our program was to bridge the gap between inpatient and outpatient medicine. We hoped to improve patient adherence with their discharge regimens, improve access to primary care physicians, and improve discharge follow-up. Since hospitalization can be overwhelming, we hoped to capture potential barriers to medical care postdischarge when patients return home while decreasing hospital readmissions. Our second- and third-year resident physicians spend as much time as needed going through the patient’s course of illness throughout their hospitalization and treatment plans to ensure their understanding and potential success.

This program benefits residents by providing medical education and patient communication opportunities. Residents must review the patient’s clinical trajectory before calling them. In this process, residents develop an understanding of routine and complex illness scripts, or pathways of common illnesses. They also prepare for potential questions about the hospitalization, new medications, and follow-up care. Lastly, residents can focus on communication skills. Without the time pressures of returning to a busy rotation, the residents spend as much time discussing the hospital course and ensuring patient understanding as needed.

At the beginning of each week, second- and third-year residents review the list of discharges from the 3 teaching teams. The list is generated by a medical service management analyst. The residents review patient records for inpatient services, laboratory results, medication changes, and proposed follow-up plans designed by the admission team prior to their phone call. The resident is also responsible for reviewing and reconciling discharge instructions and orders. Then, the resident calls the patient and reviews their hospitalization. If a patient does not answer, the resident leaves a voicemail that complies with the Health Insurance Portability and Accountability Act.

When patients answer the call, the resident follows a script (Appendix). Residents are encouraged to ask patients open-ended questions and address any new needs. They also discuss changes in symptoms, medications, functional status, and remind the patient about follow-up appointments. If imaging or specific orders were missed at discharge, the residents notify the chief resident, lead hospitalist, or deputy associate chief of staff for medical service. If additional laboratory tests need to be ordered, the resident devises a follow-up plan. If needed, specialty referrals can be placed. When residents feel there are multiple items that need to be addressed or if they notice any major concerns, they can recommend the patient present to the emergency department for evaluation. The chief resident, lead hospitalist, and deputy associate chair for medical service are available to assist with discussions about complex medical situations or new concerning symptoms. Residents document their encounters in the Computerized Patient Record System health record and any tests that need follow-up. This differs from the standard of care follow-up programs, which are conducted by primary care medicine nurses and do not fully discuss the hospitalization.

Implementation

This program was implemented as a 1-week elective for interested residents and part of the clinic rotation. The internal medicine medical service analyst pulls all discharges on Friday, which are then provided to the residents. The residents on rotation work through the discharges and find teaching team patients to follow up with and call.

Findings

Implementation of this program has yielded many benefits. The reminder of the importance of a primary care appointment has motivated patients to continue following up on an outpatient basis. Residents were also able to capture lapses in patient understanding. Residents could answer forgotten questions and help patients understand their admission pathology without time pressures. Residents have identified patients with hypoglycemia due to changed insulin regimens, set up specialist follow-up appointments, and provided additional education facilitating adherence. Additionally, several residents have expressed satisfaction with the ability to practice their communication skills. Others appreciated contributing to future patient successes.

While the focus on this article has been to share the program description, we have tabulated preliminary data. In January 2023, there were 239 internal medicine admissions; 158 admissions (66%) were teaching team patients, and 97 patients (61%) were called by a resident and spoken to regarding their care. There were 24 teaching team readmissions within 30 days, and 10 (42%) received a follow-up phone call. Eighty-one admitted patients were treated by nonteaching teams, 10 (12%) of whom were readmissions. Comparing 30-day readmission rates, 10 nonteaching team patients (12%), 10 teaching team patients (6.3%) who talk to a resident in the transition of care program were readmitted, and 24 teaching team patients who did not talk to a resident (10%) were readmitted.

The OETVC transition of care program was planned, formulated, and implemented without modeling after any other projects or institutions. This program aimed to utilize our residents as resources for patients.

Transition of care is defined as steps taken in a clinical encounter to assist with the coordination and continuity of patient care transferring between locations or levels of care.⁵ A 2018 study evaluating the utility of transition of care programs on adults aged ≥ 60 years found a reduction in rehospitalization rates, increased use of primary care services, and potential reduction in home health usage.⁶

In 2021, Johns Hopkins University School of Medicine implemented a program after polling residents and discovering their awareness of gaps in the transition of care.⁷ In 2002, pharmacists evaluated the impact of follow-up telephone calls to recently hospitalized patients. This group of pharmacists found that these calls were associated with increased patient satisfaction, resolution of medication-related problems and fewer emergency department returns.⁸

Our program differs from other transition of care programs in that resident physicians made the follow-up calls to patients. Residents could address all aspects of medical care, including new symptoms, new prescriptions, adverse events, and risk factors for readmission, or order new imaging and medications when appropriate. In the program, residents called all patients discharged after receiving care within their team. Calls were not based on risk assessments. The residents were able to speak with 61% of discharged patients. When readmission rates were compared between patients who received a resident follow-up phone call and those who did not, patients receiving the resident phone call were readmitted at a lower rate: 6.3% vs 10%, respectively.

While our data suggest a potential trend of decreased readmission, more follow-up over a longer period may be needed. We believe this program can benefit patients and our model can act as a template for other institutions interested in starting their own programs.

Challenges

Although our process is efficient, there have been some challenges. The discharge is created by the medical service management analyst and then sent to the chief resident, but there was concern that the list could be missed if either individual was unavailable. The chairperson for the department of medicine and their secretary are now involved in the process. To reduce unanswered telephone calls, residents use OETVC phones. Health Insurance Portability and Accountability Act-compliant voicemails providing a time for a follow-up call were implemented. As a result, veterans have answered their phones more regularly and are more aware of calls. Orders are generally placed by the chief resident, lead hospitalist, or chair of the medical service to ensure follow-up because residents are on rotation for 1 week at a time. Access to a physician also allows patients to discuss items unrelated to their hospitalization, introducing new symptoms, or situations requiring a resident to act with limited data.

CONCLUSIONS

The transition of care follow-up program described in this article may be beneficial for both internal medicine residents and patients. Second- and third-year residents are developing a better understanding of the trajectory of many illnesses and are given the opportunity to retrospectively analyze what they would do differently based on knowledge gained from their chart reviews. They are also given the opportunity to work on communication skills and explain courses of illnesses to patients in an easy-to-understand format without time constraints. Patients now have access to a physician following discharge to discuss any concerns with their hospitalization, condition, and follow-up. This program will continue to address barriers to care and adapt to improve the success of care transitions.

The Olin E. Teague Veterans’ Center (OETVC) in Temple, Texas, is a teaching hospital with 189 beds that provides patients access to medical, surgical, and specialty care. In 2022, 116,359 veterans received care at OETVC and 5393 inpatient admissions were noted. The inpatient ward consists of 3 teaching teams staffed by an attending physician, a second-year internal medicine resident, and 2 to 3 interns while hospitalists staff the 3 nonteaching teams. OETVC residents receive training on both routine and complex medical problems.

Each day, teaching teams discharge patients. With the complexity of discharges, there is always a risk of patients not following up with their primary care physicians, potential issues with filling medications, confusion about new medication regiments, and even potential postdischarge questions. In 1990, Holloway and colleagues evaluated potential risk factors for readmission among veterans. This study found that discharge from a geriatrics or intermediate care bed, chronic disease diagnosis, ≥ 2 procedures performed, increasing age, and distance from a veterans affairs medical center were risk factors.¹

Several community hospital studies have evaluated readmission risk factors. One from 2000 noted that patients with more hospitalizations, lower mental health function, a diagnosis of chronic obstructive pulmonary disorder, and increased satisfaction with access to emergency care were associated with increased readmission in 90 days.² Due to the readmission risks, OETVC decided to construct a program that would help these patients successfully transition from inpatient to outpatient care while establishing means to discuss their care with a physician for reassurance and guidance.

TRANSITION OF CARE PROGRAM

Transition of care programs have been implemented and evaluated in many institutions. A 2017 systematic review of transition of care programs supported the use of tailored discharge planning and postdischarge phone calls to reduce hospital readmission, noting that 6 studies demonstrated a statistically significant reduction in 30-day readmission rate.³ Another study found that pharmacy involvement in the transition of care reduced medication-related problems following discharge.⁴

Program Goals

The foundational goal of our program was to bridge the gap between inpatient and outpatient medicine. We hoped to improve patient adherence with their discharge regimens, improve access to primary care physicians, and improve discharge follow-up. Since hospitalization can be overwhelming, we hoped to capture potential barriers to medical care postdischarge when patients return home while decreasing hospital readmissions. Our second- and third-year resident physicians spend as much time as needed going through the patient’s course of illness throughout their hospitalization and treatment plans to ensure their understanding and potential success.

This program benefits residents by providing medical education and patient communication opportunities. Residents must review the patient’s clinical trajectory before calling them. In this process, residents develop an understanding of routine and complex illness scripts, or pathways of common illnesses. They also prepare for potential questions about the hospitalization, new medications, and follow-up care. Lastly, residents can focus on communication skills. Without the time pressures of returning to a busy rotation, the residents spend as much time discussing the hospital course and ensuring patient understanding as needed.

At the beginning of each week, second- and third-year residents review the list of discharges from the 3 teaching teams. The list is generated by a medical service management analyst. The residents review patient records for inpatient services, laboratory results, medication changes, and proposed follow-up plans designed by the admission team prior to their phone call. The resident is also responsible for reviewing and reconciling discharge instructions and orders. Then, the resident calls the patient and reviews their hospitalization. If a patient does not answer, the resident leaves a voicemail that complies with the Health Insurance Portability and Accountability Act.

When patients answer the call, the resident follows a script (Appendix). Residents are encouraged to ask patients open-ended questions and address any new needs. They also discuss changes in symptoms, medications, functional status, and remind the patient about follow-up appointments. If imaging or specific orders were missed at discharge, the residents notify the chief resident, lead hospitalist, or deputy associate chief of staff for medical service. If additional laboratory tests need to be ordered, the resident devises a follow-up plan. If needed, specialty referrals can be placed. When residents feel there are multiple items that need to be addressed or if they notice any major concerns, they can recommend the patient present to the emergency department for evaluation. The chief resident, lead hospitalist, and deputy associate chair for medical service are available to assist with discussions about complex medical situations or new concerning symptoms. Residents document their encounters in the Computerized Patient Record System health record and any tests that need follow-up. This differs from the standard of care follow-up programs, which are conducted by primary care medicine nurses and do not fully discuss the hospitalization.

Implementation

This program was implemented as a 1-week elective for interested residents and part of the clinic rotation. The internal medicine medical service analyst pulls all discharges on Friday, which are then provided to the residents. The residents on rotation work through the discharges and find teaching team patients to follow up with and call.

Findings

Implementation of this program has yielded many benefits. The reminder of the importance of a primary care appointment has motivated patients to continue following up on an outpatient basis. Residents were also able to capture lapses in patient understanding. Residents could answer forgotten questions and help patients understand their admission pathology without time pressures. Residents have identified patients with hypoglycemia due to changed insulin regimens, set up specialist follow-up appointments, and provided additional education facilitating adherence. Additionally, several residents have expressed satisfaction with the ability to practice their communication skills. Others appreciated contributing to future patient successes.

While the focus on this article has been to share the program description, we have tabulated preliminary data. In January 2023, there were 239 internal medicine admissions; 158 admissions (66%) were teaching team patients, and 97 patients (61%) were called by a resident and spoken to regarding their care. There were 24 teaching team readmissions within 30 days, and 10 (42%) received a follow-up phone call. Eighty-one admitted patients were treated by nonteaching teams, 10 (12%) of whom were readmissions. Comparing 30-day readmission rates, 10 nonteaching team patients (12%), 10 teaching team patients (6.3%) who talk to a resident in the transition of care program were readmitted, and 24 teaching team patients who did not talk to a resident (10%) were readmitted.

The OETVC transition of care program was planned, formulated, and implemented without modeling after any other projects or institutions. This program aimed to utilize our residents as resources for patients.

Transition of care is defined as steps taken in a clinical encounter to assist with the coordination and continuity of patient care transferring between locations or levels of care.⁵ A 2018 study evaluating the utility of transition of care programs on adults aged ≥ 60 years found a reduction in rehospitalization rates, increased use of primary care services, and potential reduction in home health usage.⁶

In 2021, Johns Hopkins University School of Medicine implemented a program after polling residents and discovering their awareness of gaps in the transition of care.⁷ In 2002, pharmacists evaluated the impact of follow-up telephone calls to recently hospitalized patients. This group of pharmacists found that these calls were associated with increased patient satisfaction, resolution of medication-related problems and fewer emergency department returns.⁸

Our program differs from other transition of care programs in that resident physicians made the follow-up calls to patients. Residents could address all aspects of medical care, including new symptoms, new prescriptions, adverse events, and risk factors for readmission, or order new imaging and medications when appropriate. In the program, residents called all patients discharged after receiving care within their team. Calls were not based on risk assessments. The residents were able to speak with 61% of discharged patients. When readmission rates were compared between patients who received a resident follow-up phone call and those who did not, patients receiving the resident phone call were readmitted at a lower rate: 6.3% vs 10%, respectively.

While our data suggest a potential trend of decreased readmission, more follow-up over a longer period may be needed. We believe this program can benefit patients and our model can act as a template for other institutions interested in starting their own programs.

Challenges

Although our process is efficient, there have been some challenges. The discharge is created by the medical service management analyst and then sent to the chief resident, but there was concern that the list could be missed if either individual was unavailable. The chairperson for the department of medicine and their secretary are now involved in the process. To reduce unanswered telephone calls, residents use OETVC phones. Health Insurance Portability and Accountability Act-compliant voicemails providing a time for a follow-up call were implemented. As a result, veterans have answered their phones more regularly and are more aware of calls. Orders are generally placed by the chief resident, lead hospitalist, or chair of the medical service to ensure follow-up because residents are on rotation for 1 week at a time. Access to a physician also allows patients to discuss items unrelated to their hospitalization, introducing new symptoms, or situations requiring a resident to act with limited data.

CONCLUSIONS

The transition of care follow-up program described in this article may be beneficial for both internal medicine residents and patients. Second- and third-year residents are developing a better understanding of the trajectory of many illnesses and are given the opportunity to retrospectively analyze what they would do differently based on knowledge gained from their chart reviews. They are also given the opportunity to work on communication skills and explain courses of illnesses to patients in an easy-to-understand format without time constraints. Patients now have access to a physician following discharge to discuss any concerns with their hospitalization, condition, and follow-up. This program will continue to address barriers to care and adapt to improve the success of care transitions.

Nurses are recognized among the most trusted professions in the United States.¹ Since the time of Florence Nightingale, nurses have been challenged to provide care to patients and soldiers with complex needs, including acute and chronic physical illness, as well as mental health issues. Nurses have traditionally met those challenges with perseverance and creativity but have also experienced stress and burnout.

A shortage of nurses has been linked to many interrelated factors including the retirement of bedside caregivers and educators, diverse care settings, expanding roles for nurses, and nurse burnout.^2-4 Therefore, there is a critical need to better understand of how nurses can be supported while they care for patients, cope with stress, and maintain positive personal and professional outcomes. The objective of this pilot project was to assess US Department of Veterans Affairs (VA) nurses’ levels of burnout and test an intervention to enhance resourcefulness skills during the COVID-19 pandemic.

Background

Stress has many definitions. Hans Selye described it as a biological response of the body to any demand.^5,6 Occupational stress is a process that occurs in which work environment stressors result in the development of psychological, behavioral, or physiological effects that can contribute to health.⁶ Occupational stress has been observed as prevalent among nurses.⁶ In 1960, Menzies identified sources of stress among nurses that include complex decision-making within a dynamic environment.⁷ Since the mid-1980s, nurses’ stress at work has increased because of legal, accreditation, ethical issues, fiscal pressures, staffing shortages, and the increasing integration of technology associated with clinical care.⁸

Sustained stress can lead to emotional exhaustion or burnout, which has been associated with nursing turnover, lower patient satisfaction, and patient safety risk.^2,9 An American Nurses Foundation survey reported that 51% of US nurses feel exhausted, 43% overwhelmed, and 36% anxious; 28% express willingness to leave the profession.² Burnout has been described as a response to physical or emotional stress leading to exhaustion, self-doubt, cynicism, and ineffectiveness.¹⁰ Employees with burnout are more likely to leave their jobs, take sick leave, and suffer from depression and relationship problems,and it affects nearly half of all US nurses, especially among critical care, pediatric, and oncology specialities.^10,11 It has been well documented that unmitigated stress can lead to burnout and contribute to nurses leaving bedside care and the health care profession.^2,3 Several studies on nursing stress and burnout have focused on its prevalence and negative outcomes.^4,7,9 However, few studies have addressed building resiliency and resourcefulness for nurses.^10,12,13

A 2021 National Academy of Medicine report advocated a multilevel approach to managing burnout and building resiliency among nurses.¹⁴ Taylor further identified specific interventions, ranging from primary prevention to treatment.¹⁵ Primary prevention could include educating nurses on self-awareness, coping strategies, and communication skills. Screening for burnout and providing resources for support would be a secondary level of intervention. For nurses who experienced severe burnout symptoms and left the workplace, strategies are sorely needed to provide healing and a return-to-work plan.¹⁵ This may include adjusting nurse schedules and nursing roles (such as admitting/discharge nurse or resource nurse).

RESILIENCY AND RESOURCEFULNESS

Rushton and colleagues describe resiliency as the “ability to face adverse situations, remain focused, and continue to be optimistic for the future.”⁴ For nurses in complex health care systems, resiliency is associated with reduced turnover and symptoms of burnout and improved mental health. Humans are thought to have an innate resiliency potential that evolves over time and fluctuates depending on the context (eg, societal conditions, moral/ethical values, commitments).⁴ It is believed that resiliency can contribute to the development of new neuropathways that can be used to manage and cope with stress, prevent burnout, and improve quality of life. However, it appears these adaptations are individualized and contingent on situations, available resources, and changing priorities.¹⁶ Consequently, resiliency may be an essential tool for nurses to combat burnout in today’s complex health care systems.¹⁷

Although resilience and resourcefulness are conceptually related, each has distinctive features.¹⁸ Celinski frames resilience as transcendence and resourcefulness as transformation.¹⁹ Thus, while resilience suggests transcendence in terms of rising above, going beyond, exceeding, or excelling; resourcefulness reflects transformation, such as making changes in thoughts, feelings, behaviors, actions, or reactions. Resourcefulness has been conceptualized as an indicator of resilience.¹⁸

Resourcefulness comprises 2 dimensions, including the use of self-help strategies (personal resourcefulness) and seeking help from others (social resourcefulness), to self-regulate one’s thoughts, feelings, and behaviors to cope with high levels of stress, anxiety, or depression.^18,20,21 Personal resourcefulness skills include the use of cognitive reframing, positive thinking, problem-solving, priority-setting, and planning ahead. Social resourcefulness involves actively seeking help from others. Formal sources of help include, but are not limited to, nursing and medical care practitioners and community organizations such as hospitals and clinics. Informal sources of help include family members, friends, peers, and coworkers.

During the COVID-19 pandemic, nurses were especially challenged to provide support for each other because of limited nursing staff and treatment options, increased complex patient assignments, shortages of supplies, and reduced support services. Many nurses, however, were able to find innovative, peer-supported strategies for coping.¹³ Nurses’ use of resourcefulness skills is believed to be indicative of their resilience. This pilot project aimed to identify and evaluate some of these strategies and resourcefulness skills.

INTERVENTION

This pilot study among VA Northeast Ohio Health Care System (VANEOHS) nurses was designed to assess nursing burnout and resourcefulness during the pandemic. Those who agreed to participate completed a baseline survey on burnout and resourcefulness. Participants agreed to review a training video on resourcefulness skills (eg, relying on and exchanging ideas with others, and reframing and using ‘positive self-talk’). They were encouraged to document their experience with familiar and new resourcefulness skills. Weekly reminders (eg, emails and phone messages) reminded and coached participants in their journey.

The study identified and implemented an existing Resourcefulness Training (RT) intervention, which was developed for informal family caregivers and found to be effective.²² We measured burnout and resourcefulness preintervention and postintervention.²³ This survey and educational intervention were reviewed by the VANEOHS institutional review board and ruled exempt. The survey also gathered information on nurses' contact with individuals infected with COVID-19.

Despite the many staffing and resource challenges during the COVID-19 pandemic, a convenience sample of 12 nurses was recruited from nursing committees that continued to have scheduled meetings. These meetings allowed time to answer questions and provide information about the study. The majority of nurses queried declined to participate, citing no time, interest, or burnout. Participants completed a baseline survey, reviewed a 30-minute RT video, and tracked their experience for 28 days. Participants completed postintervention surveys 6 weeks after the video. Details of the survey and measures can be found in previous studies.^20,21

RT is an online cognitive-behavioral intervention that teaches and reinforces personal (self-help) and social (help-seeking) resourcefulness skills that have not yet been tested in nurses or other health care professionals.^22,24 The training included social resourcefulness (eg, from family, friends, others, and professionals) and personal resourcefulness (eg, problem-solving, positive thinking, self-control, organization skills). Participants were encouraged to review the videos as often as they preferred during these 4 weeks.

All 12 survey respondents were female and had received COVID-19 vaccinations according to the federal policy at the time of data collection. The number of patients cared for with COVID-19 infections varied widely (range, 1-1000). The baseline burnout score ranged from 1 (no burnout) to 3 (1 symptom of burnout, such as physical and emotional exhaustion), with a mean score of 2.2. In the follow-up survey, the mean score was 2.0. At baseline, participants reported a variety of activities to manage stress and burnout, including times with friends and family, engaging in hobbies, and prayer. Postintervention, some participants mentioned using skills learned from RT, including reframing the situation positively by refocusing and putting stressors in perspective (Table 1).

DISCUSSION

Recent American Nurses Association efforts to develop organizational and professional goals include the importance of nurses to recognize and manage stress to prevent burnout.²⁵ The American Nurses Association Code of Ethics notes that nurses have the same duties to care for themselves as they do for others.²⁵ Nurses have demonstrated the ability to adapt and remain resilient during stressful times. VA nurses are a resourceful group. Many used resourcefulness skills to manage stress and burnout even before the pandemic. For example, nurses identified using family/friends for support and validation, as well as prayer and meditation. Some of the new activities may have been influenced/inspired by RT, such as organizing schedules for problem-solving and distraction.

Relying on family and peers emerged as an essential resourcefulness skill. Support from peers—battle buddies—has been recognized as a key strategy among combat soldiers. A battle buddy is paired with a fellow soldier for support to keep each other informed about key instructions and information. This promotes cooperative problem-solving. Outcomes associated with battle buddies include increased morale and confidence, and decreased stress.²⁵ Over time, it is hoped that these coaching/mentoring relationships will result in enhanced leadership skills. Battle buddy strategies are currently being adapted into health care environments.^12,26 Such programs need to be further evaluated and information disseminated.

Findings from this pilot program support the use of interventions such as RT to decrease burnout among nurses. This study suggests that RT should be tested in a larger more representative sample to determine efficacy.

Limitations

This pilot study was limited by its small sample size, single facility, and female-only participants; the findings are not generalizable. Nurses were recruited from VA nursing committees and may not be representative of nurses in the general population. In addition, the RT intervention may require a longer time commitment to adequately determine efficacy. Another limitation was that personal or family exposure to COVID-19 was not measured, but may be a confounding variable. An additional limitation may have been the time interval. A baseline survey was completed prior to watching the teaching video. Daily logs were to be completed for 28 days. A post survey followed at 6 weeks. It is possible that there was insufficient time for the nurses to have the opportunity to use their resourcefulness skills within the short time frame of the study. While it supports the need for further studies, findings should be interpreted cautiously and not generalized. It may be premature based on these findings to conclude that the intervention will be effective for other populations. Further studies are needed to assess nurses’ preferences for healthy coping mechanisms, including RT.

Conclusions

As the nursing shortage continues, efforts to support diverse, innovative coping strategies remain a priority postpandemic. Nurses must be vigilant in appraising and managing their ability to cope and adapt to individual stress, while also being aware of the stress their colleagues are experiencing. Educational institutions, professional organizations, and health care facilities must strive to educate and support nurses to identify not only stress, but healthy coping mechanisms.

Acknowledgments

This work was supported by the US Department of Veterans Affairs Central Office rapid response COVID-19 funding initiative, the Veteran Affairs Northeast Ohio Health Care System, and Geriatric Research, Education, and Clinical Center (GRECC). The Resourcefulness Scale, Resourcefulness Skills Scale, and the Resourcefulness Training intervention are copyrighted and were used with permission of the copyright holder, Jaclene A. Zauszniewski, MD.

Nurses are recognized among the most trusted professions in the United States.¹ Since the time of Florence Nightingale, nurses have been challenged to provide care to patients and soldiers with complex needs, including acute and chronic physical illness, as well as mental health issues. Nurses have traditionally met those challenges with perseverance and creativity but have also experienced stress and burnout.

A shortage of nurses has been linked to many interrelated factors including the retirement of bedside caregivers and educators, diverse care settings, expanding roles for nurses, and nurse burnout.^2-4 Therefore, there is a critical need to better understand of how nurses can be supported while they care for patients, cope with stress, and maintain positive personal and professional outcomes. The objective of this pilot project was to assess US Department of Veterans Affairs (VA) nurses’ levels of burnout and test an intervention to enhance resourcefulness skills during the COVID-19 pandemic.

Background

Stress has many definitions. Hans Selye described it as a biological response of the body to any demand.^5,6 Occupational stress is a process that occurs in which work environment stressors result in the development of psychological, behavioral, or physiological effects that can contribute to health.⁶ Occupational stress has been observed as prevalent among nurses.⁶ In 1960, Menzies identified sources of stress among nurses that include complex decision-making within a dynamic environment.⁷ Since the mid-1980s, nurses’ stress at work has increased because of legal, accreditation, ethical issues, fiscal pressures, staffing shortages, and the increasing integration of technology associated with clinical care.⁸

Sustained stress can lead to emotional exhaustion or burnout, which has been associated with nursing turnover, lower patient satisfaction, and patient safety risk.^2,9 An American Nurses Foundation survey reported that 51% of US nurses feel exhausted, 43% overwhelmed, and 36% anxious; 28% express willingness to leave the profession.² Burnout has been described as a response to physical or emotional stress leading to exhaustion, self-doubt, cynicism, and ineffectiveness.¹⁰ Employees with burnout are more likely to leave their jobs, take sick leave, and suffer from depression and relationship problems,and it affects nearly half of all US nurses, especially among critical care, pediatric, and oncology specialities.^10,11 It has been well documented that unmitigated stress can lead to burnout and contribute to nurses leaving bedside care and the health care profession.^2,3 Several studies on nursing stress and burnout have focused on its prevalence and negative outcomes.^4,7,9 However, few studies have addressed building resiliency and resourcefulness for nurses.^10,12,13

A 2021 National Academy of Medicine report advocated a multilevel approach to managing burnout and building resiliency among nurses.¹⁴ Taylor further identified specific interventions, ranging from primary prevention to treatment.¹⁵ Primary prevention could include educating nurses on self-awareness, coping strategies, and communication skills. Screening for burnout and providing resources for support would be a secondary level of intervention. For nurses who experienced severe burnout symptoms and left the workplace, strategies are sorely needed to provide healing and a return-to-work plan.¹⁵ This may include adjusting nurse schedules and nursing roles (such as admitting/discharge nurse or resource nurse).

RESILIENCY AND RESOURCEFULNESS

Rushton and colleagues describe resiliency as the “ability to face adverse situations, remain focused, and continue to be optimistic for the future.”⁴ For nurses in complex health care systems, resiliency is associated with reduced turnover and symptoms of burnout and improved mental health. Humans are thought to have an innate resiliency potential that evolves over time and fluctuates depending on the context (eg, societal conditions, moral/ethical values, commitments).⁴ It is believed that resiliency can contribute to the development of new neuropathways that can be used to manage and cope with stress, prevent burnout, and improve quality of life. However, it appears these adaptations are individualized and contingent on situations, available resources, and changing priorities.¹⁶ Consequently, resiliency may be an essential tool for nurses to combat burnout in today’s complex health care systems.¹⁷

Although resilience and resourcefulness are conceptually related, each has distinctive features.¹⁸ Celinski frames resilience as transcendence and resourcefulness as transformation.¹⁹ Thus, while resilience suggests transcendence in terms of rising above, going beyond, exceeding, or excelling; resourcefulness reflects transformation, such as making changes in thoughts, feelings, behaviors, actions, or reactions. Resourcefulness has been conceptualized as an indicator of resilience.¹⁸

Resourcefulness comprises 2 dimensions, including the use of self-help strategies (personal resourcefulness) and seeking help from others (social resourcefulness), to self-regulate one’s thoughts, feelings, and behaviors to cope with high levels of stress, anxiety, or depression.^18,20,21 Personal resourcefulness skills include the use of cognitive reframing, positive thinking, problem-solving, priority-setting, and planning ahead. Social resourcefulness involves actively seeking help from others. Formal sources of help include, but are not limited to, nursing and medical care practitioners and community organizations such as hospitals and clinics. Informal sources of help include family members, friends, peers, and coworkers.

During the COVID-19 pandemic, nurses were especially challenged to provide support for each other because of limited nursing staff and treatment options, increased complex patient assignments, shortages of supplies, and reduced support services. Many nurses, however, were able to find innovative, peer-supported strategies for coping.¹³ Nurses’ use of resourcefulness skills is believed to be indicative of their resilience. This pilot project aimed to identify and evaluate some of these strategies and resourcefulness skills.

INTERVENTION

This pilot study among VA Northeast Ohio Health Care System (VANEOHS) nurses was designed to assess nursing burnout and resourcefulness during the pandemic. Those who agreed to participate completed a baseline survey on burnout and resourcefulness. Participants agreed to review a training video on resourcefulness skills (eg, relying on and exchanging ideas with others, and reframing and using ‘positive self-talk’). They were encouraged to document their experience with familiar and new resourcefulness skills. Weekly reminders (eg, emails and phone messages) reminded and coached participants in their journey.

The study identified and implemented an existing Resourcefulness Training (RT) intervention, which was developed for informal family caregivers and found to be effective.²² We measured burnout and resourcefulness preintervention and postintervention.²³ This survey and educational intervention were reviewed by the VANEOHS institutional review board and ruled exempt. The survey also gathered information on nurses' contact with individuals infected with COVID-19.

Despite the many staffing and resource challenges during the COVID-19 pandemic, a convenience sample of 12 nurses was recruited from nursing committees that continued to have scheduled meetings. These meetings allowed time to answer questions and provide information about the study. The majority of nurses queried declined to participate, citing no time, interest, or burnout. Participants completed a baseline survey, reviewed a 30-minute RT video, and tracked their experience for 28 days. Participants completed postintervention surveys 6 weeks after the video. Details of the survey and measures can be found in previous studies.^20,21

RT is an online cognitive-behavioral intervention that teaches and reinforces personal (self-help) and social (help-seeking) resourcefulness skills that have not yet been tested in nurses or other health care professionals.^22,24 The training included social resourcefulness (eg, from family, friends, others, and professionals) and personal resourcefulness (eg, problem-solving, positive thinking, self-control, organization skills). Participants were encouraged to review the videos as often as they preferred during these 4 weeks.

All 12 survey respondents were female and had received COVID-19 vaccinations according to the federal policy at the time of data collection. The number of patients cared for with COVID-19 infections varied widely (range, 1-1000). The baseline burnout score ranged from 1 (no burnout) to 3 (1 symptom of burnout, such as physical and emotional exhaustion), with a mean score of 2.2. In the follow-up survey, the mean score was 2.0. At baseline, participants reported a variety of activities to manage stress and burnout, including times with friends and family, engaging in hobbies, and prayer. Postintervention, some participants mentioned using skills learned from RT, including reframing the situation positively by refocusing and putting stressors in perspective (Table 1).

DISCUSSION

Recent American Nurses Association efforts to develop organizational and professional goals include the importance of nurses to recognize and manage stress to prevent burnout.²⁵ The American Nurses Association Code of Ethics notes that nurses have the same duties to care for themselves as they do for others.²⁵ Nurses have demonstrated the ability to adapt and remain resilient during stressful times. VA nurses are a resourceful group. Many used resourcefulness skills to manage stress and burnout even before the pandemic. For example, nurses identified using family/friends for support and validation, as well as prayer and meditation. Some of the new activities may have been influenced/inspired by RT, such as organizing schedules for problem-solving and distraction.

Relying on family and peers emerged as an essential resourcefulness skill. Support from peers—battle buddies—has been recognized as a key strategy among combat soldiers. A battle buddy is paired with a fellow soldier for support to keep each other informed about key instructions and information. This promotes cooperative problem-solving. Outcomes associated with battle buddies include increased morale and confidence, and decreased stress.²⁵ Over time, it is hoped that these coaching/mentoring relationships will result in enhanced leadership skills. Battle buddy strategies are currently being adapted into health care environments.^12,26 Such programs need to be further evaluated and information disseminated.

Findings from this pilot program support the use of interventions such as RT to decrease burnout among nurses. This study suggests that RT should be tested in a larger more representative sample to determine efficacy.

Limitations

This pilot study was limited by its small sample size, single facility, and female-only participants; the findings are not generalizable. Nurses were recruited from VA nursing committees and may not be representative of nurses in the general population. In addition, the RT intervention may require a longer time commitment to adequately determine efficacy. Another limitation was that personal or family exposure to COVID-19 was not measured, but may be a confounding variable. An additional limitation may have been the time interval. A baseline survey was completed prior to watching the teaching video. Daily logs were to be completed for 28 days. A post survey followed at 6 weeks. It is possible that there was insufficient time for the nurses to have the opportunity to use their resourcefulness skills within the short time frame of the study. While it supports the need for further studies, findings should be interpreted cautiously and not generalized. It may be premature based on these findings to conclude that the intervention will be effective for other populations. Further studies are needed to assess nurses’ preferences for healthy coping mechanisms, including RT.

Conclusions

As the nursing shortage continues, efforts to support diverse, innovative coping strategies remain a priority postpandemic. Nurses must be vigilant in appraising and managing their ability to cope and adapt to individual stress, while also being aware of the stress their colleagues are experiencing. Educational institutions, professional organizations, and health care facilities must strive to educate and support nurses to identify not only stress, but healthy coping mechanisms.

Acknowledgments

This work was supported by the US Department of Veterans Affairs Central Office rapid response COVID-19 funding initiative, the Veteran Affairs Northeast Ohio Health Care System, and Geriatric Research, Education, and Clinical Center (GRECC). The Resourcefulness Scale, Resourcefulness Skills Scale, and the Resourcefulness Training intervention are copyrighted and were used with permission of the copyright holder, Jaclene A. Zauszniewski, MD.

Nurses are recognized among the most trusted professions in the United States.¹ Since the time of Florence Nightingale, nurses have been challenged to provide care to patients and soldiers with complex needs, including acute and chronic physical illness, as well as mental health issues. Nurses have traditionally met those challenges with perseverance and creativity but have also experienced stress and burnout.

A shortage of nurses has been linked to many interrelated factors including the retirement of bedside caregivers and educators, diverse care settings, expanding roles for nurses, and nurse burnout.^2-4 Therefore, there is a critical need to better understand of how nurses can be supported while they care for patients, cope with stress, and maintain positive personal and professional outcomes. The objective of this pilot project was to assess US Department of Veterans Affairs (VA) nurses’ levels of burnout and test an intervention to enhance resourcefulness skills during the COVID-19 pandemic.

Background

Stress has many definitions. Hans Selye described it as a biological response of the body to any demand.^5,6 Occupational stress is a process that occurs in which work environment stressors result in the development of psychological, behavioral, or physiological effects that can contribute to health.⁶ Occupational stress has been observed as prevalent among nurses.⁶ In 1960, Menzies identified sources of stress among nurses that include complex decision-making within a dynamic environment.⁷ Since the mid-1980s, nurses’ stress at work has increased because of legal, accreditation, ethical issues, fiscal pressures, staffing shortages, and the increasing integration of technology associated with clinical care.⁸

Sustained stress can lead to emotional exhaustion or burnout, which has been associated with nursing turnover, lower patient satisfaction, and patient safety risk.^2,9 An American Nurses Foundation survey reported that 51% of US nurses feel exhausted, 43% overwhelmed, and 36% anxious; 28% express willingness to leave the profession.² Burnout has been described as a response to physical or emotional stress leading to exhaustion, self-doubt, cynicism, and ineffectiveness.¹⁰ Employees with burnout are more likely to leave their jobs, take sick leave, and suffer from depression and relationship problems,and it affects nearly half of all US nurses, especially among critical care, pediatric, and oncology specialities.^10,11 It has been well documented that unmitigated stress can lead to burnout and contribute to nurses leaving bedside care and the health care profession.^2,3 Several studies on nursing stress and burnout have focused on its prevalence and negative outcomes.^4,7,9 However, few studies have addressed building resiliency and resourcefulness for nurses.^10,12,13

A 2021 National Academy of Medicine report advocated a multilevel approach to managing burnout and building resiliency among nurses.¹⁴ Taylor further identified specific interventions, ranging from primary prevention to treatment.¹⁵ Primary prevention could include educating nurses on self-awareness, coping strategies, and communication skills. Screening for burnout and providing resources for support would be a secondary level of intervention. For nurses who experienced severe burnout symptoms and left the workplace, strategies are sorely needed to provide healing and a return-to-work plan.¹⁵ This may include adjusting nurse schedules and nursing roles (such as admitting/discharge nurse or resource nurse).

RESILIENCY AND RESOURCEFULNESS

Rushton and colleagues describe resiliency as the “ability to face adverse situations, remain focused, and continue to be optimistic for the future.”⁴ For nurses in complex health care systems, resiliency is associated with reduced turnover and symptoms of burnout and improved mental health. Humans are thought to have an innate resiliency potential that evolves over time and fluctuates depending on the context (eg, societal conditions, moral/ethical values, commitments).⁴ It is believed that resiliency can contribute to the development of new neuropathways that can be used to manage and cope with stress, prevent burnout, and improve quality of life. However, it appears these adaptations are individualized and contingent on situations, available resources, and changing priorities.¹⁶ Consequently, resiliency may be an essential tool for nurses to combat burnout in today’s complex health care systems.¹⁷

Although resilience and resourcefulness are conceptually related, each has distinctive features.¹⁸ Celinski frames resilience as transcendence and resourcefulness as transformation.¹⁹ Thus, while resilience suggests transcendence in terms of rising above, going beyond, exceeding, or excelling; resourcefulness reflects transformation, such as making changes in thoughts, feelings, behaviors, actions, or reactions. Resourcefulness has been conceptualized as an indicator of resilience.¹⁸

Resourcefulness comprises 2 dimensions, including the use of self-help strategies (personal resourcefulness) and seeking help from others (social resourcefulness), to self-regulate one’s thoughts, feelings, and behaviors to cope with high levels of stress, anxiety, or depression.^18,20,21 Personal resourcefulness skills include the use of cognitive reframing, positive thinking, problem-solving, priority-setting, and planning ahead. Social resourcefulness involves actively seeking help from others. Formal sources of help include, but are not limited to, nursing and medical care practitioners and community organizations such as hospitals and clinics. Informal sources of help include family members, friends, peers, and coworkers.

During the COVID-19 pandemic, nurses were especially challenged to provide support for each other because of limited nursing staff and treatment options, increased complex patient assignments, shortages of supplies, and reduced support services. Many nurses, however, were able to find innovative, peer-supported strategies for coping.¹³ Nurses’ use of resourcefulness skills is believed to be indicative of their resilience. This pilot project aimed to identify and evaluate some of these strategies and resourcefulness skills.

INTERVENTION

This pilot study among VA Northeast Ohio Health Care System (VANEOHS) nurses was designed to assess nursing burnout and resourcefulness during the pandemic. Those who agreed to participate completed a baseline survey on burnout and resourcefulness. Participants agreed to review a training video on resourcefulness skills (eg, relying on and exchanging ideas with others, and reframing and using ‘positive self-talk’). They were encouraged to document their experience with familiar and new resourcefulness skills. Weekly reminders (eg, emails and phone messages) reminded and coached participants in their journey.

The study identified and implemented an existing Resourcefulness Training (RT) intervention, which was developed for informal family caregivers and found to be effective.²² We measured burnout and resourcefulness preintervention and postintervention.²³ This survey and educational intervention were reviewed by the VANEOHS institutional review board and ruled exempt. The survey also gathered information on nurses' contact with individuals infected with COVID-19.

Despite the many staffing and resource challenges during the COVID-19 pandemic, a convenience sample of 12 nurses was recruited from nursing committees that continued to have scheduled meetings. These meetings allowed time to answer questions and provide information about the study. The majority of nurses queried declined to participate, citing no time, interest, or burnout. Participants completed a baseline survey, reviewed a 30-minute RT video, and tracked their experience for 28 days. Participants completed postintervention surveys 6 weeks after the video. Details of the survey and measures can be found in previous studies.^20,21

RT is an online cognitive-behavioral intervention that teaches and reinforces personal (self-help) and social (help-seeking) resourcefulness skills that have not yet been tested in nurses or other health care professionals.^22,24 The training included social resourcefulness (eg, from family, friends, others, and professionals) and personal resourcefulness (eg, problem-solving, positive thinking, self-control, organization skills). Participants were encouraged to review the videos as often as they preferred during these 4 weeks.

All 12 survey respondents were female and had received COVID-19 vaccinations according to the federal policy at the time of data collection. The number of patients cared for with COVID-19 infections varied widely (range, 1-1000). The baseline burnout score ranged from 1 (no burnout) to 3 (1 symptom of burnout, such as physical and emotional exhaustion), with a mean score of 2.2. In the follow-up survey, the mean score was 2.0. At baseline, participants reported a variety of activities to manage stress and burnout, including times with friends and family, engaging in hobbies, and prayer. Postintervention, some participants mentioned using skills learned from RT, including reframing the situation positively by refocusing and putting stressors in perspective (Table 1).

DISCUSSION

Recent American Nurses Association efforts to develop organizational and professional goals include the importance of nurses to recognize and manage stress to prevent burnout.²⁵ The American Nurses Association Code of Ethics notes that nurses have the same duties to care for themselves as they do for others.²⁵ Nurses have demonstrated the ability to adapt and remain resilient during stressful times. VA nurses are a resourceful group. Many used resourcefulness skills to manage stress and burnout even before the pandemic. For example, nurses identified using family/friends for support and validation, as well as prayer and meditation. Some of the new activities may have been influenced/inspired by RT, such as organizing schedules for problem-solving and distraction.

Relying on family and peers emerged as an essential resourcefulness skill. Support from peers—battle buddies—has been recognized as a key strategy among combat soldiers. A battle buddy is paired with a fellow soldier for support to keep each other informed about key instructions and information. This promotes cooperative problem-solving. Outcomes associated with battle buddies include increased morale and confidence, and decreased stress.²⁵ Over time, it is hoped that these coaching/mentoring relationships will result in enhanced leadership skills. Battle buddy strategies are currently being adapted into health care environments.^12,26 Such programs need to be further evaluated and information disseminated.

Findings from this pilot program support the use of interventions such as RT to decrease burnout among nurses. This study suggests that RT should be tested in a larger more representative sample to determine efficacy.

Limitations

This pilot study was limited by its small sample size, single facility, and female-only participants; the findings are not generalizable. Nurses were recruited from VA nursing committees and may not be representative of nurses in the general population. In addition, the RT intervention may require a longer time commitment to adequately determine efficacy. Another limitation was that personal or family exposure to COVID-19 was not measured, but may be a confounding variable. An additional limitation may have been the time interval. A baseline survey was completed prior to watching the teaching video. Daily logs were to be completed for 28 days. A post survey followed at 6 weeks. It is possible that there was insufficient time for the nurses to have the opportunity to use their resourcefulness skills within the short time frame of the study. While it supports the need for further studies, findings should be interpreted cautiously and not generalized. It may be premature based on these findings to conclude that the intervention will be effective for other populations. Further studies are needed to assess nurses’ preferences for healthy coping mechanisms, including RT.

Conclusions

As the nursing shortage continues, efforts to support diverse, innovative coping strategies remain a priority postpandemic. Nurses must be vigilant in appraising and managing their ability to cope and adapt to individual stress, while also being aware of the stress their colleagues are experiencing. Educational institutions, professional organizations, and health care facilities must strive to educate and support nurses to identify not only stress, but healthy coping mechanisms.

Acknowledgments

This work was supported by the US Department of Veterans Affairs Central Office rapid response COVID-19 funding initiative, the Veteran Affairs Northeast Ohio Health Care System, and Geriatric Research, Education, and Clinical Center (GRECC). The Resourcefulness Scale, Resourcefulness Skills Scale, and the Resourcefulness Training intervention are copyrighted and were used with permission of the copyright holder, Jaclene A. Zauszniewski, MD.

Critical illness is commonly associated with interrelated conditions including pain, agitation, delirium, immobility, and sleep disruption (PADIS). Managing PADIS is often complex and includes pharmacologic and nonpharmacologic interventions.¹ Incorporating multifaceted practices to enhance PADIS management has been shown to improve several intensive care unit (ICU)-related outcomes.²

Many pharmacologic PADIS treatments are ineffective or associated with adverse effects. For example, antipsychotics used for treating ICU-related delirium have not shown improved outcomes.^3,4 Commonly used medications for agitation, such as benzodiazepines, increase delirium risk.^5,6 Because of these limitations, several nonpharmacologic interventions for PADIS have been evaluated.

Pet therapy has been implemented in some ICU settings, but is not widely adopted.⁷ Also referred to as animal-assisted activities, animal-assisted therapy, or animal-assisted interventions, pet therapy typically involves interaction between a patient and a live animal (most commonly a dog) under the direction of an animal handler, with the intention of providing therapeutic benefit. Interactions frequently include meet and greet activities such as petting, but also could include walking or other activities. Pet therapy has been reported to reduce pain, agitation, and stress among ICU patients.⁸ Introducing a pet therapy program with live animals in the ICU could be challenging because of factors such as identifying trained, accredited animals and handlers, and managing infection control and other risks.⁹ As an alternative to live pets, robotic pet therapy has been shown to be beneficial—mostly outside the ICU—in settings such as long-term care.^10,11 Although uncommon, robotic pets have been used in the ICU and hospital settings for therapeutic purposes.¹² Robotic pets reduce many concerns associated with live animals while mimicking the behaviors of live animals and potentially offering many of the same benefits.

OBSERVATIONS

The North Florida/South Georgia Veterans Health System (NF/SGVHS) implemented a novel robotic pet therapy program for patients requiring ICU care to improve the treatment of PADIS. Funding was provided through a Veterans Health Administration Innovation Grant procured by a clinical pharmacy specialist as the program’s champion. Goals of the robotic pet therapy program include reductions in: distressing symptoms associated with PADIS, use of psychoactive drugs and physical restraints, and ICU length of stay. The ICU team developed standard operating procedures and an order menu, which were integrated into the ICU prescriber ordering menu. Patients were selected for pet therapy based on PADIS scores and potential for positive response to pet therapy as assessed by the ICU team.Patients in medical and surgical ICU settings were eligible for the program. The robotic pets used in the program were Joy for AllCompanion Pets (Ageless Innovation LLC). Robotic cats and dogs were available and pets were “adopted’ by each patient (Figure). As an infection control measure, pets were not reissued or shared amongpatients and pets could be cleaned with a disinfectant solution. Nurses were primarily responsible for monitoring and documenting responses to robotic pet therapy.

It was necessary to secure buy-in from several services to successfully implement the program. The critical care clinical pharmacy specialists were responsible for ordering, storing, and dispensing the robotic pets. The NF/SGVHS innovation specialist helped secure funding, procure the robotic pet, and promote the program. The standard operating procedures for the program were developed by a multidisciplinary team with input from critical care nurses, intensivists, pharmacists, patient safety, and infection control (Table 1). Success of the program also required buy-in from ICU team members.

A retrospective cohort study was conducted to assess for improvements in PADIS symptoms and medication use post-intervention. Patients were included if they received robotic pet therapy in the ICU from July 10, 2019, to February 1, 2021. Individuals aged < 18 years or > 89 years, were pregnant, or were not receiving ICU-level care were excluded. Outcomes assessed included improvement in pain scores, agitation scores, sleep quality, resolution of delirium, and use of pain or psychoactive medications during patients’ ICU stay.

Thirty patients were included in the study (Table 2). After receiving a robotic pet, 9 (30%) patients recorded decreased pain scores, 15 (50%) recorded decreased agitation scores, 8 (27%) had resolution of delirium, and 2 (7%) described improvement in sleep. Pain medication use decreased in 12 (40%) patients and psychoactive medication use was reduced in 7 (23%) patients.

Limitations

The robotic pet therapy program has shown promising results; however, some aspects merit discussion. Evaluation of this program is limited by factors such as the observational study design, single-center patient sample, and lack of comparator group. Although no known adverse effects of robotic pet therapy were seen, it is possible that some patients may not have a favorable response. Challenges of implementing a robotic pet therapy program include cost and additional operational activities (storage, ordering, dispensing) necessary to maintain the program. Additional research is needed to evaluate the impact of robotic pet therapy on other outcomes including cost, ICU length of stay, and patient satisfaction.

CONCLUSIONS

Robotic pet therapy can be successfully implemented in the ICU and appears to provide a simple, safe, beneficial, nonpharmacologic intervention for PADIS. This study showed that many patients had favorable response to robotic pet therapy, indicating that it may be a viable alternative to traditional pet therapy. Other health systems could benefit from implementing programs similar to the robotic pet therapy program at NF/SGVHS.

Acknowledgments

The author would like to acknowledge Simran Panesar, PharmD, and Theresa Faison, PharmD, for their contributions to this project.

Critical illness is commonly associated with interrelated conditions including pain, agitation, delirium, immobility, and sleep disruption (PADIS). Managing PADIS is often complex and includes pharmacologic and nonpharmacologic interventions.¹ Incorporating multifaceted practices to enhance PADIS management has been shown to improve several intensive care unit (ICU)-related outcomes.²

Many pharmacologic PADIS treatments are ineffective or associated with adverse effects. For example, antipsychotics used for treating ICU-related delirium have not shown improved outcomes.^3,4 Commonly used medications for agitation, such as benzodiazepines, increase delirium risk.^5,6 Because of these limitations, several nonpharmacologic interventions for PADIS have been evaluated.

Pet therapy has been implemented in some ICU settings, but is not widely adopted.⁷ Also referred to as animal-assisted activities, animal-assisted therapy, or animal-assisted interventions, pet therapy typically involves interaction between a patient and a live animal (most commonly a dog) under the direction of an animal handler, with the intention of providing therapeutic benefit. Interactions frequently include meet and greet activities such as petting, but also could include walking or other activities. Pet therapy has been reported to reduce pain, agitation, and stress among ICU patients.⁸ Introducing a pet therapy program with live animals in the ICU could be challenging because of factors such as identifying trained, accredited animals and handlers, and managing infection control and other risks.⁹ As an alternative to live pets, robotic pet therapy has been shown to be beneficial—mostly outside the ICU—in settings such as long-term care.^10,11 Although uncommon, robotic pets have been used in the ICU and hospital settings for therapeutic purposes.¹² Robotic pets reduce many concerns associated with live animals while mimicking the behaviors of live animals and potentially offering many of the same benefits.

OBSERVATIONS

The North Florida/South Georgia Veterans Health System (NF/SGVHS) implemented a novel robotic pet therapy program for patients requiring ICU care to improve the treatment of PADIS. Funding was provided through a Veterans Health Administration Innovation Grant procured by a clinical pharmacy specialist as the program’s champion. Goals of the robotic pet therapy program include reductions in: distressing symptoms associated with PADIS, use of psychoactive drugs and physical restraints, and ICU length of stay. The ICU team developed standard operating procedures and an order menu, which were integrated into the ICU prescriber ordering menu. Patients were selected for pet therapy based on PADIS scores and potential for positive response to pet therapy as assessed by the ICU team.Patients in medical and surgical ICU settings were eligible for the program. The robotic pets used in the program were Joy for AllCompanion Pets (Ageless Innovation LLC). Robotic cats and dogs were available and pets were “adopted’ by each patient (Figure). As an infection control measure, pets were not reissued or shared amongpatients and pets could be cleaned with a disinfectant solution. Nurses were primarily responsible for monitoring and documenting responses to robotic pet therapy.

It was necessary to secure buy-in from several services to successfully implement the program. The critical care clinical pharmacy specialists were responsible for ordering, storing, and dispensing the robotic pets. The NF/SGVHS innovation specialist helped secure funding, procure the robotic pet, and promote the program. The standard operating procedures for the program were developed by a multidisciplinary team with input from critical care nurses, intensivists, pharmacists, patient safety, and infection control (Table 1). Success of the program also required buy-in from ICU team members.

A retrospective cohort study was conducted to assess for improvements in PADIS symptoms and medication use post-intervention. Patients were included if they received robotic pet therapy in the ICU from July 10, 2019, to February 1, 2021. Individuals aged < 18 years or > 89 years, were pregnant, or were not receiving ICU-level care were excluded. Outcomes assessed included improvement in pain scores, agitation scores, sleep quality, resolution of delirium, and use of pain or psychoactive medications during patients’ ICU stay.

Thirty patients were included in the study (Table 2). After receiving a robotic pet, 9 (30%) patients recorded decreased pain scores, 15 (50%) recorded decreased agitation scores, 8 (27%) had resolution of delirium, and 2 (7%) described improvement in sleep. Pain medication use decreased in 12 (40%) patients and psychoactive medication use was reduced in 7 (23%) patients.

Limitations

The robotic pet therapy program has shown promising results; however, some aspects merit discussion. Evaluation of this program is limited by factors such as the observational study design, single-center patient sample, and lack of comparator group. Although no known adverse effects of robotic pet therapy were seen, it is possible that some patients may not have a favorable response. Challenges of implementing a robotic pet therapy program include cost and additional operational activities (storage, ordering, dispensing) necessary to maintain the program. Additional research is needed to evaluate the impact of robotic pet therapy on other outcomes including cost, ICU length of stay, and patient satisfaction.

CONCLUSIONS

Robotic pet therapy can be successfully implemented in the ICU and appears to provide a simple, safe, beneficial, nonpharmacologic intervention for PADIS. This study showed that many patients had favorable response to robotic pet therapy, indicating that it may be a viable alternative to traditional pet therapy. Other health systems could benefit from implementing programs similar to the robotic pet therapy program at NF/SGVHS.

Acknowledgments

The author would like to acknowledge Simran Panesar, PharmD, and Theresa Faison, PharmD, for their contributions to this project.

Critical illness is commonly associated with interrelated conditions including pain, agitation, delirium, immobility, and sleep disruption (PADIS). Managing PADIS is often complex and includes pharmacologic and nonpharmacologic interventions.¹ Incorporating multifaceted practices to enhance PADIS management has been shown to improve several intensive care unit (ICU)-related outcomes.²

Many pharmacologic PADIS treatments are ineffective or associated with adverse effects. For example, antipsychotics used for treating ICU-related delirium have not shown improved outcomes.^3,4 Commonly used medications for agitation, such as benzodiazepines, increase delirium risk.^5,6 Because of these limitations, several nonpharmacologic interventions for PADIS have been evaluated.

Pet therapy has been implemented in some ICU settings, but is not widely adopted.⁷ Also referred to as animal-assisted activities, animal-assisted therapy, or animal-assisted interventions, pet therapy typically involves interaction between a patient and a live animal (most commonly a dog) under the direction of an animal handler, with the intention of providing therapeutic benefit. Interactions frequently include meet and greet activities such as petting, but also could include walking or other activities. Pet therapy has been reported to reduce pain, agitation, and stress among ICU patients.⁸ Introducing a pet therapy program with live animals in the ICU could be challenging because of factors such as identifying trained, accredited animals and handlers, and managing infection control and other risks.⁹ As an alternative to live pets, robotic pet therapy has been shown to be beneficial—mostly outside the ICU—in settings such as long-term care.^10,11 Although uncommon, robotic pets have been used in the ICU and hospital settings for therapeutic purposes.¹² Robotic pets reduce many concerns associated with live animals while mimicking the behaviors of live animals and potentially offering many of the same benefits.

OBSERVATIONS

The North Florida/South Georgia Veterans Health System (NF/SGVHS) implemented a novel robotic pet therapy program for patients requiring ICU care to improve the treatment of PADIS. Funding was provided through a Veterans Health Administration Innovation Grant procured by a clinical pharmacy specialist as the program’s champion. Goals of the robotic pet therapy program include reductions in: distressing symptoms associated with PADIS, use of psychoactive drugs and physical restraints, and ICU length of stay. The ICU team developed standard operating procedures and an order menu, which were integrated into the ICU prescriber ordering menu. Patients were selected for pet therapy based on PADIS scores and potential for positive response to pet therapy as assessed by the ICU team.Patients in medical and surgical ICU settings were eligible for the program. The robotic pets used in the program were Joy for AllCompanion Pets (Ageless Innovation LLC). Robotic cats and dogs were available and pets were “adopted’ by each patient (Figure). As an infection control measure, pets were not reissued or shared amongpatients and pets could be cleaned with a disinfectant solution. Nurses were primarily responsible for monitoring and documenting responses to robotic pet therapy.

It was necessary to secure buy-in from several services to successfully implement the program. The critical care clinical pharmacy specialists were responsible for ordering, storing, and dispensing the robotic pets. The NF/SGVHS innovation specialist helped secure funding, procure the robotic pet, and promote the program. The standard operating procedures for the program were developed by a multidisciplinary team with input from critical care nurses, intensivists, pharmacists, patient safety, and infection control (Table 1). Success of the program also required buy-in from ICU team members.

A retrospective cohort study was conducted to assess for improvements in PADIS symptoms and medication use post-intervention. Patients were included if they received robotic pet therapy in the ICU from July 10, 2019, to February 1, 2021. Individuals aged < 18 years or > 89 years, were pregnant, or were not receiving ICU-level care were excluded. Outcomes assessed included improvement in pain scores, agitation scores, sleep quality, resolution of delirium, and use of pain or psychoactive medications during patients’ ICU stay.

Thirty patients were included in the study (Table 2). After receiving a robotic pet, 9 (30%) patients recorded decreased pain scores, 15 (50%) recorded decreased agitation scores, 8 (27%) had resolution of delirium, and 2 (7%) described improvement in sleep. Pain medication use decreased in 12 (40%) patients and psychoactive medication use was reduced in 7 (23%) patients.

Limitations

The robotic pet therapy program has shown promising results; however, some aspects merit discussion. Evaluation of this program is limited by factors such as the observational study design, single-center patient sample, and lack of comparator group. Although no known adverse effects of robotic pet therapy were seen, it is possible that some patients may not have a favorable response. Challenges of implementing a robotic pet therapy program include cost and additional operational activities (storage, ordering, dispensing) necessary to maintain the program. Additional research is needed to evaluate the impact of robotic pet therapy on other outcomes including cost, ICU length of stay, and patient satisfaction.

CONCLUSIONS

Robotic pet therapy can be successfully implemented in the ICU and appears to provide a simple, safe, beneficial, nonpharmacologic intervention for PADIS. This study showed that many patients had favorable response to robotic pet therapy, indicating that it may be a viable alternative to traditional pet therapy. Other health systems could benefit from implementing programs similar to the robotic pet therapy program at NF/SGVHS.

Acknowledgments

The author would like to acknowledge Simran Panesar, PharmD, and Theresa Faison, PharmD, for their contributions to this project.

Three-dimensional (3D) printing has become a promising area of innovation in biomedical research.^1,2 Previous research in orthopedic surgery has found that customized 3D printed implants, casts, orthoses, and prosthetics (eg, prosthetic hands) matched to an individual’s unique anatomy can result in more precise placement and better surgical outcomes.^3-5 Customized prosthetics have also been found to lead to fewer complications.^3,6

Recent advances in 3D printing technology has prompted investigation from surgeons to identify how this new tool may be incorporated into patient care.^1,7 One of the most common applications of 3D printing is during preoperative planning in which surgeons gain better insight into patient-specific anatomy by using patient-specific printed models.⁸ Another promising application is the production of customized prosthetics suited to each patient’s unique anatomy.⁹ As a result, 3D printing has significantly impacted bone and cartilage restoration procedures and has the potential to completely transform the treatment of patients with debilitating musculoskeletal injuries.^3,10

The potential surrounding 3D printed prosthetics has led to their adoption by several other specialties, including otolaryngology.¹¹ The most widely used application of 3D printing among otolaryngologists is preoperative planning, and the incorporation of printed prosthetics intoreconstruction of the orbit, nasal septum, auricle, and palate has also been reported.^2,12,13 Patient-specific implants might allow otolaryngologists to better rehabilitate, reconstruct, and/or regenerate craniofacial defects using more humane procedures.¹⁴

Patients with palatomaxillary cancers are treated by prosthodontists or otolaryngologists. An impression is made with a resin–which can be painful for postoperative patients–and a prosthetic is manufactured and implanted.^15-17 Patients with cancer often see many specialists, though reconstructive care is a low priority. Many of these individuals also experience dynamic anatomic functional changes over time, leading to the need for multiple prothesis.

palatomaxillary prosthetics

This program aims to use patients’ previous computed tomography (CT) to tailor customized 3D printed palatomaxillary prosthetics to specifically fit their anatomy. Palatomaxillary defects are a source of profound disability for patients with head and neck cancers who are left with large anatomic defects as a direct result of treatment. Reconstruction of palatal defects poses unique challenges due to the complexity of patient anatomy.^18,19

3D printed prosthetics for palatomaxillary defects have not been incorporated into patient care. We reviewed previous imaging research to determine if it could be used to assist patients who struggle with their function and appearance following treatment for head and neck cancers. The primary aim was to investigate whether 3D printing was a feasible strategy for creating patient-specific palatomaxillary prosthetics. The secondary aim is to determine whether these prosthetics should be tested in the future for use in reconstruction of maxillary defects.

This study was conducted at the Veterans Affairs Palo Alto Health Care System (VAPAHCS) and was approved by the Stanford University Institutional Review Board (approval #28958, informed consent and patient contact excluded). A retrospective chart review was conducted on all patients with head and neck cancers who were treated at VAPAHCS from 2010 to 2022. Patients aged ≥ 18 years who had a palatomaxillary defect due to cancer treatment, had undergone a palatal resection, and who received treatment at any point from 2010 to 2022 were included in the review. CTs were not a specific inclusion criterion, though the quality of the scans was analyzed for eligible patients. Younger patients and those treated at VAPAHCS prior to 2010 were excluded.

There was no control group; all data was sourced from the US Department of Veterans Affairs (VA) imaging system database. Among the 3595 patients reviewed, 5 met inclusion criteria and the quality of their craniofacial anatomy CTs were analyzed. To maintain accurate craniofacial 3D modeling, CTs require a maximum of 1 mm slice thickness. Of the 5 patients who met the inclusion criteria, 4 were found to have variability in the quality of their CTs and severe defects not suitable for prosthetic reconstruction, which led to their exclusion from the study. One patient was investigated to demonstrate if making these prostheses was feasible. This patient was diagnosed with a malignant neoplasm of the hard palate, underwent a partial maxillectomy, and a palatal obturator was placed to cover the defect.

The primary data collected was patient identifiers as well as the gross anatomy and dimensions of the patients’ craniofacial anatomy, as seen in previous imaging research.²⁰ Before the imaging analysis, all personal health information was removed and the dataset was deidentified to ensure patient anonymity and noninvolvement.

CT Segmentation and 3D Printing

Using CTs of the patient’s craniofacial anatomy, we developed a model of the defects. This was achieved with deidentified CTs imported into the Food and Drug Administration (FDA)-approved computerized aid design (CAD) software, Materialise Mimics. The hard palate was segmented and isolated based off the presented scan and any holes in the image were filled using the CAD software. The model was subsequently mirrored in Materialise 3-matic to replicate an original anatomical hard palate prosthesis. The final product was converted into a 3D model and imported into Formlabs preform software to generate 3D printing supports and orient it for printing. The prosthetic was printed using FDA-approved Biocompatible Denture Base Resin by a Formlabs 3B+ printer at the Palo Alto VA Simulation Center. The 3D printed prosthesis was washed using Formlabs Form Wash 80% ethyl alcohol to remove excess resin and subsequently cured to harden the malleable resin. Supports were later removed, and the prosthesis was sanded.

The primary aim of this study was to investigate whether using CTs to create patient-specific prosthetic renderings for patients with head and neck cancer could be a feasible strategy. The CTs from the patient were successfully used to generate a 3D printed prosthesis, and the prosthesis matched the original craniofacial anatomy seen in the patient's imaging (Figure). These results demonstrate that high quality CTs can be used as a template for 3D printed prostheses for mild to moderate palatomaxillary defects.

3D Printing Costs

One liter of Denture Base Resin costs $299; prostheses use about 5 mL of resin. The average annual salary of a 3D printing technician in the United States is $42,717, or $20.54 per hour.²¹ For an experienced 3D printing technician, the time required to segment the hard palate and prepare it for 3D printing is 1 to 2 hours. The process may exceed 2 hours if the technician is presented with a lower quality CT or if the patient has a complex craniofacial anatomy.

The average time it takes to print a palatal prosthetic is 5 hours. An additional hour is needed for postprocessing, which includes washing and sanding. Therefore, the cost of the materials and labor for an average 3D printed prosthetic is about $150. A Formlabs 3B+ printer is competitively priced around $10,000. The cost for Materialise Mimics software varies, but is estimated at $16,000 at VAPAHCS. The prices for these 2 items are not included in our price estimation but should be taken into consideration.

Prosthodontist Process and Cost

The typical process of creating a palatal prosthesis by a prosthodontist begins by examining the patient, creating a stone model, then creating a wax model. Biocompatible materials are selected and processed into a mold that is trimmed and polished to the desired shape. This is followed by another patient visit to ensure the prosthesis fits properly. Follow-up care is also necessary for maintenance and comfort.

The average cost of a palatal prosthesis varies depending on the type needed (ie, metal implant, teeth replacement), the materials used, the region in which the patient is receiving care, and the complexity of the case. For complex and customizable options like those required for patients with cancer, the prostheses typically cost several thousands of dollars. The Healthcare Common Procedure Coding System code for a palatal lift prosthesis (D5955) lists prices ranging from $4000 to $8000 per prosthetic, not including the cost of the prosthodontist visits.^22,23

This program sought to determine whether imaging studies of maxillary defects are effective templates for developing 3D printed prosthetics and whether these prosthetics should be tested for future use in reconstruction of palatomaxillary defects. Our program illustrated that CTs served as feasible templates for developing hard palate prostheses for patients with palatomaxillary defects. It is important to note the CTs used were from a newer and more modern scanner and therefore yielded detailed palatal structures with higher accuracy more suitable for 3D modeling. Lower-quality CTs from the 4 patients excluded from the program were not suitable for 3D modeling. This suggests that with high-quality imaging, 3D printed prosthesis may be a viable strategy to help patients who struggle with their function following treatment for head and neck cancers.

3D printed prosthesis may also be a more patient centered and convenient option. In the traditional prosthesis creation workflow, the patient must physically bite down onto a resin (alginate or silicone) to make an impression, a very painful postoperative process that is irritating to the raw edges of the surgical bed.^15,16 Prosthodontists then create a prosthetic minus the tumor and typically secure it with clips or glue.¹⁷ Many patients also experience changes in their anatomy over time requiring them to have a new protheses created. This is particularly important in veterans with palatomaxillary defects since many VA medical centers do not have a prosthodontist on staff, making accessibility to these specialists difficult. 3D printing provides a contactless prosthetic creation process. This convenience may reduce a patient’s pain and the number of visits for which they need a specialist.

Future Directions

Additional research is needed to determine the full potential of 3D printed prosthetics. 3D printed prostheses have been effectively used for patient education in areas of presurgical planning, prosthesis creation, and trainee education.²⁴ This research represents an early step in the development of a new technology for use in otolaryngology. Specifically, many veterans with a history of head and neck cancers have sustained changes to their craniofacial anatomy following treatment. Using imaging to create 3D printed prosthetics could be very effective for these patients. Prosthetics could improve a patient’s quality of life by restoring/approximating their anatomy after cancer treatment.

Significant time and care must be taken by cancer and reconstructive surgeons to properly fit a prosthesis. Improperly fitting prosthetics leads to mucosal ulceration that then may lead to a need for fitting a new prosthetic. The advantage of 3D printed prosthetics is that they may more precisely fit the anatomy of each patient using CT results, thus potentially reducing the time needed to fit the prosthetic as well as the risk associated with an improperly fit prosthetic. 3D printed prosthesis could be used directly in the future, however, clinical trials are needed to verify its efficacy vs prosthodontic options.

Another consideration for potential future use of 3D printed prosthetics is cost. We estimated that the cost of the materials and labor of our 3D printed prosthetic to be about $150. Pricing of current molded prosthetics varies, but is often listed at several thousand dollars. Another consideration is the durability of 3D printed prosthetics vs standard prosthetics. Since we were unable to use the prosthetic in the patient, it was difficult to determine its durability. The significant cost of the 3D printer and software necessary for 3D printed prosthetics must also be considered and may be prohibitive. While many academic hospitals are considering the purchase of 3D printers and licenses, this may be challenging for resource-constrained institutions. 3D printing may also be difficult for groups without any prior experience in the field. Outsourcing to a third party is possible, though doing so adds more cost to the project. While we recognize there is a learning curve associated with adopting any new technology, it’s equally important to note that 3D printing is being rapidly integrated and has already made significant advancements in personalized medicine.^8,25,26

Limitations

This program had several limitations. First, we only obtained CTs of sufficient quality from 1 patient to generate a 3D printed prosthesis. Further research with additional patients is necessary to validate this process. Second, we were unable to trial the prosthesis in the patient because we did not have FDA approval. Additionally, it is difficult to calculate a true cost estimate for this process as materials and software costs vary dramatically across institutions as well as over time.

Conclusions

The purpose of this study was to demonstrate the possibility to develop prosthetics for the hard palate for patients suffering from palatomaxillary defects. A 3D printed prosthetic was generated that matched the patient’s craniofacial anatomy. Future research should test the feasibility of these prosthetics in patient care against a traditional prosthodontic impression. Though this is a proof-of-concept study and no prosthetics were implanted as part of this investigation, we showcase the feasibility of printing prosthetics for palatomaxillary defects. The use of 3D printed prosthetics may be a more humane process, potentially lower cost, and be more accessible to veterans.

Three-dimensional (3D) printing has become a promising area of innovation in biomedical research.^1,2 Previous research in orthopedic surgery has found that customized 3D printed implants, casts, orthoses, and prosthetics (eg, prosthetic hands) matched to an individual’s unique anatomy can result in more precise placement and better surgical outcomes.^3-5 Customized prosthetics have also been found to lead to fewer complications.^3,6

Recent advances in 3D printing technology has prompted investigation from surgeons to identify how this new tool may be incorporated into patient care.^1,7 One of the most common applications of 3D printing is during preoperative planning in which surgeons gain better insight into patient-specific anatomy by using patient-specific printed models.⁸ Another promising application is the production of customized prosthetics suited to each patient’s unique anatomy.⁹ As a result, 3D printing has significantly impacted bone and cartilage restoration procedures and has the potential to completely transform the treatment of patients with debilitating musculoskeletal injuries.^3,10

The potential surrounding 3D printed prosthetics has led to their adoption by several other specialties, including otolaryngology.¹¹ The most widely used application of 3D printing among otolaryngologists is preoperative planning, and the incorporation of printed prosthetics intoreconstruction of the orbit, nasal septum, auricle, and palate has also been reported.^2,12,13 Patient-specific implants might allow otolaryngologists to better rehabilitate, reconstruct, and/or regenerate craniofacial defects using more humane procedures.¹⁴

Patients with palatomaxillary cancers are treated by prosthodontists or otolaryngologists. An impression is made with a resin–which can be painful for postoperative patients–and a prosthetic is manufactured and implanted.^15-17 Patients with cancer often see many specialists, though reconstructive care is a low priority. Many of these individuals also experience dynamic anatomic functional changes over time, leading to the need for multiple prothesis.

palatomaxillary prosthetics

This program aims to use patients’ previous computed tomography (CT) to tailor customized 3D printed palatomaxillary prosthetics to specifically fit their anatomy. Palatomaxillary defects are a source of profound disability for patients with head and neck cancers who are left with large anatomic defects as a direct result of treatment. Reconstruction of palatal defects poses unique challenges due to the complexity of patient anatomy.^18,19

3D printed prosthetics for palatomaxillary defects have not been incorporated into patient care. We reviewed previous imaging research to determine if it could be used to assist patients who struggle with their function and appearance following treatment for head and neck cancers. The primary aim was to investigate whether 3D printing was a feasible strategy for creating patient-specific palatomaxillary prosthetics. The secondary aim is to determine whether these prosthetics should be tested in the future for use in reconstruction of maxillary defects.

This study was conducted at the Veterans Affairs Palo Alto Health Care System (VAPAHCS) and was approved by the Stanford University Institutional Review Board (approval #28958, informed consent and patient contact excluded). A retrospective chart review was conducted on all patients with head and neck cancers who were treated at VAPAHCS from 2010 to 2022. Patients aged ≥ 18 years who had a palatomaxillary defect due to cancer treatment, had undergone a palatal resection, and who received treatment at any point from 2010 to 2022 were included in the review. CTs were not a specific inclusion criterion, though the quality of the scans was analyzed for eligible patients. Younger patients and those treated at VAPAHCS prior to 2010 were excluded.

There was no control group; all data was sourced from the US Department of Veterans Affairs (VA) imaging system database. Among the 3595 patients reviewed, 5 met inclusion criteria and the quality of their craniofacial anatomy CTs were analyzed. To maintain accurate craniofacial 3D modeling, CTs require a maximum of 1 mm slice thickness. Of the 5 patients who met the inclusion criteria, 4 were found to have variability in the quality of their CTs and severe defects not suitable for prosthetic reconstruction, which led to their exclusion from the study. One patient was investigated to demonstrate if making these prostheses was feasible. This patient was diagnosed with a malignant neoplasm of the hard palate, underwent a partial maxillectomy, and a palatal obturator was placed to cover the defect.

The primary data collected was patient identifiers as well as the gross anatomy and dimensions of the patients’ craniofacial anatomy, as seen in previous imaging research.²⁰ Before the imaging analysis, all personal health information was removed and the dataset was deidentified to ensure patient anonymity and noninvolvement.

CT Segmentation and 3D Printing

Using CTs of the patient’s craniofacial anatomy, we developed a model of the defects. This was achieved with deidentified CTs imported into the Food and Drug Administration (FDA)-approved computerized aid design (CAD) software, Materialise Mimics. The hard palate was segmented and isolated based off the presented scan and any holes in the image were filled using the CAD software. The model was subsequently mirrored in Materialise 3-matic to replicate an original anatomical hard palate prosthesis. The final product was converted into a 3D model and imported into Formlabs preform software to generate 3D printing supports and orient it for printing. The prosthetic was printed using FDA-approved Biocompatible Denture Base Resin by a Formlabs 3B+ printer at the Palo Alto VA Simulation Center. The 3D printed prosthesis was washed using Formlabs Form Wash 80% ethyl alcohol to remove excess resin and subsequently cured to harden the malleable resin. Supports were later removed, and the prosthesis was sanded.

The primary aim of this study was to investigate whether using CTs to create patient-specific prosthetic renderings for patients with head and neck cancer could be a feasible strategy. The CTs from the patient were successfully used to generate a 3D printed prosthesis, and the prosthesis matched the original craniofacial anatomy seen in the patient's imaging (Figure). These results demonstrate that high quality CTs can be used as a template for 3D printed prostheses for mild to moderate palatomaxillary defects.

3D Printing Costs

One liter of Denture Base Resin costs $299; prostheses use about 5 mL of resin. The average annual salary of a 3D printing technician in the United States is $42,717, or $20.54 per hour.²¹ For an experienced 3D printing technician, the time required to segment the hard palate and prepare it for 3D printing is 1 to 2 hours. The process may exceed 2 hours if the technician is presented with a lower quality CT or if the patient has a complex craniofacial anatomy.

The average time it takes to print a palatal prosthetic is 5 hours. An additional hour is needed for postprocessing, which includes washing and sanding. Therefore, the cost of the materials and labor for an average 3D printed prosthetic is about $150. A Formlabs 3B+ printer is competitively priced around $10,000. The cost for Materialise Mimics software varies, but is estimated at $16,000 at VAPAHCS. The prices for these 2 items are not included in our price estimation but should be taken into consideration.

Prosthodontist Process and Cost

The typical process of creating a palatal prosthesis by a prosthodontist begins by examining the patient, creating a stone model, then creating a wax model. Biocompatible materials are selected and processed into a mold that is trimmed and polished to the desired shape. This is followed by another patient visit to ensure the prosthesis fits properly. Follow-up care is also necessary for maintenance and comfort.

The average cost of a palatal prosthesis varies depending on the type needed (ie, metal implant, teeth replacement), the materials used, the region in which the patient is receiving care, and the complexity of the case. For complex and customizable options like those required for patients with cancer, the prostheses typically cost several thousands of dollars. The Healthcare Common Procedure Coding System code for a palatal lift prosthesis (D5955) lists prices ranging from $4000 to $8000 per prosthetic, not including the cost of the prosthodontist visits.^22,23

This program sought to determine whether imaging studies of maxillary defects are effective templates for developing 3D printed prosthetics and whether these prosthetics should be tested for future use in reconstruction of palatomaxillary defects. Our program illustrated that CTs served as feasible templates for developing hard palate prostheses for patients with palatomaxillary defects. It is important to note the CTs used were from a newer and more modern scanner and therefore yielded detailed palatal structures with higher accuracy more suitable for 3D modeling. Lower-quality CTs from the 4 patients excluded from the program were not suitable for 3D modeling. This suggests that with high-quality imaging, 3D printed prosthesis may be a viable strategy to help patients who struggle with their function following treatment for head and neck cancers.

3D printed prosthesis may also be a more patient centered and convenient option. In the traditional prosthesis creation workflow, the patient must physically bite down onto a resin (alginate or silicone) to make an impression, a very painful postoperative process that is irritating to the raw edges of the surgical bed.^15,16 Prosthodontists then create a prosthetic minus the tumor and typically secure it with clips or glue.¹⁷ Many patients also experience changes in their anatomy over time requiring them to have a new protheses created. This is particularly important in veterans with palatomaxillary defects since many VA medical centers do not have a prosthodontist on staff, making accessibility to these specialists difficult. 3D printing provides a contactless prosthetic creation process. This convenience may reduce a patient’s pain and the number of visits for which they need a specialist.

Future Directions

Additional research is needed to determine the full potential of 3D printed prosthetics. 3D printed prostheses have been effectively used for patient education in areas of presurgical planning, prosthesis creation, and trainee education.²⁴ This research represents an early step in the development of a new technology for use in otolaryngology. Specifically, many veterans with a history of head and neck cancers have sustained changes to their craniofacial anatomy following treatment. Using imaging to create 3D printed prosthetics could be very effective for these patients. Prosthetics could improve a patient’s quality of life by restoring/approximating their anatomy after cancer treatment.

Significant time and care must be taken by cancer and reconstructive surgeons to properly fit a prosthesis. Improperly fitting prosthetics leads to mucosal ulceration that then may lead to a need for fitting a new prosthetic. The advantage of 3D printed prosthetics is that they may more precisely fit the anatomy of each patient using CT results, thus potentially reducing the time needed to fit the prosthetic as well as the risk associated with an improperly fit prosthetic. 3D printed prosthesis could be used directly in the future, however, clinical trials are needed to verify its efficacy vs prosthodontic options.

Another consideration for potential future use of 3D printed prosthetics is cost. We estimated that the cost of the materials and labor of our 3D printed prosthetic to be about $150. Pricing of current molded prosthetics varies, but is often listed at several thousand dollars. Another consideration is the durability of 3D printed prosthetics vs standard prosthetics. Since we were unable to use the prosthetic in the patient, it was difficult to determine its durability. The significant cost of the 3D printer and software necessary for 3D printed prosthetics must also be considered and may be prohibitive. While many academic hospitals are considering the purchase of 3D printers and licenses, this may be challenging for resource-constrained institutions. 3D printing may also be difficult for groups without any prior experience in the field. Outsourcing to a third party is possible, though doing so adds more cost to the project. While we recognize there is a learning curve associated with adopting any new technology, it’s equally important to note that 3D printing is being rapidly integrated and has already made significant advancements in personalized medicine.^8,25,26

Limitations

This program had several limitations. First, we only obtained CTs of sufficient quality from 1 patient to generate a 3D printed prosthesis. Further research with additional patients is necessary to validate this process. Second, we were unable to trial the prosthesis in the patient because we did not have FDA approval. Additionally, it is difficult to calculate a true cost estimate for this process as materials and software costs vary dramatically across institutions as well as over time.

Conclusions

The purpose of this study was to demonstrate the possibility to develop prosthetics for the hard palate for patients suffering from palatomaxillary defects. A 3D printed prosthetic was generated that matched the patient’s craniofacial anatomy. Future research should test the feasibility of these prosthetics in patient care against a traditional prosthodontic impression. Though this is a proof-of-concept study and no prosthetics were implanted as part of this investigation, we showcase the feasibility of printing prosthetics for palatomaxillary defects. The use of 3D printed prosthetics may be a more humane process, potentially lower cost, and be more accessible to veterans.

Three-dimensional (3D) printing has become a promising area of innovation in biomedical research.^1,2 Previous research in orthopedic surgery has found that customized 3D printed implants, casts, orthoses, and prosthetics (eg, prosthetic hands) matched to an individual’s unique anatomy can result in more precise placement and better surgical outcomes.^3-5 Customized prosthetics have also been found to lead to fewer complications.^3,6

Recent advances in 3D printing technology has prompted investigation from surgeons to identify how this new tool may be incorporated into patient care.^1,7 One of the most common applications of 3D printing is during preoperative planning in which surgeons gain better insight into patient-specific anatomy by using patient-specific printed models.⁸ Another promising application is the production of customized prosthetics suited to each patient’s unique anatomy.⁹ As a result, 3D printing has significantly impacted bone and cartilage restoration procedures and has the potential to completely transform the treatment of patients with debilitating musculoskeletal injuries.^3,10

The potential surrounding 3D printed prosthetics has led to their adoption by several other specialties, including otolaryngology.¹¹ The most widely used application of 3D printing among otolaryngologists is preoperative planning, and the incorporation of printed prosthetics intoreconstruction of the orbit, nasal septum, auricle, and palate has also been reported.^2,12,13 Patient-specific implants might allow otolaryngologists to better rehabilitate, reconstruct, and/or regenerate craniofacial defects using more humane procedures.¹⁴

Patients with palatomaxillary cancers are treated by prosthodontists or otolaryngologists. An impression is made with a resin–which can be painful for postoperative patients–and a prosthetic is manufactured and implanted.^15-17 Patients with cancer often see many specialists, though reconstructive care is a low priority. Many of these individuals also experience dynamic anatomic functional changes over time, leading to the need for multiple prothesis.

palatomaxillary prosthetics

This program aims to use patients’ previous computed tomography (CT) to tailor customized 3D printed palatomaxillary prosthetics to specifically fit their anatomy. Palatomaxillary defects are a source of profound disability for patients with head and neck cancers who are left with large anatomic defects as a direct result of treatment. Reconstruction of palatal defects poses unique challenges due to the complexity of patient anatomy.^18,19

3D printed prosthetics for palatomaxillary defects have not been incorporated into patient care. We reviewed previous imaging research to determine if it could be used to assist patients who struggle with their function and appearance following treatment for head and neck cancers. The primary aim was to investigate whether 3D printing was a feasible strategy for creating patient-specific palatomaxillary prosthetics. The secondary aim is to determine whether these prosthetics should be tested in the future for use in reconstruction of maxillary defects.

This study was conducted at the Veterans Affairs Palo Alto Health Care System (VAPAHCS) and was approved by the Stanford University Institutional Review Board (approval #28958, informed consent and patient contact excluded). A retrospective chart review was conducted on all patients with head and neck cancers who were treated at VAPAHCS from 2010 to 2022. Patients aged ≥ 18 years who had a palatomaxillary defect due to cancer treatment, had undergone a palatal resection, and who received treatment at any point from 2010 to 2022 were included in the review. CTs were not a specific inclusion criterion, though the quality of the scans was analyzed for eligible patients. Younger patients and those treated at VAPAHCS prior to 2010 were excluded.

There was no control group; all data was sourced from the US Department of Veterans Affairs (VA) imaging system database. Among the 3595 patients reviewed, 5 met inclusion criteria and the quality of their craniofacial anatomy CTs were analyzed. To maintain accurate craniofacial 3D modeling, CTs require a maximum of 1 mm slice thickness. Of the 5 patients who met the inclusion criteria, 4 were found to have variability in the quality of their CTs and severe defects not suitable for prosthetic reconstruction, which led to their exclusion from the study. One patient was investigated to demonstrate if making these prostheses was feasible. This patient was diagnosed with a malignant neoplasm of the hard palate, underwent a partial maxillectomy, and a palatal obturator was placed to cover the defect.

The primary data collected was patient identifiers as well as the gross anatomy and dimensions of the patients’ craniofacial anatomy, as seen in previous imaging research.²⁰ Before the imaging analysis, all personal health information was removed and the dataset was deidentified to ensure patient anonymity and noninvolvement.

CT Segmentation and 3D Printing

Using CTs of the patient’s craniofacial anatomy, we developed a model of the defects. This was achieved with deidentified CTs imported into the Food and Drug Administration (FDA)-approved computerized aid design (CAD) software, Materialise Mimics. The hard palate was segmented and isolated based off the presented scan and any holes in the image were filled using the CAD software. The model was subsequently mirrored in Materialise 3-matic to replicate an original anatomical hard palate prosthesis. The final product was converted into a 3D model and imported into Formlabs preform software to generate 3D printing supports and orient it for printing. The prosthetic was printed using FDA-approved Biocompatible Denture Base Resin by a Formlabs 3B+ printer at the Palo Alto VA Simulation Center. The 3D printed prosthesis was washed using Formlabs Form Wash 80% ethyl alcohol to remove excess resin and subsequently cured to harden the malleable resin. Supports were later removed, and the prosthesis was sanded.

The primary aim of this study was to investigate whether using CTs to create patient-specific prosthetic renderings for patients with head and neck cancer could be a feasible strategy. The CTs from the patient were successfully used to generate a 3D printed prosthesis, and the prosthesis matched the original craniofacial anatomy seen in the patient's imaging (Figure). These results demonstrate that high quality CTs can be used as a template for 3D printed prostheses for mild to moderate palatomaxillary defects.

3D Printing Costs

One liter of Denture Base Resin costs $299; prostheses use about 5 mL of resin. The average annual salary of a 3D printing technician in the United States is $42,717, or $20.54 per hour.²¹ For an experienced 3D printing technician, the time required to segment the hard palate and prepare it for 3D printing is 1 to 2 hours. The process may exceed 2 hours if the technician is presented with a lower quality CT or if the patient has a complex craniofacial anatomy.

The average time it takes to print a palatal prosthetic is 5 hours. An additional hour is needed for postprocessing, which includes washing and sanding. Therefore, the cost of the materials and labor for an average 3D printed prosthetic is about $150. A Formlabs 3B+ printer is competitively priced around $10,000. The cost for Materialise Mimics software varies, but is estimated at $16,000 at VAPAHCS. The prices for these 2 items are not included in our price estimation but should be taken into consideration.

Prosthodontist Process and Cost

The typical process of creating a palatal prosthesis by a prosthodontist begins by examining the patient, creating a stone model, then creating a wax model. Biocompatible materials are selected and processed into a mold that is trimmed and polished to the desired shape. This is followed by another patient visit to ensure the prosthesis fits properly. Follow-up care is also necessary for maintenance and comfort.

The average cost of a palatal prosthesis varies depending on the type needed (ie, metal implant, teeth replacement), the materials used, the region in which the patient is receiving care, and the complexity of the case. For complex and customizable options like those required for patients with cancer, the prostheses typically cost several thousands of dollars. The Healthcare Common Procedure Coding System code for a palatal lift prosthesis (D5955) lists prices ranging from $4000 to $8000 per prosthetic, not including the cost of the prosthodontist visits.^22,23

This program sought to determine whether imaging studies of maxillary defects are effective templates for developing 3D printed prosthetics and whether these prosthetics should be tested for future use in reconstruction of palatomaxillary defects. Our program illustrated that CTs served as feasible templates for developing hard palate prostheses for patients with palatomaxillary defects. It is important to note the CTs used were from a newer and more modern scanner and therefore yielded detailed palatal structures with higher accuracy more suitable for 3D modeling. Lower-quality CTs from the 4 patients excluded from the program were not suitable for 3D modeling. This suggests that with high-quality imaging, 3D printed prosthesis may be a viable strategy to help patients who struggle with their function following treatment for head and neck cancers.

3D printed prosthesis may also be a more patient centered and convenient option. In the traditional prosthesis creation workflow, the patient must physically bite down onto a resin (alginate or silicone) to make an impression, a very painful postoperative process that is irritating to the raw edges of the surgical bed.^15,16 Prosthodontists then create a prosthetic minus the tumor and typically secure it with clips or glue.¹⁷ Many patients also experience changes in their anatomy over time requiring them to have a new protheses created. This is particularly important in veterans with palatomaxillary defects since many VA medical centers do not have a prosthodontist on staff, making accessibility to these specialists difficult. 3D printing provides a contactless prosthetic creation process. This convenience may reduce a patient’s pain and the number of visits for which they need a specialist.

Future Directions

Additional research is needed to determine the full potential of 3D printed prosthetics. 3D printed prostheses have been effectively used for patient education in areas of presurgical planning, prosthesis creation, and trainee education.²⁴ This research represents an early step in the development of a new technology for use in otolaryngology. Specifically, many veterans with a history of head and neck cancers have sustained changes to their craniofacial anatomy following treatment. Using imaging to create 3D printed prosthetics could be very effective for these patients. Prosthetics could improve a patient’s quality of life by restoring/approximating their anatomy after cancer treatment.

Significant time and care must be taken by cancer and reconstructive surgeons to properly fit a prosthesis. Improperly fitting prosthetics leads to mucosal ulceration that then may lead to a need for fitting a new prosthetic. The advantage of 3D printed prosthetics is that they may more precisely fit the anatomy of each patient using CT results, thus potentially reducing the time needed to fit the prosthetic as well as the risk associated with an improperly fit prosthetic. 3D printed prosthesis could be used directly in the future, however, clinical trials are needed to verify its efficacy vs prosthodontic options.

Another consideration for potential future use of 3D printed prosthetics is cost. We estimated that the cost of the materials and labor of our 3D printed prosthetic to be about $150. Pricing of current molded prosthetics varies, but is often listed at several thousand dollars. Another consideration is the durability of 3D printed prosthetics vs standard prosthetics. Since we were unable to use the prosthetic in the patient, it was difficult to determine its durability. The significant cost of the 3D printer and software necessary for 3D printed prosthetics must also be considered and may be prohibitive. While many academic hospitals are considering the purchase of 3D printers and licenses, this may be challenging for resource-constrained institutions. 3D printing may also be difficult for groups without any prior experience in the field. Outsourcing to a third party is possible, though doing so adds more cost to the project. While we recognize there is a learning curve associated with adopting any new technology, it’s equally important to note that 3D printing is being rapidly integrated and has already made significant advancements in personalized medicine.^8,25,26

Limitations

This program had several limitations. First, we only obtained CTs of sufficient quality from 1 patient to generate a 3D printed prosthesis. Further research with additional patients is necessary to validate this process. Second, we were unable to trial the prosthesis in the patient because we did not have FDA approval. Additionally, it is difficult to calculate a true cost estimate for this process as materials and software costs vary dramatically across institutions as well as over time.

Conclusions

The purpose of this study was to demonstrate the possibility to develop prosthetics for the hard palate for patients suffering from palatomaxillary defects. A 3D printed prosthetic was generated that matched the patient’s craniofacial anatomy. Future research should test the feasibility of these prosthetics in patient care against a traditional prosthodontic impression. Though this is a proof-of-concept study and no prosthetics were implanted as part of this investigation, we showcase the feasibility of printing prosthetics for palatomaxillary defects. The use of 3D printed prosthetics may be a more humane process, potentially lower cost, and be more accessible to veterans.

Colorectal cancer (CRC) is the third-most common cancer worldwide and accounts for almost 11% of all cancer diagnoses, with > 1.9 million cases reported globally.^1,2 CRC is the second-most deadly cancer, responsible for about 935,000 deaths.¹ Over the past several decades, a steady decline in CRC incidence and mortality has been reported in developed countries, including the US.^3,4 From 2008 through 2017, an annual reduction of 3% in CRC death rates was reported in individuals aged ≥ 65 years.⁵ This decline can mainly be attributed to improvements made in health systems and advancements in CRC screening programs.^3,5

US Preventive Services Task Force (USPSTF) recommends CRC screening in individuals aged 45 to 75 years. USPSTF recommends direct visualization tests, such as colonoscopy and flexible sigmoidoscopy for CRC screening.⁶ Although colonoscopy is commonly used for CRC screening, it is an invasive procedure that requires bowel preparation and sedation, and has the potential risk of colonic perforation, bleeding, and infection. Additionally, social determinants—such as health care costs, missed work, and geographic location (eg, rural communities)—may limit colonoscopy utilization.⁷ As a result, other cost-effective, noninvasive tests such as high-sensitivity guaiac-based fecal occult blood test (gFOBT) and fecal immunochemical test (FIT) are also used for CRC screening. These tests detect occult blood in the stool of individuals who may be at risk for CRC, helping direct them to colonoscopy if they screen positive.⁸

The gFOBT relies on simple oxidation and requires a stool sample to detect the presence of the heme component of blood.⁹ If heme is present in the stool sample, it will enable the oxidation of guaiac to form a blue-colored dye when added to hydrogen peroxide. It is important to note that the oxidation component of this test may lead to false-positive results, as it may detect dietary hemoglobin present in red meat. Medications or foods that have peroxidase properties may also result in a false-positive gFOBT result. Additionally, false-negative results may be caused by antioxidants, which may interfere with the oxidation of guaiac.

FIT uses antibodies, which bind to the intact globin component of human hemoglobin.⁹ The quantity of bound antibody-hemoglobin complex is detected and measured by a variety of automated quantitative techniques. This testing strategy eliminates the need for food or medication restrictions and the subjective visual assessment of change in color, as required for the gFOBT.⁹ A 2016 meta-analysis found that FIT performed better compared with gFOBT in terms of specificity, positivity rate, number needed to scope, and number needed to screen.⁸ The FIT screening method has also been found to have greater adherence rates, which is likely due to fewer stool sampling requirements and the lack of medication or dietary restrictions, compared with gFOBT.^7,8

The COVID-19 pandemic had a drastic impact on CRC preventive care services. In March 2020, elective colonoscopies were temporarily ceased across the country and the US Department of Veterans Affairs (VA) deferred all elective surgeries and medical procedures, including screening and surveillance colonoscopies. In line with these recommendations, elective colonoscopies were temporarily ceased across the country.¹⁰ The National Cancer Institute’s Population-Based Research to Optimize the Screening Process consortium reported that CRC screening rates decreased by 82% across the US in 2020.¹¹ Public health measures are likely the main reason for this decline, but other factors may include a lack of resource availability in outpatient settings and public fear of the pandemic.¹⁰

The James A. Haley Veterans Affairs Hospital (JAHVAH) in Tampa, Florida, encouraged the use of FIT in place of colonoscopies to avoid delaying preventive services. The initiative to continue CRC screening methods via FIT was scrutinized when laboratory personnel reported that in fiscal year (FY) 2020, 62% of the FIT kits that patients returned to the laboratory were missing information or had other errors (Figure 1). These improperly returned FIT kits led to delayed processing, canceled orders, increased staff workload, and more costs for FIT repetition.

Research shows many patients often fail to adhere to the instructions for proper FIT sample collection and return. Wang and colleagues reported that of 4916 FIT samples returned to the laboratory, 971 (20%) had collection errors, and 910 (94%) of those samples were missing a sample collection date.¹² The sample collection date is important because hemoglobin degradation occurs over time, which may create false-negative FIT results. Although studies have found that sample return times of ≤ 10 days are not associated with a decrease in FIT positive rates, it is recommended to mail completed FITs within 24 hours of sample collection.¹³

Because remote screening methods like FIT were preferred during the COVID-19 pandemic, we conducted a quality improvement (QI) project to address FIT inefficiency. The aim of this initiative was to determine the root cause behind incorrectly returned FIT kits and to increase correctly collected and testable FIT kits upon initial laboratory arrival by at least 20% by the second quarter of FY 2021.

Quality Improvement Project

This QI project was conducted from July 2020 to June 2021 at the JAHVAH, which provides primary care and specialty health services to veterans in central and south Florida. The QI was designed based on the Plan-Do-Study-Act (PDSA) model of health care improvement. The QI team consisted of physicians, nurses, administrative staff, and laboratory personnel. A SIPOC (Suppliers, Input, Process, Output, Customers) map was initially designed to help clarify the different groups involved in the process of FIT kit distribution and return. This map helped the team decide who should be involved in the solution process.

The QI team performed a root cause analysis using a fishbone diagram and identified the reasons FIT kits were returned to the laboratory with errors that prevented processing. The team brainstormed potential change ideas and created an impact vs effort chart to increase the number of correctly returned and testable FIT kits upon initial arrival at the laboratory by at least 20% by the second quarter of FY 2021. We identified strengths and prioritized change ideas to improve the number of testable and correctly returned FIT kits to the hospital laboratory. These ideas included centralizing FIT kit dispersal to a new administrative group, building redundant patient reminders on kit completion and giving patients more accessible places for kit return.

Patients included in the study were adults aged 50 to 75 years seen at the JAHVAH outpatient clinic who were asked to undergo FIT CRC screening. FIT orders for other facilities were excluded. The primary endpoint of this project was to improve the number of correctly returned FITs. The number of correct and incorrect returned FITs were measured from July 2020 to June 2021. FITs returned with errors were categorized by the type of error, including: no order on file in the electronic health record (EHR), canceled test, expired test, unable to identify test, missing information, and missing collection date.

We attempted to calculate costs of FITs that were returned to the laboratory but could not be analyzed and were discarded. In FY 2020, 1568 FITs were discarded. Each FIT cost about $7.80 to process for an annualized expense of $12,230 for discarded FITs.

Root Cause Analysis

Root causes were obtained by making a fishbone diagram. From this diagram, an impact vs effort chart was created to form and prioritize ideas for our PDSA cycles. Data about correctly and incorrectly returned kits were collected monthly from laboratory personnel, then analyzed by the QI team using run charts to look for change in frequency and patterns.

To improve this process, a swim lane chart for FIT processing was assembled and later used to make a comprehensive fishbone diagram to establish the 6 main root cause errors: missing FIT EHR order, cancelled FIT EHR order, expired stool specimen, partial patient identifiers, no patient identifiers, and no stool collection date. Pareto and run charts were superimposed with the laboratory data. The most common cause of incorrectly returned FITs was no collection date.

PDSA Cycles

Beginning in January 2021, PDSA cycles from the ideas in the impact vs effort chart were used. Organization and implementation of the project occurred from July 2020 to April 2021. The team reassessed the data in April 2021 to evaluate progress after PDSA initiation. The mean rate of missing collection date dropped from 24% in FY 2020 prior to PDSA cycles to 14% in April 2021; however, the number of incorrectly returned kits was similar to the baseline level. When reviewing this discrepancy, the QI team found that although the missing collection date rate had improved, the rate of FITs with not enough information had increased from 5% in FY 2020 to 67% in April 2021 (Figure 2). After discussing with laboratory personnel, it was determined that the EHR order was missing when the process pathway changed. Our PDSA initiative changed the process pathway and different individuals were responsible for FIT dispersal. The error was quickly addressed with the help of clinical and administrative staff; a 30-day follow-up on June 21, 2021, revealed that only 9% of the patients had sent back kits with not enough information.

After troubleshooting, the team achieved a sustainable increase in the number of correctly returned FIT kits from an average of 38% before the project to 72% after 30-day follow-up.

Proper collection and return of FIT samples are vital for process efficiency for both physicians and patients. This initiative aimed to improve the rate of correctly returned FIT kits by 20%, but its final numbers showed an improvement of 33.6%. Operational benefits from this project included early detection of CRC, improved laboratory workflow, decreased FIT kit waste, and increased patient satisfaction.

The multipronged PDSA cycle attempted to increase the rate of correctly returned FIT kits. We improved kit comprehension and laboratory accessibility, and instituted redundant return reminders for patients. We also centralized a new process pathway for FIT distribution and educated physicians and support staff. Sampling and FIT return may seem like a simple procedure, but the FIT can be cumbersome for patients and directions can be confusing. Therefore, to maximize screening participation, it is essential to minimize confusion in the collection and return of a FIT sample.^14,15

This QI initiative was presented at Grand Rounds at the University of South Florida in June 2021 and has since been shared with other VA hospitals. It was also presented at the American College of Gastroenterology Conference in 2021.

Limitations

This study was a single-center QI project and focused mostly on FIT kit return rates. To fully address CRC screening, it is important to ensure that individuals with a positive screen are appropriately followed up with a colonoscopy. Although follow-up was not in the scope of this project, it is key to CRC screening in general and should be the subject of future research.

Conclusions

FIT is a useful method for CRC screening that can be particularly helpful when in-person visits are limited, as seen during the COVID-19 pandemic. This increase in demand for FITs during the pandemic revealed process deficiencies and gave JAHVAH an opportunity to improve workflow. Through the aid of a multidisciplinary team, the process to complete and return FITs improved and surpassed the goal of 20% improvement. Our goal is to continue to fine-tune the workflow and troubleshoot the system as needed.

Colorectal cancer (CRC) is the third-most common cancer worldwide and accounts for almost 11% of all cancer diagnoses, with > 1.9 million cases reported globally.^1,2 CRC is the second-most deadly cancer, responsible for about 935,000 deaths.¹ Over the past several decades, a steady decline in CRC incidence and mortality has been reported in developed countries, including the US.^3,4 From 2008 through 2017, an annual reduction of 3% in CRC death rates was reported in individuals aged ≥ 65 years.⁵ This decline can mainly be attributed to improvements made in health systems and advancements in CRC screening programs.^3,5

US Preventive Services Task Force (USPSTF) recommends CRC screening in individuals aged 45 to 75 years. USPSTF recommends direct visualization tests, such as colonoscopy and flexible sigmoidoscopy for CRC screening.⁶ Although colonoscopy is commonly used for CRC screening, it is an invasive procedure that requires bowel preparation and sedation, and has the potential risk of colonic perforation, bleeding, and infection. Additionally, social determinants—such as health care costs, missed work, and geographic location (eg, rural communities)—may limit colonoscopy utilization.⁷ As a result, other cost-effective, noninvasive tests such as high-sensitivity guaiac-based fecal occult blood test (gFOBT) and fecal immunochemical test (FIT) are also used for CRC screening. These tests detect occult blood in the stool of individuals who may be at risk for CRC, helping direct them to colonoscopy if they screen positive.⁸

The gFOBT relies on simple oxidation and requires a stool sample to detect the presence of the heme component of blood.⁹ If heme is present in the stool sample, it will enable the oxidation of guaiac to form a blue-colored dye when added to hydrogen peroxide. It is important to note that the oxidation component of this test may lead to false-positive results, as it may detect dietary hemoglobin present in red meat. Medications or foods that have peroxidase properties may also result in a false-positive gFOBT result. Additionally, false-negative results may be caused by antioxidants, which may interfere with the oxidation of guaiac.

FIT uses antibodies, which bind to the intact globin component of human hemoglobin.⁹ The quantity of bound antibody-hemoglobin complex is detected and measured by a variety of automated quantitative techniques. This testing strategy eliminates the need for food or medication restrictions and the subjective visual assessment of change in color, as required for the gFOBT.⁹ A 2016 meta-analysis found that FIT performed better compared with gFOBT in terms of specificity, positivity rate, number needed to scope, and number needed to screen.⁸ The FIT screening method has also been found to have greater adherence rates, which is likely due to fewer stool sampling requirements and the lack of medication or dietary restrictions, compared with gFOBT.^7,8

The COVID-19 pandemic had a drastic impact on CRC preventive care services. In March 2020, elective colonoscopies were temporarily ceased across the country and the US Department of Veterans Affairs (VA) deferred all elective surgeries and medical procedures, including screening and surveillance colonoscopies. In line with these recommendations, elective colonoscopies were temporarily ceased across the country.¹⁰ The National Cancer Institute’s Population-Based Research to Optimize the Screening Process consortium reported that CRC screening rates decreased by 82% across the US in 2020.¹¹ Public health measures are likely the main reason for this decline, but other factors may include a lack of resource availability in outpatient settings and public fear of the pandemic.¹⁰

The James A. Haley Veterans Affairs Hospital (JAHVAH) in Tampa, Florida, encouraged the use of FIT in place of colonoscopies to avoid delaying preventive services. The initiative to continue CRC screening methods via FIT was scrutinized when laboratory personnel reported that in fiscal year (FY) 2020, 62% of the FIT kits that patients returned to the laboratory were missing information or had other errors (Figure 1). These improperly returned FIT kits led to delayed processing, canceled orders, increased staff workload, and more costs for FIT repetition.

Research shows many patients often fail to adhere to the instructions for proper FIT sample collection and return. Wang and colleagues reported that of 4916 FIT samples returned to the laboratory, 971 (20%) had collection errors, and 910 (94%) of those samples were missing a sample collection date.¹² The sample collection date is important because hemoglobin degradation occurs over time, which may create false-negative FIT results. Although studies have found that sample return times of ≤ 10 days are not associated with a decrease in FIT positive rates, it is recommended to mail completed FITs within 24 hours of sample collection.¹³

Because remote screening methods like FIT were preferred during the COVID-19 pandemic, we conducted a quality improvement (QI) project to address FIT inefficiency. The aim of this initiative was to determine the root cause behind incorrectly returned FIT kits and to increase correctly collected and testable FIT kits upon initial laboratory arrival by at least 20% by the second quarter of FY 2021.

Quality Improvement Project

This QI project was conducted from July 2020 to June 2021 at the JAHVAH, which provides primary care and specialty health services to veterans in central and south Florida. The QI was designed based on the Plan-Do-Study-Act (PDSA) model of health care improvement. The QI team consisted of physicians, nurses, administrative staff, and laboratory personnel. A SIPOC (Suppliers, Input, Process, Output, Customers) map was initially designed to help clarify the different groups involved in the process of FIT kit distribution and return. This map helped the team decide who should be involved in the solution process.

The QI team performed a root cause analysis using a fishbone diagram and identified the reasons FIT kits were returned to the laboratory with errors that prevented processing. The team brainstormed potential change ideas and created an impact vs effort chart to increase the number of correctly returned and testable FIT kits upon initial arrival at the laboratory by at least 20% by the second quarter of FY 2021. We identified strengths and prioritized change ideas to improve the number of testable and correctly returned FIT kits to the hospital laboratory. These ideas included centralizing FIT kit dispersal to a new administrative group, building redundant patient reminders on kit completion and giving patients more accessible places for kit return.

Patients included in the study were adults aged 50 to 75 years seen at the JAHVAH outpatient clinic who were asked to undergo FIT CRC screening. FIT orders for other facilities were excluded. The primary endpoint of this project was to improve the number of correctly returned FITs. The number of correct and incorrect returned FITs were measured from July 2020 to June 2021. FITs returned with errors were categorized by the type of error, including: no order on file in the electronic health record (EHR), canceled test, expired test, unable to identify test, missing information, and missing collection date.

We attempted to calculate costs of FITs that were returned to the laboratory but could not be analyzed and were discarded. In FY 2020, 1568 FITs were discarded. Each FIT cost about $7.80 to process for an annualized expense of $12,230 for discarded FITs.

Root Cause Analysis

Root causes were obtained by making a fishbone diagram. From this diagram, an impact vs effort chart was created to form and prioritize ideas for our PDSA cycles. Data about correctly and incorrectly returned kits were collected monthly from laboratory personnel, then analyzed by the QI team using run charts to look for change in frequency and patterns.

To improve this process, a swim lane chart for FIT processing was assembled and later used to make a comprehensive fishbone diagram to establish the 6 main root cause errors: missing FIT EHR order, cancelled FIT EHR order, expired stool specimen, partial patient identifiers, no patient identifiers, and no stool collection date. Pareto and run charts were superimposed with the laboratory data. The most common cause of incorrectly returned FITs was no collection date.

PDSA Cycles

Beginning in January 2021, PDSA cycles from the ideas in the impact vs effort chart were used. Organization and implementation of the project occurred from July 2020 to April 2021. The team reassessed the data in April 2021 to evaluate progress after PDSA initiation. The mean rate of missing collection date dropped from 24% in FY 2020 prior to PDSA cycles to 14% in April 2021; however, the number of incorrectly returned kits was similar to the baseline level. When reviewing this discrepancy, the QI team found that although the missing collection date rate had improved, the rate of FITs with not enough information had increased from 5% in FY 2020 to 67% in April 2021 (Figure 2). After discussing with laboratory personnel, it was determined that the EHR order was missing when the process pathway changed. Our PDSA initiative changed the process pathway and different individuals were responsible for FIT dispersal. The error was quickly addressed with the help of clinical and administrative staff; a 30-day follow-up on June 21, 2021, revealed that only 9% of the patients had sent back kits with not enough information.

After troubleshooting, the team achieved a sustainable increase in the number of correctly returned FIT kits from an average of 38% before the project to 72% after 30-day follow-up.

Proper collection and return of FIT samples are vital for process efficiency for both physicians and patients. This initiative aimed to improve the rate of correctly returned FIT kits by 20%, but its final numbers showed an improvement of 33.6%. Operational benefits from this project included early detection of CRC, improved laboratory workflow, decreased FIT kit waste, and increased patient satisfaction.

The multipronged PDSA cycle attempted to increase the rate of correctly returned FIT kits. We improved kit comprehension and laboratory accessibility, and instituted redundant return reminders for patients. We also centralized a new process pathway for FIT distribution and educated physicians and support staff. Sampling and FIT return may seem like a simple procedure, but the FIT can be cumbersome for patients and directions can be confusing. Therefore, to maximize screening participation, it is essential to minimize confusion in the collection and return of a FIT sample.^14,15

This QI initiative was presented at Grand Rounds at the University of South Florida in June 2021 and has since been shared with other VA hospitals. It was also presented at the American College of Gastroenterology Conference in 2021.

Limitations

This study was a single-center QI project and focused mostly on FIT kit return rates. To fully address CRC screening, it is important to ensure that individuals with a positive screen are appropriately followed up with a colonoscopy. Although follow-up was not in the scope of this project, it is key to CRC screening in general and should be the subject of future research.

Conclusions

FIT is a useful method for CRC screening that can be particularly helpful when in-person visits are limited, as seen during the COVID-19 pandemic. This increase in demand for FITs during the pandemic revealed process deficiencies and gave JAHVAH an opportunity to improve workflow. Through the aid of a multidisciplinary team, the process to complete and return FITs improved and surpassed the goal of 20% improvement. Our goal is to continue to fine-tune the workflow and troubleshoot the system as needed.

Colorectal cancer (CRC) is the third-most common cancer worldwide and accounts for almost 11% of all cancer diagnoses, with > 1.9 million cases reported globally.^1,2 CRC is the second-most deadly cancer, responsible for about 935,000 deaths.¹ Over the past several decades, a steady decline in CRC incidence and mortality has been reported in developed countries, including the US.^3,4 From 2008 through 2017, an annual reduction of 3% in CRC death rates was reported in individuals aged ≥ 65 years.⁵ This decline can mainly be attributed to improvements made in health systems and advancements in CRC screening programs.^3,5

US Preventive Services Task Force (USPSTF) recommends CRC screening in individuals aged 45 to 75 years. USPSTF recommends direct visualization tests, such as colonoscopy and flexible sigmoidoscopy for CRC screening.⁶ Although colonoscopy is commonly used for CRC screening, it is an invasive procedure that requires bowel preparation and sedation, and has the potential risk of colonic perforation, bleeding, and infection. Additionally, social determinants—such as health care costs, missed work, and geographic location (eg, rural communities)—may limit colonoscopy utilization.⁷ As a result, other cost-effective, noninvasive tests such as high-sensitivity guaiac-based fecal occult blood test (gFOBT) and fecal immunochemical test (FIT) are also used for CRC screening. These tests detect occult blood in the stool of individuals who may be at risk for CRC, helping direct them to colonoscopy if they screen positive.⁸

The gFOBT relies on simple oxidation and requires a stool sample to detect the presence of the heme component of blood.⁹ If heme is present in the stool sample, it will enable the oxidation of guaiac to form a blue-colored dye when added to hydrogen peroxide. It is important to note that the oxidation component of this test may lead to false-positive results, as it may detect dietary hemoglobin present in red meat. Medications or foods that have peroxidase properties may also result in a false-positive gFOBT result. Additionally, false-negative results may be caused by antioxidants, which may interfere with the oxidation of guaiac.

FIT uses antibodies, which bind to the intact globin component of human hemoglobin.⁹ The quantity of bound antibody-hemoglobin complex is detected and measured by a variety of automated quantitative techniques. This testing strategy eliminates the need for food or medication restrictions and the subjective visual assessment of change in color, as required for the gFOBT.⁹ A 2016 meta-analysis found that FIT performed better compared with gFOBT in terms of specificity, positivity rate, number needed to scope, and number needed to screen.⁸ The FIT screening method has also been found to have greater adherence rates, which is likely due to fewer stool sampling requirements and the lack of medication or dietary restrictions, compared with gFOBT.^7,8

The COVID-19 pandemic had a drastic impact on CRC preventive care services. In March 2020, elective colonoscopies were temporarily ceased across the country and the US Department of Veterans Affairs (VA) deferred all elective surgeries and medical procedures, including screening and surveillance colonoscopies. In line with these recommendations, elective colonoscopies were temporarily ceased across the country.¹⁰ The National Cancer Institute’s Population-Based Research to Optimize the Screening Process consortium reported that CRC screening rates decreased by 82% across the US in 2020.¹¹ Public health measures are likely the main reason for this decline, but other factors may include a lack of resource availability in outpatient settings and public fear of the pandemic.¹⁰

The James A. Haley Veterans Affairs Hospital (JAHVAH) in Tampa, Florida, encouraged the use of FIT in place of colonoscopies to avoid delaying preventive services. The initiative to continue CRC screening methods via FIT was scrutinized when laboratory personnel reported that in fiscal year (FY) 2020, 62% of the FIT kits that patients returned to the laboratory were missing information or had other errors (Figure 1). These improperly returned FIT kits led to delayed processing, canceled orders, increased staff workload, and more costs for FIT repetition.

Research shows many patients often fail to adhere to the instructions for proper FIT sample collection and return. Wang and colleagues reported that of 4916 FIT samples returned to the laboratory, 971 (20%) had collection errors, and 910 (94%) of those samples were missing a sample collection date.¹² The sample collection date is important because hemoglobin degradation occurs over time, which may create false-negative FIT results. Although studies have found that sample return times of ≤ 10 days are not associated with a decrease in FIT positive rates, it is recommended to mail completed FITs within 24 hours of sample collection.¹³

Because remote screening methods like FIT were preferred during the COVID-19 pandemic, we conducted a quality improvement (QI) project to address FIT inefficiency. The aim of this initiative was to determine the root cause behind incorrectly returned FIT kits and to increase correctly collected and testable FIT kits upon initial laboratory arrival by at least 20% by the second quarter of FY 2021.

Quality Improvement Project

This QI project was conducted from July 2020 to June 2021 at the JAHVAH, which provides primary care and specialty health services to veterans in central and south Florida. The QI was designed based on the Plan-Do-Study-Act (PDSA) model of health care improvement. The QI team consisted of physicians, nurses, administrative staff, and laboratory personnel. A SIPOC (Suppliers, Input, Process, Output, Customers) map was initially designed to help clarify the different groups involved in the process of FIT kit distribution and return. This map helped the team decide who should be involved in the solution process.

The QI team performed a root cause analysis using a fishbone diagram and identified the reasons FIT kits were returned to the laboratory with errors that prevented processing. The team brainstormed potential change ideas and created an impact vs effort chart to increase the number of correctly returned and testable FIT kits upon initial arrival at the laboratory by at least 20% by the second quarter of FY 2021. We identified strengths and prioritized change ideas to improve the number of testable and correctly returned FIT kits to the hospital laboratory. These ideas included centralizing FIT kit dispersal to a new administrative group, building redundant patient reminders on kit completion and giving patients more accessible places for kit return.

Patients included in the study were adults aged 50 to 75 years seen at the JAHVAH outpatient clinic who were asked to undergo FIT CRC screening. FIT orders for other facilities were excluded. The primary endpoint of this project was to improve the number of correctly returned FITs. The number of correct and incorrect returned FITs were measured from July 2020 to June 2021. FITs returned with errors were categorized by the type of error, including: no order on file in the electronic health record (EHR), canceled test, expired test, unable to identify test, missing information, and missing collection date.

We attempted to calculate costs of FITs that were returned to the laboratory but could not be analyzed and were discarded. In FY 2020, 1568 FITs were discarded. Each FIT cost about $7.80 to process for an annualized expense of $12,230 for discarded FITs.

Root Cause Analysis

Root causes were obtained by making a fishbone diagram. From this diagram, an impact vs effort chart was created to form and prioritize ideas for our PDSA cycles. Data about correctly and incorrectly returned kits were collected monthly from laboratory personnel, then analyzed by the QI team using run charts to look for change in frequency and patterns.

To improve this process, a swim lane chart for FIT processing was assembled and later used to make a comprehensive fishbone diagram to establish the 6 main root cause errors: missing FIT EHR order, cancelled FIT EHR order, expired stool specimen, partial patient identifiers, no patient identifiers, and no stool collection date. Pareto and run charts were superimposed with the laboratory data. The most common cause of incorrectly returned FITs was no collection date.

PDSA Cycles

Beginning in January 2021, PDSA cycles from the ideas in the impact vs effort chart were used. Organization and implementation of the project occurred from July 2020 to April 2021. The team reassessed the data in April 2021 to evaluate progress after PDSA initiation. The mean rate of missing collection date dropped from 24% in FY 2020 prior to PDSA cycles to 14% in April 2021; however, the number of incorrectly returned kits was similar to the baseline level. When reviewing this discrepancy, the QI team found that although the missing collection date rate had improved, the rate of FITs with not enough information had increased from 5% in FY 2020 to 67% in April 2021 (Figure 2). After discussing with laboratory personnel, it was determined that the EHR order was missing when the process pathway changed. Our PDSA initiative changed the process pathway and different individuals were responsible for FIT dispersal. The error was quickly addressed with the help of clinical and administrative staff; a 30-day follow-up on June 21, 2021, revealed that only 9% of the patients had sent back kits with not enough information.

After troubleshooting, the team achieved a sustainable increase in the number of correctly returned FIT kits from an average of 38% before the project to 72% after 30-day follow-up.

Proper collection and return of FIT samples are vital for process efficiency for both physicians and patients. This initiative aimed to improve the rate of correctly returned FIT kits by 20%, but its final numbers showed an improvement of 33.6%. Operational benefits from this project included early detection of CRC, improved laboratory workflow, decreased FIT kit waste, and increased patient satisfaction.

The multipronged PDSA cycle attempted to increase the rate of correctly returned FIT kits. We improved kit comprehension and laboratory accessibility, and instituted redundant return reminders for patients. We also centralized a new process pathway for FIT distribution and educated physicians and support staff. Sampling and FIT return may seem like a simple procedure, but the FIT can be cumbersome for patients and directions can be confusing. Therefore, to maximize screening participation, it is essential to minimize confusion in the collection and return of a FIT sample.^14,15

This QI initiative was presented at Grand Rounds at the University of South Florida in June 2021 and has since been shared with other VA hospitals. It was also presented at the American College of Gastroenterology Conference in 2021.

Limitations

This study was a single-center QI project and focused mostly on FIT kit return rates. To fully address CRC screening, it is important to ensure that individuals with a positive screen are appropriately followed up with a colonoscopy. Although follow-up was not in the scope of this project, it is key to CRC screening in general and should be the subject of future research.

Conclusions

FIT is a useful method for CRC screening that can be particularly helpful when in-person visits are limited, as seen during the COVID-19 pandemic. This increase in demand for FITs during the pandemic revealed process deficiencies and gave JAHVAH an opportunity to improve workflow. Through the aid of a multidisciplinary team, the process to complete and return FITs improved and surpassed the goal of 20% improvement. Our goal is to continue to fine-tune the workflow and troubleshoot the system as needed.

The US Department of Veterans Affairs (VA) has partnered with academic medical centers and programs since 1946 to provide clinical training for physician residents. Ranking second in federal graduate medical education (GME) funding to the Centers for Medicare and Medicaid Services (CMS), the $850 million VA GME budget annually reimburses > 250 GME-sponsoring institutions (affiliates) of 8000 GME programs for the clinical training of 49,000 individual residents rotating through > 11,000 full-time equivalent (FTE) positions.¹ The VA also distributes $1.6 billion to VA facilities to offset the costs of conducting health professions education (HPE) (eg, facility infrastructure, salary support for VA instructors and preceptors, education office administration, and instructional equipment).² The VA financial and educational contributions account for payment of 11% of resident positions nationally and allow academic medical centers to be less reliant on CMS GME funding.^3,4 The VA contributions also provide opportunities for GME expansion,^1,5,6 educational innovations,^5,7 interprofessional and team-based care,^8,9 and quality and safety training.^10,11 The Table provides a comparison of CMS and VA GME reimbursability based on activity.

GME financing is complex, particularly the formulaic approach used by CMS, the details of which are often obscured in federal regulations. Due to this complexity and the $16 billion CMS GME budget, academic publications have focused on CMS GME financing while not fully explaining the VA GME policies and processes.^4,12-14 By comparison, the VA GME financing model is relatively straightforward and governed by different statues and VA regulations, yet sharing some of the same principles as CMS regulations. Given the challenges in CMS reimbursement to fully support the cost of resident education, as well as the educational opportunities at the VA, the VA designs its reimbursement model to assure that affiliates receive appropriate payments.^4,12,15 To ensure the continued success of VA GME partnerships, knowledge of VA GME financing has become increasingly important for designated institutional officers (DIOs) and residency program directors, particularly in light of recent investigations into oversight of the VA’s reimbursement to academic affiliates.^16-18 This report describes VA GME reimbursement and, where applicable, VA and CMS reimbursement policies are compared to highlight similarities, differences, and common principles.

VA AUTHORITY

While the VA’s primary mission is “to provide a complete hospital medical service for the medical care and treatment of veterans,”early VA leaders recognized the importance of affiliating with the nation’s academic institutions.¹⁹ In 1946, the VA Policy Memorandum Number 2 established a partnership between the VA and the academic medical community.²⁰ Additional legislation authorized specific agreements with academic affiliates for the central administration of salary and benefits for residents rotating at VA facilities. This process, known as disbursement, is an alternative payroll mechanism whereby the VA reimburses the academic affiliate for resident salary and benefits and the affiliate acts as the disbursing agent, issuing paychecks to residents.^21,22

Resident FUNDING

By policy, with rare exceptions, the VA does not sponsor residency programs due to the challenges of providing an appropriate patient mix of age, sex, and medical conditions to meet accreditation standards.⁴ Nearly all VA reimbursements are for residents in affiliate-sponsored programs, while just 1% pays for residents in legacy, VA-sponsored residency programs at 2 VA facilities. The VA budget for resident (including fellows) salary and benefits is managed by the VA Office of Academic Affiliations (OAA), the national VA office responsible for oversight, policy, and funding of VA HPE programs.

Resident Salaries and Benefits

VA funding of resident salary and benefits are analogous with CMS direct GME (DGME), which is designed to cover resident salary and benefits costs.^4,14,23 CMS DGME payments depend on a hospital’s volume of CMS inpatients and are based on a statutory formula, which uses the hospital’s resident FTE positions, the per-resident amount, and Medicare’s share of inpatient beds (Medicare patient load) to determine payments.¹² The per-resident amount is set by statute, varies geographically, and is calculated by dividing the hospital’s allowable costs of GME (percentage of CMS inpatient days) divided by the number of residents.^12,24

By comparison, the VA GME payment reimburses for each FTE based on the salary and benefits rate set by the academic affiliate. Reimbursement is calculated based on resident time spent at the VA multiplied by a daily salary rate. The daily salary rate is determined by dividing the resident’s total compensation (salary and benefits) by the number of calendar days in an academic year. Resident time spent at the VA facility is determined by obtaining rotation schedules provided by the academic affiliate and verifying resident clinical and educational activity during scheduled rotations.

Indirect Medical Education Funding

In addition to resident salary and benefits, funds to offset the cost of conducting HPE are provided to VA facilities. These funds are intended to improve and maintain necessary infrastructure for all HPE programs not just GME, including education office administration needs, teaching costs (ie, a portion of VA preceptors salary), and instructional equipment.

The Veterans Equitable Resource Allocation (VERA) is a national budgeting process for VA medical facilities that funds facility operational needs such as staff salary and benefits, infrastructure, and equipment.² The education portion of the VERA, the VERA Education Support Component (VESC), is not managed by the OAA, but rather is distributed through the VERA model to the general budget of VA facilities hosting HPE (Figure). VESC funding in the VA budget is based on labor mapping of physician time spent in education; other labor mapping categories include clinical care, research, and administration. VA facility VESC funding is calculated based on the number of paid health profession trainees (HPTs) from all professions, apportioned according to the number of FTEs for physician residents and VA-paid HPTs in other disciplines. In fiscal year 2024, VA facilities received $115,812 for each physician resident FTE position and $84,906 for each VA-paid, non-GME FTE position.

The VESC is like CMS's indirect GME funding, termed Indirect Medical Education (IME), an additional payment for each Medicare patient discharged reflecting teaching hospitals’ higher patient care costs relative to nonteaching hospitals. Described elsewhere, IME is calculated using a resident-to-bed ratio and a multiplier, which is set by statute.^4,25 While IME can be used for reimbursement for some resident clinical and educational activities(eg, research), VA VESC funds cannot be used for such activities and are part of the general facility budget and appropriated per the discretion of the medical facility director.

ESTABLISHING GME PARTNERSHIPS

An affiliation agreement establishes the administrative and legal requirements for educational relationships with academic affiliates and includes standards for conducting HPE, responsibilities for accreditation standards, program leadership, faculty, resources, supervision, academic policies, and procedures. The VA uses standardized affiliation agreement templates that have been vetted with accrediting bodies and the VA Office of General Counsel.

A disbursement agreement authorizes the VA to reimburse affiliates for resident salary and benefits for VA clinical and educational activities. The disbursement agreement details the fiscal arrangements (eg, payment in advance vs arrears, salary, and benefit rates, leave) for the reimbursement payments. Veterans Health Administration (VHA) Directive 1400.05 provides the policy and procedures for calculating reimbursement for HPT educational activities.²⁶

The VA facility designated education officer (DEO) oversees all HPE programs and coordinates the affiliation and disbursement agreement processes.²⁷ The DEO, affiliate DIO, residency program director, and VA residency site director determine the physician resident FTE positions assigned to a VA facility based on educational objectives and availability of educational resources at the VA facility, such as patient care opportunities, faculty supervisors, space, and equipment. The VA facility requests for resident FTE positions are submitted to the OAA by the facility DEO.

Once GME FTE positions are approved by the OAA, VA facilities work with their academic affiliate to submit the physician resident salary and benefit rate. Affiliate DIOs attest to the accuracy of the salary rate schedule and the local DEO submits the budget request to the OAA. Upon approval, the funds are transferred to the VA facility each fiscal year, which begins October 1. DEOs report quarterly to the OAA both budget needs and excesses based on variations in the approved FTEs due to additional VA rotations, physician resident attrition, or reassignment.

Resident Position Allocation

VA GME financing provides flexibility through periodic needs assessments and expansion initiatives. In August and December, DEOs collaborate with an academic affiliate to submit reports to the OAA confirming their projected GME needs for the next academic year. Additional positions requests are reviewed by the OAA; funding depends on budget and the educational justification. The OAA periodically issues GME expansion requests for proposal, which typically arise from legislation to address specific VA workforce needs. The VA facility DEO and affiliate GME leaders collaborate to apply for additional positions. For example, a VA GME expansion under the Veterans Access, Choice, and Accountability Act of 2014 added 1500 GME positions in 8 years for critically needed specialties and in rural and underserved areas.⁵ The Maintaining Internal Systems and Strengthening Outside Networks (MISSION) Act of 2018 authorized a pilot program for VA to fund residents at non-VA facilities with priority for Indian Health Services, Tribes and Tribal Organizations, Federally Qualified Health Centers, and US Department of Defense facilities to provide access to veterans in underserved areas.⁶

The VA GME financing system has flexibility to meet local needs for additional resident positions and to address broader VA workforce gaps through targeted expansion. Generally, CMS does not fund positions to address workforce needs, place residents in specific geographic areas, or require the training of certain types of residents.⁴ However, the Consolidated Appropriations Act of 2021 has provided the opportunity to address rural workforce needs.²⁸

Reimbursement

The VA provides reimbursement for clinical and educational activities performed in VA facilities for the benefit of veterans as well as research, didactics, meetings and conferences, annual and sick leave, and orientation. The VA also may provide reimbursement for educational activities that occur off VA grounds (eg, the VA proportional share of a residency program’s didactic sessions). The VA does not reimburse for affiliate clinical duties or administrative costs, although a national policy allows VA facilities to reimburse affiliates for some GME overhead costs.²⁹

CMS similarly reimburses for residency training time spent in patient care activities as well as orientation activities, didactics, leave, and, in some cases, research.^4,30,31 CMS makes payments to hospitals, which may include sponsoring institutions and Medicare-eligible participating training sites.^4,30,31 For both the VA and CMS, residents may not be counted twice for reimbursement by 2 federal agencies; in other words, a resident may not count for > 1 FTE.^4,30-32

GME Oversight

VA GME funding came under significant scrutiny. At a 2016 House Veterans Affairs Committee hearing, Representative Phil Roe, MD (R-Tennessee), noted that no process existed at many VA facilities for “determining trainee presence” and that many VA medical centers had “difficulty tracking resident rotations”¹⁶ A VA Office of the Inspector General investigation recommended that the VA implement policies and procedures to improve oversight to “ensure residents are fully participating in educational activities” and that the VA is “paying the correct amount” to the affiliate.¹⁷ A 2020 General Accountability Office report outlined unclear policy guidance, incomplete tracking of resident activities, and improper fiscal processes for reimbursement and reconciliation of affiliate invoices.¹⁸

In response, the OAA created an oversight and compliance unit, revised VHA Directive 1400.05 (the policy for disbursement), and improved resident tracking procedures.²⁶ The standard operating procedure that accompanied VHA Directive 1400.05 provides detailed information for the DEO and VA facility staff for tracking resident clinical and educational activities. FTE counts are essential to both VA and CMS for accurate reimbursement. The eAppendix and the Table provide a guide to reimbursable activities in the VA for the calculation of reimbursement, with a comparison to CMS.^33,34 The OAA in cooperation with other VA staff and officers periodically conducts audits to assess compliance with disbursement policy and affiliate reimbursement accuracy.

In the VA, resident activities are captured on the VA Educational Activity Record, a standardized spreadsheet to track activities and calculate reimbursement. Each VA facility hosting resident physicians manually records resident activity by the half-day. This process is labor intensive, involving both VA and affiliate staff to accurately reconcile payments. To address the workload demands, the OAA is developing an online tool that will automate aspects of the tracking process. Also, to ensure adequate staffing, the OAA is in the process of implementing an office optimization project, providing standardized position descriptions, an organizational chart, and staffing levels for DEO offices in VA facilities.

This report describes the key policies and principles of VA GME financing, highlighting the essential similarities and differences between VA and CMS. Neither the VA nor CMS regulations allow for reimbursement for > 1 FTE position per resident, a principle that underpins the assignment of resident rotations and federal funding for GME and are similar with respect to reimbursement for patient care activities, didactics, research, orientation, and scholarly activity. While reimbursable activities in the VA require physical presence and care of veteran patients, CMS also limits reimbursement to resident activities in the hospital and approved other settings if the hospital is paying for resident salary and benefits in these settings. The VA provides some flexibility for offsite activities including didactics and, in specific circumstances, remote care of veteran patients (eg, teleradiology).

The VA and CMS use different GME financing models. For example, the CMS calculations for resident FTEs are complex, whereas VA calculations reimburse the salary and benefits as set by the academic affiliate. The VA process accounts for local variation in salary rates, whereas the per-resident amount set by CMS varies regionally and does not fully account for differences in the cost of living.²⁴ Because all patients in VA facilities are veterans, VA calculations for reimbursement do not involve ratios of beds like the CMS calculations to determine a proportional share of reimbursement. The VA GME expansion tends to be more directed to VA health workforce needs than CMS, specifying the types of programs and geographic locations to address these needs.

The VA regularly reevaluates how affiliates are reimbursed for VA resident activity, balancing compliance with VA policies and the workload for VA and its affiliates. The VA obtains input from key stakeholders including DEOs, DIOs, and professional organizations such as the Association of American Medical Colleges and the Accreditation Council for Graduate Medical Education.^35,36

Looking ahead, the VA is developing an online tool to improve the accuracy of affiliate reimbursement. The VA will also implement a standardized staffing model, organizational structure, and position descriptions for DEO offices. These initiatives will help reduce the burden of tracking and verifying resident activity and continue to support the 77-year partnership between VA and its affiliated institutions.

The US Department of Veterans Affairs (VA) has partnered with academic medical centers and programs since 1946 to provide clinical training for physician residents. Ranking second in federal graduate medical education (GME) funding to the Centers for Medicare and Medicaid Services (CMS), the $850 million VA GME budget annually reimburses > 250 GME-sponsoring institutions (affiliates) of 8000 GME programs for the clinical training of 49,000 individual residents rotating through > 11,000 full-time equivalent (FTE) positions.¹ The VA also distributes $1.6 billion to VA facilities to offset the costs of conducting health professions education (HPE) (eg, facility infrastructure, salary support for VA instructors and preceptors, education office administration, and instructional equipment).² The VA financial and educational contributions account for payment of 11% of resident positions nationally and allow academic medical centers to be less reliant on CMS GME funding.^3,4 The VA contributions also provide opportunities for GME expansion,^1,5,6 educational innovations,^5,7 interprofessional and team-based care,^8,9 and quality and safety training.^10,11 The Table provides a comparison of CMS and VA GME reimbursability based on activity.

GME financing is complex, particularly the formulaic approach used by CMS, the details of which are often obscured in federal regulations. Due to this complexity and the $16 billion CMS GME budget, academic publications have focused on CMS GME financing while not fully explaining the VA GME policies and processes.^4,12-14 By comparison, the VA GME financing model is relatively straightforward and governed by different statues and VA regulations, yet sharing some of the same principles as CMS regulations. Given the challenges in CMS reimbursement to fully support the cost of resident education, as well as the educational opportunities at the VA, the VA designs its reimbursement model to assure that affiliates receive appropriate payments.^4,12,15 To ensure the continued success of VA GME partnerships, knowledge of VA GME financing has become increasingly important for designated institutional officers (DIOs) and residency program directors, particularly in light of recent investigations into oversight of the VA’s reimbursement to academic affiliates.^16-18 This report describes VA GME reimbursement and, where applicable, VA and CMS reimbursement policies are compared to highlight similarities, differences, and common principles.

VA AUTHORITY

While the VA’s primary mission is “to provide a complete hospital medical service for the medical care and treatment of veterans,”early VA leaders recognized the importance of affiliating with the nation’s academic institutions.¹⁹ In 1946, the VA Policy Memorandum Number 2 established a partnership between the VA and the academic medical community.²⁰ Additional legislation authorized specific agreements with academic affiliates for the central administration of salary and benefits for residents rotating at VA facilities. This process, known as disbursement, is an alternative payroll mechanism whereby the VA reimburses the academic affiliate for resident salary and benefits and the affiliate acts as the disbursing agent, issuing paychecks to residents.^21,22

Resident FUNDING

By policy, with rare exceptions, the VA does not sponsor residency programs due to the challenges of providing an appropriate patient mix of age, sex, and medical conditions to meet accreditation standards.⁴ Nearly all VA reimbursements are for residents in affiliate-sponsored programs, while just 1% pays for residents in legacy, VA-sponsored residency programs at 2 VA facilities. The VA budget for resident (including fellows) salary and benefits is managed by the VA Office of Academic Affiliations (OAA), the national VA office responsible for oversight, policy, and funding of VA HPE programs.

Resident Salaries and Benefits

VA funding of resident salary and benefits are analogous with CMS direct GME (DGME), which is designed to cover resident salary and benefits costs.^4,14,23 CMS DGME payments depend on a hospital’s volume of CMS inpatients and are based on a statutory formula, which uses the hospital’s resident FTE positions, the per-resident amount, and Medicare’s share of inpatient beds (Medicare patient load) to determine payments.¹² The per-resident amount is set by statute, varies geographically, and is calculated by dividing the hospital’s allowable costs of GME (percentage of CMS inpatient days) divided by the number of residents.^12,24

By comparison, the VA GME payment reimburses for each FTE based on the salary and benefits rate set by the academic affiliate. Reimbursement is calculated based on resident time spent at the VA multiplied by a daily salary rate. The daily salary rate is determined by dividing the resident’s total compensation (salary and benefits) by the number of calendar days in an academic year. Resident time spent at the VA facility is determined by obtaining rotation schedules provided by the academic affiliate and verifying resident clinical and educational activity during scheduled rotations.

Indirect Medical Education Funding

In addition to resident salary and benefits, funds to offset the cost of conducting HPE are provided to VA facilities. These funds are intended to improve and maintain necessary infrastructure for all HPE programs not just GME, including education office administration needs, teaching costs (ie, a portion of VA preceptors salary), and instructional equipment.

The Veterans Equitable Resource Allocation (VERA) is a national budgeting process for VA medical facilities that funds facility operational needs such as staff salary and benefits, infrastructure, and equipment.² The education portion of the VERA, the VERA Education Support Component (VESC), is not managed by the OAA, but rather is distributed through the VERA model to the general budget of VA facilities hosting HPE (Figure). VESC funding in the VA budget is based on labor mapping of physician time spent in education; other labor mapping categories include clinical care, research, and administration. VA facility VESC funding is calculated based on the number of paid health profession trainees (HPTs) from all professions, apportioned according to the number of FTEs for physician residents and VA-paid HPTs in other disciplines. In fiscal year 2024, VA facilities received $115,812 for each physician resident FTE position and $84,906 for each VA-paid, non-GME FTE position.

The VESC is like CMS's indirect GME funding, termed Indirect Medical Education (IME), an additional payment for each Medicare patient discharged reflecting teaching hospitals’ higher patient care costs relative to nonteaching hospitals. Described elsewhere, IME is calculated using a resident-to-bed ratio and a multiplier, which is set by statute.^4,25 While IME can be used for reimbursement for some resident clinical and educational activities(eg, research), VA VESC funds cannot be used for such activities and are part of the general facility budget and appropriated per the discretion of the medical facility director.

ESTABLISHING GME PARTNERSHIPS

An affiliation agreement establishes the administrative and legal requirements for educational relationships with academic affiliates and includes standards for conducting HPE, responsibilities for accreditation standards, program leadership, faculty, resources, supervision, academic policies, and procedures. The VA uses standardized affiliation agreement templates that have been vetted with accrediting bodies and the VA Office of General Counsel.

A disbursement agreement authorizes the VA to reimburse affiliates for resident salary and benefits for VA clinical and educational activities. The disbursement agreement details the fiscal arrangements (eg, payment in advance vs arrears, salary, and benefit rates, leave) for the reimbursement payments. Veterans Health Administration (VHA) Directive 1400.05 provides the policy and procedures for calculating reimbursement for HPT educational activities.²⁶

The VA facility designated education officer (DEO) oversees all HPE programs and coordinates the affiliation and disbursement agreement processes.²⁷ The DEO, affiliate DIO, residency program director, and VA residency site director determine the physician resident FTE positions assigned to a VA facility based on educational objectives and availability of educational resources at the VA facility, such as patient care opportunities, faculty supervisors, space, and equipment. The VA facility requests for resident FTE positions are submitted to the OAA by the facility DEO.

Once GME FTE positions are approved by the OAA, VA facilities work with their academic affiliate to submit the physician resident salary and benefit rate. Affiliate DIOs attest to the accuracy of the salary rate schedule and the local DEO submits the budget request to the OAA. Upon approval, the funds are transferred to the VA facility each fiscal year, which begins October 1. DEOs report quarterly to the OAA both budget needs and excesses based on variations in the approved FTEs due to additional VA rotations, physician resident attrition, or reassignment.

Resident Position Allocation

VA GME financing provides flexibility through periodic needs assessments and expansion initiatives. In August and December, DEOs collaborate with an academic affiliate to submit reports to the OAA confirming their projected GME needs for the next academic year. Additional positions requests are reviewed by the OAA; funding depends on budget and the educational justification. The OAA periodically issues GME expansion requests for proposal, which typically arise from legislation to address specific VA workforce needs. The VA facility DEO and affiliate GME leaders collaborate to apply for additional positions. For example, a VA GME expansion under the Veterans Access, Choice, and Accountability Act of 2014 added 1500 GME positions in 8 years for critically needed specialties and in rural and underserved areas.⁵ The Maintaining Internal Systems and Strengthening Outside Networks (MISSION) Act of 2018 authorized a pilot program for VA to fund residents at non-VA facilities with priority for Indian Health Services, Tribes and Tribal Organizations, Federally Qualified Health Centers, and US Department of Defense facilities to provide access to veterans in underserved areas.⁶

The VA GME financing system has flexibility to meet local needs for additional resident positions and to address broader VA workforce gaps through targeted expansion. Generally, CMS does not fund positions to address workforce needs, place residents in specific geographic areas, or require the training of certain types of residents.⁴ However, the Consolidated Appropriations Act of 2021 has provided the opportunity to address rural workforce needs.²⁸

Reimbursement

The VA provides reimbursement for clinical and educational activities performed in VA facilities for the benefit of veterans as well as research, didactics, meetings and conferences, annual and sick leave, and orientation. The VA also may provide reimbursement for educational activities that occur off VA grounds (eg, the VA proportional share of a residency program’s didactic sessions). The VA does not reimburse for affiliate clinical duties or administrative costs, although a national policy allows VA facilities to reimburse affiliates for some GME overhead costs.²⁹

CMS similarly reimburses for residency training time spent in patient care activities as well as orientation activities, didactics, leave, and, in some cases, research.^4,30,31 CMS makes payments to hospitals, which may include sponsoring institutions and Medicare-eligible participating training sites.^4,30,31 For both the VA and CMS, residents may not be counted twice for reimbursement by 2 federal agencies; in other words, a resident may not count for > 1 FTE.^4,30-32

GME Oversight

VA GME funding came under significant scrutiny. At a 2016 House Veterans Affairs Committee hearing, Representative Phil Roe, MD (R-Tennessee), noted that no process existed at many VA facilities for “determining trainee presence” and that many VA medical centers had “difficulty tracking resident rotations”¹⁶ A VA Office of the Inspector General investigation recommended that the VA implement policies and procedures to improve oversight to “ensure residents are fully participating in educational activities” and that the VA is “paying the correct amount” to the affiliate.¹⁷ A 2020 General Accountability Office report outlined unclear policy guidance, incomplete tracking of resident activities, and improper fiscal processes for reimbursement and reconciliation of affiliate invoices.¹⁸

In response, the OAA created an oversight and compliance unit, revised VHA Directive 1400.05 (the policy for disbursement), and improved resident tracking procedures.²⁶ The standard operating procedure that accompanied VHA Directive 1400.05 provides detailed information for the DEO and VA facility staff for tracking resident clinical and educational activities. FTE counts are essential to both VA and CMS for accurate reimbursement. The eAppendix and the Table provide a guide to reimbursable activities in the VA for the calculation of reimbursement, with a comparison to CMS.^33,34 The OAA in cooperation with other VA staff and officers periodically conducts audits to assess compliance with disbursement policy and affiliate reimbursement accuracy.

In the VA, resident activities are captured on the VA Educational Activity Record, a standardized spreadsheet to track activities and calculate reimbursement. Each VA facility hosting resident physicians manually records resident activity by the half-day. This process is labor intensive, involving both VA and affiliate staff to accurately reconcile payments. To address the workload demands, the OAA is developing an online tool that will automate aspects of the tracking process. Also, to ensure adequate staffing, the OAA is in the process of implementing an office optimization project, providing standardized position descriptions, an organizational chart, and staffing levels for DEO offices in VA facilities.

This report describes the key policies and principles of VA GME financing, highlighting the essential similarities and differences between VA and CMS. Neither the VA nor CMS regulations allow for reimbursement for > 1 FTE position per resident, a principle that underpins the assignment of resident rotations and federal funding for GME and are similar with respect to reimbursement for patient care activities, didactics, research, orientation, and scholarly activity. While reimbursable activities in the VA require physical presence and care of veteran patients, CMS also limits reimbursement to resident activities in the hospital and approved other settings if the hospital is paying for resident salary and benefits in these settings. The VA provides some flexibility for offsite activities including didactics and, in specific circumstances, remote care of veteran patients (eg, teleradiology).

The VA and CMS use different GME financing models. For example, the CMS calculations for resident FTEs are complex, whereas VA calculations reimburse the salary and benefits as set by the academic affiliate. The VA process accounts for local variation in salary rates, whereas the per-resident amount set by CMS varies regionally and does not fully account for differences in the cost of living.²⁴ Because all patients in VA facilities are veterans, VA calculations for reimbursement do not involve ratios of beds like the CMS calculations to determine a proportional share of reimbursement. The VA GME expansion tends to be more directed to VA health workforce needs than CMS, specifying the types of programs and geographic locations to address these needs.

The VA regularly reevaluates how affiliates are reimbursed for VA resident activity, balancing compliance with VA policies and the workload for VA and its affiliates. The VA obtains input from key stakeholders including DEOs, DIOs, and professional organizations such as the Association of American Medical Colleges and the Accreditation Council for Graduate Medical Education.^35,36

Looking ahead, the VA is developing an online tool to improve the accuracy of affiliate reimbursement. The VA will also implement a standardized staffing model, organizational structure, and position descriptions for DEO offices. These initiatives will help reduce the burden of tracking and verifying resident activity and continue to support the 77-year partnership between VA and its affiliated institutions.

The US Department of Veterans Affairs (VA) has partnered with academic medical centers and programs since 1946 to provide clinical training for physician residents. Ranking second in federal graduate medical education (GME) funding to the Centers for Medicare and Medicaid Services (CMS), the $850 million VA GME budget annually reimburses > 250 GME-sponsoring institutions (affiliates) of 8000 GME programs for the clinical training of 49,000 individual residents rotating through > 11,000 full-time equivalent (FTE) positions.¹ The VA also distributes $1.6 billion to VA facilities to offset the costs of conducting health professions education (HPE) (eg, facility infrastructure, salary support for VA instructors and preceptors, education office administration, and instructional equipment).² The VA financial and educational contributions account for payment of 11% of resident positions nationally and allow academic medical centers to be less reliant on CMS GME funding.^3,4 The VA contributions also provide opportunities for GME expansion,^1,5,6 educational innovations,^5,7 interprofessional and team-based care,^8,9 and quality and safety training.^10,11 The Table provides a comparison of CMS and VA GME reimbursability based on activity.

GME financing is complex, particularly the formulaic approach used by CMS, the details of which are often obscured in federal regulations. Due to this complexity and the $16 billion CMS GME budget, academic publications have focused on CMS GME financing while not fully explaining the VA GME policies and processes.^4,12-14 By comparison, the VA GME financing model is relatively straightforward and governed by different statues and VA regulations, yet sharing some of the same principles as CMS regulations. Given the challenges in CMS reimbursement to fully support the cost of resident education, as well as the educational opportunities at the VA, the VA designs its reimbursement model to assure that affiliates receive appropriate payments.^4,12,15 To ensure the continued success of VA GME partnerships, knowledge of VA GME financing has become increasingly important for designated institutional officers (DIOs) and residency program directors, particularly in light of recent investigations into oversight of the VA’s reimbursement to academic affiliates.^16-18 This report describes VA GME reimbursement and, where applicable, VA and CMS reimbursement policies are compared to highlight similarities, differences, and common principles.

VA AUTHORITY

While the VA’s primary mission is “to provide a complete hospital medical service for the medical care and treatment of veterans,”early VA leaders recognized the importance of affiliating with the nation’s academic institutions.¹⁹ In 1946, the VA Policy Memorandum Number 2 established a partnership between the VA and the academic medical community.²⁰ Additional legislation authorized specific agreements with academic affiliates for the central administration of salary and benefits for residents rotating at VA facilities. This process, known as disbursement, is an alternative payroll mechanism whereby the VA reimburses the academic affiliate for resident salary and benefits and the affiliate acts as the disbursing agent, issuing paychecks to residents.^21,22

Resident FUNDING

By policy, with rare exceptions, the VA does not sponsor residency programs due to the challenges of providing an appropriate patient mix of age, sex, and medical conditions to meet accreditation standards.⁴ Nearly all VA reimbursements are for residents in affiliate-sponsored programs, while just 1% pays for residents in legacy, VA-sponsored residency programs at 2 VA facilities. The VA budget for resident (including fellows) salary and benefits is managed by the VA Office of Academic Affiliations (OAA), the national VA office responsible for oversight, policy, and funding of VA HPE programs.

Resident Salaries and Benefits

VA funding of resident salary and benefits are analogous with CMS direct GME (DGME), which is designed to cover resident salary and benefits costs.^4,14,23 CMS DGME payments depend on a hospital’s volume of CMS inpatients and are based on a statutory formula, which uses the hospital’s resident FTE positions, the per-resident amount, and Medicare’s share of inpatient beds (Medicare patient load) to determine payments.¹² The per-resident amount is set by statute, varies geographically, and is calculated by dividing the hospital’s allowable costs of GME (percentage of CMS inpatient days) divided by the number of residents.^12,24

By comparison, the VA GME payment reimburses for each FTE based on the salary and benefits rate set by the academic affiliate. Reimbursement is calculated based on resident time spent at the VA multiplied by a daily salary rate. The daily salary rate is determined by dividing the resident’s total compensation (salary and benefits) by the number of calendar days in an academic year. Resident time spent at the VA facility is determined by obtaining rotation schedules provided by the academic affiliate and verifying resident clinical and educational activity during scheduled rotations.

Indirect Medical Education Funding

In addition to resident salary and benefits, funds to offset the cost of conducting HPE are provided to VA facilities. These funds are intended to improve and maintain necessary infrastructure for all HPE programs not just GME, including education office administration needs, teaching costs (ie, a portion of VA preceptors salary), and instructional equipment.

The Veterans Equitable Resource Allocation (VERA) is a national budgeting process for VA medical facilities that funds facility operational needs such as staff salary and benefits, infrastructure, and equipment.² The education portion of the VERA, the VERA Education Support Component (VESC), is not managed by the OAA, but rather is distributed through the VERA model to the general budget of VA facilities hosting HPE (Figure). VESC funding in the VA budget is based on labor mapping of physician time spent in education; other labor mapping categories include clinical care, research, and administration. VA facility VESC funding is calculated based on the number of paid health profession trainees (HPTs) from all professions, apportioned according to the number of FTEs for physician residents and VA-paid HPTs in other disciplines. In fiscal year 2024, VA facilities received $115,812 for each physician resident FTE position and $84,906 for each VA-paid, non-GME FTE position.

The VESC is like CMS's indirect GME funding, termed Indirect Medical Education (IME), an additional payment for each Medicare patient discharged reflecting teaching hospitals’ higher patient care costs relative to nonteaching hospitals. Described elsewhere, IME is calculated using a resident-to-bed ratio and a multiplier, which is set by statute.^4,25 While IME can be used for reimbursement for some resident clinical and educational activities(eg, research), VA VESC funds cannot be used for such activities and are part of the general facility budget and appropriated per the discretion of the medical facility director.

ESTABLISHING GME PARTNERSHIPS

An affiliation agreement establishes the administrative and legal requirements for educational relationships with academic affiliates and includes standards for conducting HPE, responsibilities for accreditation standards, program leadership, faculty, resources, supervision, academic policies, and procedures. The VA uses standardized affiliation agreement templates that have been vetted with accrediting bodies and the VA Office of General Counsel.

A disbursement agreement authorizes the VA to reimburse affiliates for resident salary and benefits for VA clinical and educational activities. The disbursement agreement details the fiscal arrangements (eg, payment in advance vs arrears, salary, and benefit rates, leave) for the reimbursement payments. Veterans Health Administration (VHA) Directive 1400.05 provides the policy and procedures for calculating reimbursement for HPT educational activities.²⁶

The VA facility designated education officer (DEO) oversees all HPE programs and coordinates the affiliation and disbursement agreement processes.²⁷ The DEO, affiliate DIO, residency program director, and VA residency site director determine the physician resident FTE positions assigned to a VA facility based on educational objectives and availability of educational resources at the VA facility, such as patient care opportunities, faculty supervisors, space, and equipment. The VA facility requests for resident FTE positions are submitted to the OAA by the facility DEO.

Once GME FTE positions are approved by the OAA, VA facilities work with their academic affiliate to submit the physician resident salary and benefit rate. Affiliate DIOs attest to the accuracy of the salary rate schedule and the local DEO submits the budget request to the OAA. Upon approval, the funds are transferred to the VA facility each fiscal year, which begins October 1. DEOs report quarterly to the OAA both budget needs and excesses based on variations in the approved FTEs due to additional VA rotations, physician resident attrition, or reassignment.

Resident Position Allocation

VA GME financing provides flexibility through periodic needs assessments and expansion initiatives. In August and December, DEOs collaborate with an academic affiliate to submit reports to the OAA confirming their projected GME needs for the next academic year. Additional positions requests are reviewed by the OAA; funding depends on budget and the educational justification. The OAA periodically issues GME expansion requests for proposal, which typically arise from legislation to address specific VA workforce needs. The VA facility DEO and affiliate GME leaders collaborate to apply for additional positions. For example, a VA GME expansion under the Veterans Access, Choice, and Accountability Act of 2014 added 1500 GME positions in 8 years for critically needed specialties and in rural and underserved areas.⁵ The Maintaining Internal Systems and Strengthening Outside Networks (MISSION) Act of 2018 authorized a pilot program for VA to fund residents at non-VA facilities with priority for Indian Health Services, Tribes and Tribal Organizations, Federally Qualified Health Centers, and US Department of Defense facilities to provide access to veterans in underserved areas.⁶

The VA GME financing system has flexibility to meet local needs for additional resident positions and to address broader VA workforce gaps through targeted expansion. Generally, CMS does not fund positions to address workforce needs, place residents in specific geographic areas, or require the training of certain types of residents.⁴ However, the Consolidated Appropriations Act of 2021 has provided the opportunity to address rural workforce needs.²⁸

Reimbursement

The VA provides reimbursement for clinical and educational activities performed in VA facilities for the benefit of veterans as well as research, didactics, meetings and conferences, annual and sick leave, and orientation. The VA also may provide reimbursement for educational activities that occur off VA grounds (eg, the VA proportional share of a residency program’s didactic sessions). The VA does not reimburse for affiliate clinical duties or administrative costs, although a national policy allows VA facilities to reimburse affiliates for some GME overhead costs.²⁹

CMS similarly reimburses for residency training time spent in patient care activities as well as orientation activities, didactics, leave, and, in some cases, research.^4,30,31 CMS makes payments to hospitals, which may include sponsoring institutions and Medicare-eligible participating training sites.^4,30,31 For both the VA and CMS, residents may not be counted twice for reimbursement by 2 federal agencies; in other words, a resident may not count for > 1 FTE.^4,30-32

GME Oversight

VA GME funding came under significant scrutiny. At a 2016 House Veterans Affairs Committee hearing, Representative Phil Roe, MD (R-Tennessee), noted that no process existed at many VA facilities for “determining trainee presence” and that many VA medical centers had “difficulty tracking resident rotations”¹⁶ A VA Office of the Inspector General investigation recommended that the VA implement policies and procedures to improve oversight to “ensure residents are fully participating in educational activities” and that the VA is “paying the correct amount” to the affiliate.¹⁷ A 2020 General Accountability Office report outlined unclear policy guidance, incomplete tracking of resident activities, and improper fiscal processes for reimbursement and reconciliation of affiliate invoices.¹⁸

In response, the OAA created an oversight and compliance unit, revised VHA Directive 1400.05 (the policy for disbursement), and improved resident tracking procedures.²⁶ The standard operating procedure that accompanied VHA Directive 1400.05 provides detailed information for the DEO and VA facility staff for tracking resident clinical and educational activities. FTE counts are essential to both VA and CMS for accurate reimbursement. The eAppendix and the Table provide a guide to reimbursable activities in the VA for the calculation of reimbursement, with a comparison to CMS.^33,34 The OAA in cooperation with other VA staff and officers periodically conducts audits to assess compliance with disbursement policy and affiliate reimbursement accuracy.

In the VA, resident activities are captured on the VA Educational Activity Record, a standardized spreadsheet to track activities and calculate reimbursement. Each VA facility hosting resident physicians manually records resident activity by the half-day. This process is labor intensive, involving both VA and affiliate staff to accurately reconcile payments. To address the workload demands, the OAA is developing an online tool that will automate aspects of the tracking process. Also, to ensure adequate staffing, the OAA is in the process of implementing an office optimization project, providing standardized position descriptions, an organizational chart, and staffing levels for DEO offices in VA facilities.

This report describes the key policies and principles of VA GME financing, highlighting the essential similarities and differences between VA and CMS. Neither the VA nor CMS regulations allow for reimbursement for > 1 FTE position per resident, a principle that underpins the assignment of resident rotations and federal funding for GME and are similar with respect to reimbursement for patient care activities, didactics, research, orientation, and scholarly activity. While reimbursable activities in the VA require physical presence and care of veteran patients, CMS also limits reimbursement to resident activities in the hospital and approved other settings if the hospital is paying for resident salary and benefits in these settings. The VA provides some flexibility for offsite activities including didactics and, in specific circumstances, remote care of veteran patients (eg, teleradiology).

The VA and CMS use different GME financing models. For example, the CMS calculations for resident FTEs are complex, whereas VA calculations reimburse the salary and benefits as set by the academic affiliate. The VA process accounts for local variation in salary rates, whereas the per-resident amount set by CMS varies regionally and does not fully account for differences in the cost of living.²⁴ Because all patients in VA facilities are veterans, VA calculations for reimbursement do not involve ratios of beds like the CMS calculations to determine a proportional share of reimbursement. The VA GME expansion tends to be more directed to VA health workforce needs than CMS, specifying the types of programs and geographic locations to address these needs.

The VA regularly reevaluates how affiliates are reimbursed for VA resident activity, balancing compliance with VA policies and the workload for VA and its affiliates. The VA obtains input from key stakeholders including DEOs, DIOs, and professional organizations such as the Association of American Medical Colleges and the Accreditation Council for Graduate Medical Education.^35,36

Looking ahead, the VA is developing an online tool to improve the accuracy of affiliate reimbursement. The VA will also implement a standardized staffing model, organizational structure, and position descriptions for DEO offices. These initiatives will help reduce the burden of tracking and verifying resident activity and continue to support the 77-year partnership between VA and its affiliated institutions.

The Veterans Health Administration (VHA) is understaffed for clinical psychologists who have specialty training in geriatrics (ie, geropsychologists) to meet the needs of aging veterans. Though only 16.8% of US adults are aged ≥ 65 years,¹ this age group comprises 45.9% of patients within the VHA.² The needs of older adults are complex and warrant specialized services from mental health clinicians trained to understand lifespan developmental processes, biological changes associated with aging, and changes in psychosocial functioning.

Older veterans (aged ≥ 65 years) present with higher rates of combined medical and mental health diagnoses compared to both younger veterans and older adults who are not veterans.³ Nearly 1 of 5 (18.1%) older veterans who use VHA services have confirmed mental health diagnoses, and an additional 25.5% have documented mental health concerns without a formal diagnosis in their health record.⁴ The clinical presentations of older veterans frequently differ from younger adults and include greater complexity. For example, older veterans face an increased risk of cognitive impairment compared to the general population, due in part to higher prevalence of posttraumatic stress, which doubles their risk of developing dementia.⁵ Additional examples of multicomplexity among older veterans may include co-occurring medical and psychiatric diagnoses, the presence of delirium, social isolation/loneliness, and concerns related to polypharmacy. These complex presentations result in significant challenges for mental health clinicians in areas like assessment (eg, accuracy of case conceptualization), intervention (eg, selection and prioritization), and consultation (eg, coordination among multiple medical and mental health specialists).

Older veterans also present with substantial resilience. Research has found that aging veterans exposed to trauma during their military service often review their memories and past experiences, which is known as later-adulthood trauma reengagement.⁶ Through this normative life review process, veterans engage with memories and experiences from their past that they previously avoided, which could lead to posttraumatic growth for some. Unfortunately, others may experience an increase in psychological distress. Mental health clinicians with specialty expertise and training in aging and lifespan development can facilitate positive outcomes to reduce distress.⁷

The United States in general, and the VHA specifically, face a growing shortage of geriatric mental health clinicians. In a 2015 American Psychological Association survey, 1528 of 4109 respondents (37.2%) reported they frequently or very frequently administered care to older adults, yet only 49 respondents (1.2%) reported geropsychology as their specialty.⁸ According to the National Provider Registry, 660 clinicians self-identified as geropsychologists (ie, those who self-reported “psychologist: adult development and aging” as an NPI Healthcare Provider taxonomy code) in the US, representing < 1% of all doctoral level psychologists.⁹ The number of psychologists who obtain board certification from the American Board of Geropsychology is even lower (only 112 clinicians as of February 2024).¹⁰ Many psychologists within the VHA treat older veterans in integrated health settings such as primary care, home-based primary care, community living centers, or behavioral health care, but many lack formal training in geropsychology.

The Geriatric Scholars Program (GSP) was developed in 2008 to address the training gap and provide education in geriatrics to VHA clinicians that treat older veterans, particularly in rural areas.^11,12 The GSP initially focused on primary care physicians, nurse practitioners, physician assistants, and pharmacists. It was later expanded to include other disciplines (ie, social work, rehabilitation therapists, and psychiatrists). In 2013, the GSP – Psychology Track (GSP-P) was developed with funding from the VHA Offices of Rural Health and Geriatrics and Extended Care specifically for psychologists.

This article describes the multicomponent longitudinal GSP-P, which has evolved to meet the target audience’s ongoing needs for knowledge, skills, and opportunities to refine practice behaviors. GSP-P received the 2020 Award for Excellence in Geropsychology Training from the Council of Professional Geropsychology Training Programs. GSP-P has grown within the context of the larger GSP and aligns with the other existing elective learning opportunities (Figure 1).

Program description

Introductory Course

Psychologist subject matter experts (SMEs) developed an intensive course in geropsychology in the absence of a similar course in the geriatric medicine board review curriculum. SMEs reviewed the guidelines for practice by professional organizations like the Pikes Peak Geropsychology Competencies,which outline knowledge and skills in various domains.¹³ SMEs integrated this review with findings from a needs assessment for postlicensed VHA psychology staff in 4 health care systems, drafted a syllabus, and circulated it to geropsychology experts for feedback. The resulting multiday course covered general mental health as well as topics particularly salient for mental health clinicians treating older veterans including suicide prevention and posttraumatic stress disorder (PTSD).¹⁴ This Geropsychology Competencies Review Course was piloted in 1 region initially before being offered nationally in 2014. The previously published evaluation findings demonstrated significant improvements from precourse to 3 months postcourse in confidence and knowledge in 4 key areas of geropsychology: General knowledge about adult development and aging, assessment, intervention, and consultation.¹⁵ These domains align with the Pikes Peak Geropsychology Competencies and are measured by a subset of items from the Geropsychology Knowledge and Skills Assessment Tool.^13,16 As of October 2023, 8 introductory courses have been held with 173 participants (Table).

Quality Improvement

Introductory course attendees also participate in an intensive day-long interactive workshop in quality improvement (QI). After completing these trainings, they apply what they have learned at their home facility by embarking on a QI project related to geriatrics. The QI projects reinforce learning and initiate practice changes not only for attendees but at times the larger health care system. Topics are selected by scholars in response to the needs they observe in their clinics. Recent GSP projects include efforts to increase screenings for depression and anxiety, improve adherence to VHA dementia policy, increase access to virtual care, and increase referrals to programs such as whole health or cognitive behavioral therapy for insomnia, a first-line treatment for insomnia in older adults.¹⁷ Another project targeted the improvement of referrals to the Compassionate Contact Corps in an effort to reduce social isolation and loneliness among older veterans.¹⁸ Evaluations demonstrate significant improvement in scholars’ confidence in related program development and management from precourse to 3 months postcourse.¹⁵

Webinars

The Addressing Geriatric Mental Health webinar series was created to introduce learners to topics that could not be covered in the introductory course. Topics were suggested by the expert reviewers of the curriculum or identified by the scholars themselves (eg, chronic pain, sexuality, or serious mental illness). A secondary function of the webinars was to reach a broader audience. Over time, scholars and webinar attendees requested opportunities to explore topics in greater depth (eg, PTSD later in life). These requests led the webinars to focus on annual themes.

The series is open to all disciplines of geriatric scholars, VHA staff, and non-VHA staff through the Veterans Affairs Talent Management System and the TRAIN Learning Network (train.org). Attendance for the 37 webinars was captured from logins to the virtual learning platform and may underestimate attendance if a group attended on a single screen. Average attendance increased from 157 attendees/webinar in 2015 to 418 attendees/webinar in 2023 (Figure 2). This may have been related to the increase in virtual learning during the COVID-19 pandemic, but represents a 166% increase in audience from the inaugural year of the series.

Advanced Learning Opportunities

To invest in the ongoing growth and development of introductory course graduates, GSP-P developed and offered an advanced workshop in 2019. This multiday workshop focused on further enhancement of geropsychology competencies, with an emphasis on treating older veterans with mental and physical comorbidities. Didactics and experiential learning exercises led by SMEs covered topics such as adjusting to chronic illness, capacity assessment, PTSD, insomnia and sleep changes, chronic pain, and psychological interventions in palliative care and hospice settings. Evaluation findings demonstrated significant improvements from precourse to 6 months postcourse in confidence and knowledge as defined by the Pikes Peak Geropsychology Competencies.¹⁹

To facilitate ongoing and individually tailored learning following the advanced workshop, scholars also developed and executed independent learning plans (ILPs) during a 6-month window with consultation from an experienced geropsychologist. Fifteen of 19 scholars (78.9%) completed ILPs with an average of 3 learning goals listed. After completing the ILPs, scholars endorsed their clinical and/or personal usefulness, citing increased confidence, enhanced skills for use with patients with complex needs, personal fulfillment, and career advancement. Most scholars noted ILPs were feasible and learning resources were accessible. Overall, the evaluation found ILPs to be a valuable way to enhance psychologists’ learning and effectiveness in treating older veterans with complex health needs.²⁰

All geriatric scholars who completed the program have additional opportunities for professional development through practicum experiences focused on specific clinical approaches to the care of older veterans, such as dementia care, pain management, geriatric assessment, and palliative care. These practica provide scholars with individualized learning experiences in an individualized or small group setting and may be conducted either in-person or virtually.

In response to an expressed need from those who completed the program, the GSP-P planning committee collaborated with an SME to develop a virtual practicum to assess patients’ decision making capacity. Evaluating capacities among older adults is a common request, yet clinicians report little to no formal training in how to conceptualize and approach these complex questions.^21,22 Utilizing an evidence-informed and structured approach promotes the balancing of an older adult’s autonomy and professional ethics. Learning capacity evaluation skills could better position psychologists to not only navigate complex ethical, legal, and clinical situations, but also serve as expert consultants to interdisciplinary teams. This virtual practicum was initiated in 2022 and to date has included 10 scholars. The practicum includes multiple modalities of learning: (1) self-directed review of core concepts; (2) attendance at 4 capacity didactics focused on introduction to evaluating capacities, medical consent and decision making, financial decision making and management, and independent living; and (3) participation in 5 group consultations on capacity evaluations conducted at their home sites. During these group consultations, additional case examples were shared to reinforce capacity concepts.

Discussion

The objective of GSP-P is to enhance geropsychology practice competencies among VHA psychologists given the outsized representation of older adults within the VHA system and their complex care needs. The curricula have significantly evolved to accomplish this, expanding the reach and investing in the continuing growth and development of scholars.

There are several elements that set GSP apart from other geriatric and geropsychology continuing medical education programs. The first is that the training is veteran focused, allowing us to discuss the unique impact military service has on aging. Similarly, because all scholars work within the integrated health care system, we can introduce and review key resources and programs that benefit all veterans and their families/care partners across the system. Through the GSP, the VA invests in ongoing professional development. Scholars can participate in additional experiential practica, webinars, and advanced workshops tailored to their individual learning needs. Lastly, the GSP works to create a community among its scholars where they can not only continue to consult with presenters/instructors, but also one another. A planned future direction for the GSP-P is to incorporate quarterly office hours and discussions for alumni to develop an increased sense of community. This may strengthen commitment to the overall VA mission, leading to increased retainment of talent who now have the knowledge, skills, and confidence to care for aging veterans.

Limitations

GSP is limited by its available funding. Additionally, the number of participants who can enroll each year in GSP-P (not including webinars) is capped by policy. Another limitation is the number of QI coaches available to mentor scholars on their projects.

Outcomes of GSP-P have been extremely favorable. Following participation in the program, we have found a significant increase in confidence in geropsychology practice among clinicians, as well as enhanced knowledge and skills across competency domains.^15,19 We have observed rising attendance in our annual webinar series and graduates of our introductory courses participate in subsequent trainings (eg, advanced workshop or virtual practicum). Several graduates of GSP-P have become board certified in geropsychology by the American Board of Geropsychology and many proceed to supervise geropsychology-focused clinical rotations for psychology practicum students, predoctoral interns, and postdoctoral fellows. This suggests that the reach of GSP-P programming may extend farther than reported in this article.

The needs of aging veterans have also changed based on cohort differences, as the population of World War II and Korean War era veterans has declined and the number of older Vietnam era veterans has grown. We expect different challenges with older Gulf War and post-9/11 era veterans. For instance, 17% of troops deployed to Iraq or Afghanistan following 9/11 experienced mild traumatic brain injury (TBI), and 59% of those experienced > 1 mild TBI.²³ Research indicates that younger post-9/11 veterans have a 3-fold risk of developing early onset dementia after experiencing a TBI.²⁴ Therefore, even though post-9/11 veterans are not older in terms of chronological age, some may experience symptoms and conditions more often occurring in older veterans. As a result, it would be beneficial for clinicians to learn about the presentation and treatment of geriatric conditions such as dementia.

Moving forward, the GSP-P should identify potential opportunities to collaborate with the non-VHA mental health community–which also faces a shortage of geriatric mental health clinicians–to extend educational opportunities to improve care for veterans in all settings (eg, cosponsor training opportunities open to both VHA and non-VHA clinicians).^8,25 Many aging veterans may receive portions of their health care outside the VHA, particularly those who reside in rural areas. Additionally, as veterans age, so do their support systems (eg, family members, friends, spouses, caregivers, and even clinicians), most of whom will receive care outside of the VHA. Community education collaborations will not only improve the care of older veterans, but also the care of older adults in the general population.

Promising directions include the adoption of the GSP model in other health care settings. Recently, Indian Health Service has adapted the model, beginning with primary care clinicians and pharmacists and is beginning to expand to other disciplines. Additional investments in VHA workforce training include the availability of geropsychology internship and fellowship training opportunities through the Office of Academic Affiliations, which provide earlier opportunities to specialize in geropsychology. Continued investment in both prelicensure and postpsychology licensure training efforts are needed within the VHA to meet the geriatric mental health needs of veterans.

Acknowledgments

The authors wish to acknowledge Terri Huh, PhD, for her contributions to the development and initiation of the GSP-P. The authors also appreciate the collaboration and quality initiative training led by Carol Callaway-Lane, DNP, ACNP-BC, and her team.

The Veterans Health Administration (VHA) is understaffed for clinical psychologists who have specialty training in geriatrics (ie, geropsychologists) to meet the needs of aging veterans. Though only 16.8% of US adults are aged ≥ 65 years,¹ this age group comprises 45.9% of patients within the VHA.² The needs of older adults are complex and warrant specialized services from mental health clinicians trained to understand lifespan developmental processes, biological changes associated with aging, and changes in psychosocial functioning.

Older veterans (aged ≥ 65 years) present with higher rates of combined medical and mental health diagnoses compared to both younger veterans and older adults who are not veterans.³ Nearly 1 of 5 (18.1%) older veterans who use VHA services have confirmed mental health diagnoses, and an additional 25.5% have documented mental health concerns without a formal diagnosis in their health record.⁴ The clinical presentations of older veterans frequently differ from younger adults and include greater complexity. For example, older veterans face an increased risk of cognitive impairment compared to the general population, due in part to higher prevalence of posttraumatic stress, which doubles their risk of developing dementia.⁵ Additional examples of multicomplexity among older veterans may include co-occurring medical and psychiatric diagnoses, the presence of delirium, social isolation/loneliness, and concerns related to polypharmacy. These complex presentations result in significant challenges for mental health clinicians in areas like assessment (eg, accuracy of case conceptualization), intervention (eg, selection and prioritization), and consultation (eg, coordination among multiple medical and mental health specialists).

Older veterans also present with substantial resilience. Research has found that aging veterans exposed to trauma during their military service often review their memories and past experiences, which is known as later-adulthood trauma reengagement.⁶ Through this normative life review process, veterans engage with memories and experiences from their past that they previously avoided, which could lead to posttraumatic growth for some. Unfortunately, others may experience an increase in psychological distress. Mental health clinicians with specialty expertise and training in aging and lifespan development can facilitate positive outcomes to reduce distress.⁷

The United States in general, and the VHA specifically, face a growing shortage of geriatric mental health clinicians. In a 2015 American Psychological Association survey, 1528 of 4109 respondents (37.2%) reported they frequently or very frequently administered care to older adults, yet only 49 respondents (1.2%) reported geropsychology as their specialty.⁸ According to the National Provider Registry, 660 clinicians self-identified as geropsychologists (ie, those who self-reported “psychologist: adult development and aging” as an NPI Healthcare Provider taxonomy code) in the US, representing < 1% of all doctoral level psychologists.⁹ The number of psychologists who obtain board certification from the American Board of Geropsychology is even lower (only 112 clinicians as of February 2024).¹⁰ Many psychologists within the VHA treat older veterans in integrated health settings such as primary care, home-based primary care, community living centers, or behavioral health care, but many lack formal training in geropsychology.

The Geriatric Scholars Program (GSP) was developed in 2008 to address the training gap and provide education in geriatrics to VHA clinicians that treat older veterans, particularly in rural areas.^11,12 The GSP initially focused on primary care physicians, nurse practitioners, physician assistants, and pharmacists. It was later expanded to include other disciplines (ie, social work, rehabilitation therapists, and psychiatrists). In 2013, the GSP – Psychology Track (GSP-P) was developed with funding from the VHA Offices of Rural Health and Geriatrics and Extended Care specifically for psychologists.

This article describes the multicomponent longitudinal GSP-P, which has evolved to meet the target audience’s ongoing needs for knowledge, skills, and opportunities to refine practice behaviors. GSP-P received the 2020 Award for Excellence in Geropsychology Training from the Council of Professional Geropsychology Training Programs. GSP-P has grown within the context of the larger GSP and aligns with the other existing elective learning opportunities (Figure 1).

Program description

Introductory Course

Psychologist subject matter experts (SMEs) developed an intensive course in geropsychology in the absence of a similar course in the geriatric medicine board review curriculum. SMEs reviewed the guidelines for practice by professional organizations like the Pikes Peak Geropsychology Competencies,which outline knowledge and skills in various domains.¹³ SMEs integrated this review with findings from a needs assessment for postlicensed VHA psychology staff in 4 health care systems, drafted a syllabus, and circulated it to geropsychology experts for feedback. The resulting multiday course covered general mental health as well as topics particularly salient for mental health clinicians treating older veterans including suicide prevention and posttraumatic stress disorder (PTSD).¹⁴ This Geropsychology Competencies Review Course was piloted in 1 region initially before being offered nationally in 2014. The previously published evaluation findings demonstrated significant improvements from precourse to 3 months postcourse in confidence and knowledge in 4 key areas of geropsychology: General knowledge about adult development and aging, assessment, intervention, and consultation.¹⁵ These domains align with the Pikes Peak Geropsychology Competencies and are measured by a subset of items from the Geropsychology Knowledge and Skills Assessment Tool.^13,16 As of October 2023, 8 introductory courses have been held with 173 participants (Table).

Quality Improvement

Introductory course attendees also participate in an intensive day-long interactive workshop in quality improvement (QI). After completing these trainings, they apply what they have learned at their home facility by embarking on a QI project related to geriatrics. The QI projects reinforce learning and initiate practice changes not only for attendees but at times the larger health care system. Topics are selected by scholars in response to the needs they observe in their clinics. Recent GSP projects include efforts to increase screenings for depression and anxiety, improve adherence to VHA dementia policy, increase access to virtual care, and increase referrals to programs such as whole health or cognitive behavioral therapy for insomnia, a first-line treatment for insomnia in older adults.¹⁷ Another project targeted the improvement of referrals to the Compassionate Contact Corps in an effort to reduce social isolation and loneliness among older veterans.¹⁸ Evaluations demonstrate significant improvement in scholars’ confidence in related program development and management from precourse to 3 months postcourse.¹⁵

Webinars

The Addressing Geriatric Mental Health webinar series was created to introduce learners to topics that could not be covered in the introductory course. Topics were suggested by the expert reviewers of the curriculum or identified by the scholars themselves (eg, chronic pain, sexuality, or serious mental illness). A secondary function of the webinars was to reach a broader audience. Over time, scholars and webinar attendees requested opportunities to explore topics in greater depth (eg, PTSD later in life). These requests led the webinars to focus on annual themes.

The series is open to all disciplines of geriatric scholars, VHA staff, and non-VHA staff through the Veterans Affairs Talent Management System and the TRAIN Learning Network (train.org). Attendance for the 37 webinars was captured from logins to the virtual learning platform and may underestimate attendance if a group attended on a single screen. Average attendance increased from 157 attendees/webinar in 2015 to 418 attendees/webinar in 2023 (Figure 2). This may have been related to the increase in virtual learning during the COVID-19 pandemic, but represents a 166% increase in audience from the inaugural year of the series.

Advanced Learning Opportunities

To invest in the ongoing growth and development of introductory course graduates, GSP-P developed and offered an advanced workshop in 2019. This multiday workshop focused on further enhancement of geropsychology competencies, with an emphasis on treating older veterans with mental and physical comorbidities. Didactics and experiential learning exercises led by SMEs covered topics such as adjusting to chronic illness, capacity assessment, PTSD, insomnia and sleep changes, chronic pain, and psychological interventions in palliative care and hospice settings. Evaluation findings demonstrated significant improvements from precourse to 6 months postcourse in confidence and knowledge as defined by the Pikes Peak Geropsychology Competencies.¹⁹

To facilitate ongoing and individually tailored learning following the advanced workshop, scholars also developed and executed independent learning plans (ILPs) during a 6-month window with consultation from an experienced geropsychologist. Fifteen of 19 scholars (78.9%) completed ILPs with an average of 3 learning goals listed. After completing the ILPs, scholars endorsed their clinical and/or personal usefulness, citing increased confidence, enhanced skills for use with patients with complex needs, personal fulfillment, and career advancement. Most scholars noted ILPs were feasible and learning resources were accessible. Overall, the evaluation found ILPs to be a valuable way to enhance psychologists’ learning and effectiveness in treating older veterans with complex health needs.²⁰

All geriatric scholars who completed the program have additional opportunities for professional development through practicum experiences focused on specific clinical approaches to the care of older veterans, such as dementia care, pain management, geriatric assessment, and palliative care. These practica provide scholars with individualized learning experiences in an individualized or small group setting and may be conducted either in-person or virtually.

In response to an expressed need from those who completed the program, the GSP-P planning committee collaborated with an SME to develop a virtual practicum to assess patients’ decision making capacity. Evaluating capacities among older adults is a common request, yet clinicians report little to no formal training in how to conceptualize and approach these complex questions.^21,22 Utilizing an evidence-informed and structured approach promotes the balancing of an older adult’s autonomy and professional ethics. Learning capacity evaluation skills could better position psychologists to not only navigate complex ethical, legal, and clinical situations, but also serve as expert consultants to interdisciplinary teams. This virtual practicum was initiated in 2022 and to date has included 10 scholars. The practicum includes multiple modalities of learning: (1) self-directed review of core concepts; (2) attendance at 4 capacity didactics focused on introduction to evaluating capacities, medical consent and decision making, financial decision making and management, and independent living; and (3) participation in 5 group consultations on capacity evaluations conducted at their home sites. During these group consultations, additional case examples were shared to reinforce capacity concepts.

Discussion

The objective of GSP-P is to enhance geropsychology practice competencies among VHA psychologists given the outsized representation of older adults within the VHA system and their complex care needs. The curricula have significantly evolved to accomplish this, expanding the reach and investing in the continuing growth and development of scholars.

There are several elements that set GSP apart from other geriatric and geropsychology continuing medical education programs. The first is that the training is veteran focused, allowing us to discuss the unique impact military service has on aging. Similarly, because all scholars work within the integrated health care system, we can introduce and review key resources and programs that benefit all veterans and their families/care partners across the system. Through the GSP, the VA invests in ongoing professional development. Scholars can participate in additional experiential practica, webinars, and advanced workshops tailored to their individual learning needs. Lastly, the GSP works to create a community among its scholars where they can not only continue to consult with presenters/instructors, but also one another. A planned future direction for the GSP-P is to incorporate quarterly office hours and discussions for alumni to develop an increased sense of community. This may strengthen commitment to the overall VA mission, leading to increased retainment of talent who now have the knowledge, skills, and confidence to care for aging veterans.

Limitations

GSP is limited by its available funding. Additionally, the number of participants who can enroll each year in GSP-P (not including webinars) is capped by policy. Another limitation is the number of QI coaches available to mentor scholars on their projects.

Outcomes of GSP-P have been extremely favorable. Following participation in the program, we have found a significant increase in confidence in geropsychology practice among clinicians, as well as enhanced knowledge and skills across competency domains.^15,19 We have observed rising attendance in our annual webinar series and graduates of our introductory courses participate in subsequent trainings (eg, advanced workshop or virtual practicum). Several graduates of GSP-P have become board certified in geropsychology by the American Board of Geropsychology and many proceed to supervise geropsychology-focused clinical rotations for psychology practicum students, predoctoral interns, and postdoctoral fellows. This suggests that the reach of GSP-P programming may extend farther than reported in this article.

The needs of aging veterans have also changed based on cohort differences, as the population of World War II and Korean War era veterans has declined and the number of older Vietnam era veterans has grown. We expect different challenges with older Gulf War and post-9/11 era veterans. For instance, 17% of troops deployed to Iraq or Afghanistan following 9/11 experienced mild traumatic brain injury (TBI), and 59% of those experienced > 1 mild TBI.²³ Research indicates that younger post-9/11 veterans have a 3-fold risk of developing early onset dementia after experiencing a TBI.²⁴ Therefore, even though post-9/11 veterans are not older in terms of chronological age, some may experience symptoms and conditions more often occurring in older veterans. As a result, it would be beneficial for clinicians to learn about the presentation and treatment of geriatric conditions such as dementia.

Moving forward, the GSP-P should identify potential opportunities to collaborate with the non-VHA mental health community–which also faces a shortage of geriatric mental health clinicians–to extend educational opportunities to improve care for veterans in all settings (eg, cosponsor training opportunities open to both VHA and non-VHA clinicians).^8,25 Many aging veterans may receive portions of their health care outside the VHA, particularly those who reside in rural areas. Additionally, as veterans age, so do their support systems (eg, family members, friends, spouses, caregivers, and even clinicians), most of whom will receive care outside of the VHA. Community education collaborations will not only improve the care of older veterans, but also the care of older adults in the general population.

Promising directions include the adoption of the GSP model in other health care settings. Recently, Indian Health Service has adapted the model, beginning with primary care clinicians and pharmacists and is beginning to expand to other disciplines. Additional investments in VHA workforce training include the availability of geropsychology internship and fellowship training opportunities through the Office of Academic Affiliations, which provide earlier opportunities to specialize in geropsychology. Continued investment in both prelicensure and postpsychology licensure training efforts are needed within the VHA to meet the geriatric mental health needs of veterans.

Acknowledgments

The authors wish to acknowledge Terri Huh, PhD, for her contributions to the development and initiation of the GSP-P. The authors also appreciate the collaboration and quality initiative training led by Carol Callaway-Lane, DNP, ACNP-BC, and her team.

The Veterans Health Administration (VHA) is understaffed for clinical psychologists who have specialty training in geriatrics (ie, geropsychologists) to meet the needs of aging veterans. Though only 16.8% of US adults are aged ≥ 65 years,¹ this age group comprises 45.9% of patients within the VHA.² The needs of older adults are complex and warrant specialized services from mental health clinicians trained to understand lifespan developmental processes, biological changes associated with aging, and changes in psychosocial functioning.

Older veterans (aged ≥ 65 years) present with higher rates of combined medical and mental health diagnoses compared to both younger veterans and older adults who are not veterans.³ Nearly 1 of 5 (18.1%) older veterans who use VHA services have confirmed mental health diagnoses, and an additional 25.5% have documented mental health concerns without a formal diagnosis in their health record.⁴ The clinical presentations of older veterans frequently differ from younger adults and include greater complexity. For example, older veterans face an increased risk of cognitive impairment compared to the general population, due in part to higher prevalence of posttraumatic stress, which doubles their risk of developing dementia.⁵ Additional examples of multicomplexity among older veterans may include co-occurring medical and psychiatric diagnoses, the presence of delirium, social isolation/loneliness, and concerns related to polypharmacy. These complex presentations result in significant challenges for mental health clinicians in areas like assessment (eg, accuracy of case conceptualization), intervention (eg, selection and prioritization), and consultation (eg, coordination among multiple medical and mental health specialists).

Older veterans also present with substantial resilience. Research has found that aging veterans exposed to trauma during their military service often review their memories and past experiences, which is known as later-adulthood trauma reengagement.⁶ Through this normative life review process, veterans engage with memories and experiences from their past that they previously avoided, which could lead to posttraumatic growth for some. Unfortunately, others may experience an increase in psychological distress. Mental health clinicians with specialty expertise and training in aging and lifespan development can facilitate positive outcomes to reduce distress.⁷

The United States in general, and the VHA specifically, face a growing shortage of geriatric mental health clinicians. In a 2015 American Psychological Association survey, 1528 of 4109 respondents (37.2%) reported they frequently or very frequently administered care to older adults, yet only 49 respondents (1.2%) reported geropsychology as their specialty.⁸ According to the National Provider Registry, 660 clinicians self-identified as geropsychologists (ie, those who self-reported “psychologist: adult development and aging” as an NPI Healthcare Provider taxonomy code) in the US, representing < 1% of all doctoral level psychologists.⁹ The number of psychologists who obtain board certification from the American Board of Geropsychology is even lower (only 112 clinicians as of February 2024).¹⁰ Many psychologists within the VHA treat older veterans in integrated health settings such as primary care, home-based primary care, community living centers, or behavioral health care, but many lack formal training in geropsychology.

The Geriatric Scholars Program (GSP) was developed in 2008 to address the training gap and provide education in geriatrics to VHA clinicians that treat older veterans, particularly in rural areas.^11,12 The GSP initially focused on primary care physicians, nurse practitioners, physician assistants, and pharmacists. It was later expanded to include other disciplines (ie, social work, rehabilitation therapists, and psychiatrists). In 2013, the GSP – Psychology Track (GSP-P) was developed with funding from the VHA Offices of Rural Health and Geriatrics and Extended Care specifically for psychologists.

This article describes the multicomponent longitudinal GSP-P, which has evolved to meet the target audience’s ongoing needs for knowledge, skills, and opportunities to refine practice behaviors. GSP-P received the 2020 Award for Excellence in Geropsychology Training from the Council of Professional Geropsychology Training Programs. GSP-P has grown within the context of the larger GSP and aligns with the other existing elective learning opportunities (Figure 1).

Program description

Introductory Course

Psychologist subject matter experts (SMEs) developed an intensive course in geropsychology in the absence of a similar course in the geriatric medicine board review curriculum. SMEs reviewed the guidelines for practice by professional organizations like the Pikes Peak Geropsychology Competencies,which outline knowledge and skills in various domains.¹³ SMEs integrated this review with findings from a needs assessment for postlicensed VHA psychology staff in 4 health care systems, drafted a syllabus, and circulated it to geropsychology experts for feedback. The resulting multiday course covered general mental health as well as topics particularly salient for mental health clinicians treating older veterans including suicide prevention and posttraumatic stress disorder (PTSD).¹⁴ This Geropsychology Competencies Review Course was piloted in 1 region initially before being offered nationally in 2014. The previously published evaluation findings demonstrated significant improvements from precourse to 3 months postcourse in confidence and knowledge in 4 key areas of geropsychology: General knowledge about adult development and aging, assessment, intervention, and consultation.¹⁵ These domains align with the Pikes Peak Geropsychology Competencies and are measured by a subset of items from the Geropsychology Knowledge and Skills Assessment Tool.^13,16 As of October 2023, 8 introductory courses have been held with 173 participants (Table).

Quality Improvement

Introductory course attendees also participate in an intensive day-long interactive workshop in quality improvement (QI). After completing these trainings, they apply what they have learned at their home facility by embarking on a QI project related to geriatrics. The QI projects reinforce learning and initiate practice changes not only for attendees but at times the larger health care system. Topics are selected by scholars in response to the needs they observe in their clinics. Recent GSP projects include efforts to increase screenings for depression and anxiety, improve adherence to VHA dementia policy, increase access to virtual care, and increase referrals to programs such as whole health or cognitive behavioral therapy for insomnia, a first-line treatment for insomnia in older adults.¹⁷ Another project targeted the improvement of referrals to the Compassionate Contact Corps in an effort to reduce social isolation and loneliness among older veterans.¹⁸ Evaluations demonstrate significant improvement in scholars’ confidence in related program development and management from precourse to 3 months postcourse.¹⁵

Webinars

The Addressing Geriatric Mental Health webinar series was created to introduce learners to topics that could not be covered in the introductory course. Topics were suggested by the expert reviewers of the curriculum or identified by the scholars themselves (eg, chronic pain, sexuality, or serious mental illness). A secondary function of the webinars was to reach a broader audience. Over time, scholars and webinar attendees requested opportunities to explore topics in greater depth (eg, PTSD later in life). These requests led the webinars to focus on annual themes.

The series is open to all disciplines of geriatric scholars, VHA staff, and non-VHA staff through the Veterans Affairs Talent Management System and the TRAIN Learning Network (train.org). Attendance for the 37 webinars was captured from logins to the virtual learning platform and may underestimate attendance if a group attended on a single screen. Average attendance increased from 157 attendees/webinar in 2015 to 418 attendees/webinar in 2023 (Figure 2). This may have been related to the increase in virtual learning during the COVID-19 pandemic, but represents a 166% increase in audience from the inaugural year of the series.

Advanced Learning Opportunities

To invest in the ongoing growth and development of introductory course graduates, GSP-P developed and offered an advanced workshop in 2019. This multiday workshop focused on further enhancement of geropsychology competencies, with an emphasis on treating older veterans with mental and physical comorbidities. Didactics and experiential learning exercises led by SMEs covered topics such as adjusting to chronic illness, capacity assessment, PTSD, insomnia and sleep changes, chronic pain, and psychological interventions in palliative care and hospice settings. Evaluation findings demonstrated significant improvements from precourse to 6 months postcourse in confidence and knowledge as defined by the Pikes Peak Geropsychology Competencies.¹⁹

To facilitate ongoing and individually tailored learning following the advanced workshop, scholars also developed and executed independent learning plans (ILPs) during a 6-month window with consultation from an experienced geropsychologist. Fifteen of 19 scholars (78.9%) completed ILPs with an average of 3 learning goals listed. After completing the ILPs, scholars endorsed their clinical and/or personal usefulness, citing increased confidence, enhanced skills for use with patients with complex needs, personal fulfillment, and career advancement. Most scholars noted ILPs were feasible and learning resources were accessible. Overall, the evaluation found ILPs to be a valuable way to enhance psychologists’ learning and effectiveness in treating older veterans with complex health needs.²⁰

All geriatric scholars who completed the program have additional opportunities for professional development through practicum experiences focused on specific clinical approaches to the care of older veterans, such as dementia care, pain management, geriatric assessment, and palliative care. These practica provide scholars with individualized learning experiences in an individualized or small group setting and may be conducted either in-person or virtually.

In response to an expressed need from those who completed the program, the GSP-P planning committee collaborated with an SME to develop a virtual practicum to assess patients’ decision making capacity. Evaluating capacities among older adults is a common request, yet clinicians report little to no formal training in how to conceptualize and approach these complex questions.^21,22 Utilizing an evidence-informed and structured approach promotes the balancing of an older adult’s autonomy and professional ethics. Learning capacity evaluation skills could better position psychologists to not only navigate complex ethical, legal, and clinical situations, but also serve as expert consultants to interdisciplinary teams. This virtual practicum was initiated in 2022 and to date has included 10 scholars. The practicum includes multiple modalities of learning: (1) self-directed review of core concepts; (2) attendance at 4 capacity didactics focused on introduction to evaluating capacities, medical consent and decision making, financial decision making and management, and independent living; and (3) participation in 5 group consultations on capacity evaluations conducted at their home sites. During these group consultations, additional case examples were shared to reinforce capacity concepts.

Discussion

The objective of GSP-P is to enhance geropsychology practice competencies among VHA psychologists given the outsized representation of older adults within the VHA system and their complex care needs. The curricula have significantly evolved to accomplish this, expanding the reach and investing in the continuing growth and development of scholars.

There are several elements that set GSP apart from other geriatric and geropsychology continuing medical education programs. The first is that the training is veteran focused, allowing us to discuss the unique impact military service has on aging. Similarly, because all scholars work within the integrated health care system, we can introduce and review key resources and programs that benefit all veterans and their families/care partners across the system. Through the GSP, the VA invests in ongoing professional development. Scholars can participate in additional experiential practica, webinars, and advanced workshops tailored to their individual learning needs. Lastly, the GSP works to create a community among its scholars where they can not only continue to consult with presenters/instructors, but also one another. A planned future direction for the GSP-P is to incorporate quarterly office hours and discussions for alumni to develop an increased sense of community. This may strengthen commitment to the overall VA mission, leading to increased retainment of talent who now have the knowledge, skills, and confidence to care for aging veterans.

Limitations

GSP is limited by its available funding. Additionally, the number of participants who can enroll each year in GSP-P (not including webinars) is capped by policy. Another limitation is the number of QI coaches available to mentor scholars on their projects.

Outcomes of GSP-P have been extremely favorable. Following participation in the program, we have found a significant increase in confidence in geropsychology practice among clinicians, as well as enhanced knowledge and skills across competency domains.^15,19 We have observed rising attendance in our annual webinar series and graduates of our introductory courses participate in subsequent trainings (eg, advanced workshop or virtual practicum). Several graduates of GSP-P have become board certified in geropsychology by the American Board of Geropsychology and many proceed to supervise geropsychology-focused clinical rotations for psychology practicum students, predoctoral interns, and postdoctoral fellows. This suggests that the reach of GSP-P programming may extend farther than reported in this article.

The needs of aging veterans have also changed based on cohort differences, as the population of World War II and Korean War era veterans has declined and the number of older Vietnam era veterans has grown. We expect different challenges with older Gulf War and post-9/11 era veterans. For instance, 17% of troops deployed to Iraq or Afghanistan following 9/11 experienced mild traumatic brain injury (TBI), and 59% of those experienced > 1 mild TBI.²³ Research indicates that younger post-9/11 veterans have a 3-fold risk of developing early onset dementia after experiencing a TBI.²⁴ Therefore, even though post-9/11 veterans are not older in terms of chronological age, some may experience symptoms and conditions more often occurring in older veterans. As a result, it would be beneficial for clinicians to learn about the presentation and treatment of geriatric conditions such as dementia.

Moving forward, the GSP-P should identify potential opportunities to collaborate with the non-VHA mental health community–which also faces a shortage of geriatric mental health clinicians–to extend educational opportunities to improve care for veterans in all settings (eg, cosponsor training opportunities open to both VHA and non-VHA clinicians).^8,25 Many aging veterans may receive portions of their health care outside the VHA, particularly those who reside in rural areas. Additionally, as veterans age, so do their support systems (eg, family members, friends, spouses, caregivers, and even clinicians), most of whom will receive care outside of the VHA. Community education collaborations will not only improve the care of older veterans, but also the care of older adults in the general population.

Promising directions include the adoption of the GSP model in other health care settings. Recently, Indian Health Service has adapted the model, beginning with primary care clinicians and pharmacists and is beginning to expand to other disciplines. Additional investments in VHA workforce training include the availability of geropsychology internship and fellowship training opportunities through the Office of Academic Affiliations, which provide earlier opportunities to specialize in geropsychology. Continued investment in both prelicensure and postpsychology licensure training efforts are needed within the VHA to meet the geriatric mental health needs of veterans.

Acknowledgments

The authors wish to acknowledge Terri Huh, PhD, for her contributions to the development and initiation of the GSP-P. The authors also appreciate the collaboration and quality initiative training led by Carol Callaway-Lane, DNP, ACNP-BC, and her team.

Urine drug screen (UDS) monitoring is a common risk-mitigation strategy tool for prescribing controlled substances.^1-3 Not only is UDS monitoring highlighted by clinical practice guidelines for opioid prescribing for chronic pain,^1,2 it has also been suggested as best practice for benzodiazepines³ and a consideration for other controlled substances. Monitoring UDSs helps confirm adherence to the prescribed treatment regimen while also screening for substance use that may increase patient risk.

UDS results can be complex and have profound implications for the patient’s treatment plan. Drug metabolites for opioids are particularly complicated; for example, synthetic and semisynthetic opioids are not detected on routine opiate immunoassays.⁴ This may lead a clinician to falsely assume the patient is not taking their fentanyl or tramadol medication as directed—or potentially even diverting—in the face of a negative opiate result.⁵ Routine UDSs are also subject to the pitfall of false-positive results due to coprescribed medications; for example, bupropion can lead to a false-positive amphetamine result, whereas sertraline can lead to a false-positive benzodiazepine result.⁶ Retrospective reviews of clinician behavior surrounding UDS interpretation have demonstrated knowledge gaps and inconsistent communication practices with patients.^7,8

Given the complexity of UDS interpretation and its close relationship with medications, pharmacists are positioned to play an important role in the process. Pharmacists are embedded in pain-management teams and involved in prescription drug monitoring programs (PDMPs) for many health systems. The Veterans Health Administration (VHA) has supported the hiring of pain management, opioid safety, and PDMP coordinators (PMOP) at its facilities to provide clinical pain-management guidance, support national initiatives, and uphold legislative requirements.⁹ In many facilities, a pharmacist is hired specifically for these positions.

Clinical dashboards have been used by pharmacists in a variety of settings.^10-13 They allow clinicians at a broad level to target interventions needed across a patient population, then produce a list of actionable patients to facilitate delivery of that intervention on an individual level.¹³ Between 2021 and 2022, a clinical dashboard to review potentially discrepant UDS results was made available for use at US Department of Veterans Affairs (VA) medical centers. Evidence exists in primary and specialty care settings that implementation of an opioid-prescribing clinical dashboard improves completion rates of risk-mitigation strategies such as UDS and opioid treatment agreements.^14,15 To our knowledge there is no published research on the use and outcomes of a clinical dashboard that allows users to efficiently review discrepant UDS results when compared to a list of currently prescribed medications.

Given the availability of the UDS dashboard at the VA Black Hills Health Care System (VABHHCS) in South Dakota and the hiring of a PMOP coordinator pharmacist, the aim of this quality improvement project was 2-fold: to implement a pharmacist-led process to monitor the UDS dashboard for potentially discrepant results and to describe the quantity and types of interventions made by the clinical pharmacist leading this process.

Quality Improvement Project

A clinical UDS dashboard was created by the VA Northwest Health Network and made available for use at VHA sites between 2021 and 2022. The UDS dashboard is housed on a secure, Power BI Report Server (Microsoft), with access restricted to only those with patient health data privileges. The dashboard identifies all local patients with a UDS that returned with a potential discrepancy, defined as an unexpected positive result (eg, a detected substance not recently prescribed or documented on the patient’s medication list) and/or an unexpected negative result (eg, a prescribed substance not detected). The UDS dashboard identifies these discrepancies by comparing the patient’s current medication list (both VHA and non-VHA) to their UDS results.

The UDS dashboard displays a summary of UDSs performed, unexpected negative results, unexpected positive results, and potential discrepancies. The user may also specify the laboratory type and time frame of interest to limit displayed results. The user can then view patient-specific data for any category. Among the data are the patient’s UDS results and the completion date, detected (or nondetected) substance(s), ordering clinician, associated medication(s) with last fill date and days’ supply, and whether a confirmatory test has been performed in the past year.

VABHHCS uses an extended UDS immunoassay (PROFILE-V, MEDTOX Diagnostics) that reports on 11 substances: opiates, oxycodone, buprenorphine, methadone, amphetamines, methamphetamine, barbiturates, benzodiazepines, cocaine metabolites, cannabinoids (tetrahydrocannabinol [THC]), and phencyclidine. These substances appear on the UDS dashboard. The project protocol initially included monitoring for tramadol but that was later removed because it was not available with this UDS immunoassay.

Pharmacist Process

Either the PMOP coordinator or pharmacy resident monitored the UDS dashboard weekly. Any patients identified as having a potential discrepancy were reviewed. If the discrepancy was determined to be significant, the PMOP coordinator or pharmacy resident would review the patient electronic health record. If warranted, the patient was contacted and asked about newly prescribed medications, missed and recent medication doses, and illicit substance use. Potential interventions during in-depth review included: (1) discussing future actions with the primary care clinician and/or prescriber of the controlled substance; (2) ordering a confirmatory test on the original urine sample; (3) evaluating for sources of potential false-positive results; (4) completing an updated PDMP if not performed within the past year; (5) referring patients for substance use disorder treatment or counseling; or (6) consulting the local narcotics review committee. A progress note was entered into the electronic health record with the findings and any actions taken, and an alert for the primary care clinician and/or prescriber of the controlled substance.

Implementation and Analysis

This quality improvement project spanned 16 weeks from June 2022 through September 2022. Any patient with a UDS that returned with a significant discrepancy was reviewed. The primary outcome was interventions made by the PMOP coordinator or pharmacy resident, as well as time taken to perform the in-depth review of each patient. Patient demographics were also collected. The protocol for this project was approved by the VABHHCS pharmacy and therapeutics committee and was determined to meet guidelines for a nonresearch quality improvement project.

Results

From June 2022 through September 2022, 700 UDSs were performed at VABHHCS with 278 (39.7%) patients identified as having a potential discrepancy based on UDS results. Sixty patients (8.6%) had significant discrepancies that warranted in-depth review. The most common reasons for determining whether a potential discrepancy was not significant included unexpected negatives due to documented non-VA medications no longer being prescribed, unexpected positives due to recent expiration of a controlled substance prescription the patient was still taking, or unexpected positives due to the detection of a substance for which the clinician was already aware. During the 16-week study period, the mean number of patients warranting in-depth review was 4 per week.

The patients were predominantly male with a mean age of 61 years, and most (87%) were prescribed at least 1 controlled substance (mean, 1.1), primarily opioids for pain management (Table 1). Most patients had recent substance risk mitigation with UDS (56%) and PDMP (65%) checks within the past year. Of the 60 patients reviewed with significant UDS discrepancies, 50% had a history of discrepant UDS results. Of the 60 UDS discrepancies, there were 37 unexpected positive results (62%), 17 unexpected negative results (28%), and 10 patients with both positive and negative results (17%). THC was the most frequently detected substance, followed by opiates, benzodiazepines, and amphetamines (Table 2).

Each in-depth review with interventions by the PMOP coordinator or pharmacy resident lasted a mean of 14 minutes (Table 3). Five patients were successfully contacted for an interview and 7 patients could not be contacted. The ordering clinician of the UDS sometimes had contacted these patients prior to the PMOP coordinator or pharmacy resident reviewing the UDS dashboard, eliminating the need for additional follow-up.

The most common pharmacist intervention was discussing future actions with the primary care clinician and/or prescriber of the controlled substance (n = 39; 65%). These conversations resulted in actions such as ordering a repeat UDS with confirmatory testing at a future date or agreeing that the clinician would discuss the results and subsequent actions with the patient at an upcoming visit. Pharmacist interventions also included 25 PDMP queries (42%) and 9 orders of confirmatory UDS on the original urine sample (15%). Only 1 patient was evaluated by the narcotics review committee, which resulted in a controlled substance flag being placed on their profile. No patients were referred to substance use disorder treatment or counseling. It was offered to and declined by 1 patient, and 3 patients were already engaged in these services.

Medication therapies that could contribute to false-positive results were also evaluated. Fourteen patients who tested positive for THC had a prescription for a nonsteroidal anti-inflammatory drug or proton-pump inhibitor, which could have created a false-positive result.⁶ One patient who tested positive for amphetamines had a prescription for phentermine.¹⁶ No other potential false-positive results were identified.

Findings of this project illustrate that the use of a clinical pharmacist to monitor a dashboard of discrepant UDS results created opportunities for collaboration with clinicians and impacted confirmatory testing and PDMP monitoring practices.

At the local level, the process had numerous benefits. First, it was a reasonable amount of workload to generate pharmacist interventions: the PMOP coordinator conducted an average of 4 in-depth reviews weekly, each lasting about 14 minutes. Thus, the UDS dashboard allowed the PMOP coordinator to actively surveil all incoming UDS results for potential discrepancies in about 1 hour each week. Pairing the automation of the UDS dashboard with the clinical judgment of the PMOP coordinator seemed to maximize efficiency. VABHHCS provides primary and secondary medical and surgical care to a rural population of approximately 20,000 patients across 5 states; the time required at facilities that serve a higher volume of patients may be greater.

Second, the project served as an opportunity for the PMOP coordinator to provide case-specific clinician education on UDS monitoring. As medication experts, pharmacists can apply their medication-related knowledge to UDS interpretation. This includes understanding drug metabolism and classification and how they apply to UDS results, as well as recognizing medication therapies that could contribute to false-positive UDS results. Research suggests that clinicians may have gaps in their knowledge and may welcome pharmacist assistance in interpreting UDS results.^7,8

Third, the project helped improve rates of confirmatory testing for those with unexpected positive UDS results. Confirmatory testing should be strongly considered if positive results would have significant implications on the future course of treatment.⁴ The PMOP coordinator ordered a confirmatory test on 9 patients using the same urine sample used to conduct the initial UDS, minimizing the burden on the patient and laboratory staff. Confirmatory testing was limited by the laboratory’s sample retention period; if the need for confirmatory testing was not recognized soon enough, the sample would no longer be available for retesting. Health systems may consider the use of reflexive confirmatory testing with UDS as an alternative approach, although this may come at an additional cost and may not be warranted in many cases (eg, only 39.7% of all potential discrepancies were deemed as significant within our project).

There were notable incidental findings in our quality improvement project. Among patients with a significant discrepancy on UDS, 50% had a history of ≥ 1 discrepant UDS result. This further emphasizes the importance of appropriate use and interpretation of UDS monitoring for all clinicians, as this may prevent prolonged and potentially inappropriate treatment regimens. Secondly, rates of mental health diagnoses among those with a significant UDS discrepancy seemed relatively high compared to population-level data. For example, among veterans, the overall lifetime prevalence of posttraumatic stress disorder is estimated to be 8.0%; in our project, 35% of patients with a significant UDS discrepancy had a posttraumatic stress disorder diagnosis.¹⁷ This relationship may be an area of further study.

Lastly, it was surprising that the overall rates of UDS and PDMP checks within the past year were 56% and 65%, respectively. VABHHCS requires veterans on controlled substances to have these risk-mitigation strategies performed annually, so our suspicion is that many were falling out due to having been most recently evaluated 12 to 16 months prior. This may represent a limitation of our data-collection method, which reviewed only the previous 12 months.

Limitations

This project was carried out over a period of only 4 months. As a result, only 60 patients received an in-depth review from the PMOP coordinator. Second, the timeliness of the intervention seemed crucial, as delayed in-depth reviews resulted in fewer opportunities to order confirmatory tests or collaborate with clinicians prior to devising an updated plan. Additionally, our process called for UDS dashboard monitoring once a week. Given that the laboratory held samples for only 48 hours, twice- or thrice-weekly review of the UDS dashboard would have allowed for more confirmatory testing, along with more immediate clinician collaboration. Most importantly, the outcomes of this project are only presented via descriptive statistics and without the results of any comparison group, making it impossible to draw firm conclusions about this approach compared to standard-care processes.

Conclusions

This quality improvement project has proven to be valuable at VABHHCS and we intend to continue this pharmacist-led process to monitor the UDS dashboard. VABHHCS leadership are also discussing UDS practices more broadly to further enhance patient management. Within the VA, the PMOP coordinator—charged with being the local coordinator of appropriate pain management and opioid safety practices—is well positioned to assume these responsibilities. Outside of the VA, a pain-management clinical pharmacist or any pharmacist embedded within primary care could similarly perform these duties. Previous literature regarding the implementation of clinical dashboards suggests that with the appropriate software engineering teams and infrastructure, this tool could also be feasibly developed and implemented at other health systems relatively quickly.¹⁴

Overall, a pharmacist-led process to efficiently monitor a dashboard of discrepant UDS results led to opportunities for collaboration with clinicians and positively impacted confirmatory testing and PDMP monitoring at a rural VA health system.

Acknowledgments

The authors express their gratitude to Patrick Spoutz, PharmD, BCPS, VISN 20 Pharmacist Executive, for introducing and sharing the UDS dashboard with our team.

Urine drug screen (UDS) monitoring is a common risk-mitigation strategy tool for prescribing controlled substances.^1-3 Not only is UDS monitoring highlighted by clinical practice guidelines for opioid prescribing for chronic pain,^1,2 it has also been suggested as best practice for benzodiazepines³ and a consideration for other controlled substances. Monitoring UDSs helps confirm adherence to the prescribed treatment regimen while also screening for substance use that may increase patient risk.

UDS results can be complex and have profound implications for the patient’s treatment plan. Drug metabolites for opioids are particularly complicated; for example, synthetic and semisynthetic opioids are not detected on routine opiate immunoassays.⁴ This may lead a clinician to falsely assume the patient is not taking their fentanyl or tramadol medication as directed—or potentially even diverting—in the face of a negative opiate result.⁵ Routine UDSs are also subject to the pitfall of false-positive results due to coprescribed medications; for example, bupropion can lead to a false-positive amphetamine result, whereas sertraline can lead to a false-positive benzodiazepine result.⁶ Retrospective reviews of clinician behavior surrounding UDS interpretation have demonstrated knowledge gaps and inconsistent communication practices with patients.^7,8

Given the complexity of UDS interpretation and its close relationship with medications, pharmacists are positioned to play an important role in the process. Pharmacists are embedded in pain-management teams and involved in prescription drug monitoring programs (PDMPs) for many health systems. The Veterans Health Administration (VHA) has supported the hiring of pain management, opioid safety, and PDMP coordinators (PMOP) at its facilities to provide clinical pain-management guidance, support national initiatives, and uphold legislative requirements.⁹ In many facilities, a pharmacist is hired specifically for these positions.

Clinical dashboards have been used by pharmacists in a variety of settings.^10-13 They allow clinicians at a broad level to target interventions needed across a patient population, then produce a list of actionable patients to facilitate delivery of that intervention on an individual level.¹³ Between 2021 and 2022, a clinical dashboard to review potentially discrepant UDS results was made available for use at US Department of Veterans Affairs (VA) medical centers. Evidence exists in primary and specialty care settings that implementation of an opioid-prescribing clinical dashboard improves completion rates of risk-mitigation strategies such as UDS and opioid treatment agreements.^14,15 To our knowledge there is no published research on the use and outcomes of a clinical dashboard that allows users to efficiently review discrepant UDS results when compared to a list of currently prescribed medications.

Given the availability of the UDS dashboard at the VA Black Hills Health Care System (VABHHCS) in South Dakota and the hiring of a PMOP coordinator pharmacist, the aim of this quality improvement project was 2-fold: to implement a pharmacist-led process to monitor the UDS dashboard for potentially discrepant results and to describe the quantity and types of interventions made by the clinical pharmacist leading this process.

Quality Improvement Project

A clinical UDS dashboard was created by the VA Northwest Health Network and made available for use at VHA sites between 2021 and 2022. The UDS dashboard is housed on a secure, Power BI Report Server (Microsoft), with access restricted to only those with patient health data privileges. The dashboard identifies all local patients with a UDS that returned with a potential discrepancy, defined as an unexpected positive result (eg, a detected substance not recently prescribed or documented on the patient’s medication list) and/or an unexpected negative result (eg, a prescribed substance not detected). The UDS dashboard identifies these discrepancies by comparing the patient’s current medication list (both VHA and non-VHA) to their UDS results.

The UDS dashboard displays a summary of UDSs performed, unexpected negative results, unexpected positive results, and potential discrepancies. The user may also specify the laboratory type and time frame of interest to limit displayed results. The user can then view patient-specific data for any category. Among the data are the patient’s UDS results and the completion date, detected (or nondetected) substance(s), ordering clinician, associated medication(s) with last fill date and days’ supply, and whether a confirmatory test has been performed in the past year.

VABHHCS uses an extended UDS immunoassay (PROFILE-V, MEDTOX Diagnostics) that reports on 11 substances: opiates, oxycodone, buprenorphine, methadone, amphetamines, methamphetamine, barbiturates, benzodiazepines, cocaine metabolites, cannabinoids (tetrahydrocannabinol [THC]), and phencyclidine. These substances appear on the UDS dashboard. The project protocol initially included monitoring for tramadol but that was later removed because it was not available with this UDS immunoassay.

Pharmacist Process

Either the PMOP coordinator or pharmacy resident monitored the UDS dashboard weekly. Any patients identified as having a potential discrepancy were reviewed. If the discrepancy was determined to be significant, the PMOP coordinator or pharmacy resident would review the patient electronic health record. If warranted, the patient was contacted and asked about newly prescribed medications, missed and recent medication doses, and illicit substance use. Potential interventions during in-depth review included: (1) discussing future actions with the primary care clinician and/or prescriber of the controlled substance; (2) ordering a confirmatory test on the original urine sample; (3) evaluating for sources of potential false-positive results; (4) completing an updated PDMP if not performed within the past year; (5) referring patients for substance use disorder treatment or counseling; or (6) consulting the local narcotics review committee. A progress note was entered into the electronic health record with the findings and any actions taken, and an alert for the primary care clinician and/or prescriber of the controlled substance.

Implementation and Analysis

This quality improvement project spanned 16 weeks from June 2022 through September 2022. Any patient with a UDS that returned with a significant discrepancy was reviewed. The primary outcome was interventions made by the PMOP coordinator or pharmacy resident, as well as time taken to perform the in-depth review of each patient. Patient demographics were also collected. The protocol for this project was approved by the VABHHCS pharmacy and therapeutics committee and was determined to meet guidelines for a nonresearch quality improvement project.

Results

From June 2022 through September 2022, 700 UDSs were performed at VABHHCS with 278 (39.7%) patients identified as having a potential discrepancy based on UDS results. Sixty patients (8.6%) had significant discrepancies that warranted in-depth review. The most common reasons for determining whether a potential discrepancy was not significant included unexpected negatives due to documented non-VA medications no longer being prescribed, unexpected positives due to recent expiration of a controlled substance prescription the patient was still taking, or unexpected positives due to the detection of a substance for which the clinician was already aware. During the 16-week study period, the mean number of patients warranting in-depth review was 4 per week.

The patients were predominantly male with a mean age of 61 years, and most (87%) were prescribed at least 1 controlled substance (mean, 1.1), primarily opioids for pain management (Table 1). Most patients had recent substance risk mitigation with UDS (56%) and PDMP (65%) checks within the past year. Of the 60 patients reviewed with significant UDS discrepancies, 50% had a history of discrepant UDS results. Of the 60 UDS discrepancies, there were 37 unexpected positive results (62%), 17 unexpected negative results (28%), and 10 patients with both positive and negative results (17%). THC was the most frequently detected substance, followed by opiates, benzodiazepines, and amphetamines (Table 2).

Each in-depth review with interventions by the PMOP coordinator or pharmacy resident lasted a mean of 14 minutes (Table 3). Five patients were successfully contacted for an interview and 7 patients could not be contacted. The ordering clinician of the UDS sometimes had contacted these patients prior to the PMOP coordinator or pharmacy resident reviewing the UDS dashboard, eliminating the need for additional follow-up.

The most common pharmacist intervention was discussing future actions with the primary care clinician and/or prescriber of the controlled substance (n = 39; 65%). These conversations resulted in actions such as ordering a repeat UDS with confirmatory testing at a future date or agreeing that the clinician would discuss the results and subsequent actions with the patient at an upcoming visit. Pharmacist interventions also included 25 PDMP queries (42%) and 9 orders of confirmatory UDS on the original urine sample (15%). Only 1 patient was evaluated by the narcotics review committee, which resulted in a controlled substance flag being placed on their profile. No patients were referred to substance use disorder treatment or counseling. It was offered to and declined by 1 patient, and 3 patients were already engaged in these services.

Medication therapies that could contribute to false-positive results were also evaluated. Fourteen patients who tested positive for THC had a prescription for a nonsteroidal anti-inflammatory drug or proton-pump inhibitor, which could have created a false-positive result.⁶ One patient who tested positive for amphetamines had a prescription for phentermine.¹⁶ No other potential false-positive results were identified.

Findings of this project illustrate that the use of a clinical pharmacist to monitor a dashboard of discrepant UDS results created opportunities for collaboration with clinicians and impacted confirmatory testing and PDMP monitoring practices.

At the local level, the process had numerous benefits. First, it was a reasonable amount of workload to generate pharmacist interventions: the PMOP coordinator conducted an average of 4 in-depth reviews weekly, each lasting about 14 minutes. Thus, the UDS dashboard allowed the PMOP coordinator to actively surveil all incoming UDS results for potential discrepancies in about 1 hour each week. Pairing the automation of the UDS dashboard with the clinical judgment of the PMOP coordinator seemed to maximize efficiency. VABHHCS provides primary and secondary medical and surgical care to a rural population of approximately 20,000 patients across 5 states; the time required at facilities that serve a higher volume of patients may be greater.

Second, the project served as an opportunity for the PMOP coordinator to provide case-specific clinician education on UDS monitoring. As medication experts, pharmacists can apply their medication-related knowledge to UDS interpretation. This includes understanding drug metabolism and classification and how they apply to UDS results, as well as recognizing medication therapies that could contribute to false-positive UDS results. Research suggests that clinicians may have gaps in their knowledge and may welcome pharmacist assistance in interpreting UDS results.^7,8

Third, the project helped improve rates of confirmatory testing for those with unexpected positive UDS results. Confirmatory testing should be strongly considered if positive results would have significant implications on the future course of treatment.⁴ The PMOP coordinator ordered a confirmatory test on 9 patients using the same urine sample used to conduct the initial UDS, minimizing the burden on the patient and laboratory staff. Confirmatory testing was limited by the laboratory’s sample retention period; if the need for confirmatory testing was not recognized soon enough, the sample would no longer be available for retesting. Health systems may consider the use of reflexive confirmatory testing with UDS as an alternative approach, although this may come at an additional cost and may not be warranted in many cases (eg, only 39.7% of all potential discrepancies were deemed as significant within our project).

There were notable incidental findings in our quality improvement project. Among patients with a significant discrepancy on UDS, 50% had a history of ≥ 1 discrepant UDS result. This further emphasizes the importance of appropriate use and interpretation of UDS monitoring for all clinicians, as this may prevent prolonged and potentially inappropriate treatment regimens. Secondly, rates of mental health diagnoses among those with a significant UDS discrepancy seemed relatively high compared to population-level data. For example, among veterans, the overall lifetime prevalence of posttraumatic stress disorder is estimated to be 8.0%; in our project, 35% of patients with a significant UDS discrepancy had a posttraumatic stress disorder diagnosis.¹⁷ This relationship may be an area of further study.

Lastly, it was surprising that the overall rates of UDS and PDMP checks within the past year were 56% and 65%, respectively. VABHHCS requires veterans on controlled substances to have these risk-mitigation strategies performed annually, so our suspicion is that many were falling out due to having been most recently evaluated 12 to 16 months prior. This may represent a limitation of our data-collection method, which reviewed only the previous 12 months.

Limitations

This project was carried out over a period of only 4 months. As a result, only 60 patients received an in-depth review from the PMOP coordinator. Second, the timeliness of the intervention seemed crucial, as delayed in-depth reviews resulted in fewer opportunities to order confirmatory tests or collaborate with clinicians prior to devising an updated plan. Additionally, our process called for UDS dashboard monitoring once a week. Given that the laboratory held samples for only 48 hours, twice- or thrice-weekly review of the UDS dashboard would have allowed for more confirmatory testing, along with more immediate clinician collaboration. Most importantly, the outcomes of this project are only presented via descriptive statistics and without the results of any comparison group, making it impossible to draw firm conclusions about this approach compared to standard-care processes.

Conclusions

This quality improvement project has proven to be valuable at VABHHCS and we intend to continue this pharmacist-led process to monitor the UDS dashboard. VABHHCS leadership are also discussing UDS practices more broadly to further enhance patient management. Within the VA, the PMOP coordinator—charged with being the local coordinator of appropriate pain management and opioid safety practices—is well positioned to assume these responsibilities. Outside of the VA, a pain-management clinical pharmacist or any pharmacist embedded within primary care could similarly perform these duties. Previous literature regarding the implementation of clinical dashboards suggests that with the appropriate software engineering teams and infrastructure, this tool could also be feasibly developed and implemented at other health systems relatively quickly.¹⁴

Overall, a pharmacist-led process to efficiently monitor a dashboard of discrepant UDS results led to opportunities for collaboration with clinicians and positively impacted confirmatory testing and PDMP monitoring at a rural VA health system.

Acknowledgments

The authors express their gratitude to Patrick Spoutz, PharmD, BCPS, VISN 20 Pharmacist Executive, for introducing and sharing the UDS dashboard with our team.

Urine drug screen (UDS) monitoring is a common risk-mitigation strategy tool for prescribing controlled substances.^1-3 Not only is UDS monitoring highlighted by clinical practice guidelines for opioid prescribing for chronic pain,^1,2 it has also been suggested as best practice for benzodiazepines³ and a consideration for other controlled substances. Monitoring UDSs helps confirm adherence to the prescribed treatment regimen while also screening for substance use that may increase patient risk.

UDS results can be complex and have profound implications for the patient’s treatment plan. Drug metabolites for opioids are particularly complicated; for example, synthetic and semisynthetic opioids are not detected on routine opiate immunoassays.⁴ This may lead a clinician to falsely assume the patient is not taking their fentanyl or tramadol medication as directed—or potentially even diverting—in the face of a negative opiate result.⁵ Routine UDSs are also subject to the pitfall of false-positive results due to coprescribed medications; for example, bupropion can lead to a false-positive amphetamine result, whereas sertraline can lead to a false-positive benzodiazepine result.⁶ Retrospective reviews of clinician behavior surrounding UDS interpretation have demonstrated knowledge gaps and inconsistent communication practices with patients.^7,8

Given the complexity of UDS interpretation and its close relationship with medications, pharmacists are positioned to play an important role in the process. Pharmacists are embedded in pain-management teams and involved in prescription drug monitoring programs (PDMPs) for many health systems. The Veterans Health Administration (VHA) has supported the hiring of pain management, opioid safety, and PDMP coordinators (PMOP) at its facilities to provide clinical pain-management guidance, support national initiatives, and uphold legislative requirements.⁹ In many facilities, a pharmacist is hired specifically for these positions.

Clinical dashboards have been used by pharmacists in a variety of settings.^10-13 They allow clinicians at a broad level to target interventions needed across a patient population, then produce a list of actionable patients to facilitate delivery of that intervention on an individual level.¹³ Between 2021 and 2022, a clinical dashboard to review potentially discrepant UDS results was made available for use at US Department of Veterans Affairs (VA) medical centers. Evidence exists in primary and specialty care settings that implementation of an opioid-prescribing clinical dashboard improves completion rates of risk-mitigation strategies such as UDS and opioid treatment agreements.^14,15 To our knowledge there is no published research on the use and outcomes of a clinical dashboard that allows users to efficiently review discrepant UDS results when compared to a list of currently prescribed medications.

Given the availability of the UDS dashboard at the VA Black Hills Health Care System (VABHHCS) in South Dakota and the hiring of a PMOP coordinator pharmacist, the aim of this quality improvement project was 2-fold: to implement a pharmacist-led process to monitor the UDS dashboard for potentially discrepant results and to describe the quantity and types of interventions made by the clinical pharmacist leading this process.

Quality Improvement Project

A clinical UDS dashboard was created by the VA Northwest Health Network and made available for use at VHA sites between 2021 and 2022. The UDS dashboard is housed on a secure, Power BI Report Server (Microsoft), with access restricted to only those with patient health data privileges. The dashboard identifies all local patients with a UDS that returned with a potential discrepancy, defined as an unexpected positive result (eg, a detected substance not recently prescribed or documented on the patient’s medication list) and/or an unexpected negative result (eg, a prescribed substance not detected). The UDS dashboard identifies these discrepancies by comparing the patient’s current medication list (both VHA and non-VHA) to their UDS results.

The UDS dashboard displays a summary of UDSs performed, unexpected negative results, unexpected positive results, and potential discrepancies. The user may also specify the laboratory type and time frame of interest to limit displayed results. The user can then view patient-specific data for any category. Among the data are the patient’s UDS results and the completion date, detected (or nondetected) substance(s), ordering clinician, associated medication(s) with last fill date and days’ supply, and whether a confirmatory test has been performed in the past year.

VABHHCS uses an extended UDS immunoassay (PROFILE-V, MEDTOX Diagnostics) that reports on 11 substances: opiates, oxycodone, buprenorphine, methadone, amphetamines, methamphetamine, barbiturates, benzodiazepines, cocaine metabolites, cannabinoids (tetrahydrocannabinol [THC]), and phencyclidine. These substances appear on the UDS dashboard. The project protocol initially included monitoring for tramadol but that was later removed because it was not available with this UDS immunoassay.

Pharmacist Process

Either the PMOP coordinator or pharmacy resident monitored the UDS dashboard weekly. Any patients identified as having a potential discrepancy were reviewed. If the discrepancy was determined to be significant, the PMOP coordinator or pharmacy resident would review the patient electronic health record. If warranted, the patient was contacted and asked about newly prescribed medications, missed and recent medication doses, and illicit substance use. Potential interventions during in-depth review included: (1) discussing future actions with the primary care clinician and/or prescriber of the controlled substance; (2) ordering a confirmatory test on the original urine sample; (3) evaluating for sources of potential false-positive results; (4) completing an updated PDMP if not performed within the past year; (5) referring patients for substance use disorder treatment or counseling; or (6) consulting the local narcotics review committee. A progress note was entered into the electronic health record with the findings and any actions taken, and an alert for the primary care clinician and/or prescriber of the controlled substance.

Implementation and Analysis

This quality improvement project spanned 16 weeks from June 2022 through September 2022. Any patient with a UDS that returned with a significant discrepancy was reviewed. The primary outcome was interventions made by the PMOP coordinator or pharmacy resident, as well as time taken to perform the in-depth review of each patient. Patient demographics were also collected. The protocol for this project was approved by the VABHHCS pharmacy and therapeutics committee and was determined to meet guidelines for a nonresearch quality improvement project.

Results

From June 2022 through September 2022, 700 UDSs were performed at VABHHCS with 278 (39.7%) patients identified as having a potential discrepancy based on UDS results. Sixty patients (8.6%) had significant discrepancies that warranted in-depth review. The most common reasons for determining whether a potential discrepancy was not significant included unexpected negatives due to documented non-VA medications no longer being prescribed, unexpected positives due to recent expiration of a controlled substance prescription the patient was still taking, or unexpected positives due to the detection of a substance for which the clinician was already aware. During the 16-week study period, the mean number of patients warranting in-depth review was 4 per week.

The patients were predominantly male with a mean age of 61 years, and most (87%) were prescribed at least 1 controlled substance (mean, 1.1), primarily opioids for pain management (Table 1). Most patients had recent substance risk mitigation with UDS (56%) and PDMP (65%) checks within the past year. Of the 60 patients reviewed with significant UDS discrepancies, 50% had a history of discrepant UDS results. Of the 60 UDS discrepancies, there were 37 unexpected positive results (62%), 17 unexpected negative results (28%), and 10 patients with both positive and negative results (17%). THC was the most frequently detected substance, followed by opiates, benzodiazepines, and amphetamines (Table 2).

Each in-depth review with interventions by the PMOP coordinator or pharmacy resident lasted a mean of 14 minutes (Table 3). Five patients were successfully contacted for an interview and 7 patients could not be contacted. The ordering clinician of the UDS sometimes had contacted these patients prior to the PMOP coordinator or pharmacy resident reviewing the UDS dashboard, eliminating the need for additional follow-up.

The most common pharmacist intervention was discussing future actions with the primary care clinician and/or prescriber of the controlled substance (n = 39; 65%). These conversations resulted in actions such as ordering a repeat UDS with confirmatory testing at a future date or agreeing that the clinician would discuss the results and subsequent actions with the patient at an upcoming visit. Pharmacist interventions also included 25 PDMP queries (42%) and 9 orders of confirmatory UDS on the original urine sample (15%). Only 1 patient was evaluated by the narcotics review committee, which resulted in a controlled substance flag being placed on their profile. No patients were referred to substance use disorder treatment or counseling. It was offered to and declined by 1 patient, and 3 patients were already engaged in these services.

Medication therapies that could contribute to false-positive results were also evaluated. Fourteen patients who tested positive for THC had a prescription for a nonsteroidal anti-inflammatory drug or proton-pump inhibitor, which could have created a false-positive result.⁶ One patient who tested positive for amphetamines had a prescription for phentermine.¹⁶ No other potential false-positive results were identified.

Findings of this project illustrate that the use of a clinical pharmacist to monitor a dashboard of discrepant UDS results created opportunities for collaboration with clinicians and impacted confirmatory testing and PDMP monitoring practices.

At the local level, the process had numerous benefits. First, it was a reasonable amount of workload to generate pharmacist interventions: the PMOP coordinator conducted an average of 4 in-depth reviews weekly, each lasting about 14 minutes. Thus, the UDS dashboard allowed the PMOP coordinator to actively surveil all incoming UDS results for potential discrepancies in about 1 hour each week. Pairing the automation of the UDS dashboard with the clinical judgment of the PMOP coordinator seemed to maximize efficiency. VABHHCS provides primary and secondary medical and surgical care to a rural population of approximately 20,000 patients across 5 states; the time required at facilities that serve a higher volume of patients may be greater.

Second, the project served as an opportunity for the PMOP coordinator to provide case-specific clinician education on UDS monitoring. As medication experts, pharmacists can apply their medication-related knowledge to UDS interpretation. This includes understanding drug metabolism and classification and how they apply to UDS results, as well as recognizing medication therapies that could contribute to false-positive UDS results. Research suggests that clinicians may have gaps in their knowledge and may welcome pharmacist assistance in interpreting UDS results.^7,8

Third, the project helped improve rates of confirmatory testing for those with unexpected positive UDS results. Confirmatory testing should be strongly considered if positive results would have significant implications on the future course of treatment.⁴ The PMOP coordinator ordered a confirmatory test on 9 patients using the same urine sample used to conduct the initial UDS, minimizing the burden on the patient and laboratory staff. Confirmatory testing was limited by the laboratory’s sample retention period; if the need for confirmatory testing was not recognized soon enough, the sample would no longer be available for retesting. Health systems may consider the use of reflexive confirmatory testing with UDS as an alternative approach, although this may come at an additional cost and may not be warranted in many cases (eg, only 39.7% of all potential discrepancies were deemed as significant within our project).

There were notable incidental findings in our quality improvement project. Among patients with a significant discrepancy on UDS, 50% had a history of ≥ 1 discrepant UDS result. This further emphasizes the importance of appropriate use and interpretation of UDS monitoring for all clinicians, as this may prevent prolonged and potentially inappropriate treatment regimens. Secondly, rates of mental health diagnoses among those with a significant UDS discrepancy seemed relatively high compared to population-level data. For example, among veterans, the overall lifetime prevalence of posttraumatic stress disorder is estimated to be 8.0%; in our project, 35% of patients with a significant UDS discrepancy had a posttraumatic stress disorder diagnosis.¹⁷ This relationship may be an area of further study.

Lastly, it was surprising that the overall rates of UDS and PDMP checks within the past year were 56% and 65%, respectively. VABHHCS requires veterans on controlled substances to have these risk-mitigation strategies performed annually, so our suspicion is that many were falling out due to having been most recently evaluated 12 to 16 months prior. This may represent a limitation of our data-collection method, which reviewed only the previous 12 months.

Limitations

This project was carried out over a period of only 4 months. As a result, only 60 patients received an in-depth review from the PMOP coordinator. Second, the timeliness of the intervention seemed crucial, as delayed in-depth reviews resulted in fewer opportunities to order confirmatory tests or collaborate with clinicians prior to devising an updated plan. Additionally, our process called for UDS dashboard monitoring once a week. Given that the laboratory held samples for only 48 hours, twice- or thrice-weekly review of the UDS dashboard would have allowed for more confirmatory testing, along with more immediate clinician collaboration. Most importantly, the outcomes of this project are only presented via descriptive statistics and without the results of any comparison group, making it impossible to draw firm conclusions about this approach compared to standard-care processes.

Conclusions

This quality improvement project has proven to be valuable at VABHHCS and we intend to continue this pharmacist-led process to monitor the UDS dashboard. VABHHCS leadership are also discussing UDS practices more broadly to further enhance patient management. Within the VA, the PMOP coordinator—charged with being the local coordinator of appropriate pain management and opioid safety practices—is well positioned to assume these responsibilities. Outside of the VA, a pain-management clinical pharmacist or any pharmacist embedded within primary care could similarly perform these duties. Previous literature regarding the implementation of clinical dashboards suggests that with the appropriate software engineering teams and infrastructure, this tool could also be feasibly developed and implemented at other health systems relatively quickly.¹⁴

Overall, a pharmacist-led process to efficiently monitor a dashboard of discrepant UDS results led to opportunities for collaboration with clinicians and positively impacted confirmatory testing and PDMP monitoring at a rural VA health system.

Acknowledgments

The authors express their gratitude to Patrick Spoutz, PharmD, BCPS, VISN 20 Pharmacist Executive, for introducing and sharing the UDS dashboard with our team.

More than 37 million Americans (11.3%) have diabetes mellitus (DM), and 90% to 95% are diagnosed with type 2 DM, including nearly 1 in 4 veterans receiving Veterans Health Administration (VHA) care.^1,2 DM is associated with serious negative health outcomes, including cardiovascular disease and subsequent complications as well as significant health care system utilization and cost.^1,3

Group interventions have been identified as a possible method of improving DM outcomes. For example, shared medical appointments (SMAs) have been identified by the VHA as holding promise for improving care and efficiency for DM and other common health conditions.⁴ Although the precise structure and SMA process for managing DM has been noted to be heterogeneous, the appointment is typically led by an interdisciplinary health care team and includes individualized assessment including medication review and adjustment, group education, and troubleshooting challenges with management in a group format.⁵ Research suggests that DM SMAs are a worthwhile treatment approach.⁵ Several studies have found that SMAs were associated with decreased hemoglobin A_1c (Hb A_1c) levels and improvement in overall disease complications and severity.⁶

The high degree of SMA heterogeneity and lack of detailed description of structure and process of SMAs studied has made meta-analysis and other synthesis of the literature difficult.⁵ Consequently, there is inadequate empirically supported guidance for clinicians and health care organizations on how to best implement SMAs and similar group-based treatments. Edelman and colleagues recommended that future research should focus on more consistent and standardized intervention structures and real-world patient- and staff-centered outcomes to address gaps in the literature.⁵ They noted that a mental health professional was utilized in only a minority of SMAs studied.⁵ Additionally, we noted a paucity of studies examining patient satisfaction with SMAs.

Another group-based intervention found to be effective in improving DM outcomes is the 6-session Stanford Diabetes Self-Management Program (DSMP), a workshop led in part by trained peers with DM. The sessions focus on educating patients on DM care and self-management tools. The workshop encourages active practice in building DM self-management skills and confidence. DSMP participation has been associated with improvement in DM-related outcomes, including Hb A_1c levels, amount of exercise, and medication adherence.⁷

While SMAs and DSMP have been shown to enhance clinical outcomes, they provide differing types of patient support. SMAs allow for frequent interaction with a health care professional (HCP) and less emphasis on behavioral health interventions. DSMPs include behavioral health professionals and peer leaders and emphasize higher levels of psychosocial support, but do not offer access to clinicians. It is possible that combining these interventions could result in better outcomes than what either could provide on their own.

In 2018, the Cincinnati Veterans Affairs Medical Center (VAMC) in Ohio offered Diabetes Basic Training, a structured DM intervention. Patients enrolled in the program participated in a 9-week intervention that included 3 SMAs and 6 DSMP sessions. During the SMAs, a clinical psychologist or psychology postdoctoral fellow skilled in motivational interviewing facilitated the group to enhance patient engagement and empowerment for improved self-management. In addition, patients participated in structured DSMP groups with an emphasis on action-planning, often surrounding nutrition, physical activity, and other health behavior change information reviewed during the SMAs.

Design and Referral

Self-management programs for chronic health conditions are often underutilized. Although HCPs may wish to connect veterans with available programs, time constraints may limit opportunities for detailed discussions with patients about specific aspects of each program. To simplify this process, a 2-hour orientation program was offered that explained individual and group DM self-management options (Figure). During this initial visit, patients met with an interdisciplinary care team (registered dietician, diabetes nurse practitioner, and behavioral health specialist) and were informed about Diabetes Basic Training, DM clinical care practices, and other related resources available at the Cincinnati VAMC (eg, cooking classes, food pantry). Patients received individualized referral recommendations and were urged to consult with their primary care practitioner to finalize their treatment plan.

Shared Medical Appointments

Diabetes Basic Training interventions had an average of 6 to 8 veterans participating in the weekly groups. The first, fifth, and final weeks were SMAs in which an interdisciplinary team collaboratively provided group-based health care for DM. The team consisted of a registered nurse, a prescriber (eg, nurse practitioner), a moderator (eg, psychologist), and a content expert (eg, nutritionist). Before each SMA began, the nurse checked-in patients in the SMA room and collected heart rate and blood pressure, and performed a diabetic foot check. Each SMA consisted of introductions, group-driven discussions (facilitated by an HCP) and troubleshooting DM self-management challenges. During group discussions, the prescriber initiated a 1-on-1 discussion with each patient in a private office regarding their recent laboratory results, medication regimen, and other aspects of DM care. The patient’s medications were refilled and/or adjusted as needed and other orders and referrals were submitted. If a patient had a medical question, the prescriber and moderator engaged the entire group so all individuals could benefit from generating and hearing answers. When discussion slowed, education was provided on topics generated by the group. Frequent topics included challenges managing DM, concerns, how DM impacted daily life and relationships, and sharing successes. As needed, HCPs spoke individually with patients following the SMA. Patients were sometimes asked, but never required, to do homework consistent with standard DM care (eg, recording what they eat or blood sugar levels). Each SMA session lasted about 2 hours.

Diabetes Self-Management Program

The second, third, fourth, sixth, seventh, and eighth weeks of the program were devoted to the DSMP. These sessions were delivered primarily by veteran peers who received appropriate training, observation, and certification. Each 2-hour educational program provided ample practice in many fundamental self-management skills, such as decision making, problem solving, and action planning. Patients were asked, but never required, to practice related skills during the sessions and to create weekly action plans to be completed between sessions that typically involved increasing exercise or improving diet. Patients were encouraged to follow up with HCPs at SMAs when they had questions requiring HCP expertise. If participants had more immediate concerns regarding their treatment plan and/or medications, they contacted their primary care practitioner prior to the next SMA.

As a part of participation in the program, psychosocial and health data and Hb A_1c levels at baseline (the closest level to 90 days prior to start) and follow-up (the closest level to 90 days after the final session) were collected.⁸ In addition, Problem Areas in Diabetes (PAID), Patient Activation Measure (PAM)-13, and Diabetes Self-Management Questionnaire (DSMQ) were administered at 3 points: during the orientation, in the first week, and in the ninth week of the program.

PAID, a 20-item self-report questionnaire designed to capture emotional distress related to having DM, is a valid and reliable scale able to detect changes over time when used in intervention studies.^9,10 PAM-13 is a 13-item measure designed to assess patient knowledge, skill, and confidence in the self-management of health or chronic conditions based on the original PAM.^11,12 Scores fall into 1 of 4 activation levels, ranging from low levels of confidence and knowledge of health management to high levels of being proactive with one’s care. The PAM-13 has been widely used within health psychology, including research among adults with multiple chronic conditions, individuals with DM or osteoarthritis, and within primary care.^13-15 The DSMQ is a statistically reliable and valid instrument that allows for user-friendly assessment of self-care behaviors associated with glycemic control.^16-18

All measures were collected as part of traditional clinical care, and we present initial program evaluation data to demonstrate potential effectiveness of the clinic model. Paired samples t tests were used to examine differences between baseline and follow-up measures for the 24 veteran participants. The age of participants who completed the program ranged from 42 to 74 years (mean, 68 years); 29% of participants were Black veterans and 12% were female. Examination of clinical outcomes indicated that veterans reported significant increases in activation levels for managing their health increasing from a baseline mean (SD) 62.1 (12.3) to 68.4 (14.5) at follow up (t[23] = 2.15, P = .04). Hb A_1c levels trended downward from a mean (SD) 8.6% (1.3) at baseline to 8.2% (1.2) at 90-day follow up (t[21] 1.05, P = .30). Similar nonsignificant trends in PAID scores were seen for pre- and postprogram reductions in emotional distress related to having DM from a mean (SD) 7.9 (5.0) at baseline to 6.3 (5.1) (t[18] = 11.51, P = .15), and enhanced self-management of glucose with a mean (SD) 6.5 (1.5) at baseline to 6.8 (1.3) at follow up (t[19] = 0.52, P = .61). The trends found in this study show promising outcomes for this pilot group-based DM treatment, though the small sample size (N = 24) limits statistical power. These findings support further exploration and expansion of interdisciplinary health programs supporting veteran self-management.

Discussion

DM is a condition of epidemic proportions that causes substantial negative health outcomes and costs at a national level. Current standards of DM care do not appear to be reversing these trends. Wider implementation of group-based treatment for DM could improve efficiency of care, increase access to quality care, and reduce burden on individual HCPs.

The VHA continues the transformation of its care system, which shifts toward a patient-centered, proactive focus on veteran well-being. This new whole health approach integrates conventional medical treatment with veteran self-empowerment in the pursuit of health goals based on individual veteran’s identified values.¹⁹ This approach emphasizes peer-led explorations of veterans’ aspirations, purpose, and individual mission, personalized health planning, and use of whole health coaches and well-being programs, with both allopathic and complementary and integrative clinical care centered around veterans’ identified goals and priorities.²⁰

Including a program like Diabetes Basic Training as a part of whole health programming could offer several benefits. Diabetes Basic Training is unique in its integration of more traditional SMA structure with psychosocial interventions including values identification and motivational interviewing strategies to enhance patient engagement. Veterans can learn from each other’s experiences and concerns, leading to better DM management knowledge and skills. The group nature of the sessions enhances opportunities for emotional support and reduced isolation, as well as peer accountability for maintaining medication adherence.

By meeting with HCPs from multiple disciplines, veterans are exposed to different perspectives on self-management techniques, including behavioral approaches for overcoming barriers to behavior change. Clinicians have more time to engage with patients, building stronger relationships and trust. SMAs are cost-efficient and time efficient, allowing HCPs to see multiple patients at once, reducing wait times and increasing the number of patients treated in a given time frame.

The COVID-19 pandemic temporarily impacted the ongoing expansion of the program, when so many services were shifted from in-person to virtual classes. Due to staffing and other logistic issues, our pilot program was suspended during that time, but plans to resume the program by early 2024 are moving forward.

CONCLUSIONS

The Diabetes Basic Training program serves as a successful model for implementation within a VAMC. Although the number of veterans with complete data available for analysis was small, the trends exhibited in the preliminary outcome data are promising. We encourage other VAMCs to replicate this program with a larger participant base and evaluate its impact on veteran health outcomes. Next steps include comparing the clinical data from treatment as usual with outcomes from DM group participants. As the program resumes, we will reinitiate recruitment efforts to increase HCP referrals to this program.

More than 37 million Americans (11.3%) have diabetes mellitus (DM), and 90% to 95% are diagnosed with type 2 DM, including nearly 1 in 4 veterans receiving Veterans Health Administration (VHA) care.^1,2 DM is associated with serious negative health outcomes, including cardiovascular disease and subsequent complications as well as significant health care system utilization and cost.^1,3

Group interventions have been identified as a possible method of improving DM outcomes. For example, shared medical appointments (SMAs) have been identified by the VHA as holding promise for improving care and efficiency for DM and other common health conditions.⁴ Although the precise structure and SMA process for managing DM has been noted to be heterogeneous, the appointment is typically led by an interdisciplinary health care team and includes individualized assessment including medication review and adjustment, group education, and troubleshooting challenges with management in a group format.⁵ Research suggests that DM SMAs are a worthwhile treatment approach.⁵ Several studies have found that SMAs were associated with decreased hemoglobin A_1c (Hb A_1c) levels and improvement in overall disease complications and severity.⁶

The high degree of SMA heterogeneity and lack of detailed description of structure and process of SMAs studied has made meta-analysis and other synthesis of the literature difficult.⁵ Consequently, there is inadequate empirically supported guidance for clinicians and health care organizations on how to best implement SMAs and similar group-based treatments. Edelman and colleagues recommended that future research should focus on more consistent and standardized intervention structures and real-world patient- and staff-centered outcomes to address gaps in the literature.⁵ They noted that a mental health professional was utilized in only a minority of SMAs studied.⁵ Additionally, we noted a paucity of studies examining patient satisfaction with SMAs.

Another group-based intervention found to be effective in improving DM outcomes is the 6-session Stanford Diabetes Self-Management Program (DSMP), a workshop led in part by trained peers with DM. The sessions focus on educating patients on DM care and self-management tools. The workshop encourages active practice in building DM self-management skills and confidence. DSMP participation has been associated with improvement in DM-related outcomes, including Hb A_1c levels, amount of exercise, and medication adherence.⁷

While SMAs and DSMP have been shown to enhance clinical outcomes, they provide differing types of patient support. SMAs allow for frequent interaction with a health care professional (HCP) and less emphasis on behavioral health interventions. DSMPs include behavioral health professionals and peer leaders and emphasize higher levels of psychosocial support, but do not offer access to clinicians. It is possible that combining these interventions could result in better outcomes than what either could provide on their own.

In 2018, the Cincinnati Veterans Affairs Medical Center (VAMC) in Ohio offered Diabetes Basic Training, a structured DM intervention. Patients enrolled in the program participated in a 9-week intervention that included 3 SMAs and 6 DSMP sessions. During the SMAs, a clinical psychologist or psychology postdoctoral fellow skilled in motivational interviewing facilitated the group to enhance patient engagement and empowerment for improved self-management. In addition, patients participated in structured DSMP groups with an emphasis on action-planning, often surrounding nutrition, physical activity, and other health behavior change information reviewed during the SMAs.

Design and Referral

Self-management programs for chronic health conditions are often underutilized. Although HCPs may wish to connect veterans with available programs, time constraints may limit opportunities for detailed discussions with patients about specific aspects of each program. To simplify this process, a 2-hour orientation program was offered that explained individual and group DM self-management options (Figure). During this initial visit, patients met with an interdisciplinary care team (registered dietician, diabetes nurse practitioner, and behavioral health specialist) and were informed about Diabetes Basic Training, DM clinical care practices, and other related resources available at the Cincinnati VAMC (eg, cooking classes, food pantry). Patients received individualized referral recommendations and were urged to consult with their primary care practitioner to finalize their treatment plan.

Shared Medical Appointments

Diabetes Basic Training interventions had an average of 6 to 8 veterans participating in the weekly groups. The first, fifth, and final weeks were SMAs in which an interdisciplinary team collaboratively provided group-based health care for DM. The team consisted of a registered nurse, a prescriber (eg, nurse practitioner), a moderator (eg, psychologist), and a content expert (eg, nutritionist). Before each SMA began, the nurse checked-in patients in the SMA room and collected heart rate and blood pressure, and performed a diabetic foot check. Each SMA consisted of introductions, group-driven discussions (facilitated by an HCP) and troubleshooting DM self-management challenges. During group discussions, the prescriber initiated a 1-on-1 discussion with each patient in a private office regarding their recent laboratory results, medication regimen, and other aspects of DM care. The patient’s medications were refilled and/or adjusted as needed and other orders and referrals were submitted. If a patient had a medical question, the prescriber and moderator engaged the entire group so all individuals could benefit from generating and hearing answers. When discussion slowed, education was provided on topics generated by the group. Frequent topics included challenges managing DM, concerns, how DM impacted daily life and relationships, and sharing successes. As needed, HCPs spoke individually with patients following the SMA. Patients were sometimes asked, but never required, to do homework consistent with standard DM care (eg, recording what they eat or blood sugar levels). Each SMA session lasted about 2 hours.

Diabetes Self-Management Program

The second, third, fourth, sixth, seventh, and eighth weeks of the program were devoted to the DSMP. These sessions were delivered primarily by veteran peers who received appropriate training, observation, and certification. Each 2-hour educational program provided ample practice in many fundamental self-management skills, such as decision making, problem solving, and action planning. Patients were asked, but never required, to practice related skills during the sessions and to create weekly action plans to be completed between sessions that typically involved increasing exercise or improving diet. Patients were encouraged to follow up with HCPs at SMAs when they had questions requiring HCP expertise. If participants had more immediate concerns regarding their treatment plan and/or medications, they contacted their primary care practitioner prior to the next SMA.

As a part of participation in the program, psychosocial and health data and Hb A_1c levels at baseline (the closest level to 90 days prior to start) and follow-up (the closest level to 90 days after the final session) were collected.⁸ In addition, Problem Areas in Diabetes (PAID), Patient Activation Measure (PAM)-13, and Diabetes Self-Management Questionnaire (DSMQ) were administered at 3 points: during the orientation, in the first week, and in the ninth week of the program.

PAID, a 20-item self-report questionnaire designed to capture emotional distress related to having DM, is a valid and reliable scale able to detect changes over time when used in intervention studies.^9,10 PAM-13 is a 13-item measure designed to assess patient knowledge, skill, and confidence in the self-management of health or chronic conditions based on the original PAM.^11,12 Scores fall into 1 of 4 activation levels, ranging from low levels of confidence and knowledge of health management to high levels of being proactive with one’s care. The PAM-13 has been widely used within health psychology, including research among adults with multiple chronic conditions, individuals with DM or osteoarthritis, and within primary care.^13-15 The DSMQ is a statistically reliable and valid instrument that allows for user-friendly assessment of self-care behaviors associated with glycemic control.^16-18

All measures were collected as part of traditional clinical care, and we present initial program evaluation data to demonstrate potential effectiveness of the clinic model. Paired samples t tests were used to examine differences between baseline and follow-up measures for the 24 veteran participants. The age of participants who completed the program ranged from 42 to 74 years (mean, 68 years); 29% of participants were Black veterans and 12% were female. Examination of clinical outcomes indicated that veterans reported significant increases in activation levels for managing their health increasing from a baseline mean (SD) 62.1 (12.3) to 68.4 (14.5) at follow up (t[23] = 2.15, P = .04). Hb A_1c levels trended downward from a mean (SD) 8.6% (1.3) at baseline to 8.2% (1.2) at 90-day follow up (t[21] 1.05, P = .30). Similar nonsignificant trends in PAID scores were seen for pre- and postprogram reductions in emotional distress related to having DM from a mean (SD) 7.9 (5.0) at baseline to 6.3 (5.1) (t[18] = 11.51, P = .15), and enhanced self-management of glucose with a mean (SD) 6.5 (1.5) at baseline to 6.8 (1.3) at follow up (t[19] = 0.52, P = .61). The trends found in this study show promising outcomes for this pilot group-based DM treatment, though the small sample size (N = 24) limits statistical power. These findings support further exploration and expansion of interdisciplinary health programs supporting veteran self-management.

Discussion

DM is a condition of epidemic proportions that causes substantial negative health outcomes and costs at a national level. Current standards of DM care do not appear to be reversing these trends. Wider implementation of group-based treatment for DM could improve efficiency of care, increase access to quality care, and reduce burden on individual HCPs.

The VHA continues the transformation of its care system, which shifts toward a patient-centered, proactive focus on veteran well-being. This new whole health approach integrates conventional medical treatment with veteran self-empowerment in the pursuit of health goals based on individual veteran’s identified values.¹⁹ This approach emphasizes peer-led explorations of veterans’ aspirations, purpose, and individual mission, personalized health planning, and use of whole health coaches and well-being programs, with both allopathic and complementary and integrative clinical care centered around veterans’ identified goals and priorities.²⁰

Including a program like Diabetes Basic Training as a part of whole health programming could offer several benefits. Diabetes Basic Training is unique in its integration of more traditional SMA structure with psychosocial interventions including values identification and motivational interviewing strategies to enhance patient engagement. Veterans can learn from each other’s experiences and concerns, leading to better DM management knowledge and skills. The group nature of the sessions enhances opportunities for emotional support and reduced isolation, as well as peer accountability for maintaining medication adherence.

By meeting with HCPs from multiple disciplines, veterans are exposed to different perspectives on self-management techniques, including behavioral approaches for overcoming barriers to behavior change. Clinicians have more time to engage with patients, building stronger relationships and trust. SMAs are cost-efficient and time efficient, allowing HCPs to see multiple patients at once, reducing wait times and increasing the number of patients treated in a given time frame.

The COVID-19 pandemic temporarily impacted the ongoing expansion of the program, when so many services were shifted from in-person to virtual classes. Due to staffing and other logistic issues, our pilot program was suspended during that time, but plans to resume the program by early 2024 are moving forward.

CONCLUSIONS

The Diabetes Basic Training program serves as a successful model for implementation within a VAMC. Although the number of veterans with complete data available for analysis was small, the trends exhibited in the preliminary outcome data are promising. We encourage other VAMCs to replicate this program with a larger participant base and evaluate its impact on veteran health outcomes. Next steps include comparing the clinical data from treatment as usual with outcomes from DM group participants. As the program resumes, we will reinitiate recruitment efforts to increase HCP referrals to this program.

More than 37 million Americans (11.3%) have diabetes mellitus (DM), and 90% to 95% are diagnosed with type 2 DM, including nearly 1 in 4 veterans receiving Veterans Health Administration (VHA) care.^1,2 DM is associated with serious negative health outcomes, including cardiovascular disease and subsequent complications as well as significant health care system utilization and cost.^1,3

Group interventions have been identified as a possible method of improving DM outcomes. For example, shared medical appointments (SMAs) have been identified by the VHA as holding promise for improving care and efficiency for DM and other common health conditions.⁴ Although the precise structure and SMA process for managing DM has been noted to be heterogeneous, the appointment is typically led by an interdisciplinary health care team and includes individualized assessment including medication review and adjustment, group education, and troubleshooting challenges with management in a group format.⁵ Research suggests that DM SMAs are a worthwhile treatment approach.⁵ Several studies have found that SMAs were associated with decreased hemoglobin A_1c (Hb A_1c) levels and improvement in overall disease complications and severity.⁶

The high degree of SMA heterogeneity and lack of detailed description of structure and process of SMAs studied has made meta-analysis and other synthesis of the literature difficult.⁵ Consequently, there is inadequate empirically supported guidance for clinicians and health care organizations on how to best implement SMAs and similar group-based treatments. Edelman and colleagues recommended that future research should focus on more consistent and standardized intervention structures and real-world patient- and staff-centered outcomes to address gaps in the literature.⁵ They noted that a mental health professional was utilized in only a minority of SMAs studied.⁵ Additionally, we noted a paucity of studies examining patient satisfaction with SMAs.

Another group-based intervention found to be effective in improving DM outcomes is the 6-session Stanford Diabetes Self-Management Program (DSMP), a workshop led in part by trained peers with DM. The sessions focus on educating patients on DM care and self-management tools. The workshop encourages active practice in building DM self-management skills and confidence. DSMP participation has been associated with improvement in DM-related outcomes, including Hb A_1c levels, amount of exercise, and medication adherence.⁷

While SMAs and DSMP have been shown to enhance clinical outcomes, they provide differing types of patient support. SMAs allow for frequent interaction with a health care professional (HCP) and less emphasis on behavioral health interventions. DSMPs include behavioral health professionals and peer leaders and emphasize higher levels of psychosocial support, but do not offer access to clinicians. It is possible that combining these interventions could result in better outcomes than what either could provide on their own.

In 2018, the Cincinnati Veterans Affairs Medical Center (VAMC) in Ohio offered Diabetes Basic Training, a structured DM intervention. Patients enrolled in the program participated in a 9-week intervention that included 3 SMAs and 6 DSMP sessions. During the SMAs, a clinical psychologist or psychology postdoctoral fellow skilled in motivational interviewing facilitated the group to enhance patient engagement and empowerment for improved self-management. In addition, patients participated in structured DSMP groups with an emphasis on action-planning, often surrounding nutrition, physical activity, and other health behavior change information reviewed during the SMAs.

Design and Referral

Self-management programs for chronic health conditions are often underutilized. Although HCPs may wish to connect veterans with available programs, time constraints may limit opportunities for detailed discussions with patients about specific aspects of each program. To simplify this process, a 2-hour orientation program was offered that explained individual and group DM self-management options (Figure). During this initial visit, patients met with an interdisciplinary care team (registered dietician, diabetes nurse practitioner, and behavioral health specialist) and were informed about Diabetes Basic Training, DM clinical care practices, and other related resources available at the Cincinnati VAMC (eg, cooking classes, food pantry). Patients received individualized referral recommendations and were urged to consult with their primary care practitioner to finalize their treatment plan.

Shared Medical Appointments

Diabetes Basic Training interventions had an average of 6 to 8 veterans participating in the weekly groups. The first, fifth, and final weeks were SMAs in which an interdisciplinary team collaboratively provided group-based health care for DM. The team consisted of a registered nurse, a prescriber (eg, nurse practitioner), a moderator (eg, psychologist), and a content expert (eg, nutritionist). Before each SMA began, the nurse checked-in patients in the SMA room and collected heart rate and blood pressure, and performed a diabetic foot check. Each SMA consisted of introductions, group-driven discussions (facilitated by an HCP) and troubleshooting DM self-management challenges. During group discussions, the prescriber initiated a 1-on-1 discussion with each patient in a private office regarding their recent laboratory results, medication regimen, and other aspects of DM care. The patient’s medications were refilled and/or adjusted as needed and other orders and referrals were submitted. If a patient had a medical question, the prescriber and moderator engaged the entire group so all individuals could benefit from generating and hearing answers. When discussion slowed, education was provided on topics generated by the group. Frequent topics included challenges managing DM, concerns, how DM impacted daily life and relationships, and sharing successes. As needed, HCPs spoke individually with patients following the SMA. Patients were sometimes asked, but never required, to do homework consistent with standard DM care (eg, recording what they eat or blood sugar levels). Each SMA session lasted about 2 hours.

Diabetes Self-Management Program

The second, third, fourth, sixth, seventh, and eighth weeks of the program were devoted to the DSMP. These sessions were delivered primarily by veteran peers who received appropriate training, observation, and certification. Each 2-hour educational program provided ample practice in many fundamental self-management skills, such as decision making, problem solving, and action planning. Patients were asked, but never required, to practice related skills during the sessions and to create weekly action plans to be completed between sessions that typically involved increasing exercise or improving diet. Patients were encouraged to follow up with HCPs at SMAs when they had questions requiring HCP expertise. If participants had more immediate concerns regarding their treatment plan and/or medications, they contacted their primary care practitioner prior to the next SMA.

As a part of participation in the program, psychosocial and health data and Hb A_1c levels at baseline (the closest level to 90 days prior to start) and follow-up (the closest level to 90 days after the final session) were collected.⁸ In addition, Problem Areas in Diabetes (PAID), Patient Activation Measure (PAM)-13, and Diabetes Self-Management Questionnaire (DSMQ) were administered at 3 points: during the orientation, in the first week, and in the ninth week of the program.

PAID, a 20-item self-report questionnaire designed to capture emotional distress related to having DM, is a valid and reliable scale able to detect changes over time when used in intervention studies.^9,10 PAM-13 is a 13-item measure designed to assess patient knowledge, skill, and confidence in the self-management of health or chronic conditions based on the original PAM.^11,12 Scores fall into 1 of 4 activation levels, ranging from low levels of confidence and knowledge of health management to high levels of being proactive with one’s care. The PAM-13 has been widely used within health psychology, including research among adults with multiple chronic conditions, individuals with DM or osteoarthritis, and within primary care.^13-15 The DSMQ is a statistically reliable and valid instrument that allows for user-friendly assessment of self-care behaviors associated with glycemic control.^16-18

All measures were collected as part of traditional clinical care, and we present initial program evaluation data to demonstrate potential effectiveness of the clinic model. Paired samples t tests were used to examine differences between baseline and follow-up measures for the 24 veteran participants. The age of participants who completed the program ranged from 42 to 74 years (mean, 68 years); 29% of participants were Black veterans and 12% were female. Examination of clinical outcomes indicated that veterans reported significant increases in activation levels for managing their health increasing from a baseline mean (SD) 62.1 (12.3) to 68.4 (14.5) at follow up (t[23] = 2.15, P = .04). Hb A_1c levels trended downward from a mean (SD) 8.6% (1.3) at baseline to 8.2% (1.2) at 90-day follow up (t[21] 1.05, P = .30). Similar nonsignificant trends in PAID scores were seen for pre- and postprogram reductions in emotional distress related to having DM from a mean (SD) 7.9 (5.0) at baseline to 6.3 (5.1) (t[18] = 11.51, P = .15), and enhanced self-management of glucose with a mean (SD) 6.5 (1.5) at baseline to 6.8 (1.3) at follow up (t[19] = 0.52, P = .61). The trends found in this study show promising outcomes for this pilot group-based DM treatment, though the small sample size (N = 24) limits statistical power. These findings support further exploration and expansion of interdisciplinary health programs supporting veteran self-management.

Discussion

DM is a condition of epidemic proportions that causes substantial negative health outcomes and costs at a national level. Current standards of DM care do not appear to be reversing these trends. Wider implementation of group-based treatment for DM could improve efficiency of care, increase access to quality care, and reduce burden on individual HCPs.

The VHA continues the transformation of its care system, which shifts toward a patient-centered, proactive focus on veteran well-being. This new whole health approach integrates conventional medical treatment with veteran self-empowerment in the pursuit of health goals based on individual veteran’s identified values.¹⁹ This approach emphasizes peer-led explorations of veterans’ aspirations, purpose, and individual mission, personalized health planning, and use of whole health coaches and well-being programs, with both allopathic and complementary and integrative clinical care centered around veterans’ identified goals and priorities.²⁰

Including a program like Diabetes Basic Training as a part of whole health programming could offer several benefits. Diabetes Basic Training is unique in its integration of more traditional SMA structure with psychosocial interventions including values identification and motivational interviewing strategies to enhance patient engagement. Veterans can learn from each other’s experiences and concerns, leading to better DM management knowledge and skills. The group nature of the sessions enhances opportunities for emotional support and reduced isolation, as well as peer accountability for maintaining medication adherence.

By meeting with HCPs from multiple disciplines, veterans are exposed to different perspectives on self-management techniques, including behavioral approaches for overcoming barriers to behavior change. Clinicians have more time to engage with patients, building stronger relationships and trust. SMAs are cost-efficient and time efficient, allowing HCPs to see multiple patients at once, reducing wait times and increasing the number of patients treated in a given time frame.

The COVID-19 pandemic temporarily impacted the ongoing expansion of the program, when so many services were shifted from in-person to virtual classes. Due to staffing and other logistic issues, our pilot program was suspended during that time, but plans to resume the program by early 2024 are moving forward.

CONCLUSIONS

The Diabetes Basic Training program serves as a successful model for implementation within a VAMC. Although the number of veterans with complete data available for analysis was small, the trends exhibited in the preliminary outcome data are promising. We encourage other VAMCs to replicate this program with a larger participant base and evaluate its impact on veteran health outcomes. Next steps include comparing the clinical data from treatment as usual with outcomes from DM group participants. As the program resumes, we will reinitiate recruitment efforts to increase HCP referrals to this program.