Program Profile

Veterans are at higher risk for suicide compared with civilian populations.¹ Firearms are the most frequent cause of death in veteran deaths by suicide, likely because about 51% of veterans own ≥ 1 firearms and firearms are the most lethal and readily available mechanism.^1-3 Unsecure firearm storage practices (eg, storing firearms unlocked, in an unsecure location, or loaded with ammunition) are associated with increased suicide risk.⁴ Conversely, secure firearm storage (ie, storing firearms locked and unloaded) is associated with lower suicide risk.⁵

A 2019 study of veterans who own firearms found that only 22.2% store all their firearms unloaded and locked, while 32.7% store ≥ 1 firearm unlocked and loaded, and 45.2% store firearms both unlocked and loaded or locked and unloaded. Only 6.3% of veterans strongly agreed that having a firearm at home increased suicide risk among household members; however, 77.2% indicated they would ensure a household member could not access firearms if they were concerned about their suicidal ideation.⁶

Another study found that 9.2% of veterans receive lethal means safety counseling from their US Department of Veterans Affairs (VA)-affiliated or non-VA health care professional.⁷ These data highlight a need to educate veterans about the increased risk for suicide associated with storing an unsecured firearm in the household and to connect this understanding to their values of service and protection of others, while simultaneously preparing them and their family members for a potential mental health crisis.

Consistent with the government’s public health approach to suicide prevention, prevention efforts should also enlist the participation of individuals outside health care.⁸ For example, prior research has found that family members are considered highly credible, and engaging them could expand the reach of lethal means safety conversations. A qualitative analysis of 29 veterans found that 17 (57%) said they preferred having a concerned significant other (CSO) (eg, spouse, adult friend, or relative) involved in their suicide prevention care, while 21 (72%) said they would prefer having a CSO assisting in the secure storage of firearms.^9,10 Some veterans may be more amenable to a conversation about firearm access and suicide risk concerns initiated by a CSO rather than by a clinician, indicating the potential benefits of educating and involving CSOs in suicide prevention.¹¹ Involving CSOs in secure firearm storage planning may also strengthen the veteran’s sense of social support, a key protective factor against suicidal ideation.¹²

CSO involvement in secure firearm storage can provide the following benefits: (1) helping the veteran create a secure storage plan, including developing approaches to secure storage; (2) understanding warning signs of suicide; (3) helping the veteran limit access to firearms during a suicidal crisis; (4) helping the veteran remember the secure storage plan; (5) helping the veteran connect with mental health services; and (6) enhancing social support. In most instances, CSOs are physically close to the veteran (eg, live in the same household) and have a greater practical ability to support and affect change with respect to changes in firearm storage practices.

This article describes the development of an intervention that incorporates CSO involvement in firearms safety efforts for veterans with guidance from VA mental health care practitioners (HCPs). The goal is to provide HCPs and other key stakeholders with a detailed description of the intervention and to suggest potential strategies for how to involve CSOs in suicide prevention.

This article follows the Guideline for Reporting Evidence-based Practice Educational interventions and Teaching checklist, which was developed to facilitate standardized reporting and replication for education interventions.¹³ Applicable portions of the checklist are outlined, with others (ie, incentives, planned/unplanned changes, attendance, and other outcomes) to be addressed in future research.

FFAST INTERVENTION

Training (FFAST) intervention promotes voluntary secure firearm storage, engages CSOs in veteran mental health care, and provides psychoeducation and skills to support crisis management. The intervention was developed for all veterans who do not securely store firearms.

Theory

The intervention incorporates motivational interviewing techniques, as ambivalence about changing firearm storage behaviors is common, particularly when veterans own firearms for safety or protection.^6,14 Motivational interviewing is a collaborative approach that addresses a client’s ambivalence to change by eliciting and exploring the client’s own arguments related to change.¹⁴ An important aspect of developing this intervention was to ensure it would be culturally relevant to veteran firearm owners and their CSOs.¹⁵ Further, involvement of the CSO is intentional and meant to boost social support, a known buffering factor against suicide risk.¹²

Objectives

This intervention’s primary objective was for veteran participants to identify secure firearm storage practices and develop a plan for implementing them, including when a veteran or other household member experiences a mental health crisis. For CSOs, the primary objective is to learn how to help the veteran connect with mental health resources if needed and support secure firearm storage as necessary. The overall goal is to learn how to identify warning signs for suicide and how to respond to a mental health crisis through a collaborative process, including securing firearms in a crisis situation.

Materials, Educational Strategies, and Instructors

Training for delivering the intervention was provided via direct consultation with the developer of the intervention and manual. The manual contains pertinent background information to provide context for the intervention’s significance and rationale, including the role of firearms in suicides and current lethal means safety initiatives. It also describes the purpose and objective of each intervention component in detail in addition to providing a script for interventionists to follow to complete each objective.

Training materials for veterans and CSOs include a single Firearms Secure Storage Planning worksheet completed during the intervention, with which the interventionist guides participants through the creation of a secure firearm storage plan (Table). Educational strategies include psychoeducation and Socratic questioning (eg, questioning focused on guiding participants toward the intervention goals) delivered verbally by the interventionist.

The intervention is delivered in person or virtually during a single 90-minute session with a veteran and CSO. Veterans and CSOs work with the interventionist to complete collaborative activities during the session and have self-directive learning activities or homework.

The intervention has 4 primary components: (1) CSO involvement; (2) psychoeducation; (3) secure firearm storage; and (4) how to respond to a mental health crisis. Each CSO should have an established relationship with the veteran, be willing and able to be present during the intervention, and remain an encouraging support person for the veteran. The interventionist emphasizes that it is part of the VA mission for staff to care about the veteran, and that initiating such contact with a CSO is meant to prioritize veteran safety and the safety of their family. Psychoeducation on mental health symptoms, suicide warning signs, veteran suicide rates and lethal means, and the benefits of secure firearm storage, is incorporated in the intervention.

The secure firearm storage component consists of 7 subcomponents: (1) general lethal means secure storage; (2) warning signs; (3) dyad communication; (4) lethal means safety when symptoms emerge; (5) coping strategies; (6) social support; and (7) emergency contacts. A lethal means safety worksheet rooted in the Stanley and Brown suicide safety plan model and implemented in VA health care settings is used to facilitate discussions of secure storage (Appendix).¹⁶

CSOs typically have little or no suicidal crisis response training, yet they likely have more interaction with the veteran on a daily basis than HCPs, putting them in a vital position to identify a crisis early and connect the veteran with the proper care. The crisis component prepares the CSO and veteran to navigate a crisis scenario so they can practice their newly developed safety plan and increase their comfort in discussing mental health and suicidal crisis.

FICTIONAL CASE STUDY

Cole, aged 59 years, is a Persian Gulf War veteran and retired police officer. His medical history includes hypothyroidism, hypertension, type 2 diabetes mellitus, chronic posttraumatic stress disorder, major depressive disorder, and insomnia.

Cole's wife of > 30 years, Sheila, joined him for the FFAST intervention. They report having 4 firearms in the home, 3 of which are loaded but stored in a lockbox and 1 that Cole reports is kept on his person for protection. Cole reports passive suicidal ideation, but no plans or intent. When discussing warning signs that a mental health crisis is building, Cole describes feeling anxious, having a change in his speech patterns, and isolating himself. Sheila agrees, but also mentions that Cole is easily angered and becomes nonverbal. Cole and Sheila express difficulty communicating and appear to have a breakthrough moment when Cole says he does not like when Sheila repeats herself, as he feels like she is “poking” at him. Sheila shares concerns for his safety and that she only repeats herself because he refuses to talk.

Cole agrees to verbalize that he is safe but needs time to process his thoughts. Sheila agrees to give him space with a plan to revisit the conversation within an agreed upon timeline. When discussing an updated secure storage plan for their firearms when a mental health crisis is building, Cole commits to allowing Sheila to store the firearm currently on his person in their gun safe, with the ammunition stored separately, and to giving her the gun safe key. They agree to implement this practice until the mental health crisis has passed.

To mitigate a potential crisis, the interventionist discusses possible internal coping strategies for Cole, including writing, reading, walking the dog, listening to music, and baking. People and social settings that could provide distraction involve going to the gym, talking to his friend Carl or his daughter Kelly, and attending the men’s ministry at church. The intervention concludes by discussing professionals or agencies that Cole and Sheila could contact during a crisis. After the intervention, Cole and Sheila are asked to rate their likelihood of using the plan they established during the conversation on a scale of 0 to 10, with 0 being highly unlikely and 10 being extremely likely. Cole responds with 9 and Sheila responds with 10.

DISCUSSION

Lethal means safety remains a critical component of veteran suicide prevention. However, lethal means safety discussions are often implemented after suicide risk has been identified, which may be too late. Thus, having these conversations early and before a crisis may be imperative. Veterans have expressed a desire to have CSOs involved in their suicide prevention treatment, and CSOs can play a key role in recognizing risk factors during everyday life. The FFAST intervention addresses many of these gaps.

Having discussions in advance of a crisis allows veterans to consider an effective secure firearm storage plan outside of the context of a crisis. Including a CSO galvanizes another person to understand a veteran’s needs and assist with secure firearm storage, identify warning signs, and support them during a crisis. These discussions occur in a context where there is less pressure than during a crisis. Features that were more appealing to veterans and their CSOs were also incorporated, such as having the dyad build a plan that is conceptually similar to other public safety initiatives (eg, a fire safety plan, tornado plan, or hurricane plan). Previous research demonstrates that veterans appreciated the nonjudgmental approach and some preferred that clinicians approach the discussion of secure firearm storage within the context of general home and family safety.¹⁷ Additionally, this intervention can build on veterans’ prior military training in preparedness.

Other potential benefits associated with the FFAST intervention include creating an opportunity to strengthen communication between the veteran and CSO. While FFAST is intended to be used with all types of CSOs, this work is consistent with preliminary data from a couples-based suicide prevention study that indicated veterans and their partners reported increases in relationship functioning and marginal decreases in suicidal ideation.¹⁸ It is possible that communication strategies gained from the current intervention could improve veterans’ relationships with their CSOs, which are associated with a greater sense of social support and reduced suicide risk.¹²

The intervention is a brief, single session that may be appealing to veterans and CSOs with full schedules. Evidence suggests that even brief, single-session interventions have a significant impact on beliefs about secure firearm storage, knowledge of lethal means safety, and confidence in having secure firearm storage conversations.¹⁹ However, clinicians should be cautious when extrapolating from the findings of the current case example, which was a one-time intervention with no follow-up.

Future Directions

Pilot testing of the proposed intervention is underway, and future research will include feedback from veterans and CSOs, as well as feasibility and acceptability data collected during the pilot process. The pilot study uses a successive cohort design with an initial 2 sets of 5 veteran and CSO dyads, and subsequent funding has expanded the pilot study to include an additional 30 dyads. Qualitative interviews will be conducted separately with each veteran and CSO, and additional constructs such as feasibility, acceptability, barriers and facilitators to implementation, and changes in secure storage will be examined. This future research may provide a deeper understanding of the broader acceptability, feasibility, and satisfaction associated with a suicide prevention intervention focused on securing firearms and involving veterans and their CSOs. These data could be used to inform future implementation trials and inform the development of an implementation strategy. In the interim, the nature of the manual is summarized in the context of the urgency of suicide prevention in this at-risk population.

Conclusions

FFAST is a novel approach to veteran firearm suicide prevention. By involving CSOs and emphasizing mental health crisis preparedness between them and veterans, the dyad can work in association with HCPs to establish and exercise secure firearm storage practices as part of an at-home safety plan. Implementation of FFAST may be beneficial for all veterans, not only those who have been identified as being at high suicide risk.

Veterans are at higher risk for suicide compared with civilian populations.¹ Firearms are the most frequent cause of death in veteran deaths by suicide, likely because about 51% of veterans own ≥ 1 firearms and firearms are the most lethal and readily available mechanism.^1-3 Unsecure firearm storage practices (eg, storing firearms unlocked, in an unsecure location, or loaded with ammunition) are associated with increased suicide risk.⁴ Conversely, secure firearm storage (ie, storing firearms locked and unloaded) is associated with lower suicide risk.⁵

A 2019 study of veterans who own firearms found that only 22.2% store all their firearms unloaded and locked, while 32.7% store ≥ 1 firearm unlocked and loaded, and 45.2% store firearms both unlocked and loaded or locked and unloaded. Only 6.3% of veterans strongly agreed that having a firearm at home increased suicide risk among household members; however, 77.2% indicated they would ensure a household member could not access firearms if they were concerned about their suicidal ideation.⁶

Another study found that 9.2% of veterans receive lethal means safety counseling from their US Department of Veterans Affairs (VA)-affiliated or non-VA health care professional.⁷ These data highlight a need to educate veterans about the increased risk for suicide associated with storing an unsecured firearm in the household and to connect this understanding to their values of service and protection of others, while simultaneously preparing them and their family members for a potential mental health crisis.

Consistent with the government’s public health approach to suicide prevention, prevention efforts should also enlist the participation of individuals outside health care.⁸ For example, prior research has found that family members are considered highly credible, and engaging them could expand the reach of lethal means safety conversations. A qualitative analysis of 29 veterans found that 17 (57%) said they preferred having a concerned significant other (CSO) (eg, spouse, adult friend, or relative) involved in their suicide prevention care, while 21 (72%) said they would prefer having a CSO assisting in the secure storage of firearms.^9,10 Some veterans may be more amenable to a conversation about firearm access and suicide risk concerns initiated by a CSO rather than by a clinician, indicating the potential benefits of educating and involving CSOs in suicide prevention.¹¹ Involving CSOs in secure firearm storage planning may also strengthen the veteran’s sense of social support, a key protective factor against suicidal ideation.¹²

CSO involvement in secure firearm storage can provide the following benefits: (1) helping the veteran create a secure storage plan, including developing approaches to secure storage; (2) understanding warning signs of suicide; (3) helping the veteran limit access to firearms during a suicidal crisis; (4) helping the veteran remember the secure storage plan; (5) helping the veteran connect with mental health services; and (6) enhancing social support. In most instances, CSOs are physically close to the veteran (eg, live in the same household) and have a greater practical ability to support and affect change with respect to changes in firearm storage practices.

This article describes the development of an intervention that incorporates CSO involvement in firearms safety efforts for veterans with guidance from VA mental health care practitioners (HCPs). The goal is to provide HCPs and other key stakeholders with a detailed description of the intervention and to suggest potential strategies for how to involve CSOs in suicide prevention.

This article follows the Guideline for Reporting Evidence-based Practice Educational interventions and Teaching checklist, which was developed to facilitate standardized reporting and replication for education interventions.¹³ Applicable portions of the checklist are outlined, with others (ie, incentives, planned/unplanned changes, attendance, and other outcomes) to be addressed in future research.

FFAST INTERVENTION

Training (FFAST) intervention promotes voluntary secure firearm storage, engages CSOs in veteran mental health care, and provides psychoeducation and skills to support crisis management. The intervention was developed for all veterans who do not securely store firearms.

Theory

The intervention incorporates motivational interviewing techniques, as ambivalence about changing firearm storage behaviors is common, particularly when veterans own firearms for safety or protection.^6,14 Motivational interviewing is a collaborative approach that addresses a client’s ambivalence to change by eliciting and exploring the client’s own arguments related to change.¹⁴ An important aspect of developing this intervention was to ensure it would be culturally relevant to veteran firearm owners and their CSOs.¹⁵ Further, involvement of the CSO is intentional and meant to boost social support, a known buffering factor against suicide risk.¹²

Objectives

This intervention’s primary objective was for veteran participants to identify secure firearm storage practices and develop a plan for implementing them, including when a veteran or other household member experiences a mental health crisis. For CSOs, the primary objective is to learn how to help the veteran connect with mental health resources if needed and support secure firearm storage as necessary. The overall goal is to learn how to identify warning signs for suicide and how to respond to a mental health crisis through a collaborative process, including securing firearms in a crisis situation.

Materials, Educational Strategies, and Instructors

Training for delivering the intervention was provided via direct consultation with the developer of the intervention and manual. The manual contains pertinent background information to provide context for the intervention’s significance and rationale, including the role of firearms in suicides and current lethal means safety initiatives. It also describes the purpose and objective of each intervention component in detail in addition to providing a script for interventionists to follow to complete each objective.

Training materials for veterans and CSOs include a single Firearms Secure Storage Planning worksheet completed during the intervention, with which the interventionist guides participants through the creation of a secure firearm storage plan (Table). Educational strategies include psychoeducation and Socratic questioning (eg, questioning focused on guiding participants toward the intervention goals) delivered verbally by the interventionist.

The intervention is delivered in person or virtually during a single 90-minute session with a veteran and CSO. Veterans and CSOs work with the interventionist to complete collaborative activities during the session and have self-directive learning activities or homework.

The intervention has 4 primary components: (1) CSO involvement; (2) psychoeducation; (3) secure firearm storage; and (4) how to respond to a mental health crisis. Each CSO should have an established relationship with the veteran, be willing and able to be present during the intervention, and remain an encouraging support person for the veteran. The interventionist emphasizes that it is part of the VA mission for staff to care about the veteran, and that initiating such contact with a CSO is meant to prioritize veteran safety and the safety of their family. Psychoeducation on mental health symptoms, suicide warning signs, veteran suicide rates and lethal means, and the benefits of secure firearm storage, is incorporated in the intervention.

The secure firearm storage component consists of 7 subcomponents: (1) general lethal means secure storage; (2) warning signs; (3) dyad communication; (4) lethal means safety when symptoms emerge; (5) coping strategies; (6) social support; and (7) emergency contacts. A lethal means safety worksheet rooted in the Stanley and Brown suicide safety plan model and implemented in VA health care settings is used to facilitate discussions of secure storage (Appendix).¹⁶

CSOs typically have little or no suicidal crisis response training, yet they likely have more interaction with the veteran on a daily basis than HCPs, putting them in a vital position to identify a crisis early and connect the veteran with the proper care. The crisis component prepares the CSO and veteran to navigate a crisis scenario so they can practice their newly developed safety plan and increase their comfort in discussing mental health and suicidal crisis.

FICTIONAL CASE STUDY

Cole, aged 59 years, is a Persian Gulf War veteran and retired police officer. His medical history includes hypothyroidism, hypertension, type 2 diabetes mellitus, chronic posttraumatic stress disorder, major depressive disorder, and insomnia.

Cole's wife of > 30 years, Sheila, joined him for the FFAST intervention. They report having 4 firearms in the home, 3 of which are loaded but stored in a lockbox and 1 that Cole reports is kept on his person for protection. Cole reports passive suicidal ideation, but no plans or intent. When discussing warning signs that a mental health crisis is building, Cole describes feeling anxious, having a change in his speech patterns, and isolating himself. Sheila agrees, but also mentions that Cole is easily angered and becomes nonverbal. Cole and Sheila express difficulty communicating and appear to have a breakthrough moment when Cole says he does not like when Sheila repeats herself, as he feels like she is “poking” at him. Sheila shares concerns for his safety and that she only repeats herself because he refuses to talk.

Cole agrees to verbalize that he is safe but needs time to process his thoughts. Sheila agrees to give him space with a plan to revisit the conversation within an agreed upon timeline. When discussing an updated secure storage plan for their firearms when a mental health crisis is building, Cole commits to allowing Sheila to store the firearm currently on his person in their gun safe, with the ammunition stored separately, and to giving her the gun safe key. They agree to implement this practice until the mental health crisis has passed.

To mitigate a potential crisis, the interventionist discusses possible internal coping strategies for Cole, including writing, reading, walking the dog, listening to music, and baking. People and social settings that could provide distraction involve going to the gym, talking to his friend Carl or his daughter Kelly, and attending the men’s ministry at church. The intervention concludes by discussing professionals or agencies that Cole and Sheila could contact during a crisis. After the intervention, Cole and Sheila are asked to rate their likelihood of using the plan they established during the conversation on a scale of 0 to 10, with 0 being highly unlikely and 10 being extremely likely. Cole responds with 9 and Sheila responds with 10.

DISCUSSION

Lethal means safety remains a critical component of veteran suicide prevention. However, lethal means safety discussions are often implemented after suicide risk has been identified, which may be too late. Thus, having these conversations early and before a crisis may be imperative. Veterans have expressed a desire to have CSOs involved in their suicide prevention treatment, and CSOs can play a key role in recognizing risk factors during everyday life. The FFAST intervention addresses many of these gaps.

Having discussions in advance of a crisis allows veterans to consider an effective secure firearm storage plan outside of the context of a crisis. Including a CSO galvanizes another person to understand a veteran’s needs and assist with secure firearm storage, identify warning signs, and support them during a crisis. These discussions occur in a context where there is less pressure than during a crisis. Features that were more appealing to veterans and their CSOs were also incorporated, such as having the dyad build a plan that is conceptually similar to other public safety initiatives (eg, a fire safety plan, tornado plan, or hurricane plan). Previous research demonstrates that veterans appreciated the nonjudgmental approach and some preferred that clinicians approach the discussion of secure firearm storage within the context of general home and family safety.¹⁷ Additionally, this intervention can build on veterans’ prior military training in preparedness.

Other potential benefits associated with the FFAST intervention include creating an opportunity to strengthen communication between the veteran and CSO. While FFAST is intended to be used with all types of CSOs, this work is consistent with preliminary data from a couples-based suicide prevention study that indicated veterans and their partners reported increases in relationship functioning and marginal decreases in suicidal ideation.¹⁸ It is possible that communication strategies gained from the current intervention could improve veterans’ relationships with their CSOs, which are associated with a greater sense of social support and reduced suicide risk.¹²

The intervention is a brief, single session that may be appealing to veterans and CSOs with full schedules. Evidence suggests that even brief, single-session interventions have a significant impact on beliefs about secure firearm storage, knowledge of lethal means safety, and confidence in having secure firearm storage conversations.¹⁹ However, clinicians should be cautious when extrapolating from the findings of the current case example, which was a one-time intervention with no follow-up.

Future Directions

Pilot testing of the proposed intervention is underway, and future research will include feedback from veterans and CSOs, as well as feasibility and acceptability data collected during the pilot process. The pilot study uses a successive cohort design with an initial 2 sets of 5 veteran and CSO dyads, and subsequent funding has expanded the pilot study to include an additional 30 dyads. Qualitative interviews will be conducted separately with each veteran and CSO, and additional constructs such as feasibility, acceptability, barriers and facilitators to implementation, and changes in secure storage will be examined. This future research may provide a deeper understanding of the broader acceptability, feasibility, and satisfaction associated with a suicide prevention intervention focused on securing firearms and involving veterans and their CSOs. These data could be used to inform future implementation trials and inform the development of an implementation strategy. In the interim, the nature of the manual is summarized in the context of the urgency of suicide prevention in this at-risk population.

Conclusions

FFAST is a novel approach to veteran firearm suicide prevention. By involving CSOs and emphasizing mental health crisis preparedness between them and veterans, the dyad can work in association with HCPs to establish and exercise secure firearm storage practices as part of an at-home safety plan. Implementation of FFAST may be beneficial for all veterans, not only those who have been identified as being at high suicide risk.

Veterans are at higher risk for suicide compared with civilian populations.¹ Firearms are the most frequent cause of death in veteran deaths by suicide, likely because about 51% of veterans own ≥ 1 firearms and firearms are the most lethal and readily available mechanism.^1-3 Unsecure firearm storage practices (eg, storing firearms unlocked, in an unsecure location, or loaded with ammunition) are associated with increased suicide risk.⁴ Conversely, secure firearm storage (ie, storing firearms locked and unloaded) is associated with lower suicide risk.⁵

A 2019 study of veterans who own firearms found that only 22.2% store all their firearms unloaded and locked, while 32.7% store ≥ 1 firearm unlocked and loaded, and 45.2% store firearms both unlocked and loaded or locked and unloaded. Only 6.3% of veterans strongly agreed that having a firearm at home increased suicide risk among household members; however, 77.2% indicated they would ensure a household member could not access firearms if they were concerned about their suicidal ideation.⁶

Another study found that 9.2% of veterans receive lethal means safety counseling from their US Department of Veterans Affairs (VA)-affiliated or non-VA health care professional.⁷ These data highlight a need to educate veterans about the increased risk for suicide associated with storing an unsecured firearm in the household and to connect this understanding to their values of service and protection of others, while simultaneously preparing them and their family members for a potential mental health crisis.

Consistent with the government’s public health approach to suicide prevention, prevention efforts should also enlist the participation of individuals outside health care.⁸ For example, prior research has found that family members are considered highly credible, and engaging them could expand the reach of lethal means safety conversations. A qualitative analysis of 29 veterans found that 17 (57%) said they preferred having a concerned significant other (CSO) (eg, spouse, adult friend, or relative) involved in their suicide prevention care, while 21 (72%) said they would prefer having a CSO assisting in the secure storage of firearms.^9,10 Some veterans may be more amenable to a conversation about firearm access and suicide risk concerns initiated by a CSO rather than by a clinician, indicating the potential benefits of educating and involving CSOs in suicide prevention.¹¹ Involving CSOs in secure firearm storage planning may also strengthen the veteran’s sense of social support, a key protective factor against suicidal ideation.¹²

CSO involvement in secure firearm storage can provide the following benefits: (1) helping the veteran create a secure storage plan, including developing approaches to secure storage; (2) understanding warning signs of suicide; (3) helping the veteran limit access to firearms during a suicidal crisis; (4) helping the veteran remember the secure storage plan; (5) helping the veteran connect with mental health services; and (6) enhancing social support. In most instances, CSOs are physically close to the veteran (eg, live in the same household) and have a greater practical ability to support and affect change with respect to changes in firearm storage practices.

This article describes the development of an intervention that incorporates CSO involvement in firearms safety efforts for veterans with guidance from VA mental health care practitioners (HCPs). The goal is to provide HCPs and other key stakeholders with a detailed description of the intervention and to suggest potential strategies for how to involve CSOs in suicide prevention.

This article follows the Guideline for Reporting Evidence-based Practice Educational interventions and Teaching checklist, which was developed to facilitate standardized reporting and replication for education interventions.¹³ Applicable portions of the checklist are outlined, with others (ie, incentives, planned/unplanned changes, attendance, and other outcomes) to be addressed in future research.

FFAST INTERVENTION

Training (FFAST) intervention promotes voluntary secure firearm storage, engages CSOs in veteran mental health care, and provides psychoeducation and skills to support crisis management. The intervention was developed for all veterans who do not securely store firearms.

Theory

The intervention incorporates motivational interviewing techniques, as ambivalence about changing firearm storage behaviors is common, particularly when veterans own firearms for safety or protection.^6,14 Motivational interviewing is a collaborative approach that addresses a client’s ambivalence to change by eliciting and exploring the client’s own arguments related to change.¹⁴ An important aspect of developing this intervention was to ensure it would be culturally relevant to veteran firearm owners and their CSOs.¹⁵ Further, involvement of the CSO is intentional and meant to boost social support, a known buffering factor against suicide risk.¹²

Objectives

This intervention’s primary objective was for veteran participants to identify secure firearm storage practices and develop a plan for implementing them, including when a veteran or other household member experiences a mental health crisis. For CSOs, the primary objective is to learn how to help the veteran connect with mental health resources if needed and support secure firearm storage as necessary. The overall goal is to learn how to identify warning signs for suicide and how to respond to a mental health crisis through a collaborative process, including securing firearms in a crisis situation.

Materials, Educational Strategies, and Instructors

Training for delivering the intervention was provided via direct consultation with the developer of the intervention and manual. The manual contains pertinent background information to provide context for the intervention’s significance and rationale, including the role of firearms in suicides and current lethal means safety initiatives. It also describes the purpose and objective of each intervention component in detail in addition to providing a script for interventionists to follow to complete each objective.

Training materials for veterans and CSOs include a single Firearms Secure Storage Planning worksheet completed during the intervention, with which the interventionist guides participants through the creation of a secure firearm storage plan (Table). Educational strategies include psychoeducation and Socratic questioning (eg, questioning focused on guiding participants toward the intervention goals) delivered verbally by the interventionist.

The intervention is delivered in person or virtually during a single 90-minute session with a veteran and CSO. Veterans and CSOs work with the interventionist to complete collaborative activities during the session and have self-directive learning activities or homework.

The intervention has 4 primary components: (1) CSO involvement; (2) psychoeducation; (3) secure firearm storage; and (4) how to respond to a mental health crisis. Each CSO should have an established relationship with the veteran, be willing and able to be present during the intervention, and remain an encouraging support person for the veteran. The interventionist emphasizes that it is part of the VA mission for staff to care about the veteran, and that initiating such contact with a CSO is meant to prioritize veteran safety and the safety of their family. Psychoeducation on mental health symptoms, suicide warning signs, veteran suicide rates and lethal means, and the benefits of secure firearm storage, is incorporated in the intervention.

The secure firearm storage component consists of 7 subcomponents: (1) general lethal means secure storage; (2) warning signs; (3) dyad communication; (4) lethal means safety when symptoms emerge; (5) coping strategies; (6) social support; and (7) emergency contacts. A lethal means safety worksheet rooted in the Stanley and Brown suicide safety plan model and implemented in VA health care settings is used to facilitate discussions of secure storage (Appendix).¹⁶

CSOs typically have little or no suicidal crisis response training, yet they likely have more interaction with the veteran on a daily basis than HCPs, putting them in a vital position to identify a crisis early and connect the veteran with the proper care. The crisis component prepares the CSO and veteran to navigate a crisis scenario so they can practice their newly developed safety plan and increase their comfort in discussing mental health and suicidal crisis.

FICTIONAL CASE STUDY

Cole, aged 59 years, is a Persian Gulf War veteran and retired police officer. His medical history includes hypothyroidism, hypertension, type 2 diabetes mellitus, chronic posttraumatic stress disorder, major depressive disorder, and insomnia.

Cole's wife of > 30 years, Sheila, joined him for the FFAST intervention. They report having 4 firearms in the home, 3 of which are loaded but stored in a lockbox and 1 that Cole reports is kept on his person for protection. Cole reports passive suicidal ideation, but no plans or intent. When discussing warning signs that a mental health crisis is building, Cole describes feeling anxious, having a change in his speech patterns, and isolating himself. Sheila agrees, but also mentions that Cole is easily angered and becomes nonverbal. Cole and Sheila express difficulty communicating and appear to have a breakthrough moment when Cole says he does not like when Sheila repeats herself, as he feels like she is “poking” at him. Sheila shares concerns for his safety and that she only repeats herself because he refuses to talk.

Cole agrees to verbalize that he is safe but needs time to process his thoughts. Sheila agrees to give him space with a plan to revisit the conversation within an agreed upon timeline. When discussing an updated secure storage plan for their firearms when a mental health crisis is building, Cole commits to allowing Sheila to store the firearm currently on his person in their gun safe, with the ammunition stored separately, and to giving her the gun safe key. They agree to implement this practice until the mental health crisis has passed.

To mitigate a potential crisis, the interventionist discusses possible internal coping strategies for Cole, including writing, reading, walking the dog, listening to music, and baking. People and social settings that could provide distraction involve going to the gym, talking to his friend Carl or his daughter Kelly, and attending the men’s ministry at church. The intervention concludes by discussing professionals or agencies that Cole and Sheila could contact during a crisis. After the intervention, Cole and Sheila are asked to rate their likelihood of using the plan they established during the conversation on a scale of 0 to 10, with 0 being highly unlikely and 10 being extremely likely. Cole responds with 9 and Sheila responds with 10.

DISCUSSION

Lethal means safety remains a critical component of veteran suicide prevention. However, lethal means safety discussions are often implemented after suicide risk has been identified, which may be too late. Thus, having these conversations early and before a crisis may be imperative. Veterans have expressed a desire to have CSOs involved in their suicide prevention treatment, and CSOs can play a key role in recognizing risk factors during everyday life. The FFAST intervention addresses many of these gaps.

Having discussions in advance of a crisis allows veterans to consider an effective secure firearm storage plan outside of the context of a crisis. Including a CSO galvanizes another person to understand a veteran’s needs and assist with secure firearm storage, identify warning signs, and support them during a crisis. These discussions occur in a context where there is less pressure than during a crisis. Features that were more appealing to veterans and their CSOs were also incorporated, such as having the dyad build a plan that is conceptually similar to other public safety initiatives (eg, a fire safety plan, tornado plan, or hurricane plan). Previous research demonstrates that veterans appreciated the nonjudgmental approach and some preferred that clinicians approach the discussion of secure firearm storage within the context of general home and family safety.¹⁷ Additionally, this intervention can build on veterans’ prior military training in preparedness.

Other potential benefits associated with the FFAST intervention include creating an opportunity to strengthen communication between the veteran and CSO. While FFAST is intended to be used with all types of CSOs, this work is consistent with preliminary data from a couples-based suicide prevention study that indicated veterans and their partners reported increases in relationship functioning and marginal decreases in suicidal ideation.¹⁸ It is possible that communication strategies gained from the current intervention could improve veterans’ relationships with their CSOs, which are associated with a greater sense of social support and reduced suicide risk.¹²

The intervention is a brief, single session that may be appealing to veterans and CSOs with full schedules. Evidence suggests that even brief, single-session interventions have a significant impact on beliefs about secure firearm storage, knowledge of lethal means safety, and confidence in having secure firearm storage conversations.¹⁹ However, clinicians should be cautious when extrapolating from the findings of the current case example, which was a one-time intervention with no follow-up.

Future Directions

Pilot testing of the proposed intervention is underway, and future research will include feedback from veterans and CSOs, as well as feasibility and acceptability data collected during the pilot process. The pilot study uses a successive cohort design with an initial 2 sets of 5 veteran and CSO dyads, and subsequent funding has expanded the pilot study to include an additional 30 dyads. Qualitative interviews will be conducted separately with each veteran and CSO, and additional constructs such as feasibility, acceptability, barriers and facilitators to implementation, and changes in secure storage will be examined. This future research may provide a deeper understanding of the broader acceptability, feasibility, and satisfaction associated with a suicide prevention intervention focused on securing firearms and involving veterans and their CSOs. These data could be used to inform future implementation trials and inform the development of an implementation strategy. In the interim, the nature of the manual is summarized in the context of the urgency of suicide prevention in this at-risk population.

Conclusions

FFAST is a novel approach to veteran firearm suicide prevention. By involving CSOs and emphasizing mental health crisis preparedness between them and veterans, the dyad can work in association with HCPs to establish and exercise secure firearm storage practices as part of an at-home safety plan. Implementation of FFAST may be beneficial for all veterans, not only those who have been identified as being at high suicide risk.

T he US Census Bureau projects that the number of older adults (aged ≥ 65 years) will exceed 49 million by 2030, and an estimated 20% (nearly 10 million) of this population will experience cognitive or mental health disorders.^1,2 The mental health workforce is not equipped to address the specialized mental health care needs of many older adults.^2,3 For example, geriatric psychiatrists specialize in the diagnosis and treatment of mental illness and cognitive disorders in the later stages of life, but their numbers are few and declining. Only 33.5% of geriatric psychiatry fellowship training slots were filled from 2017 to 2021, and only 62 fellows trained during the 2021-2022 academic year.⁴ Board-certified geriatric psychiatrists also tend to be concentrated in larger, urban, academically-affiliated medical centers, often leaving rural areas and smaller facilities without access, including facilities in the Veterans Health Administration (VHA).⁵

The VHA has been optimizing access to specialty geriatric mental health services via regional and national virtual consultation services. Seven of 19 Veterans Integrated Service Network (VISN) Clinical Resource Hubs (CRHs) have geriatric mental health teams.⁶ These provide interdisciplinary geriatric mental telehealth services, including geriatric psychiatry, for older veterans with complex care needs.^7,8 Likewise, the VHA National Expert Consultation & Specialized Services-Mental Health (NEXCSS- MH, formerly known as the National Telemental Health Center) sponsors video teleconsultations with board-certified geriatric psychiatrists and an Ask the Expert email consultation program.

This article describes the Ask the Expert Geriatric Psychiatry email program (one of several similar programs at NEXCSS-MH), building upon a symposium presented at the American Association for Geriatric Psychiatry (AAGP) annual meeting in March 2022.⁹ The program was initiated in June 2021 as a result of discussions between the National Mental Health Director, Geriatric Mental Health in the VHA Office of Mental Health and Suicide Prevention (now known as the Office of Mental Health [OMH]), and National Telemental Health Center leadership. VHA board certified geriatric psychiatrists were recruited to serve as expert consultants and respond to email questions submitted by VHA clinicians regarding the psychiatric care of older adult veterans. The results of this program identify educational needs among clinical staff and may inform the development of program materials for a range of clinicians.

Program Description

The national geriatric mental health director recruited prospective experts and met with each to assess interest and qualifications, consulting with OMH psychiatrist leaders before making selections. Five experts were initially selected; 1 later stepped down and was replaced by another, who also stepped down. The experts were board certified in psychiatry and geriatric psychiatry and held a variety of local and national leadership positions, including geriatric psychiatry fellowship director, US Department of Veterans Affairs (VA) research and clinical leader, and various roles in the AAGP; some had received teaching awards.

Operations

The national geriatric mental health director announced the program in June 2021 to VHA mental health and geriatric program email groups with reminders sent every few months. The announcement included information about the types of questions appropriate to submit, including examples of general clinical management questions that did not share patient-specific protected health information, and clarified that experts would not be conducting chart reviews because the time required for detailed chart reviews was not feasible for volunteer experts to integrate into their otherwise full-time jobs at their respective VA medical centers. The announcement also included brief biographies of the experts.

The Figure describes the daily operations of the Ask the Expert Geriatric Psychiatry email consultation program. The NEXCSS- MH developed a Microsoft Outlook mailbox and group email address where clinicians from across the VHA could submit questions. The experts, as well as the national geriatric mental health director and NEXCSS-MH staff, had access to this mailbox to track and/or respond to questions. One expert volunteered to be the program’s primary mailbox coordinator. The coordinator checked the inbox daily and assigned each question to one of the experts on a rotating basis using the color-coding feature in Outlook. The other experts were advised to check the email account at least once weekly and reply to any assigned questions.

Responding to a question entailed first determining whether the question was appropriate for the service. For example, if a question requested a chart review, the expert replied that experts could not provide chart reviews and requested that the question be reframed. Next, the expert often needed to define a specific clinical question from the information provided, as email questions often touched upon several topics. The expert provided personalized advice on diagnostic testing, nonpharmacologic treatment strategies, and/or pharmacologic treatment options. Experts also often attached relevant guidelines or review articles. The goal was to provide a response within 7 business days.

All email responses included a disclaimer indicating that the program was not intended for urgent or immediate medical advice and that the information provided was for VHA clinician education purposes only. The disclaimer explained that email communication did not establish a doctor-patient relationship between the expert and a specific veteran and that, if desired, a request for a clinical consultation could be submitted on a specific case (ie, a video teleconsultation).

Methods for Reviewing Questions

Descriptive statistics, including frequencies, means, and minimum and maximum ranges, were used to capture the number of questions the program received, type of requester, and length of time prior to response for emailed questions.⁹ Conventional content analysis procedures were used between January and October 2024 to analyze clinicians’ questions.¹⁰ Four subject matter experts (3 geriatric psychiatrists and 1 geropsychologist) served as coders, assigned in groups of 2 to review questions. Each coder independently reviewed assigned questions and identified preliminary themes. Themes were reviewed and revised using an iterative process during regular team meetings with coders to clarify and confirm interpretations. Discrepancies were discussed within team meetings to achieve consensus.

Questions received. Between February 2022 and December 2023, the program received 101 email questions. Requesters included 39 physicians, 17 nurse practitioners or physician assistants, 15 social workers, 14 psychologists, 9 nurses, 5 pharmacists, 1 dietitian, and 1 who was undetermined. Experts responded to the questions an average of 6 days after receipt (range, < 1-19); 73 responses (72%) met the 7-day goal.

Iterative changes to coded themes were made during group discussions. Multiple clinical questions were often posed within the same email. Initially, some coders identified themes solely based on reported symptoms; others identified themes based on reported and/or potential diagnostic conditions attributed to the symptom(s) described within the email. For example, some coders selected a primary theme of behavioral and psychological symptoms of dementia (BPSD) only if a behavior contributing to distress in the veteran or others was described, while others selected this theme when any psychiatric symptom (eg, psychosis) was present in the context of dementia. The group identified 1 primary theme per question based on the main clinical symptom or main concern presented. Co-occurring diagnostic conditions highlighted in the email requests were included as secondary themes, and each question could have > 1 secondary theme.

The most frequent requests related to clinical symptoms included questions about agitated behaviors, sleep and/or nightmares, and depression symptoms (Table 1). Twenty-seven of 33 email requests on agitated behaviors were related to a dementia diagnosis, as were several questions about sleep/nightmares, depression, psychosis/mania, and anxiety. Many diagnostic conditions were described in the email requests (Table 2). The most frequent condition was dementia, followed by a medical condition, depressive disorder, posttraumatic stress disorder, and/or serious mental illness.

Request for Feedback. In February 2022, an email request was sent to the 64 clinicians who asked email questions from the start of the program in June 2021 through December 2021. A second request included 11 clinicians who asked questions from January through February 2022. These requests were sent as part of preparations for the symposium on the program presented at the AAGP annual meeting in March 2022.⁹ In May 2024, feedback was requested from 37 clinicians who submitted questions from May 1, 2023, through May 15, 2024.

Requests for feedback included 6 closed-ended and 1 open-ended question: (1) Did the answer you received help inform clinical practice? (2) Did you receive a timely response? (3) What type of information was useful to you in addressing your question (ie, direct/specific answer to a clinical scenario, guidelines, articles, VA resources)? (4) Do you have access to a geriatric psychiatrist at your facility? (5) Are you likely to use Ask the Expert Geriatric Psychiatry in the future? (6) Would you use a geriatric psychiatry teleconsultation service? (7) Share suggestions for improvement. Frequencies of response selection were obtained for each question. Text responses to the open-ended question asking for suggestions for improvement were reviewed and summarized.

Responses

Thirty users responded to the feedback request (27% response rate). Respondents considered the answers received extremely (n = 14; 47%) or very much (n = 12; 40%) helpful for their clinical practice. Twenty-three respondents (77%) felt an answer was provided promptly, 7 respondents (23%) felt the answer was not timely but still useful, and none felt that the answer was too late. Respondents reported that the most useful type of information in addressing their questions was a direct/specific answer to a clinical scenario (n = 27; 90%), followed by guidelines (n = 12; 40%), articles (n = 7; 23%), and VA resources (n = 4; 13%).

Sixteen respondents (53%) reported that they rarely had ready access to a geriatric psychiatrist at their facility, 3 (10%) had access sometimes, 4 (14%) had access usually, 3 (10%) had access regularly, and 3 (10%) never had access. Twenty-seven respondents (90%) indicated they would be very likely to use the service again. If geriatric psychiatry teleconsultation and/or e-consultation were offered, many respondents indicated they would be extremely (n = 10; 33%) or very (n = 12; 40%) likely to use teleconsultation and/or e-consultation.

Suggestions for improvement included supporting experts to perform chart reviews for email questions, developing a template or consult form, holding a biweekly drop-in meeting to present questions to and discuss cases with a panel of experts, and providing further help addressing complex decisional capacity issues, delirium, and care or placement for veterans with severe behavioral issues in a rural setting.

Discussion

Although many older adults experience cognitive and mental health disorders that may benefit from management by a geriatric psychiatrist, the number of trained geriatric psychiatrists available is insufficient to allow for direct care for each patient. The Ask the Expert Geriatric Psychiatry email consultation program is one aspect of a multicomponent strategy within the VHA to increase access to specialty geriatric mental health services for veterans. A key advantage of the program is that it is not resource intensive. Experts can participate voluntarily, providing timely feedback to clinicians around the country while continuing other duties at their respective VA medical centers. Email replies to the experts’ answers elicited positive feedback on the program, include: “I found this service to be extremely helpful and I have shared the information they sent me with several other coworkers!”, “It was great!”, and “I endorsed the service to our VISN Rehabilitation and Extended Care group.”

The coding of primary and secondary themes from 101 email questions that were retained revealed the range and relative frequencies of clinical and administrative topics with which clinicians needed help. The most common (33%) theme was agitated behaviors. Nearly half of the questions (48%) were related to underlying dementia, and 29% were related to a patient’s medical comorbidities. These findings suggest that the expertise of a geriatric psychiatrist is particularly relevant when caring for older patients experiencing BPSD or patients with complex, overlapping psychiatric and medical conditions.

Despite a 27% response rate, participant feedback has been helpful. The program reached its intended audience of clinicians in rural areas and at smaller facilities with 53% of requesters reporting they rarely had access to a geriatric psychiatrist. Suggestions for improvement indicated that some clinicians desired additional support, including chart reviews, meetings with experts, and a video teleconsultation service (available through NEXCSS-MH).

Many clinicians without training in specialty geriatric mental health may require help with complex clinical presentations. For example, 39 clinicians who submitted questions to the program were physicians. Accreditation Council for Graduate Medical Education program requirements for general psychiatry residency include 4 weeks of geriatric psychiatry.¹¹ The findings of this study suggest that this level of training may not be adequate to independently care for every patient who experiences dementia or multimorbidity. Several training and mentoring initiatives have been developed to address the professional development need for psychiatrists.^12-14

The need for geriatric workforce development is significant across health care, including other mental health professions.^15,16 The VHA Geriatric Scholars program trains rural primary care practitioners, psychologists, and psychiatrists.^17,18 Likewise, consultative geriatric specialty support for primary care practitioners in rural areas is provided via the Geriatric Research Education and Clinical Center Connect program.¹⁹ The Ask the Expert Geriatric Psychiatry email program is an additional economical model to support clinician educational development and provide rapid educational responses to inform patient care.

Ask the Expert received fewer email questions than anticipated. Enhanced optimization may require more frequent and widespread announcements about the program. Clinical staff may not be aware of the program due to an overload of email communications. Likewise, it may be challenging for busy clinicians to take the time to seek consultation or recognize a potential gap in their knowledge or skills. Had more questions been submitted, the 5 volunteer experts may have had more difficulty addressing the demand. Feedback from this project may inform development of a frequently asked questions document to share with VHA teams and a drop-in office hour to pose clinical questions of geriatric psychiatry experts, as recommended by a clinician who participated in the program.

Limitations

Not all requesters were sent a request for feedback, and the response rate for the request for feedback was only 27%. As the program has evolved, it began sending a request for feedback immediately after answering each question, which may increase the odds of response. The goal of experts answering questions within 7 business days was met 72% of the time, likely an artifact of experts integrating question answering with many other duties. The mailbox coordinator has since provided email prompts to experts immediately upon being assigned a question with the goal of improving timeliness. The program did not include chart reviews or patient consultations, as neither was feasible for volunteer experts. The email consultation service is a single component of virtual consultative specialty geriatric mental health services within the VHA, including video consultations via NEXCSS-MH and regional geriatric mental health teams.

Conclusions

The need for specialty geriatric mental health services is increasing in the VHA and across the US. However, there are too few board-certified geriatric psychiatrists to provide direct patient care to all older adults with cognitive and mental health disorders. The VHA has leveraged telehealth to improve access to geriatric mental health care. The VHA Ask the Expert Geriatric Psychiatry email consultation program is a low-resource service which provides rapid feedback to clinicians nationwide on challenging clinical scenarios, many of which are dementia-related. Most users of the service who responded to requests for feedback reported that answers to their questions were helpful and timely. The email consultation program should continue to be supplemented by more comprehensive geriatric telemental health services for particularly complex cases to meet the needs of older veterans.

T he US Census Bureau projects that the number of older adults (aged ≥ 65 years) will exceed 49 million by 2030, and an estimated 20% (nearly 10 million) of this population will experience cognitive or mental health disorders.^1,2 The mental health workforce is not equipped to address the specialized mental health care needs of many older adults.^2,3 For example, geriatric psychiatrists specialize in the diagnosis and treatment of mental illness and cognitive disorders in the later stages of life, but their numbers are few and declining. Only 33.5% of geriatric psychiatry fellowship training slots were filled from 2017 to 2021, and only 62 fellows trained during the 2021-2022 academic year.⁴ Board-certified geriatric psychiatrists also tend to be concentrated in larger, urban, academically-affiliated medical centers, often leaving rural areas and smaller facilities without access, including facilities in the Veterans Health Administration (VHA).⁵

The VHA has been optimizing access to specialty geriatric mental health services via regional and national virtual consultation services. Seven of 19 Veterans Integrated Service Network (VISN) Clinical Resource Hubs (CRHs) have geriatric mental health teams.⁶ These provide interdisciplinary geriatric mental telehealth services, including geriatric psychiatry, for older veterans with complex care needs.^7,8 Likewise, the VHA National Expert Consultation & Specialized Services-Mental Health (NEXCSS- MH, formerly known as the National Telemental Health Center) sponsors video teleconsultations with board-certified geriatric psychiatrists and an Ask the Expert email consultation program.

This article describes the Ask the Expert Geriatric Psychiatry email program (one of several similar programs at NEXCSS-MH), building upon a symposium presented at the American Association for Geriatric Psychiatry (AAGP) annual meeting in March 2022.⁹ The program was initiated in June 2021 as a result of discussions between the National Mental Health Director, Geriatric Mental Health in the VHA Office of Mental Health and Suicide Prevention (now known as the Office of Mental Health [OMH]), and National Telemental Health Center leadership. VHA board certified geriatric psychiatrists were recruited to serve as expert consultants and respond to email questions submitted by VHA clinicians regarding the psychiatric care of older adult veterans. The results of this program identify educational needs among clinical staff and may inform the development of program materials for a range of clinicians.

Program Description

The national geriatric mental health director recruited prospective experts and met with each to assess interest and qualifications, consulting with OMH psychiatrist leaders before making selections. Five experts were initially selected; 1 later stepped down and was replaced by another, who also stepped down. The experts were board certified in psychiatry and geriatric psychiatry and held a variety of local and national leadership positions, including geriatric psychiatry fellowship director, US Department of Veterans Affairs (VA) research and clinical leader, and various roles in the AAGP; some had received teaching awards.

Operations

The national geriatric mental health director announced the program in June 2021 to VHA mental health and geriatric program email groups with reminders sent every few months. The announcement included information about the types of questions appropriate to submit, including examples of general clinical management questions that did not share patient-specific protected health information, and clarified that experts would not be conducting chart reviews because the time required for detailed chart reviews was not feasible for volunteer experts to integrate into their otherwise full-time jobs at their respective VA medical centers. The announcement also included brief biographies of the experts.

The Figure describes the daily operations of the Ask the Expert Geriatric Psychiatry email consultation program. The NEXCSS- MH developed a Microsoft Outlook mailbox and group email address where clinicians from across the VHA could submit questions. The experts, as well as the national geriatric mental health director and NEXCSS-MH staff, had access to this mailbox to track and/or respond to questions. One expert volunteered to be the program’s primary mailbox coordinator. The coordinator checked the inbox daily and assigned each question to one of the experts on a rotating basis using the color-coding feature in Outlook. The other experts were advised to check the email account at least once weekly and reply to any assigned questions.

Responding to a question entailed first determining whether the question was appropriate for the service. For example, if a question requested a chart review, the expert replied that experts could not provide chart reviews and requested that the question be reframed. Next, the expert often needed to define a specific clinical question from the information provided, as email questions often touched upon several topics. The expert provided personalized advice on diagnostic testing, nonpharmacologic treatment strategies, and/or pharmacologic treatment options. Experts also often attached relevant guidelines or review articles. The goal was to provide a response within 7 business days.

All email responses included a disclaimer indicating that the program was not intended for urgent or immediate medical advice and that the information provided was for VHA clinician education purposes only. The disclaimer explained that email communication did not establish a doctor-patient relationship between the expert and a specific veteran and that, if desired, a request for a clinical consultation could be submitted on a specific case (ie, a video teleconsultation).

Methods for Reviewing Questions

Descriptive statistics, including frequencies, means, and minimum and maximum ranges, were used to capture the number of questions the program received, type of requester, and length of time prior to response for emailed questions.⁹ Conventional content analysis procedures were used between January and October 2024 to analyze clinicians’ questions.¹⁰ Four subject matter experts (3 geriatric psychiatrists and 1 geropsychologist) served as coders, assigned in groups of 2 to review questions. Each coder independently reviewed assigned questions and identified preliminary themes. Themes were reviewed and revised using an iterative process during regular team meetings with coders to clarify and confirm interpretations. Discrepancies were discussed within team meetings to achieve consensus.

Questions received. Between February 2022 and December 2023, the program received 101 email questions. Requesters included 39 physicians, 17 nurse practitioners or physician assistants, 15 social workers, 14 psychologists, 9 nurses, 5 pharmacists, 1 dietitian, and 1 who was undetermined. Experts responded to the questions an average of 6 days after receipt (range, < 1-19); 73 responses (72%) met the 7-day goal.

Iterative changes to coded themes were made during group discussions. Multiple clinical questions were often posed within the same email. Initially, some coders identified themes solely based on reported symptoms; others identified themes based on reported and/or potential diagnostic conditions attributed to the symptom(s) described within the email. For example, some coders selected a primary theme of behavioral and psychological symptoms of dementia (BPSD) only if a behavior contributing to distress in the veteran or others was described, while others selected this theme when any psychiatric symptom (eg, psychosis) was present in the context of dementia. The group identified 1 primary theme per question based on the main clinical symptom or main concern presented. Co-occurring diagnostic conditions highlighted in the email requests were included as secondary themes, and each question could have > 1 secondary theme.

The most frequent requests related to clinical symptoms included questions about agitated behaviors, sleep and/or nightmares, and depression symptoms (Table 1). Twenty-seven of 33 email requests on agitated behaviors were related to a dementia diagnosis, as were several questions about sleep/nightmares, depression, psychosis/mania, and anxiety. Many diagnostic conditions were described in the email requests (Table 2). The most frequent condition was dementia, followed by a medical condition, depressive disorder, posttraumatic stress disorder, and/or serious mental illness.

Request for Feedback. In February 2022, an email request was sent to the 64 clinicians who asked email questions from the start of the program in June 2021 through December 2021. A second request included 11 clinicians who asked questions from January through February 2022. These requests were sent as part of preparations for the symposium on the program presented at the AAGP annual meeting in March 2022.⁹ In May 2024, feedback was requested from 37 clinicians who submitted questions from May 1, 2023, through May 15, 2024.

Requests for feedback included 6 closed-ended and 1 open-ended question: (1) Did the answer you received help inform clinical practice? (2) Did you receive a timely response? (3) What type of information was useful to you in addressing your question (ie, direct/specific answer to a clinical scenario, guidelines, articles, VA resources)? (4) Do you have access to a geriatric psychiatrist at your facility? (5) Are you likely to use Ask the Expert Geriatric Psychiatry in the future? (6) Would you use a geriatric psychiatry teleconsultation service? (7) Share suggestions for improvement. Frequencies of response selection were obtained for each question. Text responses to the open-ended question asking for suggestions for improvement were reviewed and summarized.

Responses

Thirty users responded to the feedback request (27% response rate). Respondents considered the answers received extremely (n = 14; 47%) or very much (n = 12; 40%) helpful for their clinical practice. Twenty-three respondents (77%) felt an answer was provided promptly, 7 respondents (23%) felt the answer was not timely but still useful, and none felt that the answer was too late. Respondents reported that the most useful type of information in addressing their questions was a direct/specific answer to a clinical scenario (n = 27; 90%), followed by guidelines (n = 12; 40%), articles (n = 7; 23%), and VA resources (n = 4; 13%).

Sixteen respondents (53%) reported that they rarely had ready access to a geriatric psychiatrist at their facility, 3 (10%) had access sometimes, 4 (14%) had access usually, 3 (10%) had access regularly, and 3 (10%) never had access. Twenty-seven respondents (90%) indicated they would be very likely to use the service again. If geriatric psychiatry teleconsultation and/or e-consultation were offered, many respondents indicated they would be extremely (n = 10; 33%) or very (n = 12; 40%) likely to use teleconsultation and/or e-consultation.

Suggestions for improvement included supporting experts to perform chart reviews for email questions, developing a template or consult form, holding a biweekly drop-in meeting to present questions to and discuss cases with a panel of experts, and providing further help addressing complex decisional capacity issues, delirium, and care or placement for veterans with severe behavioral issues in a rural setting.

Discussion

Although many older adults experience cognitive and mental health disorders that may benefit from management by a geriatric psychiatrist, the number of trained geriatric psychiatrists available is insufficient to allow for direct care for each patient. The Ask the Expert Geriatric Psychiatry email consultation program is one aspect of a multicomponent strategy within the VHA to increase access to specialty geriatric mental health services for veterans. A key advantage of the program is that it is not resource intensive. Experts can participate voluntarily, providing timely feedback to clinicians around the country while continuing other duties at their respective VA medical centers. Email replies to the experts’ answers elicited positive feedback on the program, include: “I found this service to be extremely helpful and I have shared the information they sent me with several other coworkers!”, “It was great!”, and “I endorsed the service to our VISN Rehabilitation and Extended Care group.”

The coding of primary and secondary themes from 101 email questions that were retained revealed the range and relative frequencies of clinical and administrative topics with which clinicians needed help. The most common (33%) theme was agitated behaviors. Nearly half of the questions (48%) were related to underlying dementia, and 29% were related to a patient’s medical comorbidities. These findings suggest that the expertise of a geriatric psychiatrist is particularly relevant when caring for older patients experiencing BPSD or patients with complex, overlapping psychiatric and medical conditions.

Despite a 27% response rate, participant feedback has been helpful. The program reached its intended audience of clinicians in rural areas and at smaller facilities with 53% of requesters reporting they rarely had access to a geriatric psychiatrist. Suggestions for improvement indicated that some clinicians desired additional support, including chart reviews, meetings with experts, and a video teleconsultation service (available through NEXCSS-MH).

Many clinicians without training in specialty geriatric mental health may require help with complex clinical presentations. For example, 39 clinicians who submitted questions to the program were physicians. Accreditation Council for Graduate Medical Education program requirements for general psychiatry residency include 4 weeks of geriatric psychiatry.¹¹ The findings of this study suggest that this level of training may not be adequate to independently care for every patient who experiences dementia or multimorbidity. Several training and mentoring initiatives have been developed to address the professional development need for psychiatrists.^12-14

The need for geriatric workforce development is significant across health care, including other mental health professions.^15,16 The VHA Geriatric Scholars program trains rural primary care practitioners, psychologists, and psychiatrists.^17,18 Likewise, consultative geriatric specialty support for primary care practitioners in rural areas is provided via the Geriatric Research Education and Clinical Center Connect program.¹⁹ The Ask the Expert Geriatric Psychiatry email program is an additional economical model to support clinician educational development and provide rapid educational responses to inform patient care.

Ask the Expert received fewer email questions than anticipated. Enhanced optimization may require more frequent and widespread announcements about the program. Clinical staff may not be aware of the program due to an overload of email communications. Likewise, it may be challenging for busy clinicians to take the time to seek consultation or recognize a potential gap in their knowledge or skills. Had more questions been submitted, the 5 volunteer experts may have had more difficulty addressing the demand. Feedback from this project may inform development of a frequently asked questions document to share with VHA teams and a drop-in office hour to pose clinical questions of geriatric psychiatry experts, as recommended by a clinician who participated in the program.

Limitations

Not all requesters were sent a request for feedback, and the response rate for the request for feedback was only 27%. As the program has evolved, it began sending a request for feedback immediately after answering each question, which may increase the odds of response. The goal of experts answering questions within 7 business days was met 72% of the time, likely an artifact of experts integrating question answering with many other duties. The mailbox coordinator has since provided email prompts to experts immediately upon being assigned a question with the goal of improving timeliness. The program did not include chart reviews or patient consultations, as neither was feasible for volunteer experts. The email consultation service is a single component of virtual consultative specialty geriatric mental health services within the VHA, including video consultations via NEXCSS-MH and regional geriatric mental health teams.

Conclusions

The need for specialty geriatric mental health services is increasing in the VHA and across the US. However, there are too few board-certified geriatric psychiatrists to provide direct patient care to all older adults with cognitive and mental health disorders. The VHA has leveraged telehealth to improve access to geriatric mental health care. The VHA Ask the Expert Geriatric Psychiatry email consultation program is a low-resource service which provides rapid feedback to clinicians nationwide on challenging clinical scenarios, many of which are dementia-related. Most users of the service who responded to requests for feedback reported that answers to their questions were helpful and timely. The email consultation program should continue to be supplemented by more comprehensive geriatric telemental health services for particularly complex cases to meet the needs of older veterans.

T he US Census Bureau projects that the number of older adults (aged ≥ 65 years) will exceed 49 million by 2030, and an estimated 20% (nearly 10 million) of this population will experience cognitive or mental health disorders.^1,2 The mental health workforce is not equipped to address the specialized mental health care needs of many older adults.^2,3 For example, geriatric psychiatrists specialize in the diagnosis and treatment of mental illness and cognitive disorders in the later stages of life, but their numbers are few and declining. Only 33.5% of geriatric psychiatry fellowship training slots were filled from 2017 to 2021, and only 62 fellows trained during the 2021-2022 academic year.⁴ Board-certified geriatric psychiatrists also tend to be concentrated in larger, urban, academically-affiliated medical centers, often leaving rural areas and smaller facilities without access, including facilities in the Veterans Health Administration (VHA).⁵

The VHA has been optimizing access to specialty geriatric mental health services via regional and national virtual consultation services. Seven of 19 Veterans Integrated Service Network (VISN) Clinical Resource Hubs (CRHs) have geriatric mental health teams.⁶ These provide interdisciplinary geriatric mental telehealth services, including geriatric psychiatry, for older veterans with complex care needs.^7,8 Likewise, the VHA National Expert Consultation & Specialized Services-Mental Health (NEXCSS- MH, formerly known as the National Telemental Health Center) sponsors video teleconsultations with board-certified geriatric psychiatrists and an Ask the Expert email consultation program.

This article describes the Ask the Expert Geriatric Psychiatry email program (one of several similar programs at NEXCSS-MH), building upon a symposium presented at the American Association for Geriatric Psychiatry (AAGP) annual meeting in March 2022.⁹ The program was initiated in June 2021 as a result of discussions between the National Mental Health Director, Geriatric Mental Health in the VHA Office of Mental Health and Suicide Prevention (now known as the Office of Mental Health [OMH]), and National Telemental Health Center leadership. VHA board certified geriatric psychiatrists were recruited to serve as expert consultants and respond to email questions submitted by VHA clinicians regarding the psychiatric care of older adult veterans. The results of this program identify educational needs among clinical staff and may inform the development of program materials for a range of clinicians.

Program Description

The national geriatric mental health director recruited prospective experts and met with each to assess interest and qualifications, consulting with OMH psychiatrist leaders before making selections. Five experts were initially selected; 1 later stepped down and was replaced by another, who also stepped down. The experts were board certified in psychiatry and geriatric psychiatry and held a variety of local and national leadership positions, including geriatric psychiatry fellowship director, US Department of Veterans Affairs (VA) research and clinical leader, and various roles in the AAGP; some had received teaching awards.

Operations

The national geriatric mental health director announced the program in June 2021 to VHA mental health and geriatric program email groups with reminders sent every few months. The announcement included information about the types of questions appropriate to submit, including examples of general clinical management questions that did not share patient-specific protected health information, and clarified that experts would not be conducting chart reviews because the time required for detailed chart reviews was not feasible for volunteer experts to integrate into their otherwise full-time jobs at their respective VA medical centers. The announcement also included brief biographies of the experts.

The Figure describes the daily operations of the Ask the Expert Geriatric Psychiatry email consultation program. The NEXCSS- MH developed a Microsoft Outlook mailbox and group email address where clinicians from across the VHA could submit questions. The experts, as well as the national geriatric mental health director and NEXCSS-MH staff, had access to this mailbox to track and/or respond to questions. One expert volunteered to be the program’s primary mailbox coordinator. The coordinator checked the inbox daily and assigned each question to one of the experts on a rotating basis using the color-coding feature in Outlook. The other experts were advised to check the email account at least once weekly and reply to any assigned questions.

Responding to a question entailed first determining whether the question was appropriate for the service. For example, if a question requested a chart review, the expert replied that experts could not provide chart reviews and requested that the question be reframed. Next, the expert often needed to define a specific clinical question from the information provided, as email questions often touched upon several topics. The expert provided personalized advice on diagnostic testing, nonpharmacologic treatment strategies, and/or pharmacologic treatment options. Experts also often attached relevant guidelines or review articles. The goal was to provide a response within 7 business days.

All email responses included a disclaimer indicating that the program was not intended for urgent or immediate medical advice and that the information provided was for VHA clinician education purposes only. The disclaimer explained that email communication did not establish a doctor-patient relationship between the expert and a specific veteran and that, if desired, a request for a clinical consultation could be submitted on a specific case (ie, a video teleconsultation).

Methods for Reviewing Questions

Descriptive statistics, including frequencies, means, and minimum and maximum ranges, were used to capture the number of questions the program received, type of requester, and length of time prior to response for emailed questions.⁹ Conventional content analysis procedures were used between January and October 2024 to analyze clinicians’ questions.¹⁰ Four subject matter experts (3 geriatric psychiatrists and 1 geropsychologist) served as coders, assigned in groups of 2 to review questions. Each coder independently reviewed assigned questions and identified preliminary themes. Themes were reviewed and revised using an iterative process during regular team meetings with coders to clarify and confirm interpretations. Discrepancies were discussed within team meetings to achieve consensus.

Questions received. Between February 2022 and December 2023, the program received 101 email questions. Requesters included 39 physicians, 17 nurse practitioners or physician assistants, 15 social workers, 14 psychologists, 9 nurses, 5 pharmacists, 1 dietitian, and 1 who was undetermined. Experts responded to the questions an average of 6 days after receipt (range, < 1-19); 73 responses (72%) met the 7-day goal.

Iterative changes to coded themes were made during group discussions. Multiple clinical questions were often posed within the same email. Initially, some coders identified themes solely based on reported symptoms; others identified themes based on reported and/or potential diagnostic conditions attributed to the symptom(s) described within the email. For example, some coders selected a primary theme of behavioral and psychological symptoms of dementia (BPSD) only if a behavior contributing to distress in the veteran or others was described, while others selected this theme when any psychiatric symptom (eg, psychosis) was present in the context of dementia. The group identified 1 primary theme per question based on the main clinical symptom or main concern presented. Co-occurring diagnostic conditions highlighted in the email requests were included as secondary themes, and each question could have > 1 secondary theme.

The most frequent requests related to clinical symptoms included questions about agitated behaviors, sleep and/or nightmares, and depression symptoms (Table 1). Twenty-seven of 33 email requests on agitated behaviors were related to a dementia diagnosis, as were several questions about sleep/nightmares, depression, psychosis/mania, and anxiety. Many diagnostic conditions were described in the email requests (Table 2). The most frequent condition was dementia, followed by a medical condition, depressive disorder, posttraumatic stress disorder, and/or serious mental illness.

Request for Feedback. In February 2022, an email request was sent to the 64 clinicians who asked email questions from the start of the program in June 2021 through December 2021. A second request included 11 clinicians who asked questions from January through February 2022. These requests were sent as part of preparations for the symposium on the program presented at the AAGP annual meeting in March 2022.⁹ In May 2024, feedback was requested from 37 clinicians who submitted questions from May 1, 2023, through May 15, 2024.

Requests for feedback included 6 closed-ended and 1 open-ended question: (1) Did the answer you received help inform clinical practice? (2) Did you receive a timely response? (3) What type of information was useful to you in addressing your question (ie, direct/specific answer to a clinical scenario, guidelines, articles, VA resources)? (4) Do you have access to a geriatric psychiatrist at your facility? (5) Are you likely to use Ask the Expert Geriatric Psychiatry in the future? (6) Would you use a geriatric psychiatry teleconsultation service? (7) Share suggestions for improvement. Frequencies of response selection were obtained for each question. Text responses to the open-ended question asking for suggestions for improvement were reviewed and summarized.

Responses

Thirty users responded to the feedback request (27% response rate). Respondents considered the answers received extremely (n = 14; 47%) or very much (n = 12; 40%) helpful for their clinical practice. Twenty-three respondents (77%) felt an answer was provided promptly, 7 respondents (23%) felt the answer was not timely but still useful, and none felt that the answer was too late. Respondents reported that the most useful type of information in addressing their questions was a direct/specific answer to a clinical scenario (n = 27; 90%), followed by guidelines (n = 12; 40%), articles (n = 7; 23%), and VA resources (n = 4; 13%).

Sixteen respondents (53%) reported that they rarely had ready access to a geriatric psychiatrist at their facility, 3 (10%) had access sometimes, 4 (14%) had access usually, 3 (10%) had access regularly, and 3 (10%) never had access. Twenty-seven respondents (90%) indicated they would be very likely to use the service again. If geriatric psychiatry teleconsultation and/or e-consultation were offered, many respondents indicated they would be extremely (n = 10; 33%) or very (n = 12; 40%) likely to use teleconsultation and/or e-consultation.

Suggestions for improvement included supporting experts to perform chart reviews for email questions, developing a template or consult form, holding a biweekly drop-in meeting to present questions to and discuss cases with a panel of experts, and providing further help addressing complex decisional capacity issues, delirium, and care or placement for veterans with severe behavioral issues in a rural setting.

Discussion

Although many older adults experience cognitive and mental health disorders that may benefit from management by a geriatric psychiatrist, the number of trained geriatric psychiatrists available is insufficient to allow for direct care for each patient. The Ask the Expert Geriatric Psychiatry email consultation program is one aspect of a multicomponent strategy within the VHA to increase access to specialty geriatric mental health services for veterans. A key advantage of the program is that it is not resource intensive. Experts can participate voluntarily, providing timely feedback to clinicians around the country while continuing other duties at their respective VA medical centers. Email replies to the experts’ answers elicited positive feedback on the program, include: “I found this service to be extremely helpful and I have shared the information they sent me with several other coworkers!”, “It was great!”, and “I endorsed the service to our VISN Rehabilitation and Extended Care group.”

The coding of primary and secondary themes from 101 email questions that were retained revealed the range and relative frequencies of clinical and administrative topics with which clinicians needed help. The most common (33%) theme was agitated behaviors. Nearly half of the questions (48%) were related to underlying dementia, and 29% were related to a patient’s medical comorbidities. These findings suggest that the expertise of a geriatric psychiatrist is particularly relevant when caring for older patients experiencing BPSD or patients with complex, overlapping psychiatric and medical conditions.

Despite a 27% response rate, participant feedback has been helpful. The program reached its intended audience of clinicians in rural areas and at smaller facilities with 53% of requesters reporting they rarely had access to a geriatric psychiatrist. Suggestions for improvement indicated that some clinicians desired additional support, including chart reviews, meetings with experts, and a video teleconsultation service (available through NEXCSS-MH).

Many clinicians without training in specialty geriatric mental health may require help with complex clinical presentations. For example, 39 clinicians who submitted questions to the program were physicians. Accreditation Council for Graduate Medical Education program requirements for general psychiatry residency include 4 weeks of geriatric psychiatry.¹¹ The findings of this study suggest that this level of training may not be adequate to independently care for every patient who experiences dementia or multimorbidity. Several training and mentoring initiatives have been developed to address the professional development need for psychiatrists.^12-14

The need for geriatric workforce development is significant across health care, including other mental health professions.^15,16 The VHA Geriatric Scholars program trains rural primary care practitioners, psychologists, and psychiatrists.^17,18 Likewise, consultative geriatric specialty support for primary care practitioners in rural areas is provided via the Geriatric Research Education and Clinical Center Connect program.¹⁹ The Ask the Expert Geriatric Psychiatry email program is an additional economical model to support clinician educational development and provide rapid educational responses to inform patient care.

Ask the Expert received fewer email questions than anticipated. Enhanced optimization may require more frequent and widespread announcements about the program. Clinical staff may not be aware of the program due to an overload of email communications. Likewise, it may be challenging for busy clinicians to take the time to seek consultation or recognize a potential gap in their knowledge or skills. Had more questions been submitted, the 5 volunteer experts may have had more difficulty addressing the demand. Feedback from this project may inform development of a frequently asked questions document to share with VHA teams and a drop-in office hour to pose clinical questions of geriatric psychiatry experts, as recommended by a clinician who participated in the program.

Limitations

Not all requesters were sent a request for feedback, and the response rate for the request for feedback was only 27%. As the program has evolved, it began sending a request for feedback immediately after answering each question, which may increase the odds of response. The goal of experts answering questions within 7 business days was met 72% of the time, likely an artifact of experts integrating question answering with many other duties. The mailbox coordinator has since provided email prompts to experts immediately upon being assigned a question with the goal of improving timeliness. The program did not include chart reviews or patient consultations, as neither was feasible for volunteer experts. The email consultation service is a single component of virtual consultative specialty geriatric mental health services within the VHA, including video consultations via NEXCSS-MH and regional geriatric mental health teams.

Conclusions

The need for specialty geriatric mental health services is increasing in the VHA and across the US. However, there are too few board-certified geriatric psychiatrists to provide direct patient care to all older adults with cognitive and mental health disorders. The VHA has leveraged telehealth to improve access to geriatric mental health care. The VHA Ask the Expert Geriatric Psychiatry email consultation program is a low-resource service which provides rapid feedback to clinicians nationwide on challenging clinical scenarios, many of which are dementia-related. Most users of the service who responded to requests for feedback reported that answers to their questions were helpful and timely. The email consultation program should continue to be supplemented by more comprehensive geriatric telemental health services for particularly complex cases to meet the needs of older veterans.

Cognitive behavioral therapy (CBT) is supported as an effective treatment for depression by clinical practice guidelines from the US Department of Veterans Affairs (VA) and US Department of Defense.^1-3 Despite research supporting the use of evidence-based CBT for anxiety, mood, and emotional disorders, patient access to these interventions is limited.⁴ One barrier to CBT for anxiety, mood, and emotional disorders is the traditional use of single-disorder protocols (SDPs) to separately treat each disorder (eg, major depression, generalized anxiety disorder, panic disorder).

Use of SDPs places a high burden on clinicians, requiring them to learn and competently implement multiple different manualized interventions for each anxiety, mood, or emotional disorder encountered in practice.⁴ It is common for individuals who present with an anxiety, mood, or emotional disorder to experience co-occurring disorders.⁵ Traditional SDP-based CBT may require multiple SDPs to address co-occurring disorders, extending time in treatment and increasing training burden. There is evidence that even when an SDP is used, co-occurring difficulties may decrease in intensity.⁶ Thus, evidence-based CBT for 1 presenting concern may positively affect co-occurring difficulties that are not the primary treatment target.⁶

Unified Protocol (UP) is a transdiagnostic CBT intervention for anxiety and mood disorders. UP targets emotional experiences (eg, negative affect and emotional distress) that are present in multiple anxiety, depression, and emotional disorders. UP is organized into 8 modules and uses interventions present in other CBT SDPs, such as teaching objective assessment methods to clients, motivational enhancement and goal setting, emotion psychoeducation, mindful awareness, cognitive reframing, and exposure principles as mechanisms of change (Table 1).⁴ UP is an ideal intervention for addressing a number of anxiety, mood, and emotional disorders as well as addressing cooccurring disorders within the same course of treatment.

UP has been compared to SDPs; studies illustrate that UP is equivalent to SDPs at the end of treatment and at 6 months posttreatment.^4,7 Additionally, patients who received UP experienced symptom reduction for multiple co-occurring disorders.⁷ Furthermore, patients were less likely to drop out of UP when compared with SDPs.⁴

Studies have reported positive impacts on affective disorders with UP. A 2019 meta-analysis found significant reductions in depression and anxiety-related disorders.⁸ Additionally, UP has been shown to be effective when delivered in person and via telehealth.^9,10 UP has also been successfully used in veterans.¹¹ While traditional models of UP (1-2 sessions weekly) have a lower dropout rate than SDPs, UP and CBT dropout rates still leave room for improvement.^12-14 Specifically, rates of attrition from SDP CBT and transdiagnostic CBT protocols range from 9% to 35%, and dropout reduces the likelihood of attaining a full therapeutic dose of any course of CBT. Notably, accelerated delivery of CBT (ie, ≥ 3 sessions/wk) has been shown to reduce risk of dropout.^14,15

Veterans are at increased risk for suicide, and anxiety and mood disorders are associated with increased risk of suicide attempt and death.^16,17 Very few veterans who could benefit from high-quality CBT interventions, whether SDP or UP, are able to access them. Only 6.3% of veterans received ≥1 session of an evidence-based CBT SDP for posttraumatic stress disorder (PTSD) at 6 clinics evaluated in 2010.¹⁸

We identified 2 strategies to increase access to and completion of an evidence-based CBT course. First, the use of transdiagnostic UP instead of SDPs reduced burden on therapists and allowed them to address presenting and co-occurring disorders within the same course of care. Second, the use of an accelerated model of service delivery reduced dropout risk. Training clinicians to deliver UP is efficient and cost-effective, considering clinicians use core strategies that can be tailored and flexibly applied to a range of emotional difficulties. Thus, implementing UP may decrease barriers to receiving an optimal dose of an evidence-based CBT delivered with fidelity in a time- and cost-efficient manner.⁴ Two studies have found no evidence of differences in outcomes between UP and SDPs, suggesting that training and supervising clinicians in a single transdiagnostic UP intervention may prepare them to treat heterogeneous and co-occurring anxiety, mood, and emotional disorders with less burden than learning multiple SDPs.^7,19

Delivering UP in an accelerated or massed format (≥4 sessions/wk) instead of the traditional spaced model (1 session/wk) has empirical support but has not been widely implemented. This approach, sometimes referred to as a UP-intensive outpatient program (UP-IOP) or UP-intensive outpatient track (UP-IOT), has been shown to be feasible, acceptable, and effective, with increased completion rates compared with traditional UP delivery (1-2 sessions/wk).^20-22

Ragsdale et al describe a 2-week IOP with multiple treatment tracks, including a general track.²⁰ The general track includes massed UP and additional standard services, including case management, wellness services, family services, and a single session effective behaviors group. Additional augmentation services are available when clinically indicated (eg, repetitive transcranial magnetic stimulation, transcranial direct current stimulation, psychoeducation, motivational interviewing, relapse prevention). In other words, this was an UP-IOP.²⁰

Thompson-Brenner et al described a successful implementation of massed UP applied in intensive treatment settings, such as residential and day-hospital programs, for eating disorders. Patients reported improvements in 3 UP targets: experiential avoidance, mindfulness, and anxiety.²¹ Watkins et al evaluated a 2-week IOP using CBT for comorbid substance use and mental health disorders, including prolonged exposure, UP, and relapse prevention for substance use disorders. Participants were post-9/11 veterans and activeduty personnel. Results indicated that UP reduced PTSD and depressive symptoms following treatment. Furthermore, the retention rate (91%) was higher than retention in outpatient treatment (39%-65%), supporting the IOP model as a strategy to reduce dropout.²²

Massed psychotherapy has been extended to IOP programs for PTSD treatment within the Veterans Health Administration (VHA). Yamokoski et al found that patients who completed an IOP that included massed CBT for PTSD had high retention, high satisfaction, and significant reduction in self-reported co-occurring depression symptoms. The authors also found that this model of care could be implemented and sustained within a VHA facility using minimal staffing resources.²³

The UP-IOP models described by Ragsdale et al and Watkins et al included massed UP as the primary evidence-based practice (EBP) with adjunctive groups (eg, wellness and educational groups and access to complementary interventions such as mindfulness and yoga); they found that adding these groups increased retention and patient- reported satisfaction (ie, UP-IOP).^20,22 The addition of wellness education alongside a primary EBP aligns with the VHA focus on whole health well-being and wellness. This includes understanding factors that motivate a patient toward health and well-being, providing health education, and offering access to complementary interventions such as mindfulness.²⁴

Dryden et al described the whole health transformation within VHA as a proactive approach to addressing both employee and patient wellness. Their research found that the whole health model promoted wellbeing in patients and staff and these improvements were sustained during the COVID-19 pandemic. Dryden et al also noted that virtual technologies facilitated continued whole health implementation.²⁵

The literature illustrates that (1) massed UP can be delivered with complementary education and wellness offerings that may increase retention and satisfaction by enriching treatment (eg, delivering UP-IOP); (2) whole health, including wellness education and complementary interventions (eg, mindfulness, motivational enhancement), promotes well-being in patients and clinicians; and (3) whole health education and complementary interventions can be delivered virtually.

IVET

Health Care Need

Veterans Affairs Pacific Islands Health Care System (VAPIHCS) provides medical services to veterans in the Hawaiian Islands, American Samoa, Guam, and Saipan spanning nearly 4000 miles across the Pacific Ocean. Prior to implementation of this program, veterans who received care at VAPIHCS had little to no access to UP in outpatient settings and no access to UP in residential settings. Access to UP depended on the presence of a therapist trained in UP within a given clinic and was geographically limited to the location of the UP-trained therapist. The limited outpatient access to UP was restricted to the traditional UP delivery model (eg, about 1 session/wk); thus, there was no access to accelerated UP for veterans served at VAPIHCS. In the fiscal year prior to implementation of the massed UP program, > 1000 VAPIHCS veterans had been diagnosed with obsessive-compulsive related disorder or anxiety. A massed-UP program with weekly rolling admissions would support access to UP for more veterans over 12 months, and the virtual treatment modality would reduce barriers for diverse and underserved veterans, making care more equitable and inclusive.

Successful implementation and sustainment of an EBP prompted the establishment of this UP program. In 2022, VAPIHCS launched the Intensive Virtual Evidence- Based Psychotherapy Team (iVET) for treatment of PTSD. This clinic and associated EBP project demonstrated that massed (≥3 individual sessions/wk) of prolonged exposure (PE) therapy, delivered virtually to a geographically diverse veteran population with PTSD, resulted in significant reductions in PTSD, depression, and anxiety symptoms and substance use risk factors, with very high retention rates. The iVET for PTSD program was feasible, acceptable, and effective, with veterans reporting significant improvement in quality of life and high satisfaction with their mental health services.¹⁵ Given the known benefits of transdiagnostic UP treatment (vs SDPs), the need for accelerated UP, and success with accelerated PE, our goal was to spread the EBP of massed (≥3 sessions/ wk) virtual psychotherapy to other presenting problems (eg, anxiety disorders with or without co-occurring unipolar depression) using transdiagnostic UP.

Program Description

The program implemented within outpatient mental health services at VAPIHCS was iVET for the treatment of anxiety with or without co-occurring depression. The program model consists of an accelerated course of UP and whole health education provided via VA Video Connect (VVC), the VA video telehealth platform. iVET is a 2- to 4-week program and consists of 3 parts: (1) massed individual UP for transdiagnostic treatment of anxiety and co-occurring depression, (2) group whole health and wellness classes, and (3) individual health coaching to address personal wellness goals. Programming is offered over 10-hour days to increase access across multiple time zones, especially to allow participation from Guam and Saipan.

When a patient is referred to iVET, the first contact is a video (or telephone) appointment with a registered nurse (RN) for a screening session. The screening session is designed to provide education about the program (including interventions, time commitment, and resources required for participation). Following education, the RN completes a safety screening, including screening for suicidal ideation and risk, as well as intimate partner violence risk. If urgent safety concerns are present, a licensed social worker or psychologist joins the screening to complete further risk assessment and address any safety concerns.

Following screening, patients are scheduled for a VVC telehealth intake with a licensed therapist (social worker or psychologist) to complete a diagnostic interview. Patients are sent a secure link to complete a measurement-based care (MBC) battery of self-report measures, including assessments of demographics, anxiety symptoms, depression symptoms, substance use, psychological flexibility, quality of life, and satisfaction with mental health care. The results of the diagnostic interview and self-report measures are discussed with the patient during the intake session to plan next steps and support shared decision-making. This initial VVC intake not only allows for fit assessment but also serves to troubleshoot technical difficulties with the virtual platforms.

Notably, there are minimal exclusion criteria for participation in iVET. These include active unmanaged psychosis or manic symptoms, recent suicidal crises (attempt within 8 weeks), active nonsuicidal self-injury (within 8 weeks), and moderate to severe cognitive impairment. Following intake, patients are scheduled to begin their course of care with iVET. Upon completion of intake, patients receive program materials for individual and group classes and are told they will receive email links for all VVC telehealth appointments. Patients are admitted to the iVET on a rolling basis, thereby increasing access compared with closed group and/or cohort models of care.

Patients receiving iVET attend 2 to 4 telehealth appointments daily and complete exercises between sessions. The primary iVET for Anxiety program EBP intervention is a massed or accelerated individual course of UP, which includes 8 primary components: assessment, goals and motivation, understanding emotions, mindful emotion awareness, cognitive flexibility, countering emotional behaviors, understanding and confronting physical sensations, and emotional exposures. UP is delivered in 4 to 8 individual sessions weekly (60-90 minutes each), allowing completion of the full UP protocol with fidelity in 2 to 4 weeks. In addition to primary EBP intervention, patients participate in 4 group sessions weekly (50 minutes each) of a whole health and wellness education class and have access to 1 individual health coaching session weekly (30- 60 minutes) to set wellness goals and receive coaching. During iVET, patients are invited to complete MBC batteries of self-report measures assessing anxiety symptoms, depression symptoms, substance use, psychological flexibility, quality of life, and satisfaction with mental health care at sessions 1, 5, 9, 13, and the final UP session. Following discharge from the iVET, patients are offered 1-, 3-, and 6-month individual postdischarge check-up sessions with a therapist, during which they are invited to complete MBC measures and review relapse prevention and maintenance of treatment gains. Likewise, patients are offered 1-, 3-, and 6-month individual postdischarge check-up sessions with an RN focused on maintaining wellness gains.

The iVET for Anxiety staff has 3 therapists (psychologists or social workers) and 1 RN. Additionally, the iVET for Anxiety is supported by a program manager and a program support assistant who support 2 programs total (the iVET for Anxiety plus another mental health program). The primary cost of the program is staff salary. Additional resources included computer equipment for staff and supplies (eg, printed materials for patients and office supplies). Due to the virtual environment of care, iVET staff telework and do not require physical space within VAPIHCS.

Outcomes

Veterans receiving iVET for Anxiety are invited to complete MBC multiple times, including pretreatment, during UP treatment (sessions 1, 5, 9, 13, and the final session), and posttreatment (1, 3, and 6 months). MBC measures include self-reported demographics; a 2-item measure of satisfaction with mental health services; the Acceptance and Action Questionnaire II,²⁶ the Brief Addiction Monitor-Intensive Outpatient Program, ²⁷ the Generalized Anxiety Disorder-7,²⁸ the Patient Health Questionnaire (PHQ-9),²⁹ and the Quality of Life Enjoyment and Satisfaction Questionnaire-Short Form.³⁰

Forty-two veterans completed the iVET for Anxiety program, with a retention rate of 87.5% completing . 16 sessions of massed UP (Table 2). Veterans reported reduced anxiety (P < .001), depression (P < .001), and substance use risk (P < .001). Veterans experienced improved acceptance of emotional experiences (P < .001) and quality of life (P < .001), based on paired sample t tests comparing session 1 vs final session scores on the self-reported measures. Veterans also reported high satisfaction with mental health care in iVET for Anxiety.

Veterans experienced reduced thoughts of death or suicidal ideation (SI) based on responses to item 9 of PHQ-9. When SI was categorically analyzed (presence vs absence) using PHQ-9 item 9, there was an association observed between absence of SI and completion of a course of massed UP that does not appear to be due to chance, (X² [1, N = 42) = 3.94; P = .047). In addition, veterans who completed the program showed a significant decrease in SI severity measured continuously (range, 0-3) on PHQ-9 item 9 (P < .05) (Table 3).

Future Directions

The aim of this program is to see sustained patient outcomes as iVET continues to serve more veterans. Another line of inquiry is longer- term follow-up, given that long-term follow- up was not addressed in this project. We also hope that the accelerated model of care can be applied to treatment of other presenting concerns (eg, relationship difficulties, insomnia). Expansion of accelerated mental health treatment into other federal and nonfederal health care settings is another area worthy of future inquiry. Exploration of staff satisfaction and burnout related to providing accelerated UP is another important future direction. Relatedly, assessment of the staff burden to learn 1 transdiagnostic EBP vs learning multiple SDPs is another future direction. Likewise, exploration of institutional benefits of investment in transdiagnostic training, supervision, and consultation for UP vs multiple SDPs may be important. These areas could also result in insightful, beneficial evidence of the effectiveness of massed UP to add to the existing literature.

Conclusions

UP for transdiagnostic treatment of anxiety, depressive, and emotional disorders has demonstrated reduced suffering and improved functioning and is supported by multiple clinical practice guidelines.^1-4 Federal practitioners are positioned to improve access to this intervention, thereby reducing pain and improving lives. Indeed, it is crucial to envision a future state in which access to UP for a range of anxiety and depressive disorders is improved and broad, retention rates are dramatically improved, and clinicians providing UP do not experience the high burden and burnout associated with needing to learn and implement a variety of SDPs. Development of these programs, or similar tracks within existing programs, that provide massed or accelerated UP for transdiagnostic treatment of a range of anxiety and depressive disorders with virtual delivery options, is imperative to advance improved care for patients and clinicians.

Federal health care settings treating patients with anxiety and depression, such as those within the US Department of Defense, Indian Health Services, Bureau of Prisons, and VHA, are positioned to implement programs like iVET. Moreover, at the institutional level, investment in training and supervision in the transdiagnostic UP as opposed to multiple SDPs warrants consideration. We believe this model of care has great merit and foresee a future where all patients seeking treatment for anxiety and depression have the option to complete an accelerated or massed course of transdiagnostic care with UP if they so desire. Our experiences with iVET illustrate the feasibility, acceptability, and sustainability of such programs without requiring substantial staffing and financial resources.

Cognitive behavioral therapy (CBT) is supported as an effective treatment for depression by clinical practice guidelines from the US Department of Veterans Affairs (VA) and US Department of Defense.^1-3 Despite research supporting the use of evidence-based CBT for anxiety, mood, and emotional disorders, patient access to these interventions is limited.⁴ One barrier to CBT for anxiety, mood, and emotional disorders is the traditional use of single-disorder protocols (SDPs) to separately treat each disorder (eg, major depression, generalized anxiety disorder, panic disorder).

Use of SDPs places a high burden on clinicians, requiring them to learn and competently implement multiple different manualized interventions for each anxiety, mood, or emotional disorder encountered in practice.⁴ It is common for individuals who present with an anxiety, mood, or emotional disorder to experience co-occurring disorders.⁵ Traditional SDP-based CBT may require multiple SDPs to address co-occurring disorders, extending time in treatment and increasing training burden. There is evidence that even when an SDP is used, co-occurring difficulties may decrease in intensity.⁶ Thus, evidence-based CBT for 1 presenting concern may positively affect co-occurring difficulties that are not the primary treatment target.⁶

Unified Protocol (UP) is a transdiagnostic CBT intervention for anxiety and mood disorders. UP targets emotional experiences (eg, negative affect and emotional distress) that are present in multiple anxiety, depression, and emotional disorders. UP is organized into 8 modules and uses interventions present in other CBT SDPs, such as teaching objective assessment methods to clients, motivational enhancement and goal setting, emotion psychoeducation, mindful awareness, cognitive reframing, and exposure principles as mechanisms of change (Table 1).⁴ UP is an ideal intervention for addressing a number of anxiety, mood, and emotional disorders as well as addressing cooccurring disorders within the same course of treatment.

UP has been compared to SDPs; studies illustrate that UP is equivalent to SDPs at the end of treatment and at 6 months posttreatment.^4,7 Additionally, patients who received UP experienced symptom reduction for multiple co-occurring disorders.⁷ Furthermore, patients were less likely to drop out of UP when compared with SDPs.⁴

Studies have reported positive impacts on affective disorders with UP. A 2019 meta-analysis found significant reductions in depression and anxiety-related disorders.⁸ Additionally, UP has been shown to be effective when delivered in person and via telehealth.^9,10 UP has also been successfully used in veterans.¹¹ While traditional models of UP (1-2 sessions weekly) have a lower dropout rate than SDPs, UP and CBT dropout rates still leave room for improvement.^12-14 Specifically, rates of attrition from SDP CBT and transdiagnostic CBT protocols range from 9% to 35%, and dropout reduces the likelihood of attaining a full therapeutic dose of any course of CBT. Notably, accelerated delivery of CBT (ie, ≥ 3 sessions/wk) has been shown to reduce risk of dropout.^14,15

Veterans are at increased risk for suicide, and anxiety and mood disorders are associated with increased risk of suicide attempt and death.^16,17 Very few veterans who could benefit from high-quality CBT interventions, whether SDP or UP, are able to access them. Only 6.3% of veterans received ≥1 session of an evidence-based CBT SDP for posttraumatic stress disorder (PTSD) at 6 clinics evaluated in 2010.¹⁸

We identified 2 strategies to increase access to and completion of an evidence-based CBT course. First, the use of transdiagnostic UP instead of SDPs reduced burden on therapists and allowed them to address presenting and co-occurring disorders within the same course of care. Second, the use of an accelerated model of service delivery reduced dropout risk. Training clinicians to deliver UP is efficient and cost-effective, considering clinicians use core strategies that can be tailored and flexibly applied to a range of emotional difficulties. Thus, implementing UP may decrease barriers to receiving an optimal dose of an evidence-based CBT delivered with fidelity in a time- and cost-efficient manner.⁴ Two studies have found no evidence of differences in outcomes between UP and SDPs, suggesting that training and supervising clinicians in a single transdiagnostic UP intervention may prepare them to treat heterogeneous and co-occurring anxiety, mood, and emotional disorders with less burden than learning multiple SDPs.^7,19

Delivering UP in an accelerated or massed format (≥4 sessions/wk) instead of the traditional spaced model (1 session/wk) has empirical support but has not been widely implemented. This approach, sometimes referred to as a UP-intensive outpatient program (UP-IOP) or UP-intensive outpatient track (UP-IOT), has been shown to be feasible, acceptable, and effective, with increased completion rates compared with traditional UP delivery (1-2 sessions/wk).^20-22

Ragsdale et al describe a 2-week IOP with multiple treatment tracks, including a general track.²⁰ The general track includes massed UP and additional standard services, including case management, wellness services, family services, and a single session effective behaviors group. Additional augmentation services are available when clinically indicated (eg, repetitive transcranial magnetic stimulation, transcranial direct current stimulation, psychoeducation, motivational interviewing, relapse prevention). In other words, this was an UP-IOP.²⁰

Thompson-Brenner et al described a successful implementation of massed UP applied in intensive treatment settings, such as residential and day-hospital programs, for eating disorders. Patients reported improvements in 3 UP targets: experiential avoidance, mindfulness, and anxiety.²¹ Watkins et al evaluated a 2-week IOP using CBT for comorbid substance use and mental health disorders, including prolonged exposure, UP, and relapse prevention for substance use disorders. Participants were post-9/11 veterans and activeduty personnel. Results indicated that UP reduced PTSD and depressive symptoms following treatment. Furthermore, the retention rate (91%) was higher than retention in outpatient treatment (39%-65%), supporting the IOP model as a strategy to reduce dropout.²²

Massed psychotherapy has been extended to IOP programs for PTSD treatment within the Veterans Health Administration (VHA). Yamokoski et al found that patients who completed an IOP that included massed CBT for PTSD had high retention, high satisfaction, and significant reduction in self-reported co-occurring depression symptoms. The authors also found that this model of care could be implemented and sustained within a VHA facility using minimal staffing resources.²³

The UP-IOP models described by Ragsdale et al and Watkins et al included massed UP as the primary evidence-based practice (EBP) with adjunctive groups (eg, wellness and educational groups and access to complementary interventions such as mindfulness and yoga); they found that adding these groups increased retention and patient- reported satisfaction (ie, UP-IOP).^20,22 The addition of wellness education alongside a primary EBP aligns with the VHA focus on whole health well-being and wellness. This includes understanding factors that motivate a patient toward health and well-being, providing health education, and offering access to complementary interventions such as mindfulness.²⁴

Dryden et al described the whole health transformation within VHA as a proactive approach to addressing both employee and patient wellness. Their research found that the whole health model promoted wellbeing in patients and staff and these improvements were sustained during the COVID-19 pandemic. Dryden et al also noted that virtual technologies facilitated continued whole health implementation.²⁵

The literature illustrates that (1) massed UP can be delivered with complementary education and wellness offerings that may increase retention and satisfaction by enriching treatment (eg, delivering UP-IOP); (2) whole health, including wellness education and complementary interventions (eg, mindfulness, motivational enhancement), promotes well-being in patients and clinicians; and (3) whole health education and complementary interventions can be delivered virtually.

IVET

Health Care Need

Veterans Affairs Pacific Islands Health Care System (VAPIHCS) provides medical services to veterans in the Hawaiian Islands, American Samoa, Guam, and Saipan spanning nearly 4000 miles across the Pacific Ocean. Prior to implementation of this program, veterans who received care at VAPIHCS had little to no access to UP in outpatient settings and no access to UP in residential settings. Access to UP depended on the presence of a therapist trained in UP within a given clinic and was geographically limited to the location of the UP-trained therapist. The limited outpatient access to UP was restricted to the traditional UP delivery model (eg, about 1 session/wk); thus, there was no access to accelerated UP for veterans served at VAPIHCS. In the fiscal year prior to implementation of the massed UP program, > 1000 VAPIHCS veterans had been diagnosed with obsessive-compulsive related disorder or anxiety. A massed-UP program with weekly rolling admissions would support access to UP for more veterans over 12 months, and the virtual treatment modality would reduce barriers for diverse and underserved veterans, making care more equitable and inclusive.

Successful implementation and sustainment of an EBP prompted the establishment of this UP program. In 2022, VAPIHCS launched the Intensive Virtual Evidence- Based Psychotherapy Team (iVET) for treatment of PTSD. This clinic and associated EBP project demonstrated that massed (≥3 individual sessions/wk) of prolonged exposure (PE) therapy, delivered virtually to a geographically diverse veteran population with PTSD, resulted in significant reductions in PTSD, depression, and anxiety symptoms and substance use risk factors, with very high retention rates. The iVET for PTSD program was feasible, acceptable, and effective, with veterans reporting significant improvement in quality of life and high satisfaction with their mental health services.¹⁵ Given the known benefits of transdiagnostic UP treatment (vs SDPs), the need for accelerated UP, and success with accelerated PE, our goal was to spread the EBP of massed (≥3 sessions/ wk) virtual psychotherapy to other presenting problems (eg, anxiety disorders with or without co-occurring unipolar depression) using transdiagnostic UP.

Program Description

The program implemented within outpatient mental health services at VAPIHCS was iVET for the treatment of anxiety with or without co-occurring depression. The program model consists of an accelerated course of UP and whole health education provided via VA Video Connect (VVC), the VA video telehealth platform. iVET is a 2- to 4-week program and consists of 3 parts: (1) massed individual UP for transdiagnostic treatment of anxiety and co-occurring depression, (2) group whole health and wellness classes, and (3) individual health coaching to address personal wellness goals. Programming is offered over 10-hour days to increase access across multiple time zones, especially to allow participation from Guam and Saipan.

When a patient is referred to iVET, the first contact is a video (or telephone) appointment with a registered nurse (RN) for a screening session. The screening session is designed to provide education about the program (including interventions, time commitment, and resources required for participation). Following education, the RN completes a safety screening, including screening for suicidal ideation and risk, as well as intimate partner violence risk. If urgent safety concerns are present, a licensed social worker or psychologist joins the screening to complete further risk assessment and address any safety concerns.

Following screening, patients are scheduled for a VVC telehealth intake with a licensed therapist (social worker or psychologist) to complete a diagnostic interview. Patients are sent a secure link to complete a measurement-based care (MBC) battery of self-report measures, including assessments of demographics, anxiety symptoms, depression symptoms, substance use, psychological flexibility, quality of life, and satisfaction with mental health care. The results of the diagnostic interview and self-report measures are discussed with the patient during the intake session to plan next steps and support shared decision-making. This initial VVC intake not only allows for fit assessment but also serves to troubleshoot technical difficulties with the virtual platforms.

Notably, there are minimal exclusion criteria for participation in iVET. These include active unmanaged psychosis or manic symptoms, recent suicidal crises (attempt within 8 weeks), active nonsuicidal self-injury (within 8 weeks), and moderate to severe cognitive impairment. Following intake, patients are scheduled to begin their course of care with iVET. Upon completion of intake, patients receive program materials for individual and group classes and are told they will receive email links for all VVC telehealth appointments. Patients are admitted to the iVET on a rolling basis, thereby increasing access compared with closed group and/or cohort models of care.

Patients receiving iVET attend 2 to 4 telehealth appointments daily and complete exercises between sessions. The primary iVET for Anxiety program EBP intervention is a massed or accelerated individual course of UP, which includes 8 primary components: assessment, goals and motivation, understanding emotions, mindful emotion awareness, cognitive flexibility, countering emotional behaviors, understanding and confronting physical sensations, and emotional exposures. UP is delivered in 4 to 8 individual sessions weekly (60-90 minutes each), allowing completion of the full UP protocol with fidelity in 2 to 4 weeks. In addition to primary EBP intervention, patients participate in 4 group sessions weekly (50 minutes each) of a whole health and wellness education class and have access to 1 individual health coaching session weekly (30- 60 minutes) to set wellness goals and receive coaching. During iVET, patients are invited to complete MBC batteries of self-report measures assessing anxiety symptoms, depression symptoms, substance use, psychological flexibility, quality of life, and satisfaction with mental health care at sessions 1, 5, 9, 13, and the final UP session. Following discharge from the iVET, patients are offered 1-, 3-, and 6-month individual postdischarge check-up sessions with a therapist, during which they are invited to complete MBC measures and review relapse prevention and maintenance of treatment gains. Likewise, patients are offered 1-, 3-, and 6-month individual postdischarge check-up sessions with an RN focused on maintaining wellness gains.

The iVET for Anxiety staff has 3 therapists (psychologists or social workers) and 1 RN. Additionally, the iVET for Anxiety is supported by a program manager and a program support assistant who support 2 programs total (the iVET for Anxiety plus another mental health program). The primary cost of the program is staff salary. Additional resources included computer equipment for staff and supplies (eg, printed materials for patients and office supplies). Due to the virtual environment of care, iVET staff telework and do not require physical space within VAPIHCS.

Outcomes

Veterans receiving iVET for Anxiety are invited to complete MBC multiple times, including pretreatment, during UP treatment (sessions 1, 5, 9, 13, and the final session), and posttreatment (1, 3, and 6 months). MBC measures include self-reported demographics; a 2-item measure of satisfaction with mental health services; the Acceptance and Action Questionnaire II,²⁶ the Brief Addiction Monitor-Intensive Outpatient Program, ²⁷ the Generalized Anxiety Disorder-7,²⁸ the Patient Health Questionnaire (PHQ-9),²⁹ and the Quality of Life Enjoyment and Satisfaction Questionnaire-Short Form.³⁰

Forty-two veterans completed the iVET for Anxiety program, with a retention rate of 87.5% completing . 16 sessions of massed UP (Table 2). Veterans reported reduced anxiety (P < .001), depression (P < .001), and substance use risk (P < .001). Veterans experienced improved acceptance of emotional experiences (P < .001) and quality of life (P < .001), based on paired sample t tests comparing session 1 vs final session scores on the self-reported measures. Veterans also reported high satisfaction with mental health care in iVET for Anxiety.

Veterans experienced reduced thoughts of death or suicidal ideation (SI) based on responses to item 9 of PHQ-9. When SI was categorically analyzed (presence vs absence) using PHQ-9 item 9, there was an association observed between absence of SI and completion of a course of massed UP that does not appear to be due to chance, (X² [1, N = 42) = 3.94; P = .047). In addition, veterans who completed the program showed a significant decrease in SI severity measured continuously (range, 0-3) on PHQ-9 item 9 (P < .05) (Table 3).

Future Directions

The aim of this program is to see sustained patient outcomes as iVET continues to serve more veterans. Another line of inquiry is longer- term follow-up, given that long-term follow- up was not addressed in this project. We also hope that the accelerated model of care can be applied to treatment of other presenting concerns (eg, relationship difficulties, insomnia). Expansion of accelerated mental health treatment into other federal and nonfederal health care settings is another area worthy of future inquiry. Exploration of staff satisfaction and burnout related to providing accelerated UP is another important future direction. Relatedly, assessment of the staff burden to learn 1 transdiagnostic EBP vs learning multiple SDPs is another future direction. Likewise, exploration of institutional benefits of investment in transdiagnostic training, supervision, and consultation for UP vs multiple SDPs may be important. These areas could also result in insightful, beneficial evidence of the effectiveness of massed UP to add to the existing literature.

Conclusions

UP for transdiagnostic treatment of anxiety, depressive, and emotional disorders has demonstrated reduced suffering and improved functioning and is supported by multiple clinical practice guidelines.^1-4 Federal practitioners are positioned to improve access to this intervention, thereby reducing pain and improving lives. Indeed, it is crucial to envision a future state in which access to UP for a range of anxiety and depressive disorders is improved and broad, retention rates are dramatically improved, and clinicians providing UP do not experience the high burden and burnout associated with needing to learn and implement a variety of SDPs. Development of these programs, or similar tracks within existing programs, that provide massed or accelerated UP for transdiagnostic treatment of a range of anxiety and depressive disorders with virtual delivery options, is imperative to advance improved care for patients and clinicians.

Federal health care settings treating patients with anxiety and depression, such as those within the US Department of Defense, Indian Health Services, Bureau of Prisons, and VHA, are positioned to implement programs like iVET. Moreover, at the institutional level, investment in training and supervision in the transdiagnostic UP as opposed to multiple SDPs warrants consideration. We believe this model of care has great merit and foresee a future where all patients seeking treatment for anxiety and depression have the option to complete an accelerated or massed course of transdiagnostic care with UP if they so desire. Our experiences with iVET illustrate the feasibility, acceptability, and sustainability of such programs without requiring substantial staffing and financial resources.

Cognitive behavioral therapy (CBT) is supported as an effective treatment for depression by clinical practice guidelines from the US Department of Veterans Affairs (VA) and US Department of Defense.^1-3 Despite research supporting the use of evidence-based CBT for anxiety, mood, and emotional disorders, patient access to these interventions is limited.⁴ One barrier to CBT for anxiety, mood, and emotional disorders is the traditional use of single-disorder protocols (SDPs) to separately treat each disorder (eg, major depression, generalized anxiety disorder, panic disorder).

Use of SDPs places a high burden on clinicians, requiring them to learn and competently implement multiple different manualized interventions for each anxiety, mood, or emotional disorder encountered in practice.⁴ It is common for individuals who present with an anxiety, mood, or emotional disorder to experience co-occurring disorders.⁵ Traditional SDP-based CBT may require multiple SDPs to address co-occurring disorders, extending time in treatment and increasing training burden. There is evidence that even when an SDP is used, co-occurring difficulties may decrease in intensity.⁶ Thus, evidence-based CBT for 1 presenting concern may positively affect co-occurring difficulties that are not the primary treatment target.⁶

Unified Protocol (UP) is a transdiagnostic CBT intervention for anxiety and mood disorders. UP targets emotional experiences (eg, negative affect and emotional distress) that are present in multiple anxiety, depression, and emotional disorders. UP is organized into 8 modules and uses interventions present in other CBT SDPs, such as teaching objective assessment methods to clients, motivational enhancement and goal setting, emotion psychoeducation, mindful awareness, cognitive reframing, and exposure principles as mechanisms of change (Table 1).⁴ UP is an ideal intervention for addressing a number of anxiety, mood, and emotional disorders as well as addressing cooccurring disorders within the same course of treatment.

UP has been compared to SDPs; studies illustrate that UP is equivalent to SDPs at the end of treatment and at 6 months posttreatment.^4,7 Additionally, patients who received UP experienced symptom reduction for multiple co-occurring disorders.⁷ Furthermore, patients were less likely to drop out of UP when compared with SDPs.⁴

Studies have reported positive impacts on affective disorders with UP. A 2019 meta-analysis found significant reductions in depression and anxiety-related disorders.⁸ Additionally, UP has been shown to be effective when delivered in person and via telehealth.^9,10 UP has also been successfully used in veterans.¹¹ While traditional models of UP (1-2 sessions weekly) have a lower dropout rate than SDPs, UP and CBT dropout rates still leave room for improvement.^12-14 Specifically, rates of attrition from SDP CBT and transdiagnostic CBT protocols range from 9% to 35%, and dropout reduces the likelihood of attaining a full therapeutic dose of any course of CBT. Notably, accelerated delivery of CBT (ie, ≥ 3 sessions/wk) has been shown to reduce risk of dropout.^14,15

Veterans are at increased risk for suicide, and anxiety and mood disorders are associated with increased risk of suicide attempt and death.^16,17 Very few veterans who could benefit from high-quality CBT interventions, whether SDP or UP, are able to access them. Only 6.3% of veterans received ≥1 session of an evidence-based CBT SDP for posttraumatic stress disorder (PTSD) at 6 clinics evaluated in 2010.¹⁸

We identified 2 strategies to increase access to and completion of an evidence-based CBT course. First, the use of transdiagnostic UP instead of SDPs reduced burden on therapists and allowed them to address presenting and co-occurring disorders within the same course of care. Second, the use of an accelerated model of service delivery reduced dropout risk. Training clinicians to deliver UP is efficient and cost-effective, considering clinicians use core strategies that can be tailored and flexibly applied to a range of emotional difficulties. Thus, implementing UP may decrease barriers to receiving an optimal dose of an evidence-based CBT delivered with fidelity in a time- and cost-efficient manner.⁴ Two studies have found no evidence of differences in outcomes between UP and SDPs, suggesting that training and supervising clinicians in a single transdiagnostic UP intervention may prepare them to treat heterogeneous and co-occurring anxiety, mood, and emotional disorders with less burden than learning multiple SDPs.^7,19

Delivering UP in an accelerated or massed format (≥4 sessions/wk) instead of the traditional spaced model (1 session/wk) has empirical support but has not been widely implemented. This approach, sometimes referred to as a UP-intensive outpatient program (UP-IOP) or UP-intensive outpatient track (UP-IOT), has been shown to be feasible, acceptable, and effective, with increased completion rates compared with traditional UP delivery (1-2 sessions/wk).^20-22

Ragsdale et al describe a 2-week IOP with multiple treatment tracks, including a general track.²⁰ The general track includes massed UP and additional standard services, including case management, wellness services, family services, and a single session effective behaviors group. Additional augmentation services are available when clinically indicated (eg, repetitive transcranial magnetic stimulation, transcranial direct current stimulation, psychoeducation, motivational interviewing, relapse prevention). In other words, this was an UP-IOP.²⁰

Thompson-Brenner et al described a successful implementation of massed UP applied in intensive treatment settings, such as residential and day-hospital programs, for eating disorders. Patients reported improvements in 3 UP targets: experiential avoidance, mindfulness, and anxiety.²¹ Watkins et al evaluated a 2-week IOP using CBT for comorbid substance use and mental health disorders, including prolonged exposure, UP, and relapse prevention for substance use disorders. Participants were post-9/11 veterans and activeduty personnel. Results indicated that UP reduced PTSD and depressive symptoms following treatment. Furthermore, the retention rate (91%) was higher than retention in outpatient treatment (39%-65%), supporting the IOP model as a strategy to reduce dropout.²²

Massed psychotherapy has been extended to IOP programs for PTSD treatment within the Veterans Health Administration (VHA). Yamokoski et al found that patients who completed an IOP that included massed CBT for PTSD had high retention, high satisfaction, and significant reduction in self-reported co-occurring depression symptoms. The authors also found that this model of care could be implemented and sustained within a VHA facility using minimal staffing resources.²³

The UP-IOP models described by Ragsdale et al and Watkins et al included massed UP as the primary evidence-based practice (EBP) with adjunctive groups (eg, wellness and educational groups and access to complementary interventions such as mindfulness and yoga); they found that adding these groups increased retention and patient- reported satisfaction (ie, UP-IOP).^20,22 The addition of wellness education alongside a primary EBP aligns with the VHA focus on whole health well-being and wellness. This includes understanding factors that motivate a patient toward health and well-being, providing health education, and offering access to complementary interventions such as mindfulness.²⁴

Dryden et al described the whole health transformation within VHA as a proactive approach to addressing both employee and patient wellness. Their research found that the whole health model promoted wellbeing in patients and staff and these improvements were sustained during the COVID-19 pandemic. Dryden et al also noted that virtual technologies facilitated continued whole health implementation.²⁵

The literature illustrates that (1) massed UP can be delivered with complementary education and wellness offerings that may increase retention and satisfaction by enriching treatment (eg, delivering UP-IOP); (2) whole health, including wellness education and complementary interventions (eg, mindfulness, motivational enhancement), promotes well-being in patients and clinicians; and (3) whole health education and complementary interventions can be delivered virtually.

IVET

Health Care Need

Veterans Affairs Pacific Islands Health Care System (VAPIHCS) provides medical services to veterans in the Hawaiian Islands, American Samoa, Guam, and Saipan spanning nearly 4000 miles across the Pacific Ocean. Prior to implementation of this program, veterans who received care at VAPIHCS had little to no access to UP in outpatient settings and no access to UP in residential settings. Access to UP depended on the presence of a therapist trained in UP within a given clinic and was geographically limited to the location of the UP-trained therapist. The limited outpatient access to UP was restricted to the traditional UP delivery model (eg, about 1 session/wk); thus, there was no access to accelerated UP for veterans served at VAPIHCS. In the fiscal year prior to implementation of the massed UP program, > 1000 VAPIHCS veterans had been diagnosed with obsessive-compulsive related disorder or anxiety. A massed-UP program with weekly rolling admissions would support access to UP for more veterans over 12 months, and the virtual treatment modality would reduce barriers for diverse and underserved veterans, making care more equitable and inclusive.

Successful implementation and sustainment of an EBP prompted the establishment of this UP program. In 2022, VAPIHCS launched the Intensive Virtual Evidence- Based Psychotherapy Team (iVET) for treatment of PTSD. This clinic and associated EBP project demonstrated that massed (≥3 individual sessions/wk) of prolonged exposure (PE) therapy, delivered virtually to a geographically diverse veteran population with PTSD, resulted in significant reductions in PTSD, depression, and anxiety symptoms and substance use risk factors, with very high retention rates. The iVET for PTSD program was feasible, acceptable, and effective, with veterans reporting significant improvement in quality of life and high satisfaction with their mental health services.¹⁵ Given the known benefits of transdiagnostic UP treatment (vs SDPs), the need for accelerated UP, and success with accelerated PE, our goal was to spread the EBP of massed (≥3 sessions/ wk) virtual psychotherapy to other presenting problems (eg, anxiety disorders with or without co-occurring unipolar depression) using transdiagnostic UP.

Program Description

The program implemented within outpatient mental health services at VAPIHCS was iVET for the treatment of anxiety with or without co-occurring depression. The program model consists of an accelerated course of UP and whole health education provided via VA Video Connect (VVC), the VA video telehealth platform. iVET is a 2- to 4-week program and consists of 3 parts: (1) massed individual UP for transdiagnostic treatment of anxiety and co-occurring depression, (2) group whole health and wellness classes, and (3) individual health coaching to address personal wellness goals. Programming is offered over 10-hour days to increase access across multiple time zones, especially to allow participation from Guam and Saipan.

When a patient is referred to iVET, the first contact is a video (or telephone) appointment with a registered nurse (RN) for a screening session. The screening session is designed to provide education about the program (including interventions, time commitment, and resources required for participation). Following education, the RN completes a safety screening, including screening for suicidal ideation and risk, as well as intimate partner violence risk. If urgent safety concerns are present, a licensed social worker or psychologist joins the screening to complete further risk assessment and address any safety concerns.

Following screening, patients are scheduled for a VVC telehealth intake with a licensed therapist (social worker or psychologist) to complete a diagnostic interview. Patients are sent a secure link to complete a measurement-based care (MBC) battery of self-report measures, including assessments of demographics, anxiety symptoms, depression symptoms, substance use, psychological flexibility, quality of life, and satisfaction with mental health care. The results of the diagnostic interview and self-report measures are discussed with the patient during the intake session to plan next steps and support shared decision-making. This initial VVC intake not only allows for fit assessment but also serves to troubleshoot technical difficulties with the virtual platforms.

Notably, there are minimal exclusion criteria for participation in iVET. These include active unmanaged psychosis or manic symptoms, recent suicidal crises (attempt within 8 weeks), active nonsuicidal self-injury (within 8 weeks), and moderate to severe cognitive impairment. Following intake, patients are scheduled to begin their course of care with iVET. Upon completion of intake, patients receive program materials for individual and group classes and are told they will receive email links for all VVC telehealth appointments. Patients are admitted to the iVET on a rolling basis, thereby increasing access compared with closed group and/or cohort models of care.

Patients receiving iVET attend 2 to 4 telehealth appointments daily and complete exercises between sessions. The primary iVET for Anxiety program EBP intervention is a massed or accelerated individual course of UP, which includes 8 primary components: assessment, goals and motivation, understanding emotions, mindful emotion awareness, cognitive flexibility, countering emotional behaviors, understanding and confronting physical sensations, and emotional exposures. UP is delivered in 4 to 8 individual sessions weekly (60-90 minutes each), allowing completion of the full UP protocol with fidelity in 2 to 4 weeks. In addition to primary EBP intervention, patients participate in 4 group sessions weekly (50 minutes each) of a whole health and wellness education class and have access to 1 individual health coaching session weekly (30- 60 minutes) to set wellness goals and receive coaching. During iVET, patients are invited to complete MBC batteries of self-report measures assessing anxiety symptoms, depression symptoms, substance use, psychological flexibility, quality of life, and satisfaction with mental health care at sessions 1, 5, 9, 13, and the final UP session. Following discharge from the iVET, patients are offered 1-, 3-, and 6-month individual postdischarge check-up sessions with a therapist, during which they are invited to complete MBC measures and review relapse prevention and maintenance of treatment gains. Likewise, patients are offered 1-, 3-, and 6-month individual postdischarge check-up sessions with an RN focused on maintaining wellness gains.

The iVET for Anxiety staff has 3 therapists (psychologists or social workers) and 1 RN. Additionally, the iVET for Anxiety is supported by a program manager and a program support assistant who support 2 programs total (the iVET for Anxiety plus another mental health program). The primary cost of the program is staff salary. Additional resources included computer equipment for staff and supplies (eg, printed materials for patients and office supplies). Due to the virtual environment of care, iVET staff telework and do not require physical space within VAPIHCS.

Outcomes

Veterans receiving iVET for Anxiety are invited to complete MBC multiple times, including pretreatment, during UP treatment (sessions 1, 5, 9, 13, and the final session), and posttreatment (1, 3, and 6 months). MBC measures include self-reported demographics; a 2-item measure of satisfaction with mental health services; the Acceptance and Action Questionnaire II,²⁶ the Brief Addiction Monitor-Intensive Outpatient Program, ²⁷ the Generalized Anxiety Disorder-7,²⁸ the Patient Health Questionnaire (PHQ-9),²⁹ and the Quality of Life Enjoyment and Satisfaction Questionnaire-Short Form.³⁰

Forty-two veterans completed the iVET for Anxiety program, with a retention rate of 87.5% completing . 16 sessions of massed UP (Table 2). Veterans reported reduced anxiety (P < .001), depression (P < .001), and substance use risk (P < .001). Veterans experienced improved acceptance of emotional experiences (P < .001) and quality of life (P < .001), based on paired sample t tests comparing session 1 vs final session scores on the self-reported measures. Veterans also reported high satisfaction with mental health care in iVET for Anxiety.

Veterans experienced reduced thoughts of death or suicidal ideation (SI) based on responses to item 9 of PHQ-9. When SI was categorically analyzed (presence vs absence) using PHQ-9 item 9, there was an association observed between absence of SI and completion of a course of massed UP that does not appear to be due to chance, (X² [1, N = 42) = 3.94; P = .047). In addition, veterans who completed the program showed a significant decrease in SI severity measured continuously (range, 0-3) on PHQ-9 item 9 (P < .05) (Table 3).

Future Directions

The aim of this program is to see sustained patient outcomes as iVET continues to serve more veterans. Another line of inquiry is longer- term follow-up, given that long-term follow- up was not addressed in this project. We also hope that the accelerated model of care can be applied to treatment of other presenting concerns (eg, relationship difficulties, insomnia). Expansion of accelerated mental health treatment into other federal and nonfederal health care settings is another area worthy of future inquiry. Exploration of staff satisfaction and burnout related to providing accelerated UP is another important future direction. Relatedly, assessment of the staff burden to learn 1 transdiagnostic EBP vs learning multiple SDPs is another future direction. Likewise, exploration of institutional benefits of investment in transdiagnostic training, supervision, and consultation for UP vs multiple SDPs may be important. These areas could also result in insightful, beneficial evidence of the effectiveness of massed UP to add to the existing literature.

Conclusions

UP for transdiagnostic treatment of anxiety, depressive, and emotional disorders has demonstrated reduced suffering and improved functioning and is supported by multiple clinical practice guidelines.^1-4 Federal practitioners are positioned to improve access to this intervention, thereby reducing pain and improving lives. Indeed, it is crucial to envision a future state in which access to UP for a range of anxiety and depressive disorders is improved and broad, retention rates are dramatically improved, and clinicians providing UP do not experience the high burden and burnout associated with needing to learn and implement a variety of SDPs. Development of these programs, or similar tracks within existing programs, that provide massed or accelerated UP for transdiagnostic treatment of a range of anxiety and depressive disorders with virtual delivery options, is imperative to advance improved care for patients and clinicians.

Federal health care settings treating patients with anxiety and depression, such as those within the US Department of Defense, Indian Health Services, Bureau of Prisons, and VHA, are positioned to implement programs like iVET. Moreover, at the institutional level, investment in training and supervision in the transdiagnostic UP as opposed to multiple SDPs warrants consideration. We believe this model of care has great merit and foresee a future where all patients seeking treatment for anxiety and depression have the option to complete an accelerated or massed course of transdiagnostic care with UP if they so desire. Our experiences with iVET illustrate the feasibility, acceptability, and sustainability of such programs without requiring substantial staffing and financial resources.

The US Coast Guard (USCG) operates within the US Department of Homeland Security and represents a force of > 50,000 servicemembers.¹ The missions of the service include maritime law enforcement (drug interdiction), search and rescue, and defense readiness.²

The USCG operates 42 clinics and numerous smaller sick bays of varying sizes and medical capabilities throughout the country to provide acute and routine medical services. Health services technicians (HSs) are the most common staffing component and provide much of the support services in each USCG health care setting. The HS rating, colloquially referred to as corpsmen, is achieved through a 22-week course known as “A” school that trains servicemembers in outpatient and acute care, including emergency medical technician training.³ There are about 750 USCG HSs.

Within USCG clinics, HSs conduct ambulatory intakes for outpatient appointments, administer immunizations and blood draws, requisition medical equipment and supplies, serve as a pharmacy technician, complete physical examinations, and manage referrals, among other duties. Their familiarity with different aspects of clinic operations and medical practice must be broad. To that end, corpsmen develop and reinforce their medical knowledge through various trainings, including additional courses to specialize in certain medical skills, such as pharmacy technician “C” school or dental assistant “C” school.

The USCG employs < 15 field pharmacists, most of whom serve in an ambulatory care environment.⁴ Responsibilities of USCG pharmacists include the routine reinforcement of pharmacy knowledge with HSs. For the corpsmen who are not pharmacy technicians or who have not attended pharmacy technician “C” school, the extent of their pharmacy instruction primarily came from the “A” school curriculum, of which only 1 class is specific to pharmacy. Providing routine pharmacy-related training to the HSs further cultivates their pharmacy knowledge and confidence so that they can practice more holistically. These trainings do not need to follow any specific format.

In this study, 3 pharmacists at 3 separate USCG clinics conducted a training inspired by the Jeopardy! game show with the corpsmen at their respective clinics. This study examined the effectiveness of game-based learning on the pharmacy knowledge retention of HSs at 3 USCG clinics. A secondary objective was to evaluate the baseline pharmacy knowledge of corpsmen based on specific corpsmen demographics.

Methods

As part of a USCG quality improvement study in 2024, 28 HSs at the 3 USCG clinics were provided a preintervention assessment, completed game-based educational program (intervention), and then were assessed again following the intervention.

The HSs were presented with a 25-question assessment that included 10 knowledge questions (3 on over-the-counter medications, 2 on use of medications in pregnancy, 2 on precautions and contraindications, 2 on indications, and 1 on immunizations) and 15 brand-generic matching questions. These questions were developed and reviewed by the 3 participating pharmacists to ensure that their scope was commensurate with the overall pharmacy knowledge that could be reasonably expected of corpsmen spanning various points of their HS career.

One to 7 days after the preintervention assessment, the pharmacists hosted the game-based learning modeled after Jeopardy!. The Jeopardy! categories mirrored the assessment knowledge question categories, and brand-generic nomenclature was freely discussed throughout. About 2 weeks later, the same HSs who completed the preintervention assessment and participated in the game were presented with the same assessment.

In addition to capturing the difference in scores between the 2 assessments, additional demographic data were gathered, including service time as an HS and whether they received formalized pharmacy technician training and if so, how long they have served in that capacity. Demographic data were collected to identify potential correlations between demographic characteristics and results.

Results

Twenty-eight HSs at the 3 clinics completed the game-based training and both assessments. The mean score increased from 15.1 preintervention to 17.4 postintervention (Table). Preintervention scores ranged from 1 to 24 and postintervention scores ranged from 6 to 25.

There were 19 HSs (68%) whose score increased from preintervention to postintervention and 5 (18%) had decreased scores. The largest score decrease was 4 (from 18 to 14), and the largest score increase was 11 (from 13 to 24). The mean improvement was 3.9 among the 19 HSs with increased scores

Twenty-one HSs reported no formal pharmacy technician training, 3 completed pharmacy technician “C” school, and 4 received informal on-the-job training. The mean score for the “C” school trained HSs was 23.0 preintervention and 23.7 postintervention. The mean score for HSs trained on the job was 16.0 preintervention and 18.5 postintervention. The mean score for HSs with no training was 13.9 preintervention and 16.3 postintervention.

As HSs advance in their careers, they typically assume roles with increasing technical knowledge, responsibility, and oversight, thus aligning with advancement from E-4 (third class petty officer) to E-6 (first class petty officer) and beyond. In this study, there was 1 E-3, 12 E-4s (mean time as an HS, 1.3 years), 8 E-5s (mean time as an HS, 4.8 years), and 7 E-6s (mean time as an HS, 8.6 years). The E-3 had a preintervention score of 1.0 and a postintervention score of 6.0. The E-4s had a mean change in score from pre- to postintervention of 2.4. The E-5s had a mean change in score from pre- to postintervention of 1.6. The E-6s had a mean change in score from pre- to postintervention of 2.3.

Discussion

This study is novel in its examination of the impact of game-based learning on the retention of the pharmacy knowledge of USCG corpsmen. A PubMed literature search of the phrase “((Corpsman) OR (Corpsmen)) AND (Coast Guard)” yields 135 results, though none were relevant to the USCG population described in this study. A PubMed literature search of the phrase “(Jeopardy!) AND (pharmacy)” yields 28 results, only 1 of which discusses using the game-based approach as an instructional tool.⁵ A PubMed literature search of the phrase “(game) AND (Coast Guard)” yields 55 results, none of which were specifically relevant to game-based learning in the USCG. This study appears to be among the first to discuss results and trends in game-based learning with USCG corpsmen.

The preponderance of literature for game-based learning strategies exists in children; more research in adults is needed.^6,7 With studies showing that game-based learning may impact motivation to learn and learning gains, it is unsurprising that there is some research in professional health care education. Games modeled after everything from simulated clinical scenarios to Family Feud and Chutes and Ladders-style games have been compared with traditional learning strategies. However, the results of whether game-based learning strategies improve knowledge, clinical decision-making, and motivation to learn vary, suggesting the need for more research in this field.⁸

The results of this study suggest that Jeopardy! is likely an effective instructional method for USCG corpsmen on pharmacy topics. While there were some HSs whose postintervention scores decreased, 19 (68%) had increased scores. Because the second assessment was administered about 2 weeks after the game-based learning, the results suggest some level of knowledge retention. Between these results and the informally perceived level of engagement, game-based learning could be a more stimulating alternative training method to a standard slide-based presentation.

Stratifying the data by demographics revealed additional trends, although they should be interpreted with caution due to the small sample size. The baseline results strongly illustrate the value of formalized training. It is generally expected that HSs who have completed the “C” school pharmacy technician training program should have more pharmacy knowledge than those with on-the-job or less training. The results indicate that “C” school trained and on-the-job trained HSs scored higher on the preintervention assessment (mean, 23.0 and 16.0, respectively), than those with no such experiences (mean, 13.9). Such results underscore the value of formalized training—whether as a pharmacy technician or in any other “C” school—in enhancing the medical knowledge of HSs that may allow them to hold roles of increased responsibility and medical scope.

In addition to stratification by pharmacy technician training, stratification by years of HS experience (roughly correlated to rank) yields a similar result. It would be expected that as HSs advance in their careers, they gain more exposure to various medical topics, including pharmacy. That is not always the case, however, as it is possible an HS never rotated through a pharmacy technician position or has not been recently exposed to pharmacy knowledge. Nevertheless, the results suggest that increased HS experience was likely associated with an increased baseline pharmacy knowledge, with mean preintervention scores increasing from 11.9 to 18.1 to 19.3 for E-4, E-5, and E-6, respectively.

While there are many explanations for these results, the authors hypothesize that when HSs are E-4s, they might not yet have exposure to all aspects of the clinic and are perhaps not as well-versed in pharmacy practice. An E-5—now a few years into their career—would have completed pharmacy technician “C” school or on-the-job training (if applicable), which could account for the significant jump in pharmacy knowledge scores. An E-6 can still engage in direct patient care activities but take on leadership and supervisory roles within the clinic, perhaps explaining the smaller increase in score.

In terms of increasing responsibility, many USCG corpsmen complete another schooling opportunity—Independent Duty Health Services Technician (IDHS)—so they can serve in independent duty roles, many of which are on USCG cutters. While cutters are deployed, that IDHS could be the sole medical personnel on the cutter and function in a midlevel practitioner extender role. Formalized training in pharmacy—the benefits of which are suggested through these results—or another field of medical practice would strengthen the skillset and confidence of IDHSs.

Though not formally assessed, the 3 pharmacists noted that the game-based learning was met with overwhelmingly positive feedback in terms of excitement, energy, and overall engagement.

Limitations

This cohort of individuals represents a small proportion of the total number of USCG corpsmen, and it is not fully representative of all practice settings. HSs can be assigned to USCG cutters as IDHSs, which would not be captured in this cohort. Even within a single clinic, the knowledge of HSs varies, as not all HS duties consist solely of clinical skills. Additionally, while the overall game framework was consistent among the 3 sites, there may have been unquantifiable differences in overall teaching style by the 3 pharmacists that may have resulted in different levels of content retention. Given the lack of similar studies in this population, this study can best be described as a quantitative descriptor of results rather than a statistical comparison of what instructional method works best.

Conclusions

The USCG greatly benefits from having trained and experienced HSs fulfilling mission support roles in the organization. In addition to traditional slide-based trainings, game-based learning can be considered to create engaging learning environments to support the knowledge retention of pharmacy and other medical topics for USCG corpsmen.

The US Coast Guard (USCG) operates within the US Department of Homeland Security and represents a force of > 50,000 servicemembers.¹ The missions of the service include maritime law enforcement (drug interdiction), search and rescue, and defense readiness.²

The USCG operates 42 clinics and numerous smaller sick bays of varying sizes and medical capabilities throughout the country to provide acute and routine medical services. Health services technicians (HSs) are the most common staffing component and provide much of the support services in each USCG health care setting. The HS rating, colloquially referred to as corpsmen, is achieved through a 22-week course known as “A” school that trains servicemembers in outpatient and acute care, including emergency medical technician training.³ There are about 750 USCG HSs.

Within USCG clinics, HSs conduct ambulatory intakes for outpatient appointments, administer immunizations and blood draws, requisition medical equipment and supplies, serve as a pharmacy technician, complete physical examinations, and manage referrals, among other duties. Their familiarity with different aspects of clinic operations and medical practice must be broad. To that end, corpsmen develop and reinforce their medical knowledge through various trainings, including additional courses to specialize in certain medical skills, such as pharmacy technician “C” school or dental assistant “C” school.

The USCG employs < 15 field pharmacists, most of whom serve in an ambulatory care environment.⁴ Responsibilities of USCG pharmacists include the routine reinforcement of pharmacy knowledge with HSs. For the corpsmen who are not pharmacy technicians or who have not attended pharmacy technician “C” school, the extent of their pharmacy instruction primarily came from the “A” school curriculum, of which only 1 class is specific to pharmacy. Providing routine pharmacy-related training to the HSs further cultivates their pharmacy knowledge and confidence so that they can practice more holistically. These trainings do not need to follow any specific format.

In this study, 3 pharmacists at 3 separate USCG clinics conducted a training inspired by the Jeopardy! game show with the corpsmen at their respective clinics. This study examined the effectiveness of game-based learning on the pharmacy knowledge retention of HSs at 3 USCG clinics. A secondary objective was to evaluate the baseline pharmacy knowledge of corpsmen based on specific corpsmen demographics.

Methods

As part of a USCG quality improvement study in 2024, 28 HSs at the 3 USCG clinics were provided a preintervention assessment, completed game-based educational program (intervention), and then were assessed again following the intervention.

The HSs were presented with a 25-question assessment that included 10 knowledge questions (3 on over-the-counter medications, 2 on use of medications in pregnancy, 2 on precautions and contraindications, 2 on indications, and 1 on immunizations) and 15 brand-generic matching questions. These questions were developed and reviewed by the 3 participating pharmacists to ensure that their scope was commensurate with the overall pharmacy knowledge that could be reasonably expected of corpsmen spanning various points of their HS career.

One to 7 days after the preintervention assessment, the pharmacists hosted the game-based learning modeled after Jeopardy!. The Jeopardy! categories mirrored the assessment knowledge question categories, and brand-generic nomenclature was freely discussed throughout. About 2 weeks later, the same HSs who completed the preintervention assessment and participated in the game were presented with the same assessment.

In addition to capturing the difference in scores between the 2 assessments, additional demographic data were gathered, including service time as an HS and whether they received formalized pharmacy technician training and if so, how long they have served in that capacity. Demographic data were collected to identify potential correlations between demographic characteristics and results.

Results

Twenty-eight HSs at the 3 clinics completed the game-based training and both assessments. The mean score increased from 15.1 preintervention to 17.4 postintervention (Table). Preintervention scores ranged from 1 to 24 and postintervention scores ranged from 6 to 25.

There were 19 HSs (68%) whose score increased from preintervention to postintervention and 5 (18%) had decreased scores. The largest score decrease was 4 (from 18 to 14), and the largest score increase was 11 (from 13 to 24). The mean improvement was 3.9 among the 19 HSs with increased scores

Twenty-one HSs reported no formal pharmacy technician training, 3 completed pharmacy technician “C” school, and 4 received informal on-the-job training. The mean score for the “C” school trained HSs was 23.0 preintervention and 23.7 postintervention. The mean score for HSs trained on the job was 16.0 preintervention and 18.5 postintervention. The mean score for HSs with no training was 13.9 preintervention and 16.3 postintervention.

As HSs advance in their careers, they typically assume roles with increasing technical knowledge, responsibility, and oversight, thus aligning with advancement from E-4 (third class petty officer) to E-6 (first class petty officer) and beyond. In this study, there was 1 E-3, 12 E-4s (mean time as an HS, 1.3 years), 8 E-5s (mean time as an HS, 4.8 years), and 7 E-6s (mean time as an HS, 8.6 years). The E-3 had a preintervention score of 1.0 and a postintervention score of 6.0. The E-4s had a mean change in score from pre- to postintervention of 2.4. The E-5s had a mean change in score from pre- to postintervention of 1.6. The E-6s had a mean change in score from pre- to postintervention of 2.3.

Discussion

This study is novel in its examination of the impact of game-based learning on the retention of the pharmacy knowledge of USCG corpsmen. A PubMed literature search of the phrase “((Corpsman) OR (Corpsmen)) AND (Coast Guard)” yields 135 results, though none were relevant to the USCG population described in this study. A PubMed literature search of the phrase “(Jeopardy!) AND (pharmacy)” yields 28 results, only 1 of which discusses using the game-based approach as an instructional tool.⁵ A PubMed literature search of the phrase “(game) AND (Coast Guard)” yields 55 results, none of which were specifically relevant to game-based learning in the USCG. This study appears to be among the first to discuss results and trends in game-based learning with USCG corpsmen.

The preponderance of literature for game-based learning strategies exists in children; more research in adults is needed.^6,7 With studies showing that game-based learning may impact motivation to learn and learning gains, it is unsurprising that there is some research in professional health care education. Games modeled after everything from simulated clinical scenarios to Family Feud and Chutes and Ladders-style games have been compared with traditional learning strategies. However, the results of whether game-based learning strategies improve knowledge, clinical decision-making, and motivation to learn vary, suggesting the need for more research in this field.⁸

The results of this study suggest that Jeopardy! is likely an effective instructional method for USCG corpsmen on pharmacy topics. While there were some HSs whose postintervention scores decreased, 19 (68%) had increased scores. Because the second assessment was administered about 2 weeks after the game-based learning, the results suggest some level of knowledge retention. Between these results and the informally perceived level of engagement, game-based learning could be a more stimulating alternative training method to a standard slide-based presentation.

Stratifying the data by demographics revealed additional trends, although they should be interpreted with caution due to the small sample size. The baseline results strongly illustrate the value of formalized training. It is generally expected that HSs who have completed the “C” school pharmacy technician training program should have more pharmacy knowledge than those with on-the-job or less training. The results indicate that “C” school trained and on-the-job trained HSs scored higher on the preintervention assessment (mean, 23.0 and 16.0, respectively), than those with no such experiences (mean, 13.9). Such results underscore the value of formalized training—whether as a pharmacy technician or in any other “C” school—in enhancing the medical knowledge of HSs that may allow them to hold roles of increased responsibility and medical scope.

In addition to stratification by pharmacy technician training, stratification by years of HS experience (roughly correlated to rank) yields a similar result. It would be expected that as HSs advance in their careers, they gain more exposure to various medical topics, including pharmacy. That is not always the case, however, as it is possible an HS never rotated through a pharmacy technician position or has not been recently exposed to pharmacy knowledge. Nevertheless, the results suggest that increased HS experience was likely associated with an increased baseline pharmacy knowledge, with mean preintervention scores increasing from 11.9 to 18.1 to 19.3 for E-4, E-5, and E-6, respectively.

While there are many explanations for these results, the authors hypothesize that when HSs are E-4s, they might not yet have exposure to all aspects of the clinic and are perhaps not as well-versed in pharmacy practice. An E-5—now a few years into their career—would have completed pharmacy technician “C” school or on-the-job training (if applicable), which could account for the significant jump in pharmacy knowledge scores. An E-6 can still engage in direct patient care activities but take on leadership and supervisory roles within the clinic, perhaps explaining the smaller increase in score.

In terms of increasing responsibility, many USCG corpsmen complete another schooling opportunity—Independent Duty Health Services Technician (IDHS)—so they can serve in independent duty roles, many of which are on USCG cutters. While cutters are deployed, that IDHS could be the sole medical personnel on the cutter and function in a midlevel practitioner extender role. Formalized training in pharmacy—the benefits of which are suggested through these results—or another field of medical practice would strengthen the skillset and confidence of IDHSs.

Though not formally assessed, the 3 pharmacists noted that the game-based learning was met with overwhelmingly positive feedback in terms of excitement, energy, and overall engagement.

Limitations

This cohort of individuals represents a small proportion of the total number of USCG corpsmen, and it is not fully representative of all practice settings. HSs can be assigned to USCG cutters as IDHSs, which would not be captured in this cohort. Even within a single clinic, the knowledge of HSs varies, as not all HS duties consist solely of clinical skills. Additionally, while the overall game framework was consistent among the 3 sites, there may have been unquantifiable differences in overall teaching style by the 3 pharmacists that may have resulted in different levels of content retention. Given the lack of similar studies in this population, this study can best be described as a quantitative descriptor of results rather than a statistical comparison of what instructional method works best.

Conclusions

The USCG greatly benefits from having trained and experienced HSs fulfilling mission support roles in the organization. In addition to traditional slide-based trainings, game-based learning can be considered to create engaging learning environments to support the knowledge retention of pharmacy and other medical topics for USCG corpsmen.

The US Coast Guard (USCG) operates within the US Department of Homeland Security and represents a force of > 50,000 servicemembers.¹ The missions of the service include maritime law enforcement (drug interdiction), search and rescue, and defense readiness.²

The USCG operates 42 clinics and numerous smaller sick bays of varying sizes and medical capabilities throughout the country to provide acute and routine medical services. Health services technicians (HSs) are the most common staffing component and provide much of the support services in each USCG health care setting. The HS rating, colloquially referred to as corpsmen, is achieved through a 22-week course known as “A” school that trains servicemembers in outpatient and acute care, including emergency medical technician training.³ There are about 750 USCG HSs.

Within USCG clinics, HSs conduct ambulatory intakes for outpatient appointments, administer immunizations and blood draws, requisition medical equipment and supplies, serve as a pharmacy technician, complete physical examinations, and manage referrals, among other duties. Their familiarity with different aspects of clinic operations and medical practice must be broad. To that end, corpsmen develop and reinforce their medical knowledge through various trainings, including additional courses to specialize in certain medical skills, such as pharmacy technician “C” school or dental assistant “C” school.

The USCG employs < 15 field pharmacists, most of whom serve in an ambulatory care environment.⁴ Responsibilities of USCG pharmacists include the routine reinforcement of pharmacy knowledge with HSs. For the corpsmen who are not pharmacy technicians or who have not attended pharmacy technician “C” school, the extent of their pharmacy instruction primarily came from the “A” school curriculum, of which only 1 class is specific to pharmacy. Providing routine pharmacy-related training to the HSs further cultivates their pharmacy knowledge and confidence so that they can practice more holistically. These trainings do not need to follow any specific format.

In this study, 3 pharmacists at 3 separate USCG clinics conducted a training inspired by the Jeopardy! game show with the corpsmen at their respective clinics. This study examined the effectiveness of game-based learning on the pharmacy knowledge retention of HSs at 3 USCG clinics. A secondary objective was to evaluate the baseline pharmacy knowledge of corpsmen based on specific corpsmen demographics.

Methods

As part of a USCG quality improvement study in 2024, 28 HSs at the 3 USCG clinics were provided a preintervention assessment, completed game-based educational program (intervention), and then were assessed again following the intervention.

The HSs were presented with a 25-question assessment that included 10 knowledge questions (3 on over-the-counter medications, 2 on use of medications in pregnancy, 2 on precautions and contraindications, 2 on indications, and 1 on immunizations) and 15 brand-generic matching questions. These questions were developed and reviewed by the 3 participating pharmacists to ensure that their scope was commensurate with the overall pharmacy knowledge that could be reasonably expected of corpsmen spanning various points of their HS career.

One to 7 days after the preintervention assessment, the pharmacists hosted the game-based learning modeled after Jeopardy!. The Jeopardy! categories mirrored the assessment knowledge question categories, and brand-generic nomenclature was freely discussed throughout. About 2 weeks later, the same HSs who completed the preintervention assessment and participated in the game were presented with the same assessment.

In addition to capturing the difference in scores between the 2 assessments, additional demographic data were gathered, including service time as an HS and whether they received formalized pharmacy technician training and if so, how long they have served in that capacity. Demographic data were collected to identify potential correlations between demographic characteristics and results.

Results

Twenty-eight HSs at the 3 clinics completed the game-based training and both assessments. The mean score increased from 15.1 preintervention to 17.4 postintervention (Table). Preintervention scores ranged from 1 to 24 and postintervention scores ranged from 6 to 25.

There were 19 HSs (68%) whose score increased from preintervention to postintervention and 5 (18%) had decreased scores. The largest score decrease was 4 (from 18 to 14), and the largest score increase was 11 (from 13 to 24). The mean improvement was 3.9 among the 19 HSs with increased scores

Twenty-one HSs reported no formal pharmacy technician training, 3 completed pharmacy technician “C” school, and 4 received informal on-the-job training. The mean score for the “C” school trained HSs was 23.0 preintervention and 23.7 postintervention. The mean score for HSs trained on the job was 16.0 preintervention and 18.5 postintervention. The mean score for HSs with no training was 13.9 preintervention and 16.3 postintervention.

As HSs advance in their careers, they typically assume roles with increasing technical knowledge, responsibility, and oversight, thus aligning with advancement from E-4 (third class petty officer) to E-6 (first class petty officer) and beyond. In this study, there was 1 E-3, 12 E-4s (mean time as an HS, 1.3 years), 8 E-5s (mean time as an HS, 4.8 years), and 7 E-6s (mean time as an HS, 8.6 years). The E-3 had a preintervention score of 1.0 and a postintervention score of 6.0. The E-4s had a mean change in score from pre- to postintervention of 2.4. The E-5s had a mean change in score from pre- to postintervention of 1.6. The E-6s had a mean change in score from pre- to postintervention of 2.3.

Discussion

This study is novel in its examination of the impact of game-based learning on the retention of the pharmacy knowledge of USCG corpsmen. A PubMed literature search of the phrase “((Corpsman) OR (Corpsmen)) AND (Coast Guard)” yields 135 results, though none were relevant to the USCG population described in this study. A PubMed literature search of the phrase “(Jeopardy!) AND (pharmacy)” yields 28 results, only 1 of which discusses using the game-based approach as an instructional tool.⁵ A PubMed literature search of the phrase “(game) AND (Coast Guard)” yields 55 results, none of which were specifically relevant to game-based learning in the USCG. This study appears to be among the first to discuss results and trends in game-based learning with USCG corpsmen.

The preponderance of literature for game-based learning strategies exists in children; more research in adults is needed.^6,7 With studies showing that game-based learning may impact motivation to learn and learning gains, it is unsurprising that there is some research in professional health care education. Games modeled after everything from simulated clinical scenarios to Family Feud and Chutes and Ladders-style games have been compared with traditional learning strategies. However, the results of whether game-based learning strategies improve knowledge, clinical decision-making, and motivation to learn vary, suggesting the need for more research in this field.⁸

The results of this study suggest that Jeopardy! is likely an effective instructional method for USCG corpsmen on pharmacy topics. While there were some HSs whose postintervention scores decreased, 19 (68%) had increased scores. Because the second assessment was administered about 2 weeks after the game-based learning, the results suggest some level of knowledge retention. Between these results and the informally perceived level of engagement, game-based learning could be a more stimulating alternative training method to a standard slide-based presentation.

Stratifying the data by demographics revealed additional trends, although they should be interpreted with caution due to the small sample size. The baseline results strongly illustrate the value of formalized training. It is generally expected that HSs who have completed the “C” school pharmacy technician training program should have more pharmacy knowledge than those with on-the-job or less training. The results indicate that “C” school trained and on-the-job trained HSs scored higher on the preintervention assessment (mean, 23.0 and 16.0, respectively), than those with no such experiences (mean, 13.9). Such results underscore the value of formalized training—whether as a pharmacy technician or in any other “C” school—in enhancing the medical knowledge of HSs that may allow them to hold roles of increased responsibility and medical scope.

In addition to stratification by pharmacy technician training, stratification by years of HS experience (roughly correlated to rank) yields a similar result. It would be expected that as HSs advance in their careers, they gain more exposure to various medical topics, including pharmacy. That is not always the case, however, as it is possible an HS never rotated through a pharmacy technician position or has not been recently exposed to pharmacy knowledge. Nevertheless, the results suggest that increased HS experience was likely associated with an increased baseline pharmacy knowledge, with mean preintervention scores increasing from 11.9 to 18.1 to 19.3 for E-4, E-5, and E-6, respectively.

While there are many explanations for these results, the authors hypothesize that when HSs are E-4s, they might not yet have exposure to all aspects of the clinic and are perhaps not as well-versed in pharmacy practice. An E-5—now a few years into their career—would have completed pharmacy technician “C” school or on-the-job training (if applicable), which could account for the significant jump in pharmacy knowledge scores. An E-6 can still engage in direct patient care activities but take on leadership and supervisory roles within the clinic, perhaps explaining the smaller increase in score.

In terms of increasing responsibility, many USCG corpsmen complete another schooling opportunity—Independent Duty Health Services Technician (IDHS)—so they can serve in independent duty roles, many of which are on USCG cutters. While cutters are deployed, that IDHS could be the sole medical personnel on the cutter and function in a midlevel practitioner extender role. Formalized training in pharmacy—the benefits of which are suggested through these results—or another field of medical practice would strengthen the skillset and confidence of IDHSs.

Though not formally assessed, the 3 pharmacists noted that the game-based learning was met with overwhelmingly positive feedback in terms of excitement, energy, and overall engagement.

Limitations

This cohort of individuals represents a small proportion of the total number of USCG corpsmen, and it is not fully representative of all practice settings. HSs can be assigned to USCG cutters as IDHSs, which would not be captured in this cohort. Even within a single clinic, the knowledge of HSs varies, as not all HS duties consist solely of clinical skills. Additionally, while the overall game framework was consistent among the 3 sites, there may have been unquantifiable differences in overall teaching style by the 3 pharmacists that may have resulted in different levels of content retention. Given the lack of similar studies in this population, this study can best be described as a quantitative descriptor of results rather than a statistical comparison of what instructional method works best.

Conclusions

The USCG greatly benefits from having trained and experienced HSs fulfilling mission support roles in the organization. In addition to traditional slide-based trainings, game-based learning can be considered to create engaging learning environments to support the knowledge retention of pharmacy and other medical topics for USCG corpsmen.

Learning health systems (LHS) promote a continuous process that can assist in making sense of uncertainty when confronting emerging complex conditions such as Long COVID. Long COVID is an infection-associated chronic condition that detrimentally impacts veterans, their families, and the communities in which they live. This complex condition is defined by ongoing, new, or returning symptoms following COVID-19 infection that negatively affect return to meaningful participation in social, recreational, and vocational activities.^1,2 The clinical uncertainty surrounding Long COVID is amplified by unclear etiology, prognosis, and expected course of symptoms.^3,4 Uncertainty surrounding best clinical practices, processes, and policies for Long COVID care has resulted in practice variation despite the emerging evidence base for Long COVID care.⁴ Failure to address gaps in clinical evidence and care implementation threatens to perpetuate fragmented and unnecessary care.

The context surrounding Long COVID created an urgency to rapidly address clinically relevant questions and make sense of any uncertainty. Thus, the Veterans Health Administration (VHA) funded a Long COVID Practice-Based Research Network (LC-PBRN) to build an infrastructure that supports Long COVID research nationally and promotes interdisciplinary collaboration. The LC-PBRN vision is to centralize Long COVID clinical, research, and operational activities. The research infrastructure of the LC-PBRN is designed with an LHS lens to facilitate feedback loops and integrate knowledge learned while making progress towards this vision.⁵ This article describes the phases of infrastructure development and network building, as well as associated lessons learned.

Designing the LC-PBRN Infrastructure

The LC-PBRN is a multisite operation with interdisciplinary representatives from 4 US Department of Veterans Affairs (VA) health care systems. Each site has ≥ 1 principal investigator (0.1-0.4 full-time equivalent [FTE]) and ≥ 1 project staff member (0.5-0.8 FTE). The lead site also employs data and statistical support staff (1.5 FTE). To build this infrastructure, VHA Health Services Research awarded $1 million in November 2023 to the 4 sites. The funding was distributed over 2 years. Additional funding will be required for sustainability. The components and key infrastructure elements of the LC-PBRN are outlined in the Table. The 2-year LC-PBRN implementation activities is outlined in the Appendix.

Vision

The LC-PBRN’s vision is to create an infrastructure that integrates an LHS framework by unifying the VA research approach to Long COVID to ensure veteran, clinician, operational, and researcher involvement (Figure 1). A critical aspect of this is a unifying definition of Long COVID, for which the LC-PBRN has adopted the National Academies of Science, Engineering, and Medicine (NASEM) definition: “Long COVID is an infection-associated chronic condition that occurs after SARS-CoV-2 infection and is present for at least 3 months as a continuous, relapsing and remitting, or progressive disease state that affects one or more organ systems.”⁶ This is a working definition to be refined over time, as necessary, based on new data. The LC-PBRN aligns with existing VA initiatives by serving as a centralized hub for internal and external networking. This approach ensures shareholder needs are identified, resources are allocated appropriately, and redundancy in efforts is avoided. In this spirit, the LC-PBRN maintains a long-term vision of collaborating with other systems to support national efforts to address Long COVID.

Mission and Governance

The LC-PBRN operates with an executive leadership team and 5 cores. The executive leadership team is responsible for overall LC-PBRN operations, management, and direction setting of the LC-PBRN. The executive leadership team meets weekly to provide oversight of each core, which specializes in different aspects. The cores include: Administrative, Partner Engagement and Needs Assessment, Patient Identification and Analysis, Clinical Coordination and Implementation, and Dissemination (Figure 2).

The Administrative core focuses on interagency collaboration to identify and network with key operational and agency leaders to allow for ongoing exploration of funding strategies for Long COVID research. The Administrative core manages 3 teams: an advisory board, Long COVID council, and the strategic planning team. The advisory board meets biannually to oversee achievement of LC-PBRN goals, deliverables, and tactics for meeting these goals. The advisory board includes the LC-PBRN executive leadership team and 13 interagency members from various shareholders (eg, Centers for Disease Control and Prevention, National Institutes of Health, and specialty departments within the VA).

The Long COVID council convenes quarterly to provide scientific input on important overarching issues in Long COVID research, practice, and policy. The council consists of 22 scientific representatives in VA and non-VA contexts, university affiliates, and veteran representatives. The strategic planning team convenes annually to identify how the LC-PBRN and its partners can meet the needs of the broader Long COVID ecosystem and conduct a strengths, opportunities, weaknesses, and threats analysis to identify strategic objectives and expected outcomes. The strategic planning team includes the executive leadership team and key Long COVID shareholders within VHA and affiliated partners. The Partner Engagement and Needs Assessment core aims to solicit feedback from veterans, clinicians, researchers, and operational leadership. Input is gathered through a Veteran Engagement Panel and a modified Delphi consensus process. The panel was formed using a Community Engagement Studio model to engage veterans as consultants on research.⁷ Currently, 10 members represent a range of ages, genders, racial and ethnic backgrounds, and military experience. All veterans have a history of Long COVID and are paid as consultants. Video conference panel meetings occur quarterly for 1 to 2 hours; the meeting length is shorter than typical engagement studios to accommodate for fatigue-related symptoms that may limit attention and ability to participate in longer meetings. Before each panel, the Partner Engagement and Needs Assessment core helps identify key questions and creates a structured agenda. Each panel begins with a presentation of a research study followed by a group discussion led by a trained facilitator. The modified Delphi consensus process focuses on identifying research priority areas for Long COVID within the VA. Veterans living with Long COVID, as well as clinicians and researchers who work closely with patients who have Long COVID, complete a series of progressive surveys to provide input on research priorities.

The Partner Engagement and Needs Assessment core also actively provides outreach to important partners in research, clinical care, and operational leadership to facilitate introductory meetings to (1) ask partners to describe their 5 largest pain points, (2) find pain points within the scope of LC-PBRN resources, and (3) discuss the strengths and capacity of the PBRN. During introductory meetings, communications preferences and a cadence for subsequent meetings are established. Subsequent engagement meetings aim to provide updates and codevelop solutions to emerging issues. This core maintains a living document to track engagement efforts, points of contact for identified and emerging partners, and ensure all communication is timely.

The Patient Identification and Analysis core develops a database of veterans with confirmed or suspected Long COVID. The goal is for researchers to use the database to identify potential participants for clinical trials and monitor clinical care outcomes. When possible, this core works with existing VA data to facilitate research that aligns with the LC-PBRN mission. The core can also use natural language processing and machine learning to work with researchers conducting clinical trials to help identify patients who may meet eligibility criteria.

The Clinical Coordination and Implementation core gathers information on the best practices for identifying and recruiting veterans for Long COVID research as well as compiles strategies for standardized clinical assessments that can both facilitate ongoing research and the successful implementation of evidence-based care. The Clinical Coordination and Implementation core provides support to pilot and multisite trials in 3 ways. First, it develops toolkits such as best practice strategies for recruiting participants for research, template examples of recruitment materials, and a library of patient-reported outcome measures, standardized clinical note titles and templates in use for Long COVID in the national electronic health record. Second, it partners with the Patient Identification and Analysis core to facilitate access to and use of algorithms that identify Long COVID cases based on electronic health records for recruitment. Finally, it compiles a detailed list of potential collaborating sites. The steps to facilitate patient identification and recruitment inform feasibility assessments and improve efficiency of launching pilot studies and multisite trials. The library of outcome measures, standardized clinical notes, and templates can aid and expedite data collection.

The Dissemination core focuses on developing a website, creating a dissemination plan, and actively disseminating products of the LC-PBRN and its partners. This core’s foundational framework is based on the Agency for Healthcare Research and Quality Quick-Start Guide to Dissemination for PBRNs.^8,9 The core built an internal- and external-facing website to connect users with LC-PBRN products, potential outreach contacts, and promote timely updates on LC-PBRN activities. A manual of operating procedures will be drafted to include the development of training for practitioners involved in research projects to learn the processes involved in presenting clinical results for education and training initiatives, presentations, and manuscript preparation. A toolkit will also be developed to support dissemination activities designed to reach a variety of end-users, such as education materials, policy briefings, educational briefs, newsletters, and presentations at local, regional, and national levels.

Key Partners

Key partners exist specific to the LC-PBRN and within the broader VA ecosystem, including VA clinical operations, VA research, and intra-agency collaborations.

LC-PBRN Specific. In addition to the LC-PBRN council, advisory board, and Veteran Engagement Panel discussed earlier, the LC-PBRN has 8 VA Long COVID clinical sites that have joined the network. As part of the network, these sites gain greater insight into the Long COVID ecosystem within the VA through priority access to the Long COVID Veteran Engagement Panel and recognition as members of the network. The LC-PBRN also meets monthly with pilot projects conducted at other VA facilities to learn more about how Long COVID research is being implemented and identify how the LC-PBRN can assist in troubleshooting barriers.

VA Clinical Operations. To support clinical operations, a Long COVID Field Advisory Board was formed through the VA Office of Specialty Care as an operational effort to develop clinical best practice. The LC-PBRN consults with this group on veteran engagement strategies for input on clinical guides and dissemination of practice guide materials. The LC-PBRN also partners with an existing Long COVID Community of Practice and the Office of Primary Care. The Community of Practice provides a learning space for VA staff interested in advancing Long COVID care and assists with disseminating LC-PBRN to the broader Long COVID clinical community. A member of the Office of Primary Care sits on the PBRN advisory board to provide input on engaging primary care practitioners and ensure their unique needs are considered in LC-PBRN initiatives.

VA Research & Interagency Collaborations. The LC-PBRN engages monthly with an interagency workgroup led by the US Department of Health and Human Services Office of Long COVID Research and Practice. These engagements support identification of research gaps that the VA may help address, monitor emerging funding opportunities, and foster collaborations. LC-PBRN representatives also meet with staff at the National Institutes of Health Researching COVID to Enhance Recovery initiative to identify pathways for veteran recruitment.

LHS Feedback Loops

The LC-PBRN was designed with an LHS approach in mind.¹⁰ Throughout development of the LC-PBRN, consideration was given to (1) capture data on new efforts within the Long COVID ecosystem (performance to data), (2) examine performance gaps and identify approaches for best practice (data to knowledge), and (3) implement best practices, develop toolkits, disseminate findings, and measure impacts (knowledge to performance). With this approach, the LC-PBRN is constantly evolving based on new information coming from the internal and external Long COVID ecosystem. Each element was deliberatively considered in relation to how data can be transformed into knowledge, knowledge into performance, and performance into data.

First, an important mechanism for feedback involves establishing clear channels of communication. Regular check-ins with key partners occur through virtual meetings to provide updates, assess needs and challenges, and codevelop action plans. For example, during a check-in with the Long COVID Field Advisory Board, members expressed a desire to incorporate veteran feedback into VA clinical practice recommendations. We provided expertise on different engagement modalities (eg, focus groups vs individual interviews), and collaboration occurred to identify key interview questions for veterans. This process resulted in a published clinician-facing Long COVID Nervous System Clinical Guide (available at longcovid@hhs.gov) that integrated critical feedback from veterans related to neurological symptoms.

Second, weekly executive leadership meetings include dedicated time for reflection on partner feedback, the current state of Long COVID, and contextual changes that impact deliverable priorities and timelines. Outcomes from these discussions are communicated with VHA Health Services Research and, when appropriate, to key partners to ensure alignment. For example, the Patient Identification and Analysis core was originally tasked with identifying a definition of Long COVID. However, as the broader community moved away from a singular definition, efforts were redirected toward higher-priority issues within the VA Long COVID ecosystem, including veteran enrollment in clinical trials.

Third, the Veteran Engagement Panel captures feedback from those with lived experience to inform Long COVID research and clinical efforts. The panel meetings are strategically designed to ask veterans living with Long COVID specific questions related to a given research or clinical topic of interest. For example, panel sessions with the Field Advisory Board focused on concerns articulated by veterans related to the mental health and gastroenterological symptoms associated with Long COVID. Insights from these discussions will inform development of Long COVID mental health and gastroenterological clinical care guides, with several PBRN investigators serving as subject matter experts. This collaborative approach ensures that veteran perspectives are represented in developing Long COVID clinical care processes.

Fourth, research priorities identified through the Delphi consensus process will inform development of VA Request for Funding Proposals related to Long COVID. The initial survey was developed in collaboration with veterans, clinicians, and researchers across the Veteran Engagement Panel, the Field Advisory Board, and the National Research Action Plan on Long COVID.¹¹ The process was launched in October 2024 and concluded in June 2025. The team conducted 3 consensus rounds with veterans and VA clinicians and researchers. Top priority areas included the testing assessments for diagnosing Long COVID, studying subtypes of Long COVID and treatments for each, and finding biomarkers for Long COVID. A formal publication of the results and analysis is the focus of a future publication.

Fifth, ongoing engagement with the Field Advisory Board has supported adoption of a preliminary set of clinical outcome measures. If universally adopted, these instruments may contribute to the development of a standardized data collection process and serve as common data elements collected for epidemiologic, health services, or clinical trial research.

Lessons Learned and Practice Implications

Throughout the development of the LC-PBRN, several decisions were identified that have impacted infrastructure development and implementation.

Include veterans’ voices to ensure network efforts align with patient needs. Given the novelty of Long COVID, practitioners and researchers are learning as they go. It is important to listen to individuals who live with Long COVID. Throughout the development of the LC-PBRN, veteran perspective has proven how vital it is for them to be heard when it comes to their health care. Clinicians similarly highlighted the value of incorporating patient perspectives into the development of tools and treatment strategies. Develop an interdisciplinary leadership team to foster the diverse viewpoints needed to tackle multifaceted problems. It is important to consider as many clinical and research perspectives as possible because Long COVID is a complex condition with symptoms impacting major organ systems.^12-15 Therefore, the team spans across a multitude of specialties and locations.

Set clear expectations and goals with partners to uphold timely deliverables and stay within the PBRN’s capacity. When including a multitude of partners, teams should consider each of those partners’ experiences and opinions in decision-making conversations. Expectation setting is important to ensure all partners are on the same page and understand the capacity of the LC-PBRN. This allows the team to focus its efforts, avoid being overwhelmed with requests, and provide quality deliverables.

Build engaging relationships to bridge gaps between internal and external partners. A substantial number of resources focus on building relationships with partners so they can trust the LC-PBRN has their best interests in mind. These relationships are important to ensure the VA avoids duplicate efforts. This includes prioritizing connecting partners who are working on similar efforts to promote collaboration across facilities.

Clinical practice implications. The LC-PBRN is working towards clinical practice initiatives derived from this process in partnership with the Long COVID Community of Practice and the participating clinical sites. This may include efforts to increase the uptake of standardized instruments endorsed by clinical partners that facilitate assessment of outcomes. PBRN partners can then use outcomes data to ask and answer clinically relevant research questions and assess care quality to inform the learning process that is integral to an LHS. Future dissemination efforts will be centered around individual initiatives and deliverables from the LC-PBRN.

Conclusions

PBRNs provide an important mechanism to use LHS approaches to successfully convene research around complex issues. PBRNs can support integration across the LHS cycle, allowing for multiple feedback loops, and coordinate activities that work to achieve a larger vision. PBRNs offer centralized mechanisms to collaboratively understand and address complex problems, such as Long COVID, where the uncertainty regarding how to treat occurs in tandem with the urgency to treat. The LC-PBRN model described in this article has the potential to transcend Long COVID by building infrastructure necessary to proactively address current or future clinical conditions or populations with a LHS lens. The infrastructure can require cross-system and sector collaborations, expediency, inclusivity, and patient- and family-centeredness. Future efforts will focus on building out a larger network of VHA sites, facilitating recruitment at site and veteran levels into Long COVID trials through case identification, and systematically support the standardization of clinical data for clinical utility and evaluation of quality and/or outcomes across the VHA.

Learning health systems (LHS) promote a continuous process that can assist in making sense of uncertainty when confronting emerging complex conditions such as Long COVID. Long COVID is an infection-associated chronic condition that detrimentally impacts veterans, their families, and the communities in which they live. This complex condition is defined by ongoing, new, or returning symptoms following COVID-19 infection that negatively affect return to meaningful participation in social, recreational, and vocational activities.^1,2 The clinical uncertainty surrounding Long COVID is amplified by unclear etiology, prognosis, and expected course of symptoms.^3,4 Uncertainty surrounding best clinical practices, processes, and policies for Long COVID care has resulted in practice variation despite the emerging evidence base for Long COVID care.⁴ Failure to address gaps in clinical evidence and care implementation threatens to perpetuate fragmented and unnecessary care.

The context surrounding Long COVID created an urgency to rapidly address clinically relevant questions and make sense of any uncertainty. Thus, the Veterans Health Administration (VHA) funded a Long COVID Practice-Based Research Network (LC-PBRN) to build an infrastructure that supports Long COVID research nationally and promotes interdisciplinary collaboration. The LC-PBRN vision is to centralize Long COVID clinical, research, and operational activities. The research infrastructure of the LC-PBRN is designed with an LHS lens to facilitate feedback loops and integrate knowledge learned while making progress towards this vision.⁵ This article describes the phases of infrastructure development and network building, as well as associated lessons learned.

Designing the LC-PBRN Infrastructure

The LC-PBRN is a multisite operation with interdisciplinary representatives from 4 US Department of Veterans Affairs (VA) health care systems. Each site has ≥ 1 principal investigator (0.1-0.4 full-time equivalent [FTE]) and ≥ 1 project staff member (0.5-0.8 FTE). The lead site also employs data and statistical support staff (1.5 FTE). To build this infrastructure, VHA Health Services Research awarded $1 million in November 2023 to the 4 sites. The funding was distributed over 2 years. Additional funding will be required for sustainability. The components and key infrastructure elements of the LC-PBRN are outlined in the Table. The 2-year LC-PBRN implementation activities is outlined in the Appendix.

Vision

The LC-PBRN’s vision is to create an infrastructure that integrates an LHS framework by unifying the VA research approach to Long COVID to ensure veteran, clinician, operational, and researcher involvement (Figure 1). A critical aspect of this is a unifying definition of Long COVID, for which the LC-PBRN has adopted the National Academies of Science, Engineering, and Medicine (NASEM) definition: “Long COVID is an infection-associated chronic condition that occurs after SARS-CoV-2 infection and is present for at least 3 months as a continuous, relapsing and remitting, or progressive disease state that affects one or more organ systems.”⁶ This is a working definition to be refined over time, as necessary, based on new data. The LC-PBRN aligns with existing VA initiatives by serving as a centralized hub for internal and external networking. This approach ensures shareholder needs are identified, resources are allocated appropriately, and redundancy in efforts is avoided. In this spirit, the LC-PBRN maintains a long-term vision of collaborating with other systems to support national efforts to address Long COVID.

Mission and Governance

The LC-PBRN operates with an executive leadership team and 5 cores. The executive leadership team is responsible for overall LC-PBRN operations, management, and direction setting of the LC-PBRN. The executive leadership team meets weekly to provide oversight of each core, which specializes in different aspects. The cores include: Administrative, Partner Engagement and Needs Assessment, Patient Identification and Analysis, Clinical Coordination and Implementation, and Dissemination (Figure 2).

The Administrative core focuses on interagency collaboration to identify and network with key operational and agency leaders to allow for ongoing exploration of funding strategies for Long COVID research. The Administrative core manages 3 teams: an advisory board, Long COVID council, and the strategic planning team. The advisory board meets biannually to oversee achievement of LC-PBRN goals, deliverables, and tactics for meeting these goals. The advisory board includes the LC-PBRN executive leadership team and 13 interagency members from various shareholders (eg, Centers for Disease Control and Prevention, National Institutes of Health, and specialty departments within the VA).

The Long COVID council convenes quarterly to provide scientific input on important overarching issues in Long COVID research, practice, and policy. The council consists of 22 scientific representatives in VA and non-VA contexts, university affiliates, and veteran representatives. The strategic planning team convenes annually to identify how the LC-PBRN and its partners can meet the needs of the broader Long COVID ecosystem and conduct a strengths, opportunities, weaknesses, and threats analysis to identify strategic objectives and expected outcomes. The strategic planning team includes the executive leadership team and key Long COVID shareholders within VHA and affiliated partners. The Partner Engagement and Needs Assessment core aims to solicit feedback from veterans, clinicians, researchers, and operational leadership. Input is gathered through a Veteran Engagement Panel and a modified Delphi consensus process. The panel was formed using a Community Engagement Studio model to engage veterans as consultants on research.⁷ Currently, 10 members represent a range of ages, genders, racial and ethnic backgrounds, and military experience. All veterans have a history of Long COVID and are paid as consultants. Video conference panel meetings occur quarterly for 1 to 2 hours; the meeting length is shorter than typical engagement studios to accommodate for fatigue-related symptoms that may limit attention and ability to participate in longer meetings. Before each panel, the Partner Engagement and Needs Assessment core helps identify key questions and creates a structured agenda. Each panel begins with a presentation of a research study followed by a group discussion led by a trained facilitator. The modified Delphi consensus process focuses on identifying research priority areas for Long COVID within the VA. Veterans living with Long COVID, as well as clinicians and researchers who work closely with patients who have Long COVID, complete a series of progressive surveys to provide input on research priorities.

The Partner Engagement and Needs Assessment core also actively provides outreach to important partners in research, clinical care, and operational leadership to facilitate introductory meetings to (1) ask partners to describe their 5 largest pain points, (2) find pain points within the scope of LC-PBRN resources, and (3) discuss the strengths and capacity of the PBRN. During introductory meetings, communications preferences and a cadence for subsequent meetings are established. Subsequent engagement meetings aim to provide updates and codevelop solutions to emerging issues. This core maintains a living document to track engagement efforts, points of contact for identified and emerging partners, and ensure all communication is timely.

The Patient Identification and Analysis core develops a database of veterans with confirmed or suspected Long COVID. The goal is for researchers to use the database to identify potential participants for clinical trials and monitor clinical care outcomes. When possible, this core works with existing VA data to facilitate research that aligns with the LC-PBRN mission. The core can also use natural language processing and machine learning to work with researchers conducting clinical trials to help identify patients who may meet eligibility criteria.

The Clinical Coordination and Implementation core gathers information on the best practices for identifying and recruiting veterans for Long COVID research as well as compiles strategies for standardized clinical assessments that can both facilitate ongoing research and the successful implementation of evidence-based care. The Clinical Coordination and Implementation core provides support to pilot and multisite trials in 3 ways. First, it develops toolkits such as best practice strategies for recruiting participants for research, template examples of recruitment materials, and a library of patient-reported outcome measures, standardized clinical note titles and templates in use for Long COVID in the national electronic health record. Second, it partners with the Patient Identification and Analysis core to facilitate access to and use of algorithms that identify Long COVID cases based on electronic health records for recruitment. Finally, it compiles a detailed list of potential collaborating sites. The steps to facilitate patient identification and recruitment inform feasibility assessments and improve efficiency of launching pilot studies and multisite trials. The library of outcome measures, standardized clinical notes, and templates can aid and expedite data collection.

The Dissemination core focuses on developing a website, creating a dissemination plan, and actively disseminating products of the LC-PBRN and its partners. This core’s foundational framework is based on the Agency for Healthcare Research and Quality Quick-Start Guide to Dissemination for PBRNs.^8,9 The core built an internal- and external-facing website to connect users with LC-PBRN products, potential outreach contacts, and promote timely updates on LC-PBRN activities. A manual of operating procedures will be drafted to include the development of training for practitioners involved in research projects to learn the processes involved in presenting clinical results for education and training initiatives, presentations, and manuscript preparation. A toolkit will also be developed to support dissemination activities designed to reach a variety of end-users, such as education materials, policy briefings, educational briefs, newsletters, and presentations at local, regional, and national levels.

Key Partners

Key partners exist specific to the LC-PBRN and within the broader VA ecosystem, including VA clinical operations, VA research, and intra-agency collaborations.

LC-PBRN Specific. In addition to the LC-PBRN council, advisory board, and Veteran Engagement Panel discussed earlier, the LC-PBRN has 8 VA Long COVID clinical sites that have joined the network. As part of the network, these sites gain greater insight into the Long COVID ecosystem within the VA through priority access to the Long COVID Veteran Engagement Panel and recognition as members of the network. The LC-PBRN also meets monthly with pilot projects conducted at other VA facilities to learn more about how Long COVID research is being implemented and identify how the LC-PBRN can assist in troubleshooting barriers.

VA Clinical Operations. To support clinical operations, a Long COVID Field Advisory Board was formed through the VA Office of Specialty Care as an operational effort to develop clinical best practice. The LC-PBRN consults with this group on veteran engagement strategies for input on clinical guides and dissemination of practice guide materials. The LC-PBRN also partners with an existing Long COVID Community of Practice and the Office of Primary Care. The Community of Practice provides a learning space for VA staff interested in advancing Long COVID care and assists with disseminating LC-PBRN to the broader Long COVID clinical community. A member of the Office of Primary Care sits on the PBRN advisory board to provide input on engaging primary care practitioners and ensure their unique needs are considered in LC-PBRN initiatives.

VA Research & Interagency Collaborations. The LC-PBRN engages monthly with an interagency workgroup led by the US Department of Health and Human Services Office of Long COVID Research and Practice. These engagements support identification of research gaps that the VA may help address, monitor emerging funding opportunities, and foster collaborations. LC-PBRN representatives also meet with staff at the National Institutes of Health Researching COVID to Enhance Recovery initiative to identify pathways for veteran recruitment.

LHS Feedback Loops

The LC-PBRN was designed with an LHS approach in mind.¹⁰ Throughout development of the LC-PBRN, consideration was given to (1) capture data on new efforts within the Long COVID ecosystem (performance to data), (2) examine performance gaps and identify approaches for best practice (data to knowledge), and (3) implement best practices, develop toolkits, disseminate findings, and measure impacts (knowledge to performance). With this approach, the LC-PBRN is constantly evolving based on new information coming from the internal and external Long COVID ecosystem. Each element was deliberatively considered in relation to how data can be transformed into knowledge, knowledge into performance, and performance into data.

First, an important mechanism for feedback involves establishing clear channels of communication. Regular check-ins with key partners occur through virtual meetings to provide updates, assess needs and challenges, and codevelop action plans. For example, during a check-in with the Long COVID Field Advisory Board, members expressed a desire to incorporate veteran feedback into VA clinical practice recommendations. We provided expertise on different engagement modalities (eg, focus groups vs individual interviews), and collaboration occurred to identify key interview questions for veterans. This process resulted in a published clinician-facing Long COVID Nervous System Clinical Guide (available at longcovid@hhs.gov) that integrated critical feedback from veterans related to neurological symptoms.

Second, weekly executive leadership meetings include dedicated time for reflection on partner feedback, the current state of Long COVID, and contextual changes that impact deliverable priorities and timelines. Outcomes from these discussions are communicated with VHA Health Services Research and, when appropriate, to key partners to ensure alignment. For example, the Patient Identification and Analysis core was originally tasked with identifying a definition of Long COVID. However, as the broader community moved away from a singular definition, efforts were redirected toward higher-priority issues within the VA Long COVID ecosystem, including veteran enrollment in clinical trials.

Third, the Veteran Engagement Panel captures feedback from those with lived experience to inform Long COVID research and clinical efforts. The panel meetings are strategically designed to ask veterans living with Long COVID specific questions related to a given research or clinical topic of interest. For example, panel sessions with the Field Advisory Board focused on concerns articulated by veterans related to the mental health and gastroenterological symptoms associated with Long COVID. Insights from these discussions will inform development of Long COVID mental health and gastroenterological clinical care guides, with several PBRN investigators serving as subject matter experts. This collaborative approach ensures that veteran perspectives are represented in developing Long COVID clinical care processes.

Fourth, research priorities identified through the Delphi consensus process will inform development of VA Request for Funding Proposals related to Long COVID. The initial survey was developed in collaboration with veterans, clinicians, and researchers across the Veteran Engagement Panel, the Field Advisory Board, and the National Research Action Plan on Long COVID.¹¹ The process was launched in October 2024 and concluded in June 2025. The team conducted 3 consensus rounds with veterans and VA clinicians and researchers. Top priority areas included the testing assessments for diagnosing Long COVID, studying subtypes of Long COVID and treatments for each, and finding biomarkers for Long COVID. A formal publication of the results and analysis is the focus of a future publication.

Fifth, ongoing engagement with the Field Advisory Board has supported adoption of a preliminary set of clinical outcome measures. If universally adopted, these instruments may contribute to the development of a standardized data collection process and serve as common data elements collected for epidemiologic, health services, or clinical trial research.

Lessons Learned and Practice Implications

Throughout the development of the LC-PBRN, several decisions were identified that have impacted infrastructure development and implementation.

Include veterans’ voices to ensure network efforts align with patient needs. Given the novelty of Long COVID, practitioners and researchers are learning as they go. It is important to listen to individuals who live with Long COVID. Throughout the development of the LC-PBRN, veteran perspective has proven how vital it is for them to be heard when it comes to their health care. Clinicians similarly highlighted the value of incorporating patient perspectives into the development of tools and treatment strategies. Develop an interdisciplinary leadership team to foster the diverse viewpoints needed to tackle multifaceted problems. It is important to consider as many clinical and research perspectives as possible because Long COVID is a complex condition with symptoms impacting major organ systems.^12-15 Therefore, the team spans across a multitude of specialties and locations.

Set clear expectations and goals with partners to uphold timely deliverables and stay within the PBRN’s capacity. When including a multitude of partners, teams should consider each of those partners’ experiences and opinions in decision-making conversations. Expectation setting is important to ensure all partners are on the same page and understand the capacity of the LC-PBRN. This allows the team to focus its efforts, avoid being overwhelmed with requests, and provide quality deliverables.

Build engaging relationships to bridge gaps between internal and external partners. A substantial number of resources focus on building relationships with partners so they can trust the LC-PBRN has their best interests in mind. These relationships are important to ensure the VA avoids duplicate efforts. This includes prioritizing connecting partners who are working on similar efforts to promote collaboration across facilities.

Clinical practice implications. The LC-PBRN is working towards clinical practice initiatives derived from this process in partnership with the Long COVID Community of Practice and the participating clinical sites. This may include efforts to increase the uptake of standardized instruments endorsed by clinical partners that facilitate assessment of outcomes. PBRN partners can then use outcomes data to ask and answer clinically relevant research questions and assess care quality to inform the learning process that is integral to an LHS. Future dissemination efforts will be centered around individual initiatives and deliverables from the LC-PBRN.

Conclusions

PBRNs provide an important mechanism to use LHS approaches to successfully convene research around complex issues. PBRNs can support integration across the LHS cycle, allowing for multiple feedback loops, and coordinate activities that work to achieve a larger vision. PBRNs offer centralized mechanisms to collaboratively understand and address complex problems, such as Long COVID, where the uncertainty regarding how to treat occurs in tandem with the urgency to treat. The LC-PBRN model described in this article has the potential to transcend Long COVID by building infrastructure necessary to proactively address current or future clinical conditions or populations with a LHS lens. The infrastructure can require cross-system and sector collaborations, expediency, inclusivity, and patient- and family-centeredness. Future efforts will focus on building out a larger network of VHA sites, facilitating recruitment at site and veteran levels into Long COVID trials through case identification, and systematically support the standardization of clinical data for clinical utility and evaluation of quality and/or outcomes across the VHA.

Learning health systems (LHS) promote a continuous process that can assist in making sense of uncertainty when confronting emerging complex conditions such as Long COVID. Long COVID is an infection-associated chronic condition that detrimentally impacts veterans, their families, and the communities in which they live. This complex condition is defined by ongoing, new, or returning symptoms following COVID-19 infection that negatively affect return to meaningful participation in social, recreational, and vocational activities.^1,2 The clinical uncertainty surrounding Long COVID is amplified by unclear etiology, prognosis, and expected course of symptoms.^3,4 Uncertainty surrounding best clinical practices, processes, and policies for Long COVID care has resulted in practice variation despite the emerging evidence base for Long COVID care.⁴ Failure to address gaps in clinical evidence and care implementation threatens to perpetuate fragmented and unnecessary care.

The context surrounding Long COVID created an urgency to rapidly address clinically relevant questions and make sense of any uncertainty. Thus, the Veterans Health Administration (VHA) funded a Long COVID Practice-Based Research Network (LC-PBRN) to build an infrastructure that supports Long COVID research nationally and promotes interdisciplinary collaboration. The LC-PBRN vision is to centralize Long COVID clinical, research, and operational activities. The research infrastructure of the LC-PBRN is designed with an LHS lens to facilitate feedback loops and integrate knowledge learned while making progress towards this vision.⁵ This article describes the phases of infrastructure development and network building, as well as associated lessons learned.

Designing the LC-PBRN Infrastructure

The LC-PBRN is a multisite operation with interdisciplinary representatives from 4 US Department of Veterans Affairs (VA) health care systems. Each site has ≥ 1 principal investigator (0.1-0.4 full-time equivalent [FTE]) and ≥ 1 project staff member (0.5-0.8 FTE). The lead site also employs data and statistical support staff (1.5 FTE). To build this infrastructure, VHA Health Services Research awarded $1 million in November 2023 to the 4 sites. The funding was distributed over 2 years. Additional funding will be required for sustainability. The components and key infrastructure elements of the LC-PBRN are outlined in the Table. The 2-year LC-PBRN implementation activities is outlined in the Appendix.

Vision

The LC-PBRN’s vision is to create an infrastructure that integrates an LHS framework by unifying the VA research approach to Long COVID to ensure veteran, clinician, operational, and researcher involvement (Figure 1). A critical aspect of this is a unifying definition of Long COVID, for which the LC-PBRN has adopted the National Academies of Science, Engineering, and Medicine (NASEM) definition: “Long COVID is an infection-associated chronic condition that occurs after SARS-CoV-2 infection and is present for at least 3 months as a continuous, relapsing and remitting, or progressive disease state that affects one or more organ systems.”⁶ This is a working definition to be refined over time, as necessary, based on new data. The LC-PBRN aligns with existing VA initiatives by serving as a centralized hub for internal and external networking. This approach ensures shareholder needs are identified, resources are allocated appropriately, and redundancy in efforts is avoided. In this spirit, the LC-PBRN maintains a long-term vision of collaborating with other systems to support national efforts to address Long COVID.

Mission and Governance

The LC-PBRN operates with an executive leadership team and 5 cores. The executive leadership team is responsible for overall LC-PBRN operations, management, and direction setting of the LC-PBRN. The executive leadership team meets weekly to provide oversight of each core, which specializes in different aspects. The cores include: Administrative, Partner Engagement and Needs Assessment, Patient Identification and Analysis, Clinical Coordination and Implementation, and Dissemination (Figure 2).

The Administrative core focuses on interagency collaboration to identify and network with key operational and agency leaders to allow for ongoing exploration of funding strategies for Long COVID research. The Administrative core manages 3 teams: an advisory board, Long COVID council, and the strategic planning team. The advisory board meets biannually to oversee achievement of LC-PBRN goals, deliverables, and tactics for meeting these goals. The advisory board includes the LC-PBRN executive leadership team and 13 interagency members from various shareholders (eg, Centers for Disease Control and Prevention, National Institutes of Health, and specialty departments within the VA).

The Long COVID council convenes quarterly to provide scientific input on important overarching issues in Long COVID research, practice, and policy. The council consists of 22 scientific representatives in VA and non-VA contexts, university affiliates, and veteran representatives. The strategic planning team convenes annually to identify how the LC-PBRN and its partners can meet the needs of the broader Long COVID ecosystem and conduct a strengths, opportunities, weaknesses, and threats analysis to identify strategic objectives and expected outcomes. The strategic planning team includes the executive leadership team and key Long COVID shareholders within VHA and affiliated partners. The Partner Engagement and Needs Assessment core aims to solicit feedback from veterans, clinicians, researchers, and operational leadership. Input is gathered through a Veteran Engagement Panel and a modified Delphi consensus process. The panel was formed using a Community Engagement Studio model to engage veterans as consultants on research.⁷ Currently, 10 members represent a range of ages, genders, racial and ethnic backgrounds, and military experience. All veterans have a history of Long COVID and are paid as consultants. Video conference panel meetings occur quarterly for 1 to 2 hours; the meeting length is shorter than typical engagement studios to accommodate for fatigue-related symptoms that may limit attention and ability to participate in longer meetings. Before each panel, the Partner Engagement and Needs Assessment core helps identify key questions and creates a structured agenda. Each panel begins with a presentation of a research study followed by a group discussion led by a trained facilitator. The modified Delphi consensus process focuses on identifying research priority areas for Long COVID within the VA. Veterans living with Long COVID, as well as clinicians and researchers who work closely with patients who have Long COVID, complete a series of progressive surveys to provide input on research priorities.

The Partner Engagement and Needs Assessment core also actively provides outreach to important partners in research, clinical care, and operational leadership to facilitate introductory meetings to (1) ask partners to describe their 5 largest pain points, (2) find pain points within the scope of LC-PBRN resources, and (3) discuss the strengths and capacity of the PBRN. During introductory meetings, communications preferences and a cadence for subsequent meetings are established. Subsequent engagement meetings aim to provide updates and codevelop solutions to emerging issues. This core maintains a living document to track engagement efforts, points of contact for identified and emerging partners, and ensure all communication is timely.

The Patient Identification and Analysis core develops a database of veterans with confirmed or suspected Long COVID. The goal is for researchers to use the database to identify potential participants for clinical trials and monitor clinical care outcomes. When possible, this core works with existing VA data to facilitate research that aligns with the LC-PBRN mission. The core can also use natural language processing and machine learning to work with researchers conducting clinical trials to help identify patients who may meet eligibility criteria.

The Clinical Coordination and Implementation core gathers information on the best practices for identifying and recruiting veterans for Long COVID research as well as compiles strategies for standardized clinical assessments that can both facilitate ongoing research and the successful implementation of evidence-based care. The Clinical Coordination and Implementation core provides support to pilot and multisite trials in 3 ways. First, it develops toolkits such as best practice strategies for recruiting participants for research, template examples of recruitment materials, and a library of patient-reported outcome measures, standardized clinical note titles and templates in use for Long COVID in the national electronic health record. Second, it partners with the Patient Identification and Analysis core to facilitate access to and use of algorithms that identify Long COVID cases based on electronic health records for recruitment. Finally, it compiles a detailed list of potential collaborating sites. The steps to facilitate patient identification and recruitment inform feasibility assessments and improve efficiency of launching pilot studies and multisite trials. The library of outcome measures, standardized clinical notes, and templates can aid and expedite data collection.

The Dissemination core focuses on developing a website, creating a dissemination plan, and actively disseminating products of the LC-PBRN and its partners. This core’s foundational framework is based on the Agency for Healthcare Research and Quality Quick-Start Guide to Dissemination for PBRNs.^8,9 The core built an internal- and external-facing website to connect users with LC-PBRN products, potential outreach contacts, and promote timely updates on LC-PBRN activities. A manual of operating procedures will be drafted to include the development of training for practitioners involved in research projects to learn the processes involved in presenting clinical results for education and training initiatives, presentations, and manuscript preparation. A toolkit will also be developed to support dissemination activities designed to reach a variety of end-users, such as education materials, policy briefings, educational briefs, newsletters, and presentations at local, regional, and national levels.

Key Partners

Key partners exist specific to the LC-PBRN and within the broader VA ecosystem, including VA clinical operations, VA research, and intra-agency collaborations.

LC-PBRN Specific. In addition to the LC-PBRN council, advisory board, and Veteran Engagement Panel discussed earlier, the LC-PBRN has 8 VA Long COVID clinical sites that have joined the network. As part of the network, these sites gain greater insight into the Long COVID ecosystem within the VA through priority access to the Long COVID Veteran Engagement Panel and recognition as members of the network. The LC-PBRN also meets monthly with pilot projects conducted at other VA facilities to learn more about how Long COVID research is being implemented and identify how the LC-PBRN can assist in troubleshooting barriers.

VA Clinical Operations. To support clinical operations, a Long COVID Field Advisory Board was formed through the VA Office of Specialty Care as an operational effort to develop clinical best practice. The LC-PBRN consults with this group on veteran engagement strategies for input on clinical guides and dissemination of practice guide materials. The LC-PBRN also partners with an existing Long COVID Community of Practice and the Office of Primary Care. The Community of Practice provides a learning space for VA staff interested in advancing Long COVID care and assists with disseminating LC-PBRN to the broader Long COVID clinical community. A member of the Office of Primary Care sits on the PBRN advisory board to provide input on engaging primary care practitioners and ensure their unique needs are considered in LC-PBRN initiatives.

VA Research & Interagency Collaborations. The LC-PBRN engages monthly with an interagency workgroup led by the US Department of Health and Human Services Office of Long COVID Research and Practice. These engagements support identification of research gaps that the VA may help address, monitor emerging funding opportunities, and foster collaborations. LC-PBRN representatives also meet with staff at the National Institutes of Health Researching COVID to Enhance Recovery initiative to identify pathways for veteran recruitment.

LHS Feedback Loops

The LC-PBRN was designed with an LHS approach in mind.¹⁰ Throughout development of the LC-PBRN, consideration was given to (1) capture data on new efforts within the Long COVID ecosystem (performance to data), (2) examine performance gaps and identify approaches for best practice (data to knowledge), and (3) implement best practices, develop toolkits, disseminate findings, and measure impacts (knowledge to performance). With this approach, the LC-PBRN is constantly evolving based on new information coming from the internal and external Long COVID ecosystem. Each element was deliberatively considered in relation to how data can be transformed into knowledge, knowledge into performance, and performance into data.

First, an important mechanism for feedback involves establishing clear channels of communication. Regular check-ins with key partners occur through virtual meetings to provide updates, assess needs and challenges, and codevelop action plans. For example, during a check-in with the Long COVID Field Advisory Board, members expressed a desire to incorporate veteran feedback into VA clinical practice recommendations. We provided expertise on different engagement modalities (eg, focus groups vs individual interviews), and collaboration occurred to identify key interview questions for veterans. This process resulted in a published clinician-facing Long COVID Nervous System Clinical Guide (available at longcovid@hhs.gov) that integrated critical feedback from veterans related to neurological symptoms.

Second, weekly executive leadership meetings include dedicated time for reflection on partner feedback, the current state of Long COVID, and contextual changes that impact deliverable priorities and timelines. Outcomes from these discussions are communicated with VHA Health Services Research and, when appropriate, to key partners to ensure alignment. For example, the Patient Identification and Analysis core was originally tasked with identifying a definition of Long COVID. However, as the broader community moved away from a singular definition, efforts were redirected toward higher-priority issues within the VA Long COVID ecosystem, including veteran enrollment in clinical trials.

Third, the Veteran Engagement Panel captures feedback from those with lived experience to inform Long COVID research and clinical efforts. The panel meetings are strategically designed to ask veterans living with Long COVID specific questions related to a given research or clinical topic of interest. For example, panel sessions with the Field Advisory Board focused on concerns articulated by veterans related to the mental health and gastroenterological symptoms associated with Long COVID. Insights from these discussions will inform development of Long COVID mental health and gastroenterological clinical care guides, with several PBRN investigators serving as subject matter experts. This collaborative approach ensures that veteran perspectives are represented in developing Long COVID clinical care processes.

Fourth, research priorities identified through the Delphi consensus process will inform development of VA Request for Funding Proposals related to Long COVID. The initial survey was developed in collaboration with veterans, clinicians, and researchers across the Veteran Engagement Panel, the Field Advisory Board, and the National Research Action Plan on Long COVID.¹¹ The process was launched in October 2024 and concluded in June 2025. The team conducted 3 consensus rounds with veterans and VA clinicians and researchers. Top priority areas included the testing assessments for diagnosing Long COVID, studying subtypes of Long COVID and treatments for each, and finding biomarkers for Long COVID. A formal publication of the results and analysis is the focus of a future publication.

Fifth, ongoing engagement with the Field Advisory Board has supported adoption of a preliminary set of clinical outcome measures. If universally adopted, these instruments may contribute to the development of a standardized data collection process and serve as common data elements collected for epidemiologic, health services, or clinical trial research.

Lessons Learned and Practice Implications

Throughout the development of the LC-PBRN, several decisions were identified that have impacted infrastructure development and implementation.

Include veterans’ voices to ensure network efforts align with patient needs. Given the novelty of Long COVID, practitioners and researchers are learning as they go. It is important to listen to individuals who live with Long COVID. Throughout the development of the LC-PBRN, veteran perspective has proven how vital it is for them to be heard when it comes to their health care. Clinicians similarly highlighted the value of incorporating patient perspectives into the development of tools and treatment strategies. Develop an interdisciplinary leadership team to foster the diverse viewpoints needed to tackle multifaceted problems. It is important to consider as many clinical and research perspectives as possible because Long COVID is a complex condition with symptoms impacting major organ systems.^12-15 Therefore, the team spans across a multitude of specialties and locations.

Set clear expectations and goals with partners to uphold timely deliverables and stay within the PBRN’s capacity. When including a multitude of partners, teams should consider each of those partners’ experiences and opinions in decision-making conversations. Expectation setting is important to ensure all partners are on the same page and understand the capacity of the LC-PBRN. This allows the team to focus its efforts, avoid being overwhelmed with requests, and provide quality deliverables.

Build engaging relationships to bridge gaps between internal and external partners. A substantial number of resources focus on building relationships with partners so they can trust the LC-PBRN has their best interests in mind. These relationships are important to ensure the VA avoids duplicate efforts. This includes prioritizing connecting partners who are working on similar efforts to promote collaboration across facilities.

Clinical practice implications. The LC-PBRN is working towards clinical practice initiatives derived from this process in partnership with the Long COVID Community of Practice and the participating clinical sites. This may include efforts to increase the uptake of standardized instruments endorsed by clinical partners that facilitate assessment of outcomes. PBRN partners can then use outcomes data to ask and answer clinically relevant research questions and assess care quality to inform the learning process that is integral to an LHS. Future dissemination efforts will be centered around individual initiatives and deliverables from the LC-PBRN.

Conclusions

PBRNs provide an important mechanism to use LHS approaches to successfully convene research around complex issues. PBRNs can support integration across the LHS cycle, allowing for multiple feedback loops, and coordinate activities that work to achieve a larger vision. PBRNs offer centralized mechanisms to collaboratively understand and address complex problems, such as Long COVID, where the uncertainty regarding how to treat occurs in tandem with the urgency to treat. The LC-PBRN model described in this article has the potential to transcend Long COVID by building infrastructure necessary to proactively address current or future clinical conditions or populations with a LHS lens. The infrastructure can require cross-system and sector collaborations, expediency, inclusivity, and patient- and family-centeredness. Future efforts will focus on building out a larger network of VHA sites, facilitating recruitment at site and veteran levels into Long COVID trials through case identification, and systematically support the standardization of clinical data for clinical utility and evaluation of quality and/or outcomes across the VHA.