Original Research

Tobacco smoking remains the leading cause of preventable disease and death in the United States, accounting for more than 480,000 deaths annually.¹ An estimated 50.6 million US adults (20.8%) identify as tobacco users, with even higher rates among veterans (29.2%).^2,3 Tobacco use is estimated to cost the US more than $300 billion annually in direct and indirect medical costs.⁴ According to a 2015 report, more than two-thirds of adult smokers reported a desire to quit, while only 7.5% reported successfully quitting in the past year.⁵ According to that same report, only 57.2% of smokers who had seen a health professional in the past year reported receiving advice to quit.⁵ This statistic is unfortunate, as interventions that combine behavioral and pharmacologic support can drastically increase tobacco cessation rates compared with self-help materials or no treatment.⁶

Currently, 7 first-line medications (5 nicotine, 2 nonnicotine) have been shown to increase long-term smoking abstinence rates. Varenicline was approved by the US Food and Drug Administration (FDA) in 2006 for use in adults as an aid to smoking cessation treatment. As a partial agonist of the α4β2 nicotinic acetylcholine receptor, varenicline’s mechanism of action is believed to involve reduction of nicotine’s rewarding capacity.⁷ Varenicline not only aids in complete tobacco cessation but also has been found to be effective for reducing cigarette consumption among smokers not yet willing or able to make a quit attempt.⁸ Furthermore, varenicline has demonstrated efficacy among users of smokeless tobacco in achieving continuous abstinence.⁹

Widespread adoption of varenicline into clinical practice was perhaps slowed by early concerns of psychiatric complications, prompting the FDA to issue a boxed warning for risk of serious neuropsychiatric events. This boxed warning was removed in 2016 in response to publication of the Evaluating Adverse Events in a Global Smoking Cessation Study (EAGLES). In this randomized controlled trial of more than 8000 participants, among whom 50.5% had a psychiatric disorder determined to be stable, varenicline significantly increased rates of continuous tobacco cessation compared with bupropion or the nicotine patch without an increased risk of neuropsychiatric events.¹⁰ This study underscored not only the safety of varenicline, but also its superiority over other first-line cessation products. The most recently published clinical practice guidelines recommend varenicline as a first-line agent for helping patients achieve long-term smoking cessation.^11,12

Pharmacists are uniquely positioned to provide tobacco cessation interventions given their medication expertise and accessibility to the public. Indeed, multiple studies have demonstrated the effectiveness of pharmacist-led interventions on tobacco cessation.^13-15 As of 2019, only 12 states had statutes or regulations addressing pharmacist prescribing of tobacco cessation aids without a collaborative practice agreement or local standing order.¹⁶ Until recently, most of these states limited pharmacists’ prescriptive authority to nicotine replacement therapy (NRT). New Mexico serves as the outlier, as pharmacists have been authorized to prescribe all FDA-approved tobacco cessation products since 2004. A 2014 New Mexico study consisting of > 1400 patients showed that pharmacist-provided tobacco cessation interventions, which included management of varenicline, resulted in quit rates similar to those seen with management by other health care professionals.¹⁷ Aside from this study, there is a paucity of data available to assess the impact of pharmacist management of varenicline, let alone provide a head-to-head comparison with management by other clinicians.

Within the US Department of Veterans Affairs (VA), the clinical pharmacy specialist (CPS) is credentialed as an advanced practitioner with authority to independently manage patient medication therapy for a variety of diseases specified under a scope of practice. Although CPSs have provided tobacco cessation services for years, expansion of their scope to include varenicline did not occur until June 26, 2019, at the Southern Arizona VA Health Care System (SAVAHCS). All VA prescribers must follow the same criteria for prescribing varenicline. Unless previously trialed on varenicline, patients must have failed an appropriate trial of first-line agents (NRT, bupropion, or combination therapy) or have a contraindication to use of these first-line therapies before varenicline can be considered. Exclusions to therapy would include history of serious hypersensitivity to varenicline; suicidal intent, plan, or attempt within the past 12 months; current substance use disorder other than nicotine (unless varenicline recommended or prescribed by mental health professional); or unstable mental health disorder.¹⁸

The purpose of this study was to evaluate the efficacy and safety of CPS management of varenicline compared with other clinicians. We hope that this study provides insight regarding how the expansion of CPS scope to include prescriptive authority for varenicline has affected patient outcomes.

 

Methods

This retrospective chart review was conducted using SAVAHCS electronic health records. This study was granted approval by the institutional review board and the research and development committee at SAVAHCS. Data were obtained through the Computerized Patient Record System from the information provided by the pharmacist informatics department and was recorded electronically on a secure Microsoft Excel spreadsheet.

To be eligible for this study, patients must have been aged ≥ 18 years with a varenicline prescription between July 1, 2019, and July 31, 2020. Patients were excluded if tobacco cessation was managed by community-based (non-VA) clincians or if there was a lack of documentation of tobacco use at baseline and after at least 12 weeks of varenicline therapy. Sample size was not designed to achieve statistical power. Potential patients were queried by a pharmacist specializing in clinical informatics. All patients meeting initial inclusion criteria were then screened individually to evaluate for exclusion criteria.

Data collected included baseline age, sex, race, type of tobacco use (cigarettes, smokeless, both), mean daily tobacco use, prespecified comorbidities (depression, anxiety, or other psychiatric condition), and previous cessation medications prescribed (NRT, bupropion, and previous trials of varenicline).

The primary outcomes were reduction in tobacco use calculated as change at 12 weeks from baseline (and 24 weeks if available), continuous abstinence at 12 weeks (and 24 weeks if available), adherence to varenicline therapy measured by proportion of days covered (days covered by refills during the measurement period divided by days between the first fill and the end of the measurement period), and time to first follow-up in days. For safety evaluation, charts were reviewed for documented adverse events (AEs) in the health record. These AEs were categorized as follows: gastrointestinal, mood disturbance, sleep disturbance, headache, seizures, allergy, or other.

Statistical analyses regarding veteran baseline characteristics were descriptive in nature. χ² test was used to analyze differences in complete cessation rates and AEs, whereas a Student t test was used to compare reductions of tobacco use, proportion of days covered (ie, adherence), and time to first follow-up. An α of .05 was used to determine significance.

Results

From the initial search, 255 charts met general inclusion criteria. After chart review, only 50 patients from the CPS group and 93 patients from the other clinician group met criteria to be included (Figure 1). The CPS group included pharmacists specializing in ambulatory care and outpatient mental health. The other clinician group was composed primarily of primary care practitioners, psychiatrists, and pulmonologists.

 

Overall, baseline characteristics were similar between the groups (Table 1). In the overall study population, the mean age was 57.5 years, 90% of patients were male, and 99% of patients were cigarette smokers. Baseline mean (SD) tobacco use was similar between the groups: 14.5 (10.8) vs 14.8 (8.6) cigarettes daily for the CPS and other clinician group, respectively.

 

  

While there was a significant reduction in daily cigarette use for both groups at 12 and 24 weeks (Figure 2), there was no mean (SD) between-group difference found among those patients prescribed varenicline by a CPS compared with other clinicians: -7.9 (10.4) vs -5.4 (9.8) cigarettes daily, respectively (P = .15) (Table 2). Change in tobacco use at 24 weeks and rates of complete tobacco abstinence were also not statistically significant between prescriber groups. Adherence (as evidenced by refill data) was higher in the CPS group than in the other clinician group (42% vs 31%, respectively; P = .01). There was also a significant difference in time to first follow-up; patients whose varenicline therapy was managed by a CPS had a mean (SD) follow-up time of 52 (66) vs 163 (110) days when patients were managed by other clinicians (P < .001). AEs were documented in 42% of patients in the CPS group compared with 23% of patients in the other clinician group (Table 3). The most reported AEs were gastrointestinal, as well as mood and sleep disturbances.

 

Discussion

The results of this single center study suggest that management of varenicline by CPSs is associated with similar reductions in tobacco use and abstinence rates compared with management by other clinicians. These results provide evidence that CPS management of varenicline may be as safe and effective as management by other clinicians.

Adherence rates (reported as proportion of days covered when assessing varenicline refill data) were higher on average among patients managed by a CPS compared with patients managed by other clinicians. However, this outcome may not be as reflective of adherence as initially intended, given delays in follow-up (see limitations section). Time to first follow-up was drastically different between the groups, with much sooner follow-up by CPSs compared with other clinicians. Despite similar tobacco cessation rates between groups, more frequent follow-up by CPSs helps to assess patient barriers to cessation, adherence to therapy, and AEs with varenicline. A higher percentage of AEs were documented within the CPS group that could be attributed to disparities in documentation rather than true rates of AEs. While rates of AEs were initially intended to serve as the primary safety outcome, they may instead reflect pharmacists’ diligence in monitoring and documenting tolerability of medication therapy.

Limitations

Several limitations to this study should be noted. First, the data collected were only as detailed as the extent to which prescribers documented tobacco use, previous cessation trials, and AEs; thus, various data points are likely missing within this study that could impact the results presented. In line with lack of documentation, delays in follow-up (ie, annual primary care visits) sorely undermined proportion of days covered, making these data less indicative of true medication adherence. Furthermore, this study did not account for concurrent therapies, such as combination varenicline and nicotine gum/lozenges, or behavioral treatment strategies like cessation classes.

Another limitation was that some primary care practitioners prescribed varenicline but then referred these patients to a CPS for tobacco cessation follow-up. Per the study’s protocol, these patients were included within the other clinician group, which could have brought results closer to the null. Finally, the timing of this chart review (July 1, 2019, to July 31, 2020) intersects with the start of the COVID-19 pandemic, presenting a possible confounding factor if patients’ quit attempts were hindered by the stress and isolation of the pandemic.¹⁹ All pharmacist visits during the pandemic were conducted by telephone, which may have affected results.

Conclusions

In this study of veterans receiving varenicline, management by CPSs resulted in similar reductions of tobacco use and rates of complete abstinence compared with management by other clinicians. Pharmacist management was associated with greater adherence and shorter time to first follow-up compared with other clinicians. Additional research is needed to fully characterize the impact of pharmacist management of varenicline, justify expansion of clinical pharmacist scope of practice, and ultimately enhance patient outcomes regarding tobacco cessation.

It would be interesting to see more studies outside of the VA system to determine the impact of pharmacist management of varenicline for a more heterogenous patient population. At some point, a prospective controlled trial should be conducted to overcome the various confounding factors that limit the results of retrospective chart reviews

 

Acknowledgments

This article was prepared, and research was conducted with resources and the use of facilities at the Southern Arizona Veterans Affairs Health Care System in Tucson.

Tobacco smoking remains the leading cause of preventable disease and death in the United States, accounting for more than 480,000 deaths annually.¹ An estimated 50.6 million US adults (20.8%) identify as tobacco users, with even higher rates among veterans (29.2%).^2,3 Tobacco use is estimated to cost the US more than $300 billion annually in direct and indirect medical costs.⁴ According to a 2015 report, more than two-thirds of adult smokers reported a desire to quit, while only 7.5% reported successfully quitting in the past year.⁵ According to that same report, only 57.2% of smokers who had seen a health professional in the past year reported receiving advice to quit.⁵ This statistic is unfortunate, as interventions that combine behavioral and pharmacologic support can drastically increase tobacco cessation rates compared with self-help materials or no treatment.⁶

Currently, 7 first-line medications (5 nicotine, 2 nonnicotine) have been shown to increase long-term smoking abstinence rates. Varenicline was approved by the US Food and Drug Administration (FDA) in 2006 for use in adults as an aid to smoking cessation treatment. As a partial agonist of the α4β2 nicotinic acetylcholine receptor, varenicline’s mechanism of action is believed to involve reduction of nicotine’s rewarding capacity.⁷ Varenicline not only aids in complete tobacco cessation but also has been found to be effective for reducing cigarette consumption among smokers not yet willing or able to make a quit attempt.⁸ Furthermore, varenicline has demonstrated efficacy among users of smokeless tobacco in achieving continuous abstinence.⁹

Widespread adoption of varenicline into clinical practice was perhaps slowed by early concerns of psychiatric complications, prompting the FDA to issue a boxed warning for risk of serious neuropsychiatric events. This boxed warning was removed in 2016 in response to publication of the Evaluating Adverse Events in a Global Smoking Cessation Study (EAGLES). In this randomized controlled trial of more than 8000 participants, among whom 50.5% had a psychiatric disorder determined to be stable, varenicline significantly increased rates of continuous tobacco cessation compared with bupropion or the nicotine patch without an increased risk of neuropsychiatric events.¹⁰ This study underscored not only the safety of varenicline, but also its superiority over other first-line cessation products. The most recently published clinical practice guidelines recommend varenicline as a first-line agent for helping patients achieve long-term smoking cessation.^11,12

Pharmacists are uniquely positioned to provide tobacco cessation interventions given their medication expertise and accessibility to the public. Indeed, multiple studies have demonstrated the effectiveness of pharmacist-led interventions on tobacco cessation.^13-15 As of 2019, only 12 states had statutes or regulations addressing pharmacist prescribing of tobacco cessation aids without a collaborative practice agreement or local standing order.¹⁶ Until recently, most of these states limited pharmacists’ prescriptive authority to nicotine replacement therapy (NRT). New Mexico serves as the outlier, as pharmacists have been authorized to prescribe all FDA-approved tobacco cessation products since 2004. A 2014 New Mexico study consisting of > 1400 patients showed that pharmacist-provided tobacco cessation interventions, which included management of varenicline, resulted in quit rates similar to those seen with management by other health care professionals.¹⁷ Aside from this study, there is a paucity of data available to assess the impact of pharmacist management of varenicline, let alone provide a head-to-head comparison with management by other clinicians.

Within the US Department of Veterans Affairs (VA), the clinical pharmacy specialist (CPS) is credentialed as an advanced practitioner with authority to independently manage patient medication therapy for a variety of diseases specified under a scope of practice. Although CPSs have provided tobacco cessation services for years, expansion of their scope to include varenicline did not occur until June 26, 2019, at the Southern Arizona VA Health Care System (SAVAHCS). All VA prescribers must follow the same criteria for prescribing varenicline. Unless previously trialed on varenicline, patients must have failed an appropriate trial of first-line agents (NRT, bupropion, or combination therapy) or have a contraindication to use of these first-line therapies before varenicline can be considered. Exclusions to therapy would include history of serious hypersensitivity to varenicline; suicidal intent, plan, or attempt within the past 12 months; current substance use disorder other than nicotine (unless varenicline recommended or prescribed by mental health professional); or unstable mental health disorder.¹⁸

The purpose of this study was to evaluate the efficacy and safety of CPS management of varenicline compared with other clinicians. We hope that this study provides insight regarding how the expansion of CPS scope to include prescriptive authority for varenicline has affected patient outcomes.

 

Methods

This retrospective chart review was conducted using SAVAHCS electronic health records. This study was granted approval by the institutional review board and the research and development committee at SAVAHCS. Data were obtained through the Computerized Patient Record System from the information provided by the pharmacist informatics department and was recorded electronically on a secure Microsoft Excel spreadsheet.

To be eligible for this study, patients must have been aged ≥ 18 years with a varenicline prescription between July 1, 2019, and July 31, 2020. Patients were excluded if tobacco cessation was managed by community-based (non-VA) clincians or if there was a lack of documentation of tobacco use at baseline and after at least 12 weeks of varenicline therapy. Sample size was not designed to achieve statistical power. Potential patients were queried by a pharmacist specializing in clinical informatics. All patients meeting initial inclusion criteria were then screened individually to evaluate for exclusion criteria.

Data collected included baseline age, sex, race, type of tobacco use (cigarettes, smokeless, both), mean daily tobacco use, prespecified comorbidities (depression, anxiety, or other psychiatric condition), and previous cessation medications prescribed (NRT, bupropion, and previous trials of varenicline).

The primary outcomes were reduction in tobacco use calculated as change at 12 weeks from baseline (and 24 weeks if available), continuous abstinence at 12 weeks (and 24 weeks if available), adherence to varenicline therapy measured by proportion of days covered (days covered by refills during the measurement period divided by days between the first fill and the end of the measurement period), and time to first follow-up in days. For safety evaluation, charts were reviewed for documented adverse events (AEs) in the health record. These AEs were categorized as follows: gastrointestinal, mood disturbance, sleep disturbance, headache, seizures, allergy, or other.

Statistical analyses regarding veteran baseline characteristics were descriptive in nature. χ² test was used to analyze differences in complete cessation rates and AEs, whereas a Student t test was used to compare reductions of tobacco use, proportion of days covered (ie, adherence), and time to first follow-up. An α of .05 was used to determine significance.

Results

From the initial search, 255 charts met general inclusion criteria. After chart review, only 50 patients from the CPS group and 93 patients from the other clinician group met criteria to be included (Figure 1). The CPS group included pharmacists specializing in ambulatory care and outpatient mental health. The other clinician group was composed primarily of primary care practitioners, psychiatrists, and pulmonologists.

 

Overall, baseline characteristics were similar between the groups (Table 1). In the overall study population, the mean age was 57.5 years, 90% of patients were male, and 99% of patients were cigarette smokers. Baseline mean (SD) tobacco use was similar between the groups: 14.5 (10.8) vs 14.8 (8.6) cigarettes daily for the CPS and other clinician group, respectively.

 

  

While there was a significant reduction in daily cigarette use for both groups at 12 and 24 weeks (Figure 2), there was no mean (SD) between-group difference found among those patients prescribed varenicline by a CPS compared with other clinicians: -7.9 (10.4) vs -5.4 (9.8) cigarettes daily, respectively (P = .15) (Table 2). Change in tobacco use at 24 weeks and rates of complete tobacco abstinence were also not statistically significant between prescriber groups. Adherence (as evidenced by refill data) was higher in the CPS group than in the other clinician group (42% vs 31%, respectively; P = .01). There was also a significant difference in time to first follow-up; patients whose varenicline therapy was managed by a CPS had a mean (SD) follow-up time of 52 (66) vs 163 (110) days when patients were managed by other clinicians (P < .001). AEs were documented in 42% of patients in the CPS group compared with 23% of patients in the other clinician group (Table 3). The most reported AEs were gastrointestinal, as well as mood and sleep disturbances.

 

Discussion

The results of this single center study suggest that management of varenicline by CPSs is associated with similar reductions in tobacco use and abstinence rates compared with management by other clinicians. These results provide evidence that CPS management of varenicline may be as safe and effective as management by other clinicians.

Adherence rates (reported as proportion of days covered when assessing varenicline refill data) were higher on average among patients managed by a CPS compared with patients managed by other clinicians. However, this outcome may not be as reflective of adherence as initially intended, given delays in follow-up (see limitations section). Time to first follow-up was drastically different between the groups, with much sooner follow-up by CPSs compared with other clinicians. Despite similar tobacco cessation rates between groups, more frequent follow-up by CPSs helps to assess patient barriers to cessation, adherence to therapy, and AEs with varenicline. A higher percentage of AEs were documented within the CPS group that could be attributed to disparities in documentation rather than true rates of AEs. While rates of AEs were initially intended to serve as the primary safety outcome, they may instead reflect pharmacists’ diligence in monitoring and documenting tolerability of medication therapy.

Limitations

Several limitations to this study should be noted. First, the data collected were only as detailed as the extent to which prescribers documented tobacco use, previous cessation trials, and AEs; thus, various data points are likely missing within this study that could impact the results presented. In line with lack of documentation, delays in follow-up (ie, annual primary care visits) sorely undermined proportion of days covered, making these data less indicative of true medication adherence. Furthermore, this study did not account for concurrent therapies, such as combination varenicline and nicotine gum/lozenges, or behavioral treatment strategies like cessation classes.

Another limitation was that some primary care practitioners prescribed varenicline but then referred these patients to a CPS for tobacco cessation follow-up. Per the study’s protocol, these patients were included within the other clinician group, which could have brought results closer to the null. Finally, the timing of this chart review (July 1, 2019, to July 31, 2020) intersects with the start of the COVID-19 pandemic, presenting a possible confounding factor if patients’ quit attempts were hindered by the stress and isolation of the pandemic.¹⁹ All pharmacist visits during the pandemic were conducted by telephone, which may have affected results.

Conclusions

In this study of veterans receiving varenicline, management by CPSs resulted in similar reductions of tobacco use and rates of complete abstinence compared with management by other clinicians. Pharmacist management was associated with greater adherence and shorter time to first follow-up compared with other clinicians. Additional research is needed to fully characterize the impact of pharmacist management of varenicline, justify expansion of clinical pharmacist scope of practice, and ultimately enhance patient outcomes regarding tobacco cessation.

It would be interesting to see more studies outside of the VA system to determine the impact of pharmacist management of varenicline for a more heterogenous patient population. At some point, a prospective controlled trial should be conducted to overcome the various confounding factors that limit the results of retrospective chart reviews

 

Acknowledgments

This article was prepared, and research was conducted with resources and the use of facilities at the Southern Arizona Veterans Affairs Health Care System in Tucson.

Tobacco smoking remains the leading cause of preventable disease and death in the United States, accounting for more than 480,000 deaths annually.¹ An estimated 50.6 million US adults (20.8%) identify as tobacco users, with even higher rates among veterans (29.2%).^2,3 Tobacco use is estimated to cost the US more than $300 billion annually in direct and indirect medical costs.⁴ According to a 2015 report, more than two-thirds of adult smokers reported a desire to quit, while only 7.5% reported successfully quitting in the past year.⁵ According to that same report, only 57.2% of smokers who had seen a health professional in the past year reported receiving advice to quit.⁵ This statistic is unfortunate, as interventions that combine behavioral and pharmacologic support can drastically increase tobacco cessation rates compared with self-help materials or no treatment.⁶

Currently, 7 first-line medications (5 nicotine, 2 nonnicotine) have been shown to increase long-term smoking abstinence rates. Varenicline was approved by the US Food and Drug Administration (FDA) in 2006 for use in adults as an aid to smoking cessation treatment. As a partial agonist of the α4β2 nicotinic acetylcholine receptor, varenicline’s mechanism of action is believed to involve reduction of nicotine’s rewarding capacity.⁷ Varenicline not only aids in complete tobacco cessation but also has been found to be effective for reducing cigarette consumption among smokers not yet willing or able to make a quit attempt.⁸ Furthermore, varenicline has demonstrated efficacy among users of smokeless tobacco in achieving continuous abstinence.⁹

Widespread adoption of varenicline into clinical practice was perhaps slowed by early concerns of psychiatric complications, prompting the FDA to issue a boxed warning for risk of serious neuropsychiatric events. This boxed warning was removed in 2016 in response to publication of the Evaluating Adverse Events in a Global Smoking Cessation Study (EAGLES). In this randomized controlled trial of more than 8000 participants, among whom 50.5% had a psychiatric disorder determined to be stable, varenicline significantly increased rates of continuous tobacco cessation compared with bupropion or the nicotine patch without an increased risk of neuropsychiatric events.¹⁰ This study underscored not only the safety of varenicline, but also its superiority over other first-line cessation products. The most recently published clinical practice guidelines recommend varenicline as a first-line agent for helping patients achieve long-term smoking cessation.^11,12

Pharmacists are uniquely positioned to provide tobacco cessation interventions given their medication expertise and accessibility to the public. Indeed, multiple studies have demonstrated the effectiveness of pharmacist-led interventions on tobacco cessation.^13-15 As of 2019, only 12 states had statutes or regulations addressing pharmacist prescribing of tobacco cessation aids without a collaborative practice agreement or local standing order.¹⁶ Until recently, most of these states limited pharmacists’ prescriptive authority to nicotine replacement therapy (NRT). New Mexico serves as the outlier, as pharmacists have been authorized to prescribe all FDA-approved tobacco cessation products since 2004. A 2014 New Mexico study consisting of > 1400 patients showed that pharmacist-provided tobacco cessation interventions, which included management of varenicline, resulted in quit rates similar to those seen with management by other health care professionals.¹⁷ Aside from this study, there is a paucity of data available to assess the impact of pharmacist management of varenicline, let alone provide a head-to-head comparison with management by other clinicians.

Within the US Department of Veterans Affairs (VA), the clinical pharmacy specialist (CPS) is credentialed as an advanced practitioner with authority to independently manage patient medication therapy for a variety of diseases specified under a scope of practice. Although CPSs have provided tobacco cessation services for years, expansion of their scope to include varenicline did not occur until June 26, 2019, at the Southern Arizona VA Health Care System (SAVAHCS). All VA prescribers must follow the same criteria for prescribing varenicline. Unless previously trialed on varenicline, patients must have failed an appropriate trial of first-line agents (NRT, bupropion, or combination therapy) or have a contraindication to use of these first-line therapies before varenicline can be considered. Exclusions to therapy would include history of serious hypersensitivity to varenicline; suicidal intent, plan, or attempt within the past 12 months; current substance use disorder other than nicotine (unless varenicline recommended or prescribed by mental health professional); or unstable mental health disorder.¹⁸

The purpose of this study was to evaluate the efficacy and safety of CPS management of varenicline compared with other clinicians. We hope that this study provides insight regarding how the expansion of CPS scope to include prescriptive authority for varenicline has affected patient outcomes.

 

Methods

This retrospective chart review was conducted using SAVAHCS electronic health records. This study was granted approval by the institutional review board and the research and development committee at SAVAHCS. Data were obtained through the Computerized Patient Record System from the information provided by the pharmacist informatics department and was recorded electronically on a secure Microsoft Excel spreadsheet.

To be eligible for this study, patients must have been aged ≥ 18 years with a varenicline prescription between July 1, 2019, and July 31, 2020. Patients were excluded if tobacco cessation was managed by community-based (non-VA) clincians or if there was a lack of documentation of tobacco use at baseline and after at least 12 weeks of varenicline therapy. Sample size was not designed to achieve statistical power. Potential patients were queried by a pharmacist specializing in clinical informatics. All patients meeting initial inclusion criteria were then screened individually to evaluate for exclusion criteria.

Data collected included baseline age, sex, race, type of tobacco use (cigarettes, smokeless, both), mean daily tobacco use, prespecified comorbidities (depression, anxiety, or other psychiatric condition), and previous cessation medications prescribed (NRT, bupropion, and previous trials of varenicline).

The primary outcomes were reduction in tobacco use calculated as change at 12 weeks from baseline (and 24 weeks if available), continuous abstinence at 12 weeks (and 24 weeks if available), adherence to varenicline therapy measured by proportion of days covered (days covered by refills during the measurement period divided by days between the first fill and the end of the measurement period), and time to first follow-up in days. For safety evaluation, charts were reviewed for documented adverse events (AEs) in the health record. These AEs were categorized as follows: gastrointestinal, mood disturbance, sleep disturbance, headache, seizures, allergy, or other.

Statistical analyses regarding veteran baseline characteristics were descriptive in nature. χ² test was used to analyze differences in complete cessation rates and AEs, whereas a Student t test was used to compare reductions of tobacco use, proportion of days covered (ie, adherence), and time to first follow-up. An α of .05 was used to determine significance.

Results

From the initial search, 255 charts met general inclusion criteria. After chart review, only 50 patients from the CPS group and 93 patients from the other clinician group met criteria to be included (Figure 1). The CPS group included pharmacists specializing in ambulatory care and outpatient mental health. The other clinician group was composed primarily of primary care practitioners, psychiatrists, and pulmonologists.

 

Overall, baseline characteristics were similar between the groups (Table 1). In the overall study population, the mean age was 57.5 years, 90% of patients were male, and 99% of patients were cigarette smokers. Baseline mean (SD) tobacco use was similar between the groups: 14.5 (10.8) vs 14.8 (8.6) cigarettes daily for the CPS and other clinician group, respectively.

 

  

While there was a significant reduction in daily cigarette use for both groups at 12 and 24 weeks (Figure 2), there was no mean (SD) between-group difference found among those patients prescribed varenicline by a CPS compared with other clinicians: -7.9 (10.4) vs -5.4 (9.8) cigarettes daily, respectively (P = .15) (Table 2). Change in tobacco use at 24 weeks and rates of complete tobacco abstinence were also not statistically significant between prescriber groups. Adherence (as evidenced by refill data) was higher in the CPS group than in the other clinician group (42% vs 31%, respectively; P = .01). There was also a significant difference in time to first follow-up; patients whose varenicline therapy was managed by a CPS had a mean (SD) follow-up time of 52 (66) vs 163 (110) days when patients were managed by other clinicians (P < .001). AEs were documented in 42% of patients in the CPS group compared with 23% of patients in the other clinician group (Table 3). The most reported AEs were gastrointestinal, as well as mood and sleep disturbances.

 

Discussion

The results of this single center study suggest that management of varenicline by CPSs is associated with similar reductions in tobacco use and abstinence rates compared with management by other clinicians. These results provide evidence that CPS management of varenicline may be as safe and effective as management by other clinicians.

Adherence rates (reported as proportion of days covered when assessing varenicline refill data) were higher on average among patients managed by a CPS compared with patients managed by other clinicians. However, this outcome may not be as reflective of adherence as initially intended, given delays in follow-up (see limitations section). Time to first follow-up was drastically different between the groups, with much sooner follow-up by CPSs compared with other clinicians. Despite similar tobacco cessation rates between groups, more frequent follow-up by CPSs helps to assess patient barriers to cessation, adherence to therapy, and AEs with varenicline. A higher percentage of AEs were documented within the CPS group that could be attributed to disparities in documentation rather than true rates of AEs. While rates of AEs were initially intended to serve as the primary safety outcome, they may instead reflect pharmacists’ diligence in monitoring and documenting tolerability of medication therapy.

Limitations

Several limitations to this study should be noted. First, the data collected were only as detailed as the extent to which prescribers documented tobacco use, previous cessation trials, and AEs; thus, various data points are likely missing within this study that could impact the results presented. In line with lack of documentation, delays in follow-up (ie, annual primary care visits) sorely undermined proportion of days covered, making these data less indicative of true medication adherence. Furthermore, this study did not account for concurrent therapies, such as combination varenicline and nicotine gum/lozenges, or behavioral treatment strategies like cessation classes.

Another limitation was that some primary care practitioners prescribed varenicline but then referred these patients to a CPS for tobacco cessation follow-up. Per the study’s protocol, these patients were included within the other clinician group, which could have brought results closer to the null. Finally, the timing of this chart review (July 1, 2019, to July 31, 2020) intersects with the start of the COVID-19 pandemic, presenting a possible confounding factor if patients’ quit attempts were hindered by the stress and isolation of the pandemic.¹⁹ All pharmacist visits during the pandemic were conducted by telephone, which may have affected results.

Conclusions

In this study of veterans receiving varenicline, management by CPSs resulted in similar reductions of tobacco use and rates of complete abstinence compared with management by other clinicians. Pharmacist management was associated with greater adherence and shorter time to first follow-up compared with other clinicians. Additional research is needed to fully characterize the impact of pharmacist management of varenicline, justify expansion of clinical pharmacist scope of practice, and ultimately enhance patient outcomes regarding tobacco cessation.

It would be interesting to see more studies outside of the VA system to determine the impact of pharmacist management of varenicline for a more heterogenous patient population. At some point, a prospective controlled trial should be conducted to overcome the various confounding factors that limit the results of retrospective chart reviews

 

Acknowledgments

This article was prepared, and research was conducted with resources and the use of facilities at the Southern Arizona Veterans Affairs Health Care System in Tucson.

Venous thromboembolism (VTE) presents as deep venous thromboembolism (DVT) or pulmonary embolism (PE). VTE is the third most common vascular disease and a leading cardiovascular complication.^1,2 Hospitalized patients are at increased risk of developing VTE due to multiple factors such as inflammatory processes from acute illness, recent surgery or trauma leading to hypercoagulable states, and prolonged periods of immobilization.³ Additional risk factors for complications include presence of malignancy, obesity, and prior history of VTE. About half of VTE cases in the community setting occur as a result of a hospital admission for recent or ongoing acute illness or surgery.¹ Hospitalized patients are often categorized as high risk for VTE, and this risk may persist postdischarge.⁴

The risk of hospital-associated VTE may be mitigated with either mechanical or pharmacologic thromboprophylaxis.⁵ Risk assessment models (RAMs), such as Padua Prediction Score (PPS) and IMPROVEDD, have been developed to assist in evaluating hospitalized patients’ risk of VTE and need for pharmacologic thromboprophylaxis (Table 1).^1,5 The PPS is externally validated and can assist clinicians in VTE risk assessment when integrated into clinical decision making.⁶ Patients with a PPS ≥ 4 are deemed high risk for VTE, and pharmacologic thromboprophylaxis is indicated as long as the patient is not at high risk for bleeding. IMPROVEDD added D-dimer as an additional risk factor to IMPROVE and was validated in 2017 to help predict the risk of symptomatic VTE in acutely ill patients hospitalized for up to 77 days.⁷ IMPROVEDD scores ≥ 2 identify patients at high risk for symptomatic VTE through 77 days hospitalization, while scores ≥ 4 identify patients who may qualify for extended thromboprophylaxis.⁷ Despite their utility, RAMs may not be used appropriately within clinical practice, and whether patients should receive extended-duration thromboprophylaxis postdischarge and for how long is debatable.⁵

VTE events contribute to increased health care spending, morbidity, and mortality, thus it is imperative to evaluate current hospital practices with respect to appropriate prescribing of pharmacologic thromboprophylaxis.⁸ Appropriately identifying high-risk patients and prescribing pharmacologic thromboprophylaxis to limit preventable VTEs is essential. Conversely, it is important to withhold pharmacologic thromboprophylaxis from those deemed low risk to limit bleeding complications.⁹ Health care professionals must be good stewards of anticoagulant prescribing when implementing these tools along with clinical knowledge to weigh the risks vs benefits to promote medication safety and prevent further complications.¹⁰This quality improvement project aimed to evaluate if VTE thromboprophylaxis was appropriately given or withheld in hospitalized medical patients based on PPS calculated upon admission using a link to an online calculator embedded within an admission order set. Additionally, this study aimed to characterize patients readmitted for VTE within 45 days postdischarge to generate hypotheses for future stu

 

Methods

This was an observational, retrospective cohort study that took place at the US Department of Veterans Affairs (VA) Tennessee Valley Healthcare System (TVHS). TVHS is a multisite health care system with campuses in Nashville and Murfreesboro. Clinical pharmacists employed at the study site and the primary research investigators designed this study and oversaw its execution. The study was reviewed and deemed exempt as a quality improvement study by the TVHS Institutional Review Board.

This study included adult veterans aged ≥ 18 years admitted to a general medicine floor or the medical intensive care unit between June 1, 2017, and June 30, 2020. Patients were excluded if they were on chronic therapeutic anticoagulation prior to their index hospitalization, required therapeutic anticoagulation on admission for index hospitalization (ie, acute coronary syndrome requiring a heparin drip), or were bedded within the surgical intensive care unit. All patients admitted to the TVHS within the prespecified date range were extracted from the electronic health record. A second subset of patients meeting inclusion criteria and readmitted for VTE within 45 days of index hospitalization with International Classification of Diseases, Tenth Revision (ICD-10) descriptions including thrombosis or embolism were extracted for review of a secondary endpoint. Patients with preexisting clots, history of prior DVT or PE, or history of portal vein thrombosis were not reviewed.

The primary endpoint was the percentage of patients for whom pharmacologic thromboprophylaxis was appropriately initiated or withheld based on a PPS calculated upon admission (Table 2). PPS was chosen for review as it is the only RAM currently used at TVHS. Secondary endpoints were the percentage of patients with documented rationale for ordering thromboprophylaxis when not indicated, based on PPS, or withholding despite indication as well as the number of patients readmitted to TVHS for VTE within 45 days of discharge with IMPROVEDD scores ≥ 4 and < 4 (eAppendix available at doi:10.12788/fp.0291). The primary investigators performed a manual health record review of all patients meeting inclusion criteria. Descriptive statistics were used given this was a quality improvement study, therefore, sample size and power calculations were not necessary. Data were stored in Microsoft Excel spreadsheets that were encrypted and password protected. To maintain security of personal health information, all study files were kept on the TVHS internal network, and access was limited to the research investigators.

Results

Two hundred fifty patients meeting inclusion criteria were randomly selected for review for the primary endpoint. Of the patients reviewed for the primary endpoint, 118 had a PPS < 4 and 132 a PPS ≥ 4 (Figure). Pharmacologic thromboprophylaxis was inappropriately given or withheld based on their PPS for 91 (36.4%) patients. This included 58 (49.2%) patients in the low-risk group (PPS < 4) who had thromboprophylaxis inappropriately given and 33 (25.0%) patients in the high-risk group (PPS ≥ 4) who had thromboprophylaxis inappropriately withheld. Of the 58 patients with a PPS < 4 who were given prophylaxis, only 2 (3.4%) patients had documented rationale as to why anticoagulation was administered. Of the 132 patients with a PPS ≥ 4, 44 patients had thromboprophylaxis withheld. Eleven (8.3%) patients had thromboprophylaxis appropriately withheld due to presence or concern for bleeding. Commonly documented rationale for inappropriately withholding thromboprophylaxis when indicated included use of sequential compression devices (40.9%), pancytopenia (18.2%), dual antiplatelet therapy (9.1%), or patient was ambulatory (4.5%).

 

A secondary endpoint characterized patients at highest risk for developing a VTE after hospitalization for an acute illness. Seventy patients were readmitted within 45 days of discharge from the index hospitalization with ICD descriptions for embolism or thrombosis. Only 15 of those patients were readmitted with a newly diagnosed VTE not previously identified; 14 (93.3%) had a PPS ≥ 4 upon index admission and 10 (66.7%) appropriately received pharmacologic prophylaxis within 24 hours of admission. Of the 15 patients, 3 (20.0%) did not receive pharmacologic thromboprophylaxis within 24 hours of admission and 1 (6.7%) received thromboprophylaxis despite having a PPS < 4.

Looking at IMPROVEDD scores for the 15 patients at the index hospitalization discharge, 1 (6.7%) patient had an IMPROVEDD score < 2, 11 (73.3%) patients had IMPROVEDD scores ≥ 2, and 3 (20.0%) patients had IMPROVEDD scores ≥ 4. Two of the patients with IMPROVEDD scores ≥ 4 had a history of VTE and were aged > 60 years. Of the 15 patients reviewed, 7 had a diagnosis of cancer, and 3 were actively undergoing chemotherapy.

Discussion

PPS is the RAM embedded in our system’s order set, which identifies hospitalized medical patients at risk for VTE.⁶ In the original study that validated PPS, the results suggested that implementation of preventive measures during hospitalization in patients labeled as having high thrombotic risk confers longstanding protection against thromboembolic complications in comparison with untreated patients.⁶ However, PPS must be used consistently and appropriately to realize this benefit. Our results showed that pharmacologic thromboprophylaxis is frequently inappropriately given or withheld despite the incorporation of a RAM in an admission order set, suggesting there is a significant gap between written policy and actual practice. More than one-third of patients had thromboprophylaxis given or withheld inappropriately according to the PPS calculated manually on review. With this, there is concern for over- and underprescribing of thromboprophylaxis, which increases the risk of adverse events. Overprescribing can lead to unnecessary bleeding complications, whereas underprescribing can lead to preventable VTE.

One issue identified during this study was the need for a user-friendly interface. The PPS calculator currently embedded in our admission order set is a hyperlink to an online calculator. This is time consuming and cumbersome for clinicians tending to a high volume of patients, which may cause them to overlook the calculator and estimate risk based on clinician judgement. Noted areas for improvement regarding thromboprophylaxis during inpatient admissions include the failure to implement or adhere to risk stratification protocols, lack of appropriate assessment for thromboprophylaxis, and the overutilization of pharmacologic thromboprophylaxis in low-risk patients.¹¹

Certain patients develop a VTE postdischarge despite efforts at prevention during their index hospitalization, which led us to explore our secondary endpoint looking at readmissions. Regarding thromboprophylaxis postdischarge, the duration of therapy is an area of current debate.⁵ Extended-duration thromboprophylaxis is defined as anticoagulation prescribed beyond hospitalization for up to 42 days total.^1,12 To date, there have been 5 clinical trials to evaluate the utility of extended-duration thromboprophylaxis in hospitalized medically ill patients. While routine use is not recommended by the 2018 American Society of Hematology guidelines for management of VTE, more recent data suggest certain medically ill patients may derive benefit from extended-duration thromboprophylaxis.⁴ The IMPROVEDD score aimed to address this need, which is why it was calculated on index discharge for our patients readmitted within 45 days. Research is still needed to identify such patients and RAMs for capturing these subpopulations.^1,11

Our secondary endpoint sought to characterize patients at highest risk for developing a VTE postdischarge. Of the 15 patients reviewed, 7 had a diagnosis of cancer and 3 were actively undergoing chemotherapy. With that, the Khorana Risk Score may have been a more appropriate RAM for some given the Khorana score is validated in ambulatory patients undergoing chemotherapy. D-dimer was only collected for 1 of the 15 patients, therefore, VTE risk could have been underestimated with the IMPROVEDD scores calculated. More than 75% of patients readmitted for VTE appropriately received thromboprophylaxis on index admission yet still went on to develop a VTE. It is essential to increase clinician awareness about hospital-acquired and postdischarge VTE. In line with guidance from the North American Thrombosis Forum, extended-duration thromboprophylaxis should be thoughtfully considered in high-risk patients.⁵ Pathways, including follow-up, are needed to implement postdischarge thromboprophylaxis when appropriate

 

Limitations

There were some inherent limitations to this study with its retrospective nature and small sample size. Data extraction was limited to health records within the VA, so there is a chance relevant history could be missed via incomplete documentation. Thus, our results could be an underestimation of postdischarge VTE prevalence if patients sought medical attention outside of the VA. Given this study was a retrospective chart review, data collection was limited to what was explicitly documented in the chart. Rationale for giving thromboprophylaxis when not indicated or holding when indicated may have been underestimated if clinicians did not document thoroughly in the electronic health record. Last, for the secondary endpoint reviewing the IMPROVEDD score, a D-dimer was not consistently obtained on admission, which could lead to underestimation of risk.

Conclusions

The results of this study showed that more than one-third of patients admitted to our facility within the prespecified timeframe had pharmacologic thromboprophylaxis inappropriately given or withheld according to a PPS manually calculated on admission. The PPS calculator currently embedded within our admission order set is not being utilized appropriately or consistently in clinical practice. Additionally, results from the secondary endpoint looking at IMPROVEDD scores highlight an unmet need for thromboprophylaxis at discharge. Pathways are needed to implement postdischarge thromboprophylaxis when appropriate for patients at highest thromboembolic risk.

 

Venous thromboembolism (VTE) presents as deep venous thromboembolism (DVT) or pulmonary embolism (PE). VTE is the third most common vascular disease and a leading cardiovascular complication.^1,2 Hospitalized patients are at increased risk of developing VTE due to multiple factors such as inflammatory processes from acute illness, recent surgery or trauma leading to hypercoagulable states, and prolonged periods of immobilization.³ Additional risk factors for complications include presence of malignancy, obesity, and prior history of VTE. About half of VTE cases in the community setting occur as a result of a hospital admission for recent or ongoing acute illness or surgery.¹ Hospitalized patients are often categorized as high risk for VTE, and this risk may persist postdischarge.⁴

The risk of hospital-associated VTE may be mitigated with either mechanical or pharmacologic thromboprophylaxis.⁵ Risk assessment models (RAMs), such as Padua Prediction Score (PPS) and IMPROVEDD, have been developed to assist in evaluating hospitalized patients’ risk of VTE and need for pharmacologic thromboprophylaxis (Table 1).^1,5 The PPS is externally validated and can assist clinicians in VTE risk assessment when integrated into clinical decision making.⁶ Patients with a PPS ≥ 4 are deemed high risk for VTE, and pharmacologic thromboprophylaxis is indicated as long as the patient is not at high risk for bleeding. IMPROVEDD added D-dimer as an additional risk factor to IMPROVE and was validated in 2017 to help predict the risk of symptomatic VTE in acutely ill patients hospitalized for up to 77 days.⁷ IMPROVEDD scores ≥ 2 identify patients at high risk for symptomatic VTE through 77 days hospitalization, while scores ≥ 4 identify patients who may qualify for extended thromboprophylaxis.⁷ Despite their utility, RAMs may not be used appropriately within clinical practice, and whether patients should receive extended-duration thromboprophylaxis postdischarge and for how long is debatable.⁵

VTE events contribute to increased health care spending, morbidity, and mortality, thus it is imperative to evaluate current hospital practices with respect to appropriate prescribing of pharmacologic thromboprophylaxis.⁸ Appropriately identifying high-risk patients and prescribing pharmacologic thromboprophylaxis to limit preventable VTEs is essential. Conversely, it is important to withhold pharmacologic thromboprophylaxis from those deemed low risk to limit bleeding complications.⁹ Health care professionals must be good stewards of anticoagulant prescribing when implementing these tools along with clinical knowledge to weigh the risks vs benefits to promote medication safety and prevent further complications.¹⁰This quality improvement project aimed to evaluate if VTE thromboprophylaxis was appropriately given or withheld in hospitalized medical patients based on PPS calculated upon admission using a link to an online calculator embedded within an admission order set. Additionally, this study aimed to characterize patients readmitted for VTE within 45 days postdischarge to generate hypotheses for future stu

 

Methods

This was an observational, retrospective cohort study that took place at the US Department of Veterans Affairs (VA) Tennessee Valley Healthcare System (TVHS). TVHS is a multisite health care system with campuses in Nashville and Murfreesboro. Clinical pharmacists employed at the study site and the primary research investigators designed this study and oversaw its execution. The study was reviewed and deemed exempt as a quality improvement study by the TVHS Institutional Review Board.

This study included adult veterans aged ≥ 18 years admitted to a general medicine floor or the medical intensive care unit between June 1, 2017, and June 30, 2020. Patients were excluded if they were on chronic therapeutic anticoagulation prior to their index hospitalization, required therapeutic anticoagulation on admission for index hospitalization (ie, acute coronary syndrome requiring a heparin drip), or were bedded within the surgical intensive care unit. All patients admitted to the TVHS within the prespecified date range were extracted from the electronic health record. A second subset of patients meeting inclusion criteria and readmitted for VTE within 45 days of index hospitalization with International Classification of Diseases, Tenth Revision (ICD-10) descriptions including thrombosis or embolism were extracted for review of a secondary endpoint. Patients with preexisting clots, history of prior DVT or PE, or history of portal vein thrombosis were not reviewed.

The primary endpoint was the percentage of patients for whom pharmacologic thromboprophylaxis was appropriately initiated or withheld based on a PPS calculated upon admission (Table 2). PPS was chosen for review as it is the only RAM currently used at TVHS. Secondary endpoints were the percentage of patients with documented rationale for ordering thromboprophylaxis when not indicated, based on PPS, or withholding despite indication as well as the number of patients readmitted to TVHS for VTE within 45 days of discharge with IMPROVEDD scores ≥ 4 and < 4 (eAppendix available at doi:10.12788/fp.0291). The primary investigators performed a manual health record review of all patients meeting inclusion criteria. Descriptive statistics were used given this was a quality improvement study, therefore, sample size and power calculations were not necessary. Data were stored in Microsoft Excel spreadsheets that were encrypted and password protected. To maintain security of personal health information, all study files were kept on the TVHS internal network, and access was limited to the research investigators.

Results

Two hundred fifty patients meeting inclusion criteria were randomly selected for review for the primary endpoint. Of the patients reviewed for the primary endpoint, 118 had a PPS < 4 and 132 a PPS ≥ 4 (Figure). Pharmacologic thromboprophylaxis was inappropriately given or withheld based on their PPS for 91 (36.4%) patients. This included 58 (49.2%) patients in the low-risk group (PPS < 4) who had thromboprophylaxis inappropriately given and 33 (25.0%) patients in the high-risk group (PPS ≥ 4) who had thromboprophylaxis inappropriately withheld. Of the 58 patients with a PPS < 4 who were given prophylaxis, only 2 (3.4%) patients had documented rationale as to why anticoagulation was administered. Of the 132 patients with a PPS ≥ 4, 44 patients had thromboprophylaxis withheld. Eleven (8.3%) patients had thromboprophylaxis appropriately withheld due to presence or concern for bleeding. Commonly documented rationale for inappropriately withholding thromboprophylaxis when indicated included use of sequential compression devices (40.9%), pancytopenia (18.2%), dual antiplatelet therapy (9.1%), or patient was ambulatory (4.5%).

 

A secondary endpoint characterized patients at highest risk for developing a VTE after hospitalization for an acute illness. Seventy patients were readmitted within 45 days of discharge from the index hospitalization with ICD descriptions for embolism or thrombosis. Only 15 of those patients were readmitted with a newly diagnosed VTE not previously identified; 14 (93.3%) had a PPS ≥ 4 upon index admission and 10 (66.7%) appropriately received pharmacologic prophylaxis within 24 hours of admission. Of the 15 patients, 3 (20.0%) did not receive pharmacologic thromboprophylaxis within 24 hours of admission and 1 (6.7%) received thromboprophylaxis despite having a PPS < 4.

Looking at IMPROVEDD scores for the 15 patients at the index hospitalization discharge, 1 (6.7%) patient had an IMPROVEDD score < 2, 11 (73.3%) patients had IMPROVEDD scores ≥ 2, and 3 (20.0%) patients had IMPROVEDD scores ≥ 4. Two of the patients with IMPROVEDD scores ≥ 4 had a history of VTE and were aged > 60 years. Of the 15 patients reviewed, 7 had a diagnosis of cancer, and 3 were actively undergoing chemotherapy.

Discussion

PPS is the RAM embedded in our system’s order set, which identifies hospitalized medical patients at risk for VTE.⁶ In the original study that validated PPS, the results suggested that implementation of preventive measures during hospitalization in patients labeled as having high thrombotic risk confers longstanding protection against thromboembolic complications in comparison with untreated patients.⁶ However, PPS must be used consistently and appropriately to realize this benefit. Our results showed that pharmacologic thromboprophylaxis is frequently inappropriately given or withheld despite the incorporation of a RAM in an admission order set, suggesting there is a significant gap between written policy and actual practice. More than one-third of patients had thromboprophylaxis given or withheld inappropriately according to the PPS calculated manually on review. With this, there is concern for over- and underprescribing of thromboprophylaxis, which increases the risk of adverse events. Overprescribing can lead to unnecessary bleeding complications, whereas underprescribing can lead to preventable VTE.

One issue identified during this study was the need for a user-friendly interface. The PPS calculator currently embedded in our admission order set is a hyperlink to an online calculator. This is time consuming and cumbersome for clinicians tending to a high volume of patients, which may cause them to overlook the calculator and estimate risk based on clinician judgement. Noted areas for improvement regarding thromboprophylaxis during inpatient admissions include the failure to implement or adhere to risk stratification protocols, lack of appropriate assessment for thromboprophylaxis, and the overutilization of pharmacologic thromboprophylaxis in low-risk patients.¹¹

Certain patients develop a VTE postdischarge despite efforts at prevention during their index hospitalization, which led us to explore our secondary endpoint looking at readmissions. Regarding thromboprophylaxis postdischarge, the duration of therapy is an area of current debate.⁵ Extended-duration thromboprophylaxis is defined as anticoagulation prescribed beyond hospitalization for up to 42 days total.^1,12 To date, there have been 5 clinical trials to evaluate the utility of extended-duration thromboprophylaxis in hospitalized medically ill patients. While routine use is not recommended by the 2018 American Society of Hematology guidelines for management of VTE, more recent data suggest certain medically ill patients may derive benefit from extended-duration thromboprophylaxis.⁴ The IMPROVEDD score aimed to address this need, which is why it was calculated on index discharge for our patients readmitted within 45 days. Research is still needed to identify such patients and RAMs for capturing these subpopulations.^1,11

Our secondary endpoint sought to characterize patients at highest risk for developing a VTE postdischarge. Of the 15 patients reviewed, 7 had a diagnosis of cancer and 3 were actively undergoing chemotherapy. With that, the Khorana Risk Score may have been a more appropriate RAM for some given the Khorana score is validated in ambulatory patients undergoing chemotherapy. D-dimer was only collected for 1 of the 15 patients, therefore, VTE risk could have been underestimated with the IMPROVEDD scores calculated. More than 75% of patients readmitted for VTE appropriately received thromboprophylaxis on index admission yet still went on to develop a VTE. It is essential to increase clinician awareness about hospital-acquired and postdischarge VTE. In line with guidance from the North American Thrombosis Forum, extended-duration thromboprophylaxis should be thoughtfully considered in high-risk patients.⁵ Pathways, including follow-up, are needed to implement postdischarge thromboprophylaxis when appropriate

 

Limitations

There were some inherent limitations to this study with its retrospective nature and small sample size. Data extraction was limited to health records within the VA, so there is a chance relevant history could be missed via incomplete documentation. Thus, our results could be an underestimation of postdischarge VTE prevalence if patients sought medical attention outside of the VA. Given this study was a retrospective chart review, data collection was limited to what was explicitly documented in the chart. Rationale for giving thromboprophylaxis when not indicated or holding when indicated may have been underestimated if clinicians did not document thoroughly in the electronic health record. Last, for the secondary endpoint reviewing the IMPROVEDD score, a D-dimer was not consistently obtained on admission, which could lead to underestimation of risk.

Conclusions

The results of this study showed that more than one-third of patients admitted to our facility within the prespecified timeframe had pharmacologic thromboprophylaxis inappropriately given or withheld according to a PPS manually calculated on admission. The PPS calculator currently embedded within our admission order set is not being utilized appropriately or consistently in clinical practice. Additionally, results from the secondary endpoint looking at IMPROVEDD scores highlight an unmet need for thromboprophylaxis at discharge. Pathways are needed to implement postdischarge thromboprophylaxis when appropriate for patients at highest thromboembolic risk.

 

Venous thromboembolism (VTE) presents as deep venous thromboembolism (DVT) or pulmonary embolism (PE). VTE is the third most common vascular disease and a leading cardiovascular complication.^1,2 Hospitalized patients are at increased risk of developing VTE due to multiple factors such as inflammatory processes from acute illness, recent surgery or trauma leading to hypercoagulable states, and prolonged periods of immobilization.³ Additional risk factors for complications include presence of malignancy, obesity, and prior history of VTE. About half of VTE cases in the community setting occur as a result of a hospital admission for recent or ongoing acute illness or surgery.¹ Hospitalized patients are often categorized as high risk for VTE, and this risk may persist postdischarge.⁴

The risk of hospital-associated VTE may be mitigated with either mechanical or pharmacologic thromboprophylaxis.⁵ Risk assessment models (RAMs), such as Padua Prediction Score (PPS) and IMPROVEDD, have been developed to assist in evaluating hospitalized patients’ risk of VTE and need for pharmacologic thromboprophylaxis (Table 1).^1,5 The PPS is externally validated and can assist clinicians in VTE risk assessment when integrated into clinical decision making.⁶ Patients with a PPS ≥ 4 are deemed high risk for VTE, and pharmacologic thromboprophylaxis is indicated as long as the patient is not at high risk for bleeding. IMPROVEDD added D-dimer as an additional risk factor to IMPROVE and was validated in 2017 to help predict the risk of symptomatic VTE in acutely ill patients hospitalized for up to 77 days.⁷ IMPROVEDD scores ≥ 2 identify patients at high risk for symptomatic VTE through 77 days hospitalization, while scores ≥ 4 identify patients who may qualify for extended thromboprophylaxis.⁷ Despite their utility, RAMs may not be used appropriately within clinical practice, and whether patients should receive extended-duration thromboprophylaxis postdischarge and for how long is debatable.⁵

VTE events contribute to increased health care spending, morbidity, and mortality, thus it is imperative to evaluate current hospital practices with respect to appropriate prescribing of pharmacologic thromboprophylaxis.⁸ Appropriately identifying high-risk patients and prescribing pharmacologic thromboprophylaxis to limit preventable VTEs is essential. Conversely, it is important to withhold pharmacologic thromboprophylaxis from those deemed low risk to limit bleeding complications.⁹ Health care professionals must be good stewards of anticoagulant prescribing when implementing these tools along with clinical knowledge to weigh the risks vs benefits to promote medication safety and prevent further complications.¹⁰This quality improvement project aimed to evaluate if VTE thromboprophylaxis was appropriately given or withheld in hospitalized medical patients based on PPS calculated upon admission using a link to an online calculator embedded within an admission order set. Additionally, this study aimed to characterize patients readmitted for VTE within 45 days postdischarge to generate hypotheses for future stu

 

Methods

This was an observational, retrospective cohort study that took place at the US Department of Veterans Affairs (VA) Tennessee Valley Healthcare System (TVHS). TVHS is a multisite health care system with campuses in Nashville and Murfreesboro. Clinical pharmacists employed at the study site and the primary research investigators designed this study and oversaw its execution. The study was reviewed and deemed exempt as a quality improvement study by the TVHS Institutional Review Board.

This study included adult veterans aged ≥ 18 years admitted to a general medicine floor or the medical intensive care unit between June 1, 2017, and June 30, 2020. Patients were excluded if they were on chronic therapeutic anticoagulation prior to their index hospitalization, required therapeutic anticoagulation on admission for index hospitalization (ie, acute coronary syndrome requiring a heparin drip), or were bedded within the surgical intensive care unit. All patients admitted to the TVHS within the prespecified date range were extracted from the electronic health record. A second subset of patients meeting inclusion criteria and readmitted for VTE within 45 days of index hospitalization with International Classification of Diseases, Tenth Revision (ICD-10) descriptions including thrombosis or embolism were extracted for review of a secondary endpoint. Patients with preexisting clots, history of prior DVT or PE, or history of portal vein thrombosis were not reviewed.

The primary endpoint was the percentage of patients for whom pharmacologic thromboprophylaxis was appropriately initiated or withheld based on a PPS calculated upon admission (Table 2). PPS was chosen for review as it is the only RAM currently used at TVHS. Secondary endpoints were the percentage of patients with documented rationale for ordering thromboprophylaxis when not indicated, based on PPS, or withholding despite indication as well as the number of patients readmitted to TVHS for VTE within 45 days of discharge with IMPROVEDD scores ≥ 4 and < 4 (eAppendix available at doi:10.12788/fp.0291). The primary investigators performed a manual health record review of all patients meeting inclusion criteria. Descriptive statistics were used given this was a quality improvement study, therefore, sample size and power calculations were not necessary. Data were stored in Microsoft Excel spreadsheets that were encrypted and password protected. To maintain security of personal health information, all study files were kept on the TVHS internal network, and access was limited to the research investigators.

Results

Two hundred fifty patients meeting inclusion criteria were randomly selected for review for the primary endpoint. Of the patients reviewed for the primary endpoint, 118 had a PPS < 4 and 132 a PPS ≥ 4 (Figure). Pharmacologic thromboprophylaxis was inappropriately given or withheld based on their PPS for 91 (36.4%) patients. This included 58 (49.2%) patients in the low-risk group (PPS < 4) who had thromboprophylaxis inappropriately given and 33 (25.0%) patients in the high-risk group (PPS ≥ 4) who had thromboprophylaxis inappropriately withheld. Of the 58 patients with a PPS < 4 who were given prophylaxis, only 2 (3.4%) patients had documented rationale as to why anticoagulation was administered. Of the 132 patients with a PPS ≥ 4, 44 patients had thromboprophylaxis withheld. Eleven (8.3%) patients had thromboprophylaxis appropriately withheld due to presence or concern for bleeding. Commonly documented rationale for inappropriately withholding thromboprophylaxis when indicated included use of sequential compression devices (40.9%), pancytopenia (18.2%), dual antiplatelet therapy (9.1%), or patient was ambulatory (4.5%).

 

A secondary endpoint characterized patients at highest risk for developing a VTE after hospitalization for an acute illness. Seventy patients were readmitted within 45 days of discharge from the index hospitalization with ICD descriptions for embolism or thrombosis. Only 15 of those patients were readmitted with a newly diagnosed VTE not previously identified; 14 (93.3%) had a PPS ≥ 4 upon index admission and 10 (66.7%) appropriately received pharmacologic prophylaxis within 24 hours of admission. Of the 15 patients, 3 (20.0%) did not receive pharmacologic thromboprophylaxis within 24 hours of admission and 1 (6.7%) received thromboprophylaxis despite having a PPS < 4.

Looking at IMPROVEDD scores for the 15 patients at the index hospitalization discharge, 1 (6.7%) patient had an IMPROVEDD score < 2, 11 (73.3%) patients had IMPROVEDD scores ≥ 2, and 3 (20.0%) patients had IMPROVEDD scores ≥ 4. Two of the patients with IMPROVEDD scores ≥ 4 had a history of VTE and were aged > 60 years. Of the 15 patients reviewed, 7 had a diagnosis of cancer, and 3 were actively undergoing chemotherapy.

Discussion

PPS is the RAM embedded in our system’s order set, which identifies hospitalized medical patients at risk for VTE.⁶ In the original study that validated PPS, the results suggested that implementation of preventive measures during hospitalization in patients labeled as having high thrombotic risk confers longstanding protection against thromboembolic complications in comparison with untreated patients.⁶ However, PPS must be used consistently and appropriately to realize this benefit. Our results showed that pharmacologic thromboprophylaxis is frequently inappropriately given or withheld despite the incorporation of a RAM in an admission order set, suggesting there is a significant gap between written policy and actual practice. More than one-third of patients had thromboprophylaxis given or withheld inappropriately according to the PPS calculated manually on review. With this, there is concern for over- and underprescribing of thromboprophylaxis, which increases the risk of adverse events. Overprescribing can lead to unnecessary bleeding complications, whereas underprescribing can lead to preventable VTE.

One issue identified during this study was the need for a user-friendly interface. The PPS calculator currently embedded in our admission order set is a hyperlink to an online calculator. This is time consuming and cumbersome for clinicians tending to a high volume of patients, which may cause them to overlook the calculator and estimate risk based on clinician judgement. Noted areas for improvement regarding thromboprophylaxis during inpatient admissions include the failure to implement or adhere to risk stratification protocols, lack of appropriate assessment for thromboprophylaxis, and the overutilization of pharmacologic thromboprophylaxis in low-risk patients.¹¹

Certain patients develop a VTE postdischarge despite efforts at prevention during their index hospitalization, which led us to explore our secondary endpoint looking at readmissions. Regarding thromboprophylaxis postdischarge, the duration of therapy is an area of current debate.⁵ Extended-duration thromboprophylaxis is defined as anticoagulation prescribed beyond hospitalization for up to 42 days total.^1,12 To date, there have been 5 clinical trials to evaluate the utility of extended-duration thromboprophylaxis in hospitalized medically ill patients. While routine use is not recommended by the 2018 American Society of Hematology guidelines for management of VTE, more recent data suggest certain medically ill patients may derive benefit from extended-duration thromboprophylaxis.⁴ The IMPROVEDD score aimed to address this need, which is why it was calculated on index discharge for our patients readmitted within 45 days. Research is still needed to identify such patients and RAMs for capturing these subpopulations.^1,11

Our secondary endpoint sought to characterize patients at highest risk for developing a VTE postdischarge. Of the 15 patients reviewed, 7 had a diagnosis of cancer and 3 were actively undergoing chemotherapy. With that, the Khorana Risk Score may have been a more appropriate RAM for some given the Khorana score is validated in ambulatory patients undergoing chemotherapy. D-dimer was only collected for 1 of the 15 patients, therefore, VTE risk could have been underestimated with the IMPROVEDD scores calculated. More than 75% of patients readmitted for VTE appropriately received thromboprophylaxis on index admission yet still went on to develop a VTE. It is essential to increase clinician awareness about hospital-acquired and postdischarge VTE. In line with guidance from the North American Thrombosis Forum, extended-duration thromboprophylaxis should be thoughtfully considered in high-risk patients.⁵ Pathways, including follow-up, are needed to implement postdischarge thromboprophylaxis when appropriate

 

Limitations

There were some inherent limitations to this study with its retrospective nature and small sample size. Data extraction was limited to health records within the VA, so there is a chance relevant history could be missed via incomplete documentation. Thus, our results could be an underestimation of postdischarge VTE prevalence if patients sought medical attention outside of the VA. Given this study was a retrospective chart review, data collection was limited to what was explicitly documented in the chart. Rationale for giving thromboprophylaxis when not indicated or holding when indicated may have been underestimated if clinicians did not document thoroughly in the electronic health record. Last, for the secondary endpoint reviewing the IMPROVEDD score, a D-dimer was not consistently obtained on admission, which could lead to underestimation of risk.

Conclusions

The results of this study showed that more than one-third of patients admitted to our facility within the prespecified timeframe had pharmacologic thromboprophylaxis inappropriately given or withheld according to a PPS manually calculated on admission. The PPS calculator currently embedded within our admission order set is not being utilized appropriately or consistently in clinical practice. Additionally, results from the secondary endpoint looking at IMPROVEDD scores highlight an unmet need for thromboprophylaxis at discharge. Pathways are needed to implement postdischarge thromboprophylaxis when appropriate for patients at highest thromboembolic risk.

 

COVID-19, caused by SARS-CoV-2, is more severe in individuals with underlying illnesses. Because complete social distancing might be more difficult in nursing homes and community living centers (CLCs), public health leaders and clinicians have been concerned about the epidemiology and disease course in nursing homes even before the COVID-19 pandemic.1-7 A report of a COVID-19 outbreak in a nursing home facility in King County, Washington, documented a 33.7% overall fatality rate for residents and 52.4% among the most critically ill.^4,5 The experience at King County, Washington, shows that proactive steps to identify, monitor, and apply preventive control measures is important for future outbreaks.⁵

Reverse transcriptase polymerase chain reaction (RT-PCR) testing produces a cycle threshold (CT) or cycle number (CN) that correlates with viral load and infectiousness. ^8-14 CT/CN represents the number of RT-PCR cycles required for the fluorescent signal to cross the detection threshold (exceed background level) and is inversely proportional to the viral load. Effectively, the higher the viral load, the lower the CT/ CN value (Figure 1). Tracking CT/CN values was not documented in the Washington nursing home outbreak. Reports of COVID- 19 testing in CLCs during outbreaks are sparse, and CT/CN values and demographic distribution of these veterans has not been reported.¹⁵ The CLC veteran population, with known higher vulnerability to infection and chronic diseases, is epidemiologically different from the general nursing home population.^15-18 To address these literature gaps, we present the first report of COVID- 19 testing with CT/CN value correlations in the high-risk veteran CLC population.

Methods

A retrospective review of all COVID-19 CT/CN testing at the Corporal Michael J. Crescenz Veterans Affairs Medical Center (VAMC) CLC in Philadelphia, Pennsylvania, from March 28, 2020, to April 24, 2020, was performed with a US Department of Veterans Affairs (VA) Veterans Health Information System Architecture VistA/FileMan search. Only veteran residents were included in this review. Data collected included initial and serial test results, CT/CN on positive test results, test dates, testing platform used, demographic information (age, self-reported ethnicity, and sex), and clinical follow-up information. Health records were reviewed retrospectively to identify death, the first day after diagnosis with no documented symptoms, or hospitalization status.

RT-PCR testing was performed with the Abbott RealTime SARS-CoV-2 assay on the Abbott m2000 platform and the Xpert Xpress SARS-CoV-2 assay on the Cepheid Infinity platform. The Xpert Xpress assay gave 2 CT values for the E and N2 targets on positive samples.¹⁹ For this assay to indicate a positive specimen, amplification by RT-PCR of the N2 target or both the N2 and E target is required. The Xpert Xpress assay results as presumptive positive if only the E target amplified. This assay counts a maximum of 45 cycles. The Abbott RealTime SARS-CoV-2 assay gave 1 CN derived from the RNA-dependent RNA polymerase and N targets on positive samples.20 The Abbott assay on the m2000 counts a maximum of 37 cycles. The CT/CN value is the number of cycles required by RT-PCR for the fluorescence signal to cross a threshold value exceeding background level.^19,20

Samples that are negative for COVID-19 by RT-PCR do not produce a CT/CN value. Although both instruments were used for RT-PCR, the precise CT/CN values are not interchangeable and CT/CN observations over time between the 2 instruments during the disease course would be based on CT/CN value movement (general upward or downward trend) rather than absolute CT/CN differences. Both assays have been approved by emergency use authorization as qualitative tests for the presence/absence of COVID-19. Although the CT/CN value is available to laboratory staff after test completion, the CT/CN value is not reported routinely in the patient health record. All veteran patients identified on the initial review from March 28, 2020, to April 24, 2020, had all serial COVID-19 testing recorded until November 10, 2020. The CN values at the limit of detection (LOD) for the Abbott m2000 platform from the initial validation study were reviewed for reference.²¹

Results

Of 80 patients, 25 (31%) were COVID-19 positive over the course of testing. The study population had a mean age of 73.5 years; 92% were aged > 60 years. The group was predominantly male (79 male vs 1 female). Among the 77 patients with a stated ethnicity, 39 (51%) were African American. In comparison, 43% of residents in Philadelphia County are African American (Table).^22,23 Additionally, a previously published total COVID-19 tested population by ethnicity at the same regional VAMC revealed 46.8% of tested veteran patients were African American. ²⁴ Three patients had no stated ethnicity. Among those who tested positive, 11 were African American patients, 12 were White patients, and 2 had no stated ethnicity. Four patients tested positive on their first test. The other 21 patients were positive on repeat testing. Interestingly, 6 patients had 1 initial negative test before a positive test, 6 patients had 2, 8 patients had 3, and 1 patient had 4 initial negative tests before a positive test result. Among the 25 positive patients, 22 were either positive within 10 days of the initial negative test result or initially positive (Figure 2). Three patients who tested positive after 10 days did so at 16, 20, and 21 days after the initial negative test result. Among the 25 positive patients, 23 had initial and serial testing from both the Abbott and Xpert Xpress assays. The remaining 2 positive patients had initial and serial testing from the Abbott assay exclusively.

 

Only positive COVID-19 results by RTPCR produced a CT/CN value. After disease resolution with a negative test, no CT/CN value was produced with the negative test result on either testing platform. Because repeat testing after the initial positive result took place no sooner than 10 days, we observed that the CT/CN value increased after the initial positive result until the disease resolved, and a negative result was obtained (eAppendix 1, available online at doi:10.12788/fp.0276). A t test comparing the initial CT/CN value to the value more than 10 days after the initial positive showed the CT/CN was statistically significantly higher (P < .05).

Prompt repeat testing after the initial test can show a decrease in the CT/CN value because of increasing viral load before the expected increase until disease resolution if the initial test caught the infection early. Twelve patients had a negative test result between 2 serial positive results. These negative test results occurred later, near the end of the disease course. Among the 12 patients with this positive-negativepositive CT/CN pattern, 7 were symptomatic and no longer had documented symptoms or hospitalization around the time of this positive-negative-positive pattern. Four of these individuals were asymptomatic during the entire infection course. One of the 12 patients with this pattern expired with the negative result occurring on day 27 of the disease in the context of rising CT/CN. One of these 12 patients only had a presumptive positive test result on the Cepheid because it detected only the E target with a CT value of 38.7. In 1 of the 12 patients, the negative test result occurred between 2 positive test results with CT/CN values < 20 (12.05 and 19.05 for the positive tests before and after the negative result, respectively). When the initial CT/CN values was separated based on ethnicity, the average CT/CN value for African Americans (23.3) was higher than for other ethnicities (19.9), although it did not reach statistical significance (P = .35).

 

Ten of the 25 patients testing positive were admitted to the hospital, including 1 admitted 15 days before diagnosis (patient 20) and 1 admitted 80 days after diagnosis (patient 7). Among these 10 patients, 6 were admitted to the intensive care unit, including patient 7. None of the patients were intubated. Three of the 10 admitted patients died (patients 7, 20, and 24). Patient 7 was a 79-year-old male with a history of dementia, cerebrovascular accident, hypertension, hyperlipidemia, and chronic kidney disease with symptoms of lethargy and refusal of oral intake when he was diagnosed with COVID-19. He was admitted 80 days after diagnosis for hyponatremia and acute renal failure, with death on day 87 recorded as complications from the earlier COVID-19 infection. Patient 20, an 89-year-old male with a history of dementia, chronic kidney disease, and hyperlipidemia, had been admitted with fever, cough, and leukocytosis 17 days before COVID-19 diagnosis. He continued to be symptomatic after diagnosis with development of hypotension, dehydration, and refusal of oral intake while on comfort measures/endof- life care and died 15 days after COVID- 19 infection diagnosis. Patient 24 was a 96-year-old male with history of heart failure, hypertension, coronary artery disease, prostate carcinoma, and dementia who developed a cough at the time of diagnosis; because of his underlying condition, he remained in the CLC on comfort care. His symptoms, including hypoxia, worsened until he died 7 days after diagnosis.

Among the 25 patients, 17 were symptomatic at the time of diagnosis; the 14 initially symptomatic patients who survived improved clinically and returned to baseline. Eight of the 25 patients were asymptomatic initially and 3 developed symptoms 2 to 5 days after diagnosis. Only 1 patient who remained asymptomatic was admitted for inability to adhere to quarantine at the CLC. Review of the health records of all surviving symptomatic patients showed symptom resolution with return to baseline that corresponds to an increasing CT/CN value. A 1-tailed t test comparing the initial CT/ CN at the time of diagnosis to the last CT/CN value for symptomatic patients who recovered revealed a statistically significant increase (P < .05). For the symptomatic, symptom resolution and hospital discharge took (if required) a mean 20 days (range, 7-46). Among those who were not hospitalized, symptoms resolved in 7 to 36 days (18 days). Among those requiring hospitalization at any time (excluding patients who died or were asymptomatic), symptom and hospitalization resolution took a mean 22 days (range, 10-46). Asymptomatic patients (patients 8, 10, 15, 16, and 25) also showed increasing CT/CN value during the infection course, although there was no correlation with the continued lack of symptoms.

During the initial validation of the Abbott m2000 instrument, an LOD study included concentrations of 1000, 500, 250, 100, 70, 60, and 50 virus copies/mL (eAppendix 2, available online at doi:10.12788/fp.0276).21 The average CN at 100 virus copies/mL—the manufacturer provided LOD in the instructions for use—was 25.74.20 At a concentration of one-half that (50 virus copies/mL), the average CN was 28.39.

Discussion

This is the first study in the English literature to track CT/CN values as part of serial testing of a veteran CLC. Widescale testing and repeat screening in the absence of symptoms of nursing home residents would identify those who are infected and allow providers to track viral load clearance.^9-14 CT/CN values, when serially tracked during the infection course, appear to increase with illness resolution, consistent with earlier reports that CT/CN correlates with viral load.^8-14 Serial CT/CN values that are high (> 25) and continue to increase with each test suggest progression toward disease resolution or viral RNA clearance.^8-14 After symptom resolution, patients can have a persistent low level of viral shedding (corresponding to a high CT/CN value).^10-14,25 Near the end of disease resolution, a negative serial RT-PCR sample test before a subsequent positive might be a promising clinical sign of near disease recovery. Once the viral load is low with a CT/CN significantly higher than 25, some specimens might result as negative but turn up positive on subsequent sampling with a high CT/CN value. This pattern, with attendant high CT/CN values for the positive results, are consistent with the known effect of viral load (ie, a low viral load correlates to a high CT/CN) and adequacy of specimen collection on CT/CN values^.25 If the patient’s viral load is low, the sample collected might have a viral load at or near the testing platform’s LOD.

For Abbott m2000, the manufacturer provided LOD is 100 virus copies/mL, although the instrument was able to detect virus concentrations below that level during the initial validation.²⁰ The actual LOD of the instrument at our institution is < 100 virus copies/mL. For the Cepheid Xpert Xpress SARS-CoV-2 assay, the manufacturer-provided LOD is 250 virus copies/mL.19 An LOD study including samples below the manufacturer-provided LOD was not part of the initial validation study for the Xpert Xpress assay. Nonetheless, the virus concentration of samples with very high CT values at or near the maximum CT value of 45 is expected to be at or near the platform’s actual LOD.

If the samples collected near the end of the patient’s disease course have viral loads near these low concentrations, the encouraging positive-negative-positive pattern with high CT/CN values might be a promising sign for viral clearance. On the other hand, a positive-negative-positive pattern in the setting of low CT/CN values before and after the negative test might indicate poor sampling for the negative specimen. The back-and-forth or positive-negative-positive pattern generally appears to indicate near resolution of the infection course, although clinical correlation is necessary to rule out inadequate sampling earlier in the disease course or prolonged viral RNA shedding.^9-14 In all of the surviving symptomatic patients who showed the positive-negative-positive pattern, this sign occurred around or after symptom resolution. It also is important to consider that in some patients, SARS-CoV-2 RNA might remain detectable with increasing CT/CN after symptom resolution, and samples from these patients might not result positive. Therefore, CT/CN values cannot be interpreted without considering the clinical picture.²⁵

Studies on infectiousness and virus culture from COVID-19 samples with CT/ CN correlation have shown that patients with high CT/CN at the end of their disease course might not be as infectious.^9-14,25 Because 1 patient had a presumptive positive result after the negative result, this study shows that this positive-negative-positive pattern could include presumptive positive results. Also, in the setting of a recent positive result on the same testing platform, a patient with this pattern is presumed to be positive for COVID-19 RNA because of scant viral material.

Taiwan’s public health response to the outbreak illustrates the ability to mitigate an outbreak throughout a society.²⁶ These actions could help blunt an outbreak within a civilian nursing home population.⁵ Mitigation within a veteran CLC population has been documented, but the study, which focused on mitigation, did not consider CT/CN values, demographic distribution, testing access of the studied population, or laboratory findings related to disease pathophysiology.¹⁵ A key ingredient in widescale, serial testing is the availability of a rapid turnaround from testing in-house that allowed identification within 24 hours instead of several days at a reference laboratory. ¹⁵ Rapid widescale testing would allow clinical teams to optimize the Triangle of Benefit of Widescale Timely Tests for CLC (Figure 3).¹⁵ Timely laboratory testing remains pivotal for CLC veteran residents to aid successful clinical triage and management. Reporting serial CT/CN values can provide additional information to clinicians about the disease course because CT/ CN correlates with viral load, which varies based on where the patient is in the disease course.^9-14 CT/CN values carry significant prognostic value, particularly with respect to intubation and mortality.⁸

 

Limitations

Important limitations to our study include the use of 2 separate RT-PCR platforms. Using different RT-PCR platforms is common in clinical laboratories trying to take advantage of the unique characteristics of different platforms—for example, turnaround time vs high throughput— to manage COVID-19 testing workflow.²⁵ However, the exact CT/CN values obtained from each platform might not translate to the other, and the general trend (CT/CN values are rising or falling across serial tests) rather than a single value could be useful for clinical correlation. Even when the same platform is used for the serial testing, CT/CN values can be affected by adequacy of specimen collection; therefore, clinical correlation and considering the trend in CT/CN values is necessary for interpretation.^10-14,25 Because of the known trend in viral dynamics, a positive specimen collected with a high CT/CN followed by a subsequent (within 2 days) positive specimen collected with a low CT/CN might be compatible with early detection of COVID- 19 infection in the appropriate clinical context. ^10-14 However, detection late in the infection course or even after the symptomatic disease resolved with prolonged viral shedding might show serial positive samples with increasing CT/CN values.^10-14

Patients with prolonged viral shedding might not be infectious.27 Because of the clinical correlation required for interpretation and the other factors that might affect CT/CN values, recommendations advise against using CT/CN values in clinical practice at this time, although these recommendations could change with future research.25 Serial CT/CN values have the potential, if appropriately correlated with the clinical picture, to provide useful information, such as whether the viral load of the sample is relatively high or low and increasing or decreasing.

Veterans, as a population, are more susceptible to poor health outcomes and morbidity compared with similar civilian counterparts.^2,14-16 Veteran CLC patients likely would experience worse outcomes with COVID-19, including more infections, expiration, and morbidity compared with similar general population nursing homes. Similar to what had been reported for the civilian population, a trend (high CT/CN values early in the disease course with repeat testing needed to detect all positives followed by lower CT/CN value to correlate with increased viral load and then increased CT/CN value as the infection resolved) also was observed in this veteran population.

It has been extensively documented that minority groups experience decreased health care access and worse health outcomes. ^28-30 Considering the critical medical supply shortages, including personal protective equipment, ventilators, and even testing supplies, there is the potential for a resource access disparity by ethnicity.^28-31 Because the VA does not depend on measures of wealth and privilege such as health insurance, there was no disparity noted in access to testing by race or ethnicity at the VAMC CLC. When considering the health outcome of viral load from the measured CT/CN value, the viral loads of African American patients and those of other ethnicities was not significantly different in this study.

Conclusions

This is the first study to bring up critical points including serial CT/CN value correlation in RT-PCR tests, demographic distributions demonstrating easy and equal access in a veteran nursing home to COVID-19 testing, and clinical laboratory signs related to disease pathophysiology. Unlike other populations who have undergone serial CT/CN monitoring, nursing homes represent a particularly vulnerable population who require measures to prevent the spread and mitigate outbreaks of COVID-19.^2,4,5 Test measurements obtained such as the CT/CN value during routine clinical care can provide useful information for public health, epidemiologic, or clinical purposes with appropriate correlation to clinical and other laboratory parameters. This study demonstrates early intervention of serial testing of an outbreak in a veterans nursing home with CT/CN value correlation.

COVID-19, caused by SARS-CoV-2, is more severe in individuals with underlying illnesses. Because complete social distancing might be more difficult in nursing homes and community living centers (CLCs), public health leaders and clinicians have been concerned about the epidemiology and disease course in nursing homes even before the COVID-19 pandemic.1-7 A report of a COVID-19 outbreak in a nursing home facility in King County, Washington, documented a 33.7% overall fatality rate for residents and 52.4% among the most critically ill.^4,5 The experience at King County, Washington, shows that proactive steps to identify, monitor, and apply preventive control measures is important for future outbreaks.⁵

Reverse transcriptase polymerase chain reaction (RT-PCR) testing produces a cycle threshold (CT) or cycle number (CN) that correlates with viral load and infectiousness. ^8-14 CT/CN represents the number of RT-PCR cycles required for the fluorescent signal to cross the detection threshold (exceed background level) and is inversely proportional to the viral load. Effectively, the higher the viral load, the lower the CT/ CN value (Figure 1). Tracking CT/CN values was not documented in the Washington nursing home outbreak. Reports of COVID- 19 testing in CLCs during outbreaks are sparse, and CT/CN values and demographic distribution of these veterans has not been reported.¹⁵ The CLC veteran population, with known higher vulnerability to infection and chronic diseases, is epidemiologically different from the general nursing home population.^15-18 To address these literature gaps, we present the first report of COVID- 19 testing with CT/CN value correlations in the high-risk veteran CLC population.

Methods

A retrospective review of all COVID-19 CT/CN testing at the Corporal Michael J. Crescenz Veterans Affairs Medical Center (VAMC) CLC in Philadelphia, Pennsylvania, from March 28, 2020, to April 24, 2020, was performed with a US Department of Veterans Affairs (VA) Veterans Health Information System Architecture VistA/FileMan search. Only veteran residents were included in this review. Data collected included initial and serial test results, CT/CN on positive test results, test dates, testing platform used, demographic information (age, self-reported ethnicity, and sex), and clinical follow-up information. Health records were reviewed retrospectively to identify death, the first day after diagnosis with no documented symptoms, or hospitalization status.

RT-PCR testing was performed with the Abbott RealTime SARS-CoV-2 assay on the Abbott m2000 platform and the Xpert Xpress SARS-CoV-2 assay on the Cepheid Infinity platform. The Xpert Xpress assay gave 2 CT values for the E and N2 targets on positive samples.¹⁹ For this assay to indicate a positive specimen, amplification by RT-PCR of the N2 target or both the N2 and E target is required. The Xpert Xpress assay results as presumptive positive if only the E target amplified. This assay counts a maximum of 45 cycles. The Abbott RealTime SARS-CoV-2 assay gave 1 CN derived from the RNA-dependent RNA polymerase and N targets on positive samples.20 The Abbott assay on the m2000 counts a maximum of 37 cycles. The CT/CN value is the number of cycles required by RT-PCR for the fluorescence signal to cross a threshold value exceeding background level.^19,20

Samples that are negative for COVID-19 by RT-PCR do not produce a CT/CN value. Although both instruments were used for RT-PCR, the precise CT/CN values are not interchangeable and CT/CN observations over time between the 2 instruments during the disease course would be based on CT/CN value movement (general upward or downward trend) rather than absolute CT/CN differences. Both assays have been approved by emergency use authorization as qualitative tests for the presence/absence of COVID-19. Although the CT/CN value is available to laboratory staff after test completion, the CT/CN value is not reported routinely in the patient health record. All veteran patients identified on the initial review from March 28, 2020, to April 24, 2020, had all serial COVID-19 testing recorded until November 10, 2020. The CN values at the limit of detection (LOD) for the Abbott m2000 platform from the initial validation study were reviewed for reference.²¹

Results

Of 80 patients, 25 (31%) were COVID-19 positive over the course of testing. The study population had a mean age of 73.5 years; 92% were aged > 60 years. The group was predominantly male (79 male vs 1 female). Among the 77 patients with a stated ethnicity, 39 (51%) were African American. In comparison, 43% of residents in Philadelphia County are African American (Table).^22,23 Additionally, a previously published total COVID-19 tested population by ethnicity at the same regional VAMC revealed 46.8% of tested veteran patients were African American. ²⁴ Three patients had no stated ethnicity. Among those who tested positive, 11 were African American patients, 12 were White patients, and 2 had no stated ethnicity. Four patients tested positive on their first test. The other 21 patients were positive on repeat testing. Interestingly, 6 patients had 1 initial negative test before a positive test, 6 patients had 2, 8 patients had 3, and 1 patient had 4 initial negative tests before a positive test result. Among the 25 positive patients, 22 were either positive within 10 days of the initial negative test result or initially positive (Figure 2). Three patients who tested positive after 10 days did so at 16, 20, and 21 days after the initial negative test result. Among the 25 positive patients, 23 had initial and serial testing from both the Abbott and Xpert Xpress assays. The remaining 2 positive patients had initial and serial testing from the Abbott assay exclusively.

 

Only positive COVID-19 results by RTPCR produced a CT/CN value. After disease resolution with a negative test, no CT/CN value was produced with the negative test result on either testing platform. Because repeat testing after the initial positive result took place no sooner than 10 days, we observed that the CT/CN value increased after the initial positive result until the disease resolved, and a negative result was obtained (eAppendix 1, available online at doi:10.12788/fp.0276). A t test comparing the initial CT/CN value to the value more than 10 days after the initial positive showed the CT/CN was statistically significantly higher (P < .05).

Prompt repeat testing after the initial test can show a decrease in the CT/CN value because of increasing viral load before the expected increase until disease resolution if the initial test caught the infection early. Twelve patients had a negative test result between 2 serial positive results. These negative test results occurred later, near the end of the disease course. Among the 12 patients with this positive-negativepositive CT/CN pattern, 7 were symptomatic and no longer had documented symptoms or hospitalization around the time of this positive-negative-positive pattern. Four of these individuals were asymptomatic during the entire infection course. One of the 12 patients with this pattern expired with the negative result occurring on day 27 of the disease in the context of rising CT/CN. One of these 12 patients only had a presumptive positive test result on the Cepheid because it detected only the E target with a CT value of 38.7. In 1 of the 12 patients, the negative test result occurred between 2 positive test results with CT/CN values < 20 (12.05 and 19.05 for the positive tests before and after the negative result, respectively). When the initial CT/CN values was separated based on ethnicity, the average CT/CN value for African Americans (23.3) was higher than for other ethnicities (19.9), although it did not reach statistical significance (P = .35).

 

Ten of the 25 patients testing positive were admitted to the hospital, including 1 admitted 15 days before diagnosis (patient 20) and 1 admitted 80 days after diagnosis (patient 7). Among these 10 patients, 6 were admitted to the intensive care unit, including patient 7. None of the patients were intubated. Three of the 10 admitted patients died (patients 7, 20, and 24). Patient 7 was a 79-year-old male with a history of dementia, cerebrovascular accident, hypertension, hyperlipidemia, and chronic kidney disease with symptoms of lethargy and refusal of oral intake when he was diagnosed with COVID-19. He was admitted 80 days after diagnosis for hyponatremia and acute renal failure, with death on day 87 recorded as complications from the earlier COVID-19 infection. Patient 20, an 89-year-old male with a history of dementia, chronic kidney disease, and hyperlipidemia, had been admitted with fever, cough, and leukocytosis 17 days before COVID-19 diagnosis. He continued to be symptomatic after diagnosis with development of hypotension, dehydration, and refusal of oral intake while on comfort measures/endof- life care and died 15 days after COVID- 19 infection diagnosis. Patient 24 was a 96-year-old male with history of heart failure, hypertension, coronary artery disease, prostate carcinoma, and dementia who developed a cough at the time of diagnosis; because of his underlying condition, he remained in the CLC on comfort care. His symptoms, including hypoxia, worsened until he died 7 days after diagnosis.

Among the 25 patients, 17 were symptomatic at the time of diagnosis; the 14 initially symptomatic patients who survived improved clinically and returned to baseline. Eight of the 25 patients were asymptomatic initially and 3 developed symptoms 2 to 5 days after diagnosis. Only 1 patient who remained asymptomatic was admitted for inability to adhere to quarantine at the CLC. Review of the health records of all surviving symptomatic patients showed symptom resolution with return to baseline that corresponds to an increasing CT/CN value. A 1-tailed t test comparing the initial CT/ CN at the time of diagnosis to the last CT/CN value for symptomatic patients who recovered revealed a statistically significant increase (P < .05). For the symptomatic, symptom resolution and hospital discharge took (if required) a mean 20 days (range, 7-46). Among those who were not hospitalized, symptoms resolved in 7 to 36 days (18 days). Among those requiring hospitalization at any time (excluding patients who died or were asymptomatic), symptom and hospitalization resolution took a mean 22 days (range, 10-46). Asymptomatic patients (patients 8, 10, 15, 16, and 25) also showed increasing CT/CN value during the infection course, although there was no correlation with the continued lack of symptoms.

During the initial validation of the Abbott m2000 instrument, an LOD study included concentrations of 1000, 500, 250, 100, 70, 60, and 50 virus copies/mL (eAppendix 2, available online at doi:10.12788/fp.0276).21 The average CN at 100 virus copies/mL—the manufacturer provided LOD in the instructions for use—was 25.74.20 At a concentration of one-half that (50 virus copies/mL), the average CN was 28.39.

Discussion

This is the first study in the English literature to track CT/CN values as part of serial testing of a veteran CLC. Widescale testing and repeat screening in the absence of symptoms of nursing home residents would identify those who are infected and allow providers to track viral load clearance.^9-14 CT/CN values, when serially tracked during the infection course, appear to increase with illness resolution, consistent with earlier reports that CT/CN correlates with viral load.^8-14 Serial CT/CN values that are high (> 25) and continue to increase with each test suggest progression toward disease resolution or viral RNA clearance.^8-14 After symptom resolution, patients can have a persistent low level of viral shedding (corresponding to a high CT/CN value).^10-14,25 Near the end of disease resolution, a negative serial RT-PCR sample test before a subsequent positive might be a promising clinical sign of near disease recovery. Once the viral load is low with a CT/CN significantly higher than 25, some specimens might result as negative but turn up positive on subsequent sampling with a high CT/CN value. This pattern, with attendant high CT/CN values for the positive results, are consistent with the known effect of viral load (ie, a low viral load correlates to a high CT/CN) and adequacy of specimen collection on CT/CN values^.25 If the patient’s viral load is low, the sample collected might have a viral load at or near the testing platform’s LOD.

For Abbott m2000, the manufacturer provided LOD is 100 virus copies/mL, although the instrument was able to detect virus concentrations below that level during the initial validation.²⁰ The actual LOD of the instrument at our institution is < 100 virus copies/mL. For the Cepheid Xpert Xpress SARS-CoV-2 assay, the manufacturer-provided LOD is 250 virus copies/mL.19 An LOD study including samples below the manufacturer-provided LOD was not part of the initial validation study for the Xpert Xpress assay. Nonetheless, the virus concentration of samples with very high CT values at or near the maximum CT value of 45 is expected to be at or near the platform’s actual LOD.

If the samples collected near the end of the patient’s disease course have viral loads near these low concentrations, the encouraging positive-negative-positive pattern with high CT/CN values might be a promising sign for viral clearance. On the other hand, a positive-negative-positive pattern in the setting of low CT/CN values before and after the negative test might indicate poor sampling for the negative specimen. The back-and-forth or positive-negative-positive pattern generally appears to indicate near resolution of the infection course, although clinical correlation is necessary to rule out inadequate sampling earlier in the disease course or prolonged viral RNA shedding.^9-14 In all of the surviving symptomatic patients who showed the positive-negative-positive pattern, this sign occurred around or after symptom resolution. It also is important to consider that in some patients, SARS-CoV-2 RNA might remain detectable with increasing CT/CN after symptom resolution, and samples from these patients might not result positive. Therefore, CT/CN values cannot be interpreted without considering the clinical picture.²⁵

Studies on infectiousness and virus culture from COVID-19 samples with CT/ CN correlation have shown that patients with high CT/CN at the end of their disease course might not be as infectious.^9-14,25 Because 1 patient had a presumptive positive result after the negative result, this study shows that this positive-negative-positive pattern could include presumptive positive results. Also, in the setting of a recent positive result on the same testing platform, a patient with this pattern is presumed to be positive for COVID-19 RNA because of scant viral material.

Taiwan’s public health response to the outbreak illustrates the ability to mitigate an outbreak throughout a society.²⁶ These actions could help blunt an outbreak within a civilian nursing home population.⁵ Mitigation within a veteran CLC population has been documented, but the study, which focused on mitigation, did not consider CT/CN values, demographic distribution, testing access of the studied population, or laboratory findings related to disease pathophysiology.¹⁵ A key ingredient in widescale, serial testing is the availability of a rapid turnaround from testing in-house that allowed identification within 24 hours instead of several days at a reference laboratory. ¹⁵ Rapid widescale testing would allow clinical teams to optimize the Triangle of Benefit of Widescale Timely Tests for CLC (Figure 3).¹⁵ Timely laboratory testing remains pivotal for CLC veteran residents to aid successful clinical triage and management. Reporting serial CT/CN values can provide additional information to clinicians about the disease course because CT/ CN correlates with viral load, which varies based on where the patient is in the disease course.^9-14 CT/CN values carry significant prognostic value, particularly with respect to intubation and mortality.⁸

 

Limitations

Important limitations to our study include the use of 2 separate RT-PCR platforms. Using different RT-PCR platforms is common in clinical laboratories trying to take advantage of the unique characteristics of different platforms—for example, turnaround time vs high throughput— to manage COVID-19 testing workflow.²⁵ However, the exact CT/CN values obtained from each platform might not translate to the other, and the general trend (CT/CN values are rising or falling across serial tests) rather than a single value could be useful for clinical correlation. Even when the same platform is used for the serial testing, CT/CN values can be affected by adequacy of specimen collection; therefore, clinical correlation and considering the trend in CT/CN values is necessary for interpretation.^10-14,25 Because of the known trend in viral dynamics, a positive specimen collected with a high CT/CN followed by a subsequent (within 2 days) positive specimen collected with a low CT/CN might be compatible with early detection of COVID- 19 infection in the appropriate clinical context. ^10-14 However, detection late in the infection course or even after the symptomatic disease resolved with prolonged viral shedding might show serial positive samples with increasing CT/CN values.^10-14

Patients with prolonged viral shedding might not be infectious.27 Because of the clinical correlation required for interpretation and the other factors that might affect CT/CN values, recommendations advise against using CT/CN values in clinical practice at this time, although these recommendations could change with future research.25 Serial CT/CN values have the potential, if appropriately correlated with the clinical picture, to provide useful information, such as whether the viral load of the sample is relatively high or low and increasing or decreasing.

Veterans, as a population, are more susceptible to poor health outcomes and morbidity compared with similar civilian counterparts.^2,14-16 Veteran CLC patients likely would experience worse outcomes with COVID-19, including more infections, expiration, and morbidity compared with similar general population nursing homes. Similar to what had been reported for the civilian population, a trend (high CT/CN values early in the disease course with repeat testing needed to detect all positives followed by lower CT/CN value to correlate with increased viral load and then increased CT/CN value as the infection resolved) also was observed in this veteran population.

It has been extensively documented that minority groups experience decreased health care access and worse health outcomes. ^28-30 Considering the critical medical supply shortages, including personal protective equipment, ventilators, and even testing supplies, there is the potential for a resource access disparity by ethnicity.^28-31 Because the VA does not depend on measures of wealth and privilege such as health insurance, there was no disparity noted in access to testing by race or ethnicity at the VAMC CLC. When considering the health outcome of viral load from the measured CT/CN value, the viral loads of African American patients and those of other ethnicities was not significantly different in this study.

Conclusions

This is the first study to bring up critical points including serial CT/CN value correlation in RT-PCR tests, demographic distributions demonstrating easy and equal access in a veteran nursing home to COVID-19 testing, and clinical laboratory signs related to disease pathophysiology. Unlike other populations who have undergone serial CT/CN monitoring, nursing homes represent a particularly vulnerable population who require measures to prevent the spread and mitigate outbreaks of COVID-19.^2,4,5 Test measurements obtained such as the CT/CN value during routine clinical care can provide useful information for public health, epidemiologic, or clinical purposes with appropriate correlation to clinical and other laboratory parameters. This study demonstrates early intervention of serial testing of an outbreak in a veterans nursing home with CT/CN value correlation.

COVID-19, caused by SARS-CoV-2, is more severe in individuals with underlying illnesses. Because complete social distancing might be more difficult in nursing homes and community living centers (CLCs), public health leaders and clinicians have been concerned about the epidemiology and disease course in nursing homes even before the COVID-19 pandemic.1-7 A report of a COVID-19 outbreak in a nursing home facility in King County, Washington, documented a 33.7% overall fatality rate for residents and 52.4% among the most critically ill.^4,5 The experience at King County, Washington, shows that proactive steps to identify, monitor, and apply preventive control measures is important for future outbreaks.⁵

Reverse transcriptase polymerase chain reaction (RT-PCR) testing produces a cycle threshold (CT) or cycle number (CN) that correlates with viral load and infectiousness. ^8-14 CT/CN represents the number of RT-PCR cycles required for the fluorescent signal to cross the detection threshold (exceed background level) and is inversely proportional to the viral load. Effectively, the higher the viral load, the lower the CT/ CN value (Figure 1). Tracking CT/CN values was not documented in the Washington nursing home outbreak. Reports of COVID- 19 testing in CLCs during outbreaks are sparse, and CT/CN values and demographic distribution of these veterans has not been reported.¹⁵ The CLC veteran population, with known higher vulnerability to infection and chronic diseases, is epidemiologically different from the general nursing home population.^15-18 To address these literature gaps, we present the first report of COVID- 19 testing with CT/CN value correlations in the high-risk veteran CLC population.

Methods

A retrospective review of all COVID-19 CT/CN testing at the Corporal Michael J. Crescenz Veterans Affairs Medical Center (VAMC) CLC in Philadelphia, Pennsylvania, from March 28, 2020, to April 24, 2020, was performed with a US Department of Veterans Affairs (VA) Veterans Health Information System Architecture VistA/FileMan search. Only veteran residents were included in this review. Data collected included initial and serial test results, CT/CN on positive test results, test dates, testing platform used, demographic information (age, self-reported ethnicity, and sex), and clinical follow-up information. Health records were reviewed retrospectively to identify death, the first day after diagnosis with no documented symptoms, or hospitalization status.

RT-PCR testing was performed with the Abbott RealTime SARS-CoV-2 assay on the Abbott m2000 platform and the Xpert Xpress SARS-CoV-2 assay on the Cepheid Infinity platform. The Xpert Xpress assay gave 2 CT values for the E and N2 targets on positive samples.¹⁹ For this assay to indicate a positive specimen, amplification by RT-PCR of the N2 target or both the N2 and E target is required. The Xpert Xpress assay results as presumptive positive if only the E target amplified. This assay counts a maximum of 45 cycles. The Abbott RealTime SARS-CoV-2 assay gave 1 CN derived from the RNA-dependent RNA polymerase and N targets on positive samples.20 The Abbott assay on the m2000 counts a maximum of 37 cycles. The CT/CN value is the number of cycles required by RT-PCR for the fluorescence signal to cross a threshold value exceeding background level.^19,20

Samples that are negative for COVID-19 by RT-PCR do not produce a CT/CN value. Although both instruments were used for RT-PCR, the precise CT/CN values are not interchangeable and CT/CN observations over time between the 2 instruments during the disease course would be based on CT/CN value movement (general upward or downward trend) rather than absolute CT/CN differences. Both assays have been approved by emergency use authorization as qualitative tests for the presence/absence of COVID-19. Although the CT/CN value is available to laboratory staff after test completion, the CT/CN value is not reported routinely in the patient health record. All veteran patients identified on the initial review from March 28, 2020, to April 24, 2020, had all serial COVID-19 testing recorded until November 10, 2020. The CN values at the limit of detection (LOD) for the Abbott m2000 platform from the initial validation study were reviewed for reference.²¹

Results

Of 80 patients, 25 (31%) were COVID-19 positive over the course of testing. The study population had a mean age of 73.5 years; 92% were aged > 60 years. The group was predominantly male (79 male vs 1 female). Among the 77 patients with a stated ethnicity, 39 (51%) were African American. In comparison, 43% of residents in Philadelphia County are African American (Table).^22,23 Additionally, a previously published total COVID-19 tested population by ethnicity at the same regional VAMC revealed 46.8% of tested veteran patients were African American. ²⁴ Three patients had no stated ethnicity. Among those who tested positive, 11 were African American patients, 12 were White patients, and 2 had no stated ethnicity. Four patients tested positive on their first test. The other 21 patients were positive on repeat testing. Interestingly, 6 patients had 1 initial negative test before a positive test, 6 patients had 2, 8 patients had 3, and 1 patient had 4 initial negative tests before a positive test result. Among the 25 positive patients, 22 were either positive within 10 days of the initial negative test result or initially positive (Figure 2). Three patients who tested positive after 10 days did so at 16, 20, and 21 days after the initial negative test result. Among the 25 positive patients, 23 had initial and serial testing from both the Abbott and Xpert Xpress assays. The remaining 2 positive patients had initial and serial testing from the Abbott assay exclusively.

 

Only positive COVID-19 results by RTPCR produced a CT/CN value. After disease resolution with a negative test, no CT/CN value was produced with the negative test result on either testing platform. Because repeat testing after the initial positive result took place no sooner than 10 days, we observed that the CT/CN value increased after the initial positive result until the disease resolved, and a negative result was obtained (eAppendix 1, available online at doi:10.12788/fp.0276). A t test comparing the initial CT/CN value to the value more than 10 days after the initial positive showed the CT/CN was statistically significantly higher (P < .05).

Prompt repeat testing after the initial test can show a decrease in the CT/CN value because of increasing viral load before the expected increase until disease resolution if the initial test caught the infection early. Twelve patients had a negative test result between 2 serial positive results. These negative test results occurred later, near the end of the disease course. Among the 12 patients with this positive-negativepositive CT/CN pattern, 7 were symptomatic and no longer had documented symptoms or hospitalization around the time of this positive-negative-positive pattern. Four of these individuals were asymptomatic during the entire infection course. One of the 12 patients with this pattern expired with the negative result occurring on day 27 of the disease in the context of rising CT/CN. One of these 12 patients only had a presumptive positive test result on the Cepheid because it detected only the E target with a CT value of 38.7. In 1 of the 12 patients, the negative test result occurred between 2 positive test results with CT/CN values < 20 (12.05 and 19.05 for the positive tests before and after the negative result, respectively). When the initial CT/CN values was separated based on ethnicity, the average CT/CN value for African Americans (23.3) was higher than for other ethnicities (19.9), although it did not reach statistical significance (P = .35).

 

Ten of the 25 patients testing positive were admitted to the hospital, including 1 admitted 15 days before diagnosis (patient 20) and 1 admitted 80 days after diagnosis (patient 7). Among these 10 patients, 6 were admitted to the intensive care unit, including patient 7. None of the patients were intubated. Three of the 10 admitted patients died (patients 7, 20, and 24). Patient 7 was a 79-year-old male with a history of dementia, cerebrovascular accident, hypertension, hyperlipidemia, and chronic kidney disease with symptoms of lethargy and refusal of oral intake when he was diagnosed with COVID-19. He was admitted 80 days after diagnosis for hyponatremia and acute renal failure, with death on day 87 recorded as complications from the earlier COVID-19 infection. Patient 20, an 89-year-old male with a history of dementia, chronic kidney disease, and hyperlipidemia, had been admitted with fever, cough, and leukocytosis 17 days before COVID-19 diagnosis. He continued to be symptomatic after diagnosis with development of hypotension, dehydration, and refusal of oral intake while on comfort measures/endof- life care and died 15 days after COVID- 19 infection diagnosis. Patient 24 was a 96-year-old male with history of heart failure, hypertension, coronary artery disease, prostate carcinoma, and dementia who developed a cough at the time of diagnosis; because of his underlying condition, he remained in the CLC on comfort care. His symptoms, including hypoxia, worsened until he died 7 days after diagnosis.

Among the 25 patients, 17 were symptomatic at the time of diagnosis; the 14 initially symptomatic patients who survived improved clinically and returned to baseline. Eight of the 25 patients were asymptomatic initially and 3 developed symptoms 2 to 5 days after diagnosis. Only 1 patient who remained asymptomatic was admitted for inability to adhere to quarantine at the CLC. Review of the health records of all surviving symptomatic patients showed symptom resolution with return to baseline that corresponds to an increasing CT/CN value. A 1-tailed t test comparing the initial CT/ CN at the time of diagnosis to the last CT/CN value for symptomatic patients who recovered revealed a statistically significant increase (P < .05). For the symptomatic, symptom resolution and hospital discharge took (if required) a mean 20 days (range, 7-46). Among those who were not hospitalized, symptoms resolved in 7 to 36 days (18 days). Among those requiring hospitalization at any time (excluding patients who died or were asymptomatic), symptom and hospitalization resolution took a mean 22 days (range, 10-46). Asymptomatic patients (patients 8, 10, 15, 16, and 25) also showed increasing CT/CN value during the infection course, although there was no correlation with the continued lack of symptoms.

During the initial validation of the Abbott m2000 instrument, an LOD study included concentrations of 1000, 500, 250, 100, 70, 60, and 50 virus copies/mL (eAppendix 2, available online at doi:10.12788/fp.0276).21 The average CN at 100 virus copies/mL—the manufacturer provided LOD in the instructions for use—was 25.74.20 At a concentration of one-half that (50 virus copies/mL), the average CN was 28.39.

Discussion

This is the first study in the English literature to track CT/CN values as part of serial testing of a veteran CLC. Widescale testing and repeat screening in the absence of symptoms of nursing home residents would identify those who are infected and allow providers to track viral load clearance.^9-14 CT/CN values, when serially tracked during the infection course, appear to increase with illness resolution, consistent with earlier reports that CT/CN correlates with viral load.^8-14 Serial CT/CN values that are high (> 25) and continue to increase with each test suggest progression toward disease resolution or viral RNA clearance.^8-14 After symptom resolution, patients can have a persistent low level of viral shedding (corresponding to a high CT/CN value).^10-14,25 Near the end of disease resolution, a negative serial RT-PCR sample test before a subsequent positive might be a promising clinical sign of near disease recovery. Once the viral load is low with a CT/CN significantly higher than 25, some specimens might result as negative but turn up positive on subsequent sampling with a high CT/CN value. This pattern, with attendant high CT/CN values for the positive results, are consistent with the known effect of viral load (ie, a low viral load correlates to a high CT/CN) and adequacy of specimen collection on CT/CN values^.25 If the patient’s viral load is low, the sample collected might have a viral load at or near the testing platform’s LOD.

For Abbott m2000, the manufacturer provided LOD is 100 virus copies/mL, although the instrument was able to detect virus concentrations below that level during the initial validation.²⁰ The actual LOD of the instrument at our institution is < 100 virus copies/mL. For the Cepheid Xpert Xpress SARS-CoV-2 assay, the manufacturer-provided LOD is 250 virus copies/mL.19 An LOD study including samples below the manufacturer-provided LOD was not part of the initial validation study for the Xpert Xpress assay. Nonetheless, the virus concentration of samples with very high CT values at or near the maximum CT value of 45 is expected to be at or near the platform’s actual LOD.

If the samples collected near the end of the patient’s disease course have viral loads near these low concentrations, the encouraging positive-negative-positive pattern with high CT/CN values might be a promising sign for viral clearance. On the other hand, a positive-negative-positive pattern in the setting of low CT/CN values before and after the negative test might indicate poor sampling for the negative specimen. The back-and-forth or positive-negative-positive pattern generally appears to indicate near resolution of the infection course, although clinical correlation is necessary to rule out inadequate sampling earlier in the disease course or prolonged viral RNA shedding.^9-14 In all of the surviving symptomatic patients who showed the positive-negative-positive pattern, this sign occurred around or after symptom resolution. It also is important to consider that in some patients, SARS-CoV-2 RNA might remain detectable with increasing CT/CN after symptom resolution, and samples from these patients might not result positive. Therefore, CT/CN values cannot be interpreted without considering the clinical picture.²⁵

Studies on infectiousness and virus culture from COVID-19 samples with CT/ CN correlation have shown that patients with high CT/CN at the end of their disease course might not be as infectious.^9-14,25 Because 1 patient had a presumptive positive result after the negative result, this study shows that this positive-negative-positive pattern could include presumptive positive results. Also, in the setting of a recent positive result on the same testing platform, a patient with this pattern is presumed to be positive for COVID-19 RNA because of scant viral material.

Taiwan’s public health response to the outbreak illustrates the ability to mitigate an outbreak throughout a society.²⁶ These actions could help blunt an outbreak within a civilian nursing home population.⁵ Mitigation within a veteran CLC population has been documented, but the study, which focused on mitigation, did not consider CT/CN values, demographic distribution, testing access of the studied population, or laboratory findings related to disease pathophysiology.¹⁵ A key ingredient in widescale, serial testing is the availability of a rapid turnaround from testing in-house that allowed identification within 24 hours instead of several days at a reference laboratory. ¹⁵ Rapid widescale testing would allow clinical teams to optimize the Triangle of Benefit of Widescale Timely Tests for CLC (Figure 3).¹⁵ Timely laboratory testing remains pivotal for CLC veteran residents to aid successful clinical triage and management. Reporting serial CT/CN values can provide additional information to clinicians about the disease course because CT/ CN correlates with viral load, which varies based on where the patient is in the disease course.^9-14 CT/CN values carry significant prognostic value, particularly with respect to intubation and mortality.⁸

 

Limitations

Important limitations to our study include the use of 2 separate RT-PCR platforms. Using different RT-PCR platforms is common in clinical laboratories trying to take advantage of the unique characteristics of different platforms—for example, turnaround time vs high throughput— to manage COVID-19 testing workflow.²⁵ However, the exact CT/CN values obtained from each platform might not translate to the other, and the general trend (CT/CN values are rising or falling across serial tests) rather than a single value could be useful for clinical correlation. Even when the same platform is used for the serial testing, CT/CN values can be affected by adequacy of specimen collection; therefore, clinical correlation and considering the trend in CT/CN values is necessary for interpretation.^10-14,25 Because of the known trend in viral dynamics, a positive specimen collected with a high CT/CN followed by a subsequent (within 2 days) positive specimen collected with a low CT/CN might be compatible with early detection of COVID- 19 infection in the appropriate clinical context. ^10-14 However, detection late in the infection course or even after the symptomatic disease resolved with prolonged viral shedding might show serial positive samples with increasing CT/CN values.^10-14

Patients with prolonged viral shedding might not be infectious.27 Because of the clinical correlation required for interpretation and the other factors that might affect CT/CN values, recommendations advise against using CT/CN values in clinical practice at this time, although these recommendations could change with future research.25 Serial CT/CN values have the potential, if appropriately correlated with the clinical picture, to provide useful information, such as whether the viral load of the sample is relatively high or low and increasing or decreasing.

Veterans, as a population, are more susceptible to poor health outcomes and morbidity compared with similar civilian counterparts.^2,14-16 Veteran CLC patients likely would experience worse outcomes with COVID-19, including more infections, expiration, and morbidity compared with similar general population nursing homes. Similar to what had been reported for the civilian population, a trend (high CT/CN values early in the disease course with repeat testing needed to detect all positives followed by lower CT/CN value to correlate with increased viral load and then increased CT/CN value as the infection resolved) also was observed in this veteran population.

It has been extensively documented that minority groups experience decreased health care access and worse health outcomes. ^28-30 Considering the critical medical supply shortages, including personal protective equipment, ventilators, and even testing supplies, there is the potential for a resource access disparity by ethnicity.^28-31 Because the VA does not depend on measures of wealth and privilege such as health insurance, there was no disparity noted in access to testing by race or ethnicity at the VAMC CLC. When considering the health outcome of viral load from the measured CT/CN value, the viral loads of African American patients and those of other ethnicities was not significantly different in this study.

Conclusions

This is the first study to bring up critical points including serial CT/CN value correlation in RT-PCR tests, demographic distributions demonstrating easy and equal access in a veteran nursing home to COVID-19 testing, and clinical laboratory signs related to disease pathophysiology. Unlike other populations who have undergone serial CT/CN monitoring, nursing homes represent a particularly vulnerable population who require measures to prevent the spread and mitigate outbreaks of COVID-19.^2,4,5 Test measurements obtained such as the CT/CN value during routine clinical care can provide useful information for public health, epidemiologic, or clinical purposes with appropriate correlation to clinical and other laboratory parameters. This study demonstrates early intervention of serial testing of an outbreak in a veterans nursing home with CT/CN value correlation.

The US Department of Veterans Affairs (VA) operates the largest integrated health care system in the United States, providing physical and mental health care to more than 9 million veterans enrolled each year through a national system of inpatient, outpatient, and long-term care settings.¹ As 1 of 4 statutory missions, the VA conducts the largest training effort for health professionals in cooperation with affiliated academic institutions. From 2016 through 2020, an average of 123,000 trainees from various professions received training at the VA.² Physician residents comprised the largest trainee group (37%), followed by associated health students and residents (20%), and nursing professionals (21%).² In VA, associated health professions include all health care disciplines other than allopathic and osteopathic medicine, dentistry, and nursing. The associated health professions encompass about 40 specialties, including audiology, dietetics, physical and occupational therapy, optometry, pharmacy, podiatry, psychology, and social work. 

The VA also trains a smaller number of advanced fellows to address specialties important to the nation and veterans health that are not sufficiently addressed by standard accredited professional training.³ The VA Advanced Fellowship programs include 22 postresidency, postdoctoral, and postmasters fellowships to physicians and dentists, and associated health professions, including psychologists, social workers, and pharmacists. 3 From 2015 to 2019, 57 to 61% of medical school students reported having a VA clinical training experience during medical school^.4 Of current VA employees, 20% of registered nurses, 64% of physicians, 73% of podiatrists and optometrists, and 81% of psychologists reported VA training prior to employment.⁵

Health professions education is led by the designated education officer (DEO) at each VA facility.⁶ Also known as the associate chief of staff for education (ACOS/E), the DEO is a leadership position that is accountable to local VA facility executive leadership as well as the national Office of Academic Affiliations (OAA), which directs all VA health professions training across the US.⁶ At most VA facilities, the DEO oversees clinical training and education reporting directly to the facility chief of staff. At the same time, the ACOS/E is accountable to the OAA to ensure adherence with national education directives and policy. The DEO oversees trainee programs through collaboration with training program directors, faculty, academic affiliates, and accreditation agencies across > 40 health professions.

The DEO is expected to possess expertise in leadership attributes identified by the US Office of Personnel Management as essential to build a federal corporate culture that drives results, serves customers, and builds successful teams and coalitions within and outside the VA.7 These leadership attributes include leading change, leading people, driving results, business acumen, and building coalitions.⁷ They are operationalized by OAA as 4 domains of expertise required to lead education across multiple professions, including: (1) creating and sustaining an organizational work environment that supports learning, discovery, and continuous improvement; (2) aligning and managing fiscal, human, and capital resources to meet organizational learning needs; (3) driving learning and performance results to impact organizational success; and (4) leading change and transformation through positioning and implementing innovative learning and education strategies (Table 1).⁶

In this article we describe the VA DEO leadership role and the tasks required to lead education across multiple professions within the VA health care system. Given the broad scope of leading educational programs across multiple clinical professions and the interprofessional backgrounds of DEOs across the VA, we evaluated DEO self-perceived effectiveness to impact educational decisions and behavior by professional discipline. Our evaluation question is: Are different professional education and practice backgrounds functionally capable of providing leadership over all education of health professions training programs? Finally, we describe DEOs perceptions of facilitators and barriers to performing their DEO role within the VA.

Methods

We conducted a mixed methods analysis of data collected by OAA to assess DEO needs within a multiprofessional clinical learning environment. The needs assessment was conducted by an OAA evaluator (NH) with input on instrument development and data analysis from OAA leadership (KS, MB). This evaluation is categorized as an operations activity based on VA Handbook 1200 where information generated is used for business operations and quality improvement. ⁸ The overall project was subject to administrative rather than institutional review board oversight.

A needs assessment tool was developed based on the OAA domains of expertise.⁶ Prior to its administration, the tool was piloted with 8 DEOs in the field and the survey shortened based on their feedback. DEOs were asked about individual professional characteristics (eg, clinical profession, academic appointment, type of health professions training programs at the VA site) and their self-perceived effectiveness in impacting educational decisions and behaviors on general and profession-specific tasks within each of the 4 domains of expertise on a 5-point Likert scale (1, not effective; 5, very effective). 6,9 The needs assessment also included an open-ended question asking respondents to comment on any issues they felt important to understanding DEO role effectiveness.

The needs assessment was administered online via SurveyMonkey to 132 DEOs via email in September and October 2019. The DEOs represented 148 of 160 VA facilities with health professions education; 14 DEOs covered > 1 VA facility, and 12 positions were vacant. Email reminders were sent to nonresponders after 1 week. At 2 weeks, nonresponders received telephone reminders and personalized follow-up emails from OAA staff. The response rate at the end of 3 weeks was 96%.

Data Analysis

Mixed methods analyses included quantitative analyses to identify differences in general and profession-specific self-ratings of effectiveness in influencing educational decisions and behaviors by DEO profession, and qualitative analyses to further understand DEO’s perceptions of facilitators and barriers to DEO task effectiveness.^10,11 Quantitative analyses included descriptive statistics for all variables followed by nonparametric tests including χ2 and Mann- Whitney U tests to assess differences between physician and other professional DEOs in descriptive characteristics and selfperceived effectiveness on general and profession- specific tasks. Quantitative analyses were conducted using SPSS software, version 26. Qualitative analyses consisted of rapid assessment procedures to identify facilitators and barriers to DEO effectiveness by profession using Atlas.ti version 8, which involved reviewing responses to the open-ended question and assigning each response to predetermined categories based on the organizational level it applied to (eg, individual DEO, VA facility, or external to the organization).^12,13 Responses within categories were then summarized to identify the main themes.

Results 

Completed surveys were received from 127 respondents representing 139 VA facilities. Eighty percent were physicians and 20% were other professionals, including psychologists, pharmacists, dentists, dieticians, nurses, and nonclinicians. There were no statistically significant differences between physician and other professional DEOs in the percent working full time or length of time spent working in the position. About one-third of the sample had been in the position for < 2 years, one-third had been in the position for 2 to < 5 years, and one-third had been in the role for ≥ 5 years. Eighty percent reported having a faculty appointment with an academic affiliate. While 92% of physician DEOs had a faculty appointment, only 40% of other professional DEOs did (P < .001). Most faculty appointments for both groups were with a school of medicine. More physician DEOs than other professionals had training programs at their site for physicians (P = .003) and dentists (P < .001), but there were no statistically significant differences for having associated health, nursing, or advanced fellowship training programs at their sites. Across all DEOs, 98% reported training programs at their site for associated health professions, 95% for physician training, 93% for nursing training, 59% for dental training, and 48% for advanced fellowships.

Self-Perceived Effectiveness

There were no statistically significant differences between physician and other professional DEOs on self-perceived effectiveness in impacting educational decisions or behaviors for general tasks applicable across professions (Table 2). This result held even after controlling for length of time in the position and whether the DEO had an academic appointment. Generally, both groups reported being effective on tasks in the enabling learning domain, including applying policies and procedures related to trainees who rotate through the VA and maintaining adherence with accreditation agency standards across health professions. Mean score ranges for both physician and other professional DEOs reported moderate effectiveness in aligning resources effectiveness questions (2.45-3.72 vs 2.75-3.76), driving results questions (3.02-3.60 vs 3.39-3.48), and leading change questions (3.12-3.50 vs 3.42-3.80).

For profession-specific tasks, effectiveness ratings between the 2 groups were generally not statistically significant for medical, dental, and advanced fellowship training programs (Table 3). There was a pattern of statistically significant differences between physician and other professional DEOs for associated health and nursing training programs on tasks across the 4 domains of expertise with physicians having lower mean ratings compared with other professionals. Generally, physician DEOs had higher task effectiveness when compared with other professionals for medical training programs, and other professionals had higher task effectiveness ratings than did physicians for associated health or nursing training programs.

Facilitators and Barriers

Seventy responses related to facilitators and barriers to DEO effectiveness were received (59 from physicians and 11 from other professionals). Most responses were categorized as individual level facilitators or barriers (53% for physician and 64% for other professionals). Only 3% of comments were categorized as external to the organization (all made by physicians). The themes were similar for both groups and were aggregated in Table 4. Facilitators included continuing education, having a mentor who works at a similar type of facility, maintaining balance and time management when working with different training programs, learning to work and develop relationships with training program directors, developing an overall picture of each type of health professions training program, holding regular meetings with all health training programs and academic affiliates, having a formal education service line with budget and staffing, facility executive leadership who are knowledgeable of the education mission and DEO role, having a national oversight body, and the DEO’s relationships with academic affiliates.

Barriers to role effectiveness at the individual DEO level included assignment of multiple roles and a focus on regulation and monitoring with little time for development of new programs and strategic planning. The organizational level barriers included difficulty getting core services to engage with health professions trainees and siloed education leadership. 

Discussion

DEOs oversee multiple health professions training programs within local facilities. The DEO is accountable to local VA facility leadership and a national education office to lead local health professions education at local facilities and integrate these educational activities across the national VA system.

The VA DEO role is similar to the Accreditation Council for Graduate Medical Education designated institutional official (DIO) except that the VA DEO provides oversight of > 40 health professions training programs^.14,15 The VA DEO, therefore, has broader oversight than the DIO role that focuses only on graduate physician education. Similar to the DIO, the VA DEO role initially emphasized the enabling learning and aligning resources domains to provide oversight and administration of health professions training programs. Over time, both roles have expanded to include defining and ensuring healthy clinical learning environments, aligning educational resources and training with the institutional mission, workforce, and societal needs, and creating continuous educational improvement models.^6,16,17 To accomplish these expanded goals, both the DEO and the DIO work closely with other educational leaders at the academic affiliate and the VA facility. As health professions education advances, there will be increased emphasis placed on delivering educational programs to improve clinical practice and health care outcomes.¹⁸

Our findings that DEO profession did not influence self-ratings of effectiveness to influence educational decisions or behaviors on general tasks applicable across health professions suggest that education and practice background are not factors influencing selfratings. Nor were self-ratings influenced by other factors. Since the DEO is a senior leadership position, candidates for the position already may possess managerial and leadership skills. In our sample, several individuals commented that they had prior education leadership positions, eg, training program director or had years of experience working in the VA. Similarly, having an academic appointment may not be important for the performance of general administrative tasks. However, an academic appointment may be important for effective performance of educational tasks, such as clinical teaching, didactic training, and curriculum development, which were not measured in this study.

The finding of differences in self-ratings between physicians and other professionals on profession-specific tasks for associated health and nursing suggests that physicians may require additional curriculum to enhance their knowledge in managing other professional educational programs. For nursing specifically, this finding could also reflect substantial input from the lead nurse executive in the facility. DEOs also identified practical ways to facilitate their work with multiple health professions that could immediately be put into practice, including developing relationships and enhancing communication with training program directors, faculty, and academic affiliates of each profession.

Taken together, the quantitative and qualitative findings indicate that despite differences in professional backgrounds, DEOs have high self-ratings of their own effectiveness to influence educational decisions and behaviors on general tasks they are expected to accomplish. There are some professionspecific tasks where professional background does influence self-perceived effectiveness, ie, physicians have higher self-ratings on physician-specific tasks and other professionals have higher self-ratings on associated health or nursing tasks. These perceived differences may be mitigated by increasing facilitators and decreasing barriers identified for the individual DEO, within the organization, and external to the organization.

Limitations Our findings should be interpreted with the following limitations in mind. The selfreport nature of the data opens the possibility of self-report bias or Dunning-Kruger effects where effectiveness ratings could have been overestimated by respondents.21 Although respondents were assured of their anonymity and that results would only be reported in the aggregate, there is potential for providing more positive responses on a needs assessment administered by the national education program office. We recommend further work be conducted to validate the needs assessment tool against other data collection methods, such as actual outcomes of educational effectiveness. Our study did not incorporate measures of educational effectiveness to determine whether self-perceived DEO effectiveness is translated to better trainee or learning outcomes. Before this can happen, educational policymakers must identify the most important facility-level learning outcomes. Since the DEO is a facility level educational administrator, learning efeffectiveness must be defined at the facility level. The qualitative findings could also be expanded through the application of more detailed qualitative methods, such as indepth interviews. The tasks rated by DEOs were based on OAA’s current definition of the DEO role.6 As the field advances, DEO tasks will also evolve.^22-24

Conclusions

The DEO is a senior educational leadership role that oversees all health professions training in the VA. Our findings are supportive of individuals from various health disciplines serving in the VA DEO role with responsibilities that span multiple health profession training programs. We recommend further work to validate the instrument used in this study, as well as the application of qualitative methods like indepth interviews to further our understanding of the DEO role.

The US Department of Veterans Affairs (VA) operates the largest integrated health care system in the United States, providing physical and mental health care to more than 9 million veterans enrolled each year through a national system of inpatient, outpatient, and long-term care settings.¹ As 1 of 4 statutory missions, the VA conducts the largest training effort for health professionals in cooperation with affiliated academic institutions. From 2016 through 2020, an average of 123,000 trainees from various professions received training at the VA.² Physician residents comprised the largest trainee group (37%), followed by associated health students and residents (20%), and nursing professionals (21%).² In VA, associated health professions include all health care disciplines other than allopathic and osteopathic medicine, dentistry, and nursing. The associated health professions encompass about 40 specialties, including audiology, dietetics, physical and occupational therapy, optometry, pharmacy, podiatry, psychology, and social work. 

The VA also trains a smaller number of advanced fellows to address specialties important to the nation and veterans health that are not sufficiently addressed by standard accredited professional training.³ The VA Advanced Fellowship programs include 22 postresidency, postdoctoral, and postmasters fellowships to physicians and dentists, and associated health professions, including psychologists, social workers, and pharmacists. 3 From 2015 to 2019, 57 to 61% of medical school students reported having a VA clinical training experience during medical school^.4 Of current VA employees, 20% of registered nurses, 64% of physicians, 73% of podiatrists and optometrists, and 81% of psychologists reported VA training prior to employment.⁵

Health professions education is led by the designated education officer (DEO) at each VA facility.⁶ Also known as the associate chief of staff for education (ACOS/E), the DEO is a leadership position that is accountable to local VA facility executive leadership as well as the national Office of Academic Affiliations (OAA), which directs all VA health professions training across the US.⁶ At most VA facilities, the DEO oversees clinical training and education reporting directly to the facility chief of staff. At the same time, the ACOS/E is accountable to the OAA to ensure adherence with national education directives and policy. The DEO oversees trainee programs through collaboration with training program directors, faculty, academic affiliates, and accreditation agencies across > 40 health professions.

The DEO is expected to possess expertise in leadership attributes identified by the US Office of Personnel Management as essential to build a federal corporate culture that drives results, serves customers, and builds successful teams and coalitions within and outside the VA.7 These leadership attributes include leading change, leading people, driving results, business acumen, and building coalitions.⁷ They are operationalized by OAA as 4 domains of expertise required to lead education across multiple professions, including: (1) creating and sustaining an organizational work environment that supports learning, discovery, and continuous improvement; (2) aligning and managing fiscal, human, and capital resources to meet organizational learning needs; (3) driving learning and performance results to impact organizational success; and (4) leading change and transformation through positioning and implementing innovative learning and education strategies (Table 1).⁶

In this article we describe the VA DEO leadership role and the tasks required to lead education across multiple professions within the VA health care system. Given the broad scope of leading educational programs across multiple clinical professions and the interprofessional backgrounds of DEOs across the VA, we evaluated DEO self-perceived effectiveness to impact educational decisions and behavior by professional discipline. Our evaluation question is: Are different professional education and practice backgrounds functionally capable of providing leadership over all education of health professions training programs? Finally, we describe DEOs perceptions of facilitators and barriers to performing their DEO role within the VA.

Methods

We conducted a mixed methods analysis of data collected by OAA to assess DEO needs within a multiprofessional clinical learning environment. The needs assessment was conducted by an OAA evaluator (NH) with input on instrument development and data analysis from OAA leadership (KS, MB). This evaluation is categorized as an operations activity based on VA Handbook 1200 where information generated is used for business operations and quality improvement. ⁸ The overall project was subject to administrative rather than institutional review board oversight.

A needs assessment tool was developed based on the OAA domains of expertise.⁶ Prior to its administration, the tool was piloted with 8 DEOs in the field and the survey shortened based on their feedback. DEOs were asked about individual professional characteristics (eg, clinical profession, academic appointment, type of health professions training programs at the VA site) and their self-perceived effectiveness in impacting educational decisions and behaviors on general and profession-specific tasks within each of the 4 domains of expertise on a 5-point Likert scale (1, not effective; 5, very effective). 6,9 The needs assessment also included an open-ended question asking respondents to comment on any issues they felt important to understanding DEO role effectiveness.

The needs assessment was administered online via SurveyMonkey to 132 DEOs via email in September and October 2019. The DEOs represented 148 of 160 VA facilities with health professions education; 14 DEOs covered > 1 VA facility, and 12 positions were vacant. Email reminders were sent to nonresponders after 1 week. At 2 weeks, nonresponders received telephone reminders and personalized follow-up emails from OAA staff. The response rate at the end of 3 weeks was 96%.

Data Analysis

Mixed methods analyses included quantitative analyses to identify differences in general and profession-specific self-ratings of effectiveness in influencing educational decisions and behaviors by DEO profession, and qualitative analyses to further understand DEO’s perceptions of facilitators and barriers to DEO task effectiveness.^10,11 Quantitative analyses included descriptive statistics for all variables followed by nonparametric tests including χ2 and Mann- Whitney U tests to assess differences between physician and other professional DEOs in descriptive characteristics and selfperceived effectiveness on general and profession- specific tasks. Quantitative analyses were conducted using SPSS software, version 26. Qualitative analyses consisted of rapid assessment procedures to identify facilitators and barriers to DEO effectiveness by profession using Atlas.ti version 8, which involved reviewing responses to the open-ended question and assigning each response to predetermined categories based on the organizational level it applied to (eg, individual DEO, VA facility, or external to the organization).^12,13 Responses within categories were then summarized to identify the main themes.

Results 

Completed surveys were received from 127 respondents representing 139 VA facilities. Eighty percent were physicians and 20% were other professionals, including psychologists, pharmacists, dentists, dieticians, nurses, and nonclinicians. There were no statistically significant differences between physician and other professional DEOs in the percent working full time or length of time spent working in the position. About one-third of the sample had been in the position for < 2 years, one-third had been in the position for 2 to < 5 years, and one-third had been in the role for ≥ 5 years. Eighty percent reported having a faculty appointment with an academic affiliate. While 92% of physician DEOs had a faculty appointment, only 40% of other professional DEOs did (P < .001). Most faculty appointments for both groups were with a school of medicine. More physician DEOs than other professionals had training programs at their site for physicians (P = .003) and dentists (P < .001), but there were no statistically significant differences for having associated health, nursing, or advanced fellowship training programs at their sites. Across all DEOs, 98% reported training programs at their site for associated health professions, 95% for physician training, 93% for nursing training, 59% for dental training, and 48% for advanced fellowships.

Self-Perceived Effectiveness

There were no statistically significant differences between physician and other professional DEOs on self-perceived effectiveness in impacting educational decisions or behaviors for general tasks applicable across professions (Table 2). This result held even after controlling for length of time in the position and whether the DEO had an academic appointment. Generally, both groups reported being effective on tasks in the enabling learning domain, including applying policies and procedures related to trainees who rotate through the VA and maintaining adherence with accreditation agency standards across health professions. Mean score ranges for both physician and other professional DEOs reported moderate effectiveness in aligning resources effectiveness questions (2.45-3.72 vs 2.75-3.76), driving results questions (3.02-3.60 vs 3.39-3.48), and leading change questions (3.12-3.50 vs 3.42-3.80).

For profession-specific tasks, effectiveness ratings between the 2 groups were generally not statistically significant for medical, dental, and advanced fellowship training programs (Table 3). There was a pattern of statistically significant differences between physician and other professional DEOs for associated health and nursing training programs on tasks across the 4 domains of expertise with physicians having lower mean ratings compared with other professionals. Generally, physician DEOs had higher task effectiveness when compared with other professionals for medical training programs, and other professionals had higher task effectiveness ratings than did physicians for associated health or nursing training programs.

Facilitators and Barriers

Seventy responses related to facilitators and barriers to DEO effectiveness were received (59 from physicians and 11 from other professionals). Most responses were categorized as individual level facilitators or barriers (53% for physician and 64% for other professionals). Only 3% of comments were categorized as external to the organization (all made by physicians). The themes were similar for both groups and were aggregated in Table 4. Facilitators included continuing education, having a mentor who works at a similar type of facility, maintaining balance and time management when working with different training programs, learning to work and develop relationships with training program directors, developing an overall picture of each type of health professions training program, holding regular meetings with all health training programs and academic affiliates, having a formal education service line with budget and staffing, facility executive leadership who are knowledgeable of the education mission and DEO role, having a national oversight body, and the DEO’s relationships with academic affiliates.

Barriers to role effectiveness at the individual DEO level included assignment of multiple roles and a focus on regulation and monitoring with little time for development of new programs and strategic planning. The organizational level barriers included difficulty getting core services to engage with health professions trainees and siloed education leadership. 

Discussion

DEOs oversee multiple health professions training programs within local facilities. The DEO is accountable to local VA facility leadership and a national education office to lead local health professions education at local facilities and integrate these educational activities across the national VA system.

The VA DEO role is similar to the Accreditation Council for Graduate Medical Education designated institutional official (DIO) except that the VA DEO provides oversight of > 40 health professions training programs^.14,15 The VA DEO, therefore, has broader oversight than the DIO role that focuses only on graduate physician education. Similar to the DIO, the VA DEO role initially emphasized the enabling learning and aligning resources domains to provide oversight and administration of health professions training programs. Over time, both roles have expanded to include defining and ensuring healthy clinical learning environments, aligning educational resources and training with the institutional mission, workforce, and societal needs, and creating continuous educational improvement models.^6,16,17 To accomplish these expanded goals, both the DEO and the DIO work closely with other educational leaders at the academic affiliate and the VA facility. As health professions education advances, there will be increased emphasis placed on delivering educational programs to improve clinical practice and health care outcomes.¹⁸

Our findings that DEO profession did not influence self-ratings of effectiveness to influence educational decisions or behaviors on general tasks applicable across health professions suggest that education and practice background are not factors influencing selfratings. Nor were self-ratings influenced by other factors. Since the DEO is a senior leadership position, candidates for the position already may possess managerial and leadership skills. In our sample, several individuals commented that they had prior education leadership positions, eg, training program director or had years of experience working in the VA. Similarly, having an academic appointment may not be important for the performance of general administrative tasks. However, an academic appointment may be important for effective performance of educational tasks, such as clinical teaching, didactic training, and curriculum development, which were not measured in this study.

The finding of differences in self-ratings between physicians and other professionals on profession-specific tasks for associated health and nursing suggests that physicians may require additional curriculum to enhance their knowledge in managing other professional educational programs. For nursing specifically, this finding could also reflect substantial input from the lead nurse executive in the facility. DEOs also identified practical ways to facilitate their work with multiple health professions that could immediately be put into practice, including developing relationships and enhancing communication with training program directors, faculty, and academic affiliates of each profession.

Taken together, the quantitative and qualitative findings indicate that despite differences in professional backgrounds, DEOs have high self-ratings of their own effectiveness to influence educational decisions and behaviors on general tasks they are expected to accomplish. There are some professionspecific tasks where professional background does influence self-perceived effectiveness, ie, physicians have higher self-ratings on physician-specific tasks and other professionals have higher self-ratings on associated health or nursing tasks. These perceived differences may be mitigated by increasing facilitators and decreasing barriers identified for the individual DEO, within the organization, and external to the organization.

Limitations Our findings should be interpreted with the following limitations in mind. The selfreport nature of the data opens the possibility of self-report bias or Dunning-Kruger effects where effectiveness ratings could have been overestimated by respondents.21 Although respondents were assured of their anonymity and that results would only be reported in the aggregate, there is potential for providing more positive responses on a needs assessment administered by the national education program office. We recommend further work be conducted to validate the needs assessment tool against other data collection methods, such as actual outcomes of educational effectiveness. Our study did not incorporate measures of educational effectiveness to determine whether self-perceived DEO effectiveness is translated to better trainee or learning outcomes. Before this can happen, educational policymakers must identify the most important facility-level learning outcomes. Since the DEO is a facility level educational administrator, learning efeffectiveness must be defined at the facility level. The qualitative findings could also be expanded through the application of more detailed qualitative methods, such as indepth interviews. The tasks rated by DEOs were based on OAA’s current definition of the DEO role.6 As the field advances, DEO tasks will also evolve.^22-24

Conclusions

The DEO is a senior educational leadership role that oversees all health professions training in the VA. Our findings are supportive of individuals from various health disciplines serving in the VA DEO role with responsibilities that span multiple health profession training programs. We recommend further work to validate the instrument used in this study, as well as the application of qualitative methods like indepth interviews to further our understanding of the DEO role.

The US Department of Veterans Affairs (VA) operates the largest integrated health care system in the United States, providing physical and mental health care to more than 9 million veterans enrolled each year through a national system of inpatient, outpatient, and long-term care settings.¹ As 1 of 4 statutory missions, the VA conducts the largest training effort for health professionals in cooperation with affiliated academic institutions. From 2016 through 2020, an average of 123,000 trainees from various professions received training at the VA.² Physician residents comprised the largest trainee group (37%), followed by associated health students and residents (20%), and nursing professionals (21%).² In VA, associated health professions include all health care disciplines other than allopathic and osteopathic medicine, dentistry, and nursing. The associated health professions encompass about 40 specialties, including audiology, dietetics, physical and occupational therapy, optometry, pharmacy, podiatry, psychology, and social work. 

The VA also trains a smaller number of advanced fellows to address specialties important to the nation and veterans health that are not sufficiently addressed by standard accredited professional training.³ The VA Advanced Fellowship programs include 22 postresidency, postdoctoral, and postmasters fellowships to physicians and dentists, and associated health professions, including psychologists, social workers, and pharmacists. 3 From 2015 to 2019, 57 to 61% of medical school students reported having a VA clinical training experience during medical school^.4 Of current VA employees, 20% of registered nurses, 64% of physicians, 73% of podiatrists and optometrists, and 81% of psychologists reported VA training prior to employment.⁵

Health professions education is led by the designated education officer (DEO) at each VA facility.⁶ Also known as the associate chief of staff for education (ACOS/E), the DEO is a leadership position that is accountable to local VA facility executive leadership as well as the national Office of Academic Affiliations (OAA), which directs all VA health professions training across the US.⁶ At most VA facilities, the DEO oversees clinical training and education reporting directly to the facility chief of staff. At the same time, the ACOS/E is accountable to the OAA to ensure adherence with national education directives and policy. The DEO oversees trainee programs through collaboration with training program directors, faculty, academic affiliates, and accreditation agencies across > 40 health professions.

The DEO is expected to possess expertise in leadership attributes identified by the US Office of Personnel Management as essential to build a federal corporate culture that drives results, serves customers, and builds successful teams and coalitions within and outside the VA.7 These leadership attributes include leading change, leading people, driving results, business acumen, and building coalitions.⁷ They are operationalized by OAA as 4 domains of expertise required to lead education across multiple professions, including: (1) creating and sustaining an organizational work environment that supports learning, discovery, and continuous improvement; (2) aligning and managing fiscal, human, and capital resources to meet organizational learning needs; (3) driving learning and performance results to impact organizational success; and (4) leading change and transformation through positioning and implementing innovative learning and education strategies (Table 1).⁶

In this article we describe the VA DEO leadership role and the tasks required to lead education across multiple professions within the VA health care system. Given the broad scope of leading educational programs across multiple clinical professions and the interprofessional backgrounds of DEOs across the VA, we evaluated DEO self-perceived effectiveness to impact educational decisions and behavior by professional discipline. Our evaluation question is: Are different professional education and practice backgrounds functionally capable of providing leadership over all education of health professions training programs? Finally, we describe DEOs perceptions of facilitators and barriers to performing their DEO role within the VA.

Methods

We conducted a mixed methods analysis of data collected by OAA to assess DEO needs within a multiprofessional clinical learning environment. The needs assessment was conducted by an OAA evaluator (NH) with input on instrument development and data analysis from OAA leadership (KS, MB). This evaluation is categorized as an operations activity based on VA Handbook 1200 where information generated is used for business operations and quality improvement. ⁸ The overall project was subject to administrative rather than institutional review board oversight.

A needs assessment tool was developed based on the OAA domains of expertise.⁶ Prior to its administration, the tool was piloted with 8 DEOs in the field and the survey shortened based on their feedback. DEOs were asked about individual professional characteristics (eg, clinical profession, academic appointment, type of health professions training programs at the VA site) and their self-perceived effectiveness in impacting educational decisions and behaviors on general and profession-specific tasks within each of the 4 domains of expertise on a 5-point Likert scale (1, not effective; 5, very effective). 6,9 The needs assessment also included an open-ended question asking respondents to comment on any issues they felt important to understanding DEO role effectiveness.

The needs assessment was administered online via SurveyMonkey to 132 DEOs via email in September and October 2019. The DEOs represented 148 of 160 VA facilities with health professions education; 14 DEOs covered > 1 VA facility, and 12 positions were vacant. Email reminders were sent to nonresponders after 1 week. At 2 weeks, nonresponders received telephone reminders and personalized follow-up emails from OAA staff. The response rate at the end of 3 weeks was 96%.

Data Analysis

Mixed methods analyses included quantitative analyses to identify differences in general and profession-specific self-ratings of effectiveness in influencing educational decisions and behaviors by DEO profession, and qualitative analyses to further understand DEO’s perceptions of facilitators and barriers to DEO task effectiveness.^10,11 Quantitative analyses included descriptive statistics for all variables followed by nonparametric tests including χ2 and Mann- Whitney U tests to assess differences between physician and other professional DEOs in descriptive characteristics and selfperceived effectiveness on general and profession- specific tasks. Quantitative analyses were conducted using SPSS software, version 26. Qualitative analyses consisted of rapid assessment procedures to identify facilitators and barriers to DEO effectiveness by profession using Atlas.ti version 8, which involved reviewing responses to the open-ended question and assigning each response to predetermined categories based on the organizational level it applied to (eg, individual DEO, VA facility, or external to the organization).^12,13 Responses within categories were then summarized to identify the main themes.

Results 

Completed surveys were received from 127 respondents representing 139 VA facilities. Eighty percent were physicians and 20% were other professionals, including psychologists, pharmacists, dentists, dieticians, nurses, and nonclinicians. There were no statistically significant differences between physician and other professional DEOs in the percent working full time or length of time spent working in the position. About one-third of the sample had been in the position for < 2 years, one-third had been in the position for 2 to < 5 years, and one-third had been in the role for ≥ 5 years. Eighty percent reported having a faculty appointment with an academic affiliate. While 92% of physician DEOs had a faculty appointment, only 40% of other professional DEOs did (P < .001). Most faculty appointments for both groups were with a school of medicine. More physician DEOs than other professionals had training programs at their site for physicians (P = .003) and dentists (P < .001), but there were no statistically significant differences for having associated health, nursing, or advanced fellowship training programs at their sites. Across all DEOs, 98% reported training programs at their site for associated health professions, 95% for physician training, 93% for nursing training, 59% for dental training, and 48% for advanced fellowships.

Self-Perceived Effectiveness

There were no statistically significant differences between physician and other professional DEOs on self-perceived effectiveness in impacting educational decisions or behaviors for general tasks applicable across professions (Table 2). This result held even after controlling for length of time in the position and whether the DEO had an academic appointment. Generally, both groups reported being effective on tasks in the enabling learning domain, including applying policies and procedures related to trainees who rotate through the VA and maintaining adherence with accreditation agency standards across health professions. Mean score ranges for both physician and other professional DEOs reported moderate effectiveness in aligning resources effectiveness questions (2.45-3.72 vs 2.75-3.76), driving results questions (3.02-3.60 vs 3.39-3.48), and leading change questions (3.12-3.50 vs 3.42-3.80).

For profession-specific tasks, effectiveness ratings between the 2 groups were generally not statistically significant for medical, dental, and advanced fellowship training programs (Table 3). There was a pattern of statistically significant differences between physician and other professional DEOs for associated health and nursing training programs on tasks across the 4 domains of expertise with physicians having lower mean ratings compared with other professionals. Generally, physician DEOs had higher task effectiveness when compared with other professionals for medical training programs, and other professionals had higher task effectiveness ratings than did physicians for associated health or nursing training programs.

Facilitators and Barriers

Seventy responses related to facilitators and barriers to DEO effectiveness were received (59 from physicians and 11 from other professionals). Most responses were categorized as individual level facilitators or barriers (53% for physician and 64% for other professionals). Only 3% of comments were categorized as external to the organization (all made by physicians). The themes were similar for both groups and were aggregated in Table 4. Facilitators included continuing education, having a mentor who works at a similar type of facility, maintaining balance and time management when working with different training programs, learning to work and develop relationships with training program directors, developing an overall picture of each type of health professions training program, holding regular meetings with all health training programs and academic affiliates, having a formal education service line with budget and staffing, facility executive leadership who are knowledgeable of the education mission and DEO role, having a national oversight body, and the DEO’s relationships with academic affiliates.

Barriers to role effectiveness at the individual DEO level included assignment of multiple roles and a focus on regulation and monitoring with little time for development of new programs and strategic planning. The organizational level barriers included difficulty getting core services to engage with health professions trainees and siloed education leadership. 

Discussion

DEOs oversee multiple health professions training programs within local facilities. The DEO is accountable to local VA facility leadership and a national education office to lead local health professions education at local facilities and integrate these educational activities across the national VA system.

The VA DEO role is similar to the Accreditation Council for Graduate Medical Education designated institutional official (DIO) except that the VA DEO provides oversight of > 40 health professions training programs^.14,15 The VA DEO, therefore, has broader oversight than the DIO role that focuses only on graduate physician education. Similar to the DIO, the VA DEO role initially emphasized the enabling learning and aligning resources domains to provide oversight and administration of health professions training programs. Over time, both roles have expanded to include defining and ensuring healthy clinical learning environments, aligning educational resources and training with the institutional mission, workforce, and societal needs, and creating continuous educational improvement models.^6,16,17 To accomplish these expanded goals, both the DEO and the DIO work closely with other educational leaders at the academic affiliate and the VA facility. As health professions education advances, there will be increased emphasis placed on delivering educational programs to improve clinical practice and health care outcomes.¹⁸

Our findings that DEO profession did not influence self-ratings of effectiveness to influence educational decisions or behaviors on general tasks applicable across health professions suggest that education and practice background are not factors influencing selfratings. Nor were self-ratings influenced by other factors. Since the DEO is a senior leadership position, candidates for the position already may possess managerial and leadership skills. In our sample, several individuals commented that they had prior education leadership positions, eg, training program director or had years of experience working in the VA. Similarly, having an academic appointment may not be important for the performance of general administrative tasks. However, an academic appointment may be important for effective performance of educational tasks, such as clinical teaching, didactic training, and curriculum development, which were not measured in this study.

The finding of differences in self-ratings between physicians and other professionals on profession-specific tasks for associated health and nursing suggests that physicians may require additional curriculum to enhance their knowledge in managing other professional educational programs. For nursing specifically, this finding could also reflect substantial input from the lead nurse executive in the facility. DEOs also identified practical ways to facilitate their work with multiple health professions that could immediately be put into practice, including developing relationships and enhancing communication with training program directors, faculty, and academic affiliates of each profession.

Taken together, the quantitative and qualitative findings indicate that despite differences in professional backgrounds, DEOs have high self-ratings of their own effectiveness to influence educational decisions and behaviors on general tasks they are expected to accomplish. There are some professionspecific tasks where professional background does influence self-perceived effectiveness, ie, physicians have higher self-ratings on physician-specific tasks and other professionals have higher self-ratings on associated health or nursing tasks. These perceived differences may be mitigated by increasing facilitators and decreasing barriers identified for the individual DEO, within the organization, and external to the organization.

Limitations Our findings should be interpreted with the following limitations in mind. The selfreport nature of the data opens the possibility of self-report bias or Dunning-Kruger effects where effectiveness ratings could have been overestimated by respondents.21 Although respondents were assured of their anonymity and that results would only be reported in the aggregate, there is potential for providing more positive responses on a needs assessment administered by the national education program office. We recommend further work be conducted to validate the needs assessment tool against other data collection methods, such as actual outcomes of educational effectiveness. Our study did not incorporate measures of educational effectiveness to determine whether self-perceived DEO effectiveness is translated to better trainee or learning outcomes. Before this can happen, educational policymakers must identify the most important facility-level learning outcomes. Since the DEO is a facility level educational administrator, learning efeffectiveness must be defined at the facility level. The qualitative findings could also be expanded through the application of more detailed qualitative methods, such as indepth interviews. The tasks rated by DEOs were based on OAA’s current definition of the DEO role.6 As the field advances, DEO tasks will also evolve.^22-24

Conclusions

The DEO is a senior educational leadership role that oversees all health professions training in the VA. Our findings are supportive of individuals from various health disciplines serving in the VA DEO role with responsibilities that span multiple health profession training programs. We recommend further work to validate the instrument used in this study, as well as the application of qualitative methods like indepth interviews to further our understanding of the DEO role.

This article is part of a series of articles on the Home Improvements and Structural Alterations program (HISA), a home modification (HM) program within the Veterans Health Administration (VHA). HISA is a benefit awarded to veterans with disabilities (VWDs) and is instrumental in affording physical accessibility and structural alterations to veterans’ homes.¹ The overarching goals of this project are to describe and understand HISA use by VWDs. Previous work has shown geographical variation in the number of HISA prescriptions across counties in the US (Figure 1).¹ The current work seeks to describe and predict the county-level rates of HISA use. Information about what predicts HISA utilization at the county level is important because it enhances understanding of program utilization at a national level. The long-term goal of the series is to provide knowledge about HM services within VHA to improve community-based independent living of VWDs by increasing awareness and utilization of HM services. 

Background

A health care professional (HCP) approves a HM support award by evaluating the practicality of the support to improve the built environment of a given veteran’s disability.^1,2 Previously we detailed some of the preliminary research into the HISA program, including HISA user demographic and clinical characteristics, types of HMs received, user suggestions for improvement, and geospatial analysis of HISA prescriptions concentration.^1-4

The geospatial analyses of HISA prescriptions revealed clusters of high numbers of HISA users (hot spots) and low numbers of HISA users (cold spots), indicating that HISA is either not prescribed or uniformly used across the US. The previous research prompted investigation into county-level variables that may impact HISA utilization rates. This inquiry focuses on county characteristics associated with HISA use rates, such as measures of clinical care and quality of care (eg, access to health services variables, lack of insurance, preventable hospital stays), physical environment, and sociodemographic characteristics. Clinical care and quality of care measures promote the interaction with HCPs. Moreover, access to health care is an important indicator of health outcomes.^5,6 An individual’s capacity to access health services, such as a HM program, greatly impacts well-being, safety, independence, and health.^2,4 Well-being, safety, independence, and health become compromised if individuals cannot access care, if needed care is lacking in their area, if HCPs are not available, or are unwilling to provide care due to lack of insurance coverage.^7-12 In locations where health care services are minimal due to lack of specialists or health care facilities, the quality of (or access to) care may be compromised, resulting in preventable conditions becoming problematic.^13,14 These conditions may result in unnecessary hospitalizations for conditions that could have been treated during routine care. Financial barriers to care particularly among low-income people and the uninsured have proven detrimental to health.^15,16 On the other hand, preventable hospital stays are a quality of care measure (ie, a proxy for poor quality of care). HISA operates within a health care system; thus, it is imperative to include these measures impacting health. 

In this study, we sought to identify county-level predictors—in particular, county-level proxies for access to care—that may be associated with county-level HISA use. We define HISA utilization rate as the percentage of a county’s VHA patients who have received a HISA award.

Methods

This study used data from the National Prosthetics Patient Database (NPPD), US Department of Veterans Affairs (VA) medical database inpatient and outpatient datasets, VHA Support Service Center (VSSC) data cubes, and the County Health Rankings database (CHRD). First, the study cohort was identified from NPPD users who have obtained a HISA award from fiscal years (FY) 2015 to 2018. Analysis started with FY 2015 following new regulations (38 CFR § 17) governing the operations of the HISA program.² The study cohort was matched with records from NPPD and VA inpatient and outpatient datasets to obtain information about the veterans’ demographic characteristics and their HM characteristics and costs. The number of VHA end-of-year (EOY) patients per county was extracted from the VSSC Current Enrollment Cube, which was used in calculation of the county-level HISA utilization rate.¹⁷ Finally, zip code–based locational data were used to calculate approximate drive time and distance from the HISA user’s approximate location to the facility where they received their HM prescription. Drive times and drive distances were calculated with Esri ArcGIS Pro (v2.6.3) by placing zip code centroid and VHA facilities on a nationwide road network that contains both road speeds and distances.

Calculations

Patient-level data were aggregated up to county-level variables by calculating the sum, mean, or percent per county. HISA user sample characteristics, including sex, race, rurality (urban, rural), marital status, and Class 1 vs Class 2 disability-related eligibility groups, were aggregated to the county level by calculating percentages of HISA users of the given characteristics out of total HISA users in the county. Disability-related eligibility groups (Class 1 vs Class 2 HISA users) determines the maximum lifetime award dollar amount. Specifically, those with service-connected disabilities or those with a ≥ 50% disability rating (regardless of whether or not their disability is service connected) are classified as Class 1 HISA users and are eligible to receive a maximum lifetime award of $6800. Those with a recorded disability that is not connected to their military service, and who have a disability rating of < 50% are classified as Class 2 HISA users and are eligible to receive a lifetime maximum award of $2000.

The county-level number of HISA users was used as the numerator for calculation of county-level HISA utilization rate. Counties with zero HISA users were excluded. The number of EOY VHA patients per county in FY 2018 was divided by 1000 and used as the denominator in the calculation of county-level HISA utilization rate. Thus, the outcome variable is HISA utilization rate per 1000 VHA patients in FY 2018 (HISA utilization rate).

 

County-Level Variables

County-level variables were downloaded from the 2020 CHRD.^5,6 An explanation of the CHRD model and the factors used in this study are shown in the eAppendix (available at doi: 10.12788/fp.0279).⁶ County-level aggregated HISA user data and the CHRD data were matched using county Federal Information Processing Standards codes. Access to care measures collected from CHRD included percentages uninsured and ratios of population to primary care physicians, dentists, mental health professionals, and other primary care professionals. Other CHRD measures included those for quality of care (rate of preventable hospital stay) and housing quality (percent of households with high housing costs, percent of households with overcrowding, percent of households with lack of kitchen or plumbing, percent of households with severe housing cost burden, percent of homeownership). Of secondary interest was county population rurality, as previous research findings showed the annual average of HISA users who are from rural areas ranged from 30 to 35%.

Analysis Methods

SAS (v9.4), R (v4.0.2), and RStudio (v1.3.1093) were used for data preparation and analysis.¹⁸ Multiple regression analysis was used to predict county-level utilization rate from county-level variables. Sociodemographic characteristics from CHRD and HISA data were included as important control predictors in the regression model, though our focus for this paper are the access to care and housing quality predictors.

Model diagnostics (examination of model residuals, Breusch-Godfrey test, Breusch-Pagan test) revealed significant heteroskedasticity of the model; thus, robust standard errors and associated P values were computed using the R estimatr package (v0.30.2).¹⁹ Some predictor variables of interest (eg, ratio of mental health professionals) were removed during the model building process either due to problems of multicollinearity or excessive missingness that would have resulted in listwise deletion.

Results

County-level HISA utilization rate per 1000 EOY VHA patients ranged from 0.09 to 59.7%, with a 6.6% mean and 5% median (Figure 2). The data were highly positively skewed. The final model included 33 total predictor variables (Table 1). The final regression model was a significantly better predictor of county-level HISA utilization rate than a null model (F[33-2184], 10.18; P < .001). The adjusted model R² showed that the overall model accounted for approximately 23% of variance in county-level HISA utilization rate (Table 2).

 

 

Among the primary variables of interest, percent uninsured adults and rate of preventable hospital stays emerged as significant predictors of county-level HISA utilization rate. Specifically, county percentage of uninsured adults was negatively related to county-level HISA utilization rate (b = -8.99, P = .005), indicating that the higher the proportion of uninsured adults—with all other predictors held constant—the lower the HISA utilization rate. Percent uninsured adults ranged from 2.7 to 42.4% across counties, with a mean (SD) of 12.7% (5.8%) and 11.4% median.

County rate of preventable hospital stays, however, was significantly and positively related to county-level HISA utilization rate (b = .0004, P = .009), indicating that the higher the rate of preventable hospital stays—with all other predictors held constant—the higher the HISA utilization rate. The direction of this effect is the opposite of the direction of the effect of percent uninsured adults (positive rather than negative), even though both could be considered measures of access to care. The standardized β for these 2 predictors indicate that county rate of preventable hospital stays is a somewhat stronger predictor of county-level HISA utilization rate than is county percent of uninsured adults (β = .11 and β = -.09, respectively). Rate of preventable hospital stays ranged from 683 to 16,802 across counties included in this analysis, with a mean (SD) of 4,796.5 (1659.9) and a 4669 median.

Of secondary interest was county rurality. The county-level percentage of rural residents was significantly and positively related to county-level HISA utilization rate, indicating that the higher the proportion of individuals within county considered rural—all other predictors held constant—the higher the HISA utilization rate. The mean (SD) percentage of rural residents per county was 52.3% (30.2) and 52.7 % median.

 

Discussion

This study examined whether county-level characteristics, specifically variables for access to care, quality of care, and housing quality, were predictive of a county’s HISA utilization rate. Given that this series of work on the HISA program is (to our knowledge) the first of its kind, and given the exploratory nature of this analysis, we did not have specific predictions for the effects of any one given variable. Nevertheless, some of the results were surprising, and we believe they warrant additional study. In particular, the opposing direction of effects for access to care and quality of care variables were hard to reconcile.

The county percent of uninsured adults (an access to care variable, specifically, a proxy for poor access to care) was negatively associated with county-level HISA utilization rate, whereas the county rate of preventable hospital stays (a quality of care variable, but also potentially an access to care variable, and specifically, proxies for poor quality of care or poor access to care) was positively associated with county-level HISA utilization rate. To describe the relationships more generally, one coefficient in the regression model indicated that the poorer the access to care, the lower the HISA utilization rate (higher percent of uninsured adults predicts lower HISA utilization rate), while another coefficient in the regression model indicated the poorer the quality of and access to care, the higher the HISA utilization rate (higher rate of preventable hospital stays predicts higher HISA utilization rate). Future study is warranted to disentangle and reconcile the various community-level predictors of this service.

Housing quality measures (eg, percent of households with high housing costs, percent of households with overcrowding, percent of households with lack of kitchen or plumbing, percent of households with severe housing cost burden, and percent of homeownership) are important in the consideration of whether a HM will be performed or should be performed. For example, if a person is cost burdened by the amount of expenditure spent in housing there will be little discretionary funds to perform a HM. Individuals who do not own their home may experience complications in obtaining permission from landlords to perform a HM. County-level predictors of housing quality (percent of households with high housing costs, overcrowding, and lack of kitchen or plumbing) were not significantly associated with county-level HISA utilization rate but are also nevertheless relevant to the discussion of home modifications. Of particular interest is the percent of households with lack of kitchen or plumbing variable, which was positively related to county-level HISA utilization rate although not statistically significant. HM elements related to the kitchen (eg, heighten countertop) add to the accessibility of the home allowing for the performing of activities of daily living such as cooking. Between FY 2015 and FY 2018, we discovered 131 prescriptions for kitchen (n = 90) and plumbing (n = 41) HMs, which is a very small proportion of the 30,780 total HMs (there were 24,397 bathroom HMs). The nonsignificant coefficient for this variable may reflect the small number of veterans that obtained these HM.

Limitations

The potentially conflicting direction of effects for a significant access to care variable (percent uninsured adults) and a significant access to care and quality of care variable (preventable hospital stays) are interesting and warrant additional study, but the inability to interpret or explain this apparent inconsistency constitutes a limitation of the current data and analyses presented here. Another limitation is that this analysis uses county-level predictors for what is ultimately an individual-level outcome. It would have been ideal to have both individual- and county-level data to conduct a multilevel analysis; in particular, individual-level data within counties of individuals (both veterans and nonveterans) who did not receive a HISA award (including both those who applied and were denied, and who did not apply) would be highly valuable.

Conclusions

Our continuing research into veterans’ use of HM fills a gap in the literature about the characteristics of HISA users, the impact of county-level variables on the use of HISA, and the geographic distribution and use of HISA within the VHA. While it is important to examine the influence of broader systems on individual outcomes, there could be myriad other factors that are more proximal and more closely related to whether any one individual applies for, let alone receives, a HISA award. Indeed, a low overall adjusted model R² indicates that there is considerable variability in county-level HISA utilization rate that was not accounted for by the current model; this further speaks to warranted additional study.

More research is needed to understand and account for geographical variation in HISA utilization rate across the US. However, this work serves as an exploratory first step at quantifying and predicting HISA utilization rate at a broad level, with the ultimate goal of increasing access to HMs for veterans with disabilities.

Acknowledgments

This research was supported by grant 15521 from the US Department of Veterans Affairs, Office of Rural Health. Furthermore, the research was supported in part by grant K12 HD055929 from the National Institutes of Health. We want to acknowledge Cheri E. Knecht, Project Coordinator, for her assistance throughout all aspects of our research study and for her thoughtful contributions during the writing of this manuscript.

This article is part of a series of articles on the Home Improvements and Structural Alterations program (HISA), a home modification (HM) program within the Veterans Health Administration (VHA). HISA is a benefit awarded to veterans with disabilities (VWDs) and is instrumental in affording physical accessibility and structural alterations to veterans’ homes.¹ The overarching goals of this project are to describe and understand HISA use by VWDs. Previous work has shown geographical variation in the number of HISA prescriptions across counties in the US (Figure 1).¹ The current work seeks to describe and predict the county-level rates of HISA use. Information about what predicts HISA utilization at the county level is important because it enhances understanding of program utilization at a national level. The long-term goal of the series is to provide knowledge about HM services within VHA to improve community-based independent living of VWDs by increasing awareness and utilization of HM services. 

Background

A health care professional (HCP) approves a HM support award by evaluating the practicality of the support to improve the built environment of a given veteran’s disability.^1,2 Previously we detailed some of the preliminary research into the HISA program, including HISA user demographic and clinical characteristics, types of HMs received, user suggestions for improvement, and geospatial analysis of HISA prescriptions concentration.^1-4

The geospatial analyses of HISA prescriptions revealed clusters of high numbers of HISA users (hot spots) and low numbers of HISA users (cold spots), indicating that HISA is either not prescribed or uniformly used across the US. The previous research prompted investigation into county-level variables that may impact HISA utilization rates. This inquiry focuses on county characteristics associated with HISA use rates, such as measures of clinical care and quality of care (eg, access to health services variables, lack of insurance, preventable hospital stays), physical environment, and sociodemographic characteristics. Clinical care and quality of care measures promote the interaction with HCPs. Moreover, access to health care is an important indicator of health outcomes.^5,6 An individual’s capacity to access health services, such as a HM program, greatly impacts well-being, safety, independence, and health.^2,4 Well-being, safety, independence, and health become compromised if individuals cannot access care, if needed care is lacking in their area, if HCPs are not available, or are unwilling to provide care due to lack of insurance coverage.^7-12 In locations where health care services are minimal due to lack of specialists or health care facilities, the quality of (or access to) care may be compromised, resulting in preventable conditions becoming problematic.^13,14 These conditions may result in unnecessary hospitalizations for conditions that could have been treated during routine care. Financial barriers to care particularly among low-income people and the uninsured have proven detrimental to health.^15,16 On the other hand, preventable hospital stays are a quality of care measure (ie, a proxy for poor quality of care). HISA operates within a health care system; thus, it is imperative to include these measures impacting health. 

In this study, we sought to identify county-level predictors—in particular, county-level proxies for access to care—that may be associated with county-level HISA use. We define HISA utilization rate as the percentage of a county’s VHA patients who have received a HISA award.

Methods

This study used data from the National Prosthetics Patient Database (NPPD), US Department of Veterans Affairs (VA) medical database inpatient and outpatient datasets, VHA Support Service Center (VSSC) data cubes, and the County Health Rankings database (CHRD). First, the study cohort was identified from NPPD users who have obtained a HISA award from fiscal years (FY) 2015 to 2018. Analysis started with FY 2015 following new regulations (38 CFR § 17) governing the operations of the HISA program.² The study cohort was matched with records from NPPD and VA inpatient and outpatient datasets to obtain information about the veterans’ demographic characteristics and their HM characteristics and costs. The number of VHA end-of-year (EOY) patients per county was extracted from the VSSC Current Enrollment Cube, which was used in calculation of the county-level HISA utilization rate.¹⁷ Finally, zip code–based locational data were used to calculate approximate drive time and distance from the HISA user’s approximate location to the facility where they received their HM prescription. Drive times and drive distances were calculated with Esri ArcGIS Pro (v2.6.3) by placing zip code centroid and VHA facilities on a nationwide road network that contains both road speeds and distances.

Calculations

Patient-level data were aggregated up to county-level variables by calculating the sum, mean, or percent per county. HISA user sample characteristics, including sex, race, rurality (urban, rural), marital status, and Class 1 vs Class 2 disability-related eligibility groups, were aggregated to the county level by calculating percentages of HISA users of the given characteristics out of total HISA users in the county. Disability-related eligibility groups (Class 1 vs Class 2 HISA users) determines the maximum lifetime award dollar amount. Specifically, those with service-connected disabilities or those with a ≥ 50% disability rating (regardless of whether or not their disability is service connected) are classified as Class 1 HISA users and are eligible to receive a maximum lifetime award of $6800. Those with a recorded disability that is not connected to their military service, and who have a disability rating of < 50% are classified as Class 2 HISA users and are eligible to receive a lifetime maximum award of $2000.

The county-level number of HISA users was used as the numerator for calculation of county-level HISA utilization rate. Counties with zero HISA users were excluded. The number of EOY VHA patients per county in FY 2018 was divided by 1000 and used as the denominator in the calculation of county-level HISA utilization rate. Thus, the outcome variable is HISA utilization rate per 1000 VHA patients in FY 2018 (HISA utilization rate).

 

County-Level Variables

County-level variables were downloaded from the 2020 CHRD.^5,6 An explanation of the CHRD model and the factors used in this study are shown in the eAppendix (available at doi: 10.12788/fp.0279).⁶ County-level aggregated HISA user data and the CHRD data were matched using county Federal Information Processing Standards codes. Access to care measures collected from CHRD included percentages uninsured and ratios of population to primary care physicians, dentists, mental health professionals, and other primary care professionals. Other CHRD measures included those for quality of care (rate of preventable hospital stay) and housing quality (percent of households with high housing costs, percent of households with overcrowding, percent of households with lack of kitchen or plumbing, percent of households with severe housing cost burden, percent of homeownership). Of secondary interest was county population rurality, as previous research findings showed the annual average of HISA users who are from rural areas ranged from 30 to 35%.

Analysis Methods

SAS (v9.4), R (v4.0.2), and RStudio (v1.3.1093) were used for data preparation and analysis.¹⁸ Multiple regression analysis was used to predict county-level utilization rate from county-level variables. Sociodemographic characteristics from CHRD and HISA data were included as important control predictors in the regression model, though our focus for this paper are the access to care and housing quality predictors.

Model diagnostics (examination of model residuals, Breusch-Godfrey test, Breusch-Pagan test) revealed significant heteroskedasticity of the model; thus, robust standard errors and associated P values were computed using the R estimatr package (v0.30.2).¹⁹ Some predictor variables of interest (eg, ratio of mental health professionals) were removed during the model building process either due to problems of multicollinearity or excessive missingness that would have resulted in listwise deletion.

Results

County-level HISA utilization rate per 1000 EOY VHA patients ranged from 0.09 to 59.7%, with a 6.6% mean and 5% median (Figure 2). The data were highly positively skewed. The final model included 33 total predictor variables (Table 1). The final regression model was a significantly better predictor of county-level HISA utilization rate than a null model (F[33-2184], 10.18; P < .001). The adjusted model R² showed that the overall model accounted for approximately 23% of variance in county-level HISA utilization rate (Table 2).

 

 

Among the primary variables of interest, percent uninsured adults and rate of preventable hospital stays emerged as significant predictors of county-level HISA utilization rate. Specifically, county percentage of uninsured adults was negatively related to county-level HISA utilization rate (b = -8.99, P = .005), indicating that the higher the proportion of uninsured adults—with all other predictors held constant—the lower the HISA utilization rate. Percent uninsured adults ranged from 2.7 to 42.4% across counties, with a mean (SD) of 12.7% (5.8%) and 11.4% median.

County rate of preventable hospital stays, however, was significantly and positively related to county-level HISA utilization rate (b = .0004, P = .009), indicating that the higher the rate of preventable hospital stays—with all other predictors held constant—the higher the HISA utilization rate. The direction of this effect is the opposite of the direction of the effect of percent uninsured adults (positive rather than negative), even though both could be considered measures of access to care. The standardized β for these 2 predictors indicate that county rate of preventable hospital stays is a somewhat stronger predictor of county-level HISA utilization rate than is county percent of uninsured adults (β = .11 and β = -.09, respectively). Rate of preventable hospital stays ranged from 683 to 16,802 across counties included in this analysis, with a mean (SD) of 4,796.5 (1659.9) and a 4669 median.

Of secondary interest was county rurality. The county-level percentage of rural residents was significantly and positively related to county-level HISA utilization rate, indicating that the higher the proportion of individuals within county considered rural—all other predictors held constant—the higher the HISA utilization rate. The mean (SD) percentage of rural residents per county was 52.3% (30.2) and 52.7 % median.

 

Discussion

This study examined whether county-level characteristics, specifically variables for access to care, quality of care, and housing quality, were predictive of a county’s HISA utilization rate. Given that this series of work on the HISA program is (to our knowledge) the first of its kind, and given the exploratory nature of this analysis, we did not have specific predictions for the effects of any one given variable. Nevertheless, some of the results were surprising, and we believe they warrant additional study. In particular, the opposing direction of effects for access to care and quality of care variables were hard to reconcile.

The county percent of uninsured adults (an access to care variable, specifically, a proxy for poor access to care) was negatively associated with county-level HISA utilization rate, whereas the county rate of preventable hospital stays (a quality of care variable, but also potentially an access to care variable, and specifically, proxies for poor quality of care or poor access to care) was positively associated with county-level HISA utilization rate. To describe the relationships more generally, one coefficient in the regression model indicated that the poorer the access to care, the lower the HISA utilization rate (higher percent of uninsured adults predicts lower HISA utilization rate), while another coefficient in the regression model indicated the poorer the quality of and access to care, the higher the HISA utilization rate (higher rate of preventable hospital stays predicts higher HISA utilization rate). Future study is warranted to disentangle and reconcile the various community-level predictors of this service.

Housing quality measures (eg, percent of households with high housing costs, percent of households with overcrowding, percent of households with lack of kitchen or plumbing, percent of households with severe housing cost burden, and percent of homeownership) are important in the consideration of whether a HM will be performed or should be performed. For example, if a person is cost burdened by the amount of expenditure spent in housing there will be little discretionary funds to perform a HM. Individuals who do not own their home may experience complications in obtaining permission from landlords to perform a HM. County-level predictors of housing quality (percent of households with high housing costs, overcrowding, and lack of kitchen or plumbing) were not significantly associated with county-level HISA utilization rate but are also nevertheless relevant to the discussion of home modifications. Of particular interest is the percent of households with lack of kitchen or plumbing variable, which was positively related to county-level HISA utilization rate although not statistically significant. HM elements related to the kitchen (eg, heighten countertop) add to the accessibility of the home allowing for the performing of activities of daily living such as cooking. Between FY 2015 and FY 2018, we discovered 131 prescriptions for kitchen (n = 90) and plumbing (n = 41) HMs, which is a very small proportion of the 30,780 total HMs (there were 24,397 bathroom HMs). The nonsignificant coefficient for this variable may reflect the small number of veterans that obtained these HM.

Limitations

The potentially conflicting direction of effects for a significant access to care variable (percent uninsured adults) and a significant access to care and quality of care variable (preventable hospital stays) are interesting and warrant additional study, but the inability to interpret or explain this apparent inconsistency constitutes a limitation of the current data and analyses presented here. Another limitation is that this analysis uses county-level predictors for what is ultimately an individual-level outcome. It would have been ideal to have both individual- and county-level data to conduct a multilevel analysis; in particular, individual-level data within counties of individuals (both veterans and nonveterans) who did not receive a HISA award (including both those who applied and were denied, and who did not apply) would be highly valuable.

Conclusions

Our continuing research into veterans’ use of HM fills a gap in the literature about the characteristics of HISA users, the impact of county-level variables on the use of HISA, and the geographic distribution and use of HISA within the VHA. While it is important to examine the influence of broader systems on individual outcomes, there could be myriad other factors that are more proximal and more closely related to whether any one individual applies for, let alone receives, a HISA award. Indeed, a low overall adjusted model R² indicates that there is considerable variability in county-level HISA utilization rate that was not accounted for by the current model; this further speaks to warranted additional study.

More research is needed to understand and account for geographical variation in HISA utilization rate across the US. However, this work serves as an exploratory first step at quantifying and predicting HISA utilization rate at a broad level, with the ultimate goal of increasing access to HMs for veterans with disabilities.

Acknowledgments

This research was supported by grant 15521 from the US Department of Veterans Affairs, Office of Rural Health. Furthermore, the research was supported in part by grant K12 HD055929 from the National Institutes of Health. We want to acknowledge Cheri E. Knecht, Project Coordinator, for her assistance throughout all aspects of our research study and for her thoughtful contributions during the writing of this manuscript.

This article is part of a series of articles on the Home Improvements and Structural Alterations program (HISA), a home modification (HM) program within the Veterans Health Administration (VHA). HISA is a benefit awarded to veterans with disabilities (VWDs) and is instrumental in affording physical accessibility and structural alterations to veterans’ homes.¹ The overarching goals of this project are to describe and understand HISA use by VWDs. Previous work has shown geographical variation in the number of HISA prescriptions across counties in the US (Figure 1).¹ The current work seeks to describe and predict the county-level rates of HISA use. Information about what predicts HISA utilization at the county level is important because it enhances understanding of program utilization at a national level. The long-term goal of the series is to provide knowledge about HM services within VHA to improve community-based independent living of VWDs by increasing awareness and utilization of HM services. 

Background

A health care professional (HCP) approves a HM support award by evaluating the practicality of the support to improve the built environment of a given veteran’s disability.^1,2 Previously we detailed some of the preliminary research into the HISA program, including HISA user demographic and clinical characteristics, types of HMs received, user suggestions for improvement, and geospatial analysis of HISA prescriptions concentration.^1-4

The geospatial analyses of HISA prescriptions revealed clusters of high numbers of HISA users (hot spots) and low numbers of HISA users (cold spots), indicating that HISA is either not prescribed or uniformly used across the US. The previous research prompted investigation into county-level variables that may impact HISA utilization rates. This inquiry focuses on county characteristics associated with HISA use rates, such as measures of clinical care and quality of care (eg, access to health services variables, lack of insurance, preventable hospital stays), physical environment, and sociodemographic characteristics. Clinical care and quality of care measures promote the interaction with HCPs. Moreover, access to health care is an important indicator of health outcomes.^5,6 An individual’s capacity to access health services, such as a HM program, greatly impacts well-being, safety, independence, and health.^2,4 Well-being, safety, independence, and health become compromised if individuals cannot access care, if needed care is lacking in their area, if HCPs are not available, or are unwilling to provide care due to lack of insurance coverage.^7-12 In locations where health care services are minimal due to lack of specialists or health care facilities, the quality of (or access to) care may be compromised, resulting in preventable conditions becoming problematic.^13,14 These conditions may result in unnecessary hospitalizations for conditions that could have been treated during routine care. Financial barriers to care particularly among low-income people and the uninsured have proven detrimental to health.^15,16 On the other hand, preventable hospital stays are a quality of care measure (ie, a proxy for poor quality of care). HISA operates within a health care system; thus, it is imperative to include these measures impacting health. 

In this study, we sought to identify county-level predictors—in particular, county-level proxies for access to care—that may be associated with county-level HISA use. We define HISA utilization rate as the percentage of a county’s VHA patients who have received a HISA award.

Methods

This study used data from the National Prosthetics Patient Database (NPPD), US Department of Veterans Affairs (VA) medical database inpatient and outpatient datasets, VHA Support Service Center (VSSC) data cubes, and the County Health Rankings database (CHRD). First, the study cohort was identified from NPPD users who have obtained a HISA award from fiscal years (FY) 2015 to 2018. Analysis started with FY 2015 following new regulations (38 CFR § 17) governing the operations of the HISA program.² The study cohort was matched with records from NPPD and VA inpatient and outpatient datasets to obtain information about the veterans’ demographic characteristics and their HM characteristics and costs. The number of VHA end-of-year (EOY) patients per county was extracted from the VSSC Current Enrollment Cube, which was used in calculation of the county-level HISA utilization rate.¹⁷ Finally, zip code–based locational data were used to calculate approximate drive time and distance from the HISA user’s approximate location to the facility where they received their HM prescription. Drive times and drive distances were calculated with Esri ArcGIS Pro (v2.6.3) by placing zip code centroid and VHA facilities on a nationwide road network that contains both road speeds and distances.

Calculations

Patient-level data were aggregated up to county-level variables by calculating the sum, mean, or percent per county. HISA user sample characteristics, including sex, race, rurality (urban, rural), marital status, and Class 1 vs Class 2 disability-related eligibility groups, were aggregated to the county level by calculating percentages of HISA users of the given characteristics out of total HISA users in the county. Disability-related eligibility groups (Class 1 vs Class 2 HISA users) determines the maximum lifetime award dollar amount. Specifically, those with service-connected disabilities or those with a ≥ 50% disability rating (regardless of whether or not their disability is service connected) are classified as Class 1 HISA users and are eligible to receive a maximum lifetime award of $6800. Those with a recorded disability that is not connected to their military service, and who have a disability rating of < 50% are classified as Class 2 HISA users and are eligible to receive a lifetime maximum award of $2000.

The county-level number of HISA users was used as the numerator for calculation of county-level HISA utilization rate. Counties with zero HISA users were excluded. The number of EOY VHA patients per county in FY 2018 was divided by 1000 and used as the denominator in the calculation of county-level HISA utilization rate. Thus, the outcome variable is HISA utilization rate per 1000 VHA patients in FY 2018 (HISA utilization rate).

 

County-Level Variables

County-level variables were downloaded from the 2020 CHRD.^5,6 An explanation of the CHRD model and the factors used in this study are shown in the eAppendix (available at doi: 10.12788/fp.0279).⁶ County-level aggregated HISA user data and the CHRD data were matched using county Federal Information Processing Standards codes. Access to care measures collected from CHRD included percentages uninsured and ratios of population to primary care physicians, dentists, mental health professionals, and other primary care professionals. Other CHRD measures included those for quality of care (rate of preventable hospital stay) and housing quality (percent of households with high housing costs, percent of households with overcrowding, percent of households with lack of kitchen or plumbing, percent of households with severe housing cost burden, percent of homeownership). Of secondary interest was county population rurality, as previous research findings showed the annual average of HISA users who are from rural areas ranged from 30 to 35%.

Analysis Methods

SAS (v9.4), R (v4.0.2), and RStudio (v1.3.1093) were used for data preparation and analysis.¹⁸ Multiple regression analysis was used to predict county-level utilization rate from county-level variables. Sociodemographic characteristics from CHRD and HISA data were included as important control predictors in the regression model, though our focus for this paper are the access to care and housing quality predictors.

Model diagnostics (examination of model residuals, Breusch-Godfrey test, Breusch-Pagan test) revealed significant heteroskedasticity of the model; thus, robust standard errors and associated P values were computed using the R estimatr package (v0.30.2).¹⁹ Some predictor variables of interest (eg, ratio of mental health professionals) were removed during the model building process either due to problems of multicollinearity or excessive missingness that would have resulted in listwise deletion.

Results

County-level HISA utilization rate per 1000 EOY VHA patients ranged from 0.09 to 59.7%, with a 6.6% mean and 5% median (Figure 2). The data were highly positively skewed. The final model included 33 total predictor variables (Table 1). The final regression model was a significantly better predictor of county-level HISA utilization rate than a null model (F[33-2184], 10.18; P < .001). The adjusted model R² showed that the overall model accounted for approximately 23% of variance in county-level HISA utilization rate (Table 2).

 

 

Among the primary variables of interest, percent uninsured adults and rate of preventable hospital stays emerged as significant predictors of county-level HISA utilization rate. Specifically, county percentage of uninsured adults was negatively related to county-level HISA utilization rate (b = -8.99, P = .005), indicating that the higher the proportion of uninsured adults—with all other predictors held constant—the lower the HISA utilization rate. Percent uninsured adults ranged from 2.7 to 42.4% across counties, with a mean (SD) of 12.7% (5.8%) and 11.4% median.

County rate of preventable hospital stays, however, was significantly and positively related to county-level HISA utilization rate (b = .0004, P = .009), indicating that the higher the rate of preventable hospital stays—with all other predictors held constant—the higher the HISA utilization rate. The direction of this effect is the opposite of the direction of the effect of percent uninsured adults (positive rather than negative), even though both could be considered measures of access to care. The standardized β for these 2 predictors indicate that county rate of preventable hospital stays is a somewhat stronger predictor of county-level HISA utilization rate than is county percent of uninsured adults (β = .11 and β = -.09, respectively). Rate of preventable hospital stays ranged from 683 to 16,802 across counties included in this analysis, with a mean (SD) of 4,796.5 (1659.9) and a 4669 median.

Of secondary interest was county rurality. The county-level percentage of rural residents was significantly and positively related to county-level HISA utilization rate, indicating that the higher the proportion of individuals within county considered rural—all other predictors held constant—the higher the HISA utilization rate. The mean (SD) percentage of rural residents per county was 52.3% (30.2) and 52.7 % median.

 

Discussion

This study examined whether county-level characteristics, specifically variables for access to care, quality of care, and housing quality, were predictive of a county’s HISA utilization rate. Given that this series of work on the HISA program is (to our knowledge) the first of its kind, and given the exploratory nature of this analysis, we did not have specific predictions for the effects of any one given variable. Nevertheless, some of the results were surprising, and we believe they warrant additional study. In particular, the opposing direction of effects for access to care and quality of care variables were hard to reconcile.

The county percent of uninsured adults (an access to care variable, specifically, a proxy for poor access to care) was negatively associated with county-level HISA utilization rate, whereas the county rate of preventable hospital stays (a quality of care variable, but also potentially an access to care variable, and specifically, proxies for poor quality of care or poor access to care) was positively associated with county-level HISA utilization rate. To describe the relationships more generally, one coefficient in the regression model indicated that the poorer the access to care, the lower the HISA utilization rate (higher percent of uninsured adults predicts lower HISA utilization rate), while another coefficient in the regression model indicated the poorer the quality of and access to care, the higher the HISA utilization rate (higher rate of preventable hospital stays predicts higher HISA utilization rate). Future study is warranted to disentangle and reconcile the various community-level predictors of this service.

Housing quality measures (eg, percent of households with high housing costs, percent of households with overcrowding, percent of households with lack of kitchen or plumbing, percent of households with severe housing cost burden, and percent of homeownership) are important in the consideration of whether a HM will be performed or should be performed. For example, if a person is cost burdened by the amount of expenditure spent in housing there will be little discretionary funds to perform a HM. Individuals who do not own their home may experience complications in obtaining permission from landlords to perform a HM. County-level predictors of housing quality (percent of households with high housing costs, overcrowding, and lack of kitchen or plumbing) were not significantly associated with county-level HISA utilization rate but are also nevertheless relevant to the discussion of home modifications. Of particular interest is the percent of households with lack of kitchen or plumbing variable, which was positively related to county-level HISA utilization rate although not statistically significant. HM elements related to the kitchen (eg, heighten countertop) add to the accessibility of the home allowing for the performing of activities of daily living such as cooking. Between FY 2015 and FY 2018, we discovered 131 prescriptions for kitchen (n = 90) and plumbing (n = 41) HMs, which is a very small proportion of the 30,780 total HMs (there were 24,397 bathroom HMs). The nonsignificant coefficient for this variable may reflect the small number of veterans that obtained these HM.

Limitations

The potentially conflicting direction of effects for a significant access to care variable (percent uninsured adults) and a significant access to care and quality of care variable (preventable hospital stays) are interesting and warrant additional study, but the inability to interpret or explain this apparent inconsistency constitutes a limitation of the current data and analyses presented here. Another limitation is that this analysis uses county-level predictors for what is ultimately an individual-level outcome. It would have been ideal to have both individual- and county-level data to conduct a multilevel analysis; in particular, individual-level data within counties of individuals (both veterans and nonveterans) who did not receive a HISA award (including both those who applied and were denied, and who did not apply) would be highly valuable.

Conclusions

Our continuing research into veterans’ use of HM fills a gap in the literature about the characteristics of HISA users, the impact of county-level variables on the use of HISA, and the geographic distribution and use of HISA within the VHA. While it is important to examine the influence of broader systems on individual outcomes, there could be myriad other factors that are more proximal and more closely related to whether any one individual applies for, let alone receives, a HISA award. Indeed, a low overall adjusted model R² indicates that there is considerable variability in county-level HISA utilization rate that was not accounted for by the current model; this further speaks to warranted additional study.

More research is needed to understand and account for geographical variation in HISA utilization rate across the US. However, this work serves as an exploratory first step at quantifying and predicting HISA utilization rate at a broad level, with the ultimate goal of increasing access to HMs for veterans with disabilities.

Acknowledgments

This research was supported by grant 15521 from the US Department of Veterans Affairs, Office of Rural Health. Furthermore, the research was supported in part by grant K12 HD055929 from the National Institutes of Health. We want to acknowledge Cheri E. Knecht, Project Coordinator, for her assistance throughout all aspects of our research study and for her thoughtful contributions during the writing of this manuscript.

To the Editor:

The internet is a popular resource for patients seeking information about dermatologic treatments. Hydroquinone (HQ) cream 4% is approved by the US Food and Drug Administration for skin hyperpigmentation.¹ The agency enforced the CARES (Coronavirus Aid, Relief, and Economic Security) Act and OTC (over-the-counter) Monograph Reform on September 25, 2020, to restrict distribution of OTC HQ.² Exogenous ochronosis is listed as a potential adverse effect in the prescribing information for HQ.¹

We sought to assess online resources on HQ for accuracy of information, including the recent OTC ban, as well as readability. The word hydroquinone was searched on 3 internet search engines—Google, Yahoo, and Bing—on December 12, 2020, each for the first 20 URLs (ie, websites)(total of 60 URLs). Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA)(Figure) reporting guidelines were used to assess a list of relevant websites to include in the final analysis. Website data were reviewed by both authors. Eighteen duplicates and 27 irrelevant and non–English-language URLs were excluded. The remaining 15 websites were analyzed. Based on a previously published and validated tool, a pro forma was designed to evaluate information on HQ for each website based on accountability, quality, readability, display, support, and transparency (Table).^1,3

Scores for all 15 websites are listed in eTable 1. The mean overall (total) score was 25.3 points (of a maximum possible score of 44 points; range, 18–34). The average accountability score was 6.3 (of a possible 10; range, 3–10); average quality score, 10.9 (of a possible 17; range, 5–16); and average readability score, 2.1 (of a possible 5; range, 0–5).

The mean display score was 0.3 (of a possible 4; range, 0–2); 66.7% of websites (10/15) had advertisements or irrelevant material. Only 6.7% and 13.3% of websites included relevant videos or images, respectively, on applying HQ (eTable 2). We identified only 3 photographs—across all 15 websites—that depicted skin, all of which were Fitzpatrick skin types II or III. Therefore, none of the websites included a diversity of images to indicate broad ethnic relatability.

The average support score was 2.5 (of a possible 4; range, 1–3); 20% (3/15) of URLs included chat sites, message boards, or forums, and approximately half (8/15 [53.3%]) included references. Only 7 URLs (46.7%) had been updated in the last 12 months. Only 4 (26.7%) were written by a board-certified dermatologist (eTable 2). Most (60%) websites contained advertising, though none were sponsored by a pharmaceutical company that manufactures HQ.

Only 46.7% (7/15) of websites recommended limiting a course of HQ treatment to 3 months; only 40% (6/15) mentioned shelf life or photochemical degradation when exposed to air. Although 93.3% (14/15) of URLs mentioned ochronosis, a clinical description of the condition was provided in only 33.3% (5/15)—none with images.

Only 2 sites (13.3%; Everyday Health and WebMD) met the accepted 7th-grade reading level for online patient education material; those sites scored lower on quality (9 of 17 and 6 of 17, respectively) than sites with higher overall scores.

 

None of the 15 websites studied, therefore, demonstrated optimal features on combined measures of accountability, quality, readability, display, support, and transparency regarding HQ. Notably, the American Academy of Dermatology website (www.aad.org) was not among the 15 websites studied; the AAD website mentions HQ in a section on melasma, but only minimal detail is provided.

Limitations of this study include the small number of websites analyzed and possible selection bias because only 3 internet search engines were used to identify websites for study and analysis.

Previously, we analyzed content about HQ on the video-sharing and social media platform YouTube.⁴ The most viewed YouTube videos on HQ had poor-quality information (ie, only 20% mentioned ochronosis and only 28.6% recommended sunscreen [N=70]). However, average reading level of these videos was 7th grade.^4,5 Therefore, YouTube HQ content, though comprehensible, generally is of poor quality.

By conducting a search for website content about HQ, we found that the most popular URLs had either accurate information with poor readability or lower-quality educational material that was more comprehensible. We conclude that there is a need to develop online patient education materials on HQ that are characterized by high-quality, up-to-date medical information; have been written by board-certified dermatologists; are comprehensible (ie, no more than approximately 1200 words and written at a 7th-grade reading level); and contain relevant clinical images and references. We encourage dermatologists to recognize the limitations of online patient education resources on HQ and educate patients on the proper use of the drug as well as its potential adverse effects

To the Editor:

The internet is a popular resource for patients seeking information about dermatologic treatments. Hydroquinone (HQ) cream 4% is approved by the US Food and Drug Administration for skin hyperpigmentation.¹ The agency enforced the CARES (Coronavirus Aid, Relief, and Economic Security) Act and OTC (over-the-counter) Monograph Reform on September 25, 2020, to restrict distribution of OTC HQ.² Exogenous ochronosis is listed as a potential adverse effect in the prescribing information for HQ.¹

We sought to assess online resources on HQ for accuracy of information, including the recent OTC ban, as well as readability. The word hydroquinone was searched on 3 internet search engines—Google, Yahoo, and Bing—on December 12, 2020, each for the first 20 URLs (ie, websites)(total of 60 URLs). Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA)(Figure) reporting guidelines were used to assess a list of relevant websites to include in the final analysis. Website data were reviewed by both authors. Eighteen duplicates and 27 irrelevant and non–English-language URLs were excluded. The remaining 15 websites were analyzed. Based on a previously published and validated tool, a pro forma was designed to evaluate information on HQ for each website based on accountability, quality, readability, display, support, and transparency (Table).^1,3

Scores for all 15 websites are listed in eTable 1. The mean overall (total) score was 25.3 points (of a maximum possible score of 44 points; range, 18–34). The average accountability score was 6.3 (of a possible 10; range, 3–10); average quality score, 10.9 (of a possible 17; range, 5–16); and average readability score, 2.1 (of a possible 5; range, 0–5).

The mean display score was 0.3 (of a possible 4; range, 0–2); 66.7% of websites (10/15) had advertisements or irrelevant material. Only 6.7% and 13.3% of websites included relevant videos or images, respectively, on applying HQ (eTable 2). We identified only 3 photographs—across all 15 websites—that depicted skin, all of which were Fitzpatrick skin types II or III. Therefore, none of the websites included a diversity of images to indicate broad ethnic relatability.

The average support score was 2.5 (of a possible 4; range, 1–3); 20% (3/15) of URLs included chat sites, message boards, or forums, and approximately half (8/15 [53.3%]) included references. Only 7 URLs (46.7%) had been updated in the last 12 months. Only 4 (26.7%) were written by a board-certified dermatologist (eTable 2). Most (60%) websites contained advertising, though none were sponsored by a pharmaceutical company that manufactures HQ.

Only 46.7% (7/15) of websites recommended limiting a course of HQ treatment to 3 months; only 40% (6/15) mentioned shelf life or photochemical degradation when exposed to air. Although 93.3% (14/15) of URLs mentioned ochronosis, a clinical description of the condition was provided in only 33.3% (5/15)—none with images.

Only 2 sites (13.3%; Everyday Health and WebMD) met the accepted 7th-grade reading level for online patient education material; those sites scored lower on quality (9 of 17 and 6 of 17, respectively) than sites with higher overall scores.

 

None of the 15 websites studied, therefore, demonstrated optimal features on combined measures of accountability, quality, readability, display, support, and transparency regarding HQ. Notably, the American Academy of Dermatology website (www.aad.org) was not among the 15 websites studied; the AAD website mentions HQ in a section on melasma, but only minimal detail is provided.

Limitations of this study include the small number of websites analyzed and possible selection bias because only 3 internet search engines were used to identify websites for study and analysis.

Previously, we analyzed content about HQ on the video-sharing and social media platform YouTube.⁴ The most viewed YouTube videos on HQ had poor-quality information (ie, only 20% mentioned ochronosis and only 28.6% recommended sunscreen [N=70]). However, average reading level of these videos was 7th grade.^4,5 Therefore, YouTube HQ content, though comprehensible, generally is of poor quality.

By conducting a search for website content about HQ, we found that the most popular URLs had either accurate information with poor readability or lower-quality educational material that was more comprehensible. We conclude that there is a need to develop online patient education materials on HQ that are characterized by high-quality, up-to-date medical information; have been written by board-certified dermatologists; are comprehensible (ie, no more than approximately 1200 words and written at a 7th-grade reading level); and contain relevant clinical images and references. We encourage dermatologists to recognize the limitations of online patient education resources on HQ and educate patients on the proper use of the drug as well as its potential adverse effects

To the Editor:

The internet is a popular resource for patients seeking information about dermatologic treatments. Hydroquinone (HQ) cream 4% is approved by the US Food and Drug Administration for skin hyperpigmentation.¹ The agency enforced the CARES (Coronavirus Aid, Relief, and Economic Security) Act and OTC (over-the-counter) Monograph Reform on September 25, 2020, to restrict distribution of OTC HQ.² Exogenous ochronosis is listed as a potential adverse effect in the prescribing information for HQ.¹

We sought to assess online resources on HQ for accuracy of information, including the recent OTC ban, as well as readability. The word hydroquinone was searched on 3 internet search engines—Google, Yahoo, and Bing—on December 12, 2020, each for the first 20 URLs (ie, websites)(total of 60 URLs). Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA)(Figure) reporting guidelines were used to assess a list of relevant websites to include in the final analysis. Website data were reviewed by both authors. Eighteen duplicates and 27 irrelevant and non–English-language URLs were excluded. The remaining 15 websites were analyzed. Based on a previously published and validated tool, a pro forma was designed to evaluate information on HQ for each website based on accountability, quality, readability, display, support, and transparency (Table).^1,3

Scores for all 15 websites are listed in eTable 1. The mean overall (total) score was 25.3 points (of a maximum possible score of 44 points; range, 18–34). The average accountability score was 6.3 (of a possible 10; range, 3–10); average quality score, 10.9 (of a possible 17; range, 5–16); and average readability score, 2.1 (of a possible 5; range, 0–5).

The mean display score was 0.3 (of a possible 4; range, 0–2); 66.7% of websites (10/15) had advertisements or irrelevant material. Only 6.7% and 13.3% of websites included relevant videos or images, respectively, on applying HQ (eTable 2). We identified only 3 photographs—across all 15 websites—that depicted skin, all of which were Fitzpatrick skin types II or III. Therefore, none of the websites included a diversity of images to indicate broad ethnic relatability.

The average support score was 2.5 (of a possible 4; range, 1–3); 20% (3/15) of URLs included chat sites, message boards, or forums, and approximately half (8/15 [53.3%]) included references. Only 7 URLs (46.7%) had been updated in the last 12 months. Only 4 (26.7%) were written by a board-certified dermatologist (eTable 2). Most (60%) websites contained advertising, though none were sponsored by a pharmaceutical company that manufactures HQ.

Only 46.7% (7/15) of websites recommended limiting a course of HQ treatment to 3 months; only 40% (6/15) mentioned shelf life or photochemical degradation when exposed to air. Although 93.3% (14/15) of URLs mentioned ochronosis, a clinical description of the condition was provided in only 33.3% (5/15)—none with images.

Only 2 sites (13.3%; Everyday Health and WebMD) met the accepted 7th-grade reading level for online patient education material; those sites scored lower on quality (9 of 17 and 6 of 17, respectively) than sites with higher overall scores.

 

None of the 15 websites studied, therefore, demonstrated optimal features on combined measures of accountability, quality, readability, display, support, and transparency regarding HQ. Notably, the American Academy of Dermatology website (www.aad.org) was not among the 15 websites studied; the AAD website mentions HQ in a section on melasma, but only minimal detail is provided.

Limitations of this study include the small number of websites analyzed and possible selection bias because only 3 internet search engines were used to identify websites for study and analysis.

Previously, we analyzed content about HQ on the video-sharing and social media platform YouTube.⁴ The most viewed YouTube videos on HQ had poor-quality information (ie, only 20% mentioned ochronosis and only 28.6% recommended sunscreen [N=70]). However, average reading level of these videos was 7th grade.^4,5 Therefore, YouTube HQ content, though comprehensible, generally is of poor quality.

By conducting a search for website content about HQ, we found that the most popular URLs had either accurate information with poor readability or lower-quality educational material that was more comprehensible. We conclude that there is a need to develop online patient education materials on HQ that are characterized by high-quality, up-to-date medical information; have been written by board-certified dermatologists; are comprehensible (ie, no more than approximately 1200 words and written at a 7th-grade reading level); and contain relevant clinical images and references. We encourage dermatologists to recognize the limitations of online patient education resources on HQ and educate patients on the proper use of the drug as well as its potential adverse effects

Similar to other dermatologic conditions, barriers to early care in patients with vitiligo can exacerbate health disparities.¹ Delayed treatment of vitiligo is known to hamper successful disease stabilization and repigmentation, as therapies tend to work more effectively in early stages of the disease.²

To investigate the factors associated with treatment delays for patients with vitiligo, we conducted a retrospective chart review of 102 consecutive patients with vitiligo attending an academic outpatient clinic in Austin, Texas, over 36 months.

Methods

Our sample included 102 consecutive patients with vitiligo who attended an academic outpatient clinic in Austin, Texas, from January 2017 to January 2020. Demographic information, clinical characteristics of vitiligo, and treatment data were self-reported via a standardized questionnaire given to all patients with vitiligo and gathered from medical chart review. Patient characteristics are outlined in the Table. The delay to treatment was the time (in months) from the date the patient first noticed the lesion to the start date of first treatment. This retrospective chart review was reviewed by the University of Texas at Austin institutional review board and was determined to be exempt.

Statistical Analysis—The data were analyzed descriptively with a Wilcoxon rank sum test (type I error rate of .05).

Results

Of the 102 charts that were analyzed, 45 were females and 57 were males. More than half of the patients (54.9% [56/102]) were White. Sixteen were Asian, 13 were Hispanic non-White, 11 were Black/African American, and 4 were American Indian/Alaska Native. The median age of disease onset was 21 years, minimum age was 1 year, and maximum age was 83 years. The diagnosis of vitiligo was made by a dermatologist for 72 patients and by a physician of another specialty for 20 patients. Ten patients did not declare the specialty of the diagnosing physician.

Individuals older than 21 years when their disease started had a shorter delay to treatment than individuals who noticed their first lesion at an age younger than 21 years (median, 75 months vs 13 months; P<.01). Individuals diagnosed by a dermatologist had a shorter delay to treatment than individuals diagnosed by a physician of another specialty (median, 13 months vs 58 months; P<.05). White individuals had a shorter delay to treatment than individuals with skin of color (median, 13 months vs 31 months; P=.08), though this trend did not reach statistical significance. Individuals with 1% to 25% of body surface area (BSA) affected at time of presentation to clinic also had a shorter delay to treatment than those with a greater BSA affected (median, 13 months vs 74 months; P<.06), though this trend did not reach statistical significance. Type of vitiligo (P<.8), Fitzpatrick skin type (P<.6), and smoking status (P<.7) were not associated with differential delays.

Comment

Impact of Age on Vitiligo Treatment—Our data suggest that individuals who develop vitiligo at a younger age experience longer treatment delays compared to older individuals. Reasons for this are uncertain but could include access issues, medical decision-making agency, and younger patients not remembering being treated during their youth. Our data also could be influenced by some of the adult patients in our study first noticing their lesions many years ago when treatments for vitiligo were more limited. Nevertheless, detrimental effects on quality of life in children and adolescents with vitiligo suggest that motivating younger individuals with vitiligo to seek treatment or proactively making them aware of treatment opportunities may be beneficial.³

 

Diagnosis of Vitiligo by Nondermatologists—The increase in delay to treatment when a nondermatologist diagnoses vitiligo suggests that prompt initiation of treatment or referrals to dermatology by primary care providers may not routinely be occurring.⁴ Our data indicate the need to educate primary care providers on treatment opportunities for individuals with vitiligo and that early treatment generally is more effective.⁵

Impact of Race/Ethnicity on Vitiligo Treatment—Our data also show trends for longer treatment delays for individuals with skin of color. Although this did not reach statistical significance, we hope future studies will investigate this issue, especially because patients with skin of color experience more stigmatization and quality-of-life impacts by vitiligo than White patients.⁵

Impact of BSA on Vitiligo Treatment—Our data show that patients with a smaller BSA had a shorter delay to treatment than those with a greater BSA affected. This was a unique finding given it initially was hypothesized that patients with greater BSA would seek treatment earlier because of the associated increase in quality of life impact. This trend was not statistically significant, but further investigation would be helpful to analyze the reason behind treatment delays in patients with greater BSA affected.

Conclusion

The delay to treatment in our study population was correlated with the diagnosing physician’s specialty and patient age at disease onset, with trends also observed for race and BSA affected. These findings emphasize the need to investigate specific causes of barriers to early care to promote health equity among individuals with vitiligo.

Similar to other dermatologic conditions, barriers to early care in patients with vitiligo can exacerbate health disparities.¹ Delayed treatment of vitiligo is known to hamper successful disease stabilization and repigmentation, as therapies tend to work more effectively in early stages of the disease.²

To investigate the factors associated with treatment delays for patients with vitiligo, we conducted a retrospective chart review of 102 consecutive patients with vitiligo attending an academic outpatient clinic in Austin, Texas, over 36 months.

Methods

Our sample included 102 consecutive patients with vitiligo who attended an academic outpatient clinic in Austin, Texas, from January 2017 to January 2020. Demographic information, clinical characteristics of vitiligo, and treatment data were self-reported via a standardized questionnaire given to all patients with vitiligo and gathered from medical chart review. Patient characteristics are outlined in the Table. The delay to treatment was the time (in months) from the date the patient first noticed the lesion to the start date of first treatment. This retrospective chart review was reviewed by the University of Texas at Austin institutional review board and was determined to be exempt.

Statistical Analysis—The data were analyzed descriptively with a Wilcoxon rank sum test (type I error rate of .05).

Results

Of the 102 charts that were analyzed, 45 were females and 57 were males. More than half of the patients (54.9% [56/102]) were White. Sixteen were Asian, 13 were Hispanic non-White, 11 were Black/African American, and 4 were American Indian/Alaska Native. The median age of disease onset was 21 years, minimum age was 1 year, and maximum age was 83 years. The diagnosis of vitiligo was made by a dermatologist for 72 patients and by a physician of another specialty for 20 patients. Ten patients did not declare the specialty of the diagnosing physician.

Individuals older than 21 years when their disease started had a shorter delay to treatment than individuals who noticed their first lesion at an age younger than 21 years (median, 75 months vs 13 months; P<.01). Individuals diagnosed by a dermatologist had a shorter delay to treatment than individuals diagnosed by a physician of another specialty (median, 13 months vs 58 months; P<.05). White individuals had a shorter delay to treatment than individuals with skin of color (median, 13 months vs 31 months; P=.08), though this trend did not reach statistical significance. Individuals with 1% to 25% of body surface area (BSA) affected at time of presentation to clinic also had a shorter delay to treatment than those with a greater BSA affected (median, 13 months vs 74 months; P<.06), though this trend did not reach statistical significance. Type of vitiligo (P<.8), Fitzpatrick skin type (P<.6), and smoking status (P<.7) were not associated with differential delays.

Comment

Impact of Age on Vitiligo Treatment—Our data suggest that individuals who develop vitiligo at a younger age experience longer treatment delays compared to older individuals. Reasons for this are uncertain but could include access issues, medical decision-making agency, and younger patients not remembering being treated during their youth. Our data also could be influenced by some of the adult patients in our study first noticing their lesions many years ago when treatments for vitiligo were more limited. Nevertheless, detrimental effects on quality of life in children and adolescents with vitiligo suggest that motivating younger individuals with vitiligo to seek treatment or proactively making them aware of treatment opportunities may be beneficial.³

 

Diagnosis of Vitiligo by Nondermatologists—The increase in delay to treatment when a nondermatologist diagnoses vitiligo suggests that prompt initiation of treatment or referrals to dermatology by primary care providers may not routinely be occurring.⁴ Our data indicate the need to educate primary care providers on treatment opportunities for individuals with vitiligo and that early treatment generally is more effective.⁵

Impact of Race/Ethnicity on Vitiligo Treatment—Our data also show trends for longer treatment delays for individuals with skin of color. Although this did not reach statistical significance, we hope future studies will investigate this issue, especially because patients with skin of color experience more stigmatization and quality-of-life impacts by vitiligo than White patients.⁵

Impact of BSA on Vitiligo Treatment—Our data show that patients with a smaller BSA had a shorter delay to treatment than those with a greater BSA affected. This was a unique finding given it initially was hypothesized that patients with greater BSA would seek treatment earlier because of the associated increase in quality of life impact. This trend was not statistically significant, but further investigation would be helpful to analyze the reason behind treatment delays in patients with greater BSA affected.

Conclusion

The delay to treatment in our study population was correlated with the diagnosing physician’s specialty and patient age at disease onset, with trends also observed for race and BSA affected. These findings emphasize the need to investigate specific causes of barriers to early care to promote health equity among individuals with vitiligo.

Similar to other dermatologic conditions, barriers to early care in patients with vitiligo can exacerbate health disparities.¹ Delayed treatment of vitiligo is known to hamper successful disease stabilization and repigmentation, as therapies tend to work more effectively in early stages of the disease.²

To investigate the factors associated with treatment delays for patients with vitiligo, we conducted a retrospective chart review of 102 consecutive patients with vitiligo attending an academic outpatient clinic in Austin, Texas, over 36 months.

Methods

Our sample included 102 consecutive patients with vitiligo who attended an academic outpatient clinic in Austin, Texas, from January 2017 to January 2020. Demographic information, clinical characteristics of vitiligo, and treatment data were self-reported via a standardized questionnaire given to all patients with vitiligo and gathered from medical chart review. Patient characteristics are outlined in the Table. The delay to treatment was the time (in months) from the date the patient first noticed the lesion to the start date of first treatment. This retrospective chart review was reviewed by the University of Texas at Austin institutional review board and was determined to be exempt.

Statistical Analysis—The data were analyzed descriptively with a Wilcoxon rank sum test (type I error rate of .05).

Results

Of the 102 charts that were analyzed, 45 were females and 57 were males. More than half of the patients (54.9% [56/102]) were White. Sixteen were Asian, 13 were Hispanic non-White, 11 were Black/African American, and 4 were American Indian/Alaska Native. The median age of disease onset was 21 years, minimum age was 1 year, and maximum age was 83 years. The diagnosis of vitiligo was made by a dermatologist for 72 patients and by a physician of another specialty for 20 patients. Ten patients did not declare the specialty of the diagnosing physician.

Individuals older than 21 years when their disease started had a shorter delay to treatment than individuals who noticed their first lesion at an age younger than 21 years (median, 75 months vs 13 months; P<.01). Individuals diagnosed by a dermatologist had a shorter delay to treatment than individuals diagnosed by a physician of another specialty (median, 13 months vs 58 months; P<.05). White individuals had a shorter delay to treatment than individuals with skin of color (median, 13 months vs 31 months; P=.08), though this trend did not reach statistical significance. Individuals with 1% to 25% of body surface area (BSA) affected at time of presentation to clinic also had a shorter delay to treatment than those with a greater BSA affected (median, 13 months vs 74 months; P<.06), though this trend did not reach statistical significance. Type of vitiligo (P<.8), Fitzpatrick skin type (P<.6), and smoking status (P<.7) were not associated with differential delays.

Comment

Impact of Age on Vitiligo Treatment—Our data suggest that individuals who develop vitiligo at a younger age experience longer treatment delays compared to older individuals. Reasons for this are uncertain but could include access issues, medical decision-making agency, and younger patients not remembering being treated during their youth. Our data also could be influenced by some of the adult patients in our study first noticing their lesions many years ago when treatments for vitiligo were more limited. Nevertheless, detrimental effects on quality of life in children and adolescents with vitiligo suggest that motivating younger individuals with vitiligo to seek treatment or proactively making them aware of treatment opportunities may be beneficial.³

 

Diagnosis of Vitiligo by Nondermatologists—The increase in delay to treatment when a nondermatologist diagnoses vitiligo suggests that prompt initiation of treatment or referrals to dermatology by primary care providers may not routinely be occurring.⁴ Our data indicate the need to educate primary care providers on treatment opportunities for individuals with vitiligo and that early treatment generally is more effective.⁵

Impact of Race/Ethnicity on Vitiligo Treatment—Our data also show trends for longer treatment delays for individuals with skin of color. Although this did not reach statistical significance, we hope future studies will investigate this issue, especially because patients with skin of color experience more stigmatization and quality-of-life impacts by vitiligo than White patients.⁵

Impact of BSA on Vitiligo Treatment—Our data show that patients with a smaller BSA had a shorter delay to treatment than those with a greater BSA affected. This was a unique finding given it initially was hypothesized that patients with greater BSA would seek treatment earlier because of the associated increase in quality of life impact. This trend was not statistically significant, but further investigation would be helpful to analyze the reason behind treatment delays in patients with greater BSA affected.

Conclusion

The delay to treatment in our study population was correlated with the diagnosing physician’s specialty and patient age at disease onset, with trends also observed for race and BSA affected. These findings emphasize the need to investigate specific causes of barriers to early care to promote health equity among individuals with vitiligo.

From Sharp Memorial Hospital, San Diego, CA (Drs. Poremba, Dehner, Perreiter, Semma, and Mills), Sharp Rees-Stealy Medical Group, San Diego, CA (Dr. Sakoulas), and Collaborative to Halt Antibiotic-Resistant Microbes (CHARM), Department of Pediatrics, University of California San Diego School of Medicine, La Jolla, CA (Dr. Sakoulas).

Abstract

Objective: To compare the costs of hospitalization of patients with moderate-to-severe COVID-19 who received intravenous immunoglobulin (IVIG) with those of patients of similar comorbidity and illness severity who did not.

Design: Analysis 1 was a case-control study of 10 nonventilated, moderately to severely hypoxic patients with COVID-19 who received IVIG (Privigen [CSL Behring]) matched 1:2 with 20 control patients of similar age, body mass index, degree of hypoxemia, and comorbidities. Analysis 2 consisted of patients enrolled in a previously published, randomized, open-label prospective study of 14 patients with COVID-19 receiving standard of care vs 13 patients who received standard of care plus IVIG (Octagam 10% [Octapharma]).

Setting and participants: Patients with COVID-19 with moderate-to-severe hypoxemia hospitalized at a single site located in San Diego, California.

Measurements: Direct cost of hospitalization.

Results: In the first (case-control) population, mean total direct costs, including IVIG, for the treatment group were $21,982 per IVIG-treated case vs $42,431 per case for matched non-IVIG-receiving controls, representing a net cost reduction of $20,449 (48%) per case. For the second (randomized) group, mean total direct costs, including IVIG, for the treatment group were $28,268 per case vs $62,707 per case for untreated controls, representing a net cost reduction of $34,439 (55%) per case. Of the patients who did not receive IVIG, 24% had hospital costs exceeding $80,000; none of the IVIG-treated patients had costs exceeding this amount (P = .016, Fisher exact test).

Conclusion: If allocated early to the appropriate patient type (moderate-to-severe illness without end-organ comorbidities and age <70 years), IVIG can significantly reduce hospital costs in COVID-19 care. More important, in our study it reduced the demand for scarce critical care resources during the COVID-19 pandemic.

Keywords: IVIG, SARS-CoV-2, cost saving, direct hospital costs.

Intravenous immunoglobulin (IVIG) has been available in most hospitals for 4 decades, with broad therapeutic applications in the treatment of Kawasaki disease and a variety of inflammatory, infectious, autoimmune, and viral diseases, via multifactorial mechanisms of immune modulation.¹ Reports of COVID-19−associated multisystem inflammatory syndrome in adults and children have supported the use of IVIG in treatment.^2,3 Previous studies of IVIG treatment for COVID-19 have produced mixed results. Although retrospective studies have largely been positive,^4-8 prospective clinical trials have been mixed, with some favorable results^9-11 and another, more recent study showing no benefit.¹² However, there is still considerable debate regarding whether some subgroups of patients with COVID-19 may benefit from IVIG; the studies that support this argument, however, have been diluted by broad clinical trials that lack granularity among the heterogeneity of patient characteristics and the timing of IVIG administration.^13,14 One study suggests that patients with COVID-19 who may be particularly poised to benefit from IVIG are those who are younger, have fewer comorbidities, and are treated early.⁸

At our institution, we selectively utilized IVIG to treat patients within 48 hours of rapidly increasing oxygen requirements due to COVID-19, targeting those younger than 70 years, with no previous irreversible end-organ damage, no significant comorbidities (renal failure, heart failure, dementia, active cancer malignancies), and no active treatment for cancer. We analyzed the costs of care of these IVIG (Privigen) recipients and compared them to costs for patients with COVID-19 matched by comorbidities, age, and illness severity who did not receive IVIG. To look for consistency, we examined the cost of care of COVID-19 patients who received IVIG (Octagam) as compared to controls from a previously published pilot trial.¹⁰

 

Methods

Setting and Treatment

All patients in this study were hospitalized at a single site located in San Diego, California. Treatment patients in both cohorts received IVIG 0.5 g/kg adjusted for body weight daily for 3 consecutive days.

Patient Cohort #1: Retrospective Case-Control Trial

Intravenous immunoglobulin (Privigen 10%, CSL Behring) was utilized off-label to treat moderately to severely ill non-intensive care unit (ICU) patients with COVID-19 requiring ≥3 L of oxygen by nasal cannula who were not mechanically ventilated but were considered at high risk for respiratory failure. Preset exclusion criteria for off-label use of IVIG in the treatment of COVID-19 were age >70 years, active malignancy, organ transplant recipient, renal failure, heart failure, or dementia. Controls were obtained from a list of all admitted patients with COVID-19, matched to cases 2:1 on the basis of age (±10 years), body mass index (±1), gender, comorbidities present at admission (eg, hypertension, diabetes mellitus, lung disease, or history of tobacco use), and maximum oxygen requirements within the first 48 hours of admission. In situations where more than 2 potential matched controls were identified for a patient, the 2 controls closest in age to the treatment patient were selected. One IVIG patient was excluded because only 1 matched-age control could be found. Pregnant patients who otherwise fulfilled the criteria for IVIG administration were also excluded from this analysis.

Patient Cohort #2: Prospective, Randomized, Open-Label Trial

Use of IVIG (Octagam 10%, Octapharma) in COVID-19 was studied in a previously published, prospective, open-label randomized trial.¹⁰ This pilot trial included 16 IVIG-treated patients and 17 control patients, of which 13 and 14 patients, respectively, had hospital cost data available for analysis.¹⁰ Most notably, COVID-19 patients in this study were required to have ≥4 L of oxygen via nasal cannula to maintain arterial oxygen saturationof ≤96%.

Outcomes

Cost data were independently obtained from our finance team, which provided us with the total direct cost and the total pharmaceutical cost associated with each admission. We also compared total length of stay (LOS) and ICU LOS between treatment arms, as these were presumed to be the major drivers of cost difference.

Statistics

Nonparametric comparisons of medians were performed with the Mann-Whitney U test. Comparison of means was done by Student t test. Categorical data were analyzed by Fisher exact test.

This analysis was initiated as an internal quality assessment. It received approval from the Sharp Healthcare Institutional Review Board (research@sharp.com), and was granted a waiver of subject authorization and consent given the retrospective nature of the study.

 

Results

Case-Control Analysis

A total of 10 hypoxic patients with COVID-19 received Privigen IVIG outside of clinical trial settings. None of the patients was vaccinated against SARS-CoV-2, as hospitalization occurred prior to vaccine availability. In addition, the original SARS-CoV-2 strain was circulating while these patients were hospitalized, preceding subsequent emerging variants. Oxygen requirements within the first 48 hours ranged from 3 L via nasal cannula to requiring bi-level positive pressure airway therapy with 100% oxygen; median age was 56 years and median Charlson comorbidity index was 1. These 10 patients were each matched to 2 control patients hospitalized during a comparable time period and who, based on oxygen requirements, did not receive IVIG. The 20 control patients had a median age of 58.5 years and a Charlson comorbidity index of 1 (Table 1). Rates of comorbidities, such as hypertension, diabetes mellitus, and obesity, were identical in the 2 groups. None of the patients in either group died during the index hospitalization. Fewer control patients received glucocorticoids, which was reflective of lower illness severity/degree of hypoxia in some controls.

Health care utilization in terms of costs and hospital LOS between the 2 groups are shown in Table 2. The mean total direct hospital cost per case, including IVIG and other drug costs, for the 10 IVIG-treated COVID-19 patients was $21,982 vs $42,431 for the matched controls, a reduction of $20,449 (48%) per case (P = .6187) with IVIG. This difference was heavily driven by 4 control patients (20%) with hospital costs >$80,000, marked by need for ICU transfer, mechanical ventilation during admission, and longer hospital stays. This reduction in progression to mechanical ventilation was consistent with our previously published, open-label, randomized prospective IVIG study, the financial assessment of which is reviewed below. While total direct costs were lower in the treatment arm, the mean drug cost for the treatment arm was $3122 greater than the mean drug cost in the control arm (P = .001622), consistent with the high cost of IVIG therapy (Table 2).

LOS information was obtained, as this was thought to be a primary driver of direct costs. The average LOS in the IVIG arm was 8.4 days, and the average LOS in the control arm was 13.6 days (P = NS). The average ICU LOS in the IVIG arm was 0 days, while the average ICU LOS in the control arm was 5.3 days (P = .04). As with the differences in cost, the differences in LOS were primarily driven by the 4 outlier cases in our control arm, who each had a LOS >25 days, as well as an ICU LOS >20 days.

Randomized, Open-Label, Patient Cohort Analysis

Patient characteristics, LOS, and rates of mechanical ventilation for the IVIG and control patients were previously published and showed a reduction in mechanical ventilation and hospital LOS with IVIG treatment.¹⁰ In this group of patients, 1 patient treated with IVIG (6%) and 3 patients not treated with IVIG (18%) died. To determine the consistency of these results from the case-control patients with a set of patients obtained from clinical trial randomization, we examined the health care costs of patients from the prior study.¹⁰ As with the case-control group, patients in this portion of the analysis were hospitalized before vaccines were available and prior to any identified variants.

Comparing the hospital cost of the IVIG-treated patients to the control patients from this trial revealed results similar to the matched case-control analysis discussed earlier. Average total direct cost per case, including IVIG, for the IVIG treatment group was $28,268, vs $62,707 per case for non-IVIG controls. This represented a net cost reduction of $34,439 (55%) per case, very similar to that of the prior cohort.

IVIG Reduces Costly Outlier Cases

The case-control and randomized trial groups, yielding a combined 23 IVIG and 34 control patients, showed a median cost per case of $22,578 (range $10,115-$70,929) and $22,645 (range $4723-$279,797) for the IVIG and control groups, respectively. Cases with a cost >$80,000 were 0/23 (0%) vs 8/34 (24%) in the IVIG and control groups, respectively (P = .016, Fisher exact test).

Improving care while simultaneously keeping care costs below reimbursement payment levels received from third-party payers is paramount to the financial survival of health care systems. IVIG appears to do this by reducing the number of patients with COVID-19 who progress to ICU care. We compared the costs of care of our combined case-control and randomized trial cohorts to published data on average reimbursements hospitals receive for COVID-19 care from Medicaid, Medicare, and private insurance (Figure).¹⁵ IVIG demonstrated a reduction in cases where costs exceed reimbursement. Indeed, a comparison of net revenue per case of the case-control group showed significantly higher revenue for the IVIG group compared to controls ($52,704 vs $34,712, P = .0338, Table 2).

 

Discussion

As reflected in at least 1 other study,¹⁶ our hospital had been successfully utilizing IVIG in the treatment of viral acute respiratory distress syndrome (ARDS) prior to COVID-19. Therefore, we moved quickly to perform a randomized, open-label pilot study of IVIG (Octagam 10%) in COVID-19, and noted significant clinical benefit that might translate into hospital cost savings.¹⁰ Over the course of the pandemic, evidence has accumulated that IVIG may play an important role in COVID-19 therapeutics, as summarized in a recent review.¹⁷ However, despite promising but inconsistent results, the relatively high acquisition costs of IVIG raised questions as to its pharmacoeconomic value, particularly with such a high volume of COVID-19 patients with hypoxia, in light of limited clinical data.

COVID-19 therapeutics data can be categorized into either high-quality trials showing marginal benefit for some agents or low-quality trials showing greater benefit for other agents, with IVIG studies falling into the latter category.¹⁸ This phenomenon may speak to the pathophysiological heterogeneity of the COVID-19 patient population. High-quality trials enrolling broad patient types lack the granularity to capture and single out relevant patient subsets who would derive maximal therapeutic benefit, with those subsets diluted by other patient types for which no benefit is seen. Meanwhile, the more granular low-quality trials are criticized as underpowered and lacking in translatability to practice.

Positive results from our pilot trial allowed the use of IVIG (Privigen) off-label in hospitalized COVID-19 patients restricted to specific criteria. Patients had to be moderately to severely ill, requiring >3 L of oxygen via nasal cannula; show high risk of clinical deterioration based on respiratory rate and decline in respiratory status; and have underlying comorbidities (such as hypertension, obesity, or diabetes mellitus). However, older patients (>age 70 years) and those with underlying comorbidities marked by organ failure (such as heart failure, renal failure, dementia, or receipt of organ transplant) and active malignancy were excluded, as their clinical outcome in COVID-19 may be considered less modifiable by therapeutics, while simultaneously carrying potentially a higher risk of adverse events from IVIG (volume overload, renal failure). These exclusions are reflected in the overall low Charlson comorbidity index (mean of 1) of the patients in the case-control study arm. As anticipated, we found a net cost reduction: $20,449 (48%) per case among the 10 IVIG-treated patients compared to the 20 matched controls.

We then went back to the patients from the randomized prospective trial and compared costs for the 13 of 16 IVIG patients and 14 of 17 of the control patients for whom data were available. Among untreated controls, we found a net cost reduction of $34,439 (55%) per case. The higher costs seen in the randomized patient cohort compared to the latter case-control group may be due to a combination of the fact that the treated patients had slightly higher comorbidity indices than the case-control group (median Charlson comorbidity index of 2 in both groups) and the fact that they were treated earlier in the pandemic (May/June 2020), as opposed to the case-control group patients, who were treated in November/December 2020.

It was notable that the cost savings across both groups were derived largely from the reduction in the approximately 20% to 25% of control patients who went on to critical illness, including mechanical ventilation, extracorporeal membrane oxygenation (ECMO), and prolonged ICU stays. Indeed, 8 of 34 of the control patients—but none of the 23 IVIG-treated patients—generated hospital costs in excess of $80,000, a difference that was statistically significant even for such a small sample size. Therefore, reducing these very costly outlier events translated into net savings across the board.

In addition to lowering costs, reducing progression to critical illness is extremely important during heavy waves of COVID-19, when the sheer volume of patients results in severe strain due to the relative scarcity of ICU beds, mechanical ventilators, and ECMO. Therefore, reducing the need for these resources would have a vital role that cannot be measured economically.

The major limitations of this study include the small sample size and the potential lack of generalizability of these results to all hospital centers and treating providers. Our group has considerable experience in IVIG utilization in COVID-19 and, as a result, has identified a “sweet spot,” where benefits were seen clinically and economically. However, it remains to be determined whether IVIG will benefit patients with greater illness severity, such as those in the ICU, on mechanical ventilation, or ECMO. Furthermore, while a significant morbidity and mortality burden of COVID-19 rests in extremely elderly patients and those with end-organ comorbidities such as renal failure and heart failure, it is uncertain whether their COVID-19 adverse outcomes can be improved with IVIG or other therapies. We believe such patients may limit the pharmacoeconomic value of IVIG due to their generally poorer prognosis, regardless of intervention. On the other hand, COVID-19 patients who are not that severely ill, with minimal to no hypoxia, generally will do well regardless of therapy. Therefore, IVIG intervention may be an unnecessary treatment expense. Evidence for this was suggested in our pilot trial¹⁰ and supported in a recent meta-analysis of IVIG therapy in COVID-19.¹⁹

 

Several other therapeutic options with high acquisition costs have seen an increase in use during the COVID-19 pandemic despite relatively lukewarm data. Remdesivir, the first drug found to have a beneficial effect on hospitalized patients with COVID-19, is priced at $3120 for a complete 5-day treatment course in the United States. This was in line with initial pricing models from the Institute for Clinical and Economic Review (ICER) in May 2020, assuming a mortality benefit with remdesivir use. After the SOLIDARITY trial was published, which showed no mortality benefit associated with remdesivir, ICER updated their pricing models in June 2020 and released a statement that the price of remdesivir was too high to align with demonstrated benefits.^20,21 More recent data demonstrate that remdesivir may be beneficial, but only if administered to patients with fewer than 6 days of symptoms.²² However, only a minority of patients present to the hospital early enough in their illness for remdesivir to be beneficial.²²

Tocilizumab, an interleukin-6 inhibitor, saw an increase in use during the pandemic. An 800-mg treatment course for COVID-19 costs $3584. The efficacy of this treatment option came into question after the COVACTA trial failed to show a difference in clinical status or mortality in COVID-19 patients who received tocilizumab vs placebo.^23,24 A more recent study pointed to a survival benefit of tocilizumab in COVID-19, driven by a very large sample size (>4000), yielding statistically significant, but perhaps clinically less significant, effects on survival.²⁵ This latter study points to the extremely large sample sizes required to capture statistically significant benefits of expensive interventions in COVID-19, which our data demonstrate may benefit only a fraction of patients (20%-25% of patients in the case of IVIG). A more granular clinical assessment of these other interventions is needed to be able to capture the patient subtypes where tocilizumab, remdesivir, and other therapies will be cost effective in the treatment of COVID-19 or other virally mediated cases of ARDS.

 

Conclusion

While IVIG has a high acquisition cost, the drug’s use in hypoxic COVID-19 patients resulted in reduced costs per COVID-19 case of approximately 50% and use of less critical care resources. The difference was consistent between 2 cohorts (randomized trial vs off-label use in prespecified COVID-19 patient types), IVIG products used (Octagam 10% and Privigen), and time period in the pandemic (waves 1 and 2 in May/June 2020 vs wave 3 in November/December 2020), thereby adjusting for potential differences in circulating viral strains. Furthermore, patients from both groups predated SARS-CoV-2 vaccine availability and major circulating viral variants (eg, delta, omicron), thereby eliminating confounding on outcomes posed by these factors. Control patients’ higher costs of care were driven largely by the approximately 25% of patients who required costly hospital critical care resources, a group mitigated by IVIG. When allocated to the appropriate patient type (patients with moderate-to-severe but not critical illness, <age 70 without preexisting comorbidities of end-organ failure or active cancer), IVIG can reduce hospital costs for COVID-19 care. Identification of specific patient populations where IVIG has the most anticipated benefits in viral illness is needed.

Corresponding author: George Sakoulas, MD, Sharp Rees-Stealy Medical Group, 2020 Genesee Avenue, 2nd Floor, San Diego, CA 92123; gsakoulas@health.ucsd.edu

Disclosures: Dr Sakoulas has worked as a consultant for Abbvie, Paratek, and Octapharma, has served as a speaker for Abbvie and Paratek, and has received research funding from Octapharma. The other authors did not report any disclosures.

From Sharp Memorial Hospital, San Diego, CA (Drs. Poremba, Dehner, Perreiter, Semma, and Mills), Sharp Rees-Stealy Medical Group, San Diego, CA (Dr. Sakoulas), and Collaborative to Halt Antibiotic-Resistant Microbes (CHARM), Department of Pediatrics, University of California San Diego School of Medicine, La Jolla, CA (Dr. Sakoulas).

Abstract

Objective: To compare the costs of hospitalization of patients with moderate-to-severe COVID-19 who received intravenous immunoglobulin (IVIG) with those of patients of similar comorbidity and illness severity who did not.

Design: Analysis 1 was a case-control study of 10 nonventilated, moderately to severely hypoxic patients with COVID-19 who received IVIG (Privigen [CSL Behring]) matched 1:2 with 20 control patients of similar age, body mass index, degree of hypoxemia, and comorbidities. Analysis 2 consisted of patients enrolled in a previously published, randomized, open-label prospective study of 14 patients with COVID-19 receiving standard of care vs 13 patients who received standard of care plus IVIG (Octagam 10% [Octapharma]).

Setting and participants: Patients with COVID-19 with moderate-to-severe hypoxemia hospitalized at a single site located in San Diego, California.

Measurements: Direct cost of hospitalization.

Results: In the first (case-control) population, mean total direct costs, including IVIG, for the treatment group were $21,982 per IVIG-treated case vs $42,431 per case for matched non-IVIG-receiving controls, representing a net cost reduction of $20,449 (48%) per case. For the second (randomized) group, mean total direct costs, including IVIG, for the treatment group were $28,268 per case vs $62,707 per case for untreated controls, representing a net cost reduction of $34,439 (55%) per case. Of the patients who did not receive IVIG, 24% had hospital costs exceeding $80,000; none of the IVIG-treated patients had costs exceeding this amount (P = .016, Fisher exact test).

Conclusion: If allocated early to the appropriate patient type (moderate-to-severe illness without end-organ comorbidities and age <70 years), IVIG can significantly reduce hospital costs in COVID-19 care. More important, in our study it reduced the demand for scarce critical care resources during the COVID-19 pandemic.

Keywords: IVIG, SARS-CoV-2, cost saving, direct hospital costs.

Intravenous immunoglobulin (IVIG) has been available in most hospitals for 4 decades, with broad therapeutic applications in the treatment of Kawasaki disease and a variety of inflammatory, infectious, autoimmune, and viral diseases, via multifactorial mechanisms of immune modulation.¹ Reports of COVID-19−associated multisystem inflammatory syndrome in adults and children have supported the use of IVIG in treatment.^2,3 Previous studies of IVIG treatment for COVID-19 have produced mixed results. Although retrospective studies have largely been positive,^4-8 prospective clinical trials have been mixed, with some favorable results^9-11 and another, more recent study showing no benefit.¹² However, there is still considerable debate regarding whether some subgroups of patients with COVID-19 may benefit from IVIG; the studies that support this argument, however, have been diluted by broad clinical trials that lack granularity among the heterogeneity of patient characteristics and the timing of IVIG administration.^13,14 One study suggests that patients with COVID-19 who may be particularly poised to benefit from IVIG are those who are younger, have fewer comorbidities, and are treated early.⁸

At our institution, we selectively utilized IVIG to treat patients within 48 hours of rapidly increasing oxygen requirements due to COVID-19, targeting those younger than 70 years, with no previous irreversible end-organ damage, no significant comorbidities (renal failure, heart failure, dementia, active cancer malignancies), and no active treatment for cancer. We analyzed the costs of care of these IVIG (Privigen) recipients and compared them to costs for patients with COVID-19 matched by comorbidities, age, and illness severity who did not receive IVIG. To look for consistency, we examined the cost of care of COVID-19 patients who received IVIG (Octagam) as compared to controls from a previously published pilot trial.¹⁰

 

Methods

Setting and Treatment

All patients in this study were hospitalized at a single site located in San Diego, California. Treatment patients in both cohorts received IVIG 0.5 g/kg adjusted for body weight daily for 3 consecutive days.

Patient Cohort #1: Retrospective Case-Control Trial

Intravenous immunoglobulin (Privigen 10%, CSL Behring) was utilized off-label to treat moderately to severely ill non-intensive care unit (ICU) patients with COVID-19 requiring ≥3 L of oxygen by nasal cannula who were not mechanically ventilated but were considered at high risk for respiratory failure. Preset exclusion criteria for off-label use of IVIG in the treatment of COVID-19 were age >70 years, active malignancy, organ transplant recipient, renal failure, heart failure, or dementia. Controls were obtained from a list of all admitted patients with COVID-19, matched to cases 2:1 on the basis of age (±10 years), body mass index (±1), gender, comorbidities present at admission (eg, hypertension, diabetes mellitus, lung disease, or history of tobacco use), and maximum oxygen requirements within the first 48 hours of admission. In situations where more than 2 potential matched controls were identified for a patient, the 2 controls closest in age to the treatment patient were selected. One IVIG patient was excluded because only 1 matched-age control could be found. Pregnant patients who otherwise fulfilled the criteria for IVIG administration were also excluded from this analysis.

Patient Cohort #2: Prospective, Randomized, Open-Label Trial

Use of IVIG (Octagam 10%, Octapharma) in COVID-19 was studied in a previously published, prospective, open-label randomized trial.¹⁰ This pilot trial included 16 IVIG-treated patients and 17 control patients, of which 13 and 14 patients, respectively, had hospital cost data available for analysis.¹⁰ Most notably, COVID-19 patients in this study were required to have ≥4 L of oxygen via nasal cannula to maintain arterial oxygen saturationof ≤96%.

Outcomes

Cost data were independently obtained from our finance team, which provided us with the total direct cost and the total pharmaceutical cost associated with each admission. We also compared total length of stay (LOS) and ICU LOS between treatment arms, as these were presumed to be the major drivers of cost difference.

Statistics

Nonparametric comparisons of medians were performed with the Mann-Whitney U test. Comparison of means was done by Student t test. Categorical data were analyzed by Fisher exact test.

This analysis was initiated as an internal quality assessment. It received approval from the Sharp Healthcare Institutional Review Board (research@sharp.com), and was granted a waiver of subject authorization and consent given the retrospective nature of the study.

 

Results

Case-Control Analysis

A total of 10 hypoxic patients with COVID-19 received Privigen IVIG outside of clinical trial settings. None of the patients was vaccinated against SARS-CoV-2, as hospitalization occurred prior to vaccine availability. In addition, the original SARS-CoV-2 strain was circulating while these patients were hospitalized, preceding subsequent emerging variants. Oxygen requirements within the first 48 hours ranged from 3 L via nasal cannula to requiring bi-level positive pressure airway therapy with 100% oxygen; median age was 56 years and median Charlson comorbidity index was 1. These 10 patients were each matched to 2 control patients hospitalized during a comparable time period and who, based on oxygen requirements, did not receive IVIG. The 20 control patients had a median age of 58.5 years and a Charlson comorbidity index of 1 (Table 1). Rates of comorbidities, such as hypertension, diabetes mellitus, and obesity, were identical in the 2 groups. None of the patients in either group died during the index hospitalization. Fewer control patients received glucocorticoids, which was reflective of lower illness severity/degree of hypoxia in some controls.

Health care utilization in terms of costs and hospital LOS between the 2 groups are shown in Table 2. The mean total direct hospital cost per case, including IVIG and other drug costs, for the 10 IVIG-treated COVID-19 patients was $21,982 vs $42,431 for the matched controls, a reduction of $20,449 (48%) per case (P = .6187) with IVIG. This difference was heavily driven by 4 control patients (20%) with hospital costs >$80,000, marked by need for ICU transfer, mechanical ventilation during admission, and longer hospital stays. This reduction in progression to mechanical ventilation was consistent with our previously published, open-label, randomized prospective IVIG study, the financial assessment of which is reviewed below. While total direct costs were lower in the treatment arm, the mean drug cost for the treatment arm was $3122 greater than the mean drug cost in the control arm (P = .001622), consistent with the high cost of IVIG therapy (Table 2).

LOS information was obtained, as this was thought to be a primary driver of direct costs. The average LOS in the IVIG arm was 8.4 days, and the average LOS in the control arm was 13.6 days (P = NS). The average ICU LOS in the IVIG arm was 0 days, while the average ICU LOS in the control arm was 5.3 days (P = .04). As with the differences in cost, the differences in LOS were primarily driven by the 4 outlier cases in our control arm, who each had a LOS >25 days, as well as an ICU LOS >20 days.

Randomized, Open-Label, Patient Cohort Analysis

Patient characteristics, LOS, and rates of mechanical ventilation for the IVIG and control patients were previously published and showed a reduction in mechanical ventilation and hospital LOS with IVIG treatment.¹⁰ In this group of patients, 1 patient treated with IVIG (6%) and 3 patients not treated with IVIG (18%) died. To determine the consistency of these results from the case-control patients with a set of patients obtained from clinical trial randomization, we examined the health care costs of patients from the prior study.¹⁰ As with the case-control group, patients in this portion of the analysis were hospitalized before vaccines were available and prior to any identified variants.

Comparing the hospital cost of the IVIG-treated patients to the control patients from this trial revealed results similar to the matched case-control analysis discussed earlier. Average total direct cost per case, including IVIG, for the IVIG treatment group was $28,268, vs $62,707 per case for non-IVIG controls. This represented a net cost reduction of $34,439 (55%) per case, very similar to that of the prior cohort.

IVIG Reduces Costly Outlier Cases

The case-control and randomized trial groups, yielding a combined 23 IVIG and 34 control patients, showed a median cost per case of $22,578 (range $10,115-$70,929) and $22,645 (range $4723-$279,797) for the IVIG and control groups, respectively. Cases with a cost >$80,000 were 0/23 (0%) vs 8/34 (24%) in the IVIG and control groups, respectively (P = .016, Fisher exact test).

Improving care while simultaneously keeping care costs below reimbursement payment levels received from third-party payers is paramount to the financial survival of health care systems. IVIG appears to do this by reducing the number of patients with COVID-19 who progress to ICU care. We compared the costs of care of our combined case-control and randomized trial cohorts to published data on average reimbursements hospitals receive for COVID-19 care from Medicaid, Medicare, and private insurance (Figure).¹⁵ IVIG demonstrated a reduction in cases where costs exceed reimbursement. Indeed, a comparison of net revenue per case of the case-control group showed significantly higher revenue for the IVIG group compared to controls ($52,704 vs $34,712, P = .0338, Table 2).

 

Discussion

As reflected in at least 1 other study,¹⁶ our hospital had been successfully utilizing IVIG in the treatment of viral acute respiratory distress syndrome (ARDS) prior to COVID-19. Therefore, we moved quickly to perform a randomized, open-label pilot study of IVIG (Octagam 10%) in COVID-19, and noted significant clinical benefit that might translate into hospital cost savings.¹⁰ Over the course of the pandemic, evidence has accumulated that IVIG may play an important role in COVID-19 therapeutics, as summarized in a recent review.¹⁷ However, despite promising but inconsistent results, the relatively high acquisition costs of IVIG raised questions as to its pharmacoeconomic value, particularly with such a high volume of COVID-19 patients with hypoxia, in light of limited clinical data.

COVID-19 therapeutics data can be categorized into either high-quality trials showing marginal benefit for some agents or low-quality trials showing greater benefit for other agents, with IVIG studies falling into the latter category.¹⁸ This phenomenon may speak to the pathophysiological heterogeneity of the COVID-19 patient population. High-quality trials enrolling broad patient types lack the granularity to capture and single out relevant patient subsets who would derive maximal therapeutic benefit, with those subsets diluted by other patient types for which no benefit is seen. Meanwhile, the more granular low-quality trials are criticized as underpowered and lacking in translatability to practice.

Positive results from our pilot trial allowed the use of IVIG (Privigen) off-label in hospitalized COVID-19 patients restricted to specific criteria. Patients had to be moderately to severely ill, requiring >3 L of oxygen via nasal cannula; show high risk of clinical deterioration based on respiratory rate and decline in respiratory status; and have underlying comorbidities (such as hypertension, obesity, or diabetes mellitus). However, older patients (>age 70 years) and those with underlying comorbidities marked by organ failure (such as heart failure, renal failure, dementia, or receipt of organ transplant) and active malignancy were excluded, as their clinical outcome in COVID-19 may be considered less modifiable by therapeutics, while simultaneously carrying potentially a higher risk of adverse events from IVIG (volume overload, renal failure). These exclusions are reflected in the overall low Charlson comorbidity index (mean of 1) of the patients in the case-control study arm. As anticipated, we found a net cost reduction: $20,449 (48%) per case among the 10 IVIG-treated patients compared to the 20 matched controls.

We then went back to the patients from the randomized prospective trial and compared costs for the 13 of 16 IVIG patients and 14 of 17 of the control patients for whom data were available. Among untreated controls, we found a net cost reduction of $34,439 (55%) per case. The higher costs seen in the randomized patient cohort compared to the latter case-control group may be due to a combination of the fact that the treated patients had slightly higher comorbidity indices than the case-control group (median Charlson comorbidity index of 2 in both groups) and the fact that they were treated earlier in the pandemic (May/June 2020), as opposed to the case-control group patients, who were treated in November/December 2020.

It was notable that the cost savings across both groups were derived largely from the reduction in the approximately 20% to 25% of control patients who went on to critical illness, including mechanical ventilation, extracorporeal membrane oxygenation (ECMO), and prolonged ICU stays. Indeed, 8 of 34 of the control patients—but none of the 23 IVIG-treated patients—generated hospital costs in excess of $80,000, a difference that was statistically significant even for such a small sample size. Therefore, reducing these very costly outlier events translated into net savings across the board.

In addition to lowering costs, reducing progression to critical illness is extremely important during heavy waves of COVID-19, when the sheer volume of patients results in severe strain due to the relative scarcity of ICU beds, mechanical ventilators, and ECMO. Therefore, reducing the need for these resources would have a vital role that cannot be measured economically.

The major limitations of this study include the small sample size and the potential lack of generalizability of these results to all hospital centers and treating providers. Our group has considerable experience in IVIG utilization in COVID-19 and, as a result, has identified a “sweet spot,” where benefits were seen clinically and economically. However, it remains to be determined whether IVIG will benefit patients with greater illness severity, such as those in the ICU, on mechanical ventilation, or ECMO. Furthermore, while a significant morbidity and mortality burden of COVID-19 rests in extremely elderly patients and those with end-organ comorbidities such as renal failure and heart failure, it is uncertain whether their COVID-19 adverse outcomes can be improved with IVIG or other therapies. We believe such patients may limit the pharmacoeconomic value of IVIG due to their generally poorer prognosis, regardless of intervention. On the other hand, COVID-19 patients who are not that severely ill, with minimal to no hypoxia, generally will do well regardless of therapy. Therefore, IVIG intervention may be an unnecessary treatment expense. Evidence for this was suggested in our pilot trial¹⁰ and supported in a recent meta-analysis of IVIG therapy in COVID-19.¹⁹

 

Several other therapeutic options with high acquisition costs have seen an increase in use during the COVID-19 pandemic despite relatively lukewarm data. Remdesivir, the first drug found to have a beneficial effect on hospitalized patients with COVID-19, is priced at $3120 for a complete 5-day treatment course in the United States. This was in line with initial pricing models from the Institute for Clinical and Economic Review (ICER) in May 2020, assuming a mortality benefit with remdesivir use. After the SOLIDARITY trial was published, which showed no mortality benefit associated with remdesivir, ICER updated their pricing models in June 2020 and released a statement that the price of remdesivir was too high to align with demonstrated benefits.^20,21 More recent data demonstrate that remdesivir may be beneficial, but only if administered to patients with fewer than 6 days of symptoms.²² However, only a minority of patients present to the hospital early enough in their illness for remdesivir to be beneficial.²²

Tocilizumab, an interleukin-6 inhibitor, saw an increase in use during the pandemic. An 800-mg treatment course for COVID-19 costs $3584. The efficacy of this treatment option came into question after the COVACTA trial failed to show a difference in clinical status or mortality in COVID-19 patients who received tocilizumab vs placebo.^23,24 A more recent study pointed to a survival benefit of tocilizumab in COVID-19, driven by a very large sample size (>4000), yielding statistically significant, but perhaps clinically less significant, effects on survival.²⁵ This latter study points to the extremely large sample sizes required to capture statistically significant benefits of expensive interventions in COVID-19, which our data demonstrate may benefit only a fraction of patients (20%-25% of patients in the case of IVIG). A more granular clinical assessment of these other interventions is needed to be able to capture the patient subtypes where tocilizumab, remdesivir, and other therapies will be cost effective in the treatment of COVID-19 or other virally mediated cases of ARDS.

 

Conclusion

While IVIG has a high acquisition cost, the drug’s use in hypoxic COVID-19 patients resulted in reduced costs per COVID-19 case of approximately 50% and use of less critical care resources. The difference was consistent between 2 cohorts (randomized trial vs off-label use in prespecified COVID-19 patient types), IVIG products used (Octagam 10% and Privigen), and time period in the pandemic (waves 1 and 2 in May/June 2020 vs wave 3 in November/December 2020), thereby adjusting for potential differences in circulating viral strains. Furthermore, patients from both groups predated SARS-CoV-2 vaccine availability and major circulating viral variants (eg, delta, omicron), thereby eliminating confounding on outcomes posed by these factors. Control patients’ higher costs of care were driven largely by the approximately 25% of patients who required costly hospital critical care resources, a group mitigated by IVIG. When allocated to the appropriate patient type (patients with moderate-to-severe but not critical illness, <age 70 without preexisting comorbidities of end-organ failure or active cancer), IVIG can reduce hospital costs for COVID-19 care. Identification of specific patient populations where IVIG has the most anticipated benefits in viral illness is needed.

Corresponding author: George Sakoulas, MD, Sharp Rees-Stealy Medical Group, 2020 Genesee Avenue, 2nd Floor, San Diego, CA 92123; gsakoulas@health.ucsd.edu

Disclosures: Dr Sakoulas has worked as a consultant for Abbvie, Paratek, and Octapharma, has served as a speaker for Abbvie and Paratek, and has received research funding from Octapharma. The other authors did not report any disclosures.

From Sharp Memorial Hospital, San Diego, CA (Drs. Poremba, Dehner, Perreiter, Semma, and Mills), Sharp Rees-Stealy Medical Group, San Diego, CA (Dr. Sakoulas), and Collaborative to Halt Antibiotic-Resistant Microbes (CHARM), Department of Pediatrics, University of California San Diego School of Medicine, La Jolla, CA (Dr. Sakoulas).

Abstract

Objective: To compare the costs of hospitalization of patients with moderate-to-severe COVID-19 who received intravenous immunoglobulin (IVIG) with those of patients of similar comorbidity and illness severity who did not.

Design: Analysis 1 was a case-control study of 10 nonventilated, moderately to severely hypoxic patients with COVID-19 who received IVIG (Privigen [CSL Behring]) matched 1:2 with 20 control patients of similar age, body mass index, degree of hypoxemia, and comorbidities. Analysis 2 consisted of patients enrolled in a previously published, randomized, open-label prospective study of 14 patients with COVID-19 receiving standard of care vs 13 patients who received standard of care plus IVIG (Octagam 10% [Octapharma]).

Setting and participants: Patients with COVID-19 with moderate-to-severe hypoxemia hospitalized at a single site located in San Diego, California.

Measurements: Direct cost of hospitalization.

Results: In the first (case-control) population, mean total direct costs, including IVIG, for the treatment group were $21,982 per IVIG-treated case vs $42,431 per case for matched non-IVIG-receiving controls, representing a net cost reduction of $20,449 (48%) per case. For the second (randomized) group, mean total direct costs, including IVIG, for the treatment group were $28,268 per case vs $62,707 per case for untreated controls, representing a net cost reduction of $34,439 (55%) per case. Of the patients who did not receive IVIG, 24% had hospital costs exceeding $80,000; none of the IVIG-treated patients had costs exceeding this amount (P = .016, Fisher exact test).

Conclusion: If allocated early to the appropriate patient type (moderate-to-severe illness without end-organ comorbidities and age <70 years), IVIG can significantly reduce hospital costs in COVID-19 care. More important, in our study it reduced the demand for scarce critical care resources during the COVID-19 pandemic.

Keywords: IVIG, SARS-CoV-2, cost saving, direct hospital costs.

Intravenous immunoglobulin (IVIG) has been available in most hospitals for 4 decades, with broad therapeutic applications in the treatment of Kawasaki disease and a variety of inflammatory, infectious, autoimmune, and viral diseases, via multifactorial mechanisms of immune modulation.¹ Reports of COVID-19−associated multisystem inflammatory syndrome in adults and children have supported the use of IVIG in treatment.^2,3 Previous studies of IVIG treatment for COVID-19 have produced mixed results. Although retrospective studies have largely been positive,^4-8 prospective clinical trials have been mixed, with some favorable results^9-11 and another, more recent study showing no benefit.¹² However, there is still considerable debate regarding whether some subgroups of patients with COVID-19 may benefit from IVIG; the studies that support this argument, however, have been diluted by broad clinical trials that lack granularity among the heterogeneity of patient characteristics and the timing of IVIG administration.^13,14 One study suggests that patients with COVID-19 who may be particularly poised to benefit from IVIG are those who are younger, have fewer comorbidities, and are treated early.⁸

At our institution, we selectively utilized IVIG to treat patients within 48 hours of rapidly increasing oxygen requirements due to COVID-19, targeting those younger than 70 years, with no previous irreversible end-organ damage, no significant comorbidities (renal failure, heart failure, dementia, active cancer malignancies), and no active treatment for cancer. We analyzed the costs of care of these IVIG (Privigen) recipients and compared them to costs for patients with COVID-19 matched by comorbidities, age, and illness severity who did not receive IVIG. To look for consistency, we examined the cost of care of COVID-19 patients who received IVIG (Octagam) as compared to controls from a previously published pilot trial.¹⁰

 

Methods

Setting and Treatment

All patients in this study were hospitalized at a single site located in San Diego, California. Treatment patients in both cohorts received IVIG 0.5 g/kg adjusted for body weight daily for 3 consecutive days.

Patient Cohort #1: Retrospective Case-Control Trial

Intravenous immunoglobulin (Privigen 10%, CSL Behring) was utilized off-label to treat moderately to severely ill non-intensive care unit (ICU) patients with COVID-19 requiring ≥3 L of oxygen by nasal cannula who were not mechanically ventilated but were considered at high risk for respiratory failure. Preset exclusion criteria for off-label use of IVIG in the treatment of COVID-19 were age >70 years, active malignancy, organ transplant recipient, renal failure, heart failure, or dementia. Controls were obtained from a list of all admitted patients with COVID-19, matched to cases 2:1 on the basis of age (±10 years), body mass index (±1), gender, comorbidities present at admission (eg, hypertension, diabetes mellitus, lung disease, or history of tobacco use), and maximum oxygen requirements within the first 48 hours of admission. In situations where more than 2 potential matched controls were identified for a patient, the 2 controls closest in age to the treatment patient were selected. One IVIG patient was excluded because only 1 matched-age control could be found. Pregnant patients who otherwise fulfilled the criteria for IVIG administration were also excluded from this analysis.

Patient Cohort #2: Prospective, Randomized, Open-Label Trial

Use of IVIG (Octagam 10%, Octapharma) in COVID-19 was studied in a previously published, prospective, open-label randomized trial.¹⁰ This pilot trial included 16 IVIG-treated patients and 17 control patients, of which 13 and 14 patients, respectively, had hospital cost data available for analysis.¹⁰ Most notably, COVID-19 patients in this study were required to have ≥4 L of oxygen via nasal cannula to maintain arterial oxygen saturationof ≤96%.

Outcomes

Cost data were independently obtained from our finance team, which provided us with the total direct cost and the total pharmaceutical cost associated with each admission. We also compared total length of stay (LOS) and ICU LOS between treatment arms, as these were presumed to be the major drivers of cost difference.

Statistics

Nonparametric comparisons of medians were performed with the Mann-Whitney U test. Comparison of means was done by Student t test. Categorical data were analyzed by Fisher exact test.

This analysis was initiated as an internal quality assessment. It received approval from the Sharp Healthcare Institutional Review Board (research@sharp.com), and was granted a waiver of subject authorization and consent given the retrospective nature of the study.

 

Results

Case-Control Analysis

A total of 10 hypoxic patients with COVID-19 received Privigen IVIG outside of clinical trial settings. None of the patients was vaccinated against SARS-CoV-2, as hospitalization occurred prior to vaccine availability. In addition, the original SARS-CoV-2 strain was circulating while these patients were hospitalized, preceding subsequent emerging variants. Oxygen requirements within the first 48 hours ranged from 3 L via nasal cannula to requiring bi-level positive pressure airway therapy with 100% oxygen; median age was 56 years and median Charlson comorbidity index was 1. These 10 patients were each matched to 2 control patients hospitalized during a comparable time period and who, based on oxygen requirements, did not receive IVIG. The 20 control patients had a median age of 58.5 years and a Charlson comorbidity index of 1 (Table 1). Rates of comorbidities, such as hypertension, diabetes mellitus, and obesity, were identical in the 2 groups. None of the patients in either group died during the index hospitalization. Fewer control patients received glucocorticoids, which was reflective of lower illness severity/degree of hypoxia in some controls.

Health care utilization in terms of costs and hospital LOS between the 2 groups are shown in Table 2. The mean total direct hospital cost per case, including IVIG and other drug costs, for the 10 IVIG-treated COVID-19 patients was $21,982 vs $42,431 for the matched controls, a reduction of $20,449 (48%) per case (P = .6187) with IVIG. This difference was heavily driven by 4 control patients (20%) with hospital costs >$80,000, marked by need for ICU transfer, mechanical ventilation during admission, and longer hospital stays. This reduction in progression to mechanical ventilation was consistent with our previously published, open-label, randomized prospective IVIG study, the financial assessment of which is reviewed below. While total direct costs were lower in the treatment arm, the mean drug cost for the treatment arm was $3122 greater than the mean drug cost in the control arm (P = .001622), consistent with the high cost of IVIG therapy (Table 2).

LOS information was obtained, as this was thought to be a primary driver of direct costs. The average LOS in the IVIG arm was 8.4 days, and the average LOS in the control arm was 13.6 days (P = NS). The average ICU LOS in the IVIG arm was 0 days, while the average ICU LOS in the control arm was 5.3 days (P = .04). As with the differences in cost, the differences in LOS were primarily driven by the 4 outlier cases in our control arm, who each had a LOS >25 days, as well as an ICU LOS >20 days.

Randomized, Open-Label, Patient Cohort Analysis

Patient characteristics, LOS, and rates of mechanical ventilation for the IVIG and control patients were previously published and showed a reduction in mechanical ventilation and hospital LOS with IVIG treatment.¹⁰ In this group of patients, 1 patient treated with IVIG (6%) and 3 patients not treated with IVIG (18%) died. To determine the consistency of these results from the case-control patients with a set of patients obtained from clinical trial randomization, we examined the health care costs of patients from the prior study.¹⁰ As with the case-control group, patients in this portion of the analysis were hospitalized before vaccines were available and prior to any identified variants.

Comparing the hospital cost of the IVIG-treated patients to the control patients from this trial revealed results similar to the matched case-control analysis discussed earlier. Average total direct cost per case, including IVIG, for the IVIG treatment group was $28,268, vs $62,707 per case for non-IVIG controls. This represented a net cost reduction of $34,439 (55%) per case, very similar to that of the prior cohort.

IVIG Reduces Costly Outlier Cases

The case-control and randomized trial groups, yielding a combined 23 IVIG and 34 control patients, showed a median cost per case of $22,578 (range $10,115-$70,929) and $22,645 (range $4723-$279,797) for the IVIG and control groups, respectively. Cases with a cost >$80,000 were 0/23 (0%) vs 8/34 (24%) in the IVIG and control groups, respectively (P = .016, Fisher exact test).

Improving care while simultaneously keeping care costs below reimbursement payment levels received from third-party payers is paramount to the financial survival of health care systems. IVIG appears to do this by reducing the number of patients with COVID-19 who progress to ICU care. We compared the costs of care of our combined case-control and randomized trial cohorts to published data on average reimbursements hospitals receive for COVID-19 care from Medicaid, Medicare, and private insurance (Figure).¹⁵ IVIG demonstrated a reduction in cases where costs exceed reimbursement. Indeed, a comparison of net revenue per case of the case-control group showed significantly higher revenue for the IVIG group compared to controls ($52,704 vs $34,712, P = .0338, Table 2).

 

Discussion

As reflected in at least 1 other study,¹⁶ our hospital had been successfully utilizing IVIG in the treatment of viral acute respiratory distress syndrome (ARDS) prior to COVID-19. Therefore, we moved quickly to perform a randomized, open-label pilot study of IVIG (Octagam 10%) in COVID-19, and noted significant clinical benefit that might translate into hospital cost savings.¹⁰ Over the course of the pandemic, evidence has accumulated that IVIG may play an important role in COVID-19 therapeutics, as summarized in a recent review.¹⁷ However, despite promising but inconsistent results, the relatively high acquisition costs of IVIG raised questions as to its pharmacoeconomic value, particularly with such a high volume of COVID-19 patients with hypoxia, in light of limited clinical data.

COVID-19 therapeutics data can be categorized into either high-quality trials showing marginal benefit for some agents or low-quality trials showing greater benefit for other agents, with IVIG studies falling into the latter category.¹⁸ This phenomenon may speak to the pathophysiological heterogeneity of the COVID-19 patient population. High-quality trials enrolling broad patient types lack the granularity to capture and single out relevant patient subsets who would derive maximal therapeutic benefit, with those subsets diluted by other patient types for which no benefit is seen. Meanwhile, the more granular low-quality trials are criticized as underpowered and lacking in translatability to practice.

Positive results from our pilot trial allowed the use of IVIG (Privigen) off-label in hospitalized COVID-19 patients restricted to specific criteria. Patients had to be moderately to severely ill, requiring >3 L of oxygen via nasal cannula; show high risk of clinical deterioration based on respiratory rate and decline in respiratory status; and have underlying comorbidities (such as hypertension, obesity, or diabetes mellitus). However, older patients (>age 70 years) and those with underlying comorbidities marked by organ failure (such as heart failure, renal failure, dementia, or receipt of organ transplant) and active malignancy were excluded, as their clinical outcome in COVID-19 may be considered less modifiable by therapeutics, while simultaneously carrying potentially a higher risk of adverse events from IVIG (volume overload, renal failure). These exclusions are reflected in the overall low Charlson comorbidity index (mean of 1) of the patients in the case-control study arm. As anticipated, we found a net cost reduction: $20,449 (48%) per case among the 10 IVIG-treated patients compared to the 20 matched controls.

We then went back to the patients from the randomized prospective trial and compared costs for the 13 of 16 IVIG patients and 14 of 17 of the control patients for whom data were available. Among untreated controls, we found a net cost reduction of $34,439 (55%) per case. The higher costs seen in the randomized patient cohort compared to the latter case-control group may be due to a combination of the fact that the treated patients had slightly higher comorbidity indices than the case-control group (median Charlson comorbidity index of 2 in both groups) and the fact that they were treated earlier in the pandemic (May/June 2020), as opposed to the case-control group patients, who were treated in November/December 2020.

It was notable that the cost savings across both groups were derived largely from the reduction in the approximately 20% to 25% of control patients who went on to critical illness, including mechanical ventilation, extracorporeal membrane oxygenation (ECMO), and prolonged ICU stays. Indeed, 8 of 34 of the control patients—but none of the 23 IVIG-treated patients—generated hospital costs in excess of $80,000, a difference that was statistically significant even for such a small sample size. Therefore, reducing these very costly outlier events translated into net savings across the board.

In addition to lowering costs, reducing progression to critical illness is extremely important during heavy waves of COVID-19, when the sheer volume of patients results in severe strain due to the relative scarcity of ICU beds, mechanical ventilators, and ECMO. Therefore, reducing the need for these resources would have a vital role that cannot be measured economically.

The major limitations of this study include the small sample size and the potential lack of generalizability of these results to all hospital centers and treating providers. Our group has considerable experience in IVIG utilization in COVID-19 and, as a result, has identified a “sweet spot,” where benefits were seen clinically and economically. However, it remains to be determined whether IVIG will benefit patients with greater illness severity, such as those in the ICU, on mechanical ventilation, or ECMO. Furthermore, while a significant morbidity and mortality burden of COVID-19 rests in extremely elderly patients and those with end-organ comorbidities such as renal failure and heart failure, it is uncertain whether their COVID-19 adverse outcomes can be improved with IVIG or other therapies. We believe such patients may limit the pharmacoeconomic value of IVIG due to their generally poorer prognosis, regardless of intervention. On the other hand, COVID-19 patients who are not that severely ill, with minimal to no hypoxia, generally will do well regardless of therapy. Therefore, IVIG intervention may be an unnecessary treatment expense. Evidence for this was suggested in our pilot trial¹⁰ and supported in a recent meta-analysis of IVIG therapy in COVID-19.¹⁹

 

Several other therapeutic options with high acquisition costs have seen an increase in use during the COVID-19 pandemic despite relatively lukewarm data. Remdesivir, the first drug found to have a beneficial effect on hospitalized patients with COVID-19, is priced at $3120 for a complete 5-day treatment course in the United States. This was in line with initial pricing models from the Institute for Clinical and Economic Review (ICER) in May 2020, assuming a mortality benefit with remdesivir use. After the SOLIDARITY trial was published, which showed no mortality benefit associated with remdesivir, ICER updated their pricing models in June 2020 and released a statement that the price of remdesivir was too high to align with demonstrated benefits.^20,21 More recent data demonstrate that remdesivir may be beneficial, but only if administered to patients with fewer than 6 days of symptoms.²² However, only a minority of patients present to the hospital early enough in their illness for remdesivir to be beneficial.²²

Tocilizumab, an interleukin-6 inhibitor, saw an increase in use during the pandemic. An 800-mg treatment course for COVID-19 costs $3584. The efficacy of this treatment option came into question after the COVACTA trial failed to show a difference in clinical status or mortality in COVID-19 patients who received tocilizumab vs placebo.^23,24 A more recent study pointed to a survival benefit of tocilizumab in COVID-19, driven by a very large sample size (>4000), yielding statistically significant, but perhaps clinically less significant, effects on survival.²⁵ This latter study points to the extremely large sample sizes required to capture statistically significant benefits of expensive interventions in COVID-19, which our data demonstrate may benefit only a fraction of patients (20%-25% of patients in the case of IVIG). A more granular clinical assessment of these other interventions is needed to be able to capture the patient subtypes where tocilizumab, remdesivir, and other therapies will be cost effective in the treatment of COVID-19 or other virally mediated cases of ARDS.

 

Conclusion

While IVIG has a high acquisition cost, the drug’s use in hypoxic COVID-19 patients resulted in reduced costs per COVID-19 case of approximately 50% and use of less critical care resources. The difference was consistent between 2 cohorts (randomized trial vs off-label use in prespecified COVID-19 patient types), IVIG products used (Octagam 10% and Privigen), and time period in the pandemic (waves 1 and 2 in May/June 2020 vs wave 3 in November/December 2020), thereby adjusting for potential differences in circulating viral strains. Furthermore, patients from both groups predated SARS-CoV-2 vaccine availability and major circulating viral variants (eg, delta, omicron), thereby eliminating confounding on outcomes posed by these factors. Control patients’ higher costs of care were driven largely by the approximately 25% of patients who required costly hospital critical care resources, a group mitigated by IVIG. When allocated to the appropriate patient type (patients with moderate-to-severe but not critical illness, <age 70 without preexisting comorbidities of end-organ failure or active cancer), IVIG can reduce hospital costs for COVID-19 care. Identification of specific patient populations where IVIG has the most anticipated benefits in viral illness is needed.

Corresponding author: George Sakoulas, MD, Sharp Rees-Stealy Medical Group, 2020 Genesee Avenue, 2nd Floor, San Diego, CA 92123; gsakoulas@health.ucsd.edu

Disclosures: Dr Sakoulas has worked as a consultant for Abbvie, Paratek, and Octapharma, has served as a speaker for Abbvie and Paratek, and has received research funding from Octapharma. The other authors did not report any disclosures.

To the Editor:

Skin cancer represents a notable health care burden of rising incidence.^1-3 Nondermatologist health care providers play a key role in skin cancer screening through the use of skin cancer examination (SCE)^1,4; however, several factors including poor diagnostic accuracy, low confidence, and lack of training have contributed to limited use of the SCE by these providers.^4,5 Therefore, it is important to identify and implement changes in the medical school curriculum that can facilitate improved use of SCE in clinical practice. We sought to examine factors in the medical school curriculum that influence skin cancer education.

A voluntary electronic survey was distributed through class email and social media to all medical student classes at 4 medical schools (Figure). Responses were collected between March 2 and April 20, 2020. Survey items assessed demographics and curricular factors that influence skin cancer education. Our study was approved by the Institutional Review Board for Human Research of the Medical University of South Carolina (Charleston, South Carolina).

Knowledge of the clinical features of melanoma was assessed by asking participants to correctly identify at least 5 of 6 pigmented lesions as concerning or not concerning for melanoma. Confidence in performing the SCE—the primary outcome—was measured by dichotomizing a 4-point Likert-type scale (“very confident” and “moderately confident” against “slightly confident” and “not at all confident”).

Logistic regression was used to examine curricular factors associated with confidence; descriptive statistics were used for remaining analyses. Analyses were performed using SAS 9.4 statistical software. Prior to analysis, responses from the University of South Carolina School of Medicine Greenville were excluded because the response rate was less than 20%.

The survey was distributed to 1524 students; 619 (40.6%) answered at least 1 question, with a variable response rate to each item (eTable 1). Most respondents were female (351 [56.7%]); 438 (70.8%) were White.

Most respondents said that they received 3 hours or less of general skin cancer (74.9%) or SCE-specific (93.0%) education by the end of their fourth year of medical training. Lecture was the most common method of instruction. Education was provided most often by dermatologists (48.6%), followed by general practice physicians (21.2%). Numerous (26.9%) fourth-year respondents reported that they had never observed SCE; even more (47.6%) had never performed SCE. Almost half of second- and third-year students (43.2% and 44.8%, respectively) considered themselves knowledgeable about the clinical features of melanoma, but only 31.9% of fourth-year students considered themselves knowledgeable.

Only 24.1% of fourth-year students reported confidence performing SCE (eTable 1). Students who received most of their instruction through real clinical encounters were 4.14 times more likely to be confident performing SCE than students who had been given lecture-based learning. Students who performed 1 to 3 SCE or 4 or more SCE were 3.02 and 32.25 times, respectively, more likely to be confident than students who had never performed SCE (eTable 2).

Consistent with a recent study,⁶ our results reflect the discrepancy between the burden and education of skin cancer. This is especially demonstrated by our cohort’s low confidence in performing SCE, a metric associated with both intention to perform and actual performance of SCE in practice.^4,5 We also observed a downward trend in knowledge among students who were about to enter residency, potentially indicating the need for longitudinal training.

Given curricular time constraints, it is essential that medical schools implement changes in learning that will have the greatest impact. Although our results strongly support the efficacy of hands-on clinical training, exposure to dermatology in the second half of medical school training is limited nationwide.⁶ Concentrated efforts to increase clinical exposure might help prepare future physicians in all specialties to combat the burden of this disease.

Limitations of our study include the potential for selection and recall biases. Although our survey spanned multiple institutions in different regions of the United States, results might not be universally representative.

Acknowledgments—We thank Dirk Elston, MD, and Amy Wahlquist, MS (both from Charleston, South Carolina), who helped facilitate the survey on which our research is based. We also acknowledge the assistance of Philip Carmon, MD (Columbia, South Carolina); Julie Flugel (Columbia, South Carolina); Algimantas Simpson, MD (Columbia, South Carolina); Nathan Jasperse, MD (Irvine, California); Jeremy Teruel, MD (Charleston, South Carolina); Alan Snyder, MD, MSCR (Charleston, South Carolina); John Bosland (Charleston, South Carolina); and Daniel Spangler (Greenville, South Carolina).

To the Editor:

Skin cancer represents a notable health care burden of rising incidence.^1-3 Nondermatologist health care providers play a key role in skin cancer screening through the use of skin cancer examination (SCE)^1,4; however, several factors including poor diagnostic accuracy, low confidence, and lack of training have contributed to limited use of the SCE by these providers.^4,5 Therefore, it is important to identify and implement changes in the medical school curriculum that can facilitate improved use of SCE in clinical practice. We sought to examine factors in the medical school curriculum that influence skin cancer education.

A voluntary electronic survey was distributed through class email and social media to all medical student classes at 4 medical schools (Figure). Responses were collected between March 2 and April 20, 2020. Survey items assessed demographics and curricular factors that influence skin cancer education. Our study was approved by the Institutional Review Board for Human Research of the Medical University of South Carolina (Charleston, South Carolina).

Knowledge of the clinical features of melanoma was assessed by asking participants to correctly identify at least 5 of 6 pigmented lesions as concerning or not concerning for melanoma. Confidence in performing the SCE—the primary outcome—was measured by dichotomizing a 4-point Likert-type scale (“very confident” and “moderately confident” against “slightly confident” and “not at all confident”).

Logistic regression was used to examine curricular factors associated with confidence; descriptive statistics were used for remaining analyses. Analyses were performed using SAS 9.4 statistical software. Prior to analysis, responses from the University of South Carolina School of Medicine Greenville were excluded because the response rate was less than 20%.

The survey was distributed to 1524 students; 619 (40.6%) answered at least 1 question, with a variable response rate to each item (eTable 1). Most respondents were female (351 [56.7%]); 438 (70.8%) were White.

Most respondents said that they received 3 hours or less of general skin cancer (74.9%) or SCE-specific (93.0%) education by the end of their fourth year of medical training. Lecture was the most common method of instruction. Education was provided most often by dermatologists (48.6%), followed by general practice physicians (21.2%). Numerous (26.9%) fourth-year respondents reported that they had never observed SCE; even more (47.6%) had never performed SCE. Almost half of second- and third-year students (43.2% and 44.8%, respectively) considered themselves knowledgeable about the clinical features of melanoma, but only 31.9% of fourth-year students considered themselves knowledgeable.

Only 24.1% of fourth-year students reported confidence performing SCE (eTable 1). Students who received most of their instruction through real clinical encounters were 4.14 times more likely to be confident performing SCE than students who had been given lecture-based learning. Students who performed 1 to 3 SCE or 4 or more SCE were 3.02 and 32.25 times, respectively, more likely to be confident than students who had never performed SCE (eTable 2).

Consistent with a recent study,⁶ our results reflect the discrepancy between the burden and education of skin cancer. This is especially demonstrated by our cohort’s low confidence in performing SCE, a metric associated with both intention to perform and actual performance of SCE in practice.^4,5 We also observed a downward trend in knowledge among students who were about to enter residency, potentially indicating the need for longitudinal training.

Given curricular time constraints, it is essential that medical schools implement changes in learning that will have the greatest impact. Although our results strongly support the efficacy of hands-on clinical training, exposure to dermatology in the second half of medical school training is limited nationwide.⁶ Concentrated efforts to increase clinical exposure might help prepare future physicians in all specialties to combat the burden of this disease.

Limitations of our study include the potential for selection and recall biases. Although our survey spanned multiple institutions in different regions of the United States, results might not be universally representative.

Acknowledgments—We thank Dirk Elston, MD, and Amy Wahlquist, MS (both from Charleston, South Carolina), who helped facilitate the survey on which our research is based. We also acknowledge the assistance of Philip Carmon, MD (Columbia, South Carolina); Julie Flugel (Columbia, South Carolina); Algimantas Simpson, MD (Columbia, South Carolina); Nathan Jasperse, MD (Irvine, California); Jeremy Teruel, MD (Charleston, South Carolina); Alan Snyder, MD, MSCR (Charleston, South Carolina); John Bosland (Charleston, South Carolina); and Daniel Spangler (Greenville, South Carolina).

To the Editor:

Skin cancer represents a notable health care burden of rising incidence.^1-3 Nondermatologist health care providers play a key role in skin cancer screening through the use of skin cancer examination (SCE)^1,4; however, several factors including poor diagnostic accuracy, low confidence, and lack of training have contributed to limited use of the SCE by these providers.^4,5 Therefore, it is important to identify and implement changes in the medical school curriculum that can facilitate improved use of SCE in clinical practice. We sought to examine factors in the medical school curriculum that influence skin cancer education.

A voluntary electronic survey was distributed through class email and social media to all medical student classes at 4 medical schools (Figure). Responses were collected between March 2 and April 20, 2020. Survey items assessed demographics and curricular factors that influence skin cancer education. Our study was approved by the Institutional Review Board for Human Research of the Medical University of South Carolina (Charleston, South Carolina).

Knowledge of the clinical features of melanoma was assessed by asking participants to correctly identify at least 5 of 6 pigmented lesions as concerning or not concerning for melanoma. Confidence in performing the SCE—the primary outcome—was measured by dichotomizing a 4-point Likert-type scale (“very confident” and “moderately confident” against “slightly confident” and “not at all confident”).

Logistic regression was used to examine curricular factors associated with confidence; descriptive statistics were used for remaining analyses. Analyses were performed using SAS 9.4 statistical software. Prior to analysis, responses from the University of South Carolina School of Medicine Greenville were excluded because the response rate was less than 20%.

The survey was distributed to 1524 students; 619 (40.6%) answered at least 1 question, with a variable response rate to each item (eTable 1). Most respondents were female (351 [56.7%]); 438 (70.8%) were White.

Most respondents said that they received 3 hours or less of general skin cancer (74.9%) or SCE-specific (93.0%) education by the end of their fourth year of medical training. Lecture was the most common method of instruction. Education was provided most often by dermatologists (48.6%), followed by general practice physicians (21.2%). Numerous (26.9%) fourth-year respondents reported that they had never observed SCE; even more (47.6%) had never performed SCE. Almost half of second- and third-year students (43.2% and 44.8%, respectively) considered themselves knowledgeable about the clinical features of melanoma, but only 31.9% of fourth-year students considered themselves knowledgeable.

Only 24.1% of fourth-year students reported confidence performing SCE (eTable 1). Students who received most of their instruction through real clinical encounters were 4.14 times more likely to be confident performing SCE than students who had been given lecture-based learning. Students who performed 1 to 3 SCE or 4 or more SCE were 3.02 and 32.25 times, respectively, more likely to be confident than students who had never performed SCE (eTable 2).

Consistent with a recent study,⁶ our results reflect the discrepancy between the burden and education of skin cancer. This is especially demonstrated by our cohort’s low confidence in performing SCE, a metric associated with both intention to perform and actual performance of SCE in practice.^4,5 We also observed a downward trend in knowledge among students who were about to enter residency, potentially indicating the need for longitudinal training.

Given curricular time constraints, it is essential that medical schools implement changes in learning that will have the greatest impact. Although our results strongly support the efficacy of hands-on clinical training, exposure to dermatology in the second half of medical school training is limited nationwide.⁶ Concentrated efforts to increase clinical exposure might help prepare future physicians in all specialties to combat the burden of this disease.

Limitations of our study include the potential for selection and recall biases. Although our survey spanned multiple institutions in different regions of the United States, results might not be universally representative.

Acknowledgments—We thank Dirk Elston, MD, and Amy Wahlquist, MS (both from Charleston, South Carolina), who helped facilitate the survey on which our research is based. We also acknowledge the assistance of Philip Carmon, MD (Columbia, South Carolina); Julie Flugel (Columbia, South Carolina); Algimantas Simpson, MD (Columbia, South Carolina); Nathan Jasperse, MD (Irvine, California); Jeremy Teruel, MD (Charleston, South Carolina); Alan Snyder, MD, MSCR (Charleston, South Carolina); John Bosland (Charleston, South Carolina); and Daniel Spangler (Greenville, South Carolina).

Lung cancer is the leading cause of cancer death in the United States.¹ The 2011 National Lung Screening Trial (NLST) demonstrated that low-dose computed tomography (LDCT) screening provided a 20% relative reduction in lung cancer–specific mortality.² Based on these findings, the United States Preventive Services Task Force (USPSTF) published lung cancer screening guidelines in 2013 recommending an annual LDCT of the thorax in patients aged 55 to 80 years with a 30 pack-year smoking history and who currently smoke or quit within the past 15 years. 

In 2021, the USPSTF updated its recommendations by reducing the qualifications for annual screening to a 20 pack-year smoking history.³ The updated guidelines recognized the increased risk of lung cancer for Black individuals.^4,5 Evidence suggests the 2013 screening criteria was too conservative for this population.^6,7

Similarly, US Department of Veteran Affairs (VA) patients are a population at higher risk for lung cancer due to a male predominance, presence of comorbidities, exposure to carcinogenic agents, and possibly a higher prevalence of tobacco smoking.⁸ This study sought to examine the applicability of the USPSTF guidelines in a VA health care system with a predominantly Black population.

Methods

A retrospective chart review of adult patients who were diagnosed and treated with early-stage small cell or non–small cell lung cancer (stage I or II) was performed within the Southeast Louisiana Veterans Health Care System (SLVHCS) in New Orleans. The review used data from the VA Cancer Registry from January 1, 2005, through December 31, 2017. Patients were grouped by whether they met 2013 USPSTF screening criteria at time of diagnosis vs those that did not. Data collected included type and stage of lung cancer at time of diagnosis, context of diagnosis (incidental, screening, symptomatic), diagnostic method, smoking history, and presence of chronic obstructive pulmonary disease (COPD). Patients without a clear smoking history documented in the health record were excluded.

Statistical analyses were performed with GraphPad Prism 8.0. Student t test and Fischer exact test were performed for most of the statistical analyses, with differences between groups noted to be statistically significant at a P < .05.

Results

A total of 182 patient charts were reviewed and 13 patients were excluded for missing information related to the USPSTF screening criteria. Of the 169 patients included, 122 (72%) met USPSTF screening criteria while 47 (28%) patients did not. The reasons for not meeting screening criteria were 14 patients were too young at and 9 patients were too old at time of diagnosis, 7 had a < 20 pack-year smoking history, 7 patients had quit > 15 years previously, and 12 patients met multiple exclusion criteria. The study population was 96% male and there was an overall predominance of Black patients (58%) within the sample (Table).

There was a significantly higher proportion of Black patients in the group that did not meet screening criteria compared with the group that met screening criteria (68% vs 54%, P = .04). Cancer type and stage at diagnosis were similar in both patient populations. There was a statistically significant difference in COPD diagnosis between the groups, with a larger proportion of COPD patients in the met screening criteria group (74% vs 45%, P < .001). The mean smoking history was 61.4 pack-years in the met criteria group and 43.3 pack-years in the did not meet criteria group.  

Five additional patients in the group that did not meet the 2013 USPSTF screening criteria would have met criteria if the 2021 USPSTF guidelines were applied. All 5 were Black patients. Using the 2021 guidelines, Black patients would have made up 56% of the patients who met screening criteria and 54% of the patients who did not meet screening criteria at time of diagnosis.

 

Discussion

This study sought to determine the hypothetical effectiveness of national lung cancer screening guidelines in detecting early-stage lung cancer for a high-risk population. Patients diagnosed with early-stage lung cancer were selected as these patients have improved outcomes with treatment, and thus would theoretically benefit from early detection through screening. As expected, the study population had a majority of Black veterans (58%), with a higher proportion of Black patients in the did not meet screening criteria group compared with the met screening criteria group (68% vs 54%, P = .04). This difference highlights the concern that Black individuals were being underscreened with the 2013 USPSTF guidelines.⁷ This is not all surprising as the NLST, from which the initial screening guidelines were based, included a majority White population with only 4.4% of their population being Black.² The USPSTF also cites the NELSON trial as evidence to support annual lung cancer screening, a trial that was performed in the Netherlands with a very different population compared with that of southeast Louisiana.⁹

Given concern that the old criteria were underscreening certain populations, the updated 2021 USPSTF guidelines sought to expand the screening population. In this study, the implementation of these new guidelines resulted in more Black patients meeting screening criteria.

Racial and ethnic disparities in health care in the US are no secret, as Black individuals consistently have increased disease and death rates, higher rates of unemployment, and decreased access to preventive medical care compared to White individuals.¹⁰ Despite the updated USPSTF guidelines, additional modifications to the screening criteria could improve the ability to identify high-risk patients. A modified model using data from the Prostate, Lung, Colorectal, and Ovarian Screening Trial (PLCO) incorporating COPD history, race and ethnicity, and personal history of cancer increased the sensitivity for high-risk Black ever-smokers.¹¹ Additional models and analyses also support the utility of incorporating race and ethnicity in lung cancer screening criteria.^7,12 Using race and ethnicity to guide screening criteria for cancer is not unheard of; in 2017, the US Multi-Society Task Force recommended that Black individuals start colon cancer screening at age 45 years rather than the typical age of 50 years, before updating the guidelines again in 2021 to recommend that all adults start at age 45 years.^13,14

Limitations

This study had the inherent weakness of being a retrospective study at a single institution. Additionally, the 7th edition of the International Association for the Study of Lung Cancer was published in 2010, during the 2005 to 2017 time frame from which our data was collected, leading to possible inconsistencies in staging between patients before and after 2010.¹⁵ However, these changes in staging are unlikely to significantly impact the results for in this study, since the vast majority of the patients diagnosed with lung cancer stage I or II before 2010 would still be in the those 2 stages in the 2010 edition. Finally, specific to our patient population, it was often difficult to ascertain an accurate smoking history for each patient, especially in the early years of the data set, likely due to the disruption of care caused by Hurricane Katrina.

Conclusions

In this retrospective study performed at the SLVHCS in New Orleans, a significantly higher proportion of Black patients compared with White patients with early-stage lung cancer did not meet the 2013 USPSTF lung cancer screening guidelines at time of diagnosis, highlighting the concern that this population was being underscreened. These findings demonstrate the challenges and failures of applying national guidelines to a unique, high-risk population. An individualized, risk-based screening model incorporating race and ethnicity could be more effective at diagnosing early-stage lung cancer and requires more investigation. Centralized lung cancer screening programs within the VA system could also be beneficial for early detection and treatment, as well as provide insight into the increased risk within the veteran population.

Lung cancer is the leading cause of cancer death in the United States.¹ The 2011 National Lung Screening Trial (NLST) demonstrated that low-dose computed tomography (LDCT) screening provided a 20% relative reduction in lung cancer–specific mortality.² Based on these findings, the United States Preventive Services Task Force (USPSTF) published lung cancer screening guidelines in 2013 recommending an annual LDCT of the thorax in patients aged 55 to 80 years with a 30 pack-year smoking history and who currently smoke or quit within the past 15 years. 

In 2021, the USPSTF updated its recommendations by reducing the qualifications for annual screening to a 20 pack-year smoking history.³ The updated guidelines recognized the increased risk of lung cancer for Black individuals.^4,5 Evidence suggests the 2013 screening criteria was too conservative for this population.^6,7

Similarly, US Department of Veteran Affairs (VA) patients are a population at higher risk for lung cancer due to a male predominance, presence of comorbidities, exposure to carcinogenic agents, and possibly a higher prevalence of tobacco smoking.⁸ This study sought to examine the applicability of the USPSTF guidelines in a VA health care system with a predominantly Black population.

Methods

A retrospective chart review of adult patients who were diagnosed and treated with early-stage small cell or non–small cell lung cancer (stage I or II) was performed within the Southeast Louisiana Veterans Health Care System (SLVHCS) in New Orleans. The review used data from the VA Cancer Registry from January 1, 2005, through December 31, 2017. Patients were grouped by whether they met 2013 USPSTF screening criteria at time of diagnosis vs those that did not. Data collected included type and stage of lung cancer at time of diagnosis, context of diagnosis (incidental, screening, symptomatic), diagnostic method, smoking history, and presence of chronic obstructive pulmonary disease (COPD). Patients without a clear smoking history documented in the health record were excluded.

Statistical analyses were performed with GraphPad Prism 8.0. Student t test and Fischer exact test were performed for most of the statistical analyses, with differences between groups noted to be statistically significant at a P < .05.

Results

A total of 182 patient charts were reviewed and 13 patients were excluded for missing information related to the USPSTF screening criteria. Of the 169 patients included, 122 (72%) met USPSTF screening criteria while 47 (28%) patients did not. The reasons for not meeting screening criteria were 14 patients were too young at and 9 patients were too old at time of diagnosis, 7 had a < 20 pack-year smoking history, 7 patients had quit > 15 years previously, and 12 patients met multiple exclusion criteria. The study population was 96% male and there was an overall predominance of Black patients (58%) within the sample (Table).

There was a significantly higher proportion of Black patients in the group that did not meet screening criteria compared with the group that met screening criteria (68% vs 54%, P = .04). Cancer type and stage at diagnosis were similar in both patient populations. There was a statistically significant difference in COPD diagnosis between the groups, with a larger proportion of COPD patients in the met screening criteria group (74% vs 45%, P < .001). The mean smoking history was 61.4 pack-years in the met criteria group and 43.3 pack-years in the did not meet criteria group.  

Five additional patients in the group that did not meet the 2013 USPSTF screening criteria would have met criteria if the 2021 USPSTF guidelines were applied. All 5 were Black patients. Using the 2021 guidelines, Black patients would have made up 56% of the patients who met screening criteria and 54% of the patients who did not meet screening criteria at time of diagnosis.

 

Discussion

This study sought to determine the hypothetical effectiveness of national lung cancer screening guidelines in detecting early-stage lung cancer for a high-risk population. Patients diagnosed with early-stage lung cancer were selected as these patients have improved outcomes with treatment, and thus would theoretically benefit from early detection through screening. As expected, the study population had a majority of Black veterans (58%), with a higher proportion of Black patients in the did not meet screening criteria group compared with the met screening criteria group (68% vs 54%, P = .04). This difference highlights the concern that Black individuals were being underscreened with the 2013 USPSTF guidelines.⁷ This is not all surprising as the NLST, from which the initial screening guidelines were based, included a majority White population with only 4.4% of their population being Black.² The USPSTF also cites the NELSON trial as evidence to support annual lung cancer screening, a trial that was performed in the Netherlands with a very different population compared with that of southeast Louisiana.⁹

Given concern that the old criteria were underscreening certain populations, the updated 2021 USPSTF guidelines sought to expand the screening population. In this study, the implementation of these new guidelines resulted in more Black patients meeting screening criteria.

Racial and ethnic disparities in health care in the US are no secret, as Black individuals consistently have increased disease and death rates, higher rates of unemployment, and decreased access to preventive medical care compared to White individuals.¹⁰ Despite the updated USPSTF guidelines, additional modifications to the screening criteria could improve the ability to identify high-risk patients. A modified model using data from the Prostate, Lung, Colorectal, and Ovarian Screening Trial (PLCO) incorporating COPD history, race and ethnicity, and personal history of cancer increased the sensitivity for high-risk Black ever-smokers.¹¹ Additional models and analyses also support the utility of incorporating race and ethnicity in lung cancer screening criteria.^7,12 Using race and ethnicity to guide screening criteria for cancer is not unheard of; in 2017, the US Multi-Society Task Force recommended that Black individuals start colon cancer screening at age 45 years rather than the typical age of 50 years, before updating the guidelines again in 2021 to recommend that all adults start at age 45 years.^13,14

Limitations

This study had the inherent weakness of being a retrospective study at a single institution. Additionally, the 7th edition of the International Association for the Study of Lung Cancer was published in 2010, during the 2005 to 2017 time frame from which our data was collected, leading to possible inconsistencies in staging between patients before and after 2010.¹⁵ However, these changes in staging are unlikely to significantly impact the results for in this study, since the vast majority of the patients diagnosed with lung cancer stage I or II before 2010 would still be in the those 2 stages in the 2010 edition. Finally, specific to our patient population, it was often difficult to ascertain an accurate smoking history for each patient, especially in the early years of the data set, likely due to the disruption of care caused by Hurricane Katrina.

Conclusions

In this retrospective study performed at the SLVHCS in New Orleans, a significantly higher proportion of Black patients compared with White patients with early-stage lung cancer did not meet the 2013 USPSTF lung cancer screening guidelines at time of diagnosis, highlighting the concern that this population was being underscreened. These findings demonstrate the challenges and failures of applying national guidelines to a unique, high-risk population. An individualized, risk-based screening model incorporating race and ethnicity could be more effective at diagnosing early-stage lung cancer and requires more investigation. Centralized lung cancer screening programs within the VA system could also be beneficial for early detection and treatment, as well as provide insight into the increased risk within the veteran population.

Lung cancer is the leading cause of cancer death in the United States.¹ The 2011 National Lung Screening Trial (NLST) demonstrated that low-dose computed tomography (LDCT) screening provided a 20% relative reduction in lung cancer–specific mortality.² Based on these findings, the United States Preventive Services Task Force (USPSTF) published lung cancer screening guidelines in 2013 recommending an annual LDCT of the thorax in patients aged 55 to 80 years with a 30 pack-year smoking history and who currently smoke or quit within the past 15 years. 

In 2021, the USPSTF updated its recommendations by reducing the qualifications for annual screening to a 20 pack-year smoking history.³ The updated guidelines recognized the increased risk of lung cancer for Black individuals.^4,5 Evidence suggests the 2013 screening criteria was too conservative for this population.^6,7

Similarly, US Department of Veteran Affairs (VA) patients are a population at higher risk for lung cancer due to a male predominance, presence of comorbidities, exposure to carcinogenic agents, and possibly a higher prevalence of tobacco smoking.⁸ This study sought to examine the applicability of the USPSTF guidelines in a VA health care system with a predominantly Black population.

Methods

A retrospective chart review of adult patients who were diagnosed and treated with early-stage small cell or non–small cell lung cancer (stage I or II) was performed within the Southeast Louisiana Veterans Health Care System (SLVHCS) in New Orleans. The review used data from the VA Cancer Registry from January 1, 2005, through December 31, 2017. Patients were grouped by whether they met 2013 USPSTF screening criteria at time of diagnosis vs those that did not. Data collected included type and stage of lung cancer at time of diagnosis, context of diagnosis (incidental, screening, symptomatic), diagnostic method, smoking history, and presence of chronic obstructive pulmonary disease (COPD). Patients without a clear smoking history documented in the health record were excluded.

Statistical analyses were performed with GraphPad Prism 8.0. Student t test and Fischer exact test were performed for most of the statistical analyses, with differences between groups noted to be statistically significant at a P < .05.

Results

A total of 182 patient charts were reviewed and 13 patients were excluded for missing information related to the USPSTF screening criteria. Of the 169 patients included, 122 (72%) met USPSTF screening criteria while 47 (28%) patients did not. The reasons for not meeting screening criteria were 14 patients were too young at and 9 patients were too old at time of diagnosis, 7 had a < 20 pack-year smoking history, 7 patients had quit > 15 years previously, and 12 patients met multiple exclusion criteria. The study population was 96% male and there was an overall predominance of Black patients (58%) within the sample (Table).

There was a significantly higher proportion of Black patients in the group that did not meet screening criteria compared with the group that met screening criteria (68% vs 54%, P = .04). Cancer type and stage at diagnosis were similar in both patient populations. There was a statistically significant difference in COPD diagnosis between the groups, with a larger proportion of COPD patients in the met screening criteria group (74% vs 45%, P < .001). The mean smoking history was 61.4 pack-years in the met criteria group and 43.3 pack-years in the did not meet criteria group.  

Five additional patients in the group that did not meet the 2013 USPSTF screening criteria would have met criteria if the 2021 USPSTF guidelines were applied. All 5 were Black patients. Using the 2021 guidelines, Black patients would have made up 56% of the patients who met screening criteria and 54% of the patients who did not meet screening criteria at time of diagnosis.

 

Discussion

This study sought to determine the hypothetical effectiveness of national lung cancer screening guidelines in detecting early-stage lung cancer for a high-risk population. Patients diagnosed with early-stage lung cancer were selected as these patients have improved outcomes with treatment, and thus would theoretically benefit from early detection through screening. As expected, the study population had a majority of Black veterans (58%), with a higher proportion of Black patients in the did not meet screening criteria group compared with the met screening criteria group (68% vs 54%, P = .04). This difference highlights the concern that Black individuals were being underscreened with the 2013 USPSTF guidelines.⁷ This is not all surprising as the NLST, from which the initial screening guidelines were based, included a majority White population with only 4.4% of their population being Black.² The USPSTF also cites the NELSON trial as evidence to support annual lung cancer screening, a trial that was performed in the Netherlands with a very different population compared with that of southeast Louisiana.⁹

Given concern that the old criteria were underscreening certain populations, the updated 2021 USPSTF guidelines sought to expand the screening population. In this study, the implementation of these new guidelines resulted in more Black patients meeting screening criteria.

Racial and ethnic disparities in health care in the US are no secret, as Black individuals consistently have increased disease and death rates, higher rates of unemployment, and decreased access to preventive medical care compared to White individuals.¹⁰ Despite the updated USPSTF guidelines, additional modifications to the screening criteria could improve the ability to identify high-risk patients. A modified model using data from the Prostate, Lung, Colorectal, and Ovarian Screening Trial (PLCO) incorporating COPD history, race and ethnicity, and personal history of cancer increased the sensitivity for high-risk Black ever-smokers.¹¹ Additional models and analyses also support the utility of incorporating race and ethnicity in lung cancer screening criteria.^7,12 Using race and ethnicity to guide screening criteria for cancer is not unheard of; in 2017, the US Multi-Society Task Force recommended that Black individuals start colon cancer screening at age 45 years rather than the typical age of 50 years, before updating the guidelines again in 2021 to recommend that all adults start at age 45 years.^13,14

Limitations

This study had the inherent weakness of being a retrospective study at a single institution. Additionally, the 7th edition of the International Association for the Study of Lung Cancer was published in 2010, during the 2005 to 2017 time frame from which our data was collected, leading to possible inconsistencies in staging between patients before and after 2010.¹⁵ However, these changes in staging are unlikely to significantly impact the results for in this study, since the vast majority of the patients diagnosed with lung cancer stage I or II before 2010 would still be in the those 2 stages in the 2010 edition. Finally, specific to our patient population, it was often difficult to ascertain an accurate smoking history for each patient, especially in the early years of the data set, likely due to the disruption of care caused by Hurricane Katrina.

Conclusions

In this retrospective study performed at the SLVHCS in New Orleans, a significantly higher proportion of Black patients compared with White patients with early-stage lung cancer did not meet the 2013 USPSTF lung cancer screening guidelines at time of diagnosis, highlighting the concern that this population was being underscreened. These findings demonstrate the challenges and failures of applying national guidelines to a unique, high-risk population. An individualized, risk-based screening model incorporating race and ethnicity could be more effective at diagnosing early-stage lung cancer and requires more investigation. Centralized lung cancer screening programs within the VA system could also be beneficial for early detection and treatment, as well as provide insight into the increased risk within the veteran population.