Original Research

Oritavancin and dalbavancin are long acting lipoglycopeptides indicated for the treatment of acute bacterial skin and skin structure infections (ABSSSI).^1,2 Largely due to their long half-lives, prolonged tissue concentrations at sites of infection, tolerability, and minimal requirement for therapeutic drug monitoring, these agents are attractive options in outpatient settings.^3,4 A 1- or 2-dose treatment of oritavancin and dalbavancin may be sufficient for conditions traditionally treated with outpatient parenteral antimicrobial therapy (OPAT) via peripherally inserted central catheter (PICC).

Limited research supports the use of dalbavancin and oritavancin for bone and joint infections, infective endocarditis, and bloodstream infections (BSIs). However, the US Food and Drug Administration has approved an indication for the treatment of ABSSSI.^3-9 Dosing for these off-label indications varies but typically consists of an initial intravenous (IV) dose (1000 mg, 1200 mg, or 1500 mg), with a subsequent dose 1 to 2 weeks later or administered once weekly.^6-10

Due in part to the recent availability of oritavancin and dalbavancin relative to the publication of practice guidelines, their appropriate place in therapy continues to evolve based on emerging literature.^11,12 One potential barrier of use for these medications is their cost. Based on the number of doses administered, the 2022 estimated total acquisition cost of therapy for oritavancin and dalbavancin was $1014 to $4397 and $3046 to $7150, respectively (eAppendix). Despite the high acquisition costs, these agents do not require the placement of an indwelling central line, can be administered in outpatient settings, and require minimal therapeutic dose monitoring compared to vancomycin.^13-15 This medication use evaluation (MUE) compared the total cost of treatment with oritavancin and dalbavancin vs therapies traditionally used for OPAT or prolonged IV inpatient therapy.

METHODS

This retrospective MUE was conducted at the Boise Veterans Affairs Medical Center (BVAMC), a level 2 facility with an extensive rural catchment area. BVAMC provides many OPAT services, including medications, supplies, and dressing changes after initial clinic or inpatient education. Contracted vendors may also assist with at-home nursing care using supplies provided by the BVAMC. Cases were identified using an internal database of OPAT patients and those who received oritavancin or dalbavancin between September 1, 2017, and November 1, 2022. Patients aged ≥ 18 years who received ≥ 1 dose of oritavancin or dalbavancin for ABSSSI, osteomyelitis/joint infections, endocarditis, and BSI were included. Comparator treatments consisting of ≥ 1 week of vancomycin or daptomycin for ABSSSI, osteomyelitis/joint infections, endocarditis, and BSI were identified through review of OPAT and Infectious Diseases service consults during the same timeframe. Patients were excluded if any antibiotic was prescribed by a non- VA clinician, if medications were not provided by OPAT, or if chart review did not identify an ABSSSI, osteomyelitis/ joint infection, or BSI diagnosis.

Electronic medical record review was conducted using a standardized data collection form (eAppendix). Data collected included demographics, infectious diagnosis, treatment administered, administration procedures and related visits and treatment locations, outcomes including clinical failure, adverse events (AEs), and hospital readmission.

Clinical failure was defined as readmission or death due to worsening infection or readmission secondary to a documented potential AE to the evaluated antibiotics within 90 days after initiation. Clinical failures excluded readmissions not associated with infection including comorbidities or elective procedures. AEs included new onset renal failure (serum creatinine ≥ 0.5 mg/dL), neutropenia (neutrophils ≤ 500), thrombocytopenia (platelets < 100,000), eosinophilia (> 15% eosinophils), or creatine phosphokinase > 10 times the upper limit of normal, and Clostridioides difficile (C. difficile) infection. Line complications included thrombophlebitis, local inflammation, or infection requiring line replacement (eAppendix).

A cost-minimization approach was used to assess the total cost of treatment.¹⁶ Patients who received oritavancin or dalbavancin were matched with patients that received vancomycin and daptomycin for the same indication and about 1 month of initiation through the randomization function in Microsoft Excel. This accounted for changes in personnel, nonformulary drug approvals, cost, and changes in practice during the pandemic. Costs were calculated using a decision tree as a base model (Figure 1). In this model, each treatment dyad was assessed for the presence or absence of clinical failure, adverse event (medication and line complications), and treatment setting endpoints. Cost estimates were tabulated for each patient that received treatment using published VA data, literature, pharmacoeconomist guidance, or best faith effort based on workflow. ^17-20 All cost estimates were based on 2022 figures or adjusted for inflation if obtained prior to 2022. Secondary endpoints of this analysis included estimated total cost of medication acquisition, administration supplies, laboratory monitoring, and human resources for OPAT visits or receiving home-health services.

This evaluation was classified by the BVAMC Medication Use Evaluation research determination subcommittee as a quality improvement project and was considered exempt from VA Human Subjects Research requirements based on the VA Policy Handbook guideline 1058.05.

RESULTS

The study identified 44 patients who received dalbavancin or oritavancin between September 1, 2017, and October 31, 2022. Thirty-nine patients were included in the analysis: 24 received oritavancin and 15 received dalbavancin and were matched by indication to 10 patients who received vancomycin and 8 patients who received daptomycin. Three patients could not be matched by indication of ABSSSI (Figure 2). Most patients were male, aged > 65 years, and were treated for osteomyelitis (Table 1). No patients were treated for infective endocarditis. A myriad of concomitant antibiotics were used to treat patients and culture results indicated that most infections treated with oritavancin and dalbavancin were polymicrobial.

The mean total cost of therapy per patient receiving oritavancin, dalbavancin, vancomycin, and daptomycin was $35,630, $59,612, $73,333, and $73,708, respectively (Figure 3). When stratified by indication, 27 patients (69%) in the oritavancin/dalbavancin group were treated for osteomyelitis/ joint infections (16 oritavancin, 11 dalbavancin), 9 patients (23%) were treated for BSI (6 oritavancin, 3 dalbavancin), and 3 patients (8%) were treated for ABSSSI (2 oritavancin, 1 dalbavancin). The mean cost per patient for osteomyelitis/joint infections with oritavancin, dalbavancin, vancomycin, and daptomycin was $34,678, $54,224, $87,488, and $85,044, respectively. The mean cost per patient for BSI for oritavancin, dalbavancin, vancomycin, and daptomycin was $35,048, $75,349, $40,305, and $68,068, respectively. The mean cost per patient for ABSSSI for oritavancin and dalbavancin was $44,771 and $71,672.51.

Estimated total drug cost represents the cost of drug acquisition, administration supplies, laboratory monitoring, and human resources for OPAT visits or receiving home health services. The mean cost per patient of drug-related therapy for oritavancin, dalbavancin, vancomycin, and daptomycin was $2203, $5924, $3637, and $7146, respectively (Table 2).

The mean cost per patient for osteomyelitis therapy for oritavancin, dalbavancin, vancomycin, and daptomycin was $2375, $6775, $4164, $8152, respectively. The mean cost of per patient for BSI treatment with oritavancin, dalbavancin, vancomycin, and daptomycin was $1737, $3475, $2409, and $1016, respectively. The mean cost per patient for oritavancin and dalbavancin for ABSSSI treatment, was $1553 and $3910, respectively.

Setting-related costs include expenses from inpatient admissions and postdischarge stays at community living centers (CLCs), skilled nursing facilities (SNFs), or rehabilitation facilities (RFs) for the duration of antimicrobial therapy. The mean setting-related therapy cost for osteomyelitis treatment with oritavancin, dalbavancin, vancomycin, and daptomycin was $27,852, $17,815, $83,324, and $72,856, respectively. The mean setting-related therapy cost per patient for BSI treatment with oritavancin, dalbavancin, vancomycin, and daptomycin was $33,310, $60,668, $37,734, and $67,074, respectively. The mean setting-related therapy cost per patient for ABSSSI treatment for oritavancin and dalbavancin was $43,218 and $67,762.00, respectively.

Six of 39 patients (15%) had clinical failure: 2 patients with oritavancin and 4 patients with dalbavancin. Four patients were readmitted for worsening infection and 2 for AEs. One patient (13%) in the daptomycin group had clinical failure due to readmission for worsening infection. There was no clinical failure with vancomycin. The costs associated with clinical failure per patient for oritavancin, dalbavancin, vancomycin, and daptomycin were $2925, $23,972, $0, and $3601, respectively (Table 3).

Thirty-eight patients (97%) who received oritavancin or dalbavancin had difficulty adhering to vancomycin or daptomycin OPAT. Oritavancin or dalbavancin was used in 10 patients (26%) who lacked support at home and 15 patients (38%) who had either a contraindication or previous failure with other antimicrobials, which were the most common explanations.

DISCUSSION

Long-acting lipoglycopeptides represent a potential alternative to home IV therapy that can avoid prolonged IV access with traditional OPAT. This offers significant advantages, allowing patients to be discharged from the hospital early, especially in rural areas with little OPAT infrastructure or those with logistic challenges. In this analysis, treatment with oritavancin for osteomyelitis, BSI, or ABSSSI, yielded an estimated cost savings of about $37,000 per patient, compared to treatment of matched indications with vancomycin and daptomycin. For every patient treated with dalbavancin for osteomyelitis, BSI, or ABSSSI, the cost savings was about $13,000 per patient, compared to treatment of matched indications for daptomycin and vancomycin. The estimated cost savings per patient for oritavancin was similar to previously published projections ($30,500 to $55,831).¹⁵

Cost savings were primarily driven by setting-related costs. The greatest contrast between the oritavancin and dalbavancin group compared to the vancomycin and daptomycin group was the length of stay in a postdischarge CLC, SNF, or RF setting. This analysis estimated that for every patient treated with oritavancin for osteomyelitis, the setting-related cost savings per patient was about $55,000 compared with vancomycin, and about $45,000 per patient compared with daptomycin. Furthermore, the estimated setting-related cost savings for osteomyelitis treatment with dalbavancin was about $65,000 compared with vancomycin and about $55,000 compared with daptomycin.

Clinical failure occurred with greater frequency in the oritavancin and dalbavancin groups (15%), compared with the vancomycin (0%) and daptomycin (13%) groups. Although the clinical failure rates in patients with osteomyelitis treated with oritavancin and dalbavancin compared with daptomycin were like those in previously published research (10%-30%), the rates of clinical failure for vancomycin in this analysis were lower than those in the oritavancin and dalbavancin group.^8,21,22 The discrepancy in clinical failure rates between this analysis and previous research is likely due to selection bias. Based on the percentages of clinical failure found in the analysis, it is not surprising to note that the total clinical failure-related cost per patient was higher for oritavancin and dalbavancin compared to vancomycin, but similar between oritavancin and daptomycin.

This analysis also found that 15% of patients in the oritavancin and dalbavancin group experienced an AE compared to 10% of patients in the vancomycin group and none in the daptomycin group. In the oritavancin and dalbavancin group, the 2 most common AEs were infusion-related reactions and C. difficile colitis. Although infusion related reactions are easier to correspond to oritavancin and dalbavancin, it becomes difficult to definitively attribute the occurrence of C. difficile to these drugs as many patients were receiving concomitant antibiotics. Although not a primary or secondary objective, the rate of IV-line AEs were more prevalent in the vancomycin (10%), and daptomycin (13%) groups, compared to none in the oritavancin and dalbavancin group. This finding was expected; oritavancin and dalbavancin do not require a central IV line for administration.

Pharmacoeconomic literature continues to emerge with long-acting lipoglycopeptides. A 2024 Italian retrospective single-center analysis of 62 patients reported mean cost reductions > €3200 per patient (> $3400) given dalbavancin compared with the standard of care for ABSSSI or more deep-seeded infections such as osteomyelitis.²³ A 2023 Spanish observational multicenter analysis of 124 patients with infective endocarditis demonstrated high efficacy, safety and cost-effectiveness with dalbavancin vs conventional treatments, with a mean savings of > €5548 per patient (> $6200).²⁴ An analysis of the implementation of a dalbavancin order pathway for ABSSSI to avert inpatient admissions at 11 US emergency departments found a mean cost savings of $5133 per patient and $1211 per hospitalization day avoided, compared with inpatient usual care.²⁵

Conversely, a multicenter, retrospective study of 209 patients in a community-based health care system failed to show a financial benefit for dalbavancin use when compared to standard of care for ABSSSI with higher readmission rates.²⁶ Turco et al also reported increased cost results for 64 patients who received dalbavancin vs standard of care for ABSSSI.²⁷ These discordant findings in ABSSSI studies may be impacted by the authors' patient selection choices and cost assumptions, especially with significantly cheaper oral alternatives. More data are needed to best identify the optimal therapeutic use for the long-acting lipoglycopeptides.

Limitations

The most significant limitation in this analysis was selection bias: 38 of 39 patients (97%) who received dalbavancin or oritavancin had a documented reason that described why OPAT therapy with traditional medications would not be optimal, including logistics, AEs, or clinical failures. Most patients treated with vancomycin and daptomycin were admitted into a SNF, RF, or CLC for the remainder of their treatment, allowing for closer monitoring and care compared to patients treated with oritavancin and dalbavancin, but at a greater cost. For patients sent to a community based SNF or RF, laboratory data were not available unless internally drawn or documented in the electronic medical record.

Additionally, not all cost data were available from VA sources; some were applied from literature, pharmacoeconomist, or best faith effort based on workflow. The cost data from third party contractors providing OPAT services to some BVAMC patients during the time frame of this analysis were not available. Due to its small sample size, outliers had the potential to affect averages reported and accuracy of the cost analysis. Emerging evidence suggests that daptomycin doses higher than the manufacturer-recommended regimen may be required for select indications, a factor that could affect cost, AEs, and efficacy outcomes.²⁸ The acquisition cost of oritavancin and dalbavancin may vary by institution (ie, VA contract prices vs non- VA contract prices) and change over time. A current assessment of cost is needed to best visualize institutional benefit.

Finally, while the patient demographic of this MUE was highly representative of the demographic treated at the BVAMC (males aged >65 years), it may not be applicable to external patient populations. This analysis evaluated off-label indications for these medications. Consequently, this analysis would likely not be applicable to non-VA institution, as third-party payers (eg, insurance) are unlikely to cover medications for off-label indications.

CONCLUSIONS

This study found cost savings associated with the use of oritavancin and dalbavancin compared with vancomycin and daptomycin, particularly for the treatment of osteomyelitis. As safety and efficacy data continues to emerge, the use of long-acting lipoglycopeptides appears to be an increasingly attractive alternative option compared to traditional outpatient antimicrobial therapy, depending on the structure of the program. Larger, multicenter cost-effectiveness studies are needed to further establish the impact of these novel agents.

Oritavancin and dalbavancin are long acting lipoglycopeptides indicated for the treatment of acute bacterial skin and skin structure infections (ABSSSI).^1,2 Largely due to their long half-lives, prolonged tissue concentrations at sites of infection, tolerability, and minimal requirement for therapeutic drug monitoring, these agents are attractive options in outpatient settings.^3,4 A 1- or 2-dose treatment of oritavancin and dalbavancin may be sufficient for conditions traditionally treated with outpatient parenteral antimicrobial therapy (OPAT) via peripherally inserted central catheter (PICC).

Limited research supports the use of dalbavancin and oritavancin for bone and joint infections, infective endocarditis, and bloodstream infections (BSIs). However, the US Food and Drug Administration has approved an indication for the treatment of ABSSSI.^3-9 Dosing for these off-label indications varies but typically consists of an initial intravenous (IV) dose (1000 mg, 1200 mg, or 1500 mg), with a subsequent dose 1 to 2 weeks later or administered once weekly.^6-10

Due in part to the recent availability of oritavancin and dalbavancin relative to the publication of practice guidelines, their appropriate place in therapy continues to evolve based on emerging literature.^11,12 One potential barrier of use for these medications is their cost. Based on the number of doses administered, the 2022 estimated total acquisition cost of therapy for oritavancin and dalbavancin was $1014 to $4397 and $3046 to $7150, respectively (eAppendix). Despite the high acquisition costs, these agents do not require the placement of an indwelling central line, can be administered in outpatient settings, and require minimal therapeutic dose monitoring compared to vancomycin.^13-15 This medication use evaluation (MUE) compared the total cost of treatment with oritavancin and dalbavancin vs therapies traditionally used for OPAT or prolonged IV inpatient therapy.

METHODS

This retrospective MUE was conducted at the Boise Veterans Affairs Medical Center (BVAMC), a level 2 facility with an extensive rural catchment area. BVAMC provides many OPAT services, including medications, supplies, and dressing changes after initial clinic or inpatient education. Contracted vendors may also assist with at-home nursing care using supplies provided by the BVAMC. Cases were identified using an internal database of OPAT patients and those who received oritavancin or dalbavancin between September 1, 2017, and November 1, 2022. Patients aged ≥ 18 years who received ≥ 1 dose of oritavancin or dalbavancin for ABSSSI, osteomyelitis/joint infections, endocarditis, and BSI were included. Comparator treatments consisting of ≥ 1 week of vancomycin or daptomycin for ABSSSI, osteomyelitis/joint infections, endocarditis, and BSI were identified through review of OPAT and Infectious Diseases service consults during the same timeframe. Patients were excluded if any antibiotic was prescribed by a non- VA clinician, if medications were not provided by OPAT, or if chart review did not identify an ABSSSI, osteomyelitis/ joint infection, or BSI diagnosis.

Electronic medical record review was conducted using a standardized data collection form (eAppendix). Data collected included demographics, infectious diagnosis, treatment administered, administration procedures and related visits and treatment locations, outcomes including clinical failure, adverse events (AEs), and hospital readmission.

Clinical failure was defined as readmission or death due to worsening infection or readmission secondary to a documented potential AE to the evaluated antibiotics within 90 days after initiation. Clinical failures excluded readmissions not associated with infection including comorbidities or elective procedures. AEs included new onset renal failure (serum creatinine ≥ 0.5 mg/dL), neutropenia (neutrophils ≤ 500), thrombocytopenia (platelets < 100,000), eosinophilia (> 15% eosinophils), or creatine phosphokinase > 10 times the upper limit of normal, and Clostridioides difficile (C. difficile) infection. Line complications included thrombophlebitis, local inflammation, or infection requiring line replacement (eAppendix).

A cost-minimization approach was used to assess the total cost of treatment.¹⁶ Patients who received oritavancin or dalbavancin were matched with patients that received vancomycin and daptomycin for the same indication and about 1 month of initiation through the randomization function in Microsoft Excel. This accounted for changes in personnel, nonformulary drug approvals, cost, and changes in practice during the pandemic. Costs were calculated using a decision tree as a base model (Figure 1). In this model, each treatment dyad was assessed for the presence or absence of clinical failure, adverse event (medication and line complications), and treatment setting endpoints. Cost estimates were tabulated for each patient that received treatment using published VA data, literature, pharmacoeconomist guidance, or best faith effort based on workflow. ^17-20 All cost estimates were based on 2022 figures or adjusted for inflation if obtained prior to 2022. Secondary endpoints of this analysis included estimated total cost of medication acquisition, administration supplies, laboratory monitoring, and human resources for OPAT visits or receiving home-health services.

This evaluation was classified by the BVAMC Medication Use Evaluation research determination subcommittee as a quality improvement project and was considered exempt from VA Human Subjects Research requirements based on the VA Policy Handbook guideline 1058.05.

RESULTS

The study identified 44 patients who received dalbavancin or oritavancin between September 1, 2017, and October 31, 2022. Thirty-nine patients were included in the analysis: 24 received oritavancin and 15 received dalbavancin and were matched by indication to 10 patients who received vancomycin and 8 patients who received daptomycin. Three patients could not be matched by indication of ABSSSI (Figure 2). Most patients were male, aged > 65 years, and were treated for osteomyelitis (Table 1). No patients were treated for infective endocarditis. A myriad of concomitant antibiotics were used to treat patients and culture results indicated that most infections treated with oritavancin and dalbavancin were polymicrobial.

The mean total cost of therapy per patient receiving oritavancin, dalbavancin, vancomycin, and daptomycin was $35,630, $59,612, $73,333, and $73,708, respectively (Figure 3). When stratified by indication, 27 patients (69%) in the oritavancin/dalbavancin group were treated for osteomyelitis/ joint infections (16 oritavancin, 11 dalbavancin), 9 patients (23%) were treated for BSI (6 oritavancin, 3 dalbavancin), and 3 patients (8%) were treated for ABSSSI (2 oritavancin, 1 dalbavancin). The mean cost per patient for osteomyelitis/joint infections with oritavancin, dalbavancin, vancomycin, and daptomycin was $34,678, $54,224, $87,488, and $85,044, respectively. The mean cost per patient for BSI for oritavancin, dalbavancin, vancomycin, and daptomycin was $35,048, $75,349, $40,305, and $68,068, respectively. The mean cost per patient for ABSSSI for oritavancin and dalbavancin was $44,771 and $71,672.51.

Estimated total drug cost represents the cost of drug acquisition, administration supplies, laboratory monitoring, and human resources for OPAT visits or receiving home health services. The mean cost per patient of drug-related therapy for oritavancin, dalbavancin, vancomycin, and daptomycin was $2203, $5924, $3637, and $7146, respectively (Table 2).

The mean cost per patient for osteomyelitis therapy for oritavancin, dalbavancin, vancomycin, and daptomycin was $2375, $6775, $4164, $8152, respectively. The mean cost of per patient for BSI treatment with oritavancin, dalbavancin, vancomycin, and daptomycin was $1737, $3475, $2409, and $1016, respectively. The mean cost per patient for oritavancin and dalbavancin for ABSSSI treatment, was $1553 and $3910, respectively.

Setting-related costs include expenses from inpatient admissions and postdischarge stays at community living centers (CLCs), skilled nursing facilities (SNFs), or rehabilitation facilities (RFs) for the duration of antimicrobial therapy. The mean setting-related therapy cost for osteomyelitis treatment with oritavancin, dalbavancin, vancomycin, and daptomycin was $27,852, $17,815, $83,324, and $72,856, respectively. The mean setting-related therapy cost per patient for BSI treatment with oritavancin, dalbavancin, vancomycin, and daptomycin was $33,310, $60,668, $37,734, and $67,074, respectively. The mean setting-related therapy cost per patient for ABSSSI treatment for oritavancin and dalbavancin was $43,218 and $67,762.00, respectively.

Six of 39 patients (15%) had clinical failure: 2 patients with oritavancin and 4 patients with dalbavancin. Four patients were readmitted for worsening infection and 2 for AEs. One patient (13%) in the daptomycin group had clinical failure due to readmission for worsening infection. There was no clinical failure with vancomycin. The costs associated with clinical failure per patient for oritavancin, dalbavancin, vancomycin, and daptomycin were $2925, $23,972, $0, and $3601, respectively (Table 3).

Thirty-eight patients (97%) who received oritavancin or dalbavancin had difficulty adhering to vancomycin or daptomycin OPAT. Oritavancin or dalbavancin was used in 10 patients (26%) who lacked support at home and 15 patients (38%) who had either a contraindication or previous failure with other antimicrobials, which were the most common explanations.

DISCUSSION

Long-acting lipoglycopeptides represent a potential alternative to home IV therapy that can avoid prolonged IV access with traditional OPAT. This offers significant advantages, allowing patients to be discharged from the hospital early, especially in rural areas with little OPAT infrastructure or those with logistic challenges. In this analysis, treatment with oritavancin for osteomyelitis, BSI, or ABSSSI, yielded an estimated cost savings of about $37,000 per patient, compared to treatment of matched indications with vancomycin and daptomycin. For every patient treated with dalbavancin for osteomyelitis, BSI, or ABSSSI, the cost savings was about $13,000 per patient, compared to treatment of matched indications for daptomycin and vancomycin. The estimated cost savings per patient for oritavancin was similar to previously published projections ($30,500 to $55,831).¹⁵

Cost savings were primarily driven by setting-related costs. The greatest contrast between the oritavancin and dalbavancin group compared to the vancomycin and daptomycin group was the length of stay in a postdischarge CLC, SNF, or RF setting. This analysis estimated that for every patient treated with oritavancin for osteomyelitis, the setting-related cost savings per patient was about $55,000 compared with vancomycin, and about $45,000 per patient compared with daptomycin. Furthermore, the estimated setting-related cost savings for osteomyelitis treatment with dalbavancin was about $65,000 compared with vancomycin and about $55,000 compared with daptomycin.

Clinical failure occurred with greater frequency in the oritavancin and dalbavancin groups (15%), compared with the vancomycin (0%) and daptomycin (13%) groups. Although the clinical failure rates in patients with osteomyelitis treated with oritavancin and dalbavancin compared with daptomycin were like those in previously published research (10%-30%), the rates of clinical failure for vancomycin in this analysis were lower than those in the oritavancin and dalbavancin group.^8,21,22 The discrepancy in clinical failure rates between this analysis and previous research is likely due to selection bias. Based on the percentages of clinical failure found in the analysis, it is not surprising to note that the total clinical failure-related cost per patient was higher for oritavancin and dalbavancin compared to vancomycin, but similar between oritavancin and daptomycin.

This analysis also found that 15% of patients in the oritavancin and dalbavancin group experienced an AE compared to 10% of patients in the vancomycin group and none in the daptomycin group. In the oritavancin and dalbavancin group, the 2 most common AEs were infusion-related reactions and C. difficile colitis. Although infusion related reactions are easier to correspond to oritavancin and dalbavancin, it becomes difficult to definitively attribute the occurrence of C. difficile to these drugs as many patients were receiving concomitant antibiotics. Although not a primary or secondary objective, the rate of IV-line AEs were more prevalent in the vancomycin (10%), and daptomycin (13%) groups, compared to none in the oritavancin and dalbavancin group. This finding was expected; oritavancin and dalbavancin do not require a central IV line for administration.

Pharmacoeconomic literature continues to emerge with long-acting lipoglycopeptides. A 2024 Italian retrospective single-center analysis of 62 patients reported mean cost reductions > €3200 per patient (> $3400) given dalbavancin compared with the standard of care for ABSSSI or more deep-seeded infections such as osteomyelitis.²³ A 2023 Spanish observational multicenter analysis of 124 patients with infective endocarditis demonstrated high efficacy, safety and cost-effectiveness with dalbavancin vs conventional treatments, with a mean savings of > €5548 per patient (> $6200).²⁴ An analysis of the implementation of a dalbavancin order pathway for ABSSSI to avert inpatient admissions at 11 US emergency departments found a mean cost savings of $5133 per patient and $1211 per hospitalization day avoided, compared with inpatient usual care.²⁵

Conversely, a multicenter, retrospective study of 209 patients in a community-based health care system failed to show a financial benefit for dalbavancin use when compared to standard of care for ABSSSI with higher readmission rates.²⁶ Turco et al also reported increased cost results for 64 patients who received dalbavancin vs standard of care for ABSSSI.²⁷ These discordant findings in ABSSSI studies may be impacted by the authors' patient selection choices and cost assumptions, especially with significantly cheaper oral alternatives. More data are needed to best identify the optimal therapeutic use for the long-acting lipoglycopeptides.

Limitations

The most significant limitation in this analysis was selection bias: 38 of 39 patients (97%) who received dalbavancin or oritavancin had a documented reason that described why OPAT therapy with traditional medications would not be optimal, including logistics, AEs, or clinical failures. Most patients treated with vancomycin and daptomycin were admitted into a SNF, RF, or CLC for the remainder of their treatment, allowing for closer monitoring and care compared to patients treated with oritavancin and dalbavancin, but at a greater cost. For patients sent to a community based SNF or RF, laboratory data were not available unless internally drawn or documented in the electronic medical record.

Additionally, not all cost data were available from VA sources; some were applied from literature, pharmacoeconomist, or best faith effort based on workflow. The cost data from third party contractors providing OPAT services to some BVAMC patients during the time frame of this analysis were not available. Due to its small sample size, outliers had the potential to affect averages reported and accuracy of the cost analysis. Emerging evidence suggests that daptomycin doses higher than the manufacturer-recommended regimen may be required for select indications, a factor that could affect cost, AEs, and efficacy outcomes.²⁸ The acquisition cost of oritavancin and dalbavancin may vary by institution (ie, VA contract prices vs non- VA contract prices) and change over time. A current assessment of cost is needed to best visualize institutional benefit.

Finally, while the patient demographic of this MUE was highly representative of the demographic treated at the BVAMC (males aged >65 years), it may not be applicable to external patient populations. This analysis evaluated off-label indications for these medications. Consequently, this analysis would likely not be applicable to non-VA institution, as third-party payers (eg, insurance) are unlikely to cover medications for off-label indications.

CONCLUSIONS

This study found cost savings associated with the use of oritavancin and dalbavancin compared with vancomycin and daptomycin, particularly for the treatment of osteomyelitis. As safety and efficacy data continues to emerge, the use of long-acting lipoglycopeptides appears to be an increasingly attractive alternative option compared to traditional outpatient antimicrobial therapy, depending on the structure of the program. Larger, multicenter cost-effectiveness studies are needed to further establish the impact of these novel agents.

Oritavancin and dalbavancin are long acting lipoglycopeptides indicated for the treatment of acute bacterial skin and skin structure infections (ABSSSI).^1,2 Largely due to their long half-lives, prolonged tissue concentrations at sites of infection, tolerability, and minimal requirement for therapeutic drug monitoring, these agents are attractive options in outpatient settings.^3,4 A 1- or 2-dose treatment of oritavancin and dalbavancin may be sufficient for conditions traditionally treated with outpatient parenteral antimicrobial therapy (OPAT) via peripherally inserted central catheter (PICC).

Limited research supports the use of dalbavancin and oritavancin for bone and joint infections, infective endocarditis, and bloodstream infections (BSIs). However, the US Food and Drug Administration has approved an indication for the treatment of ABSSSI.^3-9 Dosing for these off-label indications varies but typically consists of an initial intravenous (IV) dose (1000 mg, 1200 mg, or 1500 mg), with a subsequent dose 1 to 2 weeks later or administered once weekly.^6-10

Due in part to the recent availability of oritavancin and dalbavancin relative to the publication of practice guidelines, their appropriate place in therapy continues to evolve based on emerging literature.^11,12 One potential barrier of use for these medications is their cost. Based on the number of doses administered, the 2022 estimated total acquisition cost of therapy for oritavancin and dalbavancin was $1014 to $4397 and $3046 to $7150, respectively (eAppendix). Despite the high acquisition costs, these agents do not require the placement of an indwelling central line, can be administered in outpatient settings, and require minimal therapeutic dose monitoring compared to vancomycin.^13-15 This medication use evaluation (MUE) compared the total cost of treatment with oritavancin and dalbavancin vs therapies traditionally used for OPAT or prolonged IV inpatient therapy.

METHODS

This retrospective MUE was conducted at the Boise Veterans Affairs Medical Center (BVAMC), a level 2 facility with an extensive rural catchment area. BVAMC provides many OPAT services, including medications, supplies, and dressing changes after initial clinic or inpatient education. Contracted vendors may also assist with at-home nursing care using supplies provided by the BVAMC. Cases were identified using an internal database of OPAT patients and those who received oritavancin or dalbavancin between September 1, 2017, and November 1, 2022. Patients aged ≥ 18 years who received ≥ 1 dose of oritavancin or dalbavancin for ABSSSI, osteomyelitis/joint infections, endocarditis, and BSI were included. Comparator treatments consisting of ≥ 1 week of vancomycin or daptomycin for ABSSSI, osteomyelitis/joint infections, endocarditis, and BSI were identified through review of OPAT and Infectious Diseases service consults during the same timeframe. Patients were excluded if any antibiotic was prescribed by a non- VA clinician, if medications were not provided by OPAT, or if chart review did not identify an ABSSSI, osteomyelitis/ joint infection, or BSI diagnosis.

Electronic medical record review was conducted using a standardized data collection form (eAppendix). Data collected included demographics, infectious diagnosis, treatment administered, administration procedures and related visits and treatment locations, outcomes including clinical failure, adverse events (AEs), and hospital readmission.

Clinical failure was defined as readmission or death due to worsening infection or readmission secondary to a documented potential AE to the evaluated antibiotics within 90 days after initiation. Clinical failures excluded readmissions not associated with infection including comorbidities or elective procedures. AEs included new onset renal failure (serum creatinine ≥ 0.5 mg/dL), neutropenia (neutrophils ≤ 500), thrombocytopenia (platelets < 100,000), eosinophilia (> 15% eosinophils), or creatine phosphokinase > 10 times the upper limit of normal, and Clostridioides difficile (C. difficile) infection. Line complications included thrombophlebitis, local inflammation, or infection requiring line replacement (eAppendix).

A cost-minimization approach was used to assess the total cost of treatment.¹⁶ Patients who received oritavancin or dalbavancin were matched with patients that received vancomycin and daptomycin for the same indication and about 1 month of initiation through the randomization function in Microsoft Excel. This accounted for changes in personnel, nonformulary drug approvals, cost, and changes in practice during the pandemic. Costs were calculated using a decision tree as a base model (Figure 1). In this model, each treatment dyad was assessed for the presence or absence of clinical failure, adverse event (medication and line complications), and treatment setting endpoints. Cost estimates were tabulated for each patient that received treatment using published VA data, literature, pharmacoeconomist guidance, or best faith effort based on workflow. ^17-20 All cost estimates were based on 2022 figures or adjusted for inflation if obtained prior to 2022. Secondary endpoints of this analysis included estimated total cost of medication acquisition, administration supplies, laboratory monitoring, and human resources for OPAT visits or receiving home-health services.

This evaluation was classified by the BVAMC Medication Use Evaluation research determination subcommittee as a quality improvement project and was considered exempt from VA Human Subjects Research requirements based on the VA Policy Handbook guideline 1058.05.

RESULTS

The study identified 44 patients who received dalbavancin or oritavancin between September 1, 2017, and October 31, 2022. Thirty-nine patients were included in the analysis: 24 received oritavancin and 15 received dalbavancin and were matched by indication to 10 patients who received vancomycin and 8 patients who received daptomycin. Three patients could not be matched by indication of ABSSSI (Figure 2). Most patients were male, aged > 65 years, and were treated for osteomyelitis (Table 1). No patients were treated for infective endocarditis. A myriad of concomitant antibiotics were used to treat patients and culture results indicated that most infections treated with oritavancin and dalbavancin were polymicrobial.

The mean total cost of therapy per patient receiving oritavancin, dalbavancin, vancomycin, and daptomycin was $35,630, $59,612, $73,333, and $73,708, respectively (Figure 3). When stratified by indication, 27 patients (69%) in the oritavancin/dalbavancin group were treated for osteomyelitis/ joint infections (16 oritavancin, 11 dalbavancin), 9 patients (23%) were treated for BSI (6 oritavancin, 3 dalbavancin), and 3 patients (8%) were treated for ABSSSI (2 oritavancin, 1 dalbavancin). The mean cost per patient for osteomyelitis/joint infections with oritavancin, dalbavancin, vancomycin, and daptomycin was $34,678, $54,224, $87,488, and $85,044, respectively. The mean cost per patient for BSI for oritavancin, dalbavancin, vancomycin, and daptomycin was $35,048, $75,349, $40,305, and $68,068, respectively. The mean cost per patient for ABSSSI for oritavancin and dalbavancin was $44,771 and $71,672.51.

Estimated total drug cost represents the cost of drug acquisition, administration supplies, laboratory monitoring, and human resources for OPAT visits or receiving home health services. The mean cost per patient of drug-related therapy for oritavancin, dalbavancin, vancomycin, and daptomycin was $2203, $5924, $3637, and $7146, respectively (Table 2).

The mean cost per patient for osteomyelitis therapy for oritavancin, dalbavancin, vancomycin, and daptomycin was $2375, $6775, $4164, $8152, respectively. The mean cost of per patient for BSI treatment with oritavancin, dalbavancin, vancomycin, and daptomycin was $1737, $3475, $2409, and $1016, respectively. The mean cost per patient for oritavancin and dalbavancin for ABSSSI treatment, was $1553 and $3910, respectively.

Setting-related costs include expenses from inpatient admissions and postdischarge stays at community living centers (CLCs), skilled nursing facilities (SNFs), or rehabilitation facilities (RFs) for the duration of antimicrobial therapy. The mean setting-related therapy cost for osteomyelitis treatment with oritavancin, dalbavancin, vancomycin, and daptomycin was $27,852, $17,815, $83,324, and $72,856, respectively. The mean setting-related therapy cost per patient for BSI treatment with oritavancin, dalbavancin, vancomycin, and daptomycin was $33,310, $60,668, $37,734, and $67,074, respectively. The mean setting-related therapy cost per patient for ABSSSI treatment for oritavancin and dalbavancin was $43,218 and $67,762.00, respectively.

Six of 39 patients (15%) had clinical failure: 2 patients with oritavancin and 4 patients with dalbavancin. Four patients were readmitted for worsening infection and 2 for AEs. One patient (13%) in the daptomycin group had clinical failure due to readmission for worsening infection. There was no clinical failure with vancomycin. The costs associated with clinical failure per patient for oritavancin, dalbavancin, vancomycin, and daptomycin were $2925, $23,972, $0, and $3601, respectively (Table 3).

Thirty-eight patients (97%) who received oritavancin or dalbavancin had difficulty adhering to vancomycin or daptomycin OPAT. Oritavancin or dalbavancin was used in 10 patients (26%) who lacked support at home and 15 patients (38%) who had either a contraindication or previous failure with other antimicrobials, which were the most common explanations.

DISCUSSION

Long-acting lipoglycopeptides represent a potential alternative to home IV therapy that can avoid prolonged IV access with traditional OPAT. This offers significant advantages, allowing patients to be discharged from the hospital early, especially in rural areas with little OPAT infrastructure or those with logistic challenges. In this analysis, treatment with oritavancin for osteomyelitis, BSI, or ABSSSI, yielded an estimated cost savings of about $37,000 per patient, compared to treatment of matched indications with vancomycin and daptomycin. For every patient treated with dalbavancin for osteomyelitis, BSI, or ABSSSI, the cost savings was about $13,000 per patient, compared to treatment of matched indications for daptomycin and vancomycin. The estimated cost savings per patient for oritavancin was similar to previously published projections ($30,500 to $55,831).¹⁵

Cost savings were primarily driven by setting-related costs. The greatest contrast between the oritavancin and dalbavancin group compared to the vancomycin and daptomycin group was the length of stay in a postdischarge CLC, SNF, or RF setting. This analysis estimated that for every patient treated with oritavancin for osteomyelitis, the setting-related cost savings per patient was about $55,000 compared with vancomycin, and about $45,000 per patient compared with daptomycin. Furthermore, the estimated setting-related cost savings for osteomyelitis treatment with dalbavancin was about $65,000 compared with vancomycin and about $55,000 compared with daptomycin.

Clinical failure occurred with greater frequency in the oritavancin and dalbavancin groups (15%), compared with the vancomycin (0%) and daptomycin (13%) groups. Although the clinical failure rates in patients with osteomyelitis treated with oritavancin and dalbavancin compared with daptomycin were like those in previously published research (10%-30%), the rates of clinical failure for vancomycin in this analysis were lower than those in the oritavancin and dalbavancin group.^8,21,22 The discrepancy in clinical failure rates between this analysis and previous research is likely due to selection bias. Based on the percentages of clinical failure found in the analysis, it is not surprising to note that the total clinical failure-related cost per patient was higher for oritavancin and dalbavancin compared to vancomycin, but similar between oritavancin and daptomycin.

This analysis also found that 15% of patients in the oritavancin and dalbavancin group experienced an AE compared to 10% of patients in the vancomycin group and none in the daptomycin group. In the oritavancin and dalbavancin group, the 2 most common AEs were infusion-related reactions and C. difficile colitis. Although infusion related reactions are easier to correspond to oritavancin and dalbavancin, it becomes difficult to definitively attribute the occurrence of C. difficile to these drugs as many patients were receiving concomitant antibiotics. Although not a primary or secondary objective, the rate of IV-line AEs were more prevalent in the vancomycin (10%), and daptomycin (13%) groups, compared to none in the oritavancin and dalbavancin group. This finding was expected; oritavancin and dalbavancin do not require a central IV line for administration.

Pharmacoeconomic literature continues to emerge with long-acting lipoglycopeptides. A 2024 Italian retrospective single-center analysis of 62 patients reported mean cost reductions > €3200 per patient (> $3400) given dalbavancin compared with the standard of care for ABSSSI or more deep-seeded infections such as osteomyelitis.²³ A 2023 Spanish observational multicenter analysis of 124 patients with infective endocarditis demonstrated high efficacy, safety and cost-effectiveness with dalbavancin vs conventional treatments, with a mean savings of > €5548 per patient (> $6200).²⁴ An analysis of the implementation of a dalbavancin order pathway for ABSSSI to avert inpatient admissions at 11 US emergency departments found a mean cost savings of $5133 per patient and $1211 per hospitalization day avoided, compared with inpatient usual care.²⁵

Conversely, a multicenter, retrospective study of 209 patients in a community-based health care system failed to show a financial benefit for dalbavancin use when compared to standard of care for ABSSSI with higher readmission rates.²⁶ Turco et al also reported increased cost results for 64 patients who received dalbavancin vs standard of care for ABSSSI.²⁷ These discordant findings in ABSSSI studies may be impacted by the authors' patient selection choices and cost assumptions, especially with significantly cheaper oral alternatives. More data are needed to best identify the optimal therapeutic use for the long-acting lipoglycopeptides.

Limitations

The most significant limitation in this analysis was selection bias: 38 of 39 patients (97%) who received dalbavancin or oritavancin had a documented reason that described why OPAT therapy with traditional medications would not be optimal, including logistics, AEs, or clinical failures. Most patients treated with vancomycin and daptomycin were admitted into a SNF, RF, or CLC for the remainder of their treatment, allowing for closer monitoring and care compared to patients treated with oritavancin and dalbavancin, but at a greater cost. For patients sent to a community based SNF or RF, laboratory data were not available unless internally drawn or documented in the electronic medical record.

Additionally, not all cost data were available from VA sources; some were applied from literature, pharmacoeconomist, or best faith effort based on workflow. The cost data from third party contractors providing OPAT services to some BVAMC patients during the time frame of this analysis were not available. Due to its small sample size, outliers had the potential to affect averages reported and accuracy of the cost analysis. Emerging evidence suggests that daptomycin doses higher than the manufacturer-recommended regimen may be required for select indications, a factor that could affect cost, AEs, and efficacy outcomes.²⁸ The acquisition cost of oritavancin and dalbavancin may vary by institution (ie, VA contract prices vs non- VA contract prices) and change over time. A current assessment of cost is needed to best visualize institutional benefit.

Finally, while the patient demographic of this MUE was highly representative of the demographic treated at the BVAMC (males aged >65 years), it may not be applicable to external patient populations. This analysis evaluated off-label indications for these medications. Consequently, this analysis would likely not be applicable to non-VA institution, as third-party payers (eg, insurance) are unlikely to cover medications for off-label indications.

CONCLUSIONS

This study found cost savings associated with the use of oritavancin and dalbavancin compared with vancomycin and daptomycin, particularly for the treatment of osteomyelitis. As safety and efficacy data continues to emerge, the use of long-acting lipoglycopeptides appears to be an increasingly attractive alternative option compared to traditional outpatient antimicrobial therapy, depending on the structure of the program. Larger, multicenter cost-effectiveness studies are needed to further establish the impact of these novel agents.

The COVID-19 pandemic changed the education and training experiences of health care students and those set to comprise the future workforce. Apart from general training disruptions or delays due to the pandemic, behavioral health trainees such as psychologists and social workers faced limited opportunities to provide in-person services.^1-5 Trainees also experienced fewer referrals to mental health services from primary care and more disrupted, no-show, or cancelled appointments.^4-6 Behavioral health trainees experienced a limited ability to establish rapport and more difficulty providing effective services because of the limited in-person interaction presented by telehealth.⁶ The pandemic also resulted in feelings of increased isolation and decreased teamwork.^1,7 The virtual or remote setting made it more difficult for trainees to feel as if they were a member of a team or community of behavioral health professionals.^1,7

Behavioral health trainees had to adapt to conducting patient visits and educational didactics through virtual platforms.^1,3-7 Challenges included access or technological problems with online platforms and a lack of telehealth training use.^3,4,6 One study found that while both behavioral health trainees and licensed practitioners reported similar rates of telehealth use for mental health services by early April 2020, trainees had more difficulties implementing telehealth compared with licensed practitioners. This study found that US Department of Veteran Affairs (VA) facilities reported higher use of telehealth in February 2020.⁵

A mission of the VA is to provide education and training to health care professionals through partnerships with affiliated academic institutions. The VA is the largest education and training supplier for health care professions in the US. As many as 50% of psychologists in the US received some training at the VA.⁸ Additionally, more graduate-level social work students are trained at the VA than at any other organization.⁹ The VA is a major contributor to not only its own behavioral health workforce, but that of the entire country.

The VA is also the largest employer of psychologists and social workers in the US.^10,11 The VA Office of Academic Affiliations (OAA) oversees health care profession education and training at all VA facilities. In 2012, OAA began the Mental Health Education Expansion program to increase training for behavioral health professionals, including psychologists and social workers. ¹² The OAA initiative was aligned with VA training and workforce priorities.^8,12 To gauge the effectiveness of VA education and training, OAA encourages VA trainees to complete the Trainee Satisfaction Survey (TSS), which measures trainee satisfaction and the likelihood of a trainee to consider the VA for future employment.

Researchers at the Veterans Emergency Management Evaluation Center sought to understand the impact the COVID-19 pandemic had on behavioral health trainees’ experiences by examining TSS data from before and after the onset of the pandemic. This study expands on prior research among physician residents and fellows which found associations between VA training experiences and the COVID- 19 pandemic. The previous study found declines in trainee satisfaction and a decreased likelihood to consider the VA for future employment.¹³

It is important to understand the effects the pandemic had on the professional development and wellness for both physician and behavioral health professional trainees. Identifying how the pandemic impacted trainee satisfaction may help improve education programs and mitigate the impact of future public health emergencies. This is particularly important due to the shortage of behavioral health professionals in the VA and the US.^12,14

METHODS

This study used TSS data collected from August 2018 to July 2021 from 153 VA facilities. A behavioral health trainee was defined as any psychology or social work trainee who completed 1 rotation at a VA facility. Psychiatric trainees were excluded because as physicians their training programs differ markedly from those for psychology and social work. Excluding psychiatry, psychology and social work comprise the 2 largest mental health care training groups.

This study was reviewed and approved as a quality improvement project by the VA Greater Los Angeles Healthcare System (VAGLAHS) Institutional Review Board, which waived informed consent requirements. The OAA granted access to data using a process open to all VA researchers. At the time of data collection, respondents were assured their anonymity; participation was voluntary.

Measures

Any response provided before February 29, 2020, was defined as the prepandemic period. The pandemic period included any response from April 1, 2020, to July 31, 2021. Responses collected in March 2020 were excluded as it would be unclear from the survey whether the training period occurred before or after the onset of the pandemic.

To measure overall trainee satisfaction with the VA training experience, responses were grouped as satisfied (satisfied/ very satisfied) and dissatisfied (dissatisfied/ very dissatisfied). To measure a trainee’s likelihood to consider the VA for future employment as a result of their training experience, responses were grouped as likely (likely/very likely) and unlikely (unlikely/very unlikely).

Other components of satisfaction were also collected including onboarding, clinical faculty/preceptors, clinical learning environment, physical environment, working environment, and respect felt at work. If a respondent chose very dissatisfied or dissatisfied, they were subsequently asked to specify the reason for their dissatisfaction with an open-ended response. Open-ended responses were not permitted if a respondent indicated a satisfied or very satisfied response.

Statistical Analyses

Stata SE 17 was used for statistical analyses. To test the relationship between the pandemic group and the 2 separate outcome variables, logistic regressions were conducted to measure overall satisfaction and likelihood of future VA employment. Margin commands were used to calculate the difference in the probability of reporting satisfied/very satisfied and likely/very likely for the prepandemic and pandemic groups. The association of the COVID-19 group with each outcome variable was expressed as the difference in the percentage of the outcome between the prepandemic and pandemic groups. Preliminary analyses demonstrated similar effects of the pandemic on psychology and social work trainees; therefore, the groups were combined.

Rapid Coding and Thematic Analyses

Qualitative data were based on open-ended responses from behavioral health trainees when they were asked to specify the cause of dissatisfaction in the aforementioned areas of satisfaction. Methods for qualitative data included rapid coding and thematic content analyses.^15,16 Additional general information regarding the qualitative data analyses is described elsewhere.¹³ A keyword search was completed to identify all open-ended responses related to COVID-19 pandemic causes of dissatisfaction. Keywords included: virus, COVID, corona, pandemic, PPE, N95, mask, social distance, and safety. All open-ended responses were reviewed to ensure keywords were appropriately identifying pandemic-related causes of dissatisfaction and did not overlook other references to the pandemic, and to identify initial themes and corresponding definitions based on survey questions. After review, additional keywords were included in the content analyses that were related to providing mental health services using remote or telehealth options. This included the following keywords: remote, video, VVC (VA Video Connect), and tele. The research team completed a review of the initial themes and definitions and created a final coding construct with definitions before completing an independent coding of all identified pandemic-related responses. Frequency counts of each code were provided to identify which pandemic-related causes of dissatisfaction were mentioned most.

RESULTS

A total of 3950 behavioral health trainees responded to the TSS, including 2715 psychology trainees and 1235 social work trainees who indicated they received training at the VA in academic years 2018/2019, 2019/2020, or 2020/2021. The academic year 2018/2019 was considered in an effort to provide a larger sample of prepandemic trainees.

The percentage of trainees reporting satisfaction with their training decreased across prepandemic to pandemic groups. In the pandemic group, 2166 of 2324 respondents (93.2%) reported satisfaction compared to 1474 of 1555 (94.8%) in the prepandemic trainee group (P = .04; 95% CI, -3.10 to -0.08). There was no association between the pandemic group and behavioral health trainees’ reported willingness to consider the VA for future employment (Table 1). Preliminary analyses demonstrated similar effects of the pandemic on psychology and social work trainees, therefore the groups were combined, and overall effects were reported.

Pandemic-Related Dissatisfaction

Of the 3950 psychology and social work trainees who responded to the survey, 75 (1.9%) indicated dissatisfaction with their VA training experience using pandemic-related keywords. Open-ended responses were generally short (range, 1-32 words; median, 19 words). Qualitative analyses revealed 7 themes (Table 2).

The most frequently identified theme was challenges with onboarding. One respondent indicated the modified onboarding procedures in place due to the pandemic were difficult to understand and resulted in delays. Another frequently mentioned cause of dissatisfaction was limited work or office space and insufficient computer availability. This was often noted to relate to a lack of private space to conduct telehealth visits or computers that were not equipped to provide telehealth. Several respondents also noted technological issues when attempting to use VVC to provide telehealth.

Another common theme was that the pandemic diminished teamwork, generated feelings of isolation, and created unsupportive environments for trainees. For instance, some trainees indicated that COVID-19 decreased the inclusion of trainees as part of the regular staff groups and accordingly resulted in limited networking opportunities. Other causes of dissatisfaction included the pandemic’s impacts on the learning environment, such as decreases in patient volume, decreased diversity of patient cases, and a limited presence of faculty mentors. Several respondents indicated that the pandemic limited their caseloads and indicated that most patients were seen virtually. Open-ended responses from a few respondents indicated their training environments were noncompliant with social distancing, personal protective equipment requirements, or other safety guidelines.

DISCUSSION

This study illustrates the impact of the COVID-19 pandemic on the behavioral health trainee experience, which was expressed through decreased satisfaction with their clinical training at the VA. The narrative data indicated that the observed pandemic-related dissatisfaction was linked specifically to onboarding, a lack of safe and private workspaces and computers, as well as a lack of a supportive work environment.

Although the reported decrease in satisfaction was statistically significant, the effect size was not large. Additionally, while satisfaction did decrease, the trainees’ reported likelihood to consider the VA for future employment was not impacted. This may suggest psychologist and social work trainees’ perseverance and dedication to their chosen profession persisted despite the challenges presented by the pandemic. Furthermore, the qualitative data suggest potential ways to mitigate health care profession trainee challenges that can follow a crisis like the COVID-19 pandemic, although further study is warranted.

While narrative responses with pandemic-related keywords did indicate challenges specific to COVID-19 (ie, limited access to workspaces and/or computers equipped for telehealth), the overall frequency of pandemic-related responses was low. This may indicate these are institutional challenges trainees face independent of the pandemic. These findings warrant longterm attention to the satisfaction of psychology and social worker trainees’ during the pandemic. For example, additional training for the use of telehealth could be provided. One study indicated that < 61% of psychology postdoctoral fellows received telepsychology training during the pandemic, and of those who did receive training, less than half were satisfied with it.³

Similarly, strategies could be developed to ensure a more supportive learning and work environment, and provide additional networking opportunities for trainees, despite social distancing. Education specific to disaster response should be incorporated into behavioral health care professionals’ training, especially because behavioral health care professionals provided major contributions during the pandemic due to reported increases in mental health concerns (eg, anxiety and depression) during the period.^7,17,18 As the pandemic progressed, policies and procedures were established or modified to address some of these concerns because they were not necessarily limited to trainees. For example, additional training resources were developed to support the use of various telehealth technologies, virtual resources were used more often for meetings, and supervisors developed more comfort and familiarity with how to manage in a virtual or hybrid environment.

Limitations

Although the TSS data provide a large national sample of behavioral health care trainees, it only includes VA trainees, and therefore may not be completely generalizable across health care. However, because many psychologists and social workers throughout the US train at the VA, and because the VA is the largest employer of practicing psychologists and social workers, understanding the impacts felt at the VA informs institutions nationally.^8-11 The TSS has limited demographic data (eg, age, race, ethnicity, and sex), so it is unclear whether the respondent groups before and during the pandemic differed in ways that could relate to outcomes. The data also do not specify exact training dates; however, anecdotal evidence suggests respondents generally complete the survey close to the end of their training.

Additionally, open-ended narrative responses were only asked for replies that indicated dissatisfaction, precluding a more nuanced understanding of potential positive outcomes. Furthermore, the TSS is limited to questions about the trainees’ clinical experiences, but because the pandemic created many stressors, there may have been personal issues that affected their work. It is possible that changes in overall satisfaction may have been rooted in something outside of their clinical experience. Finally, the response rate for the TSS is consistently low both before and during the pandemic and includes a limited number of narrative responses.

CONCLUSIONS

The VA is an important contributor to the education, training, and composition of the behavioral health care workforce. A deeper understanding of the VA trainee experience is important to identify how to improve behavioral health care professional education and training. This is especially true as behavioral health care faces shortages within the VA and nationwide.^8,12,19

This study reinforces research that found health care trainees experienced decreased learning opportunities and telehealth-related challenges during the COVID-19 pandemic. ^13,20 Despite the observed decline in trainee satisfaction, the lack of a corresponding change in likelihood to seek employment with the VA is encouraging for VA efforts to maintain and grow its behavioral health care workforce and for similar efforts outside VA. This resilience may relate to the substantial prepandemic time invested in their professional development. Future studies should examine long term impacts of the pandemic on trainee’s clinical experience and whether the pipeline of behavioral health care workers declines over time as students that are earlier in their career paths instead chose other professions. Future research should also explore ways to improve professional development and wellness of behavioral health care trainees during disasters (eg, telehealth training, additional networking, and social support).

The COVID-19 pandemic changed the education and training experiences of health care students and those set to comprise the future workforce. Apart from general training disruptions or delays due to the pandemic, behavioral health trainees such as psychologists and social workers faced limited opportunities to provide in-person services.^1-5 Trainees also experienced fewer referrals to mental health services from primary care and more disrupted, no-show, or cancelled appointments.^4-6 Behavioral health trainees experienced a limited ability to establish rapport and more difficulty providing effective services because of the limited in-person interaction presented by telehealth.⁶ The pandemic also resulted in feelings of increased isolation and decreased teamwork.^1,7 The virtual or remote setting made it more difficult for trainees to feel as if they were a member of a team or community of behavioral health professionals.^1,7

Behavioral health trainees had to adapt to conducting patient visits and educational didactics through virtual platforms.^1,3-7 Challenges included access or technological problems with online platforms and a lack of telehealth training use.^3,4,6 One study found that while both behavioral health trainees and licensed practitioners reported similar rates of telehealth use for mental health services by early April 2020, trainees had more difficulties implementing telehealth compared with licensed practitioners. This study found that US Department of Veteran Affairs (VA) facilities reported higher use of telehealth in February 2020.⁵

A mission of the VA is to provide education and training to health care professionals through partnerships with affiliated academic institutions. The VA is the largest education and training supplier for health care professions in the US. As many as 50% of psychologists in the US received some training at the VA.⁸ Additionally, more graduate-level social work students are trained at the VA than at any other organization.⁹ The VA is a major contributor to not only its own behavioral health workforce, but that of the entire country.

The VA is also the largest employer of psychologists and social workers in the US.^10,11 The VA Office of Academic Affiliations (OAA) oversees health care profession education and training at all VA facilities. In 2012, OAA began the Mental Health Education Expansion program to increase training for behavioral health professionals, including psychologists and social workers. ¹² The OAA initiative was aligned with VA training and workforce priorities.^8,12 To gauge the effectiveness of VA education and training, OAA encourages VA trainees to complete the Trainee Satisfaction Survey (TSS), which measures trainee satisfaction and the likelihood of a trainee to consider the VA for future employment.

Researchers at the Veterans Emergency Management Evaluation Center sought to understand the impact the COVID-19 pandemic had on behavioral health trainees’ experiences by examining TSS data from before and after the onset of the pandemic. This study expands on prior research among physician residents and fellows which found associations between VA training experiences and the COVID- 19 pandemic. The previous study found declines in trainee satisfaction and a decreased likelihood to consider the VA for future employment.¹³

It is important to understand the effects the pandemic had on the professional development and wellness for both physician and behavioral health professional trainees. Identifying how the pandemic impacted trainee satisfaction may help improve education programs and mitigate the impact of future public health emergencies. This is particularly important due to the shortage of behavioral health professionals in the VA and the US.^12,14

METHODS

This study used TSS data collected from August 2018 to July 2021 from 153 VA facilities. A behavioral health trainee was defined as any psychology or social work trainee who completed 1 rotation at a VA facility. Psychiatric trainees were excluded because as physicians their training programs differ markedly from those for psychology and social work. Excluding psychiatry, psychology and social work comprise the 2 largest mental health care training groups.

This study was reviewed and approved as a quality improvement project by the VA Greater Los Angeles Healthcare System (VAGLAHS) Institutional Review Board, which waived informed consent requirements. The OAA granted access to data using a process open to all VA researchers. At the time of data collection, respondents were assured their anonymity; participation was voluntary.

Measures

Any response provided before February 29, 2020, was defined as the prepandemic period. The pandemic period included any response from April 1, 2020, to July 31, 2021. Responses collected in March 2020 were excluded as it would be unclear from the survey whether the training period occurred before or after the onset of the pandemic.

To measure overall trainee satisfaction with the VA training experience, responses were grouped as satisfied (satisfied/ very satisfied) and dissatisfied (dissatisfied/ very dissatisfied). To measure a trainee’s likelihood to consider the VA for future employment as a result of their training experience, responses were grouped as likely (likely/very likely) and unlikely (unlikely/very unlikely).

Other components of satisfaction were also collected including onboarding, clinical faculty/preceptors, clinical learning environment, physical environment, working environment, and respect felt at work. If a respondent chose very dissatisfied or dissatisfied, they were subsequently asked to specify the reason for their dissatisfaction with an open-ended response. Open-ended responses were not permitted if a respondent indicated a satisfied or very satisfied response.

Statistical Analyses

Stata SE 17 was used for statistical analyses. To test the relationship between the pandemic group and the 2 separate outcome variables, logistic regressions were conducted to measure overall satisfaction and likelihood of future VA employment. Margin commands were used to calculate the difference in the probability of reporting satisfied/very satisfied and likely/very likely for the prepandemic and pandemic groups. The association of the COVID-19 group with each outcome variable was expressed as the difference in the percentage of the outcome between the prepandemic and pandemic groups. Preliminary analyses demonstrated similar effects of the pandemic on psychology and social work trainees; therefore, the groups were combined.

Rapid Coding and Thematic Analyses

Qualitative data were based on open-ended responses from behavioral health trainees when they were asked to specify the cause of dissatisfaction in the aforementioned areas of satisfaction. Methods for qualitative data included rapid coding and thematic content analyses.^15,16 Additional general information regarding the qualitative data analyses is described elsewhere.¹³ A keyword search was completed to identify all open-ended responses related to COVID-19 pandemic causes of dissatisfaction. Keywords included: virus, COVID, corona, pandemic, PPE, N95, mask, social distance, and safety. All open-ended responses were reviewed to ensure keywords were appropriately identifying pandemic-related causes of dissatisfaction and did not overlook other references to the pandemic, and to identify initial themes and corresponding definitions based on survey questions. After review, additional keywords were included in the content analyses that were related to providing mental health services using remote or telehealth options. This included the following keywords: remote, video, VVC (VA Video Connect), and tele. The research team completed a review of the initial themes and definitions and created a final coding construct with definitions before completing an independent coding of all identified pandemic-related responses. Frequency counts of each code were provided to identify which pandemic-related causes of dissatisfaction were mentioned most.

RESULTS

A total of 3950 behavioral health trainees responded to the TSS, including 2715 psychology trainees and 1235 social work trainees who indicated they received training at the VA in academic years 2018/2019, 2019/2020, or 2020/2021. The academic year 2018/2019 was considered in an effort to provide a larger sample of prepandemic trainees.

The percentage of trainees reporting satisfaction with their training decreased across prepandemic to pandemic groups. In the pandemic group, 2166 of 2324 respondents (93.2%) reported satisfaction compared to 1474 of 1555 (94.8%) in the prepandemic trainee group (P = .04; 95% CI, -3.10 to -0.08). There was no association between the pandemic group and behavioral health trainees’ reported willingness to consider the VA for future employment (Table 1). Preliminary analyses demonstrated similar effects of the pandemic on psychology and social work trainees, therefore the groups were combined, and overall effects were reported.

Pandemic-Related Dissatisfaction

Of the 3950 psychology and social work trainees who responded to the survey, 75 (1.9%) indicated dissatisfaction with their VA training experience using pandemic-related keywords. Open-ended responses were generally short (range, 1-32 words; median, 19 words). Qualitative analyses revealed 7 themes (Table 2).

The most frequently identified theme was challenges with onboarding. One respondent indicated the modified onboarding procedures in place due to the pandemic were difficult to understand and resulted in delays. Another frequently mentioned cause of dissatisfaction was limited work or office space and insufficient computer availability. This was often noted to relate to a lack of private space to conduct telehealth visits or computers that were not equipped to provide telehealth. Several respondents also noted technological issues when attempting to use VVC to provide telehealth.

Another common theme was that the pandemic diminished teamwork, generated feelings of isolation, and created unsupportive environments for trainees. For instance, some trainees indicated that COVID-19 decreased the inclusion of trainees as part of the regular staff groups and accordingly resulted in limited networking opportunities. Other causes of dissatisfaction included the pandemic’s impacts on the learning environment, such as decreases in patient volume, decreased diversity of patient cases, and a limited presence of faculty mentors. Several respondents indicated that the pandemic limited their caseloads and indicated that most patients were seen virtually. Open-ended responses from a few respondents indicated their training environments were noncompliant with social distancing, personal protective equipment requirements, or other safety guidelines.

DISCUSSION

This study illustrates the impact of the COVID-19 pandemic on the behavioral health trainee experience, which was expressed through decreased satisfaction with their clinical training at the VA. The narrative data indicated that the observed pandemic-related dissatisfaction was linked specifically to onboarding, a lack of safe and private workspaces and computers, as well as a lack of a supportive work environment.

Although the reported decrease in satisfaction was statistically significant, the effect size was not large. Additionally, while satisfaction did decrease, the trainees’ reported likelihood to consider the VA for future employment was not impacted. This may suggest psychologist and social work trainees’ perseverance and dedication to their chosen profession persisted despite the challenges presented by the pandemic. Furthermore, the qualitative data suggest potential ways to mitigate health care profession trainee challenges that can follow a crisis like the COVID-19 pandemic, although further study is warranted.

While narrative responses with pandemic-related keywords did indicate challenges specific to COVID-19 (ie, limited access to workspaces and/or computers equipped for telehealth), the overall frequency of pandemic-related responses was low. This may indicate these are institutional challenges trainees face independent of the pandemic. These findings warrant longterm attention to the satisfaction of psychology and social worker trainees’ during the pandemic. For example, additional training for the use of telehealth could be provided. One study indicated that < 61% of psychology postdoctoral fellows received telepsychology training during the pandemic, and of those who did receive training, less than half were satisfied with it.³

Similarly, strategies could be developed to ensure a more supportive learning and work environment, and provide additional networking opportunities for trainees, despite social distancing. Education specific to disaster response should be incorporated into behavioral health care professionals’ training, especially because behavioral health care professionals provided major contributions during the pandemic due to reported increases in mental health concerns (eg, anxiety and depression) during the period.^7,17,18 As the pandemic progressed, policies and procedures were established or modified to address some of these concerns because they were not necessarily limited to trainees. For example, additional training resources were developed to support the use of various telehealth technologies, virtual resources were used more often for meetings, and supervisors developed more comfort and familiarity with how to manage in a virtual or hybrid environment.

Limitations

Although the TSS data provide a large national sample of behavioral health care trainees, it only includes VA trainees, and therefore may not be completely generalizable across health care. However, because many psychologists and social workers throughout the US train at the VA, and because the VA is the largest employer of practicing psychologists and social workers, understanding the impacts felt at the VA informs institutions nationally.^8-11 The TSS has limited demographic data (eg, age, race, ethnicity, and sex), so it is unclear whether the respondent groups before and during the pandemic differed in ways that could relate to outcomes. The data also do not specify exact training dates; however, anecdotal evidence suggests respondents generally complete the survey close to the end of their training.

Additionally, open-ended narrative responses were only asked for replies that indicated dissatisfaction, precluding a more nuanced understanding of potential positive outcomes. Furthermore, the TSS is limited to questions about the trainees’ clinical experiences, but because the pandemic created many stressors, there may have been personal issues that affected their work. It is possible that changes in overall satisfaction may have been rooted in something outside of their clinical experience. Finally, the response rate for the TSS is consistently low both before and during the pandemic and includes a limited number of narrative responses.

CONCLUSIONS

The VA is an important contributor to the education, training, and composition of the behavioral health care workforce. A deeper understanding of the VA trainee experience is important to identify how to improve behavioral health care professional education and training. This is especially true as behavioral health care faces shortages within the VA and nationwide.^8,12,19

This study reinforces research that found health care trainees experienced decreased learning opportunities and telehealth-related challenges during the COVID-19 pandemic. ^13,20 Despite the observed decline in trainee satisfaction, the lack of a corresponding change in likelihood to seek employment with the VA is encouraging for VA efforts to maintain and grow its behavioral health care workforce and for similar efforts outside VA. This resilience may relate to the substantial prepandemic time invested in their professional development. Future studies should examine long term impacts of the pandemic on trainee’s clinical experience and whether the pipeline of behavioral health care workers declines over time as students that are earlier in their career paths instead chose other professions. Future research should also explore ways to improve professional development and wellness of behavioral health care trainees during disasters (eg, telehealth training, additional networking, and social support).

The COVID-19 pandemic changed the education and training experiences of health care students and those set to comprise the future workforce. Apart from general training disruptions or delays due to the pandemic, behavioral health trainees such as psychologists and social workers faced limited opportunities to provide in-person services.^1-5 Trainees also experienced fewer referrals to mental health services from primary care and more disrupted, no-show, or cancelled appointments.^4-6 Behavioral health trainees experienced a limited ability to establish rapport and more difficulty providing effective services because of the limited in-person interaction presented by telehealth.⁶ The pandemic also resulted in feelings of increased isolation and decreased teamwork.^1,7 The virtual or remote setting made it more difficult for trainees to feel as if they were a member of a team or community of behavioral health professionals.^1,7

Behavioral health trainees had to adapt to conducting patient visits and educational didactics through virtual platforms.^1,3-7 Challenges included access or technological problems with online platforms and a lack of telehealth training use.^3,4,6 One study found that while both behavioral health trainees and licensed practitioners reported similar rates of telehealth use for mental health services by early April 2020, trainees had more difficulties implementing telehealth compared with licensed practitioners. This study found that US Department of Veteran Affairs (VA) facilities reported higher use of telehealth in February 2020.⁵

A mission of the VA is to provide education and training to health care professionals through partnerships with affiliated academic institutions. The VA is the largest education and training supplier for health care professions in the US. As many as 50% of psychologists in the US received some training at the VA.⁸ Additionally, more graduate-level social work students are trained at the VA than at any other organization.⁹ The VA is a major contributor to not only its own behavioral health workforce, but that of the entire country.

The VA is also the largest employer of psychologists and social workers in the US.^10,11 The VA Office of Academic Affiliations (OAA) oversees health care profession education and training at all VA facilities. In 2012, OAA began the Mental Health Education Expansion program to increase training for behavioral health professionals, including psychologists and social workers. ¹² The OAA initiative was aligned with VA training and workforce priorities.^8,12 To gauge the effectiveness of VA education and training, OAA encourages VA trainees to complete the Trainee Satisfaction Survey (TSS), which measures trainee satisfaction and the likelihood of a trainee to consider the VA for future employment.

Researchers at the Veterans Emergency Management Evaluation Center sought to understand the impact the COVID-19 pandemic had on behavioral health trainees’ experiences by examining TSS data from before and after the onset of the pandemic. This study expands on prior research among physician residents and fellows which found associations between VA training experiences and the COVID- 19 pandemic. The previous study found declines in trainee satisfaction and a decreased likelihood to consider the VA for future employment.¹³

It is important to understand the effects the pandemic had on the professional development and wellness for both physician and behavioral health professional trainees. Identifying how the pandemic impacted trainee satisfaction may help improve education programs and mitigate the impact of future public health emergencies. This is particularly important due to the shortage of behavioral health professionals in the VA and the US.^12,14

METHODS

This study used TSS data collected from August 2018 to July 2021 from 153 VA facilities. A behavioral health trainee was defined as any psychology or social work trainee who completed 1 rotation at a VA facility. Psychiatric trainees were excluded because as physicians their training programs differ markedly from those for psychology and social work. Excluding psychiatry, psychology and social work comprise the 2 largest mental health care training groups.

This study was reviewed and approved as a quality improvement project by the VA Greater Los Angeles Healthcare System (VAGLAHS) Institutional Review Board, which waived informed consent requirements. The OAA granted access to data using a process open to all VA researchers. At the time of data collection, respondents were assured their anonymity; participation was voluntary.

Measures

Any response provided before February 29, 2020, was defined as the prepandemic period. The pandemic period included any response from April 1, 2020, to July 31, 2021. Responses collected in March 2020 were excluded as it would be unclear from the survey whether the training period occurred before or after the onset of the pandemic.

To measure overall trainee satisfaction with the VA training experience, responses were grouped as satisfied (satisfied/ very satisfied) and dissatisfied (dissatisfied/ very dissatisfied). To measure a trainee’s likelihood to consider the VA for future employment as a result of their training experience, responses were grouped as likely (likely/very likely) and unlikely (unlikely/very unlikely).

Other components of satisfaction were also collected including onboarding, clinical faculty/preceptors, clinical learning environment, physical environment, working environment, and respect felt at work. If a respondent chose very dissatisfied or dissatisfied, they were subsequently asked to specify the reason for their dissatisfaction with an open-ended response. Open-ended responses were not permitted if a respondent indicated a satisfied or very satisfied response.

Statistical Analyses

Stata SE 17 was used for statistical analyses. To test the relationship between the pandemic group and the 2 separate outcome variables, logistic regressions were conducted to measure overall satisfaction and likelihood of future VA employment. Margin commands were used to calculate the difference in the probability of reporting satisfied/very satisfied and likely/very likely for the prepandemic and pandemic groups. The association of the COVID-19 group with each outcome variable was expressed as the difference in the percentage of the outcome between the prepandemic and pandemic groups. Preliminary analyses demonstrated similar effects of the pandemic on psychology and social work trainees; therefore, the groups were combined.

Rapid Coding and Thematic Analyses

Qualitative data were based on open-ended responses from behavioral health trainees when they were asked to specify the cause of dissatisfaction in the aforementioned areas of satisfaction. Methods for qualitative data included rapid coding and thematic content analyses.^15,16 Additional general information regarding the qualitative data analyses is described elsewhere.¹³ A keyword search was completed to identify all open-ended responses related to COVID-19 pandemic causes of dissatisfaction. Keywords included: virus, COVID, corona, pandemic, PPE, N95, mask, social distance, and safety. All open-ended responses were reviewed to ensure keywords were appropriately identifying pandemic-related causes of dissatisfaction and did not overlook other references to the pandemic, and to identify initial themes and corresponding definitions based on survey questions. After review, additional keywords were included in the content analyses that were related to providing mental health services using remote or telehealth options. This included the following keywords: remote, video, VVC (VA Video Connect), and tele. The research team completed a review of the initial themes and definitions and created a final coding construct with definitions before completing an independent coding of all identified pandemic-related responses. Frequency counts of each code were provided to identify which pandemic-related causes of dissatisfaction were mentioned most.

RESULTS

A total of 3950 behavioral health trainees responded to the TSS, including 2715 psychology trainees and 1235 social work trainees who indicated they received training at the VA in academic years 2018/2019, 2019/2020, or 2020/2021. The academic year 2018/2019 was considered in an effort to provide a larger sample of prepandemic trainees.

The percentage of trainees reporting satisfaction with their training decreased across prepandemic to pandemic groups. In the pandemic group, 2166 of 2324 respondents (93.2%) reported satisfaction compared to 1474 of 1555 (94.8%) in the prepandemic trainee group (P = .04; 95% CI, -3.10 to -0.08). There was no association between the pandemic group and behavioral health trainees’ reported willingness to consider the VA for future employment (Table 1). Preliminary analyses demonstrated similar effects of the pandemic on psychology and social work trainees, therefore the groups were combined, and overall effects were reported.

Pandemic-Related Dissatisfaction

Of the 3950 psychology and social work trainees who responded to the survey, 75 (1.9%) indicated dissatisfaction with their VA training experience using pandemic-related keywords. Open-ended responses were generally short (range, 1-32 words; median, 19 words). Qualitative analyses revealed 7 themes (Table 2).

The most frequently identified theme was challenges with onboarding. One respondent indicated the modified onboarding procedures in place due to the pandemic were difficult to understand and resulted in delays. Another frequently mentioned cause of dissatisfaction was limited work or office space and insufficient computer availability. This was often noted to relate to a lack of private space to conduct telehealth visits or computers that were not equipped to provide telehealth. Several respondents also noted technological issues when attempting to use VVC to provide telehealth.

Another common theme was that the pandemic diminished teamwork, generated feelings of isolation, and created unsupportive environments for trainees. For instance, some trainees indicated that COVID-19 decreased the inclusion of trainees as part of the regular staff groups and accordingly resulted in limited networking opportunities. Other causes of dissatisfaction included the pandemic’s impacts on the learning environment, such as decreases in patient volume, decreased diversity of patient cases, and a limited presence of faculty mentors. Several respondents indicated that the pandemic limited their caseloads and indicated that most patients were seen virtually. Open-ended responses from a few respondents indicated their training environments were noncompliant with social distancing, personal protective equipment requirements, or other safety guidelines.

DISCUSSION

This study illustrates the impact of the COVID-19 pandemic on the behavioral health trainee experience, which was expressed through decreased satisfaction with their clinical training at the VA. The narrative data indicated that the observed pandemic-related dissatisfaction was linked specifically to onboarding, a lack of safe and private workspaces and computers, as well as a lack of a supportive work environment.

Although the reported decrease in satisfaction was statistically significant, the effect size was not large. Additionally, while satisfaction did decrease, the trainees’ reported likelihood to consider the VA for future employment was not impacted. This may suggest psychologist and social work trainees’ perseverance and dedication to their chosen profession persisted despite the challenges presented by the pandemic. Furthermore, the qualitative data suggest potential ways to mitigate health care profession trainee challenges that can follow a crisis like the COVID-19 pandemic, although further study is warranted.

While narrative responses with pandemic-related keywords did indicate challenges specific to COVID-19 (ie, limited access to workspaces and/or computers equipped for telehealth), the overall frequency of pandemic-related responses was low. This may indicate these are institutional challenges trainees face independent of the pandemic. These findings warrant longterm attention to the satisfaction of psychology and social worker trainees’ during the pandemic. For example, additional training for the use of telehealth could be provided. One study indicated that < 61% of psychology postdoctoral fellows received telepsychology training during the pandemic, and of those who did receive training, less than half were satisfied with it.³

Similarly, strategies could be developed to ensure a more supportive learning and work environment, and provide additional networking opportunities for trainees, despite social distancing. Education specific to disaster response should be incorporated into behavioral health care professionals’ training, especially because behavioral health care professionals provided major contributions during the pandemic due to reported increases in mental health concerns (eg, anxiety and depression) during the period.^7,17,18 As the pandemic progressed, policies and procedures were established or modified to address some of these concerns because they were not necessarily limited to trainees. For example, additional training resources were developed to support the use of various telehealth technologies, virtual resources were used more often for meetings, and supervisors developed more comfort and familiarity with how to manage in a virtual or hybrid environment.

Limitations

Although the TSS data provide a large national sample of behavioral health care trainees, it only includes VA trainees, and therefore may not be completely generalizable across health care. However, because many psychologists and social workers throughout the US train at the VA, and because the VA is the largest employer of practicing psychologists and social workers, understanding the impacts felt at the VA informs institutions nationally.^8-11 The TSS has limited demographic data (eg, age, race, ethnicity, and sex), so it is unclear whether the respondent groups before and during the pandemic differed in ways that could relate to outcomes. The data also do not specify exact training dates; however, anecdotal evidence suggests respondents generally complete the survey close to the end of their training.

Additionally, open-ended narrative responses were only asked for replies that indicated dissatisfaction, precluding a more nuanced understanding of potential positive outcomes. Furthermore, the TSS is limited to questions about the trainees’ clinical experiences, but because the pandemic created many stressors, there may have been personal issues that affected their work. It is possible that changes in overall satisfaction may have been rooted in something outside of their clinical experience. Finally, the response rate for the TSS is consistently low both before and during the pandemic and includes a limited number of narrative responses.

CONCLUSIONS

The VA is an important contributor to the education, training, and composition of the behavioral health care workforce. A deeper understanding of the VA trainee experience is important to identify how to improve behavioral health care professional education and training. This is especially true as behavioral health care faces shortages within the VA and nationwide.^8,12,19

This study reinforces research that found health care trainees experienced decreased learning opportunities and telehealth-related challenges during the COVID-19 pandemic. ^13,20 Despite the observed decline in trainee satisfaction, the lack of a corresponding change in likelihood to seek employment with the VA is encouraging for VA efforts to maintain and grow its behavioral health care workforce and for similar efforts outside VA. This resilience may relate to the substantial prepandemic time invested in their professional development. Future studies should examine long term impacts of the pandemic on trainee’s clinical experience and whether the pipeline of behavioral health care workers declines over time as students that are earlier in their career paths instead chose other professions. Future research should also explore ways to improve professional development and wellness of behavioral health care trainees during disasters (eg, telehealth training, additional networking, and social support).

Blood cultures provide crucial evidence for diagnostic medicine, specifically aimed at identifying the presence of microbial infections in the bloodstream. Blood culturing is instrumental in diagnosing conditions such as sepsis, bacteremia, or fungemia, where the identification of the causative agent is necessary for targeted and effective treatment.¹

The process involves aseptically drawing blood into sterile culture bottles, minimizing the risk of contamination with well-established guidelines. These culture bottles contain specific growth media that support the replication of microorganisms if they are present. Once the blood specimen is collected, it incubates, allowing any potential pathogens to grow. Subsequent analysis and identification of these microorganisms enable health care professionals (HCPs) to prescribe appropriate antimicrobial therapies to treat specific infections, contributing to more effective and targeted patient care.²

The reliability of blood culture results depends on minimizing contamination risk, a challenge inherent in the procedure. Contamination can lead to false-positive results, potentially misguiding treatment.³ HCPs must adhere to strict aseptic techniques during blood draws, ensuring proper skin preparation with antiseptic solutions. The use of sterile equipment and avoiding prolonged tourniquet application helps maintain the integrity of the blood specimen. Timely inoculation of blood into culture bottles and careful handling are essential to mitigate contamination risk.² Regular training and reinforcement of proper techniques is important to uphold the accuracy of blood culture results and enhance the reliability of diagnoses and treatment decisions.³ Despite diligent contamination prevention efforts, health care systems struggle to maintain contamination rates below the 3.0% national benchmark set by the Clinical & Laboratory Standards Institute (CLSI).⁴

Blood culture contamination is a critical concern in clinical practice; it can lead to misdiagnosis, prolonged hospital stays, unnecessary antibiotic use, and increased health care costs.⁵ Monitoring blood culture contamination is integral to patient safety, avoiding inappropriate and potentially harmful treatment, providing efficient care, contributing to antibiotic stewardship, supporting cost efficiency, and maintaining quality assurance and clinical research practices for public health.⁶

The initial specimen diversion technique (ISDT) recently emerged as a potential strategy to reduce blood culture contamination rates. This technique involves diverting a small portion of the initial blood plus the skin plug from the hollow needle away from the primary collection site before filling the culture bottles. This process minimizes skin surface contaminants, providing a cleaner blood specimen for culturing.⁷

The ISDT was introduced as a result of historically elevated contamination rates.⁸ Despite implementing various mitigation methods, the US Department of Veterans Affairs (VA) Central Texas Healthcare System (VACTHCS) has struggled to meet the national benchmark of maintaining blood culture contamination < 3.0%. The VACTHCS is a 146-bed teaching hospital with about 30,000 annual visits at the Olin E. Teague Veterans Affairs Medical Center (OETVMC) emergency department (ED). VACTHCS conducted a 16-month pilot study using 2 commercially available ISDT devices and published the findings.⁸

The Military Construction, Veterans Affairs, and Related Agencies Appropriations Act, 2022 (MilCon-VA Act) committee report prioritized the reduction of blood culture contamination to < 1% to prevent health risks and harm to veterans undergoing blood testing for the diagnosis of sepsis.⁹ Because it had been 5 years since OETVMC began using an ISDT in the ED, the ISDT adaptation strategy for mitigating blood culture contamination was revisited per institution policy.

The objective of this quality improvement project was to analyze retrospective data to understand the long-term impact of ISDT use on blood culture contamination rates. We hypothesized that ISDT use would contribute to efforts to maintain OETVMC ED blood culture contamination rate below the national (3.0%) and VACTHCS (2.5%) thresholds. This project assessed the progress for reducing blood culture contamination compared with the pre-ISDT era.⁸

METHODS

This retrospective analysis compared the blood culture contamination rates 36 months before and after the introduction of the ISDT device at the OETVMC ED. The preimplementation period was from December 2014 through November 2017 (36 months) and the postimplementation period was December 2017 through November 2020 (36 months). Data were collected from the Department of Pathology and Microbiology blood culture records of all adult patients admitted to the hospital through the ED and required blood cultures for suspicion of infection. Protected health information and VA sensitive information were not collected: all data were deidentified. A total of 18,541 blood cultures were collected 36 months preimplementation and 14,865 blood cultures were collected up to 36 months postimplementation. For comparison purposes, a similar dataset was collected from patients’ blood samples drawn by phlebotomists in the laboratory, where there had been no previous issues with overcontamination; no ISDT devices were used in the collection of these samples.

Blood Culture Contamination Variable

Blood cultures were monitored using the BACT/ALERT 3D (bioMérieux) and subsequently BACT/ALERT VIRTUO (bioMérieux), with positive bottles characterized by VITEK MS Matrix Assisted Laser Desorption Ionization Time-of-Flight technology (bioMérieux) and automated susceptibility testing (VITEK 2 [bioMérieux]).¹⁰ In an updated review of blood culture contamination, the American Society for Microbiology used the College of American Pathologists' Q-Probes quality improvement studies as a guideline for classifying contamination. A sample was determined to be contaminated if ≥ 1 of the following organisms were found in only 1 bottle in a series of blood culture sets: coagulase-negative staphylococci, Micrococcus species, α-hemolytic viridans group streptococci, Corynebacterium species, Propionibacterium acnes, and Bacillus species.¹¹ The contamination assessment criteria remained unchanged, except for use of an ISDT device in blood culture collection at the ED.

The VACTHCS Infection Prevention Department ensured that the ISDT device was available and that ED nurses were trained annually on its use to collect blood cultures. Monthly reports of contamination were sent to the nursing supervisor for corrective action and retraining. The initial performance improvement project was slated for 16 months but was expanded to a 6-year period of retrospective data to obtain strong correlation.

Statistical Analysis

Contamination rates were recorded monthly from the hospital laboratory information management system for 36 months both before and after ISDT adoption. Statistical analysis was performed using a 2-tailed unpaired t-test to compare monthly contamination rates for the 2 periods with GraphPad Prism version 10.0.0 for Windows.

RESULTS

Prior to 2017, the ED reported contamination rates above the national (3.0%) and OETVMC thresholds (2.5%), with a mean of 4.5% (95% CI, 3.90-4.90).⁸ After ISDT implementation, the ED showed significant improvement with a reduction to mean 2.6% (95% CI, 2.10-3.20) (P < .001) (Figure 1). Figure 2 shows monthly blood culture contamination rates at the ED from December 2014 through November 2020. Month 36 (November 2017) shows a clear dip in contamination rate when the ISDT was introduced and month 37 to month 44 show remarkably low contamination rates. During this time, the institute experimented with 2 ISDT devices, and closer scrutiny may reveal this period as an outlier due to the monitoring of ISDT application, as previously reported.⁸

The blood culture contamination rate for samples drawn by the phlebotomists in the laboratory (excluding the ED) was calculated during the same time period (Figure 3). Non-ED contamination rates remained below 2.5% for 69 of 72 months.

DISCUSSION

The blood culture contamination rate in the OETVMC ED dropped following ISDT implementation and continued to show long-term benefits. For the 36-month period following ISDT implementation, the mean contamination rate was 2.6%, which was below the national target threshold of 3.0% and close to the OETVMC target of 2.5%. These results suggest that ISDT can have a positive impact on patient care and laboratory efficiency. Improvements in the blood contamination rates in the ED can have a positive impact on the overall hospital contamination rates.

Blood drawn by phlebotomists in the hospital laboratory infrequently had contamination rates that exceeded the 2.5% target threshold. Because the non-ED contamination rates did not change throughout the comparison period, other factors were likely not involved in the improvements seen in the ED. The decision to implement ISDT exclusively in the ED was based on its historically elevated contamination rate.⁸ Issues with blood culture contamination in EDs across various hospital systems are well documented and not unique to VACTHCS.¹²

Contamination in blood cultures can be a significant issue in the hospital. It occurs when microorganisms from the skin or environment enter the blood culture sample during collection. Moreover, it can contribute to antibiotic resistance when patients are prescribed inappropriate antibiotics. It is also important to ensure HCPs are well-trained and consistently follow standardized protocols and understand the implications of false-positive results.¹³

ISDT helps reduce false-positive results and is a significant advancement in the field of blood culture collection.^8,14 By discarding the initial blood, it ensures that only the true bloodstream sample is cultured, leading to more accurate results.¹⁵ It also may minimize the risk of contamination-related delays in diagnosis and treatment and benefits patients and health care institutions by potentially reducing hospital stays, unnecessary antibiotic use, and health care costs.

One of the ISDT device manufacturers estimated the financial impact on OETVMC based on the pilot project.⁸ While this study did not calculate the direct and indirect cost savings associated with this process improvement, the manufacturer’s website suggests that VACTHCS could annually save about $486,000.¹⁶ Furthermore, implementation of ISDT may improve laboratory efficiency, as they reduce the workload associated with identifying and reporting false-positive cultures. ⁶ ISDT devices represent a valuable tool in the efforts to reduce blood culture contamination and its wide-ranging implications in clinical settings. While ISDT alone will not be sufficient in achieving a lower threshold (< 1%) of blood culture contamination, it can be part of a multiprong effort that optimizes best practices in the collection, handling, and management of blood cultures.

Continuous quality improvement efforts and monitoring of blood culture contamination rates can help health care institutions identify problem areas and implement necessary changes. Addressing blood culture contamination can improve patient care, reduce costs, and address antibiotic resistance.

Limitations

This study was limited by its study design, which did not use a side-by-side comparison of blood cultures from groups with and without ISDT. All blood cultures from patients in the region were processed at OETVMC, which may not be representative of non-VA EDs. Part of this study took place during the COVID-19 pandemic, which may have skewed data. Additionally, hospital data were collected from a veteran population in Central Texas, and the lack of demographic diversity may not be generalizable to the greater population.

CONCLUSIONS

The findings of this study suggest ISDT may be effective in reducing blood culture contamination rates in the high-risk ED environment, which aligns with previous research. ^5,14 The ISDT may help reduce blood culture contamination rates, improving the quality of patient care and reducing health care costs. MilCon-VA mandated that all VA facilities have blood culture contamination as a metric with a goal of blood culture contamination rates < 1%.⁸ However, achieving this goal remains a challenge. Further research and continuous quality improvement efforts are necessary to achieve it. Consistently achieving a contamination threshold of < 1% may require minimizing human error. An automated robotic venipuncture device, as recently designed and reported, may be necessary to reduce human error in blood draw and contamination.¹⁶

Blood cultures provide crucial evidence for diagnostic medicine, specifically aimed at identifying the presence of microbial infections in the bloodstream. Blood culturing is instrumental in diagnosing conditions such as sepsis, bacteremia, or fungemia, where the identification of the causative agent is necessary for targeted and effective treatment.¹

The process involves aseptically drawing blood into sterile culture bottles, minimizing the risk of contamination with well-established guidelines. These culture bottles contain specific growth media that support the replication of microorganisms if they are present. Once the blood specimen is collected, it incubates, allowing any potential pathogens to grow. Subsequent analysis and identification of these microorganisms enable health care professionals (HCPs) to prescribe appropriate antimicrobial therapies to treat specific infections, contributing to more effective and targeted patient care.²

The reliability of blood culture results depends on minimizing contamination risk, a challenge inherent in the procedure. Contamination can lead to false-positive results, potentially misguiding treatment.³ HCPs must adhere to strict aseptic techniques during blood draws, ensuring proper skin preparation with antiseptic solutions. The use of sterile equipment and avoiding prolonged tourniquet application helps maintain the integrity of the blood specimen. Timely inoculation of blood into culture bottles and careful handling are essential to mitigate contamination risk.² Regular training and reinforcement of proper techniques is important to uphold the accuracy of blood culture results and enhance the reliability of diagnoses and treatment decisions.³ Despite diligent contamination prevention efforts, health care systems struggle to maintain contamination rates below the 3.0% national benchmark set by the Clinical & Laboratory Standards Institute (CLSI).⁴

Blood culture contamination is a critical concern in clinical practice; it can lead to misdiagnosis, prolonged hospital stays, unnecessary antibiotic use, and increased health care costs.⁵ Monitoring blood culture contamination is integral to patient safety, avoiding inappropriate and potentially harmful treatment, providing efficient care, contributing to antibiotic stewardship, supporting cost efficiency, and maintaining quality assurance and clinical research practices for public health.⁶

The initial specimen diversion technique (ISDT) recently emerged as a potential strategy to reduce blood culture contamination rates. This technique involves diverting a small portion of the initial blood plus the skin plug from the hollow needle away from the primary collection site before filling the culture bottles. This process minimizes skin surface contaminants, providing a cleaner blood specimen for culturing.⁷

The ISDT was introduced as a result of historically elevated contamination rates.⁸ Despite implementing various mitigation methods, the US Department of Veterans Affairs (VA) Central Texas Healthcare System (VACTHCS) has struggled to meet the national benchmark of maintaining blood culture contamination < 3.0%. The VACTHCS is a 146-bed teaching hospital with about 30,000 annual visits at the Olin E. Teague Veterans Affairs Medical Center (OETVMC) emergency department (ED). VACTHCS conducted a 16-month pilot study using 2 commercially available ISDT devices and published the findings.⁸

The Military Construction, Veterans Affairs, and Related Agencies Appropriations Act, 2022 (MilCon-VA Act) committee report prioritized the reduction of blood culture contamination to < 1% to prevent health risks and harm to veterans undergoing blood testing for the diagnosis of sepsis.⁹ Because it had been 5 years since OETVMC began using an ISDT in the ED, the ISDT adaptation strategy for mitigating blood culture contamination was revisited per institution policy.

The objective of this quality improvement project was to analyze retrospective data to understand the long-term impact of ISDT use on blood culture contamination rates. We hypothesized that ISDT use would contribute to efforts to maintain OETVMC ED blood culture contamination rate below the national (3.0%) and VACTHCS (2.5%) thresholds. This project assessed the progress for reducing blood culture contamination compared with the pre-ISDT era.⁸

METHODS

This retrospective analysis compared the blood culture contamination rates 36 months before and after the introduction of the ISDT device at the OETVMC ED. The preimplementation period was from December 2014 through November 2017 (36 months) and the postimplementation period was December 2017 through November 2020 (36 months). Data were collected from the Department of Pathology and Microbiology blood culture records of all adult patients admitted to the hospital through the ED and required blood cultures for suspicion of infection. Protected health information and VA sensitive information were not collected: all data were deidentified. A total of 18,541 blood cultures were collected 36 months preimplementation and 14,865 blood cultures were collected up to 36 months postimplementation. For comparison purposes, a similar dataset was collected from patients’ blood samples drawn by phlebotomists in the laboratory, where there had been no previous issues with overcontamination; no ISDT devices were used in the collection of these samples.

Blood Culture Contamination Variable

Blood cultures were monitored using the BACT/ALERT 3D (bioMérieux) and subsequently BACT/ALERT VIRTUO (bioMérieux), with positive bottles characterized by VITEK MS Matrix Assisted Laser Desorption Ionization Time-of-Flight technology (bioMérieux) and automated susceptibility testing (VITEK 2 [bioMérieux]).¹⁰ In an updated review of blood culture contamination, the American Society for Microbiology used the College of American Pathologists' Q-Probes quality improvement studies as a guideline for classifying contamination. A sample was determined to be contaminated if ≥ 1 of the following organisms were found in only 1 bottle in a series of blood culture sets: coagulase-negative staphylococci, Micrococcus species, α-hemolytic viridans group streptococci, Corynebacterium species, Propionibacterium acnes, and Bacillus species.¹¹ The contamination assessment criteria remained unchanged, except for use of an ISDT device in blood culture collection at the ED.

The VACTHCS Infection Prevention Department ensured that the ISDT device was available and that ED nurses were trained annually on its use to collect blood cultures. Monthly reports of contamination were sent to the nursing supervisor for corrective action and retraining. The initial performance improvement project was slated for 16 months but was expanded to a 6-year period of retrospective data to obtain strong correlation.

Statistical Analysis

Contamination rates were recorded monthly from the hospital laboratory information management system for 36 months both before and after ISDT adoption. Statistical analysis was performed using a 2-tailed unpaired t-test to compare monthly contamination rates for the 2 periods with GraphPad Prism version 10.0.0 for Windows.

RESULTS

Prior to 2017, the ED reported contamination rates above the national (3.0%) and OETVMC thresholds (2.5%), with a mean of 4.5% (95% CI, 3.90-4.90).⁸ After ISDT implementation, the ED showed significant improvement with a reduction to mean 2.6% (95% CI, 2.10-3.20) (P < .001) (Figure 1). Figure 2 shows monthly blood culture contamination rates at the ED from December 2014 through November 2020. Month 36 (November 2017) shows a clear dip in contamination rate when the ISDT was introduced and month 37 to month 44 show remarkably low contamination rates. During this time, the institute experimented with 2 ISDT devices, and closer scrutiny may reveal this period as an outlier due to the monitoring of ISDT application, as previously reported.⁸

The blood culture contamination rate for samples drawn by the phlebotomists in the laboratory (excluding the ED) was calculated during the same time period (Figure 3). Non-ED contamination rates remained below 2.5% for 69 of 72 months.

DISCUSSION

The blood culture contamination rate in the OETVMC ED dropped following ISDT implementation and continued to show long-term benefits. For the 36-month period following ISDT implementation, the mean contamination rate was 2.6%, which was below the national target threshold of 3.0% and close to the OETVMC target of 2.5%. These results suggest that ISDT can have a positive impact on patient care and laboratory efficiency. Improvements in the blood contamination rates in the ED can have a positive impact on the overall hospital contamination rates.

Blood drawn by phlebotomists in the hospital laboratory infrequently had contamination rates that exceeded the 2.5% target threshold. Because the non-ED contamination rates did not change throughout the comparison period, other factors were likely not involved in the improvements seen in the ED. The decision to implement ISDT exclusively in the ED was based on its historically elevated contamination rate.⁸ Issues with blood culture contamination in EDs across various hospital systems are well documented and not unique to VACTHCS.¹²

Contamination in blood cultures can be a significant issue in the hospital. It occurs when microorganisms from the skin or environment enter the blood culture sample during collection. Moreover, it can contribute to antibiotic resistance when patients are prescribed inappropriate antibiotics. It is also important to ensure HCPs are well-trained and consistently follow standardized protocols and understand the implications of false-positive results.¹³

ISDT helps reduce false-positive results and is a significant advancement in the field of blood culture collection.^8,14 By discarding the initial blood, it ensures that only the true bloodstream sample is cultured, leading to more accurate results.¹⁵ It also may minimize the risk of contamination-related delays in diagnosis and treatment and benefits patients and health care institutions by potentially reducing hospital stays, unnecessary antibiotic use, and health care costs.

One of the ISDT device manufacturers estimated the financial impact on OETVMC based on the pilot project.⁸ While this study did not calculate the direct and indirect cost savings associated with this process improvement, the manufacturer’s website suggests that VACTHCS could annually save about $486,000.¹⁶ Furthermore, implementation of ISDT may improve laboratory efficiency, as they reduce the workload associated with identifying and reporting false-positive cultures. ⁶ ISDT devices represent a valuable tool in the efforts to reduce blood culture contamination and its wide-ranging implications in clinical settings. While ISDT alone will not be sufficient in achieving a lower threshold (< 1%) of blood culture contamination, it can be part of a multiprong effort that optimizes best practices in the collection, handling, and management of blood cultures.

Continuous quality improvement efforts and monitoring of blood culture contamination rates can help health care institutions identify problem areas and implement necessary changes. Addressing blood culture contamination can improve patient care, reduce costs, and address antibiotic resistance.

Limitations

This study was limited by its study design, which did not use a side-by-side comparison of blood cultures from groups with and without ISDT. All blood cultures from patients in the region were processed at OETVMC, which may not be representative of non-VA EDs. Part of this study took place during the COVID-19 pandemic, which may have skewed data. Additionally, hospital data were collected from a veteran population in Central Texas, and the lack of demographic diversity may not be generalizable to the greater population.

CONCLUSIONS

The findings of this study suggest ISDT may be effective in reducing blood culture contamination rates in the high-risk ED environment, which aligns with previous research. ^5,14 The ISDT may help reduce blood culture contamination rates, improving the quality of patient care and reducing health care costs. MilCon-VA mandated that all VA facilities have blood culture contamination as a metric with a goal of blood culture contamination rates < 1%.⁸ However, achieving this goal remains a challenge. Further research and continuous quality improvement efforts are necessary to achieve it. Consistently achieving a contamination threshold of < 1% may require minimizing human error. An automated robotic venipuncture device, as recently designed and reported, may be necessary to reduce human error in blood draw and contamination.¹⁶

Blood cultures provide crucial evidence for diagnostic medicine, specifically aimed at identifying the presence of microbial infections in the bloodstream. Blood culturing is instrumental in diagnosing conditions such as sepsis, bacteremia, or fungemia, where the identification of the causative agent is necessary for targeted and effective treatment.¹

The process involves aseptically drawing blood into sterile culture bottles, minimizing the risk of contamination with well-established guidelines. These culture bottles contain specific growth media that support the replication of microorganisms if they are present. Once the blood specimen is collected, it incubates, allowing any potential pathogens to grow. Subsequent analysis and identification of these microorganisms enable health care professionals (HCPs) to prescribe appropriate antimicrobial therapies to treat specific infections, contributing to more effective and targeted patient care.²

The reliability of blood culture results depends on minimizing contamination risk, a challenge inherent in the procedure. Contamination can lead to false-positive results, potentially misguiding treatment.³ HCPs must adhere to strict aseptic techniques during blood draws, ensuring proper skin preparation with antiseptic solutions. The use of sterile equipment and avoiding prolonged tourniquet application helps maintain the integrity of the blood specimen. Timely inoculation of blood into culture bottles and careful handling are essential to mitigate contamination risk.² Regular training and reinforcement of proper techniques is important to uphold the accuracy of blood culture results and enhance the reliability of diagnoses and treatment decisions.³ Despite diligent contamination prevention efforts, health care systems struggle to maintain contamination rates below the 3.0% national benchmark set by the Clinical & Laboratory Standards Institute (CLSI).⁴

Blood culture contamination is a critical concern in clinical practice; it can lead to misdiagnosis, prolonged hospital stays, unnecessary antibiotic use, and increased health care costs.⁵ Monitoring blood culture contamination is integral to patient safety, avoiding inappropriate and potentially harmful treatment, providing efficient care, contributing to antibiotic stewardship, supporting cost efficiency, and maintaining quality assurance and clinical research practices for public health.⁶

The initial specimen diversion technique (ISDT) recently emerged as a potential strategy to reduce blood culture contamination rates. This technique involves diverting a small portion of the initial blood plus the skin plug from the hollow needle away from the primary collection site before filling the culture bottles. This process minimizes skin surface contaminants, providing a cleaner blood specimen for culturing.⁷

The ISDT was introduced as a result of historically elevated contamination rates.⁸ Despite implementing various mitigation methods, the US Department of Veterans Affairs (VA) Central Texas Healthcare System (VACTHCS) has struggled to meet the national benchmark of maintaining blood culture contamination < 3.0%. The VACTHCS is a 146-bed teaching hospital with about 30,000 annual visits at the Olin E. Teague Veterans Affairs Medical Center (OETVMC) emergency department (ED). VACTHCS conducted a 16-month pilot study using 2 commercially available ISDT devices and published the findings.⁸

The Military Construction, Veterans Affairs, and Related Agencies Appropriations Act, 2022 (MilCon-VA Act) committee report prioritized the reduction of blood culture contamination to < 1% to prevent health risks and harm to veterans undergoing blood testing for the diagnosis of sepsis.⁹ Because it had been 5 years since OETVMC began using an ISDT in the ED, the ISDT adaptation strategy for mitigating blood culture contamination was revisited per institution policy.

The objective of this quality improvement project was to analyze retrospective data to understand the long-term impact of ISDT use on blood culture contamination rates. We hypothesized that ISDT use would contribute to efforts to maintain OETVMC ED blood culture contamination rate below the national (3.0%) and VACTHCS (2.5%) thresholds. This project assessed the progress for reducing blood culture contamination compared with the pre-ISDT era.⁸

METHODS

This retrospective analysis compared the blood culture contamination rates 36 months before and after the introduction of the ISDT device at the OETVMC ED. The preimplementation period was from December 2014 through November 2017 (36 months) and the postimplementation period was December 2017 through November 2020 (36 months). Data were collected from the Department of Pathology and Microbiology blood culture records of all adult patients admitted to the hospital through the ED and required blood cultures for suspicion of infection. Protected health information and VA sensitive information were not collected: all data were deidentified. A total of 18,541 blood cultures were collected 36 months preimplementation and 14,865 blood cultures were collected up to 36 months postimplementation. For comparison purposes, a similar dataset was collected from patients’ blood samples drawn by phlebotomists in the laboratory, where there had been no previous issues with overcontamination; no ISDT devices were used in the collection of these samples.

Blood Culture Contamination Variable

Blood cultures were monitored using the BACT/ALERT 3D (bioMérieux) and subsequently BACT/ALERT VIRTUO (bioMérieux), with positive bottles characterized by VITEK MS Matrix Assisted Laser Desorption Ionization Time-of-Flight technology (bioMérieux) and automated susceptibility testing (VITEK 2 [bioMérieux]).¹⁰ In an updated review of blood culture contamination, the American Society for Microbiology used the College of American Pathologists' Q-Probes quality improvement studies as a guideline for classifying contamination. A sample was determined to be contaminated if ≥ 1 of the following organisms were found in only 1 bottle in a series of blood culture sets: coagulase-negative staphylococci, Micrococcus species, α-hemolytic viridans group streptococci, Corynebacterium species, Propionibacterium acnes, and Bacillus species.¹¹ The contamination assessment criteria remained unchanged, except for use of an ISDT device in blood culture collection at the ED.

The VACTHCS Infection Prevention Department ensured that the ISDT device was available and that ED nurses were trained annually on its use to collect blood cultures. Monthly reports of contamination were sent to the nursing supervisor for corrective action and retraining. The initial performance improvement project was slated for 16 months but was expanded to a 6-year period of retrospective data to obtain strong correlation.

Statistical Analysis

Contamination rates were recorded monthly from the hospital laboratory information management system for 36 months both before and after ISDT adoption. Statistical analysis was performed using a 2-tailed unpaired t-test to compare monthly contamination rates for the 2 periods with GraphPad Prism version 10.0.0 for Windows.

RESULTS

Prior to 2017, the ED reported contamination rates above the national (3.0%) and OETVMC thresholds (2.5%), with a mean of 4.5% (95% CI, 3.90-4.90).⁸ After ISDT implementation, the ED showed significant improvement with a reduction to mean 2.6% (95% CI, 2.10-3.20) (P < .001) (Figure 1). Figure 2 shows monthly blood culture contamination rates at the ED from December 2014 through November 2020. Month 36 (November 2017) shows a clear dip in contamination rate when the ISDT was introduced and month 37 to month 44 show remarkably low contamination rates. During this time, the institute experimented with 2 ISDT devices, and closer scrutiny may reveal this period as an outlier due to the monitoring of ISDT application, as previously reported.⁸

The blood culture contamination rate for samples drawn by the phlebotomists in the laboratory (excluding the ED) was calculated during the same time period (Figure 3). Non-ED contamination rates remained below 2.5% for 69 of 72 months.

DISCUSSION

The blood culture contamination rate in the OETVMC ED dropped following ISDT implementation and continued to show long-term benefits. For the 36-month period following ISDT implementation, the mean contamination rate was 2.6%, which was below the national target threshold of 3.0% and close to the OETVMC target of 2.5%. These results suggest that ISDT can have a positive impact on patient care and laboratory efficiency. Improvements in the blood contamination rates in the ED can have a positive impact on the overall hospital contamination rates.

Blood drawn by phlebotomists in the hospital laboratory infrequently had contamination rates that exceeded the 2.5% target threshold. Because the non-ED contamination rates did not change throughout the comparison period, other factors were likely not involved in the improvements seen in the ED. The decision to implement ISDT exclusively in the ED was based on its historically elevated contamination rate.⁸ Issues with blood culture contamination in EDs across various hospital systems are well documented and not unique to VACTHCS.¹²

Contamination in blood cultures can be a significant issue in the hospital. It occurs when microorganisms from the skin or environment enter the blood culture sample during collection. Moreover, it can contribute to antibiotic resistance when patients are prescribed inappropriate antibiotics. It is also important to ensure HCPs are well-trained and consistently follow standardized protocols and understand the implications of false-positive results.¹³

ISDT helps reduce false-positive results and is a significant advancement in the field of blood culture collection.^8,14 By discarding the initial blood, it ensures that only the true bloodstream sample is cultured, leading to more accurate results.¹⁵ It also may minimize the risk of contamination-related delays in diagnosis and treatment and benefits patients and health care institutions by potentially reducing hospital stays, unnecessary antibiotic use, and health care costs.

One of the ISDT device manufacturers estimated the financial impact on OETVMC based on the pilot project.⁸ While this study did not calculate the direct and indirect cost savings associated with this process improvement, the manufacturer’s website suggests that VACTHCS could annually save about $486,000.¹⁶ Furthermore, implementation of ISDT may improve laboratory efficiency, as they reduce the workload associated with identifying and reporting false-positive cultures. ⁶ ISDT devices represent a valuable tool in the efforts to reduce blood culture contamination and its wide-ranging implications in clinical settings. While ISDT alone will not be sufficient in achieving a lower threshold (< 1%) of blood culture contamination, it can be part of a multiprong effort that optimizes best practices in the collection, handling, and management of blood cultures.

Continuous quality improvement efforts and monitoring of blood culture contamination rates can help health care institutions identify problem areas and implement necessary changes. Addressing blood culture contamination can improve patient care, reduce costs, and address antibiotic resistance.

Limitations

This study was limited by its study design, which did not use a side-by-side comparison of blood cultures from groups with and without ISDT. All blood cultures from patients in the region were processed at OETVMC, which may not be representative of non-VA EDs. Part of this study took place during the COVID-19 pandemic, which may have skewed data. Additionally, hospital data were collected from a veteran population in Central Texas, and the lack of demographic diversity may not be generalizable to the greater population.

CONCLUSIONS

The findings of this study suggest ISDT may be effective in reducing blood culture contamination rates in the high-risk ED environment, which aligns with previous research. ^5,14 The ISDT may help reduce blood culture contamination rates, improving the quality of patient care and reducing health care costs. MilCon-VA mandated that all VA facilities have blood culture contamination as a metric with a goal of blood culture contamination rates < 1%.⁸ However, achieving this goal remains a challenge. Further research and continuous quality improvement efforts are necessary to achieve it. Consistently achieving a contamination threshold of < 1% may require minimizing human error. An automated robotic venipuncture device, as recently designed and reported, may be necessary to reduce human error in blood draw and contamination.¹⁶

To the Editor:

Online resources that are convenient and affordable play a crucial role in mitigating health inequality and improving patient access to health care information; however, the benefits are limited by the quality of information available, as medical misinformation can lead to patients engaging in harmful practices, making dangerous decisions, and even avoiding safe and effective treatments. In this study, we aimed to assess and compare the quality of patient guidance on dermatologic care generated by ChatGPT vs Reddit based on accuracy, appropriateness, and safety. It is essential to assess the quality and reliability of online health information to support patients in making informed decisions about their health.

The emergence and advancement of artificial intelligence and large language models such as ChatGPT present a new method for patients to access health care advice. ChatGPT can engage in conversation by accessing information from existing publicly available data on the internet, including books and websites, up to the year 2023 and providing humanlike responses with context.¹ ChatGPT’s access to a breadth of online evidence-based literature ensures the dissemination of quality information that is quick and without inherent bias, offering the potential to more closely align with health care professionals. ChatGPT’s use in dermatology by patients has shown efficacy, with a 98.87% approval rate by dermatologists scoring its ability to recommend appropriate medication for common dermatologic conditions.² However, ChatGPT has limitations when providing health care advice and has been observed to misunderstand health care standards, lack personalization, and offer incorrect references; currently, the latest publicly available version (ChatGPT 3.5) also is unable to analyze clinical images.^3,4

Reddit is an online social media forum that allows users to post questions and photographs to which anyone can reply and offer advice. Patients may find comfort in online communities where they can connect with others facing similar challenges related to their diagnosis. Within these communities, the responses often share users’ own lived experiences and offer support based on what has and has not worked for them. Prior research found that users intentionally seeking health information via Reddit are likely to implement the advice they receive even without verification of its credibility, suggesting a trust and receptibility to ideas offered on the platform.⁵ Furthermore, a study analyzing the dermatologic content of 17 dermatology related subreddits that had 1000 or more subscribers found that 70.6% of posts fell under the category of “seeking health/cosmetic advice.”⁶ Reddit users thus are vulnerable to receiving advice based on personal bias and exposing their health information to the public.

We hypothesized that ChatGPT would provide users with guidance that was more closely aligned with typical dermatologists’ advice due to its thorough analysis and compilation of diverse sources and recommendations available on the internet. We expected Reddit to yield recommendations of lesser quality and a diminished safety score, primarily due to the absence of credibility-vetting mechanisms and the influence of personal biases within the advice shared.

User-submitted posts to large dermatologic community Reddit forums representing a few of the most common skin conditions (r/eczema, r/acne, r/Folliculitis, r/SebDerm, r/Hidradenitis, r/keratosis, and r/Psoriasis) were retrospectively reviewed from January 2024 to March 2024. The most popular posts that did not include photographs were included in our study. Posts with photographs were excluded, as clinical images were not able to be uploaded to the publicly available ChatGPT 3.5. We collected real user questions about common skin conditions from Reddit forums and then asked ChatGPT to answer those same questions. We compared ChatGPT’s responses to the most upvoted Reddit comments to see how they matched up (eTable).

Each ChatGPT response and the top-rated Reddit comment were independently evaluated by a board certified dermatologist (S.A.) and a dermatology resident (A.H.K.). The quality of the ChatGPT and Reddit responses were determined by scoring the accuracy, appropriateness, safety consideration, and specificity on a 5-point Likert scale (1=low, 5=high). The 2 evaluators’ mean scores for each of the 4 categories were calculated based on adequate interrater reliability, which was tested using Cohen’s κ coefficient. Related-samples sign tests were used to compare ChatGPT and Reddit responses for each of the 4 categories. Analysis was completed using SPSS statistics software version 29.0 (IBM). The evaluators also were asked to provide qualitative feedback on the strengths and weaknesses of each response.

Our retrospective review yielded 20 total questions: 5 (25%) on atopic dermatitis, 4 (20%) on acne, 4 (20%) on hidradenitis suppurativa, 4 (20%) on psoriasis, 1 (5%) on folliculitis, 1 (5%) on keratosis pilaris, and 1 (5%) on seborrheic dermatitis. The number of posts was limited to 20 due to the extensive time required for grading each response. These 20 questions were selected from a larger pool of eligible posts based on factors such as clarity and relevance to common skin conditions. With regard to the types of questions that were asked, 6 (30%) were related to general management of a diagnosis, 5 (25%) were on treatment recommendations for symptom relief, 3 (15%) were on optimal utilization of current treatment regimens, 2 (10%) were on prescription side effects, 2 (10%) were on diagnosis presentation, 1 (5%) was on potential triggers of the diagnosis, and 1 (5%) was on natural treatment recommendations.

Mean (SD) evaluator scores for accuracy were significantly higher among ChatGPT responses compared with Reddit (4.63 [0.60] vs 2.60 [0.98])(P<.001). ChatGPT responses also were significantly higher for appropriateness compared with Reddit (4.55 [0.71] vs 2.58 [1.02])(P<.001) and safety consideration (4.88 [0.56] vs 2.80[0.97])(P <.001). There was no significant difference in mean specificity scores between ChatGPT and Reddit (4.25[1.02] vs 3.80 [0.70])(P=.096)(Figure).

For the Reddit responses, the weighted Cohen’s κ coefficient between the 2 evaluators was 0.200 (95% CI, –.089 to .489) for accuracy, 0.255 (95% CI, .014-.497) for appropriateness, 0.385 (95% CI, .176-.594) for safety consideration, and –0.024 (95% CI, –.177 to .129) for specificity. For the ChatGPT responses, the weighted Cohen’s κ coefficient between the 2 evaluators was 0.426 (95% CI, .122-.730) for accuracy, 0.571 (95% CI, .294-.849) for appropriateness, 0.655 (95% CI, .632-.678) for safety consideration, and 0.313 (95% CI, .043-.584) for specificity.

The strengths and weaknesses of the responses also were qualitatively analyzed. One commonly observed strength was ChatGPT’s frequent and appropriate recommendation for users to consult a dermatologist. In the case of atopic dermatitis—one of the more frequently asked about conditions—ChatGPT consistently emphasized evidence-based strategies such as gentle skin care and moisturization, reflecting alignment with clinical guidelines. Additionally, a common weakness of both ChatGPT and Reddit responses generally was the lack of personalized guidance and comprehensive discussion of the risks and benefits of specific treatments. It also was noted that neither platform consistently explored differential diagnoses—for example, distinguishing atopic dermatitis from conditions such as allergic contact dermatitis—limiting the diagnostic depth of the responses.

ChatGPT and Reddit can provide patients with quick and accessible health information for various dermatologic concerns. The results of our study demonstrated a significantly higher level of accuracy, appropriateness, and safety of responses generated by ChatGPT compared with human-generated responses on Reddit (P<.001). Both platforms offered similarly specific responses to user inquiries, demonstrating ChatGPT’s ability to comprehend user questions and draw from publicly available texts and Reddit users’ contributing insights based on their own first-hand experiences.

Reddit’s dermatologic forums often feature personal anecdotes and unique treatments described by individual users. Although specific to particular dermatologic concerns, such advice lacks an evidence-based standard of care. With the noted inherent trust of patients seeking guidance within Reddit communities, patients may follow unhelpful or potentially dangerous medical advice.⁵ A study examining 300 user-submitted posts on popular Reddit dermatology forums during the COVID-19 pandemic found that the mean scores for top-rated comments’ potential to be misleading or dangerous was 2.33 out of 5 on a Likert scale (95% CI, 2.18- 2.48).⁷ Dermatologists should be aware of the potential risks associated with dermatologic advice offered on Reddit and should caution patients against relying solely on this information without consulting a qualified dermatologist first.

Reddit’s open-forum design provides licensed dermatologists with the opportunity to disseminate evidence based information regarding dermatologic conditions. Currently, there is a subreddit (r/AskDocs) that allows users to post medical questions that can be answered by moderator-verified physicians. Participation from dermatologists in online communities such as this can improve the quality of dermatologic information shared online, combat misinformation, and promote safe skin care practices.

ChatGPT offers more accurate, appropriate, and safe information compared to Reddit responses, but its answers lack personalization. In a clinical setting, a personalized treatment plan from a physician can be tailored with a comprehensive discussion of the risks and benefits. Further, clinical settings allow for diagnosis and confirmation via biopsy and meticulous history taking to ensure that the diagnosis and treatment plan are accurate. While ChatGPT may be an option for seeking basic advice on dermatologic conditions, a licensed dermatologist should always be consulted for proper medical advice. Services such as telehealth may be another option to for patients with limited access to care.

Since ChatGPT 3.5 does not support the ability to upload images, our study acknowledges a limitation regarding the inclusion of Reddit posts containing photographs. Images can improve the response quality from both Reddit users and ChatGPT. While the updated ChatGPT 4o is capable of processing images, it requires a monthly subscription fee. The free version was chosen for use in this study, as this may reflect the most likely version that patients of low socioeconomic status would utilize to access dermatologic care; however, there is potential for growth and improvement of ChatGPT’s capability in providing medical advice.

This study compared the strengths and limitations of ChatGPT’s and Reddit’s responses to common dermatologic inquiries. ChatGPT and Reddit both show potential to be helpful sources of dermatologic health information; however, their current versions have many limitations and require caution and careful examination by patients of the guidance provided. Clinicians should be aware of these limitations when advising patients and emphasize the importance of consulting a licensed dermatologist for personalized, evidence-based care. For the best medical advice, it is always advisable to consult with a licensed dermatologist.

To the Editor:

Online resources that are convenient and affordable play a crucial role in mitigating health inequality and improving patient access to health care information; however, the benefits are limited by the quality of information available, as medical misinformation can lead to patients engaging in harmful practices, making dangerous decisions, and even avoiding safe and effective treatments. In this study, we aimed to assess and compare the quality of patient guidance on dermatologic care generated by ChatGPT vs Reddit based on accuracy, appropriateness, and safety. It is essential to assess the quality and reliability of online health information to support patients in making informed decisions about their health.

The emergence and advancement of artificial intelligence and large language models such as ChatGPT present a new method for patients to access health care advice. ChatGPT can engage in conversation by accessing information from existing publicly available data on the internet, including books and websites, up to the year 2023 and providing humanlike responses with context.¹ ChatGPT’s access to a breadth of online evidence-based literature ensures the dissemination of quality information that is quick and without inherent bias, offering the potential to more closely align with health care professionals. ChatGPT’s use in dermatology by patients has shown efficacy, with a 98.87% approval rate by dermatologists scoring its ability to recommend appropriate medication for common dermatologic conditions.² However, ChatGPT has limitations when providing health care advice and has been observed to misunderstand health care standards, lack personalization, and offer incorrect references; currently, the latest publicly available version (ChatGPT 3.5) also is unable to analyze clinical images.^3,4

Reddit is an online social media forum that allows users to post questions and photographs to which anyone can reply and offer advice. Patients may find comfort in online communities where they can connect with others facing similar challenges related to their diagnosis. Within these communities, the responses often share users’ own lived experiences and offer support based on what has and has not worked for them. Prior research found that users intentionally seeking health information via Reddit are likely to implement the advice they receive even without verification of its credibility, suggesting a trust and receptibility to ideas offered on the platform.⁵ Furthermore, a study analyzing the dermatologic content of 17 dermatology related subreddits that had 1000 or more subscribers found that 70.6% of posts fell under the category of “seeking health/cosmetic advice.”⁶ Reddit users thus are vulnerable to receiving advice based on personal bias and exposing their health information to the public.

We hypothesized that ChatGPT would provide users with guidance that was more closely aligned with typical dermatologists’ advice due to its thorough analysis and compilation of diverse sources and recommendations available on the internet. We expected Reddit to yield recommendations of lesser quality and a diminished safety score, primarily due to the absence of credibility-vetting mechanisms and the influence of personal biases within the advice shared.

User-submitted posts to large dermatologic community Reddit forums representing a few of the most common skin conditions (r/eczema, r/acne, r/Folliculitis, r/SebDerm, r/Hidradenitis, r/keratosis, and r/Psoriasis) were retrospectively reviewed from January 2024 to March 2024. The most popular posts that did not include photographs were included in our study. Posts with photographs were excluded, as clinical images were not able to be uploaded to the publicly available ChatGPT 3.5. We collected real user questions about common skin conditions from Reddit forums and then asked ChatGPT to answer those same questions. We compared ChatGPT’s responses to the most upvoted Reddit comments to see how they matched up (eTable).

Each ChatGPT response and the top-rated Reddit comment were independently evaluated by a board certified dermatologist (S.A.) and a dermatology resident (A.H.K.). The quality of the ChatGPT and Reddit responses were determined by scoring the accuracy, appropriateness, safety consideration, and specificity on a 5-point Likert scale (1=low, 5=high). The 2 evaluators’ mean scores for each of the 4 categories were calculated based on adequate interrater reliability, which was tested using Cohen’s κ coefficient. Related-samples sign tests were used to compare ChatGPT and Reddit responses for each of the 4 categories. Analysis was completed using SPSS statistics software version 29.0 (IBM). The evaluators also were asked to provide qualitative feedback on the strengths and weaknesses of each response.

Our retrospective review yielded 20 total questions: 5 (25%) on atopic dermatitis, 4 (20%) on acne, 4 (20%) on hidradenitis suppurativa, 4 (20%) on psoriasis, 1 (5%) on folliculitis, 1 (5%) on keratosis pilaris, and 1 (5%) on seborrheic dermatitis. The number of posts was limited to 20 due to the extensive time required for grading each response. These 20 questions were selected from a larger pool of eligible posts based on factors such as clarity and relevance to common skin conditions. With regard to the types of questions that were asked, 6 (30%) were related to general management of a diagnosis, 5 (25%) were on treatment recommendations for symptom relief, 3 (15%) were on optimal utilization of current treatment regimens, 2 (10%) were on prescription side effects, 2 (10%) were on diagnosis presentation, 1 (5%) was on potential triggers of the diagnosis, and 1 (5%) was on natural treatment recommendations.

Mean (SD) evaluator scores for accuracy were significantly higher among ChatGPT responses compared with Reddit (4.63 [0.60] vs 2.60 [0.98])(P<.001). ChatGPT responses also were significantly higher for appropriateness compared with Reddit (4.55 [0.71] vs 2.58 [1.02])(P<.001) and safety consideration (4.88 [0.56] vs 2.80[0.97])(P <.001). There was no significant difference in mean specificity scores between ChatGPT and Reddit (4.25[1.02] vs 3.80 [0.70])(P=.096)(Figure).

For the Reddit responses, the weighted Cohen’s κ coefficient between the 2 evaluators was 0.200 (95% CI, –.089 to .489) for accuracy, 0.255 (95% CI, .014-.497) for appropriateness, 0.385 (95% CI, .176-.594) for safety consideration, and –0.024 (95% CI, –.177 to .129) for specificity. For the ChatGPT responses, the weighted Cohen’s κ coefficient between the 2 evaluators was 0.426 (95% CI, .122-.730) for accuracy, 0.571 (95% CI, .294-.849) for appropriateness, 0.655 (95% CI, .632-.678) for safety consideration, and 0.313 (95% CI, .043-.584) for specificity.

The strengths and weaknesses of the responses also were qualitatively analyzed. One commonly observed strength was ChatGPT’s frequent and appropriate recommendation for users to consult a dermatologist. In the case of atopic dermatitis—one of the more frequently asked about conditions—ChatGPT consistently emphasized evidence-based strategies such as gentle skin care and moisturization, reflecting alignment with clinical guidelines. Additionally, a common weakness of both ChatGPT and Reddit responses generally was the lack of personalized guidance and comprehensive discussion of the risks and benefits of specific treatments. It also was noted that neither platform consistently explored differential diagnoses—for example, distinguishing atopic dermatitis from conditions such as allergic contact dermatitis—limiting the diagnostic depth of the responses.

ChatGPT and Reddit can provide patients with quick and accessible health information for various dermatologic concerns. The results of our study demonstrated a significantly higher level of accuracy, appropriateness, and safety of responses generated by ChatGPT compared with human-generated responses on Reddit (P<.001). Both platforms offered similarly specific responses to user inquiries, demonstrating ChatGPT’s ability to comprehend user questions and draw from publicly available texts and Reddit users’ contributing insights based on their own first-hand experiences.

Reddit’s dermatologic forums often feature personal anecdotes and unique treatments described by individual users. Although specific to particular dermatologic concerns, such advice lacks an evidence-based standard of care. With the noted inherent trust of patients seeking guidance within Reddit communities, patients may follow unhelpful or potentially dangerous medical advice.⁵ A study examining 300 user-submitted posts on popular Reddit dermatology forums during the COVID-19 pandemic found that the mean scores for top-rated comments’ potential to be misleading or dangerous was 2.33 out of 5 on a Likert scale (95% CI, 2.18- 2.48).⁷ Dermatologists should be aware of the potential risks associated with dermatologic advice offered on Reddit and should caution patients against relying solely on this information without consulting a qualified dermatologist first.

Reddit’s open-forum design provides licensed dermatologists with the opportunity to disseminate evidence based information regarding dermatologic conditions. Currently, there is a subreddit (r/AskDocs) that allows users to post medical questions that can be answered by moderator-verified physicians. Participation from dermatologists in online communities such as this can improve the quality of dermatologic information shared online, combat misinformation, and promote safe skin care practices.

ChatGPT offers more accurate, appropriate, and safe information compared to Reddit responses, but its answers lack personalization. In a clinical setting, a personalized treatment plan from a physician can be tailored with a comprehensive discussion of the risks and benefits. Further, clinical settings allow for diagnosis and confirmation via biopsy and meticulous history taking to ensure that the diagnosis and treatment plan are accurate. While ChatGPT may be an option for seeking basic advice on dermatologic conditions, a licensed dermatologist should always be consulted for proper medical advice. Services such as telehealth may be another option to for patients with limited access to care.

Since ChatGPT 3.5 does not support the ability to upload images, our study acknowledges a limitation regarding the inclusion of Reddit posts containing photographs. Images can improve the response quality from both Reddit users and ChatGPT. While the updated ChatGPT 4o is capable of processing images, it requires a monthly subscription fee. The free version was chosen for use in this study, as this may reflect the most likely version that patients of low socioeconomic status would utilize to access dermatologic care; however, there is potential for growth and improvement of ChatGPT’s capability in providing medical advice.

This study compared the strengths and limitations of ChatGPT’s and Reddit’s responses to common dermatologic inquiries. ChatGPT and Reddit both show potential to be helpful sources of dermatologic health information; however, their current versions have many limitations and require caution and careful examination by patients of the guidance provided. Clinicians should be aware of these limitations when advising patients and emphasize the importance of consulting a licensed dermatologist for personalized, evidence-based care. For the best medical advice, it is always advisable to consult with a licensed dermatologist.

To the Editor:

Online resources that are convenient and affordable play a crucial role in mitigating health inequality and improving patient access to health care information; however, the benefits are limited by the quality of information available, as medical misinformation can lead to patients engaging in harmful practices, making dangerous decisions, and even avoiding safe and effective treatments. In this study, we aimed to assess and compare the quality of patient guidance on dermatologic care generated by ChatGPT vs Reddit based on accuracy, appropriateness, and safety. It is essential to assess the quality and reliability of online health information to support patients in making informed decisions about their health.

The emergence and advancement of artificial intelligence and large language models such as ChatGPT present a new method for patients to access health care advice. ChatGPT can engage in conversation by accessing information from existing publicly available data on the internet, including books and websites, up to the year 2023 and providing humanlike responses with context.¹ ChatGPT’s access to a breadth of online evidence-based literature ensures the dissemination of quality information that is quick and without inherent bias, offering the potential to more closely align with health care professionals. ChatGPT’s use in dermatology by patients has shown efficacy, with a 98.87% approval rate by dermatologists scoring its ability to recommend appropriate medication for common dermatologic conditions.² However, ChatGPT has limitations when providing health care advice and has been observed to misunderstand health care standards, lack personalization, and offer incorrect references; currently, the latest publicly available version (ChatGPT 3.5) also is unable to analyze clinical images.^3,4

Reddit is an online social media forum that allows users to post questions and photographs to which anyone can reply and offer advice. Patients may find comfort in online communities where they can connect with others facing similar challenges related to their diagnosis. Within these communities, the responses often share users’ own lived experiences and offer support based on what has and has not worked for them. Prior research found that users intentionally seeking health information via Reddit are likely to implement the advice they receive even without verification of its credibility, suggesting a trust and receptibility to ideas offered on the platform.⁵ Furthermore, a study analyzing the dermatologic content of 17 dermatology related subreddits that had 1000 or more subscribers found that 70.6% of posts fell under the category of “seeking health/cosmetic advice.”⁶ Reddit users thus are vulnerable to receiving advice based on personal bias and exposing their health information to the public.

We hypothesized that ChatGPT would provide users with guidance that was more closely aligned with typical dermatologists’ advice due to its thorough analysis and compilation of diverse sources and recommendations available on the internet. We expected Reddit to yield recommendations of lesser quality and a diminished safety score, primarily due to the absence of credibility-vetting mechanisms and the influence of personal biases within the advice shared.

User-submitted posts to large dermatologic community Reddit forums representing a few of the most common skin conditions (r/eczema, r/acne, r/Folliculitis, r/SebDerm, r/Hidradenitis, r/keratosis, and r/Psoriasis) were retrospectively reviewed from January 2024 to March 2024. The most popular posts that did not include photographs were included in our study. Posts with photographs were excluded, as clinical images were not able to be uploaded to the publicly available ChatGPT 3.5. We collected real user questions about common skin conditions from Reddit forums and then asked ChatGPT to answer those same questions. We compared ChatGPT’s responses to the most upvoted Reddit comments to see how they matched up (eTable).

Each ChatGPT response and the top-rated Reddit comment were independently evaluated by a board certified dermatologist (S.A.) and a dermatology resident (A.H.K.). The quality of the ChatGPT and Reddit responses were determined by scoring the accuracy, appropriateness, safety consideration, and specificity on a 5-point Likert scale (1=low, 5=high). The 2 evaluators’ mean scores for each of the 4 categories were calculated based on adequate interrater reliability, which was tested using Cohen’s κ coefficient. Related-samples sign tests were used to compare ChatGPT and Reddit responses for each of the 4 categories. Analysis was completed using SPSS statistics software version 29.0 (IBM). The evaluators also were asked to provide qualitative feedback on the strengths and weaknesses of each response.

Our retrospective review yielded 20 total questions: 5 (25%) on atopic dermatitis, 4 (20%) on acne, 4 (20%) on hidradenitis suppurativa, 4 (20%) on psoriasis, 1 (5%) on folliculitis, 1 (5%) on keratosis pilaris, and 1 (5%) on seborrheic dermatitis. The number of posts was limited to 20 due to the extensive time required for grading each response. These 20 questions were selected from a larger pool of eligible posts based on factors such as clarity and relevance to common skin conditions. With regard to the types of questions that were asked, 6 (30%) were related to general management of a diagnosis, 5 (25%) were on treatment recommendations for symptom relief, 3 (15%) were on optimal utilization of current treatment regimens, 2 (10%) were on prescription side effects, 2 (10%) were on diagnosis presentation, 1 (5%) was on potential triggers of the diagnosis, and 1 (5%) was on natural treatment recommendations.

Mean (SD) evaluator scores for accuracy were significantly higher among ChatGPT responses compared with Reddit (4.63 [0.60] vs 2.60 [0.98])(P<.001). ChatGPT responses also were significantly higher for appropriateness compared with Reddit (4.55 [0.71] vs 2.58 [1.02])(P<.001) and safety consideration (4.88 [0.56] vs 2.80[0.97])(P <.001). There was no significant difference in mean specificity scores between ChatGPT and Reddit (4.25[1.02] vs 3.80 [0.70])(P=.096)(Figure).

For the Reddit responses, the weighted Cohen’s κ coefficient between the 2 evaluators was 0.200 (95% CI, –.089 to .489) for accuracy, 0.255 (95% CI, .014-.497) for appropriateness, 0.385 (95% CI, .176-.594) for safety consideration, and –0.024 (95% CI, –.177 to .129) for specificity. For the ChatGPT responses, the weighted Cohen’s κ coefficient between the 2 evaluators was 0.426 (95% CI, .122-.730) for accuracy, 0.571 (95% CI, .294-.849) for appropriateness, 0.655 (95% CI, .632-.678) for safety consideration, and 0.313 (95% CI, .043-.584) for specificity.

The strengths and weaknesses of the responses also were qualitatively analyzed. One commonly observed strength was ChatGPT’s frequent and appropriate recommendation for users to consult a dermatologist. In the case of atopic dermatitis—one of the more frequently asked about conditions—ChatGPT consistently emphasized evidence-based strategies such as gentle skin care and moisturization, reflecting alignment with clinical guidelines. Additionally, a common weakness of both ChatGPT and Reddit responses generally was the lack of personalized guidance and comprehensive discussion of the risks and benefits of specific treatments. It also was noted that neither platform consistently explored differential diagnoses—for example, distinguishing atopic dermatitis from conditions such as allergic contact dermatitis—limiting the diagnostic depth of the responses.

ChatGPT and Reddit can provide patients with quick and accessible health information for various dermatologic concerns. The results of our study demonstrated a significantly higher level of accuracy, appropriateness, and safety of responses generated by ChatGPT compared with human-generated responses on Reddit (P<.001). Both platforms offered similarly specific responses to user inquiries, demonstrating ChatGPT’s ability to comprehend user questions and draw from publicly available texts and Reddit users’ contributing insights based on their own first-hand experiences.

Reddit’s dermatologic forums often feature personal anecdotes and unique treatments described by individual users. Although specific to particular dermatologic concerns, such advice lacks an evidence-based standard of care. With the noted inherent trust of patients seeking guidance within Reddit communities, patients may follow unhelpful or potentially dangerous medical advice.⁵ A study examining 300 user-submitted posts on popular Reddit dermatology forums during the COVID-19 pandemic found that the mean scores for top-rated comments’ potential to be misleading or dangerous was 2.33 out of 5 on a Likert scale (95% CI, 2.18- 2.48).⁷ Dermatologists should be aware of the potential risks associated with dermatologic advice offered on Reddit and should caution patients against relying solely on this information without consulting a qualified dermatologist first.

Reddit’s open-forum design provides licensed dermatologists with the opportunity to disseminate evidence based information regarding dermatologic conditions. Currently, there is a subreddit (r/AskDocs) that allows users to post medical questions that can be answered by moderator-verified physicians. Participation from dermatologists in online communities such as this can improve the quality of dermatologic information shared online, combat misinformation, and promote safe skin care practices.

ChatGPT offers more accurate, appropriate, and safe information compared to Reddit responses, but its answers lack personalization. In a clinical setting, a personalized treatment plan from a physician can be tailored with a comprehensive discussion of the risks and benefits. Further, clinical settings allow for diagnosis and confirmation via biopsy and meticulous history taking to ensure that the diagnosis and treatment plan are accurate. While ChatGPT may be an option for seeking basic advice on dermatologic conditions, a licensed dermatologist should always be consulted for proper medical advice. Services such as telehealth may be another option to for patients with limited access to care.

Since ChatGPT 3.5 does not support the ability to upload images, our study acknowledges a limitation regarding the inclusion of Reddit posts containing photographs. Images can improve the response quality from both Reddit users and ChatGPT. While the updated ChatGPT 4o is capable of processing images, it requires a monthly subscription fee. The free version was chosen for use in this study, as this may reflect the most likely version that patients of low socioeconomic status would utilize to access dermatologic care; however, there is potential for growth and improvement of ChatGPT’s capability in providing medical advice.

This study compared the strengths and limitations of ChatGPT’s and Reddit’s responses to common dermatologic inquiries. ChatGPT and Reddit both show potential to be helpful sources of dermatologic health information; however, their current versions have many limitations and require caution and careful examination by patients of the guidance provided. Clinicians should be aware of these limitations when advising patients and emphasize the importance of consulting a licensed dermatologist for personalized, evidence-based care. For the best medical advice, it is always advisable to consult with a licensed dermatologist.

Excessive alcohol use is one of the leading preventable causes of death in the United States, responsible for about 178,000 deaths annually and an average of 488 daily deaths in 2020 and 2021.¹Alcohol-related deaths increased by 49% between 2006 and 2019.² This trend continued during the COVID-19 pandemic, with death certificates that listed alcohol increasing by > 25% from 2019 to 2020, and another 10% in 2021.³ This increase of alcohol-related deaths includes those as a direct result of chronic alcohol use, such as alcoholic cardiomyopathy, alcoholic hepatitis and cirrhosis, and alcohol-induced pancreatitis, as well as a result of acute use such as alcohol poisoning, suicide by exposure to alcohol, and alcohol-impaired driving fatalities.⁴

Excessive alcohol consumption poses other serious risks, including cases when intake is abruptly reduced without proper management. Alcohol withdrawal syndrome (AWS) can vary in severity, with potentially life-threatening complications such as hallucinations, seizures, and delirium tremens.⁵

These risks highlight the importance of professional intervention and support, not only to mitigate risks associated with AWS, but provide a pathway towards recovery from alcohol use disorder (AUD).

According to the 2022 National Survey on Drug Use and Health, 28.8 million US adults had AUD in the prior year, yet only 7.6% of these individuals received treatment and an even smaller group (2.2%) received medication-assisted treatment for alcohol.^6,7 This is despite American Psychiatric Association guidelines for the pharmacological treatment of patients with AUD, including the use of naltrexone, acamprosate, disulfiram, topiramate, or gabapentin, depending on therapy goals, past medication trials, medication contraindications, and patient preference.⁸ Several of these medications are approved by the US Food and Drug Administration (FDA) for the treatment of AUD and have support for effectiveness from randomized controlled trials and meta-analyses.^9-11

Clinical practice guidelines for the management of substance use disorders (SUDs) from the US Department of Veterans Affairs (VA) and US Department of Defense have strong recommendations for naltrexone and topiramate as first-line pharmacotherapies for moderate to severe AUD. Acamprosate and disulfiram are weak recommendations as alternative options. Gabapentin is a weak recommendation for cases where first-line treatments are contraindicated or ineffective. The guidelines emphasize the importance of a comprehensive approach to AUD treatment, including psychosocial interventions in addition to pharmacotherapy.¹²

A 2023 national survey found veterans reported higher alcohol consumption than nonveterans.¹³ At the end of fiscal year 2023, > 4.4 million veterans—6% of Veterans Health Administration patients—had been diagnosed with AUD.14 However, > 87% of these patients nationally, and 88% of Veterans Integrated Service Network (VISN) 21 patients, were not receiving naltrexone, acamprosate, disulfiram, or topiramate as part of their treatment. The VA Academic Detailing Service (ADS) now includes AUD pharmacotherapy as a campaign focus, highlighting its importance. The ADS is a pharmacy educational outreach program that uses unbiased clinical guidelines to promote aligning prescribing behavior with best practices. Academic detailing methods include speaking with health care practitioners (HCPs), and direct-to-consumer (DTC) patient education.

ADS campaigns include DTC educational handouts. Past ADS projects and research using DTC have demonstrated a significant improvement in outcomes and positively influencing patients’ pharmacotherapy treatment. ^15,16 A VA quality improvement project found a positive correlation between the initiation of AUD pharmacotherapy and engagement with mental health care following the distribution of AUD DTC patient education. ¹⁷ This project aimed to apply the same principles of prior research to explore the use of DTC across multiple facilities within VISN 21 to increase AUD pharmacotherapy. VISN 21 includes VA facilities and clinics across the Pacific Islands, Nevada, and California and serves about 350,000 veterans.

METHODS

A prospective cohort of VISN 21 veterans with or at high risk for AUD was identified using the VA ADS AUD Dashboard. The cohort included those not on acamprosate, disulfiram, naltrexone, topiramate, or gabapentin for treatment of AUD and had an elevated Alcohol Use Disorder Identification Test-Consumption (AUDIT-C) score of ≥ 6 (high risk) with an AUD diagnosis or ≥ 8 (severe risk) without a diagnosis. The AUDIT-C scores used in the dashboard are supported by the VA AUD clinician guide as the minimum scores when AUD pharmacotherapy should be offered to patients.¹⁸ Prescriptions filled outside the VA were not included in this dashboard.

Data and patient information were collected using the VA Corporate Data Warehouse. To be eligible, veterans needed a valid mailing address within the VISN 21 region and a primary care, mental health, or SUD clinician prescriber visit scheduled between October 1, 2023, and January 31, 2024. Veterans were excluded if they were in hospice, had a 1-year mortality risk score > 50% based on their Care Assessment Need (CAN) score, or facility leadership opted out of project involvement. Patients with both severe renal and hepatic impairments were excluded because they were ineligible for AUD pharmacotherapy. However, veterans with either renal or hepatic impairment (but not both) were included, as they could be potential candidates for ≥ 1 AUD pharmacotherapy option.

Initial correspondence with facilities was initiated through local academic detailers. A local champion was identified for the 1 facility without an academic detailer. Facilities could opt in or out of the project. Approval was provided by the local pharmacy and therapeutics committee, pharmacy, primary care, or psychiatry leadership. Approval process and clinician involvement varied by site.

Education

The selected AUD patient education was designed and approved by the national VA ADS (eappendix). The DTC patient education provided general knowledge about alcohol, including what constitutes a standard amount of alcohol, what is considered heavy drinking, risks of heavy drinking, creating a plan with a clinician to reduce and manage withdrawal symptoms, and additional resources. The DTC was accompanied by a cover letter that included a local facility contact number.

A centralized mailing facility was used for all materials. VA Northern California Health Care System provided the funding to cover the cost of postage. The list of veterans to be contacted was updated on a rolling basis and DTC education was mailed 2 weeks prior to their scheduled prescriber visit.

The eligible cohort of 1260 veterans received DTC education. A comparator group of 2048 veterans that did not receive DTC education was obtained retrospectively by using the same inclusion and exclusion criteria with a scheduled primary care, mental health, or SUD HCP visit from October 1, 2022, to January 31, 2023. The outcomes assessed were within 30 days of the scheduled visit, with the primary outcome as the initiation of AUD-related pharmacotherapy and the secondary outcome as the placement of a consultation for mental health or SUD services. Any consultations sent to Behavioral Health, Addiction, Mental Health, Psychiatric, and SUD services following the HCP visit, within the specified time frame, were used for the secondary outcome.

Matching and Analysis

A 1-to-1 nearest neighbor propensity score (PS) matching without replacement was used to pair the 1260 veterans from the intervention group with similarly scored comparator group veterans for a PS-matched final dataset of 2520 veterans. The PS model was a multivariate logistic regression with the outcome being exposure and comparator group status. Baseline characteristics used in the PS model were age, birth sex, race, facility of care, baseline AUDIT-C score, and days between project start and scheduled appointment. Covariate imbalance for the PS-matched sample was assessed to ensure the standardized mean difference for all covariates fell under a 0.1 threshold (Figure).¹⁹

A frequency table was provided to compare the discrete distributions of the baseline characteristics in the intervention and comparator groups. Logistic regression analysis was performed to evaluate the association between DTC education exposure and pharmacotherapy initiation, while controlling for potential confounders. Univariate and multivariate P value results for each variable included in the model were reported along with the multivariate odds ratios (ORs) and their associated 95% CIs. Logistic regression analyses were run for both outcomes. Each model included the exposure and comparator group status as well as the baseline characteristics included in the PS model. Statistical significance was set at P < .05. All statistical analyses were performed with R version 4.2.1.

RESULTS

Two of 7 VISN 21 sites did not participate, and 3 had restrictions on participation. DTC education was mailed about 2 weeks prior to scheduled visit for 1260 veterans; 53.6% identified as White, 37.6% were aged 41 to 60 years, and 79.2% had an AUDIT-C ≥ 8 (Table 1). Of those mailed education, there were 173 no-show appointments (13.7%). Thirty-two veterans (2.5%) in the DTC group and 33 veterans (2.6%) in the comparator group received an AUD-related pharmacotherapy prescription (P = .88) (Table 2). One hundred seventy-one veterans (13.6%) in the DTC group and 160 veterans (12.7%) in the comparator group had a consult placed for mental health or SUD services within 30 days of their appointment (P = .59) (Table 3).

DISCUSSION

This project did not yield statistically significant differences in either the primary or secondary outcomes within the 30-day follow-up window and found limited impact from the DTC educational outreach to veterans. The percentage of veterans that received AUD-related pharmacotherapy or consultations for mental health or SUD services was similarly low in the DTC and comparator groups. These findings suggest that although DTC education may raise awareness, it may not be sufficient on its own to drive changes in prescribing behavior or referral patterns without system-level support.

Addiction is a complex disease faced with stigma and requiring readiness by both the HCP and patient to move forward in support and treatment. The consequences of stigma can be severe: the more stigma perceived by a person with AUD, the less likely they are to seek treatment.²⁰ Stigma may exist even within HCPs and may lead to compromised care including shortened visits, less engagement, and less empathy.¹⁹ Cultural attitude towards alcohol use and intoxication can also be influenced through a wide range of sources including social media, movies, music, and television. Studies have shown targeted alcohol marketing may result in the development of positive beliefs about drinking and expand environments where alcohol use is socially acceptable and encouraged.²¹ These factors can impact drinking behavior, including the onset of drinking, binge drinking, and increased alcohol consumption.²²

Three VISN 21 sites in this study had restrictions on or excluded primary care from participation. Leadership at some of these facilities were concerned that primary care teams did not have the bandwidth to take on additional items and/or there was variable primary care readiness for initiating AUD pharmacotherapy. Further attempts should be made to integrate primary care into the process of initiating AUD treatment as significant research suggests that integrated care models for AUD may be associated with improved process and outcome measures of care.²³

There are several differences between this quality improvement project and prior research investigating the impact of DTC education for other conditions, such as the EMPOWER randomized controlled trial and VISN 22 project, which both demonstrated effectiveness of DTC education for reducing benzodiazepine use in geriatric veterans. ^15,16 These studies focused on reducing or stopping pharmacotherapy use, whereas this project sought to promote the initiation of AUD pharmacotherapy. These studies evaluated outcomes at least 6 months postindex date, whereas this project evaluated outcomes within 30 days postappointment. Furthermore, the educational content varied significantly. Other projects provided patients with information focused on specific medications and interventions, such as benzodiazepine tapering, while this project mailed general information on heavy drinking, its risks, and strategies for cutting back, without mentioning pharmacotherapy. The DTC material used in this project was chosen because it was a preapproved national VA ADS resource, which expedited the project timeline by avoiding the need for additional approvals at each participating site. These differences may impact the observed effectiveness of DTC education in this project, especially regarding the primary outcome.

Strengths and Limitations

This quality improvement project sent a large sample of veterans DTC education in a clinical setting across multiple sites. Additionally, PS matching methods were used to balance covariates between the comparator and DTC education group, thereby simulating a randomized controlled trial and reducing selection bias. The project brought attention to the VISN 21 AUD treatment rates, stimulated conversation across sites about available treatments and resources for AUD, and sparked collaboration between academic detailing, mental health, and primary care services. The time frame for visits was selected during the winter; the National Institute on Alcohol Abuse and Alcoholism notes this is a time when people may be more likely to engage in excessive alcohol consumption than at other times of the year.²⁴

The 30-day time frame for outcomes may have been too short to observe changes in prescribing or referral patterns. Additionally, the comparator group was comprised of veterans seen from October 1, 2022, to January 31, 2023, where seasonal timing may have influenced alcohol consumption behaviors and skewed the results. There were also no-show appointments in the DTC education group (13.7%), though it is likely some patients rescheduled and still received AUD pharmacotherapy within 30 days of the original appointment. Finally, it was not possible to confirm whether a patient opened and read the education that was mailed to them. This may be another reason to explore electronic distribution of DTC education. This all may have contributed to the lack of statistically significant differences in both the primary and secondary outcomes.

There was a high level of variability between facility participation in the project. Two of 7 sites did not participate, and 3 sites restricted primary care engagement. This represents a significant limitation, particularly for the secondary outcome of placing consultations for MH or SUD services. Facilities that only included mental health or SUD HCPs may have resulted in lower consultation rates due to their inherent specialization, reducing the likelihood of self-referrals.

The project may overestimate prescribed AUD pharmacotherapy in the primary outcome due to potential misclassification of medications. While the project adhered to the national VA ADS AUD dashboard’s definition of AUD pharmacotherapy, including acamprosate, disulfiram, naltrexone, topiramate, and gabapentin, some of these medications have multiple indications. For example, gabapentin is commonly prescribed for peripheral neuropathy, and topiramate is used to treat migraines and seizures. The multipurpose use adds uncertainty about whether they were prescribed specifically for AUD treatment, especially in cases where the HCP is responsible for treating a broad range of disease states, as in primary care.

CONCLUSIONS

Results of this quality improvement project did not show a statistically significant difference between patients sent DTC education and the comparator group for the initiation of AUD pharmacotherapy or placement of a consult to mental health or SUD services within 30 days of their scheduled visit. Future studies may seek to implement stricter criteria to confirm the intended use of topiramate and gabapentin, such as looking for keywords in the prescription instructions for use, performing chart reviews, and/or only including these medications if prescribed by a mental health or SUD HCP. Alternatively, future studies may consider limiting the analysis to only FDA-approved AUD medications: acamprosate, disulfiram, and naltrexone. It is vital to continue to enhance primary care HCP readiness to treat AUD, given the existing relationships and trust they often have with patients. Electronic methods for distributing DTC education could also be advantageous, as these methods may have the ability to track whether a message has been opened and read. Despite a lack of statistical significance, this project sparked crucial conversations and collaboration around AUD, available treatments, and addressing potential barriers to connecting patients to care within VISN 21.

Excessive alcohol use is one of the leading preventable causes of death in the United States, responsible for about 178,000 deaths annually and an average of 488 daily deaths in 2020 and 2021.¹Alcohol-related deaths increased by 49% between 2006 and 2019.² This trend continued during the COVID-19 pandemic, with death certificates that listed alcohol increasing by > 25% from 2019 to 2020, and another 10% in 2021.³ This increase of alcohol-related deaths includes those as a direct result of chronic alcohol use, such as alcoholic cardiomyopathy, alcoholic hepatitis and cirrhosis, and alcohol-induced pancreatitis, as well as a result of acute use such as alcohol poisoning, suicide by exposure to alcohol, and alcohol-impaired driving fatalities.⁴

Excessive alcohol consumption poses other serious risks, including cases when intake is abruptly reduced without proper management. Alcohol withdrawal syndrome (AWS) can vary in severity, with potentially life-threatening complications such as hallucinations, seizures, and delirium tremens.⁵

These risks highlight the importance of professional intervention and support, not only to mitigate risks associated with AWS, but provide a pathway towards recovery from alcohol use disorder (AUD).

According to the 2022 National Survey on Drug Use and Health, 28.8 million US adults had AUD in the prior year, yet only 7.6% of these individuals received treatment and an even smaller group (2.2%) received medication-assisted treatment for alcohol.^6,7 This is despite American Psychiatric Association guidelines for the pharmacological treatment of patients with AUD, including the use of naltrexone, acamprosate, disulfiram, topiramate, or gabapentin, depending on therapy goals, past medication trials, medication contraindications, and patient preference.⁸ Several of these medications are approved by the US Food and Drug Administration (FDA) for the treatment of AUD and have support for effectiveness from randomized controlled trials and meta-analyses.^9-11

Clinical practice guidelines for the management of substance use disorders (SUDs) from the US Department of Veterans Affairs (VA) and US Department of Defense have strong recommendations for naltrexone and topiramate as first-line pharmacotherapies for moderate to severe AUD. Acamprosate and disulfiram are weak recommendations as alternative options. Gabapentin is a weak recommendation for cases where first-line treatments are contraindicated or ineffective. The guidelines emphasize the importance of a comprehensive approach to AUD treatment, including psychosocial interventions in addition to pharmacotherapy.¹²

A 2023 national survey found veterans reported higher alcohol consumption than nonveterans.¹³ At the end of fiscal year 2023, > 4.4 million veterans—6% of Veterans Health Administration patients—had been diagnosed with AUD.14 However, > 87% of these patients nationally, and 88% of Veterans Integrated Service Network (VISN) 21 patients, were not receiving naltrexone, acamprosate, disulfiram, or topiramate as part of their treatment. The VA Academic Detailing Service (ADS) now includes AUD pharmacotherapy as a campaign focus, highlighting its importance. The ADS is a pharmacy educational outreach program that uses unbiased clinical guidelines to promote aligning prescribing behavior with best practices. Academic detailing methods include speaking with health care practitioners (HCPs), and direct-to-consumer (DTC) patient education.

ADS campaigns include DTC educational handouts. Past ADS projects and research using DTC have demonstrated a significant improvement in outcomes and positively influencing patients’ pharmacotherapy treatment. ^15,16 A VA quality improvement project found a positive correlation between the initiation of AUD pharmacotherapy and engagement with mental health care following the distribution of AUD DTC patient education. ¹⁷ This project aimed to apply the same principles of prior research to explore the use of DTC across multiple facilities within VISN 21 to increase AUD pharmacotherapy. VISN 21 includes VA facilities and clinics across the Pacific Islands, Nevada, and California and serves about 350,000 veterans.

METHODS

A prospective cohort of VISN 21 veterans with or at high risk for AUD was identified using the VA ADS AUD Dashboard. The cohort included those not on acamprosate, disulfiram, naltrexone, topiramate, or gabapentin for treatment of AUD and had an elevated Alcohol Use Disorder Identification Test-Consumption (AUDIT-C) score of ≥ 6 (high risk) with an AUD diagnosis or ≥ 8 (severe risk) without a diagnosis. The AUDIT-C scores used in the dashboard are supported by the VA AUD clinician guide as the minimum scores when AUD pharmacotherapy should be offered to patients.¹⁸ Prescriptions filled outside the VA were not included in this dashboard.

Data and patient information were collected using the VA Corporate Data Warehouse. To be eligible, veterans needed a valid mailing address within the VISN 21 region and a primary care, mental health, or SUD clinician prescriber visit scheduled between October 1, 2023, and January 31, 2024. Veterans were excluded if they were in hospice, had a 1-year mortality risk score > 50% based on their Care Assessment Need (CAN) score, or facility leadership opted out of project involvement. Patients with both severe renal and hepatic impairments were excluded because they were ineligible for AUD pharmacotherapy. However, veterans with either renal or hepatic impairment (but not both) were included, as they could be potential candidates for ≥ 1 AUD pharmacotherapy option.

Initial correspondence with facilities was initiated through local academic detailers. A local champion was identified for the 1 facility without an academic detailer. Facilities could opt in or out of the project. Approval was provided by the local pharmacy and therapeutics committee, pharmacy, primary care, or psychiatry leadership. Approval process and clinician involvement varied by site.

Education

The selected AUD patient education was designed and approved by the national VA ADS (eappendix). The DTC patient education provided general knowledge about alcohol, including what constitutes a standard amount of alcohol, what is considered heavy drinking, risks of heavy drinking, creating a plan with a clinician to reduce and manage withdrawal symptoms, and additional resources. The DTC was accompanied by a cover letter that included a local facility contact number.

A centralized mailing facility was used for all materials. VA Northern California Health Care System provided the funding to cover the cost of postage. The list of veterans to be contacted was updated on a rolling basis and DTC education was mailed 2 weeks prior to their scheduled prescriber visit.

The eligible cohort of 1260 veterans received DTC education. A comparator group of 2048 veterans that did not receive DTC education was obtained retrospectively by using the same inclusion and exclusion criteria with a scheduled primary care, mental health, or SUD HCP visit from October 1, 2022, to January 31, 2023. The outcomes assessed were within 30 days of the scheduled visit, with the primary outcome as the initiation of AUD-related pharmacotherapy and the secondary outcome as the placement of a consultation for mental health or SUD services. Any consultations sent to Behavioral Health, Addiction, Mental Health, Psychiatric, and SUD services following the HCP visit, within the specified time frame, were used for the secondary outcome.

Matching and Analysis

A 1-to-1 nearest neighbor propensity score (PS) matching without replacement was used to pair the 1260 veterans from the intervention group with similarly scored comparator group veterans for a PS-matched final dataset of 2520 veterans. The PS model was a multivariate logistic regression with the outcome being exposure and comparator group status. Baseline characteristics used in the PS model were age, birth sex, race, facility of care, baseline AUDIT-C score, and days between project start and scheduled appointment. Covariate imbalance for the PS-matched sample was assessed to ensure the standardized mean difference for all covariates fell under a 0.1 threshold (Figure).¹⁹

A frequency table was provided to compare the discrete distributions of the baseline characteristics in the intervention and comparator groups. Logistic regression analysis was performed to evaluate the association between DTC education exposure and pharmacotherapy initiation, while controlling for potential confounders. Univariate and multivariate P value results for each variable included in the model were reported along with the multivariate odds ratios (ORs) and their associated 95% CIs. Logistic regression analyses were run for both outcomes. Each model included the exposure and comparator group status as well as the baseline characteristics included in the PS model. Statistical significance was set at P < .05. All statistical analyses were performed with R version 4.2.1.

RESULTS

Two of 7 VISN 21 sites did not participate, and 3 had restrictions on participation. DTC education was mailed about 2 weeks prior to scheduled visit for 1260 veterans; 53.6% identified as White, 37.6% were aged 41 to 60 years, and 79.2% had an AUDIT-C ≥ 8 (Table 1). Of those mailed education, there were 173 no-show appointments (13.7%). Thirty-two veterans (2.5%) in the DTC group and 33 veterans (2.6%) in the comparator group received an AUD-related pharmacotherapy prescription (P = .88) (Table 2). One hundred seventy-one veterans (13.6%) in the DTC group and 160 veterans (12.7%) in the comparator group had a consult placed for mental health or SUD services within 30 days of their appointment (P = .59) (Table 3).

DISCUSSION

This project did not yield statistically significant differences in either the primary or secondary outcomes within the 30-day follow-up window and found limited impact from the DTC educational outreach to veterans. The percentage of veterans that received AUD-related pharmacotherapy or consultations for mental health or SUD services was similarly low in the DTC and comparator groups. These findings suggest that although DTC education may raise awareness, it may not be sufficient on its own to drive changes in prescribing behavior or referral patterns without system-level support.

Addiction is a complex disease faced with stigma and requiring readiness by both the HCP and patient to move forward in support and treatment. The consequences of stigma can be severe: the more stigma perceived by a person with AUD, the less likely they are to seek treatment.²⁰ Stigma may exist even within HCPs and may lead to compromised care including shortened visits, less engagement, and less empathy.¹⁹ Cultural attitude towards alcohol use and intoxication can also be influenced through a wide range of sources including social media, movies, music, and television. Studies have shown targeted alcohol marketing may result in the development of positive beliefs about drinking and expand environments where alcohol use is socially acceptable and encouraged.²¹ These factors can impact drinking behavior, including the onset of drinking, binge drinking, and increased alcohol consumption.²²

Three VISN 21 sites in this study had restrictions on or excluded primary care from participation. Leadership at some of these facilities were concerned that primary care teams did not have the bandwidth to take on additional items and/or there was variable primary care readiness for initiating AUD pharmacotherapy. Further attempts should be made to integrate primary care into the process of initiating AUD treatment as significant research suggests that integrated care models for AUD may be associated with improved process and outcome measures of care.²³

There are several differences between this quality improvement project and prior research investigating the impact of DTC education for other conditions, such as the EMPOWER randomized controlled trial and VISN 22 project, which both demonstrated effectiveness of DTC education for reducing benzodiazepine use in geriatric veterans. ^15,16 These studies focused on reducing or stopping pharmacotherapy use, whereas this project sought to promote the initiation of AUD pharmacotherapy. These studies evaluated outcomes at least 6 months postindex date, whereas this project evaluated outcomes within 30 days postappointment. Furthermore, the educational content varied significantly. Other projects provided patients with information focused on specific medications and interventions, such as benzodiazepine tapering, while this project mailed general information on heavy drinking, its risks, and strategies for cutting back, without mentioning pharmacotherapy. The DTC material used in this project was chosen because it was a preapproved national VA ADS resource, which expedited the project timeline by avoiding the need for additional approvals at each participating site. These differences may impact the observed effectiveness of DTC education in this project, especially regarding the primary outcome.

Strengths and Limitations

This quality improvement project sent a large sample of veterans DTC education in a clinical setting across multiple sites. Additionally, PS matching methods were used to balance covariates between the comparator and DTC education group, thereby simulating a randomized controlled trial and reducing selection bias. The project brought attention to the VISN 21 AUD treatment rates, stimulated conversation across sites about available treatments and resources for AUD, and sparked collaboration between academic detailing, mental health, and primary care services. The time frame for visits was selected during the winter; the National Institute on Alcohol Abuse and Alcoholism notes this is a time when people may be more likely to engage in excessive alcohol consumption than at other times of the year.²⁴

The 30-day time frame for outcomes may have been too short to observe changes in prescribing or referral patterns. Additionally, the comparator group was comprised of veterans seen from October 1, 2022, to January 31, 2023, where seasonal timing may have influenced alcohol consumption behaviors and skewed the results. There were also no-show appointments in the DTC education group (13.7%), though it is likely some patients rescheduled and still received AUD pharmacotherapy within 30 days of the original appointment. Finally, it was not possible to confirm whether a patient opened and read the education that was mailed to them. This may be another reason to explore electronic distribution of DTC education. This all may have contributed to the lack of statistically significant differences in both the primary and secondary outcomes.

There was a high level of variability between facility participation in the project. Two of 7 sites did not participate, and 3 sites restricted primary care engagement. This represents a significant limitation, particularly for the secondary outcome of placing consultations for MH or SUD services. Facilities that only included mental health or SUD HCPs may have resulted in lower consultation rates due to their inherent specialization, reducing the likelihood of self-referrals.

The project may overestimate prescribed AUD pharmacotherapy in the primary outcome due to potential misclassification of medications. While the project adhered to the national VA ADS AUD dashboard’s definition of AUD pharmacotherapy, including acamprosate, disulfiram, naltrexone, topiramate, and gabapentin, some of these medications have multiple indications. For example, gabapentin is commonly prescribed for peripheral neuropathy, and topiramate is used to treat migraines and seizures. The multipurpose use adds uncertainty about whether they were prescribed specifically for AUD treatment, especially in cases where the HCP is responsible for treating a broad range of disease states, as in primary care.

CONCLUSIONS

Results of this quality improvement project did not show a statistically significant difference between patients sent DTC education and the comparator group for the initiation of AUD pharmacotherapy or placement of a consult to mental health or SUD services within 30 days of their scheduled visit. Future studies may seek to implement stricter criteria to confirm the intended use of topiramate and gabapentin, such as looking for keywords in the prescription instructions for use, performing chart reviews, and/or only including these medications if prescribed by a mental health or SUD HCP. Alternatively, future studies may consider limiting the analysis to only FDA-approved AUD medications: acamprosate, disulfiram, and naltrexone. It is vital to continue to enhance primary care HCP readiness to treat AUD, given the existing relationships and trust they often have with patients. Electronic methods for distributing DTC education could also be advantageous, as these methods may have the ability to track whether a message has been opened and read. Despite a lack of statistical significance, this project sparked crucial conversations and collaboration around AUD, available treatments, and addressing potential barriers to connecting patients to care within VISN 21.

Excessive alcohol use is one of the leading preventable causes of death in the United States, responsible for about 178,000 deaths annually and an average of 488 daily deaths in 2020 and 2021.¹Alcohol-related deaths increased by 49% between 2006 and 2019.² This trend continued during the COVID-19 pandemic, with death certificates that listed alcohol increasing by > 25% from 2019 to 2020, and another 10% in 2021.³ This increase of alcohol-related deaths includes those as a direct result of chronic alcohol use, such as alcoholic cardiomyopathy, alcoholic hepatitis and cirrhosis, and alcohol-induced pancreatitis, as well as a result of acute use such as alcohol poisoning, suicide by exposure to alcohol, and alcohol-impaired driving fatalities.⁴

Excessive alcohol consumption poses other serious risks, including cases when intake is abruptly reduced without proper management. Alcohol withdrawal syndrome (AWS) can vary in severity, with potentially life-threatening complications such as hallucinations, seizures, and delirium tremens.⁵

These risks highlight the importance of professional intervention and support, not only to mitigate risks associated with AWS, but provide a pathway towards recovery from alcohol use disorder (AUD).

According to the 2022 National Survey on Drug Use and Health, 28.8 million US adults had AUD in the prior year, yet only 7.6% of these individuals received treatment and an even smaller group (2.2%) received medication-assisted treatment for alcohol.^6,7 This is despite American Psychiatric Association guidelines for the pharmacological treatment of patients with AUD, including the use of naltrexone, acamprosate, disulfiram, topiramate, or gabapentin, depending on therapy goals, past medication trials, medication contraindications, and patient preference.⁸ Several of these medications are approved by the US Food and Drug Administration (FDA) for the treatment of AUD and have support for effectiveness from randomized controlled trials and meta-analyses.^9-11

Clinical practice guidelines for the management of substance use disorders (SUDs) from the US Department of Veterans Affairs (VA) and US Department of Defense have strong recommendations for naltrexone and topiramate as first-line pharmacotherapies for moderate to severe AUD. Acamprosate and disulfiram are weak recommendations as alternative options. Gabapentin is a weak recommendation for cases where first-line treatments are contraindicated or ineffective. The guidelines emphasize the importance of a comprehensive approach to AUD treatment, including psychosocial interventions in addition to pharmacotherapy.¹²

A 2023 national survey found veterans reported higher alcohol consumption than nonveterans.¹³ At the end of fiscal year 2023, > 4.4 million veterans—6% of Veterans Health Administration patients—had been diagnosed with AUD.14 However, > 87% of these patients nationally, and 88% of Veterans Integrated Service Network (VISN) 21 patients, were not receiving naltrexone, acamprosate, disulfiram, or topiramate as part of their treatment. The VA Academic Detailing Service (ADS) now includes AUD pharmacotherapy as a campaign focus, highlighting its importance. The ADS is a pharmacy educational outreach program that uses unbiased clinical guidelines to promote aligning prescribing behavior with best practices. Academic detailing methods include speaking with health care practitioners (HCPs), and direct-to-consumer (DTC) patient education.

ADS campaigns include DTC educational handouts. Past ADS projects and research using DTC have demonstrated a significant improvement in outcomes and positively influencing patients’ pharmacotherapy treatment. ^15,16 A VA quality improvement project found a positive correlation between the initiation of AUD pharmacotherapy and engagement with mental health care following the distribution of AUD DTC patient education. ¹⁷ This project aimed to apply the same principles of prior research to explore the use of DTC across multiple facilities within VISN 21 to increase AUD pharmacotherapy. VISN 21 includes VA facilities and clinics across the Pacific Islands, Nevada, and California and serves about 350,000 veterans.

METHODS

A prospective cohort of VISN 21 veterans with or at high risk for AUD was identified using the VA ADS AUD Dashboard. The cohort included those not on acamprosate, disulfiram, naltrexone, topiramate, or gabapentin for treatment of AUD and had an elevated Alcohol Use Disorder Identification Test-Consumption (AUDIT-C) score of ≥ 6 (high risk) with an AUD diagnosis or ≥ 8 (severe risk) without a diagnosis. The AUDIT-C scores used in the dashboard are supported by the VA AUD clinician guide as the minimum scores when AUD pharmacotherapy should be offered to patients.¹⁸ Prescriptions filled outside the VA were not included in this dashboard.

Data and patient information were collected using the VA Corporate Data Warehouse. To be eligible, veterans needed a valid mailing address within the VISN 21 region and a primary care, mental health, or SUD clinician prescriber visit scheduled between October 1, 2023, and January 31, 2024. Veterans were excluded if they were in hospice, had a 1-year mortality risk score > 50% based on their Care Assessment Need (CAN) score, or facility leadership opted out of project involvement. Patients with both severe renal and hepatic impairments were excluded because they were ineligible for AUD pharmacotherapy. However, veterans with either renal or hepatic impairment (but not both) were included, as they could be potential candidates for ≥ 1 AUD pharmacotherapy option.

Initial correspondence with facilities was initiated through local academic detailers. A local champion was identified for the 1 facility without an academic detailer. Facilities could opt in or out of the project. Approval was provided by the local pharmacy and therapeutics committee, pharmacy, primary care, or psychiatry leadership. Approval process and clinician involvement varied by site.

Education

The selected AUD patient education was designed and approved by the national VA ADS (eappendix). The DTC patient education provided general knowledge about alcohol, including what constitutes a standard amount of alcohol, what is considered heavy drinking, risks of heavy drinking, creating a plan with a clinician to reduce and manage withdrawal symptoms, and additional resources. The DTC was accompanied by a cover letter that included a local facility contact number.

A centralized mailing facility was used for all materials. VA Northern California Health Care System provided the funding to cover the cost of postage. The list of veterans to be contacted was updated on a rolling basis and DTC education was mailed 2 weeks prior to their scheduled prescriber visit.

The eligible cohort of 1260 veterans received DTC education. A comparator group of 2048 veterans that did not receive DTC education was obtained retrospectively by using the same inclusion and exclusion criteria with a scheduled primary care, mental health, or SUD HCP visit from October 1, 2022, to January 31, 2023. The outcomes assessed were within 30 days of the scheduled visit, with the primary outcome as the initiation of AUD-related pharmacotherapy and the secondary outcome as the placement of a consultation for mental health or SUD services. Any consultations sent to Behavioral Health, Addiction, Mental Health, Psychiatric, and SUD services following the HCP visit, within the specified time frame, were used for the secondary outcome.

Matching and Analysis

A 1-to-1 nearest neighbor propensity score (PS) matching without replacement was used to pair the 1260 veterans from the intervention group with similarly scored comparator group veterans for a PS-matched final dataset of 2520 veterans. The PS model was a multivariate logistic regression with the outcome being exposure and comparator group status. Baseline characteristics used in the PS model were age, birth sex, race, facility of care, baseline AUDIT-C score, and days between project start and scheduled appointment. Covariate imbalance for the PS-matched sample was assessed to ensure the standardized mean difference for all covariates fell under a 0.1 threshold (Figure).¹⁹

A frequency table was provided to compare the discrete distributions of the baseline characteristics in the intervention and comparator groups. Logistic regression analysis was performed to evaluate the association between DTC education exposure and pharmacotherapy initiation, while controlling for potential confounders. Univariate and multivariate P value results for each variable included in the model were reported along with the multivariate odds ratios (ORs) and their associated 95% CIs. Logistic regression analyses were run for both outcomes. Each model included the exposure and comparator group status as well as the baseline characteristics included in the PS model. Statistical significance was set at P < .05. All statistical analyses were performed with R version 4.2.1.

RESULTS

Two of 7 VISN 21 sites did not participate, and 3 had restrictions on participation. DTC education was mailed about 2 weeks prior to scheduled visit for 1260 veterans; 53.6% identified as White, 37.6% were aged 41 to 60 years, and 79.2% had an AUDIT-C ≥ 8 (Table 1). Of those mailed education, there were 173 no-show appointments (13.7%). Thirty-two veterans (2.5%) in the DTC group and 33 veterans (2.6%) in the comparator group received an AUD-related pharmacotherapy prescription (P = .88) (Table 2). One hundred seventy-one veterans (13.6%) in the DTC group and 160 veterans (12.7%) in the comparator group had a consult placed for mental health or SUD services within 30 days of their appointment (P = .59) (Table 3).

DISCUSSION

This project did not yield statistically significant differences in either the primary or secondary outcomes within the 30-day follow-up window and found limited impact from the DTC educational outreach to veterans. The percentage of veterans that received AUD-related pharmacotherapy or consultations for mental health or SUD services was similarly low in the DTC and comparator groups. These findings suggest that although DTC education may raise awareness, it may not be sufficient on its own to drive changes in prescribing behavior or referral patterns without system-level support.

Addiction is a complex disease faced with stigma and requiring readiness by both the HCP and patient to move forward in support and treatment. The consequences of stigma can be severe: the more stigma perceived by a person with AUD, the less likely they are to seek treatment.²⁰ Stigma may exist even within HCPs and may lead to compromised care including shortened visits, less engagement, and less empathy.¹⁹ Cultural attitude towards alcohol use and intoxication can also be influenced through a wide range of sources including social media, movies, music, and television. Studies have shown targeted alcohol marketing may result in the development of positive beliefs about drinking and expand environments where alcohol use is socially acceptable and encouraged.²¹ These factors can impact drinking behavior, including the onset of drinking, binge drinking, and increased alcohol consumption.²²

Three VISN 21 sites in this study had restrictions on or excluded primary care from participation. Leadership at some of these facilities were concerned that primary care teams did not have the bandwidth to take on additional items and/or there was variable primary care readiness for initiating AUD pharmacotherapy. Further attempts should be made to integrate primary care into the process of initiating AUD treatment as significant research suggests that integrated care models for AUD may be associated with improved process and outcome measures of care.²³

There are several differences between this quality improvement project and prior research investigating the impact of DTC education for other conditions, such as the EMPOWER randomized controlled trial and VISN 22 project, which both demonstrated effectiveness of DTC education for reducing benzodiazepine use in geriatric veterans. ^15,16 These studies focused on reducing or stopping pharmacotherapy use, whereas this project sought to promote the initiation of AUD pharmacotherapy. These studies evaluated outcomes at least 6 months postindex date, whereas this project evaluated outcomes within 30 days postappointment. Furthermore, the educational content varied significantly. Other projects provided patients with information focused on specific medications and interventions, such as benzodiazepine tapering, while this project mailed general information on heavy drinking, its risks, and strategies for cutting back, without mentioning pharmacotherapy. The DTC material used in this project was chosen because it was a preapproved national VA ADS resource, which expedited the project timeline by avoiding the need for additional approvals at each participating site. These differences may impact the observed effectiveness of DTC education in this project, especially regarding the primary outcome.

Strengths and Limitations

This quality improvement project sent a large sample of veterans DTC education in a clinical setting across multiple sites. Additionally, PS matching methods were used to balance covariates between the comparator and DTC education group, thereby simulating a randomized controlled trial and reducing selection bias. The project brought attention to the VISN 21 AUD treatment rates, stimulated conversation across sites about available treatments and resources for AUD, and sparked collaboration between academic detailing, mental health, and primary care services. The time frame for visits was selected during the winter; the National Institute on Alcohol Abuse and Alcoholism notes this is a time when people may be more likely to engage in excessive alcohol consumption than at other times of the year.²⁴

The 30-day time frame for outcomes may have been too short to observe changes in prescribing or referral patterns. Additionally, the comparator group was comprised of veterans seen from October 1, 2022, to January 31, 2023, where seasonal timing may have influenced alcohol consumption behaviors and skewed the results. There were also no-show appointments in the DTC education group (13.7%), though it is likely some patients rescheduled and still received AUD pharmacotherapy within 30 days of the original appointment. Finally, it was not possible to confirm whether a patient opened and read the education that was mailed to them. This may be another reason to explore electronic distribution of DTC education. This all may have contributed to the lack of statistically significant differences in both the primary and secondary outcomes.

There was a high level of variability between facility participation in the project. Two of 7 sites did not participate, and 3 sites restricted primary care engagement. This represents a significant limitation, particularly for the secondary outcome of placing consultations for MH or SUD services. Facilities that only included mental health or SUD HCPs may have resulted in lower consultation rates due to their inherent specialization, reducing the likelihood of self-referrals.

The project may overestimate prescribed AUD pharmacotherapy in the primary outcome due to potential misclassification of medications. While the project adhered to the national VA ADS AUD dashboard’s definition of AUD pharmacotherapy, including acamprosate, disulfiram, naltrexone, topiramate, and gabapentin, some of these medications have multiple indications. For example, gabapentin is commonly prescribed for peripheral neuropathy, and topiramate is used to treat migraines and seizures. The multipurpose use adds uncertainty about whether they were prescribed specifically for AUD treatment, especially in cases where the HCP is responsible for treating a broad range of disease states, as in primary care.

CONCLUSIONS

Results of this quality improvement project did not show a statistically significant difference between patients sent DTC education and the comparator group for the initiation of AUD pharmacotherapy or placement of a consult to mental health or SUD services within 30 days of their scheduled visit. Future studies may seek to implement stricter criteria to confirm the intended use of topiramate and gabapentin, such as looking for keywords in the prescription instructions for use, performing chart reviews, and/or only including these medications if prescribed by a mental health or SUD HCP. Alternatively, future studies may consider limiting the analysis to only FDA-approved AUD medications: acamprosate, disulfiram, and naltrexone. It is vital to continue to enhance primary care HCP readiness to treat AUD, given the existing relationships and trust they often have with patients. Electronic methods for distributing DTC education could also be advantageous, as these methods may have the ability to track whether a message has been opened and read. Despite a lack of statistical significance, this project sparked crucial conversations and collaboration around AUD, available treatments, and addressing potential barriers to connecting patients to care within VISN 21.

Store-and-forward teledermatology (SFT) allows clinical images and information to be sent to a dermatologist for evaluation. In fiscal year (FY) 2018, 117,780 SFT consultations were completed in the Veterans Health Administration. Continued growth is expected since SFT has proven to be an effective method for improving access to face-to-face (FTF) dermatology care.¹ In the same period, the US Department of Veterans Affairs (VA) Puget Sound Health Care System (VAPSHCS) completed 12,563 consultations in a mean 1.1 days from entry into episode of care (EEC), according to data reported by VA Teledermatology Program Administrator Chris Foster.

Obtaining a prompt consultation is reported to be an overwhelming advantage of using SFT.^2-5 Rapid turnaround may appear to make SFT specialist care more accessible to veterans, yet this is an oversimplification. The process of delivering care (rather than consultation) through SFT is more complex than reading the images and reporting the findings. When a skin condition is identified by a primary care clinician and that person decides to request an SFT consultation, a complex set of tasks and handoffs is set into motion. A swim-lane diagram illustrates the numerous steps and handoffs that go into delivering care to a patient with a malignant melanoma on the SFT platform compared to FTF care, which requires fewer handoffs (Figure).

This process improvement project examined whether handoffs necessitated by SFT care lengthened the timeline of care for biopsy-proven primary cutaneous malignant melanoma. The stakes of delay in care are high. A 2018 study using the National Cancer Database found that a delay of > 30 days from biopsy to definitive excision (the date definitive surgical procedure for the condition is performed) resulted in a measurable increase in melanoma-related mortality. ⁶ This study sought to identify areas where the SFT timeline of care could be shortened.

Methods

This retrospective cohort study was approved by the VAPSHCS Institutional Review Board. The study drew from secondary data obtained from VistA, the VA Corporate Data Warehouse, the Veterans Integrated Service Network (VISN) 20 database, the American Academy of Dermatology Teledermatology Program database, and the VA Computerized Patient Record System.

Patients registered for ≥ 1 year at VAPSHCS with a diagnosis of primary cutaneous malignant melanoma by the Pathology service between January 1, 2006, and December 31, 2013, were included. Patients with metastatic or recurrent melanoma were excluded.

Cases were randomly selected from a melanoma database previously validated and used for another quality improvement project.⁷ There were initially 115 patient cases extracted from this database for both the FTF and SFT groups. Eighty-seven SFT and 107 FTF cases met inclusion criteria. To further analyze these groups, we split the FTF group into 2 subgroups: FTF dermatology (patients whose melanomas were entered into care in a dermatology clinic) and FTF primary care (patients whose melanomas were entered into care in primary care or a nondermatology setting).

The timeline of care was divided into 2 major time intervals: (1) entry into episode of care (EEC; the date a lesion was first documented in the electronic health record) to biopsy; and (2) biopsy to definitive excision. The SFT process was divided into the following intervals: EEC to imaging request (the date a clinician requested imaging); imaging request to imaging completion (the date an imager photographed a patient’s lesion); imaging completion to SFT consultation request (the date the SFT consultation was requested); SFT consultation request to consultation completion (the date an SFT reader completed the consultation request for a patient); and SFT consultation completion to biopsy. Mean and median interval lengths were compared between groups and additional analyses identified steps that may have contributed to delays in care.

To address potential bias based on access to care for rural veterans, SFT and FTF primary care cases were categorized into groups based on their location: (1) EEC and biopsy conducted at the same facility; (2) EEC and biopsy conducted at different facilities within the same health care system (main health care facility and its community-based outpatient clinics); and (3) EEC and biopsy conducted at different health care systems.

Statistics

Means, medians, and SDs were calculated in Excel. The Mann-Whitney U test was used to compare SFT medians to the FTF data and X² test was used to compare proportions for secondary analyses.

Results

The median (mean) interval from EEC to definitive excision was 73 days (85) for SFT and 58 days (73) for FTF (P = .004) (Table). To understand this difference, the distribution of intervals from EEC to biopsy and biopsy to definitive excision were calculated. Only 38% of SFT cases were biopsied within 20 days compared to 65% of FTF cases (P < .001). The difference in time from biopsy to definitive excision distributions were not statistically significant, suggesting that the difference is actually a reflection of the differences seen in the period between EEC and biopsy.

EEC and biopsy occurred at the same facility in 85% and 82% of FTF primary care and SFT cases, respectively. EEC and biopsy occurred at different facilities within the same health care system in 15% and 16% of FTF primary care and SFT cases, respectively. EEC and biopsy occurred at different health care systems in 0% and 2% of FTF primary care and SFT cases, respectively. Geographic bias did not impact results for either group of veterans.

The interval between EEC and biopsy was shorter for FTF dermatology cases than for FTF primary care cases. For FTF dermatology cases, 96% were biopsied within 20 days compared with 34% of FTF primary care cases (P < .001).

To further analyze the difference in the EEC to biopsy interval duration between SFT and FTF primary care the timeline was divided into smaller steps: EEC to imaging completion, imaging completion to SFT consult completion, and SFT consult completion to biopsy. From EEC to SFT consult completion, SFT cases took a median of 6.0 days and a mean of 12.3 days, reflecting the administrative handoffs that must occur in SFT. A total of 82% of FTF primary care cases were entered into care and consultation was requested on the same day, while this was true for only 1% of SFT cases.

Since mortality data were not collected, the frequency of in situ melanomas and invasive melanomas (pathologic stage pT1a or greater) was used as a proxy for comparing outcomes. No significant difference was found in the frequency of in situ vs invasive melanomas in the SFT and FTF dermatology groups; however, there was a much higher frequency of invasive melanomas in the FTF primary care group (P = .007).

Discussion

This study compared the time to treatment for SFT vs FTF and identified important differences. The episode of care for melanomas diagnosed by SFT was statistically significantly longer (15 days) than those diagnosed by FTF. The interval between biopsy and definitive excision was a median of 34 and 38 days, and a mean of 48 and 44 days for SFT and FTF, respectively, which were not statistically significant. The difference in the total duration of the interval between EEC and definitive excision was accounted for by the duration of the interval from EEC to biopsy. When excluding dermatology clinic cases from the FTF group, there was no difference in the interval between EEC and biopsy for SFT and FTF primary care. The handoffs in SFT accounted for a median of 6 days and mean of 12 days, a significant portion of the timeline, and is a target for process improvement. The delay necessitated by handoffs did not significantly affect the distribution of in situ and invasive melanomas in the SFT and FTF dermatology groups. This suggests that SFT may have better outcomes than FTF primary care.

There has been extensive research on the timeline from the patient initially noticing a lesion to the EEC.^8-11 There is also a body of research on the timeline from biopsy to definitive excision. ^6,12-16 However, there has been little research on the timeline between EEC and biopsy, which comprises a large portion of the overall timeline of both SFT care and FTF care. This study analyzed the delays that can occur in this interval. When patients first enter FTF dermatology care, this timeline is quite short because lesions are often biopsied on the same day. When patients enter into care with their primary or nondermatology clinician, there can be significant delays.

Since the stakes are high when it comes to treating melanoma, it is important to minimize the overall timeline. A 6-day median and 12-day mean were established as targets for teledermatology handoffs. Ideally, a lesion should be entered into an episode of care, imaged, and sent for consultation on the same day. To help further understand delays in administrative handoffs, we stratified the SFT cases by VISN 20 sites and spoke with an administrator at a top performing site. Between 2006 and 2013, this site had a dedicated full-time imager as well as a backup imager that ensured images were taken quickly, usually on the same day the lesion was entered into care. Unfortunately, this is not the standard at all VISN 20 sites and certainly contributes to the overall delay in care in SFT

Minimizing the timeline of care is possible, as shown by the Danish health system, which developed a fast-track referral system after recognizing the need to minimize delays between the presentation, diagnosis, and treatment of cutaneous melanomas. In Denmark, a patient who presents to a general practitioner with a suspicious lesion is referred to secondary care for excision biopsy within 6 days. Diagnosis is made within 2 weeks, and, if necessary, definitive excision is offered within 9 days of the diagnosis. This translates into a maximum 20-day EEC to biopsy timeline and maximum 29-day EEC to definitive excision timeline. Although an intervention such as this may be difficult to implement in the United States due to its size and decentralized health care system, it would, however, be more realistic within the VA due to its centralized structure. The Danish system shows that with appropriate resource allocation and strict timeframes for treatment referrals, the timeline can be minimized.¹⁷

Despite the delay in the SFT timeline, this study found no significant difference between the distribution of in situ vs invasive melanomas in FTF dermatology and SFT groups. One possible explanation for this is that SFT increases access to dermatologist care, meaning clinicians may be more willing to consult SFT for less advanced– appearing lesions.

The finding that SFT diagnosed a larger proportion of in situ melanomas than FTF primary care is consistent with the findings of Ferrándiz et al, who reported that the mean Breslow thickness was significantly lower among patients in an SFT group compared to patients in an FTF group consisting of general practitioners. ¹⁸ However, the study population was not randomized and the results may have been impacted by ascertainment bias. Ferrándiz et al hypothesized that clinicians may have a lower threshold for consulting teledermatology, resulting in lower mean Breslow thicknesses.¹⁸ Karavan et al found the opposite results, with a higher mean Breslow thickness in SFT compared to a primary care FTF group.¹⁹ The data presented here suggest that SFT has room for process improvement yet is essentially equivalent to FTF dermatology in terms of outcomes.

Limitations

The majority of patients in this study were aged > 50 years, White, and male. The results may not be representative for other populations. The study was relatively small compared to studies that looked at other aspects of the melanoma care timeline. The study was not powered to ascertain mortality, the most important metric for melanoma.

Conclusions

The episode of care was significantly longer for melanomas diagnosed by SFT than those diagnosed by FTF; however, timelines were not statistically different when FTF lesions entered into care in dermatology were excluded. A median 6-day and mean 12.3-day delay in administrative handoffs occurred at the beginning of the SFT process and is a target for process improvement. Considering the high stakes of melanoma, the SFT timeline could be reduced if EEC, imaging, and SFT consultation all happened in the same day.

Store-and-forward teledermatology (SFT) allows clinical images and information to be sent to a dermatologist for evaluation. In fiscal year (FY) 2018, 117,780 SFT consultations were completed in the Veterans Health Administration. Continued growth is expected since SFT has proven to be an effective method for improving access to face-to-face (FTF) dermatology care.¹ In the same period, the US Department of Veterans Affairs (VA) Puget Sound Health Care System (VAPSHCS) completed 12,563 consultations in a mean 1.1 days from entry into episode of care (EEC), according to data reported by VA Teledermatology Program Administrator Chris Foster.

Obtaining a prompt consultation is reported to be an overwhelming advantage of using SFT.^2-5 Rapid turnaround may appear to make SFT specialist care more accessible to veterans, yet this is an oversimplification. The process of delivering care (rather than consultation) through SFT is more complex than reading the images and reporting the findings. When a skin condition is identified by a primary care clinician and that person decides to request an SFT consultation, a complex set of tasks and handoffs is set into motion. A swim-lane diagram illustrates the numerous steps and handoffs that go into delivering care to a patient with a malignant melanoma on the SFT platform compared to FTF care, which requires fewer handoffs (Figure).

This process improvement project examined whether handoffs necessitated by SFT care lengthened the timeline of care for biopsy-proven primary cutaneous malignant melanoma. The stakes of delay in care are high. A 2018 study using the National Cancer Database found that a delay of > 30 days from biopsy to definitive excision (the date definitive surgical procedure for the condition is performed) resulted in a measurable increase in melanoma-related mortality. ⁶ This study sought to identify areas where the SFT timeline of care could be shortened.

Methods

This retrospective cohort study was approved by the VAPSHCS Institutional Review Board. The study drew from secondary data obtained from VistA, the VA Corporate Data Warehouse, the Veterans Integrated Service Network (VISN) 20 database, the American Academy of Dermatology Teledermatology Program database, and the VA Computerized Patient Record System.

Patients registered for ≥ 1 year at VAPSHCS with a diagnosis of primary cutaneous malignant melanoma by the Pathology service between January 1, 2006, and December 31, 2013, were included. Patients with metastatic or recurrent melanoma were excluded.

Cases were randomly selected from a melanoma database previously validated and used for another quality improvement project.⁷ There were initially 115 patient cases extracted from this database for both the FTF and SFT groups. Eighty-seven SFT and 107 FTF cases met inclusion criteria. To further analyze these groups, we split the FTF group into 2 subgroups: FTF dermatology (patients whose melanomas were entered into care in a dermatology clinic) and FTF primary care (patients whose melanomas were entered into care in primary care or a nondermatology setting).

The timeline of care was divided into 2 major time intervals: (1) entry into episode of care (EEC; the date a lesion was first documented in the electronic health record) to biopsy; and (2) biopsy to definitive excision. The SFT process was divided into the following intervals: EEC to imaging request (the date a clinician requested imaging); imaging request to imaging completion (the date an imager photographed a patient’s lesion); imaging completion to SFT consultation request (the date the SFT consultation was requested); SFT consultation request to consultation completion (the date an SFT reader completed the consultation request for a patient); and SFT consultation completion to biopsy. Mean and median interval lengths were compared between groups and additional analyses identified steps that may have contributed to delays in care.

To address potential bias based on access to care for rural veterans, SFT and FTF primary care cases were categorized into groups based on their location: (1) EEC and biopsy conducted at the same facility; (2) EEC and biopsy conducted at different facilities within the same health care system (main health care facility and its community-based outpatient clinics); and (3) EEC and biopsy conducted at different health care systems.

Statistics

Means, medians, and SDs were calculated in Excel. The Mann-Whitney U test was used to compare SFT medians to the FTF data and X² test was used to compare proportions for secondary analyses.

Results

The median (mean) interval from EEC to definitive excision was 73 days (85) for SFT and 58 days (73) for FTF (P = .004) (Table). To understand this difference, the distribution of intervals from EEC to biopsy and biopsy to definitive excision were calculated. Only 38% of SFT cases were biopsied within 20 days compared to 65% of FTF cases (P < .001). The difference in time from biopsy to definitive excision distributions were not statistically significant, suggesting that the difference is actually a reflection of the differences seen in the period between EEC and biopsy.

EEC and biopsy occurred at the same facility in 85% and 82% of FTF primary care and SFT cases, respectively. EEC and biopsy occurred at different facilities within the same health care system in 15% and 16% of FTF primary care and SFT cases, respectively. EEC and biopsy occurred at different health care systems in 0% and 2% of FTF primary care and SFT cases, respectively. Geographic bias did not impact results for either group of veterans.

The interval between EEC and biopsy was shorter for FTF dermatology cases than for FTF primary care cases. For FTF dermatology cases, 96% were biopsied within 20 days compared with 34% of FTF primary care cases (P < .001).

To further analyze the difference in the EEC to biopsy interval duration between SFT and FTF primary care the timeline was divided into smaller steps: EEC to imaging completion, imaging completion to SFT consult completion, and SFT consult completion to biopsy. From EEC to SFT consult completion, SFT cases took a median of 6.0 days and a mean of 12.3 days, reflecting the administrative handoffs that must occur in SFT. A total of 82% of FTF primary care cases were entered into care and consultation was requested on the same day, while this was true for only 1% of SFT cases.

Since mortality data were not collected, the frequency of in situ melanomas and invasive melanomas (pathologic stage pT1a or greater) was used as a proxy for comparing outcomes. No significant difference was found in the frequency of in situ vs invasive melanomas in the SFT and FTF dermatology groups; however, there was a much higher frequency of invasive melanomas in the FTF primary care group (P = .007).

Discussion

This study compared the time to treatment for SFT vs FTF and identified important differences. The episode of care for melanomas diagnosed by SFT was statistically significantly longer (15 days) than those diagnosed by FTF. The interval between biopsy and definitive excision was a median of 34 and 38 days, and a mean of 48 and 44 days for SFT and FTF, respectively, which were not statistically significant. The difference in the total duration of the interval between EEC and definitive excision was accounted for by the duration of the interval from EEC to biopsy. When excluding dermatology clinic cases from the FTF group, there was no difference in the interval between EEC and biopsy for SFT and FTF primary care. The handoffs in SFT accounted for a median of 6 days and mean of 12 days, a significant portion of the timeline, and is a target for process improvement. The delay necessitated by handoffs did not significantly affect the distribution of in situ and invasive melanomas in the SFT and FTF dermatology groups. This suggests that SFT may have better outcomes than FTF primary care.

There has been extensive research on the timeline from the patient initially noticing a lesion to the EEC.^8-11 There is also a body of research on the timeline from biopsy to definitive excision. ^6,12-16 However, there has been little research on the timeline between EEC and biopsy, which comprises a large portion of the overall timeline of both SFT care and FTF care. This study analyzed the delays that can occur in this interval. When patients first enter FTF dermatology care, this timeline is quite short because lesions are often biopsied on the same day. When patients enter into care with their primary or nondermatology clinician, there can be significant delays.

Since the stakes are high when it comes to treating melanoma, it is important to minimize the overall timeline. A 6-day median and 12-day mean were established as targets for teledermatology handoffs. Ideally, a lesion should be entered into an episode of care, imaged, and sent for consultation on the same day. To help further understand delays in administrative handoffs, we stratified the SFT cases by VISN 20 sites and spoke with an administrator at a top performing site. Between 2006 and 2013, this site had a dedicated full-time imager as well as a backup imager that ensured images were taken quickly, usually on the same day the lesion was entered into care. Unfortunately, this is not the standard at all VISN 20 sites and certainly contributes to the overall delay in care in SFT

Minimizing the timeline of care is possible, as shown by the Danish health system, which developed a fast-track referral system after recognizing the need to minimize delays between the presentation, diagnosis, and treatment of cutaneous melanomas. In Denmark, a patient who presents to a general practitioner with a suspicious lesion is referred to secondary care for excision biopsy within 6 days. Diagnosis is made within 2 weeks, and, if necessary, definitive excision is offered within 9 days of the diagnosis. This translates into a maximum 20-day EEC to biopsy timeline and maximum 29-day EEC to definitive excision timeline. Although an intervention such as this may be difficult to implement in the United States due to its size and decentralized health care system, it would, however, be more realistic within the VA due to its centralized structure. The Danish system shows that with appropriate resource allocation and strict timeframes for treatment referrals, the timeline can be minimized.¹⁷

Despite the delay in the SFT timeline, this study found no significant difference between the distribution of in situ vs invasive melanomas in FTF dermatology and SFT groups. One possible explanation for this is that SFT increases access to dermatologist care, meaning clinicians may be more willing to consult SFT for less advanced– appearing lesions.

The finding that SFT diagnosed a larger proportion of in situ melanomas than FTF primary care is consistent with the findings of Ferrándiz et al, who reported that the mean Breslow thickness was significantly lower among patients in an SFT group compared to patients in an FTF group consisting of general practitioners. ¹⁸ However, the study population was not randomized and the results may have been impacted by ascertainment bias. Ferrándiz et al hypothesized that clinicians may have a lower threshold for consulting teledermatology, resulting in lower mean Breslow thicknesses.¹⁸ Karavan et al found the opposite results, with a higher mean Breslow thickness in SFT compared to a primary care FTF group.¹⁹ The data presented here suggest that SFT has room for process improvement yet is essentially equivalent to FTF dermatology in terms of outcomes.

Limitations

The majority of patients in this study were aged > 50 years, White, and male. The results may not be representative for other populations. The study was relatively small compared to studies that looked at other aspects of the melanoma care timeline. The study was not powered to ascertain mortality, the most important metric for melanoma.

Conclusions

The episode of care was significantly longer for melanomas diagnosed by SFT than those diagnosed by FTF; however, timelines were not statistically different when FTF lesions entered into care in dermatology were excluded. A median 6-day and mean 12.3-day delay in administrative handoffs occurred at the beginning of the SFT process and is a target for process improvement. Considering the high stakes of melanoma, the SFT timeline could be reduced if EEC, imaging, and SFT consultation all happened in the same day.

Store-and-forward teledermatology (SFT) allows clinical images and information to be sent to a dermatologist for evaluation. In fiscal year (FY) 2018, 117,780 SFT consultations were completed in the Veterans Health Administration. Continued growth is expected since SFT has proven to be an effective method for improving access to face-to-face (FTF) dermatology care.¹ In the same period, the US Department of Veterans Affairs (VA) Puget Sound Health Care System (VAPSHCS) completed 12,563 consultations in a mean 1.1 days from entry into episode of care (EEC), according to data reported by VA Teledermatology Program Administrator Chris Foster.

Obtaining a prompt consultation is reported to be an overwhelming advantage of using SFT.^2-5 Rapid turnaround may appear to make SFT specialist care more accessible to veterans, yet this is an oversimplification. The process of delivering care (rather than consultation) through SFT is more complex than reading the images and reporting the findings. When a skin condition is identified by a primary care clinician and that person decides to request an SFT consultation, a complex set of tasks and handoffs is set into motion. A swim-lane diagram illustrates the numerous steps and handoffs that go into delivering care to a patient with a malignant melanoma on the SFT platform compared to FTF care, which requires fewer handoffs (Figure).

This process improvement project examined whether handoffs necessitated by SFT care lengthened the timeline of care for biopsy-proven primary cutaneous malignant melanoma. The stakes of delay in care are high. A 2018 study using the National Cancer Database found that a delay of > 30 days from biopsy to definitive excision (the date definitive surgical procedure for the condition is performed) resulted in a measurable increase in melanoma-related mortality. ⁶ This study sought to identify areas where the SFT timeline of care could be shortened.

Methods

This retrospective cohort study was approved by the VAPSHCS Institutional Review Board. The study drew from secondary data obtained from VistA, the VA Corporate Data Warehouse, the Veterans Integrated Service Network (VISN) 20 database, the American Academy of Dermatology Teledermatology Program database, and the VA Computerized Patient Record System.

Patients registered for ≥ 1 year at VAPSHCS with a diagnosis of primary cutaneous malignant melanoma by the Pathology service between January 1, 2006, and December 31, 2013, were included. Patients with metastatic or recurrent melanoma were excluded.

Cases were randomly selected from a melanoma database previously validated and used for another quality improvement project.⁷ There were initially 115 patient cases extracted from this database for both the FTF and SFT groups. Eighty-seven SFT and 107 FTF cases met inclusion criteria. To further analyze these groups, we split the FTF group into 2 subgroups: FTF dermatology (patients whose melanomas were entered into care in a dermatology clinic) and FTF primary care (patients whose melanomas were entered into care in primary care or a nondermatology setting).

The timeline of care was divided into 2 major time intervals: (1) entry into episode of care (EEC; the date a lesion was first documented in the electronic health record) to biopsy; and (2) biopsy to definitive excision. The SFT process was divided into the following intervals: EEC to imaging request (the date a clinician requested imaging); imaging request to imaging completion (the date an imager photographed a patient’s lesion); imaging completion to SFT consultation request (the date the SFT consultation was requested); SFT consultation request to consultation completion (the date an SFT reader completed the consultation request for a patient); and SFT consultation completion to biopsy. Mean and median interval lengths were compared between groups and additional analyses identified steps that may have contributed to delays in care.

To address potential bias based on access to care for rural veterans, SFT and FTF primary care cases were categorized into groups based on their location: (1) EEC and biopsy conducted at the same facility; (2) EEC and biopsy conducted at different facilities within the same health care system (main health care facility and its community-based outpatient clinics); and (3) EEC and biopsy conducted at different health care systems.

Statistics

Means, medians, and SDs were calculated in Excel. The Mann-Whitney U test was used to compare SFT medians to the FTF data and X² test was used to compare proportions for secondary analyses.

Results

The median (mean) interval from EEC to definitive excision was 73 days (85) for SFT and 58 days (73) for FTF (P = .004) (Table). To understand this difference, the distribution of intervals from EEC to biopsy and biopsy to definitive excision were calculated. Only 38% of SFT cases were biopsied within 20 days compared to 65% of FTF cases (P < .001). The difference in time from biopsy to definitive excision distributions were not statistically significant, suggesting that the difference is actually a reflection of the differences seen in the period between EEC and biopsy.

EEC and biopsy occurred at the same facility in 85% and 82% of FTF primary care and SFT cases, respectively. EEC and biopsy occurred at different facilities within the same health care system in 15% and 16% of FTF primary care and SFT cases, respectively. EEC and biopsy occurred at different health care systems in 0% and 2% of FTF primary care and SFT cases, respectively. Geographic bias did not impact results for either group of veterans.

The interval between EEC and biopsy was shorter for FTF dermatology cases than for FTF primary care cases. For FTF dermatology cases, 96% were biopsied within 20 days compared with 34% of FTF primary care cases (P < .001).

To further analyze the difference in the EEC to biopsy interval duration between SFT and FTF primary care the timeline was divided into smaller steps: EEC to imaging completion, imaging completion to SFT consult completion, and SFT consult completion to biopsy. From EEC to SFT consult completion, SFT cases took a median of 6.0 days and a mean of 12.3 days, reflecting the administrative handoffs that must occur in SFT. A total of 82% of FTF primary care cases were entered into care and consultation was requested on the same day, while this was true for only 1% of SFT cases.

Since mortality data were not collected, the frequency of in situ melanomas and invasive melanomas (pathologic stage pT1a or greater) was used as a proxy for comparing outcomes. No significant difference was found in the frequency of in situ vs invasive melanomas in the SFT and FTF dermatology groups; however, there was a much higher frequency of invasive melanomas in the FTF primary care group (P = .007).

Discussion

This study compared the time to treatment for SFT vs FTF and identified important differences. The episode of care for melanomas diagnosed by SFT was statistically significantly longer (15 days) than those diagnosed by FTF. The interval between biopsy and definitive excision was a median of 34 and 38 days, and a mean of 48 and 44 days for SFT and FTF, respectively, which were not statistically significant. The difference in the total duration of the interval between EEC and definitive excision was accounted for by the duration of the interval from EEC to biopsy. When excluding dermatology clinic cases from the FTF group, there was no difference in the interval between EEC and biopsy for SFT and FTF primary care. The handoffs in SFT accounted for a median of 6 days and mean of 12 days, a significant portion of the timeline, and is a target for process improvement. The delay necessitated by handoffs did not significantly affect the distribution of in situ and invasive melanomas in the SFT and FTF dermatology groups. This suggests that SFT may have better outcomes than FTF primary care.

There has been extensive research on the timeline from the patient initially noticing a lesion to the EEC.^8-11 There is also a body of research on the timeline from biopsy to definitive excision. ^6,12-16 However, there has been little research on the timeline between EEC and biopsy, which comprises a large portion of the overall timeline of both SFT care and FTF care. This study analyzed the delays that can occur in this interval. When patients first enter FTF dermatology care, this timeline is quite short because lesions are often biopsied on the same day. When patients enter into care with their primary or nondermatology clinician, there can be significant delays.

Since the stakes are high when it comes to treating melanoma, it is important to minimize the overall timeline. A 6-day median and 12-day mean were established as targets for teledermatology handoffs. Ideally, a lesion should be entered into an episode of care, imaged, and sent for consultation on the same day. To help further understand delays in administrative handoffs, we stratified the SFT cases by VISN 20 sites and spoke with an administrator at a top performing site. Between 2006 and 2013, this site had a dedicated full-time imager as well as a backup imager that ensured images were taken quickly, usually on the same day the lesion was entered into care. Unfortunately, this is not the standard at all VISN 20 sites and certainly contributes to the overall delay in care in SFT

Minimizing the timeline of care is possible, as shown by the Danish health system, which developed a fast-track referral system after recognizing the need to minimize delays between the presentation, diagnosis, and treatment of cutaneous melanomas. In Denmark, a patient who presents to a general practitioner with a suspicious lesion is referred to secondary care for excision biopsy within 6 days. Diagnosis is made within 2 weeks, and, if necessary, definitive excision is offered within 9 days of the diagnosis. This translates into a maximum 20-day EEC to biopsy timeline and maximum 29-day EEC to definitive excision timeline. Although an intervention such as this may be difficult to implement in the United States due to its size and decentralized health care system, it would, however, be more realistic within the VA due to its centralized structure. The Danish system shows that with appropriate resource allocation and strict timeframes for treatment referrals, the timeline can be minimized.¹⁷

Despite the delay in the SFT timeline, this study found no significant difference between the distribution of in situ vs invasive melanomas in FTF dermatology and SFT groups. One possible explanation for this is that SFT increases access to dermatologist care, meaning clinicians may be more willing to consult SFT for less advanced– appearing lesions.

The finding that SFT diagnosed a larger proportion of in situ melanomas than FTF primary care is consistent with the findings of Ferrándiz et al, who reported that the mean Breslow thickness was significantly lower among patients in an SFT group compared to patients in an FTF group consisting of general practitioners. ¹⁸ However, the study population was not randomized and the results may have been impacted by ascertainment bias. Ferrándiz et al hypothesized that clinicians may have a lower threshold for consulting teledermatology, resulting in lower mean Breslow thicknesses.¹⁸ Karavan et al found the opposite results, with a higher mean Breslow thickness in SFT compared to a primary care FTF group.¹⁹ The data presented here suggest that SFT has room for process improvement yet is essentially equivalent to FTF dermatology in terms of outcomes.

Limitations

The majority of patients in this study were aged > 50 years, White, and male. The results may not be representative for other populations. The study was relatively small compared to studies that looked at other aspects of the melanoma care timeline. The study was not powered to ascertain mortality, the most important metric for melanoma.

Conclusions

The episode of care was significantly longer for melanomas diagnosed by SFT than those diagnosed by FTF; however, timelines were not statistically different when FTF lesions entered into care in dermatology were excluded. A median 6-day and mean 12.3-day delay in administrative handoffs occurred at the beginning of the SFT process and is a target for process improvement. Considering the high stakes of melanoma, the SFT timeline could be reduced if EEC, imaging, and SFT consultation all happened in the same day.

To the Editor:

Early skin cancer detection improves patient outcomes¹; however, socioeconomic and racial disparities may impact access to dermatologic care.² Although non-Hispanic White individuals have a high incidence of skin cancer, they experience higher melanoma-specific survival rates than non-White patients, who often receive later-stage diagnoses and experience higher mortality.² Furthermore, racial/ ethnic minorities often face longer surgery wait times after diagnosis and have lower socioeconomic status (SES) and less favorable health insurance coverage, contributing to poorer outcomes.^2,3

To examine access to full-body skin examinations (FBSEs) by board-certified dermatologists in Los Angeles (LA) County, California, we analyzed the availability of FBSEs based on racial demographics, income, and insurance type (Medicaid [Medi-Cal] vs private [Blue Cross Blue Shield (BCBS)]). Demographic data by zip code were obtained from the US Census Bureau.⁴ This validated metric highlights socioeconomic disparities and minimizes data gaps^5,6 and was used to assess health care access among different population subgroups. Dermatologists’ contact information was obtained from the Find a Dermatologist page on the American Academy of Dermatology website and the listed phone numbers of their practice were used to contact them. Practices with board-certified dermatologists accepting new patients were included in the study; practices were not included if they had exclusive insurance plans; were pediatric, cosmetic, or research only; or were nonresponsive to calls. From August 2022 to September 2022, each practice was called twice within a 36-hour period—once by a simulated patient with Medi-Cal and once by a simulated patient with BCBS—and were asked about availability for new patient FBSE appointments and accepted insurance types. Data were analyzed using SAS software (SAS Institute Inc.).

Los Angeles County comprises 269 zip codes, of which 82 (30.5%) have dermatology practices. Of 213 total dermatologists in LA County listed on the American Academy of Dermatology website, 193 (90.6%) met preliminary criteria, and 169 (79.3%) were successfully contacted. Almost all (94.6% [160/169]) accepted new patients for FBSEs; of those, 63.1% (101/160) accepted only private insurance, 16.9% (27/160) accepted both private insurance and Medi-Cal, and 16.2% (26/160) did not accept any insurance. Racial predominance for each dermatology practice was analyzed by zip code (Table). Dermatologists included in our study were significantly more concentrated in predominantly non- Hispanic White areas of LA County vs predominantly Hispanic areas (P<.0001). Notably, the average income in predominantly non-Hispanic White zip codes ($114,757.74) was significantly higher than in predominantly Hispanic areas ($58,278.54)(P=.001)(Table).⁴

In LA County, 40.1% (108/269) of zip codes have no racial majority, 28.2% (76/269) are predominantly Hispanic, 27.5% (74/269) are predominantly non-Hispanic White, 2.2% (6/269) are predominantly Black, and 1.9% (5/269) are predominantly Asian.⁴ There are no dermatologists in predominantly Black zip codes, 2 in predominantly Asian zip codes, 14 in predominantly Hispanic zip codes, 38 in zip codes with no racial majority, and 106 in predominantly non-Hispanic White zip codes. There are significantly more dermatologists in predominantly non-Hispanic White zip codes compared to predominantly Hispanic zip codes (P<.0001). In LA County, the average income in predominantly Asian, non-Hispanic White, and Hispanic zip codes was $93,594, $114,757.84, and $58,278.54, respectively, in 2021.4 The average income in predominantly non-Hispanic White zip codes was significantly higher than in predominantly Hispanic zip codes (P=.001). There were no income data available for predominantly Black zip codes or zip codes with no racial majority.

The results from our study revealed potential barriers to FBSEs for racial and ethnic minorities in LA County, which supports previous research on the impact of SES, race, and insurance on access to dermatologic care.^2,3 Predominantly Hispanic zip codes have significantly lower income (P<.0001) and fewer dermatologists (P=.001) compared to zip codes that are predominantly non-Hispanic White, reflecting how lower SES correlates with worse health outcomes and higher melanoma mortality. Conversely, predominantly non-Hispanic White areas with higher income have better access to dermatologists, which may contribute to the improved melanoma survival rates among White patients. Additionally, most dermatologists accept only private insurance, further highlighting the disparity in FBSE access for non-White patients across LA County. While our study focused on FBSE access, our findings may point to a wider barrier to dermatologic care, especially in zip codes with fewer dermatologists. Further studies are needed to determine whether these areas also face barriers to accessing primary care.

Our study was limited by the exclusion of nonphysician providers (eg, nurse practitioners, physician assistants), a small sample size, and lack of available economic data for predominantly Black zip codes.⁴ Additionally, the exclusion of practices with exclusive insurance plans (eg, Kaiser Permanente) limited the generalizability of our findings, as our results did not account for the populations served by these practices. Furthermore, our analysis did not account for variations in practice size or the proportion of care provided to patients with different insurance types, which could impact overall accessibility. Additional studies are needed to explore the impact of these factors on access to general dermatologic care and not just FBSEs.

Racial/ethnic minorities and lower SES populations face major barriers to FBSE access in LA County, such as difficulty finding a dermatologist in their area or one who accepts Medi-Cal. Addressing these disparities is crucial for improving skin cancer outcomes. Further research is needed to develop strategies to eliminate these barriers to dermatologic care, such as increasing access to teledermatology, offering mobile dermatology clinics, and improving insurance coverage.

To the Editor:

Early skin cancer detection improves patient outcomes¹; however, socioeconomic and racial disparities may impact access to dermatologic care.² Although non-Hispanic White individuals have a high incidence of skin cancer, they experience higher melanoma-specific survival rates than non-White patients, who often receive later-stage diagnoses and experience higher mortality.² Furthermore, racial/ ethnic minorities often face longer surgery wait times after diagnosis and have lower socioeconomic status (SES) and less favorable health insurance coverage, contributing to poorer outcomes.^2,3

To examine access to full-body skin examinations (FBSEs) by board-certified dermatologists in Los Angeles (LA) County, California, we analyzed the availability of FBSEs based on racial demographics, income, and insurance type (Medicaid [Medi-Cal] vs private [Blue Cross Blue Shield (BCBS)]). Demographic data by zip code were obtained from the US Census Bureau.⁴ This validated metric highlights socioeconomic disparities and minimizes data gaps^5,6 and was used to assess health care access among different population subgroups. Dermatologists’ contact information was obtained from the Find a Dermatologist page on the American Academy of Dermatology website and the listed phone numbers of their practice were used to contact them. Practices with board-certified dermatologists accepting new patients were included in the study; practices were not included if they had exclusive insurance plans; were pediatric, cosmetic, or research only; or were nonresponsive to calls. From August 2022 to September 2022, each practice was called twice within a 36-hour period—once by a simulated patient with Medi-Cal and once by a simulated patient with BCBS—and were asked about availability for new patient FBSE appointments and accepted insurance types. Data were analyzed using SAS software (SAS Institute Inc.).

Los Angeles County comprises 269 zip codes, of which 82 (30.5%) have dermatology practices. Of 213 total dermatologists in LA County listed on the American Academy of Dermatology website, 193 (90.6%) met preliminary criteria, and 169 (79.3%) were successfully contacted. Almost all (94.6% [160/169]) accepted new patients for FBSEs; of those, 63.1% (101/160) accepted only private insurance, 16.9% (27/160) accepted both private insurance and Medi-Cal, and 16.2% (26/160) did not accept any insurance. Racial predominance for each dermatology practice was analyzed by zip code (Table). Dermatologists included in our study were significantly more concentrated in predominantly non- Hispanic White areas of LA County vs predominantly Hispanic areas (P<.0001). Notably, the average income in predominantly non-Hispanic White zip codes ($114,757.74) was significantly higher than in predominantly Hispanic areas ($58,278.54)(P=.001)(Table).⁴

In LA County, 40.1% (108/269) of zip codes have no racial majority, 28.2% (76/269) are predominantly Hispanic, 27.5% (74/269) are predominantly non-Hispanic White, 2.2% (6/269) are predominantly Black, and 1.9% (5/269) are predominantly Asian.⁴ There are no dermatologists in predominantly Black zip codes, 2 in predominantly Asian zip codes, 14 in predominantly Hispanic zip codes, 38 in zip codes with no racial majority, and 106 in predominantly non-Hispanic White zip codes. There are significantly more dermatologists in predominantly non-Hispanic White zip codes compared to predominantly Hispanic zip codes (P<.0001). In LA County, the average income in predominantly Asian, non-Hispanic White, and Hispanic zip codes was $93,594, $114,757.84, and $58,278.54, respectively, in 2021.4 The average income in predominantly non-Hispanic White zip codes was significantly higher than in predominantly Hispanic zip codes (P=.001). There were no income data available for predominantly Black zip codes or zip codes with no racial majority.

The results from our study revealed potential barriers to FBSEs for racial and ethnic minorities in LA County, which supports previous research on the impact of SES, race, and insurance on access to dermatologic care.^2,3 Predominantly Hispanic zip codes have significantly lower income (P<.0001) and fewer dermatologists (P=.001) compared to zip codes that are predominantly non-Hispanic White, reflecting how lower SES correlates with worse health outcomes and higher melanoma mortality. Conversely, predominantly non-Hispanic White areas with higher income have better access to dermatologists, which may contribute to the improved melanoma survival rates among White patients. Additionally, most dermatologists accept only private insurance, further highlighting the disparity in FBSE access for non-White patients across LA County. While our study focused on FBSE access, our findings may point to a wider barrier to dermatologic care, especially in zip codes with fewer dermatologists. Further studies are needed to determine whether these areas also face barriers to accessing primary care.

Our study was limited by the exclusion of nonphysician providers (eg, nurse practitioners, physician assistants), a small sample size, and lack of available economic data for predominantly Black zip codes.⁴ Additionally, the exclusion of practices with exclusive insurance plans (eg, Kaiser Permanente) limited the generalizability of our findings, as our results did not account for the populations served by these practices. Furthermore, our analysis did not account for variations in practice size or the proportion of care provided to patients with different insurance types, which could impact overall accessibility. Additional studies are needed to explore the impact of these factors on access to general dermatologic care and not just FBSEs.

Racial/ethnic minorities and lower SES populations face major barriers to FBSE access in LA County, such as difficulty finding a dermatologist in their area or one who accepts Medi-Cal. Addressing these disparities is crucial for improving skin cancer outcomes. Further research is needed to develop strategies to eliminate these barriers to dermatologic care, such as increasing access to teledermatology, offering mobile dermatology clinics, and improving insurance coverage.

To the Editor:

Early skin cancer detection improves patient outcomes¹; however, socioeconomic and racial disparities may impact access to dermatologic care.² Although non-Hispanic White individuals have a high incidence of skin cancer, they experience higher melanoma-specific survival rates than non-White patients, who often receive later-stage diagnoses and experience higher mortality.² Furthermore, racial/ ethnic minorities often face longer surgery wait times after diagnosis and have lower socioeconomic status (SES) and less favorable health insurance coverage, contributing to poorer outcomes.^2,3

To examine access to full-body skin examinations (FBSEs) by board-certified dermatologists in Los Angeles (LA) County, California, we analyzed the availability of FBSEs based on racial demographics, income, and insurance type (Medicaid [Medi-Cal] vs private [Blue Cross Blue Shield (BCBS)]). Demographic data by zip code were obtained from the US Census Bureau.⁴ This validated metric highlights socioeconomic disparities and minimizes data gaps^5,6 and was used to assess health care access among different population subgroups. Dermatologists’ contact information was obtained from the Find a Dermatologist page on the American Academy of Dermatology website and the listed phone numbers of their practice were used to contact them. Practices with board-certified dermatologists accepting new patients were included in the study; practices were not included if they had exclusive insurance plans; were pediatric, cosmetic, or research only; or were nonresponsive to calls. From August 2022 to September 2022, each practice was called twice within a 36-hour period—once by a simulated patient with Medi-Cal and once by a simulated patient with BCBS—and were asked about availability for new patient FBSE appointments and accepted insurance types. Data were analyzed using SAS software (SAS Institute Inc.).

Los Angeles County comprises 269 zip codes, of which 82 (30.5%) have dermatology practices. Of 213 total dermatologists in LA County listed on the American Academy of Dermatology website, 193 (90.6%) met preliminary criteria, and 169 (79.3%) were successfully contacted. Almost all (94.6% [160/169]) accepted new patients for FBSEs; of those, 63.1% (101/160) accepted only private insurance, 16.9% (27/160) accepted both private insurance and Medi-Cal, and 16.2% (26/160) did not accept any insurance. Racial predominance for each dermatology practice was analyzed by zip code (Table). Dermatologists included in our study were significantly more concentrated in predominantly non- Hispanic White areas of LA County vs predominantly Hispanic areas (P<.0001). Notably, the average income in predominantly non-Hispanic White zip codes ($114,757.74) was significantly higher than in predominantly Hispanic areas ($58,278.54)(P=.001)(Table).⁴

In LA County, 40.1% (108/269) of zip codes have no racial majority, 28.2% (76/269) are predominantly Hispanic, 27.5% (74/269) are predominantly non-Hispanic White, 2.2% (6/269) are predominantly Black, and 1.9% (5/269) are predominantly Asian.⁴ There are no dermatologists in predominantly Black zip codes, 2 in predominantly Asian zip codes, 14 in predominantly Hispanic zip codes, 38 in zip codes with no racial majority, and 106 in predominantly non-Hispanic White zip codes. There are significantly more dermatologists in predominantly non-Hispanic White zip codes compared to predominantly Hispanic zip codes (P<.0001). In LA County, the average income in predominantly Asian, non-Hispanic White, and Hispanic zip codes was $93,594, $114,757.84, and $58,278.54, respectively, in 2021.4 The average income in predominantly non-Hispanic White zip codes was significantly higher than in predominantly Hispanic zip codes (P=.001). There were no income data available for predominantly Black zip codes or zip codes with no racial majority.

The results from our study revealed potential barriers to FBSEs for racial and ethnic minorities in LA County, which supports previous research on the impact of SES, race, and insurance on access to dermatologic care.^2,3 Predominantly Hispanic zip codes have significantly lower income (P<.0001) and fewer dermatologists (P=.001) compared to zip codes that are predominantly non-Hispanic White, reflecting how lower SES correlates with worse health outcomes and higher melanoma mortality. Conversely, predominantly non-Hispanic White areas with higher income have better access to dermatologists, which may contribute to the improved melanoma survival rates among White patients. Additionally, most dermatologists accept only private insurance, further highlighting the disparity in FBSE access for non-White patients across LA County. While our study focused on FBSE access, our findings may point to a wider barrier to dermatologic care, especially in zip codes with fewer dermatologists. Further studies are needed to determine whether these areas also face barriers to accessing primary care.

Our study was limited by the exclusion of nonphysician providers (eg, nurse practitioners, physician assistants), a small sample size, and lack of available economic data for predominantly Black zip codes.⁴ Additionally, the exclusion of practices with exclusive insurance plans (eg, Kaiser Permanente) limited the generalizability of our findings, as our results did not account for the populations served by these practices. Furthermore, our analysis did not account for variations in practice size or the proportion of care provided to patients with different insurance types, which could impact overall accessibility. Additional studies are needed to explore the impact of these factors on access to general dermatologic care and not just FBSEs.

Racial/ethnic minorities and lower SES populations face major barriers to FBSE access in LA County, such as difficulty finding a dermatologist in their area or one who accepts Medi-Cal. Addressing these disparities is crucial for improving skin cancer outcomes. Further research is needed to develop strategies to eliminate these barriers to dermatologic care, such as increasing access to teledermatology, offering mobile dermatology clinics, and improving insurance coverage.

To the Editor:

The incidence of nonmelanoma skin cancer (NMSC) is rapidly increasing worldwide. Due to its highly curable nature when treated early, accurate diagnosis is the cornerstone to good patient outcomes.¹ Accurate diagnosis of skin cancer and subsequent treatment decisions rely heavily on the congruence between clinical observations and histopathologic assessments. Clinical misdiagnosis of a malignant lesion can lead to delayed and suboptimal treatment, which may contribute to serious complications such as metastasis or even mortality. In this study, data from clinically diagnosed basal cell carcinomas (BCCs) and squamous cell carcinomas (SCCs) were compared to their identified histopathologic subtype classifications. The accuracy of the clinical diagnosis of these NMSCs was assessed by determining the rate of misdiagnosis and the respective positive predictive value (PPV).

A retrospective review of medical records from a private dermatology practice in Lubbock, Texas, was conducted to identify patients diagnosed with NMSC from January 1, 2017, through December 31, 2021. A total of 11,229 NMSCs were diagnosed and treated in 5877 patients. Of the NMSCs diagnosed, 11,145 were identified as keratinocyte carcinomas and were classified as BCCs or SCCs. The accuracy of the clinical diagnoses was determined by comparison to the histologic subtype identified via biopsy of the lesion. Although the use of a dermatoscope during the clinical encounter was not formally recorded, reports from the examining dermatologists indicated it was not used in the majority of cases.

If a lesion was clinically diagnosed as a BCC but was identified as a subtype of SCC on histology (or vice versa), the lesion was considered to be mismatched. The number of mismatched lesions and the mismatch rate for each lesion type/subtype is recorded in the Table. Of the total 11,145 keratinocyte carcinomas included in our study, there was an overall 10.63% mismatch rate, with 1185 of the malignancies having a differing clinical diagnosis (eg, BCC vs SCC) from the histologic findings. The clinical mismatch rate was notably higher for SCC compared to BCC (15.83% vs 7.03%, respectively).

The Table provides a breakdown of the BCC subtypes identified by histology with their computed mismatch rate and PPV. It is worth clarifying that lesions classified as more than one BCC subtype per the histologic findings were diagnosed as mixed BCC; these were further classified as mixed-aggressive BCC (if at least one aggressive BCC subtype was present) and mixed nonaggressive BCC (if no aggressive BCC subtype was present). Overall, BCCs were less likely to be misdiagnosed, with an average PPV of 92.97% compared to 84.17% for SCCs. Basosquamous BCC was the BCC subtype with the highest mismatch rate (25.48%), while sclerosing BCC has the lowest overall mismatch rate (1.33%). The most common malignancy was BCC, with nodular BCC being the most common subtype.

The Table also breaks down the SCC subtypes, reporting the most commonly misdiagnosed of any BCC or SCC subtype to be poorly differentiated SCC (mismatch rate, 38.46%). The lowest mismatch rate of the SCC subtypes was 5.97% for well-differentiated SCC.

There was an overall PPV of 89.37% in clinically evaluated malignancies and their respective histologic subtypes. Basal cell carcinoma had a lower overall mismatch rate of 7.03% compared to 15.83% in SCC. The most common misdiagnosis was attributed to poorly differentiated SCC (mismatch rate, 38.46%), while the least common misdiagnosed malignancy was sclerosing BCC (1.33%). The high mismatch rate of poorly differentiated SCC may be due to its diverging presentation from a typical SCC as a flat lesion with the absence of scaling, keratin, or bleeding, leading to the misdiagnosis of BCC.²

Accurate clinical diagnosis of NMSCs is the basis for further evaluation and treatment that should ensue in a timely manner; however, accurately identifying BCCs vs SCCs solely based on clinical examination can be challenging due to variable manifestations and overlapping features. Basal cell carcinoma commonly presents as a shiny pink/flesh-colored nodule, macule, or patch with surface telangiectasia, sometimes appearing with ulceration or crusting.³ Alternatively, SCC typically appears as a firm, sharply demarcated, red nodule with a thick overlying scale.⁴ Definitive diagnoses can be difficult upon clinical examination since these features can be shared between the 2 subtypes. To aid in these uncertainties, a growing number of clinicians are implementing the use of dermoscopy in their everyday practice.

Dermoscopy is an extremely useful tool in improving the diagnostic accuracy of skin cancers compared to examination with the naked eye, as it provides detailed visualization of specific structures and patterns in skin cancer lesions.⁵ The dermoscopic appearance of BCC is characterized by pearly blue-gray or translucent globules with arborizing vessels, spoke-wheel structures, and leaflike areas.^5,6 Conversely, dermoscopic features of SCC may include a milky-red globule with a scaly, sharply demarcated, crusted lesion with polymorphous vasculature, sometimes resembling a persistent sore or nonhealing wound.^4,5 Though the use of dermoscopy can aid in diagnosis upon initial examination, certain factors such as trauma, ulceration, and previous treatments that distorted the lesion’s architecture may lead to misdiagnosis. Furthermore, the distinct vascular patterns found in BCC and SCC may be mistaken for each other and therefore lead to misdiagnosis upon examination.⁷ Other variables that may complicate diagnosis include the location of the lesion, its size, and the presence of other skin conditions or nearby lesions.

The primary limitation of the current study was the limited scope of the data, as they were derived from patients seen at one private dermatology practice, preventing the generalizability of our findings. However, our results show trends similar to those observed in other studies analyzing the clinical accuracy of skin cancer diagnoses, with higher PPVs for BCC compared to SCC. A study by Ahnlide and Bjellerup⁸ was based in a hospital dermatology department and demonstrated a PPV of 85.5% for BCC compared to 92.97% in our study; for SCC, the PPV was 67.3% compared to 84.17% in our study. In another study by Heal et al,⁹ data were collected from an Australian registry that included records of all histologically confirmed skin cancers from December 1996 to October 1999 from 202 general practitioners and 42 specialists, including 1 dermatologist. The PPVs for BCC and SCC were 72.7% and 49.4%, respectively. Although our results indicated higher PPVs compared to these 2 studies, some of the discrepancies can be accounted for by the differences in clinical setting as well as the lack of expertise of nondermatologist physicians in identifying skin malignancies in the study by Heal et al.⁹

The current study was further limited by the lack of data quantifying the number of lesions clinically suspected to be malignant but found to be histologically benign. It is typical for clinicians to have a low threshold to biopsy a suspicious lesion with atypical features (eg, rapid evolution and growth, bleeding, crusting). Furthermore, the identification of risk factors in the patient’s medical and family history (eg, exposure to radiation, personal or family history of skin cancers) can heavily influence a clinician’s decision to biopsy a lesion with an atypical appearance.¹⁰ Many benign lesions are biopsied to avoid missing a diagnosis of malignancy. Consequently, our results suggest a high degree of clinical misdiagnosis of BCCs and SCCs. Obtaining data on the number of lesions suspected to be BCC or SCC that were found to be histologically benign would be a valuable addition to our study, as it would provide a measurable insight into the sensitivity of clinicians’ decision-making to identify a lesion as suspicious and warranting biopsy.

While clinical diagnosis plays a vital role in identifying suspected NMSCs such as BCC and SCC, its accuracy can be limited even with the use of dermoscopy. Overall, our data have shown a high rate of diagnostic accuracy upon suspicion of malignancy, but the different variables that affect clinical presentation promote histologic diagnosis to prevail as the gold standard.

To the Editor:

The incidence of nonmelanoma skin cancer (NMSC) is rapidly increasing worldwide. Due to its highly curable nature when treated early, accurate diagnosis is the cornerstone to good patient outcomes.¹ Accurate diagnosis of skin cancer and subsequent treatment decisions rely heavily on the congruence between clinical observations and histopathologic assessments. Clinical misdiagnosis of a malignant lesion can lead to delayed and suboptimal treatment, which may contribute to serious complications such as metastasis or even mortality. In this study, data from clinically diagnosed basal cell carcinomas (BCCs) and squamous cell carcinomas (SCCs) were compared to their identified histopathologic subtype classifications. The accuracy of the clinical diagnosis of these NMSCs was assessed by determining the rate of misdiagnosis and the respective positive predictive value (PPV).

A retrospective review of medical records from a private dermatology practice in Lubbock, Texas, was conducted to identify patients diagnosed with NMSC from January 1, 2017, through December 31, 2021. A total of 11,229 NMSCs were diagnosed and treated in 5877 patients. Of the NMSCs diagnosed, 11,145 were identified as keratinocyte carcinomas and were classified as BCCs or SCCs. The accuracy of the clinical diagnoses was determined by comparison to the histologic subtype identified via biopsy of the lesion. Although the use of a dermatoscope during the clinical encounter was not formally recorded, reports from the examining dermatologists indicated it was not used in the majority of cases.

If a lesion was clinically diagnosed as a BCC but was identified as a subtype of SCC on histology (or vice versa), the lesion was considered to be mismatched. The number of mismatched lesions and the mismatch rate for each lesion type/subtype is recorded in the Table. Of the total 11,145 keratinocyte carcinomas included in our study, there was an overall 10.63% mismatch rate, with 1185 of the malignancies having a differing clinical diagnosis (eg, BCC vs SCC) from the histologic findings. The clinical mismatch rate was notably higher for SCC compared to BCC (15.83% vs 7.03%, respectively).

The Table provides a breakdown of the BCC subtypes identified by histology with their computed mismatch rate and PPV. It is worth clarifying that lesions classified as more than one BCC subtype per the histologic findings were diagnosed as mixed BCC; these were further classified as mixed-aggressive BCC (if at least one aggressive BCC subtype was present) and mixed nonaggressive BCC (if no aggressive BCC subtype was present). Overall, BCCs were less likely to be misdiagnosed, with an average PPV of 92.97% compared to 84.17% for SCCs. Basosquamous BCC was the BCC subtype with the highest mismatch rate (25.48%), while sclerosing BCC has the lowest overall mismatch rate (1.33%). The most common malignancy was BCC, with nodular BCC being the most common subtype.

The Table also breaks down the SCC subtypes, reporting the most commonly misdiagnosed of any BCC or SCC subtype to be poorly differentiated SCC (mismatch rate, 38.46%). The lowest mismatch rate of the SCC subtypes was 5.97% for well-differentiated SCC.

There was an overall PPV of 89.37% in clinically evaluated malignancies and their respective histologic subtypes. Basal cell carcinoma had a lower overall mismatch rate of 7.03% compared to 15.83% in SCC. The most common misdiagnosis was attributed to poorly differentiated SCC (mismatch rate, 38.46%), while the least common misdiagnosed malignancy was sclerosing BCC (1.33%). The high mismatch rate of poorly differentiated SCC may be due to its diverging presentation from a typical SCC as a flat lesion with the absence of scaling, keratin, or bleeding, leading to the misdiagnosis of BCC.²

Accurate clinical diagnosis of NMSCs is the basis for further evaluation and treatment that should ensue in a timely manner; however, accurately identifying BCCs vs SCCs solely based on clinical examination can be challenging due to variable manifestations and overlapping features. Basal cell carcinoma commonly presents as a shiny pink/flesh-colored nodule, macule, or patch with surface telangiectasia, sometimes appearing with ulceration or crusting.³ Alternatively, SCC typically appears as a firm, sharply demarcated, red nodule with a thick overlying scale.⁴ Definitive diagnoses can be difficult upon clinical examination since these features can be shared between the 2 subtypes. To aid in these uncertainties, a growing number of clinicians are implementing the use of dermoscopy in their everyday practice.

Dermoscopy is an extremely useful tool in improving the diagnostic accuracy of skin cancers compared to examination with the naked eye, as it provides detailed visualization of specific structures and patterns in skin cancer lesions.⁵ The dermoscopic appearance of BCC is characterized by pearly blue-gray or translucent globules with arborizing vessels, spoke-wheel structures, and leaflike areas.^5,6 Conversely, dermoscopic features of SCC may include a milky-red globule with a scaly, sharply demarcated, crusted lesion with polymorphous vasculature, sometimes resembling a persistent sore or nonhealing wound.^4,5 Though the use of dermoscopy can aid in diagnosis upon initial examination, certain factors such as trauma, ulceration, and previous treatments that distorted the lesion’s architecture may lead to misdiagnosis. Furthermore, the distinct vascular patterns found in BCC and SCC may be mistaken for each other and therefore lead to misdiagnosis upon examination.⁷ Other variables that may complicate diagnosis include the location of the lesion, its size, and the presence of other skin conditions or nearby lesions.

The primary limitation of the current study was the limited scope of the data, as they were derived from patients seen at one private dermatology practice, preventing the generalizability of our findings. However, our results show trends similar to those observed in other studies analyzing the clinical accuracy of skin cancer diagnoses, with higher PPVs for BCC compared to SCC. A study by Ahnlide and Bjellerup⁸ was based in a hospital dermatology department and demonstrated a PPV of 85.5% for BCC compared to 92.97% in our study; for SCC, the PPV was 67.3% compared to 84.17% in our study. In another study by Heal et al,⁹ data were collected from an Australian registry that included records of all histologically confirmed skin cancers from December 1996 to October 1999 from 202 general practitioners and 42 specialists, including 1 dermatologist. The PPVs for BCC and SCC were 72.7% and 49.4%, respectively. Although our results indicated higher PPVs compared to these 2 studies, some of the discrepancies can be accounted for by the differences in clinical setting as well as the lack of expertise of nondermatologist physicians in identifying skin malignancies in the study by Heal et al.⁹

The current study was further limited by the lack of data quantifying the number of lesions clinically suspected to be malignant but found to be histologically benign. It is typical for clinicians to have a low threshold to biopsy a suspicious lesion with atypical features (eg, rapid evolution and growth, bleeding, crusting). Furthermore, the identification of risk factors in the patient’s medical and family history (eg, exposure to radiation, personal or family history of skin cancers) can heavily influence a clinician’s decision to biopsy a lesion with an atypical appearance.¹⁰ Many benign lesions are biopsied to avoid missing a diagnosis of malignancy. Consequently, our results suggest a high degree of clinical misdiagnosis of BCCs and SCCs. Obtaining data on the number of lesions suspected to be BCC or SCC that were found to be histologically benign would be a valuable addition to our study, as it would provide a measurable insight into the sensitivity of clinicians’ decision-making to identify a lesion as suspicious and warranting biopsy.

While clinical diagnosis plays a vital role in identifying suspected NMSCs such as BCC and SCC, its accuracy can be limited even with the use of dermoscopy. Overall, our data have shown a high rate of diagnostic accuracy upon suspicion of malignancy, but the different variables that affect clinical presentation promote histologic diagnosis to prevail as the gold standard.

To the Editor:

The incidence of nonmelanoma skin cancer (NMSC) is rapidly increasing worldwide. Due to its highly curable nature when treated early, accurate diagnosis is the cornerstone to good patient outcomes.¹ Accurate diagnosis of skin cancer and subsequent treatment decisions rely heavily on the congruence between clinical observations and histopathologic assessments. Clinical misdiagnosis of a malignant lesion can lead to delayed and suboptimal treatment, which may contribute to serious complications such as metastasis or even mortality. In this study, data from clinically diagnosed basal cell carcinomas (BCCs) and squamous cell carcinomas (SCCs) were compared to their identified histopathologic subtype classifications. The accuracy of the clinical diagnosis of these NMSCs was assessed by determining the rate of misdiagnosis and the respective positive predictive value (PPV).

A retrospective review of medical records from a private dermatology practice in Lubbock, Texas, was conducted to identify patients diagnosed with NMSC from January 1, 2017, through December 31, 2021. A total of 11,229 NMSCs were diagnosed and treated in 5877 patients. Of the NMSCs diagnosed, 11,145 were identified as keratinocyte carcinomas and were classified as BCCs or SCCs. The accuracy of the clinical diagnoses was determined by comparison to the histologic subtype identified via biopsy of the lesion. Although the use of a dermatoscope during the clinical encounter was not formally recorded, reports from the examining dermatologists indicated it was not used in the majority of cases.

If a lesion was clinically diagnosed as a BCC but was identified as a subtype of SCC on histology (or vice versa), the lesion was considered to be mismatched. The number of mismatched lesions and the mismatch rate for each lesion type/subtype is recorded in the Table. Of the total 11,145 keratinocyte carcinomas included in our study, there was an overall 10.63% mismatch rate, with 1185 of the malignancies having a differing clinical diagnosis (eg, BCC vs SCC) from the histologic findings. The clinical mismatch rate was notably higher for SCC compared to BCC (15.83% vs 7.03%, respectively).

The Table provides a breakdown of the BCC subtypes identified by histology with their computed mismatch rate and PPV. It is worth clarifying that lesions classified as more than one BCC subtype per the histologic findings were diagnosed as mixed BCC; these were further classified as mixed-aggressive BCC (if at least one aggressive BCC subtype was present) and mixed nonaggressive BCC (if no aggressive BCC subtype was present). Overall, BCCs were less likely to be misdiagnosed, with an average PPV of 92.97% compared to 84.17% for SCCs. Basosquamous BCC was the BCC subtype with the highest mismatch rate (25.48%), while sclerosing BCC has the lowest overall mismatch rate (1.33%). The most common malignancy was BCC, with nodular BCC being the most common subtype.

The Table also breaks down the SCC subtypes, reporting the most commonly misdiagnosed of any BCC or SCC subtype to be poorly differentiated SCC (mismatch rate, 38.46%). The lowest mismatch rate of the SCC subtypes was 5.97% for well-differentiated SCC.

There was an overall PPV of 89.37% in clinically evaluated malignancies and their respective histologic subtypes. Basal cell carcinoma had a lower overall mismatch rate of 7.03% compared to 15.83% in SCC. The most common misdiagnosis was attributed to poorly differentiated SCC (mismatch rate, 38.46%), while the least common misdiagnosed malignancy was sclerosing BCC (1.33%). The high mismatch rate of poorly differentiated SCC may be due to its diverging presentation from a typical SCC as a flat lesion with the absence of scaling, keratin, or bleeding, leading to the misdiagnosis of BCC.²

Accurate clinical diagnosis of NMSCs is the basis for further evaluation and treatment that should ensue in a timely manner; however, accurately identifying BCCs vs SCCs solely based on clinical examination can be challenging due to variable manifestations and overlapping features. Basal cell carcinoma commonly presents as a shiny pink/flesh-colored nodule, macule, or patch with surface telangiectasia, sometimes appearing with ulceration or crusting.³ Alternatively, SCC typically appears as a firm, sharply demarcated, red nodule with a thick overlying scale.⁴ Definitive diagnoses can be difficult upon clinical examination since these features can be shared between the 2 subtypes. To aid in these uncertainties, a growing number of clinicians are implementing the use of dermoscopy in their everyday practice.

Dermoscopy is an extremely useful tool in improving the diagnostic accuracy of skin cancers compared to examination with the naked eye, as it provides detailed visualization of specific structures and patterns in skin cancer lesions.⁵ The dermoscopic appearance of BCC is characterized by pearly blue-gray or translucent globules with arborizing vessels, spoke-wheel structures, and leaflike areas.^5,6 Conversely, dermoscopic features of SCC may include a milky-red globule with a scaly, sharply demarcated, crusted lesion with polymorphous vasculature, sometimes resembling a persistent sore or nonhealing wound.^4,5 Though the use of dermoscopy can aid in diagnosis upon initial examination, certain factors such as trauma, ulceration, and previous treatments that distorted the lesion’s architecture may lead to misdiagnosis. Furthermore, the distinct vascular patterns found in BCC and SCC may be mistaken for each other and therefore lead to misdiagnosis upon examination.⁷ Other variables that may complicate diagnosis include the location of the lesion, its size, and the presence of other skin conditions or nearby lesions.

The primary limitation of the current study was the limited scope of the data, as they were derived from patients seen at one private dermatology practice, preventing the generalizability of our findings. However, our results show trends similar to those observed in other studies analyzing the clinical accuracy of skin cancer diagnoses, with higher PPVs for BCC compared to SCC. A study by Ahnlide and Bjellerup⁸ was based in a hospital dermatology department and demonstrated a PPV of 85.5% for BCC compared to 92.97% in our study; for SCC, the PPV was 67.3% compared to 84.17% in our study. In another study by Heal et al,⁹ data were collected from an Australian registry that included records of all histologically confirmed skin cancers from December 1996 to October 1999 from 202 general practitioners and 42 specialists, including 1 dermatologist. The PPVs for BCC and SCC were 72.7% and 49.4%, respectively. Although our results indicated higher PPVs compared to these 2 studies, some of the discrepancies can be accounted for by the differences in clinical setting as well as the lack of expertise of nondermatologist physicians in identifying skin malignancies in the study by Heal et al.⁹

The current study was further limited by the lack of data quantifying the number of lesions clinically suspected to be malignant but found to be histologically benign. It is typical for clinicians to have a low threshold to biopsy a suspicious lesion with atypical features (eg, rapid evolution and growth, bleeding, crusting). Furthermore, the identification of risk factors in the patient’s medical and family history (eg, exposure to radiation, personal or family history of skin cancers) can heavily influence a clinician’s decision to biopsy a lesion with an atypical appearance.¹⁰ Many benign lesions are biopsied to avoid missing a diagnosis of malignancy. Consequently, our results suggest a high degree of clinical misdiagnosis of BCCs and SCCs. Obtaining data on the number of lesions suspected to be BCC or SCC that were found to be histologically benign would be a valuable addition to our study, as it would provide a measurable insight into the sensitivity of clinicians’ decision-making to identify a lesion as suspicious and warranting biopsy.

While clinical diagnosis plays a vital role in identifying suspected NMSCs such as BCC and SCC, its accuracy can be limited even with the use of dermoscopy. Overall, our data have shown a high rate of diagnostic accuracy upon suspicion of malignancy, but the different variables that affect clinical presentation promote histologic diagnosis to prevail as the gold standard.