Original Research

Isotretinoin is used in the treatment of nodulocystic and severe papulopustular acne. During the treatment period, laboratory monitoring is recommended to identify the risk for complications such as hepatotoxicity, teratogenicity, rhabdomyolysis, hyperlipidemia, and pancreatitis.¹ There is a lack of consensus of the frequency of follow-up of laboratory parameters during isotretinoin treatment. This study evaluated the changes in laboratory parameters used in daily practice for patients with acne who were treated with isotretinoin to determine the optimum test repetition frequency.

Materials and Methods

We conducted a retrospective study of data from patients who received oral isotretinoin therapy for acne between January 2021 and July 2022 via the electronic medical records at Konya Numune Hospital and Konya Private Medova Hospital (both in Konya, Turkey). Patients who received an oral isotretinoin total cumulative dose greater than 120 mg/kg were included in the study. Patient demographic data; cumulative isotretinoin doses; and alanine transaminase (ALT), aspartate transaminase (AST), γ-glutamyltransferase (GGT), creatinine kinase (CK), low-density lipoprotein cholesterol (LDL-C), and triglyceride (TG) levels during treatment were recorded. Baseline laboratory levels of those parameters were compared with levels of the same parameters from the second and fourth months of treatment. Comparisons for all parameters were made between the second- and fourth-month levels. Reference ranges are shown in Table 1. Abnormalities were graded according to the National Cancer Institute Common Terminology Criteria for Adverse Events v3.0 grading system.² This study was approved by the Karatay University (Konya, Turkey) ethical committee.

Statistical Analysis—The descriptive statistics of the measurements were presented as means, standard deviations, or medians (first and third quartiles). With respect to the normal distribution, the consistency of the measurements was evaluated with the Kolmogorov-Smirnov test, and small deviations from the normal distribution were observed. Changes in laboratory measurements were evaluated with simple repeated-measures analysis of variance, and changes that differed significantly were determined by a Holm-Sidak post hoc test. Relationships between total cumulative doses and laboratory measurements at second visits were evaluated by the Pearson correlation analysis. The statistical significance level was P<.05. SPSS Statistics 23 (IBM) was used in the calculations.

Results

Consecutive Data at Baseline and Follow-up—A total of 415 patients with a mean age (SD) of 21.49 (7.25) years (range, 12–53 years) were included in our study. The mean total cumulative dose (SD) of the patients was 7267.27 (1878.4) mg. The consecutive data of the means of the laboratory parameters are shown in Table 1 and Figure 1. There was no significant change in the ALT levels between baseline and the fourth month as well as between the second- and fourth-month assessments (both P=.311). When comparing the differences among AST, GGT, and LDL-C measurements, the levels increased significantly between baseline and the second month and between baseline and the fourth month (all P<.001). There was no significant difference in CK levels at all assessments (all P=.304). When the differences between TG measurements were compared, the changes between baseline and the second month (P<.001), baseline and the fourth month (P<.001), and the second and fourth months (P=.013) were significant (Figure 1).

Abnormal Laboratory Measurements—The distribution of abnormal laboratory measurements during treatment is shown in Table 2 and Figure 2. Grade 3 or higher elevations of liver transaminases (ALT, AST) and GGT were observed in fewer than 2% of patients during treatment compared with baseline (grade 3 elevations of ALT and AST together in 2 patients; grade 4 AST elevation in 1 patient; grade 3 elevations of ALT, AST, and GGT combined in 1 patient; isolated grade 3 GGT elevation in 1 patient). All of the patients who developed grade 3 liver transaminases and isolated grade 3 GGT elevation had improved values when these were rechecked within 2 weeks.

In the patient who developed hepatotoxicity in the second month, the ALT level rose from a baseline of 19 U/L to 169 U/L, the AST level from a baseline of 19 U/L to 61 U/L, and the GGT level from a baseline of 24 U/L to 124 U/L. The patient was asymptomatic. Liver function test levels returned to reference range 4 weeks after discontinuation of therapy. Hepatotoxicity did not recur after treatment was re-administered.

The patient who developed grade 4 AST elevation (364 U/L) experienced fatigue and myalgia. He had done vigorous exercise up to 2 days before the test and also had a grade 4 CK elevation (12,310 U/L). He was thought to have isotretinoin-related rhabdomyolysis. His treatment was discontinued, and he was advised to hydrate and rest. Treatment was re-started after 2 weeks. With frequent laboratory monitoring and avoidance of vigorous physical activity, the patient completed the remaining course of isotretinoin without any laboratory abnormalities or symptoms.

 

Creatinine kinase abnormalities in the second and fourth months compared with baseline were not statistically significant. The patients with grade 3 or higher CK elevations, except for the case with rhabdomyolysis, had no clinical signs or other characteristic laboratory findings of rhabdomyolysis.

Hypercholesterolemia (LDL-C ≥130 mg/dL) occurred most frequently, with a maximum of 280 mg/dL in 1 patient (in the fourth month) and less than 250 mg/dL in all other patients. Hypercholesterolemia occurred in 183 (44.1%) patients in the second month and in 166 (40.0%) patients in the fourth month. However, baseline abnormalities also were frequent (86 [20.7%]), and hypercholesterolemia persisted in the second and fourth months in all of these patients.

It was observed that the patients with TG abnormalities increased continuously in the second (99 [23.9%]) and fourth (113 [27.2%]) months compared with baseline (49 [11.8%]). Grade 3 TG elevations were observed in 2.2% of patients (n=9; 5 patients in the second month, 4 patients in the fourth month) during treatment compared with baseline, and all patients had grade 1 or 2 hypertriglyceridemia at baseline. Of the patients with grade 3 TG elevation, 3 patients in the second month and 2 patients in the fourth month were obese at baseline. No grade 4 TG elevations were observed. Complications related to hyperlipidemia, such as pancreatitis, were observed in 1 patient. No patient terminated treatment because of lipid abnormalities. The treatment of our patients with major hypercholesterolemia and/or grade 3 hypertriglyceridemia was interrupted. The hyperlipidemia of these patients was controlled by a low-fat diet and a short-term dose reduction.

Relationship Between Total Cumulative Dose and Laboratory Parameters—The relationships between the total cumulative dose and changes up to the fourth month are presented in Table 3. As the total dose increased, the changes in TG and LDL-C levels significantly increased in the fourth month (both P=.001). However, the degree of these relationships was weak. No significant correlation was found between the periodic changes of other laboratory parameters and the total dose.

Comment

The parameters followed in our study show that TG levels tend to increase continuously from baseline during isotretinoin treatment, while ALT, AST, GGT, and LDL-C levels increase in the second month and decrease at 4 months. Although this same trend occurs with CK levels, the change was not statistically significant. The most common laboratory abnormality in our study was hyperlipidemia. Levels of LDL-C and TG were both found to be statistically elevated in the second and fourth months of treatment compared with baseline. Parthasarathy et al³ reported that obesity had an important role in the increase of lipid levels in patients using isotretinoin at baseline. In our study, 5 of 9 patients (55.6%) with grade 3 TG elevation were obese, which supports the theory that obesity plays an important role in the increase in lipid levels. Up-to-date laboratory follow-up of lipids suggests that there is no need to follow up serum lipids after the second month of treatment. Patients with risk factors for hyperlipidemia, such as abdominal obesity and familial hyperlipidemia, do not require further follow-up if there is no increase in serum lipids in the first month of treatment.¹ The presence of grade 1 or 2 hypertriglyceridemia at baseline in all our patients with grade 3 TG elevation may suggest that periodic laboratory follow-up during isotretinoin treatment is necessary to detect patients with grade 3 and higher TG levels.

The lack of knowledge of other risk factors (eg, familial hyperlipidemia, insulin resistance) for hyperlipidemia in all patients at baseline may be a limitation of our study. Although hypercholesterolemia persisted in the follow-up of our patients with initial LDL-C abnormalities, hypercholesterolemia over 250 mg/dL was very rare (1 patient). Possible complications associated with serum lipid abnormalities are pancreatitis and metabolic syndrome.⁴ In our study, none of the patients with lipid abnormalities had any relevant clinical sequelae. The dose-dependent elevation of the changes in LDL-C and TG (Table 3) may be important to predict the significant elevation of lipids and the associated complications in patients with a high total cumulative dose target that may require a long treatment duration. However, considering the short follow-up periods in our patients, the absence of clinical sequelae may be misleading. There are differences in recommendations between the US and European guidelines for isotretinoin dosage. Although the US guidelines recommend a total cumulative dose target, the European guidelines recommend low-dose isotretinoin daily for at least 6 months instead of a cumulative dose.^5,6 The relationship between change in lipids and total cumulative dose in our study may not be similar in patients treated with the dosing regimen recommended by the European guidelines, as our patients received a total cumulative dose instead of a daily low-dose isotretinoin regimen, unlike the European guidelines.⁵

Most liver transaminase abnormalities were detected in the second month. Abnormalities in GGT were seen in the second month and remained elevated at the next follow-up. However, clinically important grade 3 transaminase and GGT elevations were rare. It has been reported that GGT levels are more specific than transaminases in measuring hepatotoxicity.⁷ The fact that our patient with hepatotoxicity had a grade 3 GGT elevation in addition to grade 3 transaminase elevations supports that GGT elevation is more specific than transaminase levels in measuring hepatotoxicity. When these parameters were rechecked in our patients with grade 3 transaminase elevations, except in the case of hepatotoxicity, transaminase elevations did not recur, and GGT elevations did not accompany elevated transaminases, which suggested that transaminases may be elevated due to an extrahepatic origin (eg, hemolysis, exercise).

Rhabdomyolysis secondary to isotretinoin is rare in the literature of acne studies. In addition to clinical findings such as myalgia and fatigue, increased CK and abnormal liver enzymes, specifically AST, suggest the development of rhabdomyolysis.⁸ Our patient who developed rhabdomyolysis also had a recent history of vigorous exercise, grade 4 CK, and AST elevations. Other patients with isolated grade 3 CK elevations were informed about possible clinical signs of rhabdomyolysis, and they were able to complete their courses without any incident. According to a study by Landau et al,⁹ isotretinoin-associated hyperCKemia has been reported as benign. Similarly, our study found that isolated CK elevation during isotretinoin treatment may be misleading as a sign of rhabdomyolysis. Instead, CK monitoring may be more appropriate and cost-effective in patients with suspected clinical signs of rhabdomyolysis or in those with major elevations in transaminases, especially AST.

Conclusion

According to our study, hyperlipidemia was the most common complication in acne patients using isotretinoin. It may be appropriate to monitor the TG level at 2-month intervals in patients with grade 1 or 2 TG elevation at baseline to detect the possible risk for developing grade 3 hyperlipidemia. Periodic monitoring of LDL-C and TG levels may be appropriate, especially in patients who require a high total cumulative dose of isotretinoin. Clinically important liver enzyme abnormalities were rare in our study. Our findings support the idea that routine monthly monitoring of normal laboratory parameters is unnecessary and wasteful. Additionally, periodic monitoring of abnormal laboratory parameters should be considered on an individual basis.

Isotretinoin is used in the treatment of nodulocystic and severe papulopustular acne. During the treatment period, laboratory monitoring is recommended to identify the risk for complications such as hepatotoxicity, teratogenicity, rhabdomyolysis, hyperlipidemia, and pancreatitis.¹ There is a lack of consensus of the frequency of follow-up of laboratory parameters during isotretinoin treatment. This study evaluated the changes in laboratory parameters used in daily practice for patients with acne who were treated with isotretinoin to determine the optimum test repetition frequency.

Materials and Methods

We conducted a retrospective study of data from patients who received oral isotretinoin therapy for acne between January 2021 and July 2022 via the electronic medical records at Konya Numune Hospital and Konya Private Medova Hospital (both in Konya, Turkey). Patients who received an oral isotretinoin total cumulative dose greater than 120 mg/kg were included in the study. Patient demographic data; cumulative isotretinoin doses; and alanine transaminase (ALT), aspartate transaminase (AST), γ-glutamyltransferase (GGT), creatinine kinase (CK), low-density lipoprotein cholesterol (LDL-C), and triglyceride (TG) levels during treatment were recorded. Baseline laboratory levels of those parameters were compared with levels of the same parameters from the second and fourth months of treatment. Comparisons for all parameters were made between the second- and fourth-month levels. Reference ranges are shown in Table 1. Abnormalities were graded according to the National Cancer Institute Common Terminology Criteria for Adverse Events v3.0 grading system.² This study was approved by the Karatay University (Konya, Turkey) ethical committee.

Statistical Analysis—The descriptive statistics of the measurements were presented as means, standard deviations, or medians (first and third quartiles). With respect to the normal distribution, the consistency of the measurements was evaluated with the Kolmogorov-Smirnov test, and small deviations from the normal distribution were observed. Changes in laboratory measurements were evaluated with simple repeated-measures analysis of variance, and changes that differed significantly were determined by a Holm-Sidak post hoc test. Relationships between total cumulative doses and laboratory measurements at second visits were evaluated by the Pearson correlation analysis. The statistical significance level was P<.05. SPSS Statistics 23 (IBM) was used in the calculations.

Results

Consecutive Data at Baseline and Follow-up—A total of 415 patients with a mean age (SD) of 21.49 (7.25) years (range, 12–53 years) were included in our study. The mean total cumulative dose (SD) of the patients was 7267.27 (1878.4) mg. The consecutive data of the means of the laboratory parameters are shown in Table 1 and Figure 1. There was no significant change in the ALT levels between baseline and the fourth month as well as between the second- and fourth-month assessments (both P=.311). When comparing the differences among AST, GGT, and LDL-C measurements, the levels increased significantly between baseline and the second month and between baseline and the fourth month (all P<.001). There was no significant difference in CK levels at all assessments (all P=.304). When the differences between TG measurements were compared, the changes between baseline and the second month (P<.001), baseline and the fourth month (P<.001), and the second and fourth months (P=.013) were significant (Figure 1).

Abnormal Laboratory Measurements—The distribution of abnormal laboratory measurements during treatment is shown in Table 2 and Figure 2. Grade 3 or higher elevations of liver transaminases (ALT, AST) and GGT were observed in fewer than 2% of patients during treatment compared with baseline (grade 3 elevations of ALT and AST together in 2 patients; grade 4 AST elevation in 1 patient; grade 3 elevations of ALT, AST, and GGT combined in 1 patient; isolated grade 3 GGT elevation in 1 patient). All of the patients who developed grade 3 liver transaminases and isolated grade 3 GGT elevation had improved values when these were rechecked within 2 weeks.

In the patient who developed hepatotoxicity in the second month, the ALT level rose from a baseline of 19 U/L to 169 U/L, the AST level from a baseline of 19 U/L to 61 U/L, and the GGT level from a baseline of 24 U/L to 124 U/L. The patient was asymptomatic. Liver function test levels returned to reference range 4 weeks after discontinuation of therapy. Hepatotoxicity did not recur after treatment was re-administered.

The patient who developed grade 4 AST elevation (364 U/L) experienced fatigue and myalgia. He had done vigorous exercise up to 2 days before the test and also had a grade 4 CK elevation (12,310 U/L). He was thought to have isotretinoin-related rhabdomyolysis. His treatment was discontinued, and he was advised to hydrate and rest. Treatment was re-started after 2 weeks. With frequent laboratory monitoring and avoidance of vigorous physical activity, the patient completed the remaining course of isotretinoin without any laboratory abnormalities or symptoms.

 

Creatinine kinase abnormalities in the second and fourth months compared with baseline were not statistically significant. The patients with grade 3 or higher CK elevations, except for the case with rhabdomyolysis, had no clinical signs or other characteristic laboratory findings of rhabdomyolysis.

Hypercholesterolemia (LDL-C ≥130 mg/dL) occurred most frequently, with a maximum of 280 mg/dL in 1 patient (in the fourth month) and less than 250 mg/dL in all other patients. Hypercholesterolemia occurred in 183 (44.1%) patients in the second month and in 166 (40.0%) patients in the fourth month. However, baseline abnormalities also were frequent (86 [20.7%]), and hypercholesterolemia persisted in the second and fourth months in all of these patients.

It was observed that the patients with TG abnormalities increased continuously in the second (99 [23.9%]) and fourth (113 [27.2%]) months compared with baseline (49 [11.8%]). Grade 3 TG elevations were observed in 2.2% of patients (n=9; 5 patients in the second month, 4 patients in the fourth month) during treatment compared with baseline, and all patients had grade 1 or 2 hypertriglyceridemia at baseline. Of the patients with grade 3 TG elevation, 3 patients in the second month and 2 patients in the fourth month were obese at baseline. No grade 4 TG elevations were observed. Complications related to hyperlipidemia, such as pancreatitis, were observed in 1 patient. No patient terminated treatment because of lipid abnormalities. The treatment of our patients with major hypercholesterolemia and/or grade 3 hypertriglyceridemia was interrupted. The hyperlipidemia of these patients was controlled by a low-fat diet and a short-term dose reduction.

Relationship Between Total Cumulative Dose and Laboratory Parameters—The relationships between the total cumulative dose and changes up to the fourth month are presented in Table 3. As the total dose increased, the changes in TG and LDL-C levels significantly increased in the fourth month (both P=.001). However, the degree of these relationships was weak. No significant correlation was found between the periodic changes of other laboratory parameters and the total dose.

Comment

The parameters followed in our study show that TG levels tend to increase continuously from baseline during isotretinoin treatment, while ALT, AST, GGT, and LDL-C levels increase in the second month and decrease at 4 months. Although this same trend occurs with CK levels, the change was not statistically significant. The most common laboratory abnormality in our study was hyperlipidemia. Levels of LDL-C and TG were both found to be statistically elevated in the second and fourth months of treatment compared with baseline. Parthasarathy et al³ reported that obesity had an important role in the increase of lipid levels in patients using isotretinoin at baseline. In our study, 5 of 9 patients (55.6%) with grade 3 TG elevation were obese, which supports the theory that obesity plays an important role in the increase in lipid levels. Up-to-date laboratory follow-up of lipids suggests that there is no need to follow up serum lipids after the second month of treatment. Patients with risk factors for hyperlipidemia, such as abdominal obesity and familial hyperlipidemia, do not require further follow-up if there is no increase in serum lipids in the first month of treatment.¹ The presence of grade 1 or 2 hypertriglyceridemia at baseline in all our patients with grade 3 TG elevation may suggest that periodic laboratory follow-up during isotretinoin treatment is necessary to detect patients with grade 3 and higher TG levels.

The lack of knowledge of other risk factors (eg, familial hyperlipidemia, insulin resistance) for hyperlipidemia in all patients at baseline may be a limitation of our study. Although hypercholesterolemia persisted in the follow-up of our patients with initial LDL-C abnormalities, hypercholesterolemia over 250 mg/dL was very rare (1 patient). Possible complications associated with serum lipid abnormalities are pancreatitis and metabolic syndrome.⁴ In our study, none of the patients with lipid abnormalities had any relevant clinical sequelae. The dose-dependent elevation of the changes in LDL-C and TG (Table 3) may be important to predict the significant elevation of lipids and the associated complications in patients with a high total cumulative dose target that may require a long treatment duration. However, considering the short follow-up periods in our patients, the absence of clinical sequelae may be misleading. There are differences in recommendations between the US and European guidelines for isotretinoin dosage. Although the US guidelines recommend a total cumulative dose target, the European guidelines recommend low-dose isotretinoin daily for at least 6 months instead of a cumulative dose.^5,6 The relationship between change in lipids and total cumulative dose in our study may not be similar in patients treated with the dosing regimen recommended by the European guidelines, as our patients received a total cumulative dose instead of a daily low-dose isotretinoin regimen, unlike the European guidelines.⁵

Most liver transaminase abnormalities were detected in the second month. Abnormalities in GGT were seen in the second month and remained elevated at the next follow-up. However, clinically important grade 3 transaminase and GGT elevations were rare. It has been reported that GGT levels are more specific than transaminases in measuring hepatotoxicity.⁷ The fact that our patient with hepatotoxicity had a grade 3 GGT elevation in addition to grade 3 transaminase elevations supports that GGT elevation is more specific than transaminase levels in measuring hepatotoxicity. When these parameters were rechecked in our patients with grade 3 transaminase elevations, except in the case of hepatotoxicity, transaminase elevations did not recur, and GGT elevations did not accompany elevated transaminases, which suggested that transaminases may be elevated due to an extrahepatic origin (eg, hemolysis, exercise).

Rhabdomyolysis secondary to isotretinoin is rare in the literature of acne studies. In addition to clinical findings such as myalgia and fatigue, increased CK and abnormal liver enzymes, specifically AST, suggest the development of rhabdomyolysis.⁸ Our patient who developed rhabdomyolysis also had a recent history of vigorous exercise, grade 4 CK, and AST elevations. Other patients with isolated grade 3 CK elevations were informed about possible clinical signs of rhabdomyolysis, and they were able to complete their courses without any incident. According to a study by Landau et al,⁹ isotretinoin-associated hyperCKemia has been reported as benign. Similarly, our study found that isolated CK elevation during isotretinoin treatment may be misleading as a sign of rhabdomyolysis. Instead, CK monitoring may be more appropriate and cost-effective in patients with suspected clinical signs of rhabdomyolysis or in those with major elevations in transaminases, especially AST.

Conclusion

According to our study, hyperlipidemia was the most common complication in acne patients using isotretinoin. It may be appropriate to monitor the TG level at 2-month intervals in patients with grade 1 or 2 TG elevation at baseline to detect the possible risk for developing grade 3 hyperlipidemia. Periodic monitoring of LDL-C and TG levels may be appropriate, especially in patients who require a high total cumulative dose of isotretinoin. Clinically important liver enzyme abnormalities were rare in our study. Our findings support the idea that routine monthly monitoring of normal laboratory parameters is unnecessary and wasteful. Additionally, periodic monitoring of abnormal laboratory parameters should be considered on an individual basis.

Isotretinoin is used in the treatment of nodulocystic and severe papulopustular acne. During the treatment period, laboratory monitoring is recommended to identify the risk for complications such as hepatotoxicity, teratogenicity, rhabdomyolysis, hyperlipidemia, and pancreatitis.¹ There is a lack of consensus of the frequency of follow-up of laboratory parameters during isotretinoin treatment. This study evaluated the changes in laboratory parameters used in daily practice for patients with acne who were treated with isotretinoin to determine the optimum test repetition frequency.

Materials and Methods

We conducted a retrospective study of data from patients who received oral isotretinoin therapy for acne between January 2021 and July 2022 via the electronic medical records at Konya Numune Hospital and Konya Private Medova Hospital (both in Konya, Turkey). Patients who received an oral isotretinoin total cumulative dose greater than 120 mg/kg were included in the study. Patient demographic data; cumulative isotretinoin doses; and alanine transaminase (ALT), aspartate transaminase (AST), γ-glutamyltransferase (GGT), creatinine kinase (CK), low-density lipoprotein cholesterol (LDL-C), and triglyceride (TG) levels during treatment were recorded. Baseline laboratory levels of those parameters were compared with levels of the same parameters from the second and fourth months of treatment. Comparisons for all parameters were made between the second- and fourth-month levels. Reference ranges are shown in Table 1. Abnormalities were graded according to the National Cancer Institute Common Terminology Criteria for Adverse Events v3.0 grading system.² This study was approved by the Karatay University (Konya, Turkey) ethical committee.

Statistical Analysis—The descriptive statistics of the measurements were presented as means, standard deviations, or medians (first and third quartiles). With respect to the normal distribution, the consistency of the measurements was evaluated with the Kolmogorov-Smirnov test, and small deviations from the normal distribution were observed. Changes in laboratory measurements were evaluated with simple repeated-measures analysis of variance, and changes that differed significantly were determined by a Holm-Sidak post hoc test. Relationships between total cumulative doses and laboratory measurements at second visits were evaluated by the Pearson correlation analysis. The statistical significance level was P<.05. SPSS Statistics 23 (IBM) was used in the calculations.

Results

Consecutive Data at Baseline and Follow-up—A total of 415 patients with a mean age (SD) of 21.49 (7.25) years (range, 12–53 years) were included in our study. The mean total cumulative dose (SD) of the patients was 7267.27 (1878.4) mg. The consecutive data of the means of the laboratory parameters are shown in Table 1 and Figure 1. There was no significant change in the ALT levels between baseline and the fourth month as well as between the second- and fourth-month assessments (both P=.311). When comparing the differences among AST, GGT, and LDL-C measurements, the levels increased significantly between baseline and the second month and between baseline and the fourth month (all P<.001). There was no significant difference in CK levels at all assessments (all P=.304). When the differences between TG measurements were compared, the changes between baseline and the second month (P<.001), baseline and the fourth month (P<.001), and the second and fourth months (P=.013) were significant (Figure 1).

Abnormal Laboratory Measurements—The distribution of abnormal laboratory measurements during treatment is shown in Table 2 and Figure 2. Grade 3 or higher elevations of liver transaminases (ALT, AST) and GGT were observed in fewer than 2% of patients during treatment compared with baseline (grade 3 elevations of ALT and AST together in 2 patients; grade 4 AST elevation in 1 patient; grade 3 elevations of ALT, AST, and GGT combined in 1 patient; isolated grade 3 GGT elevation in 1 patient). All of the patients who developed grade 3 liver transaminases and isolated grade 3 GGT elevation had improved values when these were rechecked within 2 weeks.

In the patient who developed hepatotoxicity in the second month, the ALT level rose from a baseline of 19 U/L to 169 U/L, the AST level from a baseline of 19 U/L to 61 U/L, and the GGT level from a baseline of 24 U/L to 124 U/L. The patient was asymptomatic. Liver function test levels returned to reference range 4 weeks after discontinuation of therapy. Hepatotoxicity did not recur after treatment was re-administered.

The patient who developed grade 4 AST elevation (364 U/L) experienced fatigue and myalgia. He had done vigorous exercise up to 2 days before the test and also had a grade 4 CK elevation (12,310 U/L). He was thought to have isotretinoin-related rhabdomyolysis. His treatment was discontinued, and he was advised to hydrate and rest. Treatment was re-started after 2 weeks. With frequent laboratory monitoring and avoidance of vigorous physical activity, the patient completed the remaining course of isotretinoin without any laboratory abnormalities or symptoms.

 

Creatinine kinase abnormalities in the second and fourth months compared with baseline were not statistically significant. The patients with grade 3 or higher CK elevations, except for the case with rhabdomyolysis, had no clinical signs or other characteristic laboratory findings of rhabdomyolysis.

Hypercholesterolemia (LDL-C ≥130 mg/dL) occurred most frequently, with a maximum of 280 mg/dL in 1 patient (in the fourth month) and less than 250 mg/dL in all other patients. Hypercholesterolemia occurred in 183 (44.1%) patients in the second month and in 166 (40.0%) patients in the fourth month. However, baseline abnormalities also were frequent (86 [20.7%]), and hypercholesterolemia persisted in the second and fourth months in all of these patients.

It was observed that the patients with TG abnormalities increased continuously in the second (99 [23.9%]) and fourth (113 [27.2%]) months compared with baseline (49 [11.8%]). Grade 3 TG elevations were observed in 2.2% of patients (n=9; 5 patients in the second month, 4 patients in the fourth month) during treatment compared with baseline, and all patients had grade 1 or 2 hypertriglyceridemia at baseline. Of the patients with grade 3 TG elevation, 3 patients in the second month and 2 patients in the fourth month were obese at baseline. No grade 4 TG elevations were observed. Complications related to hyperlipidemia, such as pancreatitis, were observed in 1 patient. No patient terminated treatment because of lipid abnormalities. The treatment of our patients with major hypercholesterolemia and/or grade 3 hypertriglyceridemia was interrupted. The hyperlipidemia of these patients was controlled by a low-fat diet and a short-term dose reduction.

Relationship Between Total Cumulative Dose and Laboratory Parameters—The relationships between the total cumulative dose and changes up to the fourth month are presented in Table 3. As the total dose increased, the changes in TG and LDL-C levels significantly increased in the fourth month (both P=.001). However, the degree of these relationships was weak. No significant correlation was found between the periodic changes of other laboratory parameters and the total dose.

Comment

The parameters followed in our study show that TG levels tend to increase continuously from baseline during isotretinoin treatment, while ALT, AST, GGT, and LDL-C levels increase in the second month and decrease at 4 months. Although this same trend occurs with CK levels, the change was not statistically significant. The most common laboratory abnormality in our study was hyperlipidemia. Levels of LDL-C and TG were both found to be statistically elevated in the second and fourth months of treatment compared with baseline. Parthasarathy et al³ reported that obesity had an important role in the increase of lipid levels in patients using isotretinoin at baseline. In our study, 5 of 9 patients (55.6%) with grade 3 TG elevation were obese, which supports the theory that obesity plays an important role in the increase in lipid levels. Up-to-date laboratory follow-up of lipids suggests that there is no need to follow up serum lipids after the second month of treatment. Patients with risk factors for hyperlipidemia, such as abdominal obesity and familial hyperlipidemia, do not require further follow-up if there is no increase in serum lipids in the first month of treatment.¹ The presence of grade 1 or 2 hypertriglyceridemia at baseline in all our patients with grade 3 TG elevation may suggest that periodic laboratory follow-up during isotretinoin treatment is necessary to detect patients with grade 3 and higher TG levels.

The lack of knowledge of other risk factors (eg, familial hyperlipidemia, insulin resistance) for hyperlipidemia in all patients at baseline may be a limitation of our study. Although hypercholesterolemia persisted in the follow-up of our patients with initial LDL-C abnormalities, hypercholesterolemia over 250 mg/dL was very rare (1 patient). Possible complications associated with serum lipid abnormalities are pancreatitis and metabolic syndrome.⁴ In our study, none of the patients with lipid abnormalities had any relevant clinical sequelae. The dose-dependent elevation of the changes in LDL-C and TG (Table 3) may be important to predict the significant elevation of lipids and the associated complications in patients with a high total cumulative dose target that may require a long treatment duration. However, considering the short follow-up periods in our patients, the absence of clinical sequelae may be misleading. There are differences in recommendations between the US and European guidelines for isotretinoin dosage. Although the US guidelines recommend a total cumulative dose target, the European guidelines recommend low-dose isotretinoin daily for at least 6 months instead of a cumulative dose.^5,6 The relationship between change in lipids and total cumulative dose in our study may not be similar in patients treated with the dosing regimen recommended by the European guidelines, as our patients received a total cumulative dose instead of a daily low-dose isotretinoin regimen, unlike the European guidelines.⁵

Most liver transaminase abnormalities were detected in the second month. Abnormalities in GGT were seen in the second month and remained elevated at the next follow-up. However, clinically important grade 3 transaminase and GGT elevations were rare. It has been reported that GGT levels are more specific than transaminases in measuring hepatotoxicity.⁷ The fact that our patient with hepatotoxicity had a grade 3 GGT elevation in addition to grade 3 transaminase elevations supports that GGT elevation is more specific than transaminase levels in measuring hepatotoxicity. When these parameters were rechecked in our patients with grade 3 transaminase elevations, except in the case of hepatotoxicity, transaminase elevations did not recur, and GGT elevations did not accompany elevated transaminases, which suggested that transaminases may be elevated due to an extrahepatic origin (eg, hemolysis, exercise).

Rhabdomyolysis secondary to isotretinoin is rare in the literature of acne studies. In addition to clinical findings such as myalgia and fatigue, increased CK and abnormal liver enzymes, specifically AST, suggest the development of rhabdomyolysis.⁸ Our patient who developed rhabdomyolysis also had a recent history of vigorous exercise, grade 4 CK, and AST elevations. Other patients with isolated grade 3 CK elevations were informed about possible clinical signs of rhabdomyolysis, and they were able to complete their courses without any incident. According to a study by Landau et al,⁹ isotretinoin-associated hyperCKemia has been reported as benign. Similarly, our study found that isolated CK elevation during isotretinoin treatment may be misleading as a sign of rhabdomyolysis. Instead, CK monitoring may be more appropriate and cost-effective in patients with suspected clinical signs of rhabdomyolysis or in those with major elevations in transaminases, especially AST.

Conclusion

According to our study, hyperlipidemia was the most common complication in acne patients using isotretinoin. It may be appropriate to monitor the TG level at 2-month intervals in patients with grade 1 or 2 TG elevation at baseline to detect the possible risk for developing grade 3 hyperlipidemia. Periodic monitoring of LDL-C and TG levels may be appropriate, especially in patients who require a high total cumulative dose of isotretinoin. Clinically important liver enzyme abnormalities were rare in our study. Our findings support the idea that routine monthly monitoring of normal laboratory parameters is unnecessary and wasteful. Additionally, periodic monitoring of abnormal laboratory parameters should be considered on an individual basis.

Days or weeks before a scheduled surgical or invasive procedure involving anesthesia, evaluations are conducted to assess a patient’s condition and risk, optimize their status, and prepare them for their procedure. A comprehensive pre-anesthesia evaluation visit includes a history of present illness, the evaluation of comorbidities and medication use, the assessment of health habits such as alcohol and tobacco use, functional capacity and nutritional assessments, and the identification of social support deficiencies that may influence recovery. It also includes a focused physical examination and laboratory and other ancillary testing as needed and may include optimization interventions such as anemia management or prehabilitation. Conducting pre-anesthesia evaluations before surgery has been shown to reduce delays and cancellations, unnecessary preprocedure testing, hospital length of stay, and in-hospital mortality.^1-4

The pre-anesthesia evaluation is usually conducted in person, although other modalities have been in use for several years and have accelerated since the advent of the COVID-19 pandemic. Specifically, audio-only telephone visits are used in many settings to conduct abbreviated forms of a pre-anesthesia evaluation, typically for less-invasive procedures. When patients are evaluated over the telephone, the physical examination and testing are deferred until the day of the procedure. Another modality is the use of synchronous video telehealth. Emerging evidence for the use of video-based care in anesthesiology provides encouraging results. Several institutions have proven the technological feasibility of performing preoperative evaluations via video.^5,6 Compared with in-person evaluations, these visits seem to have similar surgery cancellation rates, improved patient satisfaction, and reduced wait times and costs.^7-9

As part of a quality improvement project, we studied the use of telehealth for pre-anesthesia evaluations within the US Department of Veterans Affairs (VA). An internal review found overall low utilization of these modalities before the COVID-19 pandemic that accelerated toward telehealth during the pandemic: The largest uptake was with telephone visits. Given the increasing adoption of telehealth for pre-anesthesia evaluations and the marked preference for telephone over video modalities among VA practitioners during the COVID-19 pandemic, we sought to understand the barriers and facilitators to the adoption of telephone- and video-based pre-anesthesia evaluation visits within the VA.

Methods

Our objective was to assess health care practitioners’ (HCPs) preferences regarding pre-anesthesia evaluation modalities (in-person, telephone, or video), and the perceived advantages and barriers to adoption for each modality. We followed the Strengthening the Reporting of Observational studies in Epidemiology (STROBE) guideline and Checklist for statistical Assessment of Medical Papers (CHAMP) statement.^10,11 The survey was deemed a quality improvement activity that was exempt from institutional review board oversight by the VA National Anesthesia Program Office and the VA Office of Connected Care.

A survey was distributed to all VA anesthesiology service chiefs via email between April 27, 2022, and May 3, 2022. Three emails were sent to each participant (initial invitation and 2 reminders). The respondents were asked to identify themselves by facility and role and to indicate whether their anesthesiology service performed any pre-anesthesia evaluations, including any telephone- or video-based evaluations; and whether their service has a dedicated pre-anesthesia evaluation clinic.

A second set of questions referred to the use of telephone- and video-based preprocedure evaluations. The questions were based on branch logic and depended on the respondent’s answers concerning their use of telephone- and video-based evaluations. Questions included statements about perceived barriers to the adoption of these pre-anesthesia evaluation modalities. Each item was rated on a 5-point Likert scale, (completely disagree [1] to completely agree [5]). A third section measured acceptability and feasibility of video using the validated Acceptability of Intervention Measure (AIM) and Feasibility of Intervention Measure (FIM) questionnaires.¹² These instruments are 4-item measures of implementation outcomes that are often considered indicators of implementation success.¹³ Acceptability is the perception among implementation stakeholders that a given treatment, service, practice, or innovation is agreeable, palatable, or satisfactory. Feasibility is defined as the extent to which a new treatment or an innovation can be successfully used or carried out within a given agency or setting.¹³ The criterion for acceptability is personal, meaning that different HCPs may have differing needs, preferences, and expectations regarding the same intervention. The criterion for feasibility is practical. An intervention may be considered to be feasible if the required tasks can be performed easily or conveniently. Finally, 2 open-ended questions allowed respondents to identify the most important factor that allowed the implementation of telehealth for pre-anesthesia evaluations in their service, and provide comments about the use of telehealth for pre-anesthesia evaluations at the VA. All questions were developed by the authors except for the 2 implementation measure instruments.

The survey was administered using an electronic survey platform (Qualtrics, version April 2022) and sent by email alongside a brief introductory video. Participation was voluntary and anonymous, as no personal information was collected. Responses were attributed to each facility, using the self-declared affiliation. When an affiliation was not provided, we deduced it using the latitude/longitude of the respondent, a feature included in the survey software. No incentives were provided. Data were stored and maintained in a secure VA server. All completed surveys were included. Some facilities had > 1 complete response, and all were included. Facilities that provided > 1 response and where responses were discordant, we clarified with the facility service chief. Incomplete responses were excluded from the analysis.

 

Statistics

For this analysis, the 2 positive sentiment responses (agree and completely agree) and the 2 negative sentiment responses (disagree and completely disagree) in the Likert scale were collapsed into single categories (good and poor, respectively). The neither agree nor disagree responses were coded as neutral. Our analysis began with a visual exploration of all variables to evaluate the frequency, percentage, and near-zero variance for categorical variables.¹⁴ Near-zero variance occurs when a categorical variable has a low frequency of unique values over the sample size (ie, the variable is almost constant), and we addressed it by combining different variable categorizations. We handled missing values through imputation algorithms followed by sensitivity analyses to verify whether our results were stable with and without imputation. We performed comparisons for the exploratory analysis using P values for one-way analysis of variance tests for numeric variables and χ² tests for categorical variables. We considered P values < .05 to be statistically significant. We also used correlation matrices and plots as exploratory analysis tools to better understand all items’ correlations. We used Pearson, polychoric, and polyserial correlation tests as appropriate for numeric, ordinal, and logical items.

Our modeling strategy involved a series of generalized linear models (GLMs) with a Gaussian family, ie, multiple linear regression models, to assess the association between (1) facilities’ preferences regarding pre-anesthesia evaluation modalities; (2) advantages between modalities; and (3) barriers to the adoption of telehealth and the ability to perform different pre-anesthesia evaluation-related tasks. In addition, we used backward deletion to reach the most parsimonious model based on a series of likelihood-ratio tests comparing nested models. Results are reported as predicted means with 95% confidence intervals, with results being interpreted as significant when any 2 predicted means do not overlap between different estimates along with P for trends < .001. We performed all analyses using the R language.¹⁵

Results

Of 109 surveyed facilities, 50 (46%) responded to the survey. The final study sample included 67 responses, and 55 were included in the analysis. Twelve responses were excluded from the analysis as they were either incomplete or test responses. Three facilities had > 1 complete response (2 facilities had 2 responses and 1 facility had 4 responses), and these were all included in the analysis.

Thirty-six locations were complex inpatient facilities, and 32 (89%) had pre-anesthesia evaluation clinics (Table 1).

Twenty-two facilities reported using both telephone and video, 11 telephone only, 5 video only, and 12 neither. Considering the 55 individual responses, 25 respondents reported using both telephone and video, 12 reported using telephone only, 5 using video only, and 13 reported using neither telephone nor video for pre-anesthesia evaluations.

The ability to obtain a history of present illness was rated good/very good via telephone for 34 respondents (92%) and 25 for video (86%). Assessing comorbidities and health habits was rated good/very good via telephone for 32 respondents (89%) and 31 respondents (86%), respectively, and via video for 24 respondents (83%) and 23 respondents (79%), respectively (Figure 1).

Fewer respondents rated the ability to estimate exercise capacity or mental health pathology good/very good: 26 respondents (70%) and 23 respondents (62%) for telephone, respectively, and 18 (62%) and 17 (59%) for video, respectively. The ability to assess nutritional status was rated lowest with 9 respondents (24%) rating it positively for telephone and 15 (52%) for video.

To compare differences between the 2 remote pre-anesthesia evaluation modalities, we created GLMs evaluating the association between each modality and the perceived ability to perform the tasks. For GLMs, we transformed the values of the categories into numerical (ie, 1, poor; 2, neutral; 3, good). Compared with telephone, video was rated more favorably regarding the assessment of nutritional status (mean, 2.1; 95% CI, 1.8-2.3 vs mean, 2.4; 95% CI, 2.2-2.7; P = .04) (eAppendix 1, available at doi:10.12788/fp.0387). No other significant differences in ratings existed between the 2 remote pre-anesthesia evaluation modalities.

The most significant barriers (cited as significant or very significant in the survey) included the inability to perform a physical examination, which was noted by 13 respondents (72%) and 15 respondents (60%) for telephone and video, respectively. The inability to obtain vital signs was rated as a significant barrier for telephone by 12 respondents (67%) and for video by 15 respondents (60%)(Figure 2).

Other less-cited barriers included concerns about patient safety and risk; patient preference; cultural barriers; lack of support from staff; and lack of evidence for its effectiveness. Specific to video care, patients’ lack of access to a computer was cited as a barrier by 12 respondents (48%), whereas only 3 (17%) cited lack of telephone as a barrier. Lastly, lack of information technology support was cited as a barrier for video visits by 8 respondents (32%). To determine differences in perceived barriers to the implementation of phone vs video pre-anesthesia evaluations, we created GLM evaluating the association between these 2 modalities and the perceived ability to perform commonly performed pre-anesthesia evaluation visit tasks. For GLM, again we transformed the values of the categories into numeric (ie, not a significant barrier, 1; somewhat a barrier, 2; a significant barrier, 3). There were no significant differences in ratings between the 2 remote pre-anesthesia evaluation modalities (eAppendix 2, available at doi:10.12788/fp.0387).

The average FIM score was 3.7, with the highest score among respondents who used both phone and video (Table 2). The average AIM score was 3.4, with the highest score among respondents who used both telehealth modalities. The internal consistency of the implementation measures was excellent (Cronbach’s α 0.95 and 0.975 for FIM and AIM, respectively).

 

Discussion

We surveyed 109 anesthesiology services across the VA regarding barriers to implementing telephone- and video-based pre-anesthesia evaluation visits. We found that 12 (23%) of the 50 anesthesiology services responding to this survey still conduct the totality of their pre-anesthesia evaluations in person. This represents an opportunity to further disseminate the appropriate use of telehealth and potentially reduce travel time, costs, and low-value testing, as it is well established that remote pre-anesthesia evaluations for low-risk procedures are safe and effective.⁶

We also found no difference between telephone and video regarding users’ perceived ability to perform any of the basic pre-anesthesia evaluation tasks except for assessing patients’ nutritional status, which was rated as easier using video than telephone. According to those not using telephone and/or video, the biggest barriers to implementation of telehealth visits were the inability to obtain vital signs and to perform a physical examination. This finding was unexpected, as facilities that conduct remote evaluations typically defer these tasks to the day of surgery, a practice that has been well established and shown to be safe and efficient. Respondents also identified patient-level factors (eg, patient preference, lack of telephone or computer) as significant barriers. Finally, feasibility ratings were higher than acceptability ratings with regards to the implementation of telehealth.

In 2004, the first use of telehealth for pre-anesthesia evaluations was reported by Wong and colleagues.¹⁶ Since then, several case series and a literature review have documented the efficacy, safety, and patient and HCP satisfaction with the use of telehealth for pre-anesthesia evaluations. A study by Mullen-Fortino and colleagues showed reduced visit times when telehealth was used for pre-anesthesia evaluation.⁸ Another study at VA hospitals showed that 88% of veterans reported that telemedicine saved them time and money.¹⁷ A report of 35 patients in rural Australia reported 98% satisfaction with the video quality of the visit, 95% perceived efficacy, and 87% preference for telehealth compared with driving to be seen in person.¹⁸ These reports conflict with the perceptions of the respondents of our survey, who identified patient preference as an important barrier to adoption of telehealth. Given these findings, research is needed on veterans’ perceptions on the use of telehealth modalities for pre-anesthesia evaluations; if their perceptions are similarly favorable, it will be important to communicate this information to HCPs and leadership, which may help increase subsequent telehealth adoption.

Despite the reported safety, efficacy, and high satisfaction of video visits among anesthesiology teams conducting pre-anesthesia evaluations, its use remains low at VA. We have found that most facilities in the VA system chose telephone platforms during the COVID-19 pandemic. One possibility is that the adoption of video modalities among pre-anesthesia evaluation clinics in the VA system is resource intensive or difficult from the HCP’s perspective. When combined with the lack of perceived advantages over telephone as we found in our survey, most practitioners resort to the technologically less demanding and more familiar telephone platform. The results from FIM and AIM support this. While both telephone and video have high feasibility scores, acceptability scores are lower for video, even among those currently using this technology. Our findings do not rule out the utility of video-based care in perioperative medicine. Rather than a yes/no proposition, future studies need to establish the precise indications for video for pre-anesthesia evaluations; that is, situations where video visits offer an advantage over telephone. For example, video could be used to deliver preoperative optimization therapies, such as supervised exercise or mental health interventions or to guide the achievement of certain milestones before surgery in patients with chronic conditions, such as target glucose values or the treatment of anemia. Future studies should explore the perceived benefits of video over telephone among centers offering these more advanced optimization interventions.

Limitations

We received responses from a subset of VA anesthesiology services; therefore, they may not be representative of the entire VA system. Facilities designated by the VA as inpatient complex were overrepresented (72% of our sample vs 50% of the total facilities nationally), and ambulatory centers (those designed by the VA as ambulatory procedural center with basic or advanced capabilities) were underrepresented (2% of our sample vs 22% nationally). Despite this, the response rate was high, and no geographic area appeared to be underrepresented. In addition, we surveyed pre-anesthesia evaluation facilities led by anesthesiologists, and the results may not be representative of the preferences of HCPs working in nonanesthesiology led pre-anesthesia evaluation clinics. Finally, just 11 facilities used both telephone and video; therefore, a true direct comparison between these 2 platforms was limited. The VA serves a unique patient population, and the findings may not be completely applicable to the non-VA population.

Conclusions

We found no significant perceived advantages of video over telephone in the ability to conduct routine pre-anesthesia evaluations among a sample of anesthesiology HCPs in the VA except for the perceived ability to assess nutritional status. HCPs with no telehealth experience cited the inability to perform a physical examination and obtain vital signs as the most significant barriers to implementation. Respondents not using telephone cited concerns about safety. Video visits in this clinical setting had additional perceived barriers to implementation, such as lack of information technology and staff support and patient-level barriers. Video had lower acceptability by HCPs. Given findings that pre-anesthesia evaluations can be conducted effectively via telehealth and have high levels of patient satisfaction, future work should focus on increasing uptake of these remote modalities. Additionally, research on the most appropriate uses of video visits within perioperative care is also needed.

Days or weeks before a scheduled surgical or invasive procedure involving anesthesia, evaluations are conducted to assess a patient’s condition and risk, optimize their status, and prepare them for their procedure. A comprehensive pre-anesthesia evaluation visit includes a history of present illness, the evaluation of comorbidities and medication use, the assessment of health habits such as alcohol and tobacco use, functional capacity and nutritional assessments, and the identification of social support deficiencies that may influence recovery. It also includes a focused physical examination and laboratory and other ancillary testing as needed and may include optimization interventions such as anemia management or prehabilitation. Conducting pre-anesthesia evaluations before surgery has been shown to reduce delays and cancellations, unnecessary preprocedure testing, hospital length of stay, and in-hospital mortality.^1-4

The pre-anesthesia evaluation is usually conducted in person, although other modalities have been in use for several years and have accelerated since the advent of the COVID-19 pandemic. Specifically, audio-only telephone visits are used in many settings to conduct abbreviated forms of a pre-anesthesia evaluation, typically for less-invasive procedures. When patients are evaluated over the telephone, the physical examination and testing are deferred until the day of the procedure. Another modality is the use of synchronous video telehealth. Emerging evidence for the use of video-based care in anesthesiology provides encouraging results. Several institutions have proven the technological feasibility of performing preoperative evaluations via video.^5,6 Compared with in-person evaluations, these visits seem to have similar surgery cancellation rates, improved patient satisfaction, and reduced wait times and costs.^7-9

As part of a quality improvement project, we studied the use of telehealth for pre-anesthesia evaluations within the US Department of Veterans Affairs (VA). An internal review found overall low utilization of these modalities before the COVID-19 pandemic that accelerated toward telehealth during the pandemic: The largest uptake was with telephone visits. Given the increasing adoption of telehealth for pre-anesthesia evaluations and the marked preference for telephone over video modalities among VA practitioners during the COVID-19 pandemic, we sought to understand the barriers and facilitators to the adoption of telephone- and video-based pre-anesthesia evaluation visits within the VA.

Methods

Our objective was to assess health care practitioners’ (HCPs) preferences regarding pre-anesthesia evaluation modalities (in-person, telephone, or video), and the perceived advantages and barriers to adoption for each modality. We followed the Strengthening the Reporting of Observational studies in Epidemiology (STROBE) guideline and Checklist for statistical Assessment of Medical Papers (CHAMP) statement.^10,11 The survey was deemed a quality improvement activity that was exempt from institutional review board oversight by the VA National Anesthesia Program Office and the VA Office of Connected Care.

A survey was distributed to all VA anesthesiology service chiefs via email between April 27, 2022, and May 3, 2022. Three emails were sent to each participant (initial invitation and 2 reminders). The respondents were asked to identify themselves by facility and role and to indicate whether their anesthesiology service performed any pre-anesthesia evaluations, including any telephone- or video-based evaluations; and whether their service has a dedicated pre-anesthesia evaluation clinic.

A second set of questions referred to the use of telephone- and video-based preprocedure evaluations. The questions were based on branch logic and depended on the respondent’s answers concerning their use of telephone- and video-based evaluations. Questions included statements about perceived barriers to the adoption of these pre-anesthesia evaluation modalities. Each item was rated on a 5-point Likert scale, (completely disagree [1] to completely agree [5]). A third section measured acceptability and feasibility of video using the validated Acceptability of Intervention Measure (AIM) and Feasibility of Intervention Measure (FIM) questionnaires.¹² These instruments are 4-item measures of implementation outcomes that are often considered indicators of implementation success.¹³ Acceptability is the perception among implementation stakeholders that a given treatment, service, practice, or innovation is agreeable, palatable, or satisfactory. Feasibility is defined as the extent to which a new treatment or an innovation can be successfully used or carried out within a given agency or setting.¹³ The criterion for acceptability is personal, meaning that different HCPs may have differing needs, preferences, and expectations regarding the same intervention. The criterion for feasibility is practical. An intervention may be considered to be feasible if the required tasks can be performed easily or conveniently. Finally, 2 open-ended questions allowed respondents to identify the most important factor that allowed the implementation of telehealth for pre-anesthesia evaluations in their service, and provide comments about the use of telehealth for pre-anesthesia evaluations at the VA. All questions were developed by the authors except for the 2 implementation measure instruments.

The survey was administered using an electronic survey platform (Qualtrics, version April 2022) and sent by email alongside a brief introductory video. Participation was voluntary and anonymous, as no personal information was collected. Responses were attributed to each facility, using the self-declared affiliation. When an affiliation was not provided, we deduced it using the latitude/longitude of the respondent, a feature included in the survey software. No incentives were provided. Data were stored and maintained in a secure VA server. All completed surveys were included. Some facilities had > 1 complete response, and all were included. Facilities that provided > 1 response and where responses were discordant, we clarified with the facility service chief. Incomplete responses were excluded from the analysis.

 

Statistics

For this analysis, the 2 positive sentiment responses (agree and completely agree) and the 2 negative sentiment responses (disagree and completely disagree) in the Likert scale were collapsed into single categories (good and poor, respectively). The neither agree nor disagree responses were coded as neutral. Our analysis began with a visual exploration of all variables to evaluate the frequency, percentage, and near-zero variance for categorical variables.¹⁴ Near-zero variance occurs when a categorical variable has a low frequency of unique values over the sample size (ie, the variable is almost constant), and we addressed it by combining different variable categorizations. We handled missing values through imputation algorithms followed by sensitivity analyses to verify whether our results were stable with and without imputation. We performed comparisons for the exploratory analysis using P values for one-way analysis of variance tests for numeric variables and χ² tests for categorical variables. We considered P values < .05 to be statistically significant. We also used correlation matrices and plots as exploratory analysis tools to better understand all items’ correlations. We used Pearson, polychoric, and polyserial correlation tests as appropriate for numeric, ordinal, and logical items.

Our modeling strategy involved a series of generalized linear models (GLMs) with a Gaussian family, ie, multiple linear regression models, to assess the association between (1) facilities’ preferences regarding pre-anesthesia evaluation modalities; (2) advantages between modalities; and (3) barriers to the adoption of telehealth and the ability to perform different pre-anesthesia evaluation-related tasks. In addition, we used backward deletion to reach the most parsimonious model based on a series of likelihood-ratio tests comparing nested models. Results are reported as predicted means with 95% confidence intervals, with results being interpreted as significant when any 2 predicted means do not overlap between different estimates along with P for trends < .001. We performed all analyses using the R language.¹⁵

Results

Of 109 surveyed facilities, 50 (46%) responded to the survey. The final study sample included 67 responses, and 55 were included in the analysis. Twelve responses were excluded from the analysis as they were either incomplete or test responses. Three facilities had > 1 complete response (2 facilities had 2 responses and 1 facility had 4 responses), and these were all included in the analysis.

Thirty-six locations were complex inpatient facilities, and 32 (89%) had pre-anesthesia evaluation clinics (Table 1).

Twenty-two facilities reported using both telephone and video, 11 telephone only, 5 video only, and 12 neither. Considering the 55 individual responses, 25 respondents reported using both telephone and video, 12 reported using telephone only, 5 using video only, and 13 reported using neither telephone nor video for pre-anesthesia evaluations.

The ability to obtain a history of present illness was rated good/very good via telephone for 34 respondents (92%) and 25 for video (86%). Assessing comorbidities and health habits was rated good/very good via telephone for 32 respondents (89%) and 31 respondents (86%), respectively, and via video for 24 respondents (83%) and 23 respondents (79%), respectively (Figure 1).

Fewer respondents rated the ability to estimate exercise capacity or mental health pathology good/very good: 26 respondents (70%) and 23 respondents (62%) for telephone, respectively, and 18 (62%) and 17 (59%) for video, respectively. The ability to assess nutritional status was rated lowest with 9 respondents (24%) rating it positively for telephone and 15 (52%) for video.

To compare differences between the 2 remote pre-anesthesia evaluation modalities, we created GLMs evaluating the association between each modality and the perceived ability to perform the tasks. For GLMs, we transformed the values of the categories into numerical (ie, 1, poor; 2, neutral; 3, good). Compared with telephone, video was rated more favorably regarding the assessment of nutritional status (mean, 2.1; 95% CI, 1.8-2.3 vs mean, 2.4; 95% CI, 2.2-2.7; P = .04) (eAppendix 1, available at doi:10.12788/fp.0387). No other significant differences in ratings existed between the 2 remote pre-anesthesia evaluation modalities.

The most significant barriers (cited as significant or very significant in the survey) included the inability to perform a physical examination, which was noted by 13 respondents (72%) and 15 respondents (60%) for telephone and video, respectively. The inability to obtain vital signs was rated as a significant barrier for telephone by 12 respondents (67%) and for video by 15 respondents (60%)(Figure 2).

Other less-cited barriers included concerns about patient safety and risk; patient preference; cultural barriers; lack of support from staff; and lack of evidence for its effectiveness. Specific to video care, patients’ lack of access to a computer was cited as a barrier by 12 respondents (48%), whereas only 3 (17%) cited lack of telephone as a barrier. Lastly, lack of information technology support was cited as a barrier for video visits by 8 respondents (32%). To determine differences in perceived barriers to the implementation of phone vs video pre-anesthesia evaluations, we created GLM evaluating the association between these 2 modalities and the perceived ability to perform commonly performed pre-anesthesia evaluation visit tasks. For GLM, again we transformed the values of the categories into numeric (ie, not a significant barrier, 1; somewhat a barrier, 2; a significant barrier, 3). There were no significant differences in ratings between the 2 remote pre-anesthesia evaluation modalities (eAppendix 2, available at doi:10.12788/fp.0387).

The average FIM score was 3.7, with the highest score among respondents who used both phone and video (Table 2). The average AIM score was 3.4, with the highest score among respondents who used both telehealth modalities. The internal consistency of the implementation measures was excellent (Cronbach’s α 0.95 and 0.975 for FIM and AIM, respectively).

 

Discussion

We surveyed 109 anesthesiology services across the VA regarding barriers to implementing telephone- and video-based pre-anesthesia evaluation visits. We found that 12 (23%) of the 50 anesthesiology services responding to this survey still conduct the totality of their pre-anesthesia evaluations in person. This represents an opportunity to further disseminate the appropriate use of telehealth and potentially reduce travel time, costs, and low-value testing, as it is well established that remote pre-anesthesia evaluations for low-risk procedures are safe and effective.⁶

We also found no difference between telephone and video regarding users’ perceived ability to perform any of the basic pre-anesthesia evaluation tasks except for assessing patients’ nutritional status, which was rated as easier using video than telephone. According to those not using telephone and/or video, the biggest barriers to implementation of telehealth visits were the inability to obtain vital signs and to perform a physical examination. This finding was unexpected, as facilities that conduct remote evaluations typically defer these tasks to the day of surgery, a practice that has been well established and shown to be safe and efficient. Respondents also identified patient-level factors (eg, patient preference, lack of telephone or computer) as significant barriers. Finally, feasibility ratings were higher than acceptability ratings with regards to the implementation of telehealth.

In 2004, the first use of telehealth for pre-anesthesia evaluations was reported by Wong and colleagues.¹⁶ Since then, several case series and a literature review have documented the efficacy, safety, and patient and HCP satisfaction with the use of telehealth for pre-anesthesia evaluations. A study by Mullen-Fortino and colleagues showed reduced visit times when telehealth was used for pre-anesthesia evaluation.⁸ Another study at VA hospitals showed that 88% of veterans reported that telemedicine saved them time and money.¹⁷ A report of 35 patients in rural Australia reported 98% satisfaction with the video quality of the visit, 95% perceived efficacy, and 87% preference for telehealth compared with driving to be seen in person.¹⁸ These reports conflict with the perceptions of the respondents of our survey, who identified patient preference as an important barrier to adoption of telehealth. Given these findings, research is needed on veterans’ perceptions on the use of telehealth modalities for pre-anesthesia evaluations; if their perceptions are similarly favorable, it will be important to communicate this information to HCPs and leadership, which may help increase subsequent telehealth adoption.

Despite the reported safety, efficacy, and high satisfaction of video visits among anesthesiology teams conducting pre-anesthesia evaluations, its use remains low at VA. We have found that most facilities in the VA system chose telephone platforms during the COVID-19 pandemic. One possibility is that the adoption of video modalities among pre-anesthesia evaluation clinics in the VA system is resource intensive or difficult from the HCP’s perspective. When combined with the lack of perceived advantages over telephone as we found in our survey, most practitioners resort to the technologically less demanding and more familiar telephone platform. The results from FIM and AIM support this. While both telephone and video have high feasibility scores, acceptability scores are lower for video, even among those currently using this technology. Our findings do not rule out the utility of video-based care in perioperative medicine. Rather than a yes/no proposition, future studies need to establish the precise indications for video for pre-anesthesia evaluations; that is, situations where video visits offer an advantage over telephone. For example, video could be used to deliver preoperative optimization therapies, such as supervised exercise or mental health interventions or to guide the achievement of certain milestones before surgery in patients with chronic conditions, such as target glucose values or the treatment of anemia. Future studies should explore the perceived benefits of video over telephone among centers offering these more advanced optimization interventions.

Limitations

We received responses from a subset of VA anesthesiology services; therefore, they may not be representative of the entire VA system. Facilities designated by the VA as inpatient complex were overrepresented (72% of our sample vs 50% of the total facilities nationally), and ambulatory centers (those designed by the VA as ambulatory procedural center with basic or advanced capabilities) were underrepresented (2% of our sample vs 22% nationally). Despite this, the response rate was high, and no geographic area appeared to be underrepresented. In addition, we surveyed pre-anesthesia evaluation facilities led by anesthesiologists, and the results may not be representative of the preferences of HCPs working in nonanesthesiology led pre-anesthesia evaluation clinics. Finally, just 11 facilities used both telephone and video; therefore, a true direct comparison between these 2 platforms was limited. The VA serves a unique patient population, and the findings may not be completely applicable to the non-VA population.

Conclusions

We found no significant perceived advantages of video over telephone in the ability to conduct routine pre-anesthesia evaluations among a sample of anesthesiology HCPs in the VA except for the perceived ability to assess nutritional status. HCPs with no telehealth experience cited the inability to perform a physical examination and obtain vital signs as the most significant barriers to implementation. Respondents not using telephone cited concerns about safety. Video visits in this clinical setting had additional perceived barriers to implementation, such as lack of information technology and staff support and patient-level barriers. Video had lower acceptability by HCPs. Given findings that pre-anesthesia evaluations can be conducted effectively via telehealth and have high levels of patient satisfaction, future work should focus on increasing uptake of these remote modalities. Additionally, research on the most appropriate uses of video visits within perioperative care is also needed.

Days or weeks before a scheduled surgical or invasive procedure involving anesthesia, evaluations are conducted to assess a patient’s condition and risk, optimize their status, and prepare them for their procedure. A comprehensive pre-anesthesia evaluation visit includes a history of present illness, the evaluation of comorbidities and medication use, the assessment of health habits such as alcohol and tobacco use, functional capacity and nutritional assessments, and the identification of social support deficiencies that may influence recovery. It also includes a focused physical examination and laboratory and other ancillary testing as needed and may include optimization interventions such as anemia management or prehabilitation. Conducting pre-anesthesia evaluations before surgery has been shown to reduce delays and cancellations, unnecessary preprocedure testing, hospital length of stay, and in-hospital mortality.^1-4

The pre-anesthesia evaluation is usually conducted in person, although other modalities have been in use for several years and have accelerated since the advent of the COVID-19 pandemic. Specifically, audio-only telephone visits are used in many settings to conduct abbreviated forms of a pre-anesthesia evaluation, typically for less-invasive procedures. When patients are evaluated over the telephone, the physical examination and testing are deferred until the day of the procedure. Another modality is the use of synchronous video telehealth. Emerging evidence for the use of video-based care in anesthesiology provides encouraging results. Several institutions have proven the technological feasibility of performing preoperative evaluations via video.^5,6 Compared with in-person evaluations, these visits seem to have similar surgery cancellation rates, improved patient satisfaction, and reduced wait times and costs.^7-9

As part of a quality improvement project, we studied the use of telehealth for pre-anesthesia evaluations within the US Department of Veterans Affairs (VA). An internal review found overall low utilization of these modalities before the COVID-19 pandemic that accelerated toward telehealth during the pandemic: The largest uptake was with telephone visits. Given the increasing adoption of telehealth for pre-anesthesia evaluations and the marked preference for telephone over video modalities among VA practitioners during the COVID-19 pandemic, we sought to understand the barriers and facilitators to the adoption of telephone- and video-based pre-anesthesia evaluation visits within the VA.

Methods

Our objective was to assess health care practitioners’ (HCPs) preferences regarding pre-anesthesia evaluation modalities (in-person, telephone, or video), and the perceived advantages and barriers to adoption for each modality. We followed the Strengthening the Reporting of Observational studies in Epidemiology (STROBE) guideline and Checklist for statistical Assessment of Medical Papers (CHAMP) statement.^10,11 The survey was deemed a quality improvement activity that was exempt from institutional review board oversight by the VA National Anesthesia Program Office and the VA Office of Connected Care.

A survey was distributed to all VA anesthesiology service chiefs via email between April 27, 2022, and May 3, 2022. Three emails were sent to each participant (initial invitation and 2 reminders). The respondents were asked to identify themselves by facility and role and to indicate whether their anesthesiology service performed any pre-anesthesia evaluations, including any telephone- or video-based evaluations; and whether their service has a dedicated pre-anesthesia evaluation clinic.

A second set of questions referred to the use of telephone- and video-based preprocedure evaluations. The questions were based on branch logic and depended on the respondent’s answers concerning their use of telephone- and video-based evaluations. Questions included statements about perceived barriers to the adoption of these pre-anesthesia evaluation modalities. Each item was rated on a 5-point Likert scale, (completely disagree [1] to completely agree [5]). A third section measured acceptability and feasibility of video using the validated Acceptability of Intervention Measure (AIM) and Feasibility of Intervention Measure (FIM) questionnaires.¹² These instruments are 4-item measures of implementation outcomes that are often considered indicators of implementation success.¹³ Acceptability is the perception among implementation stakeholders that a given treatment, service, practice, or innovation is agreeable, palatable, or satisfactory. Feasibility is defined as the extent to which a new treatment or an innovation can be successfully used or carried out within a given agency or setting.¹³ The criterion for acceptability is personal, meaning that different HCPs may have differing needs, preferences, and expectations regarding the same intervention. The criterion for feasibility is practical. An intervention may be considered to be feasible if the required tasks can be performed easily or conveniently. Finally, 2 open-ended questions allowed respondents to identify the most important factor that allowed the implementation of telehealth for pre-anesthesia evaluations in their service, and provide comments about the use of telehealth for pre-anesthesia evaluations at the VA. All questions were developed by the authors except for the 2 implementation measure instruments.

The survey was administered using an electronic survey platform (Qualtrics, version April 2022) and sent by email alongside a brief introductory video. Participation was voluntary and anonymous, as no personal information was collected. Responses were attributed to each facility, using the self-declared affiliation. When an affiliation was not provided, we deduced it using the latitude/longitude of the respondent, a feature included in the survey software. No incentives were provided. Data were stored and maintained in a secure VA server. All completed surveys were included. Some facilities had > 1 complete response, and all were included. Facilities that provided > 1 response and where responses were discordant, we clarified with the facility service chief. Incomplete responses were excluded from the analysis.

 

Statistics

For this analysis, the 2 positive sentiment responses (agree and completely agree) and the 2 negative sentiment responses (disagree and completely disagree) in the Likert scale were collapsed into single categories (good and poor, respectively). The neither agree nor disagree responses were coded as neutral. Our analysis began with a visual exploration of all variables to evaluate the frequency, percentage, and near-zero variance for categorical variables.¹⁴ Near-zero variance occurs when a categorical variable has a low frequency of unique values over the sample size (ie, the variable is almost constant), and we addressed it by combining different variable categorizations. We handled missing values through imputation algorithms followed by sensitivity analyses to verify whether our results were stable with and without imputation. We performed comparisons for the exploratory analysis using P values for one-way analysis of variance tests for numeric variables and χ² tests for categorical variables. We considered P values < .05 to be statistically significant. We also used correlation matrices and plots as exploratory analysis tools to better understand all items’ correlations. We used Pearson, polychoric, and polyserial correlation tests as appropriate for numeric, ordinal, and logical items.

Our modeling strategy involved a series of generalized linear models (GLMs) with a Gaussian family, ie, multiple linear regression models, to assess the association between (1) facilities’ preferences regarding pre-anesthesia evaluation modalities; (2) advantages between modalities; and (3) barriers to the adoption of telehealth and the ability to perform different pre-anesthesia evaluation-related tasks. In addition, we used backward deletion to reach the most parsimonious model based on a series of likelihood-ratio tests comparing nested models. Results are reported as predicted means with 95% confidence intervals, with results being interpreted as significant when any 2 predicted means do not overlap between different estimates along with P for trends < .001. We performed all analyses using the R language.¹⁵

Results

Of 109 surveyed facilities, 50 (46%) responded to the survey. The final study sample included 67 responses, and 55 were included in the analysis. Twelve responses were excluded from the analysis as they were either incomplete or test responses. Three facilities had > 1 complete response (2 facilities had 2 responses and 1 facility had 4 responses), and these were all included in the analysis.

Thirty-six locations were complex inpatient facilities, and 32 (89%) had pre-anesthesia evaluation clinics (Table 1).

Twenty-two facilities reported using both telephone and video, 11 telephone only, 5 video only, and 12 neither. Considering the 55 individual responses, 25 respondents reported using both telephone and video, 12 reported using telephone only, 5 using video only, and 13 reported using neither telephone nor video for pre-anesthesia evaluations.

The ability to obtain a history of present illness was rated good/very good via telephone for 34 respondents (92%) and 25 for video (86%). Assessing comorbidities and health habits was rated good/very good via telephone for 32 respondents (89%) and 31 respondents (86%), respectively, and via video for 24 respondents (83%) and 23 respondents (79%), respectively (Figure 1).

Fewer respondents rated the ability to estimate exercise capacity or mental health pathology good/very good: 26 respondents (70%) and 23 respondents (62%) for telephone, respectively, and 18 (62%) and 17 (59%) for video, respectively. The ability to assess nutritional status was rated lowest with 9 respondents (24%) rating it positively for telephone and 15 (52%) for video.

To compare differences between the 2 remote pre-anesthesia evaluation modalities, we created GLMs evaluating the association between each modality and the perceived ability to perform the tasks. For GLMs, we transformed the values of the categories into numerical (ie, 1, poor; 2, neutral; 3, good). Compared with telephone, video was rated more favorably regarding the assessment of nutritional status (mean, 2.1; 95% CI, 1.8-2.3 vs mean, 2.4; 95% CI, 2.2-2.7; P = .04) (eAppendix 1, available at doi:10.12788/fp.0387). No other significant differences in ratings existed between the 2 remote pre-anesthesia evaluation modalities.

The most significant barriers (cited as significant or very significant in the survey) included the inability to perform a physical examination, which was noted by 13 respondents (72%) and 15 respondents (60%) for telephone and video, respectively. The inability to obtain vital signs was rated as a significant barrier for telephone by 12 respondents (67%) and for video by 15 respondents (60%)(Figure 2).

Other less-cited barriers included concerns about patient safety and risk; patient preference; cultural barriers; lack of support from staff; and lack of evidence for its effectiveness. Specific to video care, patients’ lack of access to a computer was cited as a barrier by 12 respondents (48%), whereas only 3 (17%) cited lack of telephone as a barrier. Lastly, lack of information technology support was cited as a barrier for video visits by 8 respondents (32%). To determine differences in perceived barriers to the implementation of phone vs video pre-anesthesia evaluations, we created GLM evaluating the association between these 2 modalities and the perceived ability to perform commonly performed pre-anesthesia evaluation visit tasks. For GLM, again we transformed the values of the categories into numeric (ie, not a significant barrier, 1; somewhat a barrier, 2; a significant barrier, 3). There were no significant differences in ratings between the 2 remote pre-anesthesia evaluation modalities (eAppendix 2, available at doi:10.12788/fp.0387).

The average FIM score was 3.7, with the highest score among respondents who used both phone and video (Table 2). The average AIM score was 3.4, with the highest score among respondents who used both telehealth modalities. The internal consistency of the implementation measures was excellent (Cronbach’s α 0.95 and 0.975 for FIM and AIM, respectively).

 

Discussion

We surveyed 109 anesthesiology services across the VA regarding barriers to implementing telephone- and video-based pre-anesthesia evaluation visits. We found that 12 (23%) of the 50 anesthesiology services responding to this survey still conduct the totality of their pre-anesthesia evaluations in person. This represents an opportunity to further disseminate the appropriate use of telehealth and potentially reduce travel time, costs, and low-value testing, as it is well established that remote pre-anesthesia evaluations for low-risk procedures are safe and effective.⁶

We also found no difference between telephone and video regarding users’ perceived ability to perform any of the basic pre-anesthesia evaluation tasks except for assessing patients’ nutritional status, which was rated as easier using video than telephone. According to those not using telephone and/or video, the biggest barriers to implementation of telehealth visits were the inability to obtain vital signs and to perform a physical examination. This finding was unexpected, as facilities that conduct remote evaluations typically defer these tasks to the day of surgery, a practice that has been well established and shown to be safe and efficient. Respondents also identified patient-level factors (eg, patient preference, lack of telephone or computer) as significant barriers. Finally, feasibility ratings were higher than acceptability ratings with regards to the implementation of telehealth.

In 2004, the first use of telehealth for pre-anesthesia evaluations was reported by Wong and colleagues.¹⁶ Since then, several case series and a literature review have documented the efficacy, safety, and patient and HCP satisfaction with the use of telehealth for pre-anesthesia evaluations. A study by Mullen-Fortino and colleagues showed reduced visit times when telehealth was used for pre-anesthesia evaluation.⁸ Another study at VA hospitals showed that 88% of veterans reported that telemedicine saved them time and money.¹⁷ A report of 35 patients in rural Australia reported 98% satisfaction with the video quality of the visit, 95% perceived efficacy, and 87% preference for telehealth compared with driving to be seen in person.¹⁸ These reports conflict with the perceptions of the respondents of our survey, who identified patient preference as an important barrier to adoption of telehealth. Given these findings, research is needed on veterans’ perceptions on the use of telehealth modalities for pre-anesthesia evaluations; if their perceptions are similarly favorable, it will be important to communicate this information to HCPs and leadership, which may help increase subsequent telehealth adoption.

Despite the reported safety, efficacy, and high satisfaction of video visits among anesthesiology teams conducting pre-anesthesia evaluations, its use remains low at VA. We have found that most facilities in the VA system chose telephone platforms during the COVID-19 pandemic. One possibility is that the adoption of video modalities among pre-anesthesia evaluation clinics in the VA system is resource intensive or difficult from the HCP’s perspective. When combined with the lack of perceived advantages over telephone as we found in our survey, most practitioners resort to the technologically less demanding and more familiar telephone platform. The results from FIM and AIM support this. While both telephone and video have high feasibility scores, acceptability scores are lower for video, even among those currently using this technology. Our findings do not rule out the utility of video-based care in perioperative medicine. Rather than a yes/no proposition, future studies need to establish the precise indications for video for pre-anesthesia evaluations; that is, situations where video visits offer an advantage over telephone. For example, video could be used to deliver preoperative optimization therapies, such as supervised exercise or mental health interventions or to guide the achievement of certain milestones before surgery in patients with chronic conditions, such as target glucose values or the treatment of anemia. Future studies should explore the perceived benefits of video over telephone among centers offering these more advanced optimization interventions.

Limitations

We received responses from a subset of VA anesthesiology services; therefore, they may not be representative of the entire VA system. Facilities designated by the VA as inpatient complex were overrepresented (72% of our sample vs 50% of the total facilities nationally), and ambulatory centers (those designed by the VA as ambulatory procedural center with basic or advanced capabilities) were underrepresented (2% of our sample vs 22% nationally). Despite this, the response rate was high, and no geographic area appeared to be underrepresented. In addition, we surveyed pre-anesthesia evaluation facilities led by anesthesiologists, and the results may not be representative of the preferences of HCPs working in nonanesthesiology led pre-anesthesia evaluation clinics. Finally, just 11 facilities used both telephone and video; therefore, a true direct comparison between these 2 platforms was limited. The VA serves a unique patient population, and the findings may not be completely applicable to the non-VA population.

Conclusions

We found no significant perceived advantages of video over telephone in the ability to conduct routine pre-anesthesia evaluations among a sample of anesthesiology HCPs in the VA except for the perceived ability to assess nutritional status. HCPs with no telehealth experience cited the inability to perform a physical examination and obtain vital signs as the most significant barriers to implementation. Respondents not using telephone cited concerns about safety. Video visits in this clinical setting had additional perceived barriers to implementation, such as lack of information technology and staff support and patient-level barriers. Video had lower acceptability by HCPs. Given findings that pre-anesthesia evaluations can be conducted effectively via telehealth and have high levels of patient satisfaction, future work should focus on increasing uptake of these remote modalities. Additionally, research on the most appropriate uses of video visits within perioperative care is also needed.

To the Editor:

Peer-reviewed publications are important determinants for promotions, academic leadership, and grants in dermatology.¹ The impact of the COVID-19 pandemic on dermatology research productivity remains an area of investigation. We sought to determine authorship trends for males and females during the pandemic.

A cross-sectional retrospective study of the top 20 dermatology journals—determined by impact factor and Google Scholar H5-index—was conducted to identify manuscripts with submission date specified prepandemic (May 1, 2019–October 31, 2019) and during the pandemic (May 1, 2020–October 31, 2020). Submission date, first/last author name, sex, and affiliated country were extracted. Single authors were designated as first authors. Gender API (https://gender-api.com/en/) classified gender. A χ² test (P<.05) compared differences in proportions of female first/last authors from 2019 to 2020.

Overall, 811 and 1061 articles submitted in 2019 and 2020, respectively, were included. There were 1517 articles submitted to clinical journals and 355 articles submitted to basic science journals (Table). For the 7 clinical journals included, there was a 7.7% decrease in the proportion of female last authors in 2020 vs 2019 (P=.002), with the largest decrease between August and September 2020. Although other comparisons did not yield statistically significant differences (P>.05 all)(Table), several trends were observed. For clinical journals, there was a 1.8% decrease in the proportion of female first authors. For the 4 basic science journals included, there was a 4.9% increase and a 0.3% decrease in percentages of female first and last authors, respectively, for 2020 vs 2019.

Our findings indicate that the COVID-19 pandemic may have impacted female authors’ productivity in clinical dermatology publications. In a survey-based study for 2010 to 2011, female physician-researchers (n=437) spent 8.5 more hours per week on domestic activities and childcare and were more likely to take time off for childcare if their partner worked full time compared with males (n=612)(42.6% vs 12.4%, respectively).² Our observation that female last authors had a significant decrease in publications may suggest that this population had a disproportionate burden of domestic labor and childcare during the pandemic. It is possible that last authors, who generally are more senior researchers, may be more likely to have childcare, eldercare, and other types of domestic responsibilities. Similarly, in a study of surgery submissions (n=1068), there were 6%, 7%, and 4% decreases in percentages of female last, corresponding, and first authors, respectively, from 2019 to 2020.³Our study had limitations. Only 11 journals were analyzed because others did not have specified submission dates. Some journals only provided submission information for a subset of articles (eg, those published in the In Press section), which may have accounted for the large discrepancy in submission numbers for 2019 to 2020. Gender could not be determined for 1% of authors and was limited to female and male. Although our study submission time frame (May–October 2020) aimed at identifying research conducted during the height of the COVID-19 pandemic, some of these studies may have been conducted months or years before the pandemic. Future studies should focus on longer and more comprehensive time frames. Finally, estimated dates of stay-at-home orders fail to consider differences within countries.

The proportion of female US-affiliated first and last authors publishing in dermatology journals increased from 12% to 48% in 1976 and from 6% to 31% in 2006,⁴ which is encouraging. However, a gender gap persists, with one-third of National Institutes of Health grants in dermatology and one-fourth of research project grants in dermatology awarded to women.⁵ Consequences of the pandemic on academic productivity may include fewer women represented in higher academic ranks, lower compensation, and lower career satisfaction compared with men.¹ We urge academic institutions and funding agencies to recognize and take action to mitigate long-term sequelae. Extended grant end dates and submission periods, funding opportunities dedicated to women, and prioritization of female-authored submissions are some strategies that can safeguard equitable career progression in dermatology research.

To the Editor:

Peer-reviewed publications are important determinants for promotions, academic leadership, and grants in dermatology.¹ The impact of the COVID-19 pandemic on dermatology research productivity remains an area of investigation. We sought to determine authorship trends for males and females during the pandemic.

A cross-sectional retrospective study of the top 20 dermatology journals—determined by impact factor and Google Scholar H5-index—was conducted to identify manuscripts with submission date specified prepandemic (May 1, 2019–October 31, 2019) and during the pandemic (May 1, 2020–October 31, 2020). Submission date, first/last author name, sex, and affiliated country were extracted. Single authors were designated as first authors. Gender API (https://gender-api.com/en/) classified gender. A χ² test (P<.05) compared differences in proportions of female first/last authors from 2019 to 2020.

Overall, 811 and 1061 articles submitted in 2019 and 2020, respectively, were included. There were 1517 articles submitted to clinical journals and 355 articles submitted to basic science journals (Table). For the 7 clinical journals included, there was a 7.7% decrease in the proportion of female last authors in 2020 vs 2019 (P=.002), with the largest decrease between August and September 2020. Although other comparisons did not yield statistically significant differences (P>.05 all)(Table), several trends were observed. For clinical journals, there was a 1.8% decrease in the proportion of female first authors. For the 4 basic science journals included, there was a 4.9% increase and a 0.3% decrease in percentages of female first and last authors, respectively, for 2020 vs 2019.

Our findings indicate that the COVID-19 pandemic may have impacted female authors’ productivity in clinical dermatology publications. In a survey-based study for 2010 to 2011, female physician-researchers (n=437) spent 8.5 more hours per week on domestic activities and childcare and were more likely to take time off for childcare if their partner worked full time compared with males (n=612)(42.6% vs 12.4%, respectively).² Our observation that female last authors had a significant decrease in publications may suggest that this population had a disproportionate burden of domestic labor and childcare during the pandemic. It is possible that last authors, who generally are more senior researchers, may be more likely to have childcare, eldercare, and other types of domestic responsibilities. Similarly, in a study of surgery submissions (n=1068), there were 6%, 7%, and 4% decreases in percentages of female last, corresponding, and first authors, respectively, from 2019 to 2020.³Our study had limitations. Only 11 journals were analyzed because others did not have specified submission dates. Some journals only provided submission information for a subset of articles (eg, those published in the In Press section), which may have accounted for the large discrepancy in submission numbers for 2019 to 2020. Gender could not be determined for 1% of authors and was limited to female and male. Although our study submission time frame (May–October 2020) aimed at identifying research conducted during the height of the COVID-19 pandemic, some of these studies may have been conducted months or years before the pandemic. Future studies should focus on longer and more comprehensive time frames. Finally, estimated dates of stay-at-home orders fail to consider differences within countries.

The proportion of female US-affiliated first and last authors publishing in dermatology journals increased from 12% to 48% in 1976 and from 6% to 31% in 2006,⁴ which is encouraging. However, a gender gap persists, with one-third of National Institutes of Health grants in dermatology and one-fourth of research project grants in dermatology awarded to women.⁵ Consequences of the pandemic on academic productivity may include fewer women represented in higher academic ranks, lower compensation, and lower career satisfaction compared with men.¹ We urge academic institutions and funding agencies to recognize and take action to mitigate long-term sequelae. Extended grant end dates and submission periods, funding opportunities dedicated to women, and prioritization of female-authored submissions are some strategies that can safeguard equitable career progression in dermatology research.

To the Editor:

Peer-reviewed publications are important determinants for promotions, academic leadership, and grants in dermatology.¹ The impact of the COVID-19 pandemic on dermatology research productivity remains an area of investigation. We sought to determine authorship trends for males and females during the pandemic.

A cross-sectional retrospective study of the top 20 dermatology journals—determined by impact factor and Google Scholar H5-index—was conducted to identify manuscripts with submission date specified prepandemic (May 1, 2019–October 31, 2019) and during the pandemic (May 1, 2020–October 31, 2020). Submission date, first/last author name, sex, and affiliated country were extracted. Single authors were designated as first authors. Gender API (https://gender-api.com/en/) classified gender. A χ² test (P<.05) compared differences in proportions of female first/last authors from 2019 to 2020.

Overall, 811 and 1061 articles submitted in 2019 and 2020, respectively, were included. There were 1517 articles submitted to clinical journals and 355 articles submitted to basic science journals (Table). For the 7 clinical journals included, there was a 7.7% decrease in the proportion of female last authors in 2020 vs 2019 (P=.002), with the largest decrease between August and September 2020. Although other comparisons did not yield statistically significant differences (P>.05 all)(Table), several trends were observed. For clinical journals, there was a 1.8% decrease in the proportion of female first authors. For the 4 basic science journals included, there was a 4.9% increase and a 0.3% decrease in percentages of female first and last authors, respectively, for 2020 vs 2019.

Our findings indicate that the COVID-19 pandemic may have impacted female authors’ productivity in clinical dermatology publications. In a survey-based study for 2010 to 2011, female physician-researchers (n=437) spent 8.5 more hours per week on domestic activities and childcare and were more likely to take time off for childcare if their partner worked full time compared with males (n=612)(42.6% vs 12.4%, respectively).² Our observation that female last authors had a significant decrease in publications may suggest that this population had a disproportionate burden of domestic labor and childcare during the pandemic. It is possible that last authors, who generally are more senior researchers, may be more likely to have childcare, eldercare, and other types of domestic responsibilities. Similarly, in a study of surgery submissions (n=1068), there were 6%, 7%, and 4% decreases in percentages of female last, corresponding, and first authors, respectively, from 2019 to 2020.³Our study had limitations. Only 11 journals were analyzed because others did not have specified submission dates. Some journals only provided submission information for a subset of articles (eg, those published in the In Press section), which may have accounted for the large discrepancy in submission numbers for 2019 to 2020. Gender could not be determined for 1% of authors and was limited to female and male. Although our study submission time frame (May–October 2020) aimed at identifying research conducted during the height of the COVID-19 pandemic, some of these studies may have been conducted months or years before the pandemic. Future studies should focus on longer and more comprehensive time frames. Finally, estimated dates of stay-at-home orders fail to consider differences within countries.

The proportion of female US-affiliated first and last authors publishing in dermatology journals increased from 12% to 48% in 1976 and from 6% to 31% in 2006,⁴ which is encouraging. However, a gender gap persists, with one-third of National Institutes of Health grants in dermatology and one-fourth of research project grants in dermatology awarded to women.⁵ Consequences of the pandemic on academic productivity may include fewer women represented in higher academic ranks, lower compensation, and lower career satisfaction compared with men.¹ We urge academic institutions and funding agencies to recognize and take action to mitigate long-term sequelae. Extended grant end dates and submission periods, funding opportunities dedicated to women, and prioritization of female-authored submissions are some strategies that can safeguard equitable career progression in dermatology research.

Frailty is an age-associated, nonspecific vulnerability to adverse health outcomes. Frailty can also be described as a complex of symptoms characterized by impaired stress tolerance due to a decline in the functionality of different organs.¹ The prevalence of frailty varies widely depending on the method of measurement and the population studied.^2-4 It is a nonconstant factor that increases with age. A deficit accumulation frailty index (FI) is one method used to measure frailty.⁵ This approach sees frailty as a multidimensional risk state measured by quantity rather than the nature of health concerns. A deficit accumulation FI does not require physical testing but correlates well with other phenotypic FIs.⁶ It is, however, time consuming, as ≥ 30 deficits need to be measured to offer greater stability to the frailty estimate.

Health care is seeing increasing utilization of big data analytics to derive predictive models and help with resource allocation. There are currently 2 existing automated tools to predict health care utilization and mortality at the US Department of Veterans Affairs (VA): the VA Frailty Index (VA-FI-10) and the Care Assessment Need (CAN). VA-FI-10 is an International Statistical Classification of Diseases, Tenth Revision (ICD-10) update of the VA-FI that was created in March 2021. The VA-FI-10 is a claims-based frailty assessment tool using 31 health deficits. Calculating the VA-FI-10 requires defining an index date and lookback period (typically 3 years) relative to which it will be calculated.⁷

CAN is a set of risk-stratifying statistical models run on veterans receiving VA primary care services as part of a patient aligned care team (PACT) using electronic health record data.⁸ Each veteran is stratified based on the individual’s risks of hospitalization, death, and hospitalization or death. These 3 events are predicted for 90-day and 1-year time periods for a total of 6 distinct outcomes. CAN is currently on its third iteration (CAN 2.5) and scores range from 0 (low) to 99 (high). CAN scores are updated weekly. The 1-year hospitalization probabilities for all patients range from 0.8% to 93.1%. For patients with a CAN score of 50, the probability of being hospitalized within a year ranges from 4.5% to 5.2%, which increases to 32.2% to 36% for veterans with a CAN score of 95. The probability range widens significantly (32.2%-93.1%) for patients in the top 5 CAN scores (95-99).

CAN scores are a potential screening tool for frailty among older adults; they are generated automatically and provide acceptable diagnostic accuracy. Hence, the CAN score may be a useful tool for primary care practitioners for the detection of frailty in their patients. The CAN score has shown a moderate positive association with the FRAIL Scale.^9,10 The population-based studies that have used the FI approach (differing FIs, depending on the data available) give robust results: People accumulate an average of 0.03 deficits per year after the age of 70 years.¹¹ Interventions to delay or reverse frailty have not been clearly defined with heterogeneity in the definition of frailty and measurement of frailty outcomes.^12,13 The prevalence of frailty in the veteran population is substantially higher than the prevalence in community populations with a similar age distribution. There is also mounting evidence that veterans accumulate deficits more rapidly than their civilian counterparts.¹⁴

COVID-19 was declared a pandemic in March 2020 and had many impacts on global health that were most marked in the first year. These included reductions in hospital visits for non-COVID-19 health concerns, a reduction in completed screening tests, an initial reduction in other infectious diseases (attributable to quarantines), and an increase or worsening of mental health concerns.^15,16

We aimed to investigate whether frailty increased disproportionately in a subset of older veterans in the first year of the COVID-19 pandemic when compared with the previous year using CAN scores. This single institution, longitudinal cohort study was determined to be exempt from institutional review board review but was approved by the Phoenix VA Health Care System (PVAHCS) Research and Development Committee.

 

Methods

The Office of Clinical Systems Development and Evaluation (CSDE–10E2A) produces a weekly CAN Score Report to help identify the highest-risk patients in a primary care panel or cohort. CAN scores range from 0 (lowest risk) to 99 (highest risk), indicating how likely a patient is to experience hospitalization or death compared with other VA patients. CAN scores are calculated with statistical prediction models that use data elements from the following Corporate Data Warehouse (CDW) domains: demographics, health care utilization, laboratory tests, medical conditions, medications, and vital signs (eAppendix available online at 10.12788/fp.0385).

The CAN Score Report is generated weekly and stored on a CDW server. A patient will receive all 6 distinct CAN scores if they are: (1) assigned to a primary care PACT on the risk date; (2) a veteran; (3) not hospitalized in a VA facility on the risk date; and (4) alive as of the risk date. New to CAN 2.5 is that patients who meet criteria 1, 2, and 4 but are hospitalized in a VA facility on the risk date will receive CAN scores for the 1-year and 90-day mortality models.

Utilizing VA Informatics and Computing Infrastructure (VA HSR RES 13-457, US Department of Veterans Affairs [2008]), we obtained 2 lists of veterans aged 70 to 75 years on February 8, 2019, with a calculated CAN score of ≥ 75 for 1-year mortality and 1-year hospitalization on that date. A veteran with a CAN score of ≥ 75 is likely to be prefrail or frail.^9,10 Veterans who did not have a corresponding calculated CAN score on February 7, 2020, and February 12, 2021, were excluded. COVID-19 was declared a public health emergency in the United States on January 31, 2020, and the World Health Organization declared COVID-19 a pandemic on March 11, 2020.¹⁷ We picked February 7, 2020, within this time frame and without any other special significance. We picked additional CAN score calculation dates approximately 1 year prior and 1 year after this date. Veterans had to be alive on February 12, 2021, (the last date of the CAN score) to be included in the cohorts.

Statistical Analyses

The difference in CAN score from one year to the next was calculated for each patient. The difference between 2019 and 2020 was compared with the difference between 2020 to 2021 using a paired t test. Yearly CAN score values were analyzed using repeated measures analysis of variance. The number of patients that showed an increase in CAN score (ie, increased risk of either mortality or hospitalization within the year) or a decrease (lower risk) was compared using the χ² test. IBM SPSS v26 and GraphPad Prism v18 were used for statistical analysis. P < .05 was considered statistically significant.

Results

There were 3538 veterans at PVAHCS who met the inclusion criteria and had a 1-year mortality CAN score ≥ 75 on February 8, 2019.

We excluded 6 veterans from the final analysis due to lack of 1-year mortality CAN score for 2020 or 2021. The final number included in the analysis was 3532 (Figure 1). The mean (SD) age was 71.8 (1.3) years. There were 3488 male (98.8%) and 44 female (1.2%) veterans represented (Table 1).

Our data show a decrease in mean 1-year mortality CAN score in this subset of older frail veterans by 4.9 (95% CI, -5.3 to -4.5) in the year preceding the COVID-19 pandemic (Table 2). The 1-year mean mortality CAN score increased significantly by 0.2 (95% CI, -0.3 to 0.6; P < .0001 vs pre-COVID) in this same subset of patients after the first year of the COVID-19 pandemic (Figure 2).

Mean CAN scores for 1-year mortality were 81.5 (95% CI, 81.2 to 81.7), 76.5 (95% CI, 76.1 to 77.0), and 76.7 (95% CI, 76.2 to 77.2) for 2019, 2020, and 2021, respectively.

 

In the hospitalization group, there were 6046 veterans in the analysis; 57 veterans missing a 1-year hospitalization CAN score that were excluded. The mean age was 71.7 (1.3) years and included 5874 male (97.2%) and 172 female (2.8%) veterans. There was a decline in mean 1-year hospitalization CAN scores in our subset of frail older veterans by 2.8 (95% CI, -3.1 to -2.6) in the year preceding the COVID-19 pandemic. This mean decline slowed significantly to 1.5 (95% CI, -1.8 to -1.2; P < .0001) after the first year of the COVID-19 pandemic. Mean CAN scores for 1-year hospitalization were 84.6 (95% CI, 84.4 to 84.8), 81.8 (95% CI, 81.5 to 82.1), and 80.2 (95% CI, 79.9 to 80.6) for 2019, 2020, and 2021, respectively.

We also calculated the number of veterans with increasing, stable, and decreasing CAN scores across each of our defined periods in both the 1-year mortality and hospitalization groups.

The subset of veterans with stable/no change in CAN scores was the smallest in both groups (Figure 3).

A previous study used a 1-year combined hospitalization or mortality event CAN score as the most all-inclusive measure of frailty but determined that it was possible that 1 of the other 5 CAN risk measures could perform better in predicting frailty.¹⁰ We collected and presented data for 1-year mortality and hospitalization CAN scores. There were declines in pandemic-related US hospitalizations for illnesses not related to COVID-19 during the first few months of the pandemic.¹⁸ This may or may not have affected the 1-year hospitalization CAN score data; thus, we used the 1-year mortality CAN score data to predict frailty.

Discussion

We studied frailty trends in an older veteran subpopulation enrolled at the PVAHCS 1 year prior and into the COVID-19 pandemic using CAN scores. Frailty is a dynamic state. Previous frailty assessments aimed to identify patients at the highest risk of death. With the advent of advanced therapeutics for several diseases, the number of medical conditions that are now managed as chronic illnesses continues to grow. There is a role for repeated measures of frailty to try to identify frailty trends.¹⁹ These trends may assist us in resource allocation, identifying interventions that work and those that do not.

Some studies have shown an overall declining lethality of frailty. This may reflect improvements in the care and management of chronic conditions, screening tests, and increased awareness of healthy lifestyles.²⁰ Another study of frailty trajectories in a veteran population in the 5 years preceding death showed multiple trajectories (stable, gradually increasing, rapidly increasing, and recovering).¹⁹

The PACT is a primary care model implemented at VA medical centers in April 2010. It is a patient-centered medical home model (PCMH) with several components. The VA treats a population of socioeconomically vulnerable patients with complex chronic illness management needs. Some of the components of a PACT model relevant to our study include facilitated self-management support for veterans in between practitioner visits via care partners, peer-to-peer and transitional care programs, physical activity and diet programs, primary care mental health, integration between primary and specialty care, and telehealth.²¹ A previous study has shown that VA primary care clinics with the most PCMH components in place had greater improvements in several chronic disease quality measures than in clinics with a lower number of PCMH components.²²

 

Limitations

Our study is limited by our older veteran population demographics. We chose only a subset of older veterans at a single VA center for this study and cannot extrapolate the results to all older frail veterans or community dwelling older adults. Robust individuals may also transition to prefrailty and frailty over longer periods; our study monitored frailty trends over 2 years.

CAN scores are not quality measures to improve upon. Allocation and utilization of additional resources may clinically benefit a patient but increase their CAN scores. Although our results are statistically significant, we are unable to make any conclusions about clinical significance.

Conclusions

Our study results indicate frailty as determined by 1-year mortality CAN scores significantly increased in a subset of older veterans during the first year of the COVID-19 pandemic when compared with the previous year. Whether this change in frailty is temporary or long lasting remains to be seen. Automated CAN scores can be effectively utilized to monitor frailty trends in certain veteran populations over longer periods.

Acknowledgments

This material is the result of work supported with resources and the use of facilities at the Phoenix Veterans Affairs Health Care System.

Frailty is an age-associated, nonspecific vulnerability to adverse health outcomes. Frailty can also be described as a complex of symptoms characterized by impaired stress tolerance due to a decline in the functionality of different organs.¹ The prevalence of frailty varies widely depending on the method of measurement and the population studied.^2-4 It is a nonconstant factor that increases with age. A deficit accumulation frailty index (FI) is one method used to measure frailty.⁵ This approach sees frailty as a multidimensional risk state measured by quantity rather than the nature of health concerns. A deficit accumulation FI does not require physical testing but correlates well with other phenotypic FIs.⁶ It is, however, time consuming, as ≥ 30 deficits need to be measured to offer greater stability to the frailty estimate.

Health care is seeing increasing utilization of big data analytics to derive predictive models and help with resource allocation. There are currently 2 existing automated tools to predict health care utilization and mortality at the US Department of Veterans Affairs (VA): the VA Frailty Index (VA-FI-10) and the Care Assessment Need (CAN). VA-FI-10 is an International Statistical Classification of Diseases, Tenth Revision (ICD-10) update of the VA-FI that was created in March 2021. The VA-FI-10 is a claims-based frailty assessment tool using 31 health deficits. Calculating the VA-FI-10 requires defining an index date and lookback period (typically 3 years) relative to which it will be calculated.⁷

CAN is a set of risk-stratifying statistical models run on veterans receiving VA primary care services as part of a patient aligned care team (PACT) using electronic health record data.⁸ Each veteran is stratified based on the individual’s risks of hospitalization, death, and hospitalization or death. These 3 events are predicted for 90-day and 1-year time periods for a total of 6 distinct outcomes. CAN is currently on its third iteration (CAN 2.5) and scores range from 0 (low) to 99 (high). CAN scores are updated weekly. The 1-year hospitalization probabilities for all patients range from 0.8% to 93.1%. For patients with a CAN score of 50, the probability of being hospitalized within a year ranges from 4.5% to 5.2%, which increases to 32.2% to 36% for veterans with a CAN score of 95. The probability range widens significantly (32.2%-93.1%) for patients in the top 5 CAN scores (95-99).

CAN scores are a potential screening tool for frailty among older adults; they are generated automatically and provide acceptable diagnostic accuracy. Hence, the CAN score may be a useful tool for primary care practitioners for the detection of frailty in their patients. The CAN score has shown a moderate positive association with the FRAIL Scale.^9,10 The population-based studies that have used the FI approach (differing FIs, depending on the data available) give robust results: People accumulate an average of 0.03 deficits per year after the age of 70 years.¹¹ Interventions to delay or reverse frailty have not been clearly defined with heterogeneity in the definition of frailty and measurement of frailty outcomes.^12,13 The prevalence of frailty in the veteran population is substantially higher than the prevalence in community populations with a similar age distribution. There is also mounting evidence that veterans accumulate deficits more rapidly than their civilian counterparts.¹⁴

COVID-19 was declared a pandemic in March 2020 and had many impacts on global health that were most marked in the first year. These included reductions in hospital visits for non-COVID-19 health concerns, a reduction in completed screening tests, an initial reduction in other infectious diseases (attributable to quarantines), and an increase or worsening of mental health concerns.^15,16

We aimed to investigate whether frailty increased disproportionately in a subset of older veterans in the first year of the COVID-19 pandemic when compared with the previous year using CAN scores. This single institution, longitudinal cohort study was determined to be exempt from institutional review board review but was approved by the Phoenix VA Health Care System (PVAHCS) Research and Development Committee.

 

Methods

The Office of Clinical Systems Development and Evaluation (CSDE–10E2A) produces a weekly CAN Score Report to help identify the highest-risk patients in a primary care panel or cohort. CAN scores range from 0 (lowest risk) to 99 (highest risk), indicating how likely a patient is to experience hospitalization or death compared with other VA patients. CAN scores are calculated with statistical prediction models that use data elements from the following Corporate Data Warehouse (CDW) domains: demographics, health care utilization, laboratory tests, medical conditions, medications, and vital signs (eAppendix available online at 10.12788/fp.0385).

The CAN Score Report is generated weekly and stored on a CDW server. A patient will receive all 6 distinct CAN scores if they are: (1) assigned to a primary care PACT on the risk date; (2) a veteran; (3) not hospitalized in a VA facility on the risk date; and (4) alive as of the risk date. New to CAN 2.5 is that patients who meet criteria 1, 2, and 4 but are hospitalized in a VA facility on the risk date will receive CAN scores for the 1-year and 90-day mortality models.

Utilizing VA Informatics and Computing Infrastructure (VA HSR RES 13-457, US Department of Veterans Affairs [2008]), we obtained 2 lists of veterans aged 70 to 75 years on February 8, 2019, with a calculated CAN score of ≥ 75 for 1-year mortality and 1-year hospitalization on that date. A veteran with a CAN score of ≥ 75 is likely to be prefrail or frail.^9,10 Veterans who did not have a corresponding calculated CAN score on February 7, 2020, and February 12, 2021, were excluded. COVID-19 was declared a public health emergency in the United States on January 31, 2020, and the World Health Organization declared COVID-19 a pandemic on March 11, 2020.¹⁷ We picked February 7, 2020, within this time frame and without any other special significance. We picked additional CAN score calculation dates approximately 1 year prior and 1 year after this date. Veterans had to be alive on February 12, 2021, (the last date of the CAN score) to be included in the cohorts.

Statistical Analyses

The difference in CAN score from one year to the next was calculated for each patient. The difference between 2019 and 2020 was compared with the difference between 2020 to 2021 using a paired t test. Yearly CAN score values were analyzed using repeated measures analysis of variance. The number of patients that showed an increase in CAN score (ie, increased risk of either mortality or hospitalization within the year) or a decrease (lower risk) was compared using the χ² test. IBM SPSS v26 and GraphPad Prism v18 were used for statistical analysis. P < .05 was considered statistically significant.

Results

There were 3538 veterans at PVAHCS who met the inclusion criteria and had a 1-year mortality CAN score ≥ 75 on February 8, 2019.

We excluded 6 veterans from the final analysis due to lack of 1-year mortality CAN score for 2020 or 2021. The final number included in the analysis was 3532 (Figure 1). The mean (SD) age was 71.8 (1.3) years. There were 3488 male (98.8%) and 44 female (1.2%) veterans represented (Table 1).

Our data show a decrease in mean 1-year mortality CAN score in this subset of older frail veterans by 4.9 (95% CI, -5.3 to -4.5) in the year preceding the COVID-19 pandemic (Table 2). The 1-year mean mortality CAN score increased significantly by 0.2 (95% CI, -0.3 to 0.6; P < .0001 vs pre-COVID) in this same subset of patients after the first year of the COVID-19 pandemic (Figure 2).

Mean CAN scores for 1-year mortality were 81.5 (95% CI, 81.2 to 81.7), 76.5 (95% CI, 76.1 to 77.0), and 76.7 (95% CI, 76.2 to 77.2) for 2019, 2020, and 2021, respectively.

 

In the hospitalization group, there were 6046 veterans in the analysis; 57 veterans missing a 1-year hospitalization CAN score that were excluded. The mean age was 71.7 (1.3) years and included 5874 male (97.2%) and 172 female (2.8%) veterans. There was a decline in mean 1-year hospitalization CAN scores in our subset of frail older veterans by 2.8 (95% CI, -3.1 to -2.6) in the year preceding the COVID-19 pandemic. This mean decline slowed significantly to 1.5 (95% CI, -1.8 to -1.2; P < .0001) after the first year of the COVID-19 pandemic. Mean CAN scores for 1-year hospitalization were 84.6 (95% CI, 84.4 to 84.8), 81.8 (95% CI, 81.5 to 82.1), and 80.2 (95% CI, 79.9 to 80.6) for 2019, 2020, and 2021, respectively.

We also calculated the number of veterans with increasing, stable, and decreasing CAN scores across each of our defined periods in both the 1-year mortality and hospitalization groups.

The subset of veterans with stable/no change in CAN scores was the smallest in both groups (Figure 3).

A previous study used a 1-year combined hospitalization or mortality event CAN score as the most all-inclusive measure of frailty but determined that it was possible that 1 of the other 5 CAN risk measures could perform better in predicting frailty.¹⁰ We collected and presented data for 1-year mortality and hospitalization CAN scores. There were declines in pandemic-related US hospitalizations for illnesses not related to COVID-19 during the first few months of the pandemic.¹⁸ This may or may not have affected the 1-year hospitalization CAN score data; thus, we used the 1-year mortality CAN score data to predict frailty.

Discussion

We studied frailty trends in an older veteran subpopulation enrolled at the PVAHCS 1 year prior and into the COVID-19 pandemic using CAN scores. Frailty is a dynamic state. Previous frailty assessments aimed to identify patients at the highest risk of death. With the advent of advanced therapeutics for several diseases, the number of medical conditions that are now managed as chronic illnesses continues to grow. There is a role for repeated measures of frailty to try to identify frailty trends.¹⁹ These trends may assist us in resource allocation, identifying interventions that work and those that do not.

Some studies have shown an overall declining lethality of frailty. This may reflect improvements in the care and management of chronic conditions, screening tests, and increased awareness of healthy lifestyles.²⁰ Another study of frailty trajectories in a veteran population in the 5 years preceding death showed multiple trajectories (stable, gradually increasing, rapidly increasing, and recovering).¹⁹

The PACT is a primary care model implemented at VA medical centers in April 2010. It is a patient-centered medical home model (PCMH) with several components. The VA treats a population of socioeconomically vulnerable patients with complex chronic illness management needs. Some of the components of a PACT model relevant to our study include facilitated self-management support for veterans in between practitioner visits via care partners, peer-to-peer and transitional care programs, physical activity and diet programs, primary care mental health, integration between primary and specialty care, and telehealth.²¹ A previous study has shown that VA primary care clinics with the most PCMH components in place had greater improvements in several chronic disease quality measures than in clinics with a lower number of PCMH components.²²

 

Limitations

Our study is limited by our older veteran population demographics. We chose only a subset of older veterans at a single VA center for this study and cannot extrapolate the results to all older frail veterans or community dwelling older adults. Robust individuals may also transition to prefrailty and frailty over longer periods; our study monitored frailty trends over 2 years.

CAN scores are not quality measures to improve upon. Allocation and utilization of additional resources may clinically benefit a patient but increase their CAN scores. Although our results are statistically significant, we are unable to make any conclusions about clinical significance.

Conclusions

Our study results indicate frailty as determined by 1-year mortality CAN scores significantly increased in a subset of older veterans during the first year of the COVID-19 pandemic when compared with the previous year. Whether this change in frailty is temporary or long lasting remains to be seen. Automated CAN scores can be effectively utilized to monitor frailty trends in certain veteran populations over longer periods.

Acknowledgments

This material is the result of work supported with resources and the use of facilities at the Phoenix Veterans Affairs Health Care System.

Frailty is an age-associated, nonspecific vulnerability to adverse health outcomes. Frailty can also be described as a complex of symptoms characterized by impaired stress tolerance due to a decline in the functionality of different organs.¹ The prevalence of frailty varies widely depending on the method of measurement and the population studied.^2-4 It is a nonconstant factor that increases with age. A deficit accumulation frailty index (FI) is one method used to measure frailty.⁵ This approach sees frailty as a multidimensional risk state measured by quantity rather than the nature of health concerns. A deficit accumulation FI does not require physical testing but correlates well with other phenotypic FIs.⁶ It is, however, time consuming, as ≥ 30 deficits need to be measured to offer greater stability to the frailty estimate.

Health care is seeing increasing utilization of big data analytics to derive predictive models and help with resource allocation. There are currently 2 existing automated tools to predict health care utilization and mortality at the US Department of Veterans Affairs (VA): the VA Frailty Index (VA-FI-10) and the Care Assessment Need (CAN). VA-FI-10 is an International Statistical Classification of Diseases, Tenth Revision (ICD-10) update of the VA-FI that was created in March 2021. The VA-FI-10 is a claims-based frailty assessment tool using 31 health deficits. Calculating the VA-FI-10 requires defining an index date and lookback period (typically 3 years) relative to which it will be calculated.⁷

CAN is a set of risk-stratifying statistical models run on veterans receiving VA primary care services as part of a patient aligned care team (PACT) using electronic health record data.⁸ Each veteran is stratified based on the individual’s risks of hospitalization, death, and hospitalization or death. These 3 events are predicted for 90-day and 1-year time periods for a total of 6 distinct outcomes. CAN is currently on its third iteration (CAN 2.5) and scores range from 0 (low) to 99 (high). CAN scores are updated weekly. The 1-year hospitalization probabilities for all patients range from 0.8% to 93.1%. For patients with a CAN score of 50, the probability of being hospitalized within a year ranges from 4.5% to 5.2%, which increases to 32.2% to 36% for veterans with a CAN score of 95. The probability range widens significantly (32.2%-93.1%) for patients in the top 5 CAN scores (95-99).

CAN scores are a potential screening tool for frailty among older adults; they are generated automatically and provide acceptable diagnostic accuracy. Hence, the CAN score may be a useful tool for primary care practitioners for the detection of frailty in their patients. The CAN score has shown a moderate positive association with the FRAIL Scale.^9,10 The population-based studies that have used the FI approach (differing FIs, depending on the data available) give robust results: People accumulate an average of 0.03 deficits per year after the age of 70 years.¹¹ Interventions to delay or reverse frailty have not been clearly defined with heterogeneity in the definition of frailty and measurement of frailty outcomes.^12,13 The prevalence of frailty in the veteran population is substantially higher than the prevalence in community populations with a similar age distribution. There is also mounting evidence that veterans accumulate deficits more rapidly than their civilian counterparts.¹⁴

COVID-19 was declared a pandemic in March 2020 and had many impacts on global health that were most marked in the first year. These included reductions in hospital visits for non-COVID-19 health concerns, a reduction in completed screening tests, an initial reduction in other infectious diseases (attributable to quarantines), and an increase or worsening of mental health concerns.^15,16

We aimed to investigate whether frailty increased disproportionately in a subset of older veterans in the first year of the COVID-19 pandemic when compared with the previous year using CAN scores. This single institution, longitudinal cohort study was determined to be exempt from institutional review board review but was approved by the Phoenix VA Health Care System (PVAHCS) Research and Development Committee.

 

Methods

The Office of Clinical Systems Development and Evaluation (CSDE–10E2A) produces a weekly CAN Score Report to help identify the highest-risk patients in a primary care panel or cohort. CAN scores range from 0 (lowest risk) to 99 (highest risk), indicating how likely a patient is to experience hospitalization or death compared with other VA patients. CAN scores are calculated with statistical prediction models that use data elements from the following Corporate Data Warehouse (CDW) domains: demographics, health care utilization, laboratory tests, medical conditions, medications, and vital signs (eAppendix available online at 10.12788/fp.0385).

The CAN Score Report is generated weekly and stored on a CDW server. A patient will receive all 6 distinct CAN scores if they are: (1) assigned to a primary care PACT on the risk date; (2) a veteran; (3) not hospitalized in a VA facility on the risk date; and (4) alive as of the risk date. New to CAN 2.5 is that patients who meet criteria 1, 2, and 4 but are hospitalized in a VA facility on the risk date will receive CAN scores for the 1-year and 90-day mortality models.

Utilizing VA Informatics and Computing Infrastructure (VA HSR RES 13-457, US Department of Veterans Affairs [2008]), we obtained 2 lists of veterans aged 70 to 75 years on February 8, 2019, with a calculated CAN score of ≥ 75 for 1-year mortality and 1-year hospitalization on that date. A veteran with a CAN score of ≥ 75 is likely to be prefrail or frail.^9,10 Veterans who did not have a corresponding calculated CAN score on February 7, 2020, and February 12, 2021, were excluded. COVID-19 was declared a public health emergency in the United States on January 31, 2020, and the World Health Organization declared COVID-19 a pandemic on March 11, 2020.¹⁷ We picked February 7, 2020, within this time frame and without any other special significance. We picked additional CAN score calculation dates approximately 1 year prior and 1 year after this date. Veterans had to be alive on February 12, 2021, (the last date of the CAN score) to be included in the cohorts.

Statistical Analyses

The difference in CAN score from one year to the next was calculated for each patient. The difference between 2019 and 2020 was compared with the difference between 2020 to 2021 using a paired t test. Yearly CAN score values were analyzed using repeated measures analysis of variance. The number of patients that showed an increase in CAN score (ie, increased risk of either mortality or hospitalization within the year) or a decrease (lower risk) was compared using the χ² test. IBM SPSS v26 and GraphPad Prism v18 were used for statistical analysis. P < .05 was considered statistically significant.

Results

There were 3538 veterans at PVAHCS who met the inclusion criteria and had a 1-year mortality CAN score ≥ 75 on February 8, 2019.

We excluded 6 veterans from the final analysis due to lack of 1-year mortality CAN score for 2020 or 2021. The final number included in the analysis was 3532 (Figure 1). The mean (SD) age was 71.8 (1.3) years. There were 3488 male (98.8%) and 44 female (1.2%) veterans represented (Table 1).

Our data show a decrease in mean 1-year mortality CAN score in this subset of older frail veterans by 4.9 (95% CI, -5.3 to -4.5) in the year preceding the COVID-19 pandemic (Table 2). The 1-year mean mortality CAN score increased significantly by 0.2 (95% CI, -0.3 to 0.6; P < .0001 vs pre-COVID) in this same subset of patients after the first year of the COVID-19 pandemic (Figure 2).

Mean CAN scores for 1-year mortality were 81.5 (95% CI, 81.2 to 81.7), 76.5 (95% CI, 76.1 to 77.0), and 76.7 (95% CI, 76.2 to 77.2) for 2019, 2020, and 2021, respectively.

 

In the hospitalization group, there were 6046 veterans in the analysis; 57 veterans missing a 1-year hospitalization CAN score that were excluded. The mean age was 71.7 (1.3) years and included 5874 male (97.2%) and 172 female (2.8%) veterans. There was a decline in mean 1-year hospitalization CAN scores in our subset of frail older veterans by 2.8 (95% CI, -3.1 to -2.6) in the year preceding the COVID-19 pandemic. This mean decline slowed significantly to 1.5 (95% CI, -1.8 to -1.2; P < .0001) after the first year of the COVID-19 pandemic. Mean CAN scores for 1-year hospitalization were 84.6 (95% CI, 84.4 to 84.8), 81.8 (95% CI, 81.5 to 82.1), and 80.2 (95% CI, 79.9 to 80.6) for 2019, 2020, and 2021, respectively.

We also calculated the number of veterans with increasing, stable, and decreasing CAN scores across each of our defined periods in both the 1-year mortality and hospitalization groups.

The subset of veterans with stable/no change in CAN scores was the smallest in both groups (Figure 3).

A previous study used a 1-year combined hospitalization or mortality event CAN score as the most all-inclusive measure of frailty but determined that it was possible that 1 of the other 5 CAN risk measures could perform better in predicting frailty.¹⁰ We collected and presented data for 1-year mortality and hospitalization CAN scores. There were declines in pandemic-related US hospitalizations for illnesses not related to COVID-19 during the first few months of the pandemic.¹⁸ This may or may not have affected the 1-year hospitalization CAN score data; thus, we used the 1-year mortality CAN score data to predict frailty.

Discussion

We studied frailty trends in an older veteran subpopulation enrolled at the PVAHCS 1 year prior and into the COVID-19 pandemic using CAN scores. Frailty is a dynamic state. Previous frailty assessments aimed to identify patients at the highest risk of death. With the advent of advanced therapeutics for several diseases, the number of medical conditions that are now managed as chronic illnesses continues to grow. There is a role for repeated measures of frailty to try to identify frailty trends.¹⁹ These trends may assist us in resource allocation, identifying interventions that work and those that do not.

Some studies have shown an overall declining lethality of frailty. This may reflect improvements in the care and management of chronic conditions, screening tests, and increased awareness of healthy lifestyles.²⁰ Another study of frailty trajectories in a veteran population in the 5 years preceding death showed multiple trajectories (stable, gradually increasing, rapidly increasing, and recovering).¹⁹

The PACT is a primary care model implemented at VA medical centers in April 2010. It is a patient-centered medical home model (PCMH) with several components. The VA treats a population of socioeconomically vulnerable patients with complex chronic illness management needs. Some of the components of a PACT model relevant to our study include facilitated self-management support for veterans in between practitioner visits via care partners, peer-to-peer and transitional care programs, physical activity and diet programs, primary care mental health, integration between primary and specialty care, and telehealth.²¹ A previous study has shown that VA primary care clinics with the most PCMH components in place had greater improvements in several chronic disease quality measures than in clinics with a lower number of PCMH components.²²

 

Limitations

Our study is limited by our older veteran population demographics. We chose only a subset of older veterans at a single VA center for this study and cannot extrapolate the results to all older frail veterans or community dwelling older adults. Robust individuals may also transition to prefrailty and frailty over longer periods; our study monitored frailty trends over 2 years.

CAN scores are not quality measures to improve upon. Allocation and utilization of additional resources may clinically benefit a patient but increase their CAN scores. Although our results are statistically significant, we are unable to make any conclusions about clinical significance.

Conclusions

Our study results indicate frailty as determined by 1-year mortality CAN scores significantly increased in a subset of older veterans during the first year of the COVID-19 pandemic when compared with the previous year. Whether this change in frailty is temporary or long lasting remains to be seen. Automated CAN scores can be effectively utilized to monitor frailty trends in certain veteran populations over longer periods.

Acknowledgments

This material is the result of work supported with resources and the use of facilities at the Phoenix Veterans Affairs Health Care System.

The US Coast Guard (USCG) operates within the US Department of Homeland Security during times of peace and represents a force of > 55,000 active-duty service members (ADSMs), civilians, and reservists. ADSMs account for about 40,000 USCG personnel. The missions of the USCG include activities such as maritime law enforcement (drug interdiction), search and rescue, and defense readiness.¹ Akin to other US Department of Defense (DoD) services, USCG ADSMs are required to maintain medical readiness to maximize operational success.

Whereas the DoD centralizes its health care services at military treatment facilities, USCG health care tends to be dispersed to smaller clinics and sickbays across large geographic areas. The USCG operates 42 clinics of varying sizes and medical capabilities, providing outpatient, dentistry, pharmacy, laboratory, radiology, physical therapy, optometry, and other health care services. Many ADSMs are evaluated by a USCG medical officer in these outpatient clinics, and ADSMs may choose to fill prescriptions at the in-house pharmacy if present at that clinic.

The USCG has 14 field pharmacists. In addition to the standard dispensing role at their respective clinics, USCG pharmacists provide regional oversight of pharmaceutical services for USCG units within their area of responsibility (AOR). Therefore, USCG pharmacists clinically, operationally, and logistically support these regional assets within their AOR while serving the traditional pharmacist role. USCG pharmacists have access to ADSM electronic health records (EHRs) when evaluating prescription orders, similar to other ambulatory care settings.

New recruits and accessions into the USCG are first screened for disqualifying health conditions, and ADSMs are required to maintain medical readiness throughout their careers.² Therefore, this population tends to be younger and overall healthier compared with the general population. Equally important, medication errors or inappropriate prescribing in the ADSM group could negatively affect their duty status and mission readiness of the USCG in addition to exposing the ADSM to medication-related harms.

Duty status is an important and unique consideration in this population. ADSMs are expected to be deployable worldwide and physically and mentally capable of executing all duties associated with their position. Duty status implications and the perceived ability to stand watch are tied to an ADMS’s specialty, training, and unit role. Duty status is based on various frameworks like the USCG Medical Manual, Aeromedical Policy Letters, and other governing documents.³ Duty status determinations are initiated by privileged USCG medical practitioners and may be executed in consultation with relevant commands and other subject matter experts. An inappropriately dosed antibiotic prescription, for example, can extend the duration that an ADSM would be considered unfit for full duty due to prolonged illness. Accordingly, being on a limited duty status may negatively affect USCG total mission readiness as a whole. USCG pharmacists play a vital role in optimizing ADSMs’ medication therapies to ensure safety and efficacy.

Currently no published literature explores the number of medication interventions or the impact of those interventions made by USCG pharmacists. This study aimed to quantify the number, duty status impact, and replicability of medication interventions made by one pharmacist at the USCG Base Alameda clinic over 6 months.

 

Methods

As part of a USCG quality improvement study, a pharmacist tracked all medication interventions on a spreadsheet at USCG Base Alameda clinic from July 1, 2021, to December 31, 2021. The study defined a medication intervention as a communication with the prescriber with the intention to change the medication, strength, dose, dosage form, quantity, or instructions. Each intervention was subcategorized as either a drug therapy problem (DTP) or a non-DTP intervention. Interventions were divided into 7 categories.

Each DTP intervention was evaluated in a retrospective chart review by a panel of USCG pharmacists to assess for duty status severity and replicability. For duty status severity, the panel reviewed the intervention after considering patient-specific factors and determined whether the original prescribing (had there not been an intervention) could have reasonably resulted in a change of duty status for the ADSM from a fit for full duty (FFFD) status to a different duty status (eg, fit for limited duty [FFLD]). This duty status review factored in potential impacts across multiple positions and billets, including aviators (pilots) and divers. In addition, the panel, whose members all have prior community pharmacy experience, assessed replicability by determining whether the same intervention could have reasonably been made in the absence of access to the patient EHR, as would be common in a community pharmacy setting.

Interventions without an identified DTP were considered non-DTP interventions. These interventions involved recommendations for a more cost-effective medication or a similar in stock therapeutic option to minimize delay of patient care. The spreadsheet also included the date, medication name, medication class, specific intervention made, outcome, and other descriptive comments.

Results

During the 6-month period, 1751 prescriptions were dispensed at USCG Base Alameda pharmacy with 116 interventions (7%).

Most interventions (n = 111, 96%) were accepted by the prescriber. Of the 116 interventions, 64 (55%) were DTP interventions; 21 of the DTP interventions (33%) were indication, 20 effectiveness (31%), 19 safety (30%), and 4 adherence (6%) (Table 1).

Among the DTP interventions, 26 (41%) dealt with an inappropriate dose, 13 (20%) were for medication omission, 7 (11%) for inappropriate dosage form, and 6 (9%) for excess medication (Table 2).

Fourteen interventions (22%) impacted duty status, and 18 (28%) were made because the pharmacist had EHR access. Among 51 non-DTP interventions, 34 (67%) minimized delay in patient care, and 17 (33%) cost-savings interventions were made, resulting in about $1700 in savings. Antibiotics had the most interventions (n = 28: 10 DTP and 18 non-DTP).

Discussion

This study is novel in examining the impact of a pharmacist’s medication interventions in a USCG ambulatory care practice setting. A PubMed literature search of the phrases “Coast Guard AND pharmacy” or “Coast Guard AND pharmacy AND intervention” yielded no results specific to pharmacy interventions in a USCG setting. However, the 2021 implementation of the enterprise-wide MHS GENESIS EHR may support additional tracking and analysis tools in the future.

Pharmacist interventions have been studied in diverse patient populations and practice settings, and most conclude that pharmacists make meaningful interventions at their respective organizations.^4-7 Many of these studies were conducted at open-door health care systems, whereas USCG clinics serve ADSMs nearly exclusively. The ADSM population tends to be younger and healthier due to age requirements and medical accession and retention standards.

It is important to recognize the value of a USCG pharmacist in identifying and rectifying potential medication errors, particularly those that may affect the ability to stand duty for ADSMs. An example intervention includes changing the daily starting dose of citalopram from the ordered 30 mg to the intended 10 mg. Inappropriately prescribed medication regimens may increase the incidence of adverse effects or prolong duration to therapeutic efficacy, which impairs the ability to stand duty. There were 3 circumstances where the prescriber had ordered the medication for an incorrect ADSM that were rectified by the pharmacist. If left unchanged, these errors could negatively affect the ADSM’s overall health, well-being, and duty status.

The acceptance rate for interventions in this study was 96%. The literature suggests a highly variable acceptance rate of pharmacist interventions when examined across various practice settings, health systems, and geographic locations.^8-10 This study’s comparatively high rate could be due to the pharmacist-prescriber relationships at USCG clinics. By virtue of colocatation and teamwork initiatives, the pharmacist has the opportunity to develop positive rapport with physicians, physician assistants, and other clinic staff.

Having access to EHRs allowed the pharmacist to make 18 of the DTP interventions. Chart access is not unique to the USCG and is common in other ambulatory care settings. Those 18 interventions, such as reconciling a prescription ordered as fluticasone/salmeterol but recorded in the EHR as “will prescribe montelukast,” were deemed possible because of EHR access. Such interventions could potentially be lost if ADSMs solely received their pharmaceutical care elsewhere.

USCG uses independent duty health services technicians (IDHSs) who practice in settings where a medical officer is not present, such as at smaller sickbays or aboard Coast Guard cutters. In this study, an IDHS had mistakenly created a medication order for the medical officer to sign for bupropion SR, when the ADSM had been taking and was intended to continue taking bupropion XL. This order was signed off by the medical officer, but this oversight was identified and corrected by the pharmacist before dispensing. This indicates that there is a vital educational role that the USCG pharmacist fulfills when working with health care team members within the AOR.

Equally important to consider are the non-DTP interventions. In a military setting, minimizations of delay in care are a high priority. There were 34 instances where the pharmacist made an intervention to recommend a similar therapeutic medication that was in stock to ensure that the ADSM had timely access to the medication without the need for prior authorization. In the context of short-notice, mission-critical deployments that may last for multiple months, recognizing medication shortages or other inventory constraints and recommending therapeutic alternatives ensures that the USCG can maintain a ready posture for missions in addition to providing timely and quality patient care.

Saving about $1700 over 6 months is also important. While this was not explicitly evaluated in the study, prescribers may not be acutely aware of medication pricing. There are often significant price differences between different formulations of the same medication (eg, naproxen delayed-release vs tablets). Because USCG pharmacists are responsible for ordering medications and managing their regional budget within the AOR, they are best poised to make cost-savings recommendations. These interventions suggest that USCG pharmacists must continue to remain actively involved in the patient care team alongside physicians, physician assistants, nurses, and corpsmen. Throughout this setting and in so many others, patients’ health outcomes improve when pharmacists are more engaged in the pharmacotherapy care plan.

 

Limitations

Currently, the USCG does not publish ADSM demographic or health-related data, making it difficult to evaluate these interventions in the context of age, gender, or type of disease. Accordingly, potential directions for future research include how USCG pharmacists’ interventions are stratified by duty station and initial diagnosis. Such studies may support future models where USCG pharmacists are providing targeted education to prescribers based on disease or medication classes.

This analysis may have limited applicability to other practice settings even within USCG. Most USCG clinics have a limited number of medical officers; indeed, many have only one, and clinics with pharmacies typically have 1 to 5 medical officers aboard. USCG medical officers have a multitude of other duties, which may impact prescribing patterns and pharmacist interventions. Statistical analyses were limited by the dearth of baseline data or comparative literature. Finally, the assessment of DTP interventions’ impact did not use an official measurement tool like the US Department of Veterans Affairs’ Safety Assessment Code matrix.¹¹ Instead, the study used the internal USCG pharmacist panel for the fitness for duty consideration as the main stratification of the DTP interventions’ duty status severity, because maintaining medical readiness is the top priority for a USCG clinic.

Conclusions

The multifaceted role of pharmacists in USCG clinics includes collaborating with the patient care team to make pharmacy interventions that have significant impacts on ADSMs’ wellness and the USCG mission. The ADSMs of this nation deserve quality medical care that translates into mission readiness, and the USCG pharmacy force stands ready to support that goal.

Acknowledgments

The authors acknowledge the contributions of CDR Christopher Janik, US Coast Guard Headquarters, and LCDR Darin Schneider, US Coast Guard D11 Regional Practice Manager, in the drafting of the manuscript.

The US Coast Guard (USCG) operates within the US Department of Homeland Security during times of peace and represents a force of > 55,000 active-duty service members (ADSMs), civilians, and reservists. ADSMs account for about 40,000 USCG personnel. The missions of the USCG include activities such as maritime law enforcement (drug interdiction), search and rescue, and defense readiness.¹ Akin to other US Department of Defense (DoD) services, USCG ADSMs are required to maintain medical readiness to maximize operational success.

Whereas the DoD centralizes its health care services at military treatment facilities, USCG health care tends to be dispersed to smaller clinics and sickbays across large geographic areas. The USCG operates 42 clinics of varying sizes and medical capabilities, providing outpatient, dentistry, pharmacy, laboratory, radiology, physical therapy, optometry, and other health care services. Many ADSMs are evaluated by a USCG medical officer in these outpatient clinics, and ADSMs may choose to fill prescriptions at the in-house pharmacy if present at that clinic.

The USCG has 14 field pharmacists. In addition to the standard dispensing role at their respective clinics, USCG pharmacists provide regional oversight of pharmaceutical services for USCG units within their area of responsibility (AOR). Therefore, USCG pharmacists clinically, operationally, and logistically support these regional assets within their AOR while serving the traditional pharmacist role. USCG pharmacists have access to ADSM electronic health records (EHRs) when evaluating prescription orders, similar to other ambulatory care settings.

New recruits and accessions into the USCG are first screened for disqualifying health conditions, and ADSMs are required to maintain medical readiness throughout their careers.² Therefore, this population tends to be younger and overall healthier compared with the general population. Equally important, medication errors or inappropriate prescribing in the ADSM group could negatively affect their duty status and mission readiness of the USCG in addition to exposing the ADSM to medication-related harms.

Duty status is an important and unique consideration in this population. ADSMs are expected to be deployable worldwide and physically and mentally capable of executing all duties associated with their position. Duty status implications and the perceived ability to stand watch are tied to an ADMS’s specialty, training, and unit role. Duty status is based on various frameworks like the USCG Medical Manual, Aeromedical Policy Letters, and other governing documents.³ Duty status determinations are initiated by privileged USCG medical practitioners and may be executed in consultation with relevant commands and other subject matter experts. An inappropriately dosed antibiotic prescription, for example, can extend the duration that an ADSM would be considered unfit for full duty due to prolonged illness. Accordingly, being on a limited duty status may negatively affect USCG total mission readiness as a whole. USCG pharmacists play a vital role in optimizing ADSMs’ medication therapies to ensure safety and efficacy.

Currently no published literature explores the number of medication interventions or the impact of those interventions made by USCG pharmacists. This study aimed to quantify the number, duty status impact, and replicability of medication interventions made by one pharmacist at the USCG Base Alameda clinic over 6 months.

 

Methods

As part of a USCG quality improvement study, a pharmacist tracked all medication interventions on a spreadsheet at USCG Base Alameda clinic from July 1, 2021, to December 31, 2021. The study defined a medication intervention as a communication with the prescriber with the intention to change the medication, strength, dose, dosage form, quantity, or instructions. Each intervention was subcategorized as either a drug therapy problem (DTP) or a non-DTP intervention. Interventions were divided into 7 categories.

Each DTP intervention was evaluated in a retrospective chart review by a panel of USCG pharmacists to assess for duty status severity and replicability. For duty status severity, the panel reviewed the intervention after considering patient-specific factors and determined whether the original prescribing (had there not been an intervention) could have reasonably resulted in a change of duty status for the ADSM from a fit for full duty (FFFD) status to a different duty status (eg, fit for limited duty [FFLD]). This duty status review factored in potential impacts across multiple positions and billets, including aviators (pilots) and divers. In addition, the panel, whose members all have prior community pharmacy experience, assessed replicability by determining whether the same intervention could have reasonably been made in the absence of access to the patient EHR, as would be common in a community pharmacy setting.

Interventions without an identified DTP were considered non-DTP interventions. These interventions involved recommendations for a more cost-effective medication or a similar in stock therapeutic option to minimize delay of patient care. The spreadsheet also included the date, medication name, medication class, specific intervention made, outcome, and other descriptive comments.

Results

During the 6-month period, 1751 prescriptions were dispensed at USCG Base Alameda pharmacy with 116 interventions (7%).

Most interventions (n = 111, 96%) were accepted by the prescriber. Of the 116 interventions, 64 (55%) were DTP interventions; 21 of the DTP interventions (33%) were indication, 20 effectiveness (31%), 19 safety (30%), and 4 adherence (6%) (Table 1).

Among the DTP interventions, 26 (41%) dealt with an inappropriate dose, 13 (20%) were for medication omission, 7 (11%) for inappropriate dosage form, and 6 (9%) for excess medication (Table 2).

Fourteen interventions (22%) impacted duty status, and 18 (28%) were made because the pharmacist had EHR access. Among 51 non-DTP interventions, 34 (67%) minimized delay in patient care, and 17 (33%) cost-savings interventions were made, resulting in about $1700 in savings. Antibiotics had the most interventions (n = 28: 10 DTP and 18 non-DTP).

Discussion

This study is novel in examining the impact of a pharmacist’s medication interventions in a USCG ambulatory care practice setting. A PubMed literature search of the phrases “Coast Guard AND pharmacy” or “Coast Guard AND pharmacy AND intervention” yielded no results specific to pharmacy interventions in a USCG setting. However, the 2021 implementation of the enterprise-wide MHS GENESIS EHR may support additional tracking and analysis tools in the future.

Pharmacist interventions have been studied in diverse patient populations and practice settings, and most conclude that pharmacists make meaningful interventions at their respective organizations.^4-7 Many of these studies were conducted at open-door health care systems, whereas USCG clinics serve ADSMs nearly exclusively. The ADSM population tends to be younger and healthier due to age requirements and medical accession and retention standards.

It is important to recognize the value of a USCG pharmacist in identifying and rectifying potential medication errors, particularly those that may affect the ability to stand duty for ADSMs. An example intervention includes changing the daily starting dose of citalopram from the ordered 30 mg to the intended 10 mg. Inappropriately prescribed medication regimens may increase the incidence of adverse effects or prolong duration to therapeutic efficacy, which impairs the ability to stand duty. There were 3 circumstances where the prescriber had ordered the medication for an incorrect ADSM that were rectified by the pharmacist. If left unchanged, these errors could negatively affect the ADSM’s overall health, well-being, and duty status.

The acceptance rate for interventions in this study was 96%. The literature suggests a highly variable acceptance rate of pharmacist interventions when examined across various practice settings, health systems, and geographic locations.^8-10 This study’s comparatively high rate could be due to the pharmacist-prescriber relationships at USCG clinics. By virtue of colocatation and teamwork initiatives, the pharmacist has the opportunity to develop positive rapport with physicians, physician assistants, and other clinic staff.

Having access to EHRs allowed the pharmacist to make 18 of the DTP interventions. Chart access is not unique to the USCG and is common in other ambulatory care settings. Those 18 interventions, such as reconciling a prescription ordered as fluticasone/salmeterol but recorded in the EHR as “will prescribe montelukast,” were deemed possible because of EHR access. Such interventions could potentially be lost if ADSMs solely received their pharmaceutical care elsewhere.

USCG uses independent duty health services technicians (IDHSs) who practice in settings where a medical officer is not present, such as at smaller sickbays or aboard Coast Guard cutters. In this study, an IDHS had mistakenly created a medication order for the medical officer to sign for bupropion SR, when the ADSM had been taking and was intended to continue taking bupropion XL. This order was signed off by the medical officer, but this oversight was identified and corrected by the pharmacist before dispensing. This indicates that there is a vital educational role that the USCG pharmacist fulfills when working with health care team members within the AOR.

Equally important to consider are the non-DTP interventions. In a military setting, minimizations of delay in care are a high priority. There were 34 instances where the pharmacist made an intervention to recommend a similar therapeutic medication that was in stock to ensure that the ADSM had timely access to the medication without the need for prior authorization. In the context of short-notice, mission-critical deployments that may last for multiple months, recognizing medication shortages or other inventory constraints and recommending therapeutic alternatives ensures that the USCG can maintain a ready posture for missions in addition to providing timely and quality patient care.

Saving about $1700 over 6 months is also important. While this was not explicitly evaluated in the study, prescribers may not be acutely aware of medication pricing. There are often significant price differences between different formulations of the same medication (eg, naproxen delayed-release vs tablets). Because USCG pharmacists are responsible for ordering medications and managing their regional budget within the AOR, they are best poised to make cost-savings recommendations. These interventions suggest that USCG pharmacists must continue to remain actively involved in the patient care team alongside physicians, physician assistants, nurses, and corpsmen. Throughout this setting and in so many others, patients’ health outcomes improve when pharmacists are more engaged in the pharmacotherapy care plan.

 

Limitations

Currently, the USCG does not publish ADSM demographic or health-related data, making it difficult to evaluate these interventions in the context of age, gender, or type of disease. Accordingly, potential directions for future research include how USCG pharmacists’ interventions are stratified by duty station and initial diagnosis. Such studies may support future models where USCG pharmacists are providing targeted education to prescribers based on disease or medication classes.

This analysis may have limited applicability to other practice settings even within USCG. Most USCG clinics have a limited number of medical officers; indeed, many have only one, and clinics with pharmacies typically have 1 to 5 medical officers aboard. USCG medical officers have a multitude of other duties, which may impact prescribing patterns and pharmacist interventions. Statistical analyses were limited by the dearth of baseline data or comparative literature. Finally, the assessment of DTP interventions’ impact did not use an official measurement tool like the US Department of Veterans Affairs’ Safety Assessment Code matrix.¹¹ Instead, the study used the internal USCG pharmacist panel for the fitness for duty consideration as the main stratification of the DTP interventions’ duty status severity, because maintaining medical readiness is the top priority for a USCG clinic.

Conclusions

The multifaceted role of pharmacists in USCG clinics includes collaborating with the patient care team to make pharmacy interventions that have significant impacts on ADSMs’ wellness and the USCG mission. The ADSMs of this nation deserve quality medical care that translates into mission readiness, and the USCG pharmacy force stands ready to support that goal.

Acknowledgments

The authors acknowledge the contributions of CDR Christopher Janik, US Coast Guard Headquarters, and LCDR Darin Schneider, US Coast Guard D11 Regional Practice Manager, in the drafting of the manuscript.

The US Coast Guard (USCG) operates within the US Department of Homeland Security during times of peace and represents a force of > 55,000 active-duty service members (ADSMs), civilians, and reservists. ADSMs account for about 40,000 USCG personnel. The missions of the USCG include activities such as maritime law enforcement (drug interdiction), search and rescue, and defense readiness.¹ Akin to other US Department of Defense (DoD) services, USCG ADSMs are required to maintain medical readiness to maximize operational success.

Whereas the DoD centralizes its health care services at military treatment facilities, USCG health care tends to be dispersed to smaller clinics and sickbays across large geographic areas. The USCG operates 42 clinics of varying sizes and medical capabilities, providing outpatient, dentistry, pharmacy, laboratory, radiology, physical therapy, optometry, and other health care services. Many ADSMs are evaluated by a USCG medical officer in these outpatient clinics, and ADSMs may choose to fill prescriptions at the in-house pharmacy if present at that clinic.

The USCG has 14 field pharmacists. In addition to the standard dispensing role at their respective clinics, USCG pharmacists provide regional oversight of pharmaceutical services for USCG units within their area of responsibility (AOR). Therefore, USCG pharmacists clinically, operationally, and logistically support these regional assets within their AOR while serving the traditional pharmacist role. USCG pharmacists have access to ADSM electronic health records (EHRs) when evaluating prescription orders, similar to other ambulatory care settings.

New recruits and accessions into the USCG are first screened for disqualifying health conditions, and ADSMs are required to maintain medical readiness throughout their careers.² Therefore, this population tends to be younger and overall healthier compared with the general population. Equally important, medication errors or inappropriate prescribing in the ADSM group could negatively affect their duty status and mission readiness of the USCG in addition to exposing the ADSM to medication-related harms.

Duty status is an important and unique consideration in this population. ADSMs are expected to be deployable worldwide and physically and mentally capable of executing all duties associated with their position. Duty status implications and the perceived ability to stand watch are tied to an ADMS’s specialty, training, and unit role. Duty status is based on various frameworks like the USCG Medical Manual, Aeromedical Policy Letters, and other governing documents.³ Duty status determinations are initiated by privileged USCG medical practitioners and may be executed in consultation with relevant commands and other subject matter experts. An inappropriately dosed antibiotic prescription, for example, can extend the duration that an ADSM would be considered unfit for full duty due to prolonged illness. Accordingly, being on a limited duty status may negatively affect USCG total mission readiness as a whole. USCG pharmacists play a vital role in optimizing ADSMs’ medication therapies to ensure safety and efficacy.

Currently no published literature explores the number of medication interventions or the impact of those interventions made by USCG pharmacists. This study aimed to quantify the number, duty status impact, and replicability of medication interventions made by one pharmacist at the USCG Base Alameda clinic over 6 months.

 

Methods

As part of a USCG quality improvement study, a pharmacist tracked all medication interventions on a spreadsheet at USCG Base Alameda clinic from July 1, 2021, to December 31, 2021. The study defined a medication intervention as a communication with the prescriber with the intention to change the medication, strength, dose, dosage form, quantity, or instructions. Each intervention was subcategorized as either a drug therapy problem (DTP) or a non-DTP intervention. Interventions were divided into 7 categories.

Each DTP intervention was evaluated in a retrospective chart review by a panel of USCG pharmacists to assess for duty status severity and replicability. For duty status severity, the panel reviewed the intervention after considering patient-specific factors and determined whether the original prescribing (had there not been an intervention) could have reasonably resulted in a change of duty status for the ADSM from a fit for full duty (FFFD) status to a different duty status (eg, fit for limited duty [FFLD]). This duty status review factored in potential impacts across multiple positions and billets, including aviators (pilots) and divers. In addition, the panel, whose members all have prior community pharmacy experience, assessed replicability by determining whether the same intervention could have reasonably been made in the absence of access to the patient EHR, as would be common in a community pharmacy setting.

Interventions without an identified DTP were considered non-DTP interventions. These interventions involved recommendations for a more cost-effective medication or a similar in stock therapeutic option to minimize delay of patient care. The spreadsheet also included the date, medication name, medication class, specific intervention made, outcome, and other descriptive comments.

Results

During the 6-month period, 1751 prescriptions were dispensed at USCG Base Alameda pharmacy with 116 interventions (7%).

Most interventions (n = 111, 96%) were accepted by the prescriber. Of the 116 interventions, 64 (55%) were DTP interventions; 21 of the DTP interventions (33%) were indication, 20 effectiveness (31%), 19 safety (30%), and 4 adherence (6%) (Table 1).

Among the DTP interventions, 26 (41%) dealt with an inappropriate dose, 13 (20%) were for medication omission, 7 (11%) for inappropriate dosage form, and 6 (9%) for excess medication (Table 2).

Fourteen interventions (22%) impacted duty status, and 18 (28%) were made because the pharmacist had EHR access. Among 51 non-DTP interventions, 34 (67%) minimized delay in patient care, and 17 (33%) cost-savings interventions were made, resulting in about $1700 in savings. Antibiotics had the most interventions (n = 28: 10 DTP and 18 non-DTP).

Discussion

This study is novel in examining the impact of a pharmacist’s medication interventions in a USCG ambulatory care practice setting. A PubMed literature search of the phrases “Coast Guard AND pharmacy” or “Coast Guard AND pharmacy AND intervention” yielded no results specific to pharmacy interventions in a USCG setting. However, the 2021 implementation of the enterprise-wide MHS GENESIS EHR may support additional tracking and analysis tools in the future.

Pharmacist interventions have been studied in diverse patient populations and practice settings, and most conclude that pharmacists make meaningful interventions at their respective organizations.^4-7 Many of these studies were conducted at open-door health care systems, whereas USCG clinics serve ADSMs nearly exclusively. The ADSM population tends to be younger and healthier due to age requirements and medical accession and retention standards.

It is important to recognize the value of a USCG pharmacist in identifying and rectifying potential medication errors, particularly those that may affect the ability to stand duty for ADSMs. An example intervention includes changing the daily starting dose of citalopram from the ordered 30 mg to the intended 10 mg. Inappropriately prescribed medication regimens may increase the incidence of adverse effects or prolong duration to therapeutic efficacy, which impairs the ability to stand duty. There were 3 circumstances where the prescriber had ordered the medication for an incorrect ADSM that were rectified by the pharmacist. If left unchanged, these errors could negatively affect the ADSM’s overall health, well-being, and duty status.

The acceptance rate for interventions in this study was 96%. The literature suggests a highly variable acceptance rate of pharmacist interventions when examined across various practice settings, health systems, and geographic locations.^8-10 This study’s comparatively high rate could be due to the pharmacist-prescriber relationships at USCG clinics. By virtue of colocatation and teamwork initiatives, the pharmacist has the opportunity to develop positive rapport with physicians, physician assistants, and other clinic staff.

Having access to EHRs allowed the pharmacist to make 18 of the DTP interventions. Chart access is not unique to the USCG and is common in other ambulatory care settings. Those 18 interventions, such as reconciling a prescription ordered as fluticasone/salmeterol but recorded in the EHR as “will prescribe montelukast,” were deemed possible because of EHR access. Such interventions could potentially be lost if ADSMs solely received their pharmaceutical care elsewhere.

USCG uses independent duty health services technicians (IDHSs) who practice in settings where a medical officer is not present, such as at smaller sickbays or aboard Coast Guard cutters. In this study, an IDHS had mistakenly created a medication order for the medical officer to sign for bupropion SR, when the ADSM had been taking and was intended to continue taking bupropion XL. This order was signed off by the medical officer, but this oversight was identified and corrected by the pharmacist before dispensing. This indicates that there is a vital educational role that the USCG pharmacist fulfills when working with health care team members within the AOR.

Equally important to consider are the non-DTP interventions. In a military setting, minimizations of delay in care are a high priority. There were 34 instances where the pharmacist made an intervention to recommend a similar therapeutic medication that was in stock to ensure that the ADSM had timely access to the medication without the need for prior authorization. In the context of short-notice, mission-critical deployments that may last for multiple months, recognizing medication shortages or other inventory constraints and recommending therapeutic alternatives ensures that the USCG can maintain a ready posture for missions in addition to providing timely and quality patient care.

Saving about $1700 over 6 months is also important. While this was not explicitly evaluated in the study, prescribers may not be acutely aware of medication pricing. There are often significant price differences between different formulations of the same medication (eg, naproxen delayed-release vs tablets). Because USCG pharmacists are responsible for ordering medications and managing their regional budget within the AOR, they are best poised to make cost-savings recommendations. These interventions suggest that USCG pharmacists must continue to remain actively involved in the patient care team alongside physicians, physician assistants, nurses, and corpsmen. Throughout this setting and in so many others, patients’ health outcomes improve when pharmacists are more engaged in the pharmacotherapy care plan.

 

Limitations

Currently, the USCG does not publish ADSM demographic or health-related data, making it difficult to evaluate these interventions in the context of age, gender, or type of disease. Accordingly, potential directions for future research include how USCG pharmacists’ interventions are stratified by duty station and initial diagnosis. Such studies may support future models where USCG pharmacists are providing targeted education to prescribers based on disease or medication classes.

This analysis may have limited applicability to other practice settings even within USCG. Most USCG clinics have a limited number of medical officers; indeed, many have only one, and clinics with pharmacies typically have 1 to 5 medical officers aboard. USCG medical officers have a multitude of other duties, which may impact prescribing patterns and pharmacist interventions. Statistical analyses were limited by the dearth of baseline data or comparative literature. Finally, the assessment of DTP interventions’ impact did not use an official measurement tool like the US Department of Veterans Affairs’ Safety Assessment Code matrix.¹¹ Instead, the study used the internal USCG pharmacist panel for the fitness for duty consideration as the main stratification of the DTP interventions’ duty status severity, because maintaining medical readiness is the top priority for a USCG clinic.

Conclusions

The multifaceted role of pharmacists in USCG clinics includes collaborating with the patient care team to make pharmacy interventions that have significant impacts on ADSMs’ wellness and the USCG mission. The ADSMs of this nation deserve quality medical care that translates into mission readiness, and the USCG pharmacy force stands ready to support that goal.

Acknowledgments

The authors acknowledge the contributions of CDR Christopher Janik, US Coast Guard Headquarters, and LCDR Darin Schneider, US Coast Guard D11 Regional Practice Manager, in the drafting of the manuscript.

To the Editor:

Dermatology is one of the most competitive residencies for matching, with a 57.5% match rate in 2022.¹ Our prior study of research-mentor relationships among matched dermatology applicants corroborated the importance of home programs (HPs) and program connections.² Therefore, our current objective was to compare profiles of matched dermatology applicants without HPs vs those with HPs.

We searched websites of 139 dermatology programs nationwide and found 1736 matched applicants from 2016 to 2020; of them, 323 did not have HPs. We determined program rank by research output using Doximity Residency Navigator (https://www.doximity.com/residency/). Advanced degrees (ADs) of applicants were identified using program websites and LinkedIn. A PubMed search was conducted for number of articles published by each applicant before September 15 of their match year. For applicants without HPs, we identified the senior author on each publication. The senior author publishing with an applicant most often was considered the research mentor. Two-tailed independent t tests and χ² tests were used to determine statistical significance (P<.05).

On average, matched applicants without HPs matched in lower-ranked (74.4) and smaller (12.4) programs compared with matched applicants with HPs (45.3 [P<.0001] and 15.1 [P<.0001], respectively)(eTable). The mean number of publications was similar between matched applicants with HPs and without HPs (5.64 and 4.80, respectively; P=.0525) as well as the percentage with ADs (14.7% and 11.5%, respectively; P=.0953). Overall, 14.8% of matched applicants without HPs matched at their mentors’ institutions.

Data were obtained for matched international applicants as a subset of non-HP applicants. Despite attending medical schools without associated HPs in the United States, international applicants matched at similarly ranked (44.3) and sized (15.0) programs, on average, compared with HP applicants. The mean number of publications was higher for international applicants (11.4) vs domestic applicants (5.33). International applicants more often had ADs (23.8%) and 60.1% of them held doctor of philosophy degrees. Overall, 40.5% of international applicants matched at their mentors’ institutions.

Our study suggests that matched dermatology applicants with and without HPs had similar achievements, on average, for the number of publications and percentage with ADs. However, non-HP applicants matched at lower-ranked programs than HP applicants. Therefore, applicants without HPs should strongly consider cultivating program connections, especially if they desire to match at higher-ranked dermatology programs. To illustrate, the rate of matching at research mentors’ institutions was approximately 3-times higher for international applicants than non-HP applicants overall. Despite the disadvantages of applying as international applicants, they were able to match at substantially higher-ranked dermatology programs than non-HP applicants. International applicants may have a longer time investment—the number of years from obtaining their medical degree or US medical license to matching—giving them time to produce quality research and develop meaningful relationships at an institution. Additionally, our prior study of the top 25 dermatology residencies showed that 26.2% of successful applicants matched at their research mentors’ institutions, with almost half of this subset matching at their HPs, where their mentors also practiced.² Because of the potential benefits of having program connections, applicants without HPs should seek dermatology research mentors, especially via highly beneficial in-person networking opportunities (eg, away rotations, conferences) that had previously been limited during the COVID-19 pandemic.³ Formal mentorship programs giving priority to students without HPs recently have been developed, which only begins to address the inequities in the dermatology residency application process.⁴

Study limitations include lack of resident information on 15 program websites, missed publications due to applicant name changes, not accounting for abstracts and posters, and inability to collect data on unmatched applicants.

We hope that our study alleviates some concerns that applicants without HPs may have regarding applying for dermatology residency and encourages those with a genuine interest in dermatology to pursue the specialty, provided they find a strong research mentor. Residency programs should be cognizant of the unique challenges that non-HP applicants face for matching.

To the Editor:

Dermatology is one of the most competitive residencies for matching, with a 57.5% match rate in 2022.¹ Our prior study of research-mentor relationships among matched dermatology applicants corroborated the importance of home programs (HPs) and program connections.² Therefore, our current objective was to compare profiles of matched dermatology applicants without HPs vs those with HPs.

We searched websites of 139 dermatology programs nationwide and found 1736 matched applicants from 2016 to 2020; of them, 323 did not have HPs. We determined program rank by research output using Doximity Residency Navigator (https://www.doximity.com/residency/). Advanced degrees (ADs) of applicants were identified using program websites and LinkedIn. A PubMed search was conducted for number of articles published by each applicant before September 15 of their match year. For applicants without HPs, we identified the senior author on each publication. The senior author publishing with an applicant most often was considered the research mentor. Two-tailed independent t tests and χ² tests were used to determine statistical significance (P<.05).

On average, matched applicants without HPs matched in lower-ranked (74.4) and smaller (12.4) programs compared with matched applicants with HPs (45.3 [P<.0001] and 15.1 [P<.0001], respectively)(eTable). The mean number of publications was similar between matched applicants with HPs and without HPs (5.64 and 4.80, respectively; P=.0525) as well as the percentage with ADs (14.7% and 11.5%, respectively; P=.0953). Overall, 14.8% of matched applicants without HPs matched at their mentors’ institutions.

Data were obtained for matched international applicants as a subset of non-HP applicants. Despite attending medical schools without associated HPs in the United States, international applicants matched at similarly ranked (44.3) and sized (15.0) programs, on average, compared with HP applicants. The mean number of publications was higher for international applicants (11.4) vs domestic applicants (5.33). International applicants more often had ADs (23.8%) and 60.1% of them held doctor of philosophy degrees. Overall, 40.5% of international applicants matched at their mentors’ institutions.

Our study suggests that matched dermatology applicants with and without HPs had similar achievements, on average, for the number of publications and percentage with ADs. However, non-HP applicants matched at lower-ranked programs than HP applicants. Therefore, applicants without HPs should strongly consider cultivating program connections, especially if they desire to match at higher-ranked dermatology programs. To illustrate, the rate of matching at research mentors’ institutions was approximately 3-times higher for international applicants than non-HP applicants overall. Despite the disadvantages of applying as international applicants, they were able to match at substantially higher-ranked dermatology programs than non-HP applicants. International applicants may have a longer time investment—the number of years from obtaining their medical degree or US medical license to matching—giving them time to produce quality research and develop meaningful relationships at an institution. Additionally, our prior study of the top 25 dermatology residencies showed that 26.2% of successful applicants matched at their research mentors’ institutions, with almost half of this subset matching at their HPs, where their mentors also practiced.² Because of the potential benefits of having program connections, applicants without HPs should seek dermatology research mentors, especially via highly beneficial in-person networking opportunities (eg, away rotations, conferences) that had previously been limited during the COVID-19 pandemic.³ Formal mentorship programs giving priority to students without HPs recently have been developed, which only begins to address the inequities in the dermatology residency application process.⁴

Study limitations include lack of resident information on 15 program websites, missed publications due to applicant name changes, not accounting for abstracts and posters, and inability to collect data on unmatched applicants.

We hope that our study alleviates some concerns that applicants without HPs may have regarding applying for dermatology residency and encourages those with a genuine interest in dermatology to pursue the specialty, provided they find a strong research mentor. Residency programs should be cognizant of the unique challenges that non-HP applicants face for matching.

To the Editor:

Dermatology is one of the most competitive residencies for matching, with a 57.5% match rate in 2022.¹ Our prior study of research-mentor relationships among matched dermatology applicants corroborated the importance of home programs (HPs) and program connections.² Therefore, our current objective was to compare profiles of matched dermatology applicants without HPs vs those with HPs.

We searched websites of 139 dermatology programs nationwide and found 1736 matched applicants from 2016 to 2020; of them, 323 did not have HPs. We determined program rank by research output using Doximity Residency Navigator (https://www.doximity.com/residency/). Advanced degrees (ADs) of applicants were identified using program websites and LinkedIn. A PubMed search was conducted for number of articles published by each applicant before September 15 of their match year. For applicants without HPs, we identified the senior author on each publication. The senior author publishing with an applicant most often was considered the research mentor. Two-tailed independent t tests and χ² tests were used to determine statistical significance (P<.05).

On average, matched applicants without HPs matched in lower-ranked (74.4) and smaller (12.4) programs compared with matched applicants with HPs (45.3 [P<.0001] and 15.1 [P<.0001], respectively)(eTable). The mean number of publications was similar between matched applicants with HPs and without HPs (5.64 and 4.80, respectively; P=.0525) as well as the percentage with ADs (14.7% and 11.5%, respectively; P=.0953). Overall, 14.8% of matched applicants without HPs matched at their mentors’ institutions.

Data were obtained for matched international applicants as a subset of non-HP applicants. Despite attending medical schools without associated HPs in the United States, international applicants matched at similarly ranked (44.3) and sized (15.0) programs, on average, compared with HP applicants. The mean number of publications was higher for international applicants (11.4) vs domestic applicants (5.33). International applicants more often had ADs (23.8%) and 60.1% of them held doctor of philosophy degrees. Overall, 40.5% of international applicants matched at their mentors’ institutions.

Our study suggests that matched dermatology applicants with and without HPs had similar achievements, on average, for the number of publications and percentage with ADs. However, non-HP applicants matched at lower-ranked programs than HP applicants. Therefore, applicants without HPs should strongly consider cultivating program connections, especially if they desire to match at higher-ranked dermatology programs. To illustrate, the rate of matching at research mentors’ institutions was approximately 3-times higher for international applicants than non-HP applicants overall. Despite the disadvantages of applying as international applicants, they were able to match at substantially higher-ranked dermatology programs than non-HP applicants. International applicants may have a longer time investment—the number of years from obtaining their medical degree or US medical license to matching—giving them time to produce quality research and develop meaningful relationships at an institution. Additionally, our prior study of the top 25 dermatology residencies showed that 26.2% of successful applicants matched at their research mentors’ institutions, with almost half of this subset matching at their HPs, where their mentors also practiced.² Because of the potential benefits of having program connections, applicants without HPs should seek dermatology research mentors, especially via highly beneficial in-person networking opportunities (eg, away rotations, conferences) that had previously been limited during the COVID-19 pandemic.³ Formal mentorship programs giving priority to students without HPs recently have been developed, which only begins to address the inequities in the dermatology residency application process.⁴

Study limitations include lack of resident information on 15 program websites, missed publications due to applicant name changes, not accounting for abstracts and posters, and inability to collect data on unmatched applicants.

We hope that our study alleviates some concerns that applicants without HPs may have regarding applying for dermatology residency and encourages those with a genuine interest in dermatology to pursue the specialty, provided they find a strong research mentor. Residency programs should be cognizant of the unique challenges that non-HP applicants face for matching.

Patients are finding it easier to use online resources to discover health care providers who fit their personalized needs. In the United States, approximately 70% of individuals use the internet to find health care information, and 80% are influenced by the information presented to them on health care websites.¹ Patients utilize the internet to better understand treatments offered by providers and their prices as well as how other patients have rated their experience. Providers in private practice also have noticed that many patients are referring themselves vs obtaining a referral from another provider.² As a result, it is critical for practice websites to have information that is of value to their patients, including the unique qualities and treatments offered. The purpose of this study was to analyze the differences between the content presented on dermatology private practice websites and academic institutional websites.

Methods

Websites Searched —All 140 academic dermatology programs, including both allopathic and osteopathic programs, were queried from the Association of American Medical Colleges (AAMC) database in March 2022. ³ First, the dermatology departmental websites for each program were analyzed to see if they contained information pertinent to patients. Any website that lacked this information or only had information relevant to the dermatology residency program was excluded from the study. After exclusion, a total of 113 websites were used in the academic website cohort. The private practices were found through an incognito Google search with the search term dermatologist and matched to be within 5 miles of each academic institution. The private practices that included at least one board-certified dermatologist and received the highest number of reviews on Google compared to other practices in the same region—a measure of online reputation—were selected to be in the private practice cohort (N = 113). Any duplicate practices, practices belonging to the same conglomerate company, or multispecialty clinics were excluded from the study. Board-certified dermatologists were confirmed using the Find a Dermatologist tool on the American Academy of Dermatology (AAD) website. ⁴

Website Assessments —Each website was assessed using 23 criteria divided into 4 categories: practice, physician(s), patient, and treatment/procedure (Table). Criteria for social media and publicity were further assessed. Criteria for social media included links on the website to a Facebook page, an Instagram account, a Twitter account, a Pinterest account, a LinkedIn account, a blog, a Yelp page, a YouTube channel, and/or any other social media. Criteria for publicity included links on the website to local television news, national news, newspapers, and/or magazines. ^5-8 Ease of site access was determined if the website was the first search result found on Google when searching for each website. Nondermatology professionals included listing of mid-level providers or researchers.

Four individuals (V.S.J., A.C.B., M.E.O., and M.B.B.) independently assessed each of the websites using the established criteria. Each criterion was defined and discussed prior to data collection to maintain consistency. The criteria were determined as being present if the website clearly displayed, stated, explained, or linked to the relevant content. If the website did not directly contain the content, it was determined that the criteria were absent. One other individual (J.P.) independently cross-examined the data for consistency and evaluated for any discrepancies. ⁸

A raw analysis was done between each cohort. Another analysis was done that controlled for population density and the proportionate population age in each city ⁹ in which an academic institution/private practice was located. We proposed that more densely populated cities naturally may have more competition between practices, which may result in more optimized websites. ¹⁰ We also anticipated similar findings in cities with younger populations, as the younger demographic may be more likely to utilize and value online information when compared to older populations. ¹¹ The websites for each cohort were equally divided into 3 tiers of population density (not shown) and population age (not shown).

Statistical Analysis —Statistical analysis was completed using descriptive statistics, χ ² testing, and Fisher exact tests where appropriate with a predetermined level of significance of P < .05 in Microsoft Excel.

Results

Demographics —A total of 226 websites from both private practices and academic institutions were evaluated. Of them, only 108 private practices and 108 academic institutions listed practicing dermatologists on their site. Of 108 private practices, 76 (70.4%) had more than one practicing board-certified dermatologist. Of 108 academic institutions, all 108 (100%) institutions had more than one practicing board-certified dermatologist.

 

Of the dermatologists who practiced at academic institutions (n=2014) and private practices (n=817), 1157 (57.4%) and 419 (51.2%) were females, respectively. The population density of the cities with each of these practices/institutions ranged from 137 individuals per square kilometer to 11,232 individuals per square kilometer (mean [SD] population density, 2579 [2485] individuals per square kilometer). Densely populated, moderately populated, and sparsely populated cities had a median population density of 4618, 1708, and 760 individuals per square kilometer, respectively. The data also were divided into 3 age groups. In the older population tier, the median percentage of individuals older than 64 years was 14.2%, the median percentage of individuals aged 18 to 64 years was 63.8%, and the median percentage of individuals aged 5 to 17 years was 14.9%. In the moderately aged population tier, the median percentage of individuals older than 64 years was 10.2%, the median percentage of individuals aged 18 to 64 years was 70.3%, and the median percentage of individuals aged 5 to 17 years was 13.6%. In the younger population tier, the median percentage of individuals older than 64 years was 12%, the median percentage of individuals aged 18 to 64 years was 66.8%, and the median percentage of individuals aged 5 to 17 years was 15%.

Practice and Physician Content—In the raw analysis (Figure), the most commonly listed types of content (>90% of websites) in both private practice and academic sites was address (range, 95% to 100%), telephone number (range, 97% to 100%), and dermatologist profiles (both 92%). The least commonly listed types of content in both cohorts was publicity (range, 20% to 23%). Private practices were more likely to list profiles of nondermatology professionals (73% vs 56%; P<.02), email (47% vs 17%; P<.0001), and social media (29% vs 8%; P<.0001) compared with academic institution websites. Although Facebook was the most-linked social media account for both groups, 75% of private practice sites included the link compared with 16% of academic institutions. Academic institutions were more likely to list fellowship availability (66% vs 1%; P<.0001). Accessing each website was significantly easier in the private practice cohort (99% vs 61%; P<.0001).

When controlling for population density, private practices were only more likely to list nondermatology professionals’ profiles in densely populated cities when compared with academic institutions (73% vs 41%; P<.01). Academic institutions continued to list fellowship availability more often than private practices regardless of population density. The same trend was observed for private practices with ease of site access and listing of social media.

When controlling for population age, similar trends were seen as when controlling for population density. However, private practices listing nondermatology professionals’ profiles was only more likely in the cities with a proportionately younger population when compared with academic institutions (74% vs 47%; P<.04). 

Patient and Treatment/Procedure—The most commonly listed content types on both private practice websites and academic institution websites were available treatments/procedures (range, 89% to 98%). The least commonly listed content included financing for elective procedures (range, 4% to 16%), consultation fees (range, 1% to 2%), FAQs (frequently asked questions)(range, 4% to 20%), and HIPAA (Health Insurance Portability and Accountability Act) policy (range, 12% to 22%). Private practices were more likely to list patient testimonials (52% vs 35%; P<.005), financing (16% vs 4%; P<.005), FAQs (20% vs 4%; P<.001), online appointments (77% vs 56%; P<.001), available treatments/procedures (98% vs 86%; P<.004), product advertisements (66% vs 16%; P<.0001), pictures of dermatology conditions (33% vs 13%; P<.001), and HIPAA policy (22% vs 12%; P<.04). Academic institutions were more likely to list research trials (65% vs 13%; P<.0001).

When controlling for population density, private practices were only more likely to list patient testimonials in densely populated (P=.035) and moderately populated cities (P=.019). The same trend was observed for online appointments in densely populated (P=.0023) and moderately populated cities (P=.037). Private practices continued to list product availability more often than academic institutions regardless of population density or population age. Academic institutions also continued to list research trials more often than private practices regardless of population density or population age. 

Comment

Our study uniquely analyzed the differences in website content between private practices and academic institutions in dermatology. Of the 140 academic institutions accredited by the Accreditation Council for Graduate Medical Education (ACGME), only 113 had patient-pertinent websites.

 

Access to Websites —There was a significant difference in many website content criteria between the 2 groups. Private practice sites were easier to access via a Google search when compared with academic sites, which likely is influenced by the Google search algorithm that ranks websites higher based on several criteria including but not limited to keyword use in the title tag, link popularity of the site, and historic ranking. ^12,13 Academic sites often were only accessible through portals found on their main institutional site or institution’s residency site.

Role of Social Media —Social media has been found to assist in educating patients on medical practices as well as selecting a physician. ^14,15 Our study found that private practice websites listed links to social media more often than their academic counterparts. Social media consumption is increasing, in part due to the COVID-19 pandemic, and it may be optimal for patients and practices alike to include links on their websites. ¹⁶ Facebook and Instagram were listed more often on private practice sites when compared with academic institution sites, which was similar to a recent study analyzing the websites of plastic surgery private practices (N = 310) in which 90% of private practices included some type of social media, with Instagram and Facebook being the most used. ⁸ Social networking accounts can act as convenient platforms for marketing, providing patient education, and generating referrals, which suggests that the prominence of their usage in private practice poses benefits in patient decision-making when seeking care. ^17-19 A study analyzing the impact of Facebook in medicine concluded that a Facebook page can serve as an effective vehicle for medical education, particularly in younger generations that favor technology-oriented teaching methods. ²⁰ A survey on trends in cosmetic facial procedures in plastic surgery found that the most influential online methods patients used for choosing their providers were social media platforms and practice websites. Front-page placement on Google also was commonly associated with the number of social media followers. ^21,22 A lack of social media prominence could hinder a website’s potential to reach patients.

Communication With Practices —Our study also found significant differences in other metrics related to a patient’s ability to directly communicate with a practice, such as physical addresses, telephone numbers, products available for direct purchase, and online appointment booking, all of which were listed more often on private practice websites compared with academic institution websites. Online appointment booking also was found more frequently on private practice websites. Although physical addresses and telephone numbers were listed significantly more often on private practice sites, this information was ubiquitous and easily accessible elsewhere. Academic institution websites listed research trials and fellowship training significantly more often than private practices. These differences imply a divergence in focus between private practices and academic institutions, likely because academic institutions are funded in large part from research grants, begetting a cycle of academic contribution. ²³ In contrast, private practices may not rely as heavily on academic revenue and may be more likely to prioritize other revenue streams such as product sales. ²⁴  

HIPAA Policy —Surprisingly, HIPAA policy rarely was listed on any private (22%) or academic site (12%). Conversely, in the plastic surgery study, HIPAA policy was listed much more often, with more than half of private practices with board-certified plastic surgeons accredited in the year 2015 including it on their website, ⁸ which may suggest that surgically oriented specialties, particularly cosmetic subspecialties, aim to more noticeably display their privacy policies for patient reassurance.

Study Limitations —There are several limitations of our study. First, it is common for a conglomerate company to own multiple private practices in different specialties. As with academic sites, private practice sites may be limited by the hosting platforms, which often are tedious to navigate. Also noteworthy is the emergence of designated social media management positions—both by practice employees and by third-party firms ²⁵ —but the impact of these positions in private practices and academic institutions has not been fully explored. Finally, inclusion criteria and standardized criteria definitions were chosen based on the precedent established by the authors of similar analyses in plastic surgery and radiology. ^5-8 Further investigation into the most valued aspects of care by patients within the context of the type of practice chosen would be valuable in refining inclusion criteria. Additionally, this study did not stratify the data collected based on factors such as gender, race, and geographical location; studies conducted on website traffic analysis patterns that focus on these aspects likely would further explain the significance of these findings. Differences in the length of time to the next available appointment between private practices and academic institutions also may help support our findings. Finally, there is a need for further investigation into the preferences of patients themselves garnered from website traffic alone.

Conclusion

Our study examined a diverse compilation of private practice and academic institution websites and uncovered numerous differences in content. As technology and health care continuously evolve, it is imperative that both private practices and academic institutions are actively adapting to optimize their online presence. In doing so, patients will be better equipped at accessing provider information, gaining familiarity with the practice, and understanding treatment options.  

Patients are finding it easier to use online resources to discover health care providers who fit their personalized needs. In the United States, approximately 70% of individuals use the internet to find health care information, and 80% are influenced by the information presented to them on health care websites.¹ Patients utilize the internet to better understand treatments offered by providers and their prices as well as how other patients have rated their experience. Providers in private practice also have noticed that many patients are referring themselves vs obtaining a referral from another provider.² As a result, it is critical for practice websites to have information that is of value to their patients, including the unique qualities and treatments offered. The purpose of this study was to analyze the differences between the content presented on dermatology private practice websites and academic institutional websites.

Methods

Websites Searched —All 140 academic dermatology programs, including both allopathic and osteopathic programs, were queried from the Association of American Medical Colleges (AAMC) database in March 2022. ³ First, the dermatology departmental websites for each program were analyzed to see if they contained information pertinent to patients. Any website that lacked this information or only had information relevant to the dermatology residency program was excluded from the study. After exclusion, a total of 113 websites were used in the academic website cohort. The private practices were found through an incognito Google search with the search term dermatologist and matched to be within 5 miles of each academic institution. The private practices that included at least one board-certified dermatologist and received the highest number of reviews on Google compared to other practices in the same region—a measure of online reputation—were selected to be in the private practice cohort (N = 113). Any duplicate practices, practices belonging to the same conglomerate company, or multispecialty clinics were excluded from the study. Board-certified dermatologists were confirmed using the Find a Dermatologist tool on the American Academy of Dermatology (AAD) website. ⁴

Website Assessments —Each website was assessed using 23 criteria divided into 4 categories: practice, physician(s), patient, and treatment/procedure (Table). Criteria for social media and publicity were further assessed. Criteria for social media included links on the website to a Facebook page, an Instagram account, a Twitter account, a Pinterest account, a LinkedIn account, a blog, a Yelp page, a YouTube channel, and/or any other social media. Criteria for publicity included links on the website to local television news, national news, newspapers, and/or magazines. ^5-8 Ease of site access was determined if the website was the first search result found on Google when searching for each website. Nondermatology professionals included listing of mid-level providers or researchers.

Four individuals (V.S.J., A.C.B., M.E.O., and M.B.B.) independently assessed each of the websites using the established criteria. Each criterion was defined and discussed prior to data collection to maintain consistency. The criteria were determined as being present if the website clearly displayed, stated, explained, or linked to the relevant content. If the website did not directly contain the content, it was determined that the criteria were absent. One other individual (J.P.) independently cross-examined the data for consistency and evaluated for any discrepancies. ⁸

A raw analysis was done between each cohort. Another analysis was done that controlled for population density and the proportionate population age in each city ⁹ in which an academic institution/private practice was located. We proposed that more densely populated cities naturally may have more competition between practices, which may result in more optimized websites. ¹⁰ We also anticipated similar findings in cities with younger populations, as the younger demographic may be more likely to utilize and value online information when compared to older populations. ¹¹ The websites for each cohort were equally divided into 3 tiers of population density (not shown) and population age (not shown).

Statistical Analysis —Statistical analysis was completed using descriptive statistics, χ ² testing, and Fisher exact tests where appropriate with a predetermined level of significance of P < .05 in Microsoft Excel.

Results

Demographics —A total of 226 websites from both private practices and academic institutions were evaluated. Of them, only 108 private practices and 108 academic institutions listed practicing dermatologists on their site. Of 108 private practices, 76 (70.4%) had more than one practicing board-certified dermatologist. Of 108 academic institutions, all 108 (100%) institutions had more than one practicing board-certified dermatologist.

 

Of the dermatologists who practiced at academic institutions (n=2014) and private practices (n=817), 1157 (57.4%) and 419 (51.2%) were females, respectively. The population density of the cities with each of these practices/institutions ranged from 137 individuals per square kilometer to 11,232 individuals per square kilometer (mean [SD] population density, 2579 [2485] individuals per square kilometer). Densely populated, moderately populated, and sparsely populated cities had a median population density of 4618, 1708, and 760 individuals per square kilometer, respectively. The data also were divided into 3 age groups. In the older population tier, the median percentage of individuals older than 64 years was 14.2%, the median percentage of individuals aged 18 to 64 years was 63.8%, and the median percentage of individuals aged 5 to 17 years was 14.9%. In the moderately aged population tier, the median percentage of individuals older than 64 years was 10.2%, the median percentage of individuals aged 18 to 64 years was 70.3%, and the median percentage of individuals aged 5 to 17 years was 13.6%. In the younger population tier, the median percentage of individuals older than 64 years was 12%, the median percentage of individuals aged 18 to 64 years was 66.8%, and the median percentage of individuals aged 5 to 17 years was 15%.

Practice and Physician Content—In the raw analysis (Figure), the most commonly listed types of content (>90% of websites) in both private practice and academic sites was address (range, 95% to 100%), telephone number (range, 97% to 100%), and dermatologist profiles (both 92%). The least commonly listed types of content in both cohorts was publicity (range, 20% to 23%). Private practices were more likely to list profiles of nondermatology professionals (73% vs 56%; P<.02), email (47% vs 17%; P<.0001), and social media (29% vs 8%; P<.0001) compared with academic institution websites. Although Facebook was the most-linked social media account for both groups, 75% of private practice sites included the link compared with 16% of academic institutions. Academic institutions were more likely to list fellowship availability (66% vs 1%; P<.0001). Accessing each website was significantly easier in the private practice cohort (99% vs 61%; P<.0001).

When controlling for population density, private practices were only more likely to list nondermatology professionals’ profiles in densely populated cities when compared with academic institutions (73% vs 41%; P<.01). Academic institutions continued to list fellowship availability more often than private practices regardless of population density. The same trend was observed for private practices with ease of site access and listing of social media.

When controlling for population age, similar trends were seen as when controlling for population density. However, private practices listing nondermatology professionals’ profiles was only more likely in the cities with a proportionately younger population when compared with academic institutions (74% vs 47%; P<.04). 

Patient and Treatment/Procedure—The most commonly listed content types on both private practice websites and academic institution websites were available treatments/procedures (range, 89% to 98%). The least commonly listed content included financing for elective procedures (range, 4% to 16%), consultation fees (range, 1% to 2%), FAQs (frequently asked questions)(range, 4% to 20%), and HIPAA (Health Insurance Portability and Accountability Act) policy (range, 12% to 22%). Private practices were more likely to list patient testimonials (52% vs 35%; P<.005), financing (16% vs 4%; P<.005), FAQs (20% vs 4%; P<.001), online appointments (77% vs 56%; P<.001), available treatments/procedures (98% vs 86%; P<.004), product advertisements (66% vs 16%; P<.0001), pictures of dermatology conditions (33% vs 13%; P<.001), and HIPAA policy (22% vs 12%; P<.04). Academic institutions were more likely to list research trials (65% vs 13%; P<.0001).

When controlling for population density, private practices were only more likely to list patient testimonials in densely populated (P=.035) and moderately populated cities (P=.019). The same trend was observed for online appointments in densely populated (P=.0023) and moderately populated cities (P=.037). Private practices continued to list product availability more often than academic institutions regardless of population density or population age. Academic institutions also continued to list research trials more often than private practices regardless of population density or population age. 

Comment

Our study uniquely analyzed the differences in website content between private practices and academic institutions in dermatology. Of the 140 academic institutions accredited by the Accreditation Council for Graduate Medical Education (ACGME), only 113 had patient-pertinent websites.

 

Access to Websites —There was a significant difference in many website content criteria between the 2 groups. Private practice sites were easier to access via a Google search when compared with academic sites, which likely is influenced by the Google search algorithm that ranks websites higher based on several criteria including but not limited to keyword use in the title tag, link popularity of the site, and historic ranking. ^12,13 Academic sites often were only accessible through portals found on their main institutional site or institution’s residency site.

Role of Social Media —Social media has been found to assist in educating patients on medical practices as well as selecting a physician. ^14,15 Our study found that private practice websites listed links to social media more often than their academic counterparts. Social media consumption is increasing, in part due to the COVID-19 pandemic, and it may be optimal for patients and practices alike to include links on their websites. ¹⁶ Facebook and Instagram were listed more often on private practice sites when compared with academic institution sites, which was similar to a recent study analyzing the websites of plastic surgery private practices (N = 310) in which 90% of private practices included some type of social media, with Instagram and Facebook being the most used. ⁸ Social networking accounts can act as convenient platforms for marketing, providing patient education, and generating referrals, which suggests that the prominence of their usage in private practice poses benefits in patient decision-making when seeking care. ^17-19 A study analyzing the impact of Facebook in medicine concluded that a Facebook page can serve as an effective vehicle for medical education, particularly in younger generations that favor technology-oriented teaching methods. ²⁰ A survey on trends in cosmetic facial procedures in plastic surgery found that the most influential online methods patients used for choosing their providers were social media platforms and practice websites. Front-page placement on Google also was commonly associated with the number of social media followers. ^21,22 A lack of social media prominence could hinder a website’s potential to reach patients.

Communication With Practices —Our study also found significant differences in other metrics related to a patient’s ability to directly communicate with a practice, such as physical addresses, telephone numbers, products available for direct purchase, and online appointment booking, all of which were listed more often on private practice websites compared with academic institution websites. Online appointment booking also was found more frequently on private practice websites. Although physical addresses and telephone numbers were listed significantly more often on private practice sites, this information was ubiquitous and easily accessible elsewhere. Academic institution websites listed research trials and fellowship training significantly more often than private practices. These differences imply a divergence in focus between private practices and academic institutions, likely because academic institutions are funded in large part from research grants, begetting a cycle of academic contribution. ²³ In contrast, private practices may not rely as heavily on academic revenue and may be more likely to prioritize other revenue streams such as product sales. ²⁴  

HIPAA Policy —Surprisingly, HIPAA policy rarely was listed on any private (22%) or academic site (12%). Conversely, in the plastic surgery study, HIPAA policy was listed much more often, with more than half of private practices with board-certified plastic surgeons accredited in the year 2015 including it on their website, ⁸ which may suggest that surgically oriented specialties, particularly cosmetic subspecialties, aim to more noticeably display their privacy policies for patient reassurance.

Study Limitations —There are several limitations of our study. First, it is common for a conglomerate company to own multiple private practices in different specialties. As with academic sites, private practice sites may be limited by the hosting platforms, which often are tedious to navigate. Also noteworthy is the emergence of designated social media management positions—both by practice employees and by third-party firms ²⁵ —but the impact of these positions in private practices and academic institutions has not been fully explored. Finally, inclusion criteria and standardized criteria definitions were chosen based on the precedent established by the authors of similar analyses in plastic surgery and radiology. ^5-8 Further investigation into the most valued aspects of care by patients within the context of the type of practice chosen would be valuable in refining inclusion criteria. Additionally, this study did not stratify the data collected based on factors such as gender, race, and geographical location; studies conducted on website traffic analysis patterns that focus on these aspects likely would further explain the significance of these findings. Differences in the length of time to the next available appointment between private practices and academic institutions also may help support our findings. Finally, there is a need for further investigation into the preferences of patients themselves garnered from website traffic alone.

Conclusion

Our study examined a diverse compilation of private practice and academic institution websites and uncovered numerous differences in content. As technology and health care continuously evolve, it is imperative that both private practices and academic institutions are actively adapting to optimize their online presence. In doing so, patients will be better equipped at accessing provider information, gaining familiarity with the practice, and understanding treatment options.  

Patients are finding it easier to use online resources to discover health care providers who fit their personalized needs. In the United States, approximately 70% of individuals use the internet to find health care information, and 80% are influenced by the information presented to them on health care websites.¹ Patients utilize the internet to better understand treatments offered by providers and their prices as well as how other patients have rated their experience. Providers in private practice also have noticed that many patients are referring themselves vs obtaining a referral from another provider.² As a result, it is critical for practice websites to have information that is of value to their patients, including the unique qualities and treatments offered. The purpose of this study was to analyze the differences between the content presented on dermatology private practice websites and academic institutional websites.

Methods

Websites Searched —All 140 academic dermatology programs, including both allopathic and osteopathic programs, were queried from the Association of American Medical Colleges (AAMC) database in March 2022. ³ First, the dermatology departmental websites for each program were analyzed to see if they contained information pertinent to patients. Any website that lacked this information or only had information relevant to the dermatology residency program was excluded from the study. After exclusion, a total of 113 websites were used in the academic website cohort. The private practices were found through an incognito Google search with the search term dermatologist and matched to be within 5 miles of each academic institution. The private practices that included at least one board-certified dermatologist and received the highest number of reviews on Google compared to other practices in the same region—a measure of online reputation—were selected to be in the private practice cohort (N = 113). Any duplicate practices, practices belonging to the same conglomerate company, or multispecialty clinics were excluded from the study. Board-certified dermatologists were confirmed using the Find a Dermatologist tool on the American Academy of Dermatology (AAD) website. ⁴

Website Assessments —Each website was assessed using 23 criteria divided into 4 categories: practice, physician(s), patient, and treatment/procedure (Table). Criteria for social media and publicity were further assessed. Criteria for social media included links on the website to a Facebook page, an Instagram account, a Twitter account, a Pinterest account, a LinkedIn account, a blog, a Yelp page, a YouTube channel, and/or any other social media. Criteria for publicity included links on the website to local television news, national news, newspapers, and/or magazines. ^5-8 Ease of site access was determined if the website was the first search result found on Google when searching for each website. Nondermatology professionals included listing of mid-level providers or researchers.

Four individuals (V.S.J., A.C.B., M.E.O., and M.B.B.) independently assessed each of the websites using the established criteria. Each criterion was defined and discussed prior to data collection to maintain consistency. The criteria were determined as being present if the website clearly displayed, stated, explained, or linked to the relevant content. If the website did not directly contain the content, it was determined that the criteria were absent. One other individual (J.P.) independently cross-examined the data for consistency and evaluated for any discrepancies. ⁸

A raw analysis was done between each cohort. Another analysis was done that controlled for population density and the proportionate population age in each city ⁹ in which an academic institution/private practice was located. We proposed that more densely populated cities naturally may have more competition between practices, which may result in more optimized websites. ¹⁰ We also anticipated similar findings in cities with younger populations, as the younger demographic may be more likely to utilize and value online information when compared to older populations. ¹¹ The websites for each cohort were equally divided into 3 tiers of population density (not shown) and population age (not shown).

Statistical Analysis —Statistical analysis was completed using descriptive statistics, χ ² testing, and Fisher exact tests where appropriate with a predetermined level of significance of P < .05 in Microsoft Excel.

Results

Demographics —A total of 226 websites from both private practices and academic institutions were evaluated. Of them, only 108 private practices and 108 academic institutions listed practicing dermatologists on their site. Of 108 private practices, 76 (70.4%) had more than one practicing board-certified dermatologist. Of 108 academic institutions, all 108 (100%) institutions had more than one practicing board-certified dermatologist.

 

Of the dermatologists who practiced at academic institutions (n=2014) and private practices (n=817), 1157 (57.4%) and 419 (51.2%) were females, respectively. The population density of the cities with each of these practices/institutions ranged from 137 individuals per square kilometer to 11,232 individuals per square kilometer (mean [SD] population density, 2579 [2485] individuals per square kilometer). Densely populated, moderately populated, and sparsely populated cities had a median population density of 4618, 1708, and 760 individuals per square kilometer, respectively. The data also were divided into 3 age groups. In the older population tier, the median percentage of individuals older than 64 years was 14.2%, the median percentage of individuals aged 18 to 64 years was 63.8%, and the median percentage of individuals aged 5 to 17 years was 14.9%. In the moderately aged population tier, the median percentage of individuals older than 64 years was 10.2%, the median percentage of individuals aged 18 to 64 years was 70.3%, and the median percentage of individuals aged 5 to 17 years was 13.6%. In the younger population tier, the median percentage of individuals older than 64 years was 12%, the median percentage of individuals aged 18 to 64 years was 66.8%, and the median percentage of individuals aged 5 to 17 years was 15%.

Practice and Physician Content—In the raw analysis (Figure), the most commonly listed types of content (>90% of websites) in both private practice and academic sites was address (range, 95% to 100%), telephone number (range, 97% to 100%), and dermatologist profiles (both 92%). The least commonly listed types of content in both cohorts was publicity (range, 20% to 23%). Private practices were more likely to list profiles of nondermatology professionals (73% vs 56%; P<.02), email (47% vs 17%; P<.0001), and social media (29% vs 8%; P<.0001) compared with academic institution websites. Although Facebook was the most-linked social media account for both groups, 75% of private practice sites included the link compared with 16% of academic institutions. Academic institutions were more likely to list fellowship availability (66% vs 1%; P<.0001). Accessing each website was significantly easier in the private practice cohort (99% vs 61%; P<.0001).

When controlling for population density, private practices were only more likely to list nondermatology professionals’ profiles in densely populated cities when compared with academic institutions (73% vs 41%; P<.01). Academic institutions continued to list fellowship availability more often than private practices regardless of population density. The same trend was observed for private practices with ease of site access and listing of social media.

When controlling for population age, similar trends were seen as when controlling for population density. However, private practices listing nondermatology professionals’ profiles was only more likely in the cities with a proportionately younger population when compared with academic institutions (74% vs 47%; P<.04). 

Patient and Treatment/Procedure—The most commonly listed content types on both private practice websites and academic institution websites were available treatments/procedures (range, 89% to 98%). The least commonly listed content included financing for elective procedures (range, 4% to 16%), consultation fees (range, 1% to 2%), FAQs (frequently asked questions)(range, 4% to 20%), and HIPAA (Health Insurance Portability and Accountability Act) policy (range, 12% to 22%). Private practices were more likely to list patient testimonials (52% vs 35%; P<.005), financing (16% vs 4%; P<.005), FAQs (20% vs 4%; P<.001), online appointments (77% vs 56%; P<.001), available treatments/procedures (98% vs 86%; P<.004), product advertisements (66% vs 16%; P<.0001), pictures of dermatology conditions (33% vs 13%; P<.001), and HIPAA policy (22% vs 12%; P<.04). Academic institutions were more likely to list research trials (65% vs 13%; P<.0001).

When controlling for population density, private practices were only more likely to list patient testimonials in densely populated (P=.035) and moderately populated cities (P=.019). The same trend was observed for online appointments in densely populated (P=.0023) and moderately populated cities (P=.037). Private practices continued to list product availability more often than academic institutions regardless of population density or population age. Academic institutions also continued to list research trials more often than private practices regardless of population density or population age. 

Comment

Our study uniquely analyzed the differences in website content between private practices and academic institutions in dermatology. Of the 140 academic institutions accredited by the Accreditation Council for Graduate Medical Education (ACGME), only 113 had patient-pertinent websites.

 

Access to Websites —There was a significant difference in many website content criteria between the 2 groups. Private practice sites were easier to access via a Google search when compared with academic sites, which likely is influenced by the Google search algorithm that ranks websites higher based on several criteria including but not limited to keyword use in the title tag, link popularity of the site, and historic ranking. ^12,13 Academic sites often were only accessible through portals found on their main institutional site or institution’s residency site.

Role of Social Media —Social media has been found to assist in educating patients on medical practices as well as selecting a physician. ^14,15 Our study found that private practice websites listed links to social media more often than their academic counterparts. Social media consumption is increasing, in part due to the COVID-19 pandemic, and it may be optimal for patients and practices alike to include links on their websites. ¹⁶ Facebook and Instagram were listed more often on private practice sites when compared with academic institution sites, which was similar to a recent study analyzing the websites of plastic surgery private practices (N = 310) in which 90% of private practices included some type of social media, with Instagram and Facebook being the most used. ⁸ Social networking accounts can act as convenient platforms for marketing, providing patient education, and generating referrals, which suggests that the prominence of their usage in private practice poses benefits in patient decision-making when seeking care. ^17-19 A study analyzing the impact of Facebook in medicine concluded that a Facebook page can serve as an effective vehicle for medical education, particularly in younger generations that favor technology-oriented teaching methods. ²⁰ A survey on trends in cosmetic facial procedures in plastic surgery found that the most influential online methods patients used for choosing their providers were social media platforms and practice websites. Front-page placement on Google also was commonly associated with the number of social media followers. ^21,22 A lack of social media prominence could hinder a website’s potential to reach patients.

Communication With Practices —Our study also found significant differences in other metrics related to a patient’s ability to directly communicate with a practice, such as physical addresses, telephone numbers, products available for direct purchase, and online appointment booking, all of which were listed more often on private practice websites compared with academic institution websites. Online appointment booking also was found more frequently on private practice websites. Although physical addresses and telephone numbers were listed significantly more often on private practice sites, this information was ubiquitous and easily accessible elsewhere. Academic institution websites listed research trials and fellowship training significantly more often than private practices. These differences imply a divergence in focus between private practices and academic institutions, likely because academic institutions are funded in large part from research grants, begetting a cycle of academic contribution. ²³ In contrast, private practices may not rely as heavily on academic revenue and may be more likely to prioritize other revenue streams such as product sales. ²⁴  

HIPAA Policy —Surprisingly, HIPAA policy rarely was listed on any private (22%) or academic site (12%). Conversely, in the plastic surgery study, HIPAA policy was listed much more often, with more than half of private practices with board-certified plastic surgeons accredited in the year 2015 including it on their website, ⁸ which may suggest that surgically oriented specialties, particularly cosmetic subspecialties, aim to more noticeably display their privacy policies for patient reassurance.

Study Limitations —There are several limitations of our study. First, it is common for a conglomerate company to own multiple private practices in different specialties. As with academic sites, private practice sites may be limited by the hosting platforms, which often are tedious to navigate. Also noteworthy is the emergence of designated social media management positions—both by practice employees and by third-party firms ²⁵ —but the impact of these positions in private practices and academic institutions has not been fully explored. Finally, inclusion criteria and standardized criteria definitions were chosen based on the precedent established by the authors of similar analyses in plastic surgery and radiology. ^5-8 Further investigation into the most valued aspects of care by patients within the context of the type of practice chosen would be valuable in refining inclusion criteria. Additionally, this study did not stratify the data collected based on factors such as gender, race, and geographical location; studies conducted on website traffic analysis patterns that focus on these aspects likely would further explain the significance of these findings. Differences in the length of time to the next available appointment between private practices and academic institutions also may help support our findings. Finally, there is a need for further investigation into the preferences of patients themselves garnered from website traffic alone.

Conclusion

Our study examined a diverse compilation of private practice and academic institution websites and uncovered numerous differences in content. As technology and health care continuously evolve, it is imperative that both private practices and academic institutions are actively adapting to optimize their online presence. In doing so, patients will be better equipped at accessing provider information, gaining familiarity with the practice, and understanding treatment options.  

Psoriasis is an inflammatory autoimmune skin condition that affects approximately 125 million individuals worldwide, with approximately 8 million patients in the United States.¹ Psoriasis not only involves a cosmetic component but also comprises other comorbidities, such as psoriatic arthritis, cardiovascular disease, and psychiatric disorders, that can influence patient quality of life.^2-4 In addition, the costs associated with psoriasis are substantial, with an estimated economic burden of $35.2 billion in the United States in 2015.⁵ Given the prevalence of psoriasis and its many effects on patients, it is important that providers have high-quality evidence regarding efficacious treatment options.

Systematic reviews, which compile all available evidence on a subject to answer a specific question, represent the gold standard of research.⁶ However, studies have demonstrated that when referencing research literature, physicians tend to read only the abstract of a study rather than the entire article.^7,8 A study by Marcelo et al⁸ showed that residents at a tertiary care center answered clinical questions using only the abstract of a paper 69% of the time. Based on these findings, it is imperative that the results of systematic reviews be accurately reported in their abstracts because they can influence patient care.

Referencing only the abstracts of systematic reviews can be problematic if the abstract contains spin. Spin is a form of reporting that inappropriately highlights the benefits of a treatment with greater emphasis than what is shown by the results.⁹ Research has identified the presence of spin in the abstracts of randomized controlled trials.^10-12 For example, Cooper et al¹⁰ found that 70% (33/47) of abstracts in otolaryngology randomized controlled trials contained spin. Additionally, Arthur et al¹¹ and Austin et al¹² had similar findings within abstracts of orthopedic and obesity trials, where 44.8% (112/250) and 46.7% (21/45) contained spin, respectively. Ottwell et al¹³ found that the presence of spin in abstracts is not limited to randomized controlled trials; they demonstrated that the abstracts of nearly one-third (31% [11/36]) of systematic reviews focused on the treatment of acne vulgaris contained spin.

In our study, we aimed to evaluate the presence of spin in the abstracts of systematic reviews focused on the treatment of psoriasis.

Methods

Reproducibility and Reporting—Our study did not meet the regulatory definition for human subjects research per the US Code of Federal Regulations because the study did not involve human research subjects. The study also was not subject to review by the institutional review board. Our protocol, data set, analysis scripts, extraction forms, and other material related to the study have been placed on Open Science Framework to provide transparency and ensure reproducibility. To further allow for analytic reproducibility, our data set was given to an independent laboratory and reanalyzed with a masked approach. Our study was carried out alongside other studies assessing spin in systematic reviews regarding different specialties and disease states. Because these studies were similar in design, this methodology also has been reported elsewhere. The Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA)¹⁴ and the guidelines for meta-epidemiological studies developed by Murad and Wang¹⁵ were used in drafting this article.

Search Strategy—The search strategies for the MEDLINE (Ovid) and Embase (Ovid) databases were created by a systematic review librarian (D.N.W.) to identify systematic reviews and meta-analyses regarding treatments for psoriasis (Figure 1). The searches were performed on June 2, 2020, and uploaded to Rayyan, a systematic review screening platform.¹⁶ After duplicates were removed, the records were screened for eligibility by 2 authors (C.H. and A.L.) using the titles and abstracts. Screening was conducted independently while each of these authors was masked to the other’s results; disagreements were resolved through discussion.

Eligibility Criteria—An article had to meet the following criteria for inclusion in our study: (1) be a systematic review with or without a meta-analysis; (2) relate to the treatment of psoriasis; and (3) be written in English and include human patients only. The PRISMA definition of systematic reviews and meta-analyses was applied.¹⁷

 

Training—Various training occurred throughout our study to ensure understanding of each step and mitigate subjectivity. Before beginning screening, 2 investigators (C.H. and A.L.) completed the Introduction to Systematic Review and Meta-Analysis course offered by Johns Hopkins University.¹⁸ They also underwent 2 days of online and in-person training on the definition and interpretation of the 9 most severe types of spin found in the abstracts of systematic reviews as defined by Yavchitz et al.⁹ Finally, they were trained to use A MeaSurement Tool to Assess systematic Reviews (AMSTAR-2) to appraise the methodological quality of each systematic review. Our protocol contained an outline of all training modules used.

Data Extraction—The investigators (C.H. and A.L.) analyzed included abstracts for the 9 most severe types of spin (Table 1). Data were extracted in a masked duplicate fashion using the Google form. AMSTAR-2 was used to assess systematic reviews for methodological quality. AMSTAR-2 is an appraisal tool consisting of a 16-item checklist for systematic reviews or meta-analyses. Scores range from critically low to high based on the methodological quality of the review. Interrater reliability of AMSTAR-2 scores has been moderate to high across studies. Construct validity coefficients have been high with the original AMSTAR instrument (r=0.91) and the Risk of Bias in Systematic Reviews instrument (r=0.84).¹⁹

During data extraction from each included systematic review, the following additional items were obtained: (1) the date the review was received; (2) intervention type (ie, pharmacologic, nonpharmacologic, surgery, light therapy, mixed); (3) the funding source(s) for each systematic review (ie, industry, private, public, none, not mentioned, hospital, a combination of funding not including industry, a combination of funding including industry, other); (4) whether the journal submission guidelines suggested adherence to PRISMA guidelines; (5) whether the review discussed adherence to PRISMA¹⁴ or PRISMA for Abstracts²⁰ (PRISMA-A); (6) the publishing journal’s 5-year impact factor; and (6) the country of the systematic review’s origin. When data extraction was complete, investigators (C.H. and A.L.) were unmasked and met to resolve any disagreements by discussion. Two authors (R.O. or M.V.) served as arbiters in the case that an agreement between C.H. and A.L. could not be reached.

Statistical Analysis—Frequencies and percentages were calculated to evaluate the most common types of spin found within systematic reviews and meta-analyses. One author (M.H.) prespecified the possibility of a binary logistic regression and calculated a power analysis to determine sample size, as stated in our protocol. Our final sample size of 173 was not powered to perform the multivariable logistic regression; therefore, we calculated unadjusted odds ratios to enable assessing relationships between the presence of spin in abstracts and the various study characteristics. We used Stata 16.1 for all analyses, and all analytic decisions can be found in our protocol.

Results

General Characteristics—Our systematic search of MEDLINE and Embase returned 3200 articles, of which 665 were duplicates that were removed. An additional 2253 articles were excluded during initial abstract and title screening, and full-text screening led to the exclusion of another 109 articles. In total, 173 systematic reviews were included for data extraction. Figure 2 illustrates the screening process with the rationale for all exclusions.

Of the 173 included systematic reviews and meta-analyses, 150 (86.7%) focused on pharmacologic interventions. The majority of studies did not mention adhering to PRISMA guidelines (125/173 [72.3%]), and the publishing journals recommended their authors adhere to PRISMA for only 66 (38.2%) of the included articles. For the articles that received funding (90/173 [52.0%]), industry sources were the most common funding source (40/90 [44.4%]), followed by private (27/90 [30%]) and public funding sources (23/90 [25.6%]). Of the remaining studies, 46 articles did not include a funding statement (46/83 [55.4%]), and 37 studies were not funded (37/83 [44.6%]). The average (SD) 5-year impact factor of our included journals was 4.68 (4.64). Systematic reviews were from 31 different countries. All studies were received by their respective journals between the years 2000 and 2020 (Table 2).

Abstracts Containing Spin—We found that 37 (21.4%) of the abstracts of systematic reviews focused on psoriasis treatments contained at least 1 type of spin. Some abstracts had more than 1 type; thus, a total of 51 different instances of spin were detected. Spin type 6—selective reporting of or overemphasis on harm outcomes or analysis favoring the safety of the experimental intervention—was the most common type ofspin, found in 19 of 173 abstracts (11.0%). The most severe type of spin—type 1 (conclusion contains recommendations for clinical practice not supported by the findings)—occurred in only 1 abstract (0.6%). Spin type 8 did not occur in any of the abstracts (Table 1). There was no statistically significant association between the presence of spin and any of the study characteristics (Table 2).

 

AMSTAR Ratings—After using AMSTAR-2 to appraise the included systematic reviews, we found that 6 (3.5%) of the 173 studies could be rated as high; 36 (20.8%) as moderate; 25 (14.5%) as low; and 106 (61.3%) as critically low. Of the 37 abstracts containing spin, 2 (5.4%) had an AMSTAR-2 rating of high, 10 (27%) had a rating of moderate, 6 (16.2%) had a rating of low, and 19 (51.4%) had a rating of critically low (Table 2). No statistically significant associations were seen between abstracts found to have spin and the AMSTAR-2 rating of the review.

Nearly all (160/173 [92.5%]) of the included reviews were compliant with the inclusion of Population, Intervention, Comparison, and Outcome (PICO) method. Only 17 of 173 (9.8%) reviews reported funding sources for the studies included. See Table 3 for all AMSTAR-2 items.

Comment

Primary Findings—We evaluated the abstracts of systematic reviews for the treatment of psoriasis and found that more than one-fifth of them contained spin. Our study contributes to the existing literature surrounding spin. Spin in randomized controlled trials is well documented across several fields of medicine, including otolaryngology,¹⁰ obesity medicine,¹² dermatology,²¹ anesthesiology,²² psychiatry,²³ orthopedics,²⁴ emergency medicine,²⁵ oncology,²⁶ and cardiology.²⁷ More recently, studies have emerged evaluating the presence of spin in systematic reviews. Specific to dermatology, one study found that 74% (84/113) of systematic reviews related to atopic dermatitis treatment contained spin.²⁸ Additionally, Ottwell et al¹³ identified spin in 31% (11/36) of the systematic reviews related to the treatment of acne vulgaris, which is similar to our results for systematic reviews focused on psoriasis treatments. When comparing the presence of spin in abstracts of systematic reviews from the field of dermatology with other specialties, dermatology-focused systematic reviews appear to contain more spin in the abstract than systematic reviews focused on tinnitus and glaucoma therapies.^29,30 However, systematic reviews from the field of dermatology appear to contain less spin than systematic reviews focused on therapies for lower back pain.³¹ For example, Nascimento et al³¹ found that 80% (53/66) of systematic reviews focused on low-back pain treatments contained spin.

Examples of Spin—The most common type of spin found in our study was type 6.⁹ An example of spin type 6 can be found in an article by Bai et al³² that investigated the short-term efficacy and safety of multiple interleukin inhibitors for the treatment of plaque psoriasis. The conclusion of the abstract states, “Risankizumab appeared to have relatively high efficacy and low risk.” However, in the results section, the authors showed that risankizumab had the highest risk of serious adverse events and was ranked highest for discontinuation because of adverse events when compared with other interleukin inhibitors. Here, the presence of spin in the abstract may mislead the reader to accept the “low risk” of risankizumab without understanding the study’s full results.³²

Another example of selective reporting of harm outcomes in a systematic review can be found in the article by Wu et al,³³ which focused on assessing IL-17 antagonists for the treatment of plaque psoriasis. The conclusion of the abstract indicated that IL-17 antagonists should be accepted as safe; however, in the results section, the authors discussed serious safety concerns with brodalumab, including the death of 4 patients from suicide.³³ This example of spin type 6 highlights how the overgeneralization of a drug’s safety profile neglects serious harm outcomes that are critical to patient safety. In fact, against the safety claims of Wu et al,³³ brodalumab later received a boxed warning from the US Food and Drug Administration after 6 patients died from suicide while receiving the drug, which led to early discontinuation of the trials.^34,35 Although studies suggest this relationship is not causal,^34-36 the purpose of our study was not to investigate this association but to highlight the importance of this finding. Thus, with this example of spin in mind, we offer recommendations that we believe will improve reporting in abstracts as well as quality of patient care.

Recommendations for Reporting in Abstracts—Regarding the boxed warning³⁷ for brodalumab because of suicidal ideation and behavior, the US Food and Drug Administration recommends that prior to prescribing brodalumab, clinicians consider the potential benefits and risks in patients with a history of depression and/or suicidal ideation or behavior. However, a clinician would not adequately assess the full risks and benefits when an abstract, such as that for the article by Wu et al,³³ contains spin through selectively reporting harm outcomes. Arguably, clinicians could just read the full text; however, research confirms that abstracts often are utilized by clinicians and commonly are used to guide clinical decisions.^7,38 It is reasonable that clinicians would use abstracts in this fashion because they provide a quick synopsis of the full article’s findings and are widely available to clinicians who may not have access to article databases. Initiatives are in place to improve the quality of reporting in an abstract, such as PRISMA-A,²⁰ but even this fails to address spin. In fact, it may suggest spin because checklist item 10 of PRISMA-A advises authors of systematic reviews to provide a “general interpretation of the results and important implications.” This item is concerning because it suggests that the authors interpret importance rather than the clinician who prescribes the drug and is ultimately responsible for patient safety. Therefore, we recommend a reform to abstract reporting and an update to PRISMA-A that leads authors to report all benefits and risks encountered instead of reporting what the authors define as important.

Strengths and Limitations—Our study has several strengths as well as limitations. One of these strengths is that our protocol was strictly adhered to; any deviations were noted and added as an amendment. Our protocol, data, and all study artifacts were made freely available online on the Open Science Framework to strengthen reproducibility (https://osf.io/zrxh8/). Investigators underwent training to ensure comprehension of spin and systematic review designs. All data were extracted in masked duplicate fashion per the Cochrane Handbook for Systematic Reviews of Interventions.³⁹

Regarding limitations, only 2 databases were searched—MEDLINE and Embase. Therefore, our screening process may not have included every available systematic review on the treatment of psoriasis. Journal impact factors may be inaccurate for the systematic reviews that were published earlier in our data date range; however, we attempted to negate this limitation by using a 5-year average. Our study characteristic regarding PRISMA adherence did not account for studies published before the PRISMA statement release; we also could not access prior submission guidelines to determine when a journal began recommending PRISMA adherence. Another limitation of our study was the intrinsic subjectivity behind spin. Some may disagree with our classifications. Finally, our cross-sectional design should not be generalized to study types that are not systematic reviews or published in other journals during different periods.

Conclusion

Evidence of spin was present in many of the abstracts of systematic reviews pertaining to the treatment of psoriasis. Future clinical research should investigate any reporting of spin and search for ways to better reduce spin within literature. Continued research is necessary to evaluate the presence of spin within dermatology and other specialties.

Psoriasis is an inflammatory autoimmune skin condition that affects approximately 125 million individuals worldwide, with approximately 8 million patients in the United States.¹ Psoriasis not only involves a cosmetic component but also comprises other comorbidities, such as psoriatic arthritis, cardiovascular disease, and psychiatric disorders, that can influence patient quality of life.^2-4 In addition, the costs associated with psoriasis are substantial, with an estimated economic burden of $35.2 billion in the United States in 2015.⁵ Given the prevalence of psoriasis and its many effects on patients, it is important that providers have high-quality evidence regarding efficacious treatment options.

Systematic reviews, which compile all available evidence on a subject to answer a specific question, represent the gold standard of research.⁶ However, studies have demonstrated that when referencing research literature, physicians tend to read only the abstract of a study rather than the entire article.^7,8 A study by Marcelo et al⁸ showed that residents at a tertiary care center answered clinical questions using only the abstract of a paper 69% of the time. Based on these findings, it is imperative that the results of systematic reviews be accurately reported in their abstracts because they can influence patient care.

Referencing only the abstracts of systematic reviews can be problematic if the abstract contains spin. Spin is a form of reporting that inappropriately highlights the benefits of a treatment with greater emphasis than what is shown by the results.⁹ Research has identified the presence of spin in the abstracts of randomized controlled trials.^10-12 For example, Cooper et al¹⁰ found that 70% (33/47) of abstracts in otolaryngology randomized controlled trials contained spin. Additionally, Arthur et al¹¹ and Austin et al¹² had similar findings within abstracts of orthopedic and obesity trials, where 44.8% (112/250) and 46.7% (21/45) contained spin, respectively. Ottwell et al¹³ found that the presence of spin in abstracts is not limited to randomized controlled trials; they demonstrated that the abstracts of nearly one-third (31% [11/36]) of systematic reviews focused on the treatment of acne vulgaris contained spin.

In our study, we aimed to evaluate the presence of spin in the abstracts of systematic reviews focused on the treatment of psoriasis.

Methods

Reproducibility and Reporting—Our study did not meet the regulatory definition for human subjects research per the US Code of Federal Regulations because the study did not involve human research subjects. The study also was not subject to review by the institutional review board. Our protocol, data set, analysis scripts, extraction forms, and other material related to the study have been placed on Open Science Framework to provide transparency and ensure reproducibility. To further allow for analytic reproducibility, our data set was given to an independent laboratory and reanalyzed with a masked approach. Our study was carried out alongside other studies assessing spin in systematic reviews regarding different specialties and disease states. Because these studies were similar in design, this methodology also has been reported elsewhere. The Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA)¹⁴ and the guidelines for meta-epidemiological studies developed by Murad and Wang¹⁵ were used in drafting this article.

Search Strategy—The search strategies for the MEDLINE (Ovid) and Embase (Ovid) databases were created by a systematic review librarian (D.N.W.) to identify systematic reviews and meta-analyses regarding treatments for psoriasis (Figure 1). The searches were performed on June 2, 2020, and uploaded to Rayyan, a systematic review screening platform.¹⁶ After duplicates were removed, the records were screened for eligibility by 2 authors (C.H. and A.L.) using the titles and abstracts. Screening was conducted independently while each of these authors was masked to the other’s results; disagreements were resolved through discussion.

Eligibility Criteria—An article had to meet the following criteria for inclusion in our study: (1) be a systematic review with or without a meta-analysis; (2) relate to the treatment of psoriasis; and (3) be written in English and include human patients only. The PRISMA definition of systematic reviews and meta-analyses was applied.¹⁷

 

Training—Various training occurred throughout our study to ensure understanding of each step and mitigate subjectivity. Before beginning screening, 2 investigators (C.H. and A.L.) completed the Introduction to Systematic Review and Meta-Analysis course offered by Johns Hopkins University.¹⁸ They also underwent 2 days of online and in-person training on the definition and interpretation of the 9 most severe types of spin found in the abstracts of systematic reviews as defined by Yavchitz et al.⁹ Finally, they were trained to use A MeaSurement Tool to Assess systematic Reviews (AMSTAR-2) to appraise the methodological quality of each systematic review. Our protocol contained an outline of all training modules used.

Data Extraction—The investigators (C.H. and A.L.) analyzed included abstracts for the 9 most severe types of spin (Table 1). Data were extracted in a masked duplicate fashion using the Google form. AMSTAR-2 was used to assess systematic reviews for methodological quality. AMSTAR-2 is an appraisal tool consisting of a 16-item checklist for systematic reviews or meta-analyses. Scores range from critically low to high based on the methodological quality of the review. Interrater reliability of AMSTAR-2 scores has been moderate to high across studies. Construct validity coefficients have been high with the original AMSTAR instrument (r=0.91) and the Risk of Bias in Systematic Reviews instrument (r=0.84).¹⁹

During data extraction from each included systematic review, the following additional items were obtained: (1) the date the review was received; (2) intervention type (ie, pharmacologic, nonpharmacologic, surgery, light therapy, mixed); (3) the funding source(s) for each systematic review (ie, industry, private, public, none, not mentioned, hospital, a combination of funding not including industry, a combination of funding including industry, other); (4) whether the journal submission guidelines suggested adherence to PRISMA guidelines; (5) whether the review discussed adherence to PRISMA¹⁴ or PRISMA for Abstracts²⁰ (PRISMA-A); (6) the publishing journal’s 5-year impact factor; and (6) the country of the systematic review’s origin. When data extraction was complete, investigators (C.H. and A.L.) were unmasked and met to resolve any disagreements by discussion. Two authors (R.O. or M.V.) served as arbiters in the case that an agreement between C.H. and A.L. could not be reached.

Statistical Analysis—Frequencies and percentages were calculated to evaluate the most common types of spin found within systematic reviews and meta-analyses. One author (M.H.) prespecified the possibility of a binary logistic regression and calculated a power analysis to determine sample size, as stated in our protocol. Our final sample size of 173 was not powered to perform the multivariable logistic regression; therefore, we calculated unadjusted odds ratios to enable assessing relationships between the presence of spin in abstracts and the various study characteristics. We used Stata 16.1 for all analyses, and all analytic decisions can be found in our protocol.

Results

General Characteristics—Our systematic search of MEDLINE and Embase returned 3200 articles, of which 665 were duplicates that were removed. An additional 2253 articles were excluded during initial abstract and title screening, and full-text screening led to the exclusion of another 109 articles. In total, 173 systematic reviews were included for data extraction. Figure 2 illustrates the screening process with the rationale for all exclusions.

Of the 173 included systematic reviews and meta-analyses, 150 (86.7%) focused on pharmacologic interventions. The majority of studies did not mention adhering to PRISMA guidelines (125/173 [72.3%]), and the publishing journals recommended their authors adhere to PRISMA for only 66 (38.2%) of the included articles. For the articles that received funding (90/173 [52.0%]), industry sources were the most common funding source (40/90 [44.4%]), followed by private (27/90 [30%]) and public funding sources (23/90 [25.6%]). Of the remaining studies, 46 articles did not include a funding statement (46/83 [55.4%]), and 37 studies were not funded (37/83 [44.6%]). The average (SD) 5-year impact factor of our included journals was 4.68 (4.64). Systematic reviews were from 31 different countries. All studies were received by their respective journals between the years 2000 and 2020 (Table 2).

Abstracts Containing Spin—We found that 37 (21.4%) of the abstracts of systematic reviews focused on psoriasis treatments contained at least 1 type of spin. Some abstracts had more than 1 type; thus, a total of 51 different instances of spin were detected. Spin type 6—selective reporting of or overemphasis on harm outcomes or analysis favoring the safety of the experimental intervention—was the most common type ofspin, found in 19 of 173 abstracts (11.0%). The most severe type of spin—type 1 (conclusion contains recommendations for clinical practice not supported by the findings)—occurred in only 1 abstract (0.6%). Spin type 8 did not occur in any of the abstracts (Table 1). There was no statistically significant association between the presence of spin and any of the study characteristics (Table 2).

 

AMSTAR Ratings—After using AMSTAR-2 to appraise the included systematic reviews, we found that 6 (3.5%) of the 173 studies could be rated as high; 36 (20.8%) as moderate; 25 (14.5%) as low; and 106 (61.3%) as critically low. Of the 37 abstracts containing spin, 2 (5.4%) had an AMSTAR-2 rating of high, 10 (27%) had a rating of moderate, 6 (16.2%) had a rating of low, and 19 (51.4%) had a rating of critically low (Table 2). No statistically significant associations were seen between abstracts found to have spin and the AMSTAR-2 rating of the review.

Nearly all (160/173 [92.5%]) of the included reviews were compliant with the inclusion of Population, Intervention, Comparison, and Outcome (PICO) method. Only 17 of 173 (9.8%) reviews reported funding sources for the studies included. See Table 3 for all AMSTAR-2 items.

Comment

Primary Findings—We evaluated the abstracts of systematic reviews for the treatment of psoriasis and found that more than one-fifth of them contained spin. Our study contributes to the existing literature surrounding spin. Spin in randomized controlled trials is well documented across several fields of medicine, including otolaryngology,¹⁰ obesity medicine,¹² dermatology,²¹ anesthesiology,²² psychiatry,²³ orthopedics,²⁴ emergency medicine,²⁵ oncology,²⁶ and cardiology.²⁷ More recently, studies have emerged evaluating the presence of spin in systematic reviews. Specific to dermatology, one study found that 74% (84/113) of systematic reviews related to atopic dermatitis treatment contained spin.²⁸ Additionally, Ottwell et al¹³ identified spin in 31% (11/36) of the systematic reviews related to the treatment of acne vulgaris, which is similar to our results for systematic reviews focused on psoriasis treatments. When comparing the presence of spin in abstracts of systematic reviews from the field of dermatology with other specialties, dermatology-focused systematic reviews appear to contain more spin in the abstract than systematic reviews focused on tinnitus and glaucoma therapies.^29,30 However, systematic reviews from the field of dermatology appear to contain less spin than systematic reviews focused on therapies for lower back pain.³¹ For example, Nascimento et al³¹ found that 80% (53/66) of systematic reviews focused on low-back pain treatments contained spin.

Examples of Spin—The most common type of spin found in our study was type 6.⁹ An example of spin type 6 can be found in an article by Bai et al³² that investigated the short-term efficacy and safety of multiple interleukin inhibitors for the treatment of plaque psoriasis. The conclusion of the abstract states, “Risankizumab appeared to have relatively high efficacy and low risk.” However, in the results section, the authors showed that risankizumab had the highest risk of serious adverse events and was ranked highest for discontinuation because of adverse events when compared with other interleukin inhibitors. Here, the presence of spin in the abstract may mislead the reader to accept the “low risk” of risankizumab without understanding the study’s full results.³²

Another example of selective reporting of harm outcomes in a systematic review can be found in the article by Wu et al,³³ which focused on assessing IL-17 antagonists for the treatment of plaque psoriasis. The conclusion of the abstract indicated that IL-17 antagonists should be accepted as safe; however, in the results section, the authors discussed serious safety concerns with brodalumab, including the death of 4 patients from suicide.³³ This example of spin type 6 highlights how the overgeneralization of a drug’s safety profile neglects serious harm outcomes that are critical to patient safety. In fact, against the safety claims of Wu et al,³³ brodalumab later received a boxed warning from the US Food and Drug Administration after 6 patients died from suicide while receiving the drug, which led to early discontinuation of the trials.^34,35 Although studies suggest this relationship is not causal,^34-36 the purpose of our study was not to investigate this association but to highlight the importance of this finding. Thus, with this example of spin in mind, we offer recommendations that we believe will improve reporting in abstracts as well as quality of patient care.

Recommendations for Reporting in Abstracts—Regarding the boxed warning³⁷ for brodalumab because of suicidal ideation and behavior, the US Food and Drug Administration recommends that prior to prescribing brodalumab, clinicians consider the potential benefits and risks in patients with a history of depression and/or suicidal ideation or behavior. However, a clinician would not adequately assess the full risks and benefits when an abstract, such as that for the article by Wu et al,³³ contains spin through selectively reporting harm outcomes. Arguably, clinicians could just read the full text; however, research confirms that abstracts often are utilized by clinicians and commonly are used to guide clinical decisions.^7,38 It is reasonable that clinicians would use abstracts in this fashion because they provide a quick synopsis of the full article’s findings and are widely available to clinicians who may not have access to article databases. Initiatives are in place to improve the quality of reporting in an abstract, such as PRISMA-A,²⁰ but even this fails to address spin. In fact, it may suggest spin because checklist item 10 of PRISMA-A advises authors of systematic reviews to provide a “general interpretation of the results and important implications.” This item is concerning because it suggests that the authors interpret importance rather than the clinician who prescribes the drug and is ultimately responsible for patient safety. Therefore, we recommend a reform to abstract reporting and an update to PRISMA-A that leads authors to report all benefits and risks encountered instead of reporting what the authors define as important.

Strengths and Limitations—Our study has several strengths as well as limitations. One of these strengths is that our protocol was strictly adhered to; any deviations were noted and added as an amendment. Our protocol, data, and all study artifacts were made freely available online on the Open Science Framework to strengthen reproducibility (https://osf.io/zrxh8/). Investigators underwent training to ensure comprehension of spin and systematic review designs. All data were extracted in masked duplicate fashion per the Cochrane Handbook for Systematic Reviews of Interventions.³⁹

Regarding limitations, only 2 databases were searched—MEDLINE and Embase. Therefore, our screening process may not have included every available systematic review on the treatment of psoriasis. Journal impact factors may be inaccurate for the systematic reviews that were published earlier in our data date range; however, we attempted to negate this limitation by using a 5-year average. Our study characteristic regarding PRISMA adherence did not account for studies published before the PRISMA statement release; we also could not access prior submission guidelines to determine when a journal began recommending PRISMA adherence. Another limitation of our study was the intrinsic subjectivity behind spin. Some may disagree with our classifications. Finally, our cross-sectional design should not be generalized to study types that are not systematic reviews or published in other journals during different periods.

Conclusion

Evidence of spin was present in many of the abstracts of systematic reviews pertaining to the treatment of psoriasis. Future clinical research should investigate any reporting of spin and search for ways to better reduce spin within literature. Continued research is necessary to evaluate the presence of spin within dermatology and other specialties.

Psoriasis is an inflammatory autoimmune skin condition that affects approximately 125 million individuals worldwide, with approximately 8 million patients in the United States.¹ Psoriasis not only involves a cosmetic component but also comprises other comorbidities, such as psoriatic arthritis, cardiovascular disease, and psychiatric disorders, that can influence patient quality of life.^2-4 In addition, the costs associated with psoriasis are substantial, with an estimated economic burden of $35.2 billion in the United States in 2015.⁵ Given the prevalence of psoriasis and its many effects on patients, it is important that providers have high-quality evidence regarding efficacious treatment options.

Systematic reviews, which compile all available evidence on a subject to answer a specific question, represent the gold standard of research.⁶ However, studies have demonstrated that when referencing research literature, physicians tend to read only the abstract of a study rather than the entire article.^7,8 A study by Marcelo et al⁸ showed that residents at a tertiary care center answered clinical questions using only the abstract of a paper 69% of the time. Based on these findings, it is imperative that the results of systematic reviews be accurately reported in their abstracts because they can influence patient care.

Referencing only the abstracts of systematic reviews can be problematic if the abstract contains spin. Spin is a form of reporting that inappropriately highlights the benefits of a treatment with greater emphasis than what is shown by the results.⁹ Research has identified the presence of spin in the abstracts of randomized controlled trials.^10-12 For example, Cooper et al¹⁰ found that 70% (33/47) of abstracts in otolaryngology randomized controlled trials contained spin. Additionally, Arthur et al¹¹ and Austin et al¹² had similar findings within abstracts of orthopedic and obesity trials, where 44.8% (112/250) and 46.7% (21/45) contained spin, respectively. Ottwell et al¹³ found that the presence of spin in abstracts is not limited to randomized controlled trials; they demonstrated that the abstracts of nearly one-third (31% [11/36]) of systematic reviews focused on the treatment of acne vulgaris contained spin.

In our study, we aimed to evaluate the presence of spin in the abstracts of systematic reviews focused on the treatment of psoriasis.

Methods

Reproducibility and Reporting—Our study did not meet the regulatory definition for human subjects research per the US Code of Federal Regulations because the study did not involve human research subjects. The study also was not subject to review by the institutional review board. Our protocol, data set, analysis scripts, extraction forms, and other material related to the study have been placed on Open Science Framework to provide transparency and ensure reproducibility. To further allow for analytic reproducibility, our data set was given to an independent laboratory and reanalyzed with a masked approach. Our study was carried out alongside other studies assessing spin in systematic reviews regarding different specialties and disease states. Because these studies were similar in design, this methodology also has been reported elsewhere. The Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA)¹⁴ and the guidelines for meta-epidemiological studies developed by Murad and Wang¹⁵ were used in drafting this article.

Search Strategy—The search strategies for the MEDLINE (Ovid) and Embase (Ovid) databases were created by a systematic review librarian (D.N.W.) to identify systematic reviews and meta-analyses regarding treatments for psoriasis (Figure 1). The searches were performed on June 2, 2020, and uploaded to Rayyan, a systematic review screening platform.¹⁶ After duplicates were removed, the records were screened for eligibility by 2 authors (C.H. and A.L.) using the titles and abstracts. Screening was conducted independently while each of these authors was masked to the other’s results; disagreements were resolved through discussion.

Eligibility Criteria—An article had to meet the following criteria for inclusion in our study: (1) be a systematic review with or without a meta-analysis; (2) relate to the treatment of psoriasis; and (3) be written in English and include human patients only. The PRISMA definition of systematic reviews and meta-analyses was applied.¹⁷

 

Training—Various training occurred throughout our study to ensure understanding of each step and mitigate subjectivity. Before beginning screening, 2 investigators (C.H. and A.L.) completed the Introduction to Systematic Review and Meta-Analysis course offered by Johns Hopkins University.¹⁸ They also underwent 2 days of online and in-person training on the definition and interpretation of the 9 most severe types of spin found in the abstracts of systematic reviews as defined by Yavchitz et al.⁹ Finally, they were trained to use A MeaSurement Tool to Assess systematic Reviews (AMSTAR-2) to appraise the methodological quality of each systematic review. Our protocol contained an outline of all training modules used.

Data Extraction—The investigators (C.H. and A.L.) analyzed included abstracts for the 9 most severe types of spin (Table 1). Data were extracted in a masked duplicate fashion using the Google form. AMSTAR-2 was used to assess systematic reviews for methodological quality. AMSTAR-2 is an appraisal tool consisting of a 16-item checklist for systematic reviews or meta-analyses. Scores range from critically low to high based on the methodological quality of the review. Interrater reliability of AMSTAR-2 scores has been moderate to high across studies. Construct validity coefficients have been high with the original AMSTAR instrument (r=0.91) and the Risk of Bias in Systematic Reviews instrument (r=0.84).¹⁹

During data extraction from each included systematic review, the following additional items were obtained: (1) the date the review was received; (2) intervention type (ie, pharmacologic, nonpharmacologic, surgery, light therapy, mixed); (3) the funding source(s) for each systematic review (ie, industry, private, public, none, not mentioned, hospital, a combination of funding not including industry, a combination of funding including industry, other); (4) whether the journal submission guidelines suggested adherence to PRISMA guidelines; (5) whether the review discussed adherence to PRISMA¹⁴ or PRISMA for Abstracts²⁰ (PRISMA-A); (6) the publishing journal’s 5-year impact factor; and (6) the country of the systematic review’s origin. When data extraction was complete, investigators (C.H. and A.L.) were unmasked and met to resolve any disagreements by discussion. Two authors (R.O. or M.V.) served as arbiters in the case that an agreement between C.H. and A.L. could not be reached.

Statistical Analysis—Frequencies and percentages were calculated to evaluate the most common types of spin found within systematic reviews and meta-analyses. One author (M.H.) prespecified the possibility of a binary logistic regression and calculated a power analysis to determine sample size, as stated in our protocol. Our final sample size of 173 was not powered to perform the multivariable logistic regression; therefore, we calculated unadjusted odds ratios to enable assessing relationships between the presence of spin in abstracts and the various study characteristics. We used Stata 16.1 for all analyses, and all analytic decisions can be found in our protocol.

Results

General Characteristics—Our systematic search of MEDLINE and Embase returned 3200 articles, of which 665 were duplicates that were removed. An additional 2253 articles were excluded during initial abstract and title screening, and full-text screening led to the exclusion of another 109 articles. In total, 173 systematic reviews were included for data extraction. Figure 2 illustrates the screening process with the rationale for all exclusions.

Of the 173 included systematic reviews and meta-analyses, 150 (86.7%) focused on pharmacologic interventions. The majority of studies did not mention adhering to PRISMA guidelines (125/173 [72.3%]), and the publishing journals recommended their authors adhere to PRISMA for only 66 (38.2%) of the included articles. For the articles that received funding (90/173 [52.0%]), industry sources were the most common funding source (40/90 [44.4%]), followed by private (27/90 [30%]) and public funding sources (23/90 [25.6%]). Of the remaining studies, 46 articles did not include a funding statement (46/83 [55.4%]), and 37 studies were not funded (37/83 [44.6%]). The average (SD) 5-year impact factor of our included journals was 4.68 (4.64). Systematic reviews were from 31 different countries. All studies were received by their respective journals between the years 2000 and 2020 (Table 2).

Abstracts Containing Spin—We found that 37 (21.4%) of the abstracts of systematic reviews focused on psoriasis treatments contained at least 1 type of spin. Some abstracts had more than 1 type; thus, a total of 51 different instances of spin were detected. Spin type 6—selective reporting of or overemphasis on harm outcomes or analysis favoring the safety of the experimental intervention—was the most common type ofspin, found in 19 of 173 abstracts (11.0%). The most severe type of spin—type 1 (conclusion contains recommendations for clinical practice not supported by the findings)—occurred in only 1 abstract (0.6%). Spin type 8 did not occur in any of the abstracts (Table 1). There was no statistically significant association between the presence of spin and any of the study characteristics (Table 2).

 

AMSTAR Ratings—After using AMSTAR-2 to appraise the included systematic reviews, we found that 6 (3.5%) of the 173 studies could be rated as high; 36 (20.8%) as moderate; 25 (14.5%) as low; and 106 (61.3%) as critically low. Of the 37 abstracts containing spin, 2 (5.4%) had an AMSTAR-2 rating of high, 10 (27%) had a rating of moderate, 6 (16.2%) had a rating of low, and 19 (51.4%) had a rating of critically low (Table 2). No statistically significant associations were seen between abstracts found to have spin and the AMSTAR-2 rating of the review.

Nearly all (160/173 [92.5%]) of the included reviews were compliant with the inclusion of Population, Intervention, Comparison, and Outcome (PICO) method. Only 17 of 173 (9.8%) reviews reported funding sources for the studies included. See Table 3 for all AMSTAR-2 items.

Comment

Primary Findings—We evaluated the abstracts of systematic reviews for the treatment of psoriasis and found that more than one-fifth of them contained spin. Our study contributes to the existing literature surrounding spin. Spin in randomized controlled trials is well documented across several fields of medicine, including otolaryngology,¹⁰ obesity medicine,¹² dermatology,²¹ anesthesiology,²² psychiatry,²³ orthopedics,²⁴ emergency medicine,²⁵ oncology,²⁶ and cardiology.²⁷ More recently, studies have emerged evaluating the presence of spin in systematic reviews. Specific to dermatology, one study found that 74% (84/113) of systematic reviews related to atopic dermatitis treatment contained spin.²⁸ Additionally, Ottwell et al¹³ identified spin in 31% (11/36) of the systematic reviews related to the treatment of acne vulgaris, which is similar to our results for systematic reviews focused on psoriasis treatments. When comparing the presence of spin in abstracts of systematic reviews from the field of dermatology with other specialties, dermatology-focused systematic reviews appear to contain more spin in the abstract than systematic reviews focused on tinnitus and glaucoma therapies.^29,30 However, systematic reviews from the field of dermatology appear to contain less spin than systematic reviews focused on therapies for lower back pain.³¹ For example, Nascimento et al³¹ found that 80% (53/66) of systematic reviews focused on low-back pain treatments contained spin.

Examples of Spin—The most common type of spin found in our study was type 6.⁹ An example of spin type 6 can be found in an article by Bai et al³² that investigated the short-term efficacy and safety of multiple interleukin inhibitors for the treatment of plaque psoriasis. The conclusion of the abstract states, “Risankizumab appeared to have relatively high efficacy and low risk.” However, in the results section, the authors showed that risankizumab had the highest risk of serious adverse events and was ranked highest for discontinuation because of adverse events when compared with other interleukin inhibitors. Here, the presence of spin in the abstract may mislead the reader to accept the “low risk” of risankizumab without understanding the study’s full results.³²

Another example of selective reporting of harm outcomes in a systematic review can be found in the article by Wu et al,³³ which focused on assessing IL-17 antagonists for the treatment of plaque psoriasis. The conclusion of the abstract indicated that IL-17 antagonists should be accepted as safe; however, in the results section, the authors discussed serious safety concerns with brodalumab, including the death of 4 patients from suicide.³³ This example of spin type 6 highlights how the overgeneralization of a drug’s safety profile neglects serious harm outcomes that are critical to patient safety. In fact, against the safety claims of Wu et al,³³ brodalumab later received a boxed warning from the US Food and Drug Administration after 6 patients died from suicide while receiving the drug, which led to early discontinuation of the trials.^34,35 Although studies suggest this relationship is not causal,^34-36 the purpose of our study was not to investigate this association but to highlight the importance of this finding. Thus, with this example of spin in mind, we offer recommendations that we believe will improve reporting in abstracts as well as quality of patient care.

Recommendations for Reporting in Abstracts—Regarding the boxed warning³⁷ for brodalumab because of suicidal ideation and behavior, the US Food and Drug Administration recommends that prior to prescribing brodalumab, clinicians consider the potential benefits and risks in patients with a history of depression and/or suicidal ideation or behavior. However, a clinician would not adequately assess the full risks and benefits when an abstract, such as that for the article by Wu et al,³³ contains spin through selectively reporting harm outcomes. Arguably, clinicians could just read the full text; however, research confirms that abstracts often are utilized by clinicians and commonly are used to guide clinical decisions.^7,38 It is reasonable that clinicians would use abstracts in this fashion because they provide a quick synopsis of the full article’s findings and are widely available to clinicians who may not have access to article databases. Initiatives are in place to improve the quality of reporting in an abstract, such as PRISMA-A,²⁰ but even this fails to address spin. In fact, it may suggest spin because checklist item 10 of PRISMA-A advises authors of systematic reviews to provide a “general interpretation of the results and important implications.” This item is concerning because it suggests that the authors interpret importance rather than the clinician who prescribes the drug and is ultimately responsible for patient safety. Therefore, we recommend a reform to abstract reporting and an update to PRISMA-A that leads authors to report all benefits and risks encountered instead of reporting what the authors define as important.

Strengths and Limitations—Our study has several strengths as well as limitations. One of these strengths is that our protocol was strictly adhered to; any deviations were noted and added as an amendment. Our protocol, data, and all study artifacts were made freely available online on the Open Science Framework to strengthen reproducibility (https://osf.io/zrxh8/). Investigators underwent training to ensure comprehension of spin and systematic review designs. All data were extracted in masked duplicate fashion per the Cochrane Handbook for Systematic Reviews of Interventions.³⁹

Regarding limitations, only 2 databases were searched—MEDLINE and Embase. Therefore, our screening process may not have included every available systematic review on the treatment of psoriasis. Journal impact factors may be inaccurate for the systematic reviews that were published earlier in our data date range; however, we attempted to negate this limitation by using a 5-year average. Our study characteristic regarding PRISMA adherence did not account for studies published before the PRISMA statement release; we also could not access prior submission guidelines to determine when a journal began recommending PRISMA adherence. Another limitation of our study was the intrinsic subjectivity behind spin. Some may disagree with our classifications. Finally, our cross-sectional design should not be generalized to study types that are not systematic reviews or published in other journals during different periods.

Conclusion

Evidence of spin was present in many of the abstracts of systematic reviews pertaining to the treatment of psoriasis. Future clinical research should investigate any reporting of spin and search for ways to better reduce spin within literature. Continued research is necessary to evaluate the presence of spin within dermatology and other specialties.

Sleep apnea is a heterogeneous group of conditions that may be attributable to a wide array of underlying conditions, with varying contributions of obstructive or central sleep-disordered breathing. The spectrum from obstructive sleep apnea (OSA) to central sleep apnea (CSA) includes mixed sleep apnea, treatment-emergent CSA (TECSA), and Cheyne-Stokes respiration (CSR).¹ The pathophysiologic causes of CSA can be attributed to delayed cardiopulmonary circulation in heart failure, decreased brainstem ventilatory response due to stroke, blunting of central chemoreceptors in chronic opioid use, and/or stimulation of the Hering-Breuer reflex from activation of pulmonary stretch receptors after initiating positive airway pressure (PAP) for treatment of OSA.^2,3 Medications are commonly implicated in many forms of sleep-disordered breathing; importantly, opioids and benzodiazepines may blunt the respiratory drive, leading to CSA, and/or impair upper airway patency, resulting in or worsening OSA.

Continuous positive airway pressure (CPAP) therapy is largely ineffective in correcting CSA or improving outcomes and is often poorly tolerated in these patients.⁴ Adaptive servo-ventilation (ASV) is a form of bilevel PAP (BPAP) therapy that delivers variable adjusting pressure support, primarily to treat CSA. PAP also may relieve upper airway obstructions, thereby effectively treating any comorbid obstructive component. ASV has been well documented to improve sleep-related disorders and improve apnea-hypopnea index (AHI) in patients with CSA. However, longitudinal data have demonstrated increased mortality in patients with heart failure with reduced ejection fraction (HFrEF) who were treated with ASV.⁵ Since the SERVE-HF trial results came to light in 2015, there has been no consensus regarding the optimal use, if any, of ASV therapy.^6-8 This is partly related to the inability to fully explain the study’s major findings, which were unexpected at the time, and partly due to the absence of similar relevant mortality data in patients with CSA but without HFrEF.

TECSA may present in some patients with OSA who are new to PAP therapy. These events are frequently seen during PAP titration sleep studies, though patients can also experience significant TECSA shortly after initiating home PAP therapy. TECSA is felt to result from a combination of stimulating pulmonary stretch receptors and lowering arterial carbon dioxide below the apneic threshold. Chemoreceptors located in the medulla respond by attenuating the respiratory drive.⁹ Previous studies have shown most cases of mild TECSA resolve over time with CPAP treatment. However, in patients with persistent or worsening TECSA, ASV may be considered as an alternative to CPAP.

The prevalence of OSA in the veteran population is estimated to be as high as 60%, considerably higher than the general population estimation.¹⁰ Patients with more significant comorbidities may also experience a higher frequency of central events. Patients with CSA have also been shown to have a higher risk for cardiac-related hospital admissions, providing plausible justification for correcting CSA.¹⁰

In the current study, we aim to characterize the group of patients using ASV therapy in the modern era. We will assess the objective efficacy and adherence of ASV therapy in patients with primarily CSA compared with those having primarily OSA (ie, TECSA). Secondarily, we aim to identify baseline clinical and polysomnographic features that may be predictive of ASV adherence, as a surrogate for subjective benefit.¹¹ In the wake of the SERVE-HF study, the sleep medicine community has paused prescribing ASV therapy for CSA. We hope to provide more perspective on the treatment of veterans with CSA and identify the patient groups that would benefit most from ASV therapy.

 

Methods

This retrospective chart review examined patients prescribed ASV therapy at the Hampton Veterans Affairs Medical Center (HVAMC) in Virginia who had therapy data between January 1, 2015, and April 30, 2020. The start date was chosen to approximate the phase-in of wireless PAP devices at HVAMC and to correspond with the release of preliminary results from the SERVE-HF trial.

Patients were initially identified through a query into commercial wireless PAP management databases and cross-referenced with HVAMC patients. Adherence and efficacy data were obtained from the most recent clinical PAP data, which allowed for the evaluation of patients who discontinued therapy for reasons other than intolerance. Clinical, demographic, and polysomnography (PSG) data were obtained from the electronic health record. One patient, identified through the database query but not found in the electronic health record, was excluded. In cases of missing PSG data, especially AHI or similar values, all attempts were made to calculate the data with other provided values. This study was determined to be exempt by the HVAMC Institutional Review Board (protocol #20-01).

Statistics

Statistical analyses were designed to compare clinical characteristics and adherence to therapy of those with primarily CSA on PSG and those with primarily OSA. Because it was not currently known how many patients would fit into each of these categories, we also planned secondary comparisons of the clinical and PSG characteristics of those patients who were adherent with therapy and those who were not. Adherence with ASV therapy was defined as device use for ≥ 4 hours for ≥ 70% of nights.

Comparisons between the means of 2 normally distributed groups were performed with an unpaired t test. Comparisons between 2 nonnormally distributed groups and groups of dates were done with the Mann-Whitney U test. The normality of a group distribution was determined using D’Agostino-Pearson omnibus normality test. Two groups of dichotomous variables were compared with the Fisher exact test. P value < .05 was considered statistically significant.

Results

Thirty-one patients were prescribed ASV therapy and had follow-up at HVAMC since 2015. All patients were male. The mean (SD) age was 67.2 (11.4) years, mean body mass index (BMI) was 34.0 (5.9), and the mean (SD) Epworth Sleepiness Scale (ESS) score was 10.9 (5.8). Patient comorbidities included 30 (97%) with hypertension, 17 (55%) with diabetes mellitus, 16 (52%) with coronary artery disease, and 11 (35%) with congestive heart failure. Three patients had no echocardiogram or other documentation of left ventricular ejection fraction (LVEF). One of these patients had voluntarily stopped using PAP therapy, another had been erroneously started on ASV (ordered for fixed BPAP), and the third had since been retitrated to CPAP. In the 28 patients with documented LVEF, the mean (SD) LVEF was 61.8% (6.9). Ten patients (32%) had opioids documented on their medication lists and 6 (19%) had benzodiazepines.

The median date of diagnostic sleep testing was January 9, 2015, and testing was completed after the release of the initial field safety notice regarding the SERVE-HF trial preliminary findings May 13, 2015, for 14 patients (45%).¹² On diagnostic sleep testing, the mean (SD) AHI was 47.3 (25.6) events/h and the median (IQR) oxygen saturation (SpO₂) nadir was 82% (78-84). Three patients (10%) were initially diagnosed with CSA, 19 (61%) with OSA, and 9 (29%) with both. Sixteen patients (52%) had ASV with fixed expiratory PAP (EPAP), and 15 (48%) had variable adjusting EPAP. Mean (SD) usage of ASV was 6.5 (2.6) hours and 66.0% (34.2) of nights for ≥ 4 hours. Mean (SD) titrated EPAP (set or 90th/95th percentile autotitrated) was 10.1 (3.4) cm H₂O and inspiratory PAP (IPAP) (90th/95th percentile) was 17.1 (3.3) cm H₂O. The median (IQR) residual AHI on ASV was 2.7 events/h (1.1-5.1), apnea index (AI) was 0.4 (0.1-1.0), and hypopnea index (HI) was 1.4 (1.0-3.2); the residual central and obstructive events were not available in most cases.

 

Adherence

Seventeen patients (55%) met the minimum adherence criteria of ≥ 4 hours of usage for ≥ 70% of the nights. There were no significant differences in age, BMI, sex, race, comorbidities, medications, or ESS when comparing patients who were adherent to ASC and those who were not (Table 1). The date of diagnostic sleep testing and sleep architecture, including sleep latency, total sleep time, sleep efficiency, wake after sleep onset, arousal index, and percentage of rapid eye movement stage sleep were similar between the adherent patients and nonadherent patients (Table 2).

The overall AHI mean (SD) on diagnostic PSG were also similar between the adherent group (52.3 [24.8] events/h) and nonadherent group (45.1 [27.0] events/h) (P = .47). The mean (SD) for obstructive AHI (obstructive apneas, mixed apneas, obstructive hypopneas, and undifferentiated hypopneas per hour of sleep) were higher in the adherent group as a percentage of the total AHI: 81.5% (27.9) in the adherent group vs 46.7% (38.4%) in the nonadherent group; P = .02) (Figure 1).

This difference was primarily driven by a difference in mean (SD) HI: 29.7 (16.5) in the adherent group vs. 15.3 (12.1) in the nonadherent group (P = .04).

There were no significant differences between the proportions of patients on ASV with set EPAP or the titrated EPAP and IPAP. The median (IQR) residual AHI was lower in the adherent group compared with the nonadherent group, both in absolute values (1.7 [0.9-3.2] events/h vs 4.7 [2.4-10.3] events/h, respectively [P = .004]), and as a percentage of the pretreatment AHI (3.1% [2.5-6.0] vs 10.2% [5.3-34.4], respectively; P = .002) (Figure 2).

Primarily Obstructive Sleep Apnea

Sleep apnea was a mixed picture of obstructive and central events in many patients. Only 3 patients had “pure” CSA. Thus, we were unable to define discrete comparison groups based on the sleep-disordered breathing phenotype. We identified 19 patients with primarily OSA (ie, initially diagnosed with OSA, OSA with TECSA, or complex sleep apnea). The mean (SD) age was 66.1 (12.8) years, BMI was 36.2 (4.7), and ESS was 11.4 (5.6). The mean (SD) baseline AHI was 46.9 (29.5), obstructive AHI was 40.5 (30.4), and central AHI was 0.4 (1.2); the median (IQR) SpO₂ nadir was 81% (78%-84%). The mean (SD) titrated EPAP was 10.2 (3.5) cm H₂O, and the 90th/95th percentile IPAP was 17.9 (3.5) cm H₂O. The mean (SD) usage of ASV was 7.9 (5.3) hours with 11 patients (58%) meeting the minimum standard for adherence to ASV therapy.

No significant differences were seen between the adherent and nonadherent groups in clinical or demographic characteristics or date of diagnostic sleep testing (eAppendix, available online at doi:10.12788/fp.0374). In baseline sleep studies the mean (SD) HI was 32.3 (15.8) in the adherent group compared with 14.7 (8.8) in the nonadherent group (P = .049). In contrast, obstructive AHI was not significantly lower in the adherent group: 51.9 (30.9) in the adherent group compared with 22.2 (20.6) in the nonadherent group (P = .09). The median (IQR) residual AHI on ASV as a percentage of the pretreatment AHI was 3.0% (2.4%-6.5%) in the adherent group compared with 11.3% (5.4%-89.1%) in the nonadherent group, a statistically significant difference (P = .01). No other significant differences were seen between the groups.

 

Discussion

This study describes a real-world cohort of patients using ASV therapy and the characteristics associated with benefit from therapy. The patients that were prescribed and started ASV therapy most often had a significant degree of obstructive component to sleep-disordered breathing, whether primary OSA with TECSA or comorbid OSA and CSA. Moreover, we found that a higher obstructive AHI on the baseline PSG was associated with adherence to ASV therapy. Another important finding was that a lower residual AHI on ASV as a proportion of the baseline was associated with PAP adherence. Adherent patients had similar clinical characteristics as the nonadherent patients, including comorbidities, severity of sleep-disordered breathing, and obesity.

Though the results of the SERVE-HF trial have dampened the enthusiasm somewhat, ASV therapy has long been considered an effective and well-tolerated treatment for many types of CSA.¹³ In fact, treatments that can eliminate the central AHI are fairly limited.^4,14 Our data suggest that ASV is also effective and tolerated in OSA with TECSA and/or comorbid CSA. Recent studies suggest that CSA resolves spontaneously in a majority of TECSA patients within 2 to 3 months of regular CPAP use.¹⁵ Other estimates suggest that persistent TESCA may be present in 2% of patients with OSA on treatment.¹⁶

Given the high and rising prevalence of OSA, many people are at risk for clinically significant TESCA. Another retrospective case series found that 72% of patients that failed treatment with CPAP or BPAP during PSG, met diagnostic criteria (at the time) for CSA; ASV was objectively beneficial in these patients.¹⁷ ASV can be an especially useful modality to treat OSA in patients with CSA that either prevents tolerance of standard therapies or causes clinical consequences, presuming the patient does not also have HFrEF.¹⁸ The long-term outcomes of treatment with ASV therapy remain a matter of debate.

The SERVE-HF trial remains among the only studies that have assessed the mortality effects of CSA treatments, with unfavorable findings. Treatment of OSA has been associated with favorable chronic health benefits, though recent studies have questioned the degree of benefit attributable to OSA treatment.^19-24 Similar studies have not been done for comorbidities represented by our study cohort (ie, OSA with TECSA and/or comorbid CSA).

The lack of CSA diagnosis alone in our cohort may be partially attributable to changing practice patterns following the SERVE-HF trial, though it is not clear from these data why a higher baseline obstructive AHI was associated with adherence to ASV therapy. Our data in this regard are somewhat at odds with the preliminary results of the ADVENT-HF trial. In that study, adherence to ASV therapy in patients with predominantly OSA declined significantly more than in patients with predominantly CSA.²⁵ Most of our patients were diagnosed with predominantly OSA, so a direct comparison with the CSA group is problematic; additionally, the primary brand and the pressure adjustments algorithm used in our study differed from the ADVENT-HF trial.

OSA and CSA may present with similar clinical symptoms, including sleep fragmentation, insomnia, and excessive daytime sleepiness; however, the degree of symptomatology, especially daytime sleepiness, and the response to treatment, may be less in CSA.^2,26 Both the subjective report of symptoms (ESS) and PSG measures of sleep fragmentation were similar in our patients, again likely explained by the predominance of obstructive events.

The pathophysiology of CSA is more varied than OSA, which is probably relevant in this case. ASV was originally designed for the management of CSA with CSR, accomplishing this goal by stabilizing the periods of central apnea and hyperpnea characteristic of CSR.²⁷ Although other forms of CSA demonstrate breathing patterns distinct from CSR, ASV has become an accepted treatment for most of these. It is plausible that the long-term subjective benefit and tolerance of ASV in CSA without CSR is less than for CSA with CSR or OSA. None of the patients in our study had CSA with CSR.

Ultimately, it may be the objective treatment effect that lends to adherence, as has been shown previously in OSA patients; our group of adherent patients showed a greater improvement in AHI, relative to baseline, than the nonadherent patients did.²⁸ The technology behind ASV therapy can greatly reduce the frequencies of central apneas, yet this same treatment effectively splints the upper airway and even more effectively eliminates obstructive apneas and hypopneas. Variable adjusting EPAP devices would plausibly provide even more benefit in these patients, as has been shown in prior studies.²⁹ To the contrary, our small sample of patients with TESCA showed a nonsignificant trend toward adherence with fixed EPAP ASV.

Opioid use was substantial in our population, without significant differences between the groups. CPAP therapy is ineffective in improving opioid-associated CSA. In a recent study, 20 patients on opioid therapy with CSA were treated with CPAP therapy; after several weeks, the average therapeutic use was 4 to 5 hours per night and CPAP was abandoned in favor of ASV therapy due to persistent central apnea. ASV treatment was associated with a considerable reduction in central apnea index, AHI, arousal index, and oxygen desaturations in a remarkable improvement over CPAP.³⁰

 

Limitations and Future Directions

This retrospective, single-center study may have limited applicability to other populations. Adherence was used as a surrogate for subjective benefit from treatment, though benefit was not confirmed by the patients directly. Only patients seen in follow-up for documentation of the ASV download were identified for inclusion and data analysis. As a single center, we risk homogeneity in the treatment algorithms, though sleep medicine treatments are often decided at the time of the sleep studies. Studies and treatment recommendations were made at a variety of sites, including our sleep center, other US Department of Veterans Affairs hospitals, in the community network, and at US Department of Defense centers. Our population was homogenous in some ways; notably, 100% of our group was male, which is substantially higher than both the veteran population and the general population. Risk factors for OSA and CSA are more common in male patients, which may partially explain this anomaly. Lastly, with our small sample size, there is increased risk that the results seen occurred by chance.

There are several areas for further study. A larger multicenter study may permit these results to be generalized to the population and should include subjective measures of benefit. Patients with primarily CSA were largely absent in our group and may be the focus of future studies; data on predictors of treatment adherence in CSA are lacking. With the availability of consistent older adherence data, comparisons may be made between the efficacies of clinical practice habits, including treatment efficacy, before and after the results of the SERVE-HF trial became known.

Conclusions

In selected patients with preserved LVEF, ASV therapy appears especially effective in patients with OSA combined with CSA. Adherence to ASV treatment was associated with higher obstructive AHI during the baseline PSG and with a greater reduction in the AHI. This understanding may help guide sleep specialists in personalizing treatments for sleep-disordered breathing. Because objective efficacy appears to be important for therapy adherence, clinicians should be able to consistently determine the obstructive and central components of the residual AHI, thus taking all information into account when optimizing the treatment. Additionally, both OSA and CSA pressure requirements should be considered when developing ASV devices.

Acknowledgments

We thank Martha Harper, RRT, of Hampton Veterans Affairs Medical Center (HVAMC) for helping to identify our patients and assisting with data collection. This material is the result of work supported with resources and the use of HVAMC facilities.

Sleep apnea is a heterogeneous group of conditions that may be attributable to a wide array of underlying conditions, with varying contributions of obstructive or central sleep-disordered breathing. The spectrum from obstructive sleep apnea (OSA) to central sleep apnea (CSA) includes mixed sleep apnea, treatment-emergent CSA (TECSA), and Cheyne-Stokes respiration (CSR).¹ The pathophysiologic causes of CSA can be attributed to delayed cardiopulmonary circulation in heart failure, decreased brainstem ventilatory response due to stroke, blunting of central chemoreceptors in chronic opioid use, and/or stimulation of the Hering-Breuer reflex from activation of pulmonary stretch receptors after initiating positive airway pressure (PAP) for treatment of OSA.^2,3 Medications are commonly implicated in many forms of sleep-disordered breathing; importantly, opioids and benzodiazepines may blunt the respiratory drive, leading to CSA, and/or impair upper airway patency, resulting in or worsening OSA.

Continuous positive airway pressure (CPAP) therapy is largely ineffective in correcting CSA or improving outcomes and is often poorly tolerated in these patients.⁴ Adaptive servo-ventilation (ASV) is a form of bilevel PAP (BPAP) therapy that delivers variable adjusting pressure support, primarily to treat CSA. PAP also may relieve upper airway obstructions, thereby effectively treating any comorbid obstructive component. ASV has been well documented to improve sleep-related disorders and improve apnea-hypopnea index (AHI) in patients with CSA. However, longitudinal data have demonstrated increased mortality in patients with heart failure with reduced ejection fraction (HFrEF) who were treated with ASV.⁵ Since the SERVE-HF trial results came to light in 2015, there has been no consensus regarding the optimal use, if any, of ASV therapy.^6-8 This is partly related to the inability to fully explain the study’s major findings, which were unexpected at the time, and partly due to the absence of similar relevant mortality data in patients with CSA but without HFrEF.

TECSA may present in some patients with OSA who are new to PAP therapy. These events are frequently seen during PAP titration sleep studies, though patients can also experience significant TECSA shortly after initiating home PAP therapy. TECSA is felt to result from a combination of stimulating pulmonary stretch receptors and lowering arterial carbon dioxide below the apneic threshold. Chemoreceptors located in the medulla respond by attenuating the respiratory drive.⁹ Previous studies have shown most cases of mild TECSA resolve over time with CPAP treatment. However, in patients with persistent or worsening TECSA, ASV may be considered as an alternative to CPAP.

The prevalence of OSA in the veteran population is estimated to be as high as 60%, considerably higher than the general population estimation.¹⁰ Patients with more significant comorbidities may also experience a higher frequency of central events. Patients with CSA have also been shown to have a higher risk for cardiac-related hospital admissions, providing plausible justification for correcting CSA.¹⁰

In the current study, we aim to characterize the group of patients using ASV therapy in the modern era. We will assess the objective efficacy and adherence of ASV therapy in patients with primarily CSA compared with those having primarily OSA (ie, TECSA). Secondarily, we aim to identify baseline clinical and polysomnographic features that may be predictive of ASV adherence, as a surrogate for subjective benefit.¹¹ In the wake of the SERVE-HF study, the sleep medicine community has paused prescribing ASV therapy for CSA. We hope to provide more perspective on the treatment of veterans with CSA and identify the patient groups that would benefit most from ASV therapy.

 

Methods

This retrospective chart review examined patients prescribed ASV therapy at the Hampton Veterans Affairs Medical Center (HVAMC) in Virginia who had therapy data between January 1, 2015, and April 30, 2020. The start date was chosen to approximate the phase-in of wireless PAP devices at HVAMC and to correspond with the release of preliminary results from the SERVE-HF trial.

Patients were initially identified through a query into commercial wireless PAP management databases and cross-referenced with HVAMC patients. Adherence and efficacy data were obtained from the most recent clinical PAP data, which allowed for the evaluation of patients who discontinued therapy for reasons other than intolerance. Clinical, demographic, and polysomnography (PSG) data were obtained from the electronic health record. One patient, identified through the database query but not found in the electronic health record, was excluded. In cases of missing PSG data, especially AHI or similar values, all attempts were made to calculate the data with other provided values. This study was determined to be exempt by the HVAMC Institutional Review Board (protocol #20-01).

Statistics

Statistical analyses were designed to compare clinical characteristics and adherence to therapy of those with primarily CSA on PSG and those with primarily OSA. Because it was not currently known how many patients would fit into each of these categories, we also planned secondary comparisons of the clinical and PSG characteristics of those patients who were adherent with therapy and those who were not. Adherence with ASV therapy was defined as device use for ≥ 4 hours for ≥ 70% of nights.

Comparisons between the means of 2 normally distributed groups were performed with an unpaired t test. Comparisons between 2 nonnormally distributed groups and groups of dates were done with the Mann-Whitney U test. The normality of a group distribution was determined using D’Agostino-Pearson omnibus normality test. Two groups of dichotomous variables were compared with the Fisher exact test. P value < .05 was considered statistically significant.

Results

Thirty-one patients were prescribed ASV therapy and had follow-up at HVAMC since 2015. All patients were male. The mean (SD) age was 67.2 (11.4) years, mean body mass index (BMI) was 34.0 (5.9), and the mean (SD) Epworth Sleepiness Scale (ESS) score was 10.9 (5.8). Patient comorbidities included 30 (97%) with hypertension, 17 (55%) with diabetes mellitus, 16 (52%) with coronary artery disease, and 11 (35%) with congestive heart failure. Three patients had no echocardiogram or other documentation of left ventricular ejection fraction (LVEF). One of these patients had voluntarily stopped using PAP therapy, another had been erroneously started on ASV (ordered for fixed BPAP), and the third had since been retitrated to CPAP. In the 28 patients with documented LVEF, the mean (SD) LVEF was 61.8% (6.9). Ten patients (32%) had opioids documented on their medication lists and 6 (19%) had benzodiazepines.

The median date of diagnostic sleep testing was January 9, 2015, and testing was completed after the release of the initial field safety notice regarding the SERVE-HF trial preliminary findings May 13, 2015, for 14 patients (45%).¹² On diagnostic sleep testing, the mean (SD) AHI was 47.3 (25.6) events/h and the median (IQR) oxygen saturation (SpO₂) nadir was 82% (78-84). Three patients (10%) were initially diagnosed with CSA, 19 (61%) with OSA, and 9 (29%) with both. Sixteen patients (52%) had ASV with fixed expiratory PAP (EPAP), and 15 (48%) had variable adjusting EPAP. Mean (SD) usage of ASV was 6.5 (2.6) hours and 66.0% (34.2) of nights for ≥ 4 hours. Mean (SD) titrated EPAP (set or 90th/95th percentile autotitrated) was 10.1 (3.4) cm H₂O and inspiratory PAP (IPAP) (90th/95th percentile) was 17.1 (3.3) cm H₂O. The median (IQR) residual AHI on ASV was 2.7 events/h (1.1-5.1), apnea index (AI) was 0.4 (0.1-1.0), and hypopnea index (HI) was 1.4 (1.0-3.2); the residual central and obstructive events were not available in most cases.

 

Adherence

Seventeen patients (55%) met the minimum adherence criteria of ≥ 4 hours of usage for ≥ 70% of the nights. There were no significant differences in age, BMI, sex, race, comorbidities, medications, or ESS when comparing patients who were adherent to ASC and those who were not (Table 1). The date of diagnostic sleep testing and sleep architecture, including sleep latency, total sleep time, sleep efficiency, wake after sleep onset, arousal index, and percentage of rapid eye movement stage sleep were similar between the adherent patients and nonadherent patients (Table 2).

The overall AHI mean (SD) on diagnostic PSG were also similar between the adherent group (52.3 [24.8] events/h) and nonadherent group (45.1 [27.0] events/h) (P = .47). The mean (SD) for obstructive AHI (obstructive apneas, mixed apneas, obstructive hypopneas, and undifferentiated hypopneas per hour of sleep) were higher in the adherent group as a percentage of the total AHI: 81.5% (27.9) in the adherent group vs 46.7% (38.4%) in the nonadherent group; P = .02) (Figure 1).

This difference was primarily driven by a difference in mean (SD) HI: 29.7 (16.5) in the adherent group vs. 15.3 (12.1) in the nonadherent group (P = .04).

There were no significant differences between the proportions of patients on ASV with set EPAP or the titrated EPAP and IPAP. The median (IQR) residual AHI was lower in the adherent group compared with the nonadherent group, both in absolute values (1.7 [0.9-3.2] events/h vs 4.7 [2.4-10.3] events/h, respectively [P = .004]), and as a percentage of the pretreatment AHI (3.1% [2.5-6.0] vs 10.2% [5.3-34.4], respectively; P = .002) (Figure 2).

Primarily Obstructive Sleep Apnea

Sleep apnea was a mixed picture of obstructive and central events in many patients. Only 3 patients had “pure” CSA. Thus, we were unable to define discrete comparison groups based on the sleep-disordered breathing phenotype. We identified 19 patients with primarily OSA (ie, initially diagnosed with OSA, OSA with TECSA, or complex sleep apnea). The mean (SD) age was 66.1 (12.8) years, BMI was 36.2 (4.7), and ESS was 11.4 (5.6). The mean (SD) baseline AHI was 46.9 (29.5), obstructive AHI was 40.5 (30.4), and central AHI was 0.4 (1.2); the median (IQR) SpO₂ nadir was 81% (78%-84%). The mean (SD) titrated EPAP was 10.2 (3.5) cm H₂O, and the 90th/95th percentile IPAP was 17.9 (3.5) cm H₂O. The mean (SD) usage of ASV was 7.9 (5.3) hours with 11 patients (58%) meeting the minimum standard for adherence to ASV therapy.

No significant differences were seen between the adherent and nonadherent groups in clinical or demographic characteristics or date of diagnostic sleep testing (eAppendix, available online at doi:10.12788/fp.0374). In baseline sleep studies the mean (SD) HI was 32.3 (15.8) in the adherent group compared with 14.7 (8.8) in the nonadherent group (P = .049). In contrast, obstructive AHI was not significantly lower in the adherent group: 51.9 (30.9) in the adherent group compared with 22.2 (20.6) in the nonadherent group (P = .09). The median (IQR) residual AHI on ASV as a percentage of the pretreatment AHI was 3.0% (2.4%-6.5%) in the adherent group compared with 11.3% (5.4%-89.1%) in the nonadherent group, a statistically significant difference (P = .01). No other significant differences were seen between the groups.

 

Discussion

This study describes a real-world cohort of patients using ASV therapy and the characteristics associated with benefit from therapy. The patients that were prescribed and started ASV therapy most often had a significant degree of obstructive component to sleep-disordered breathing, whether primary OSA with TECSA or comorbid OSA and CSA. Moreover, we found that a higher obstructive AHI on the baseline PSG was associated with adherence to ASV therapy. Another important finding was that a lower residual AHI on ASV as a proportion of the baseline was associated with PAP adherence. Adherent patients had similar clinical characteristics as the nonadherent patients, including comorbidities, severity of sleep-disordered breathing, and obesity.

Though the results of the SERVE-HF trial have dampened the enthusiasm somewhat, ASV therapy has long been considered an effective and well-tolerated treatment for many types of CSA.¹³ In fact, treatments that can eliminate the central AHI are fairly limited.^4,14 Our data suggest that ASV is also effective and tolerated in OSA with TECSA and/or comorbid CSA. Recent studies suggest that CSA resolves spontaneously in a majority of TECSA patients within 2 to 3 months of regular CPAP use.¹⁵ Other estimates suggest that persistent TESCA may be present in 2% of patients with OSA on treatment.¹⁶

Given the high and rising prevalence of OSA, many people are at risk for clinically significant TESCA. Another retrospective case series found that 72% of patients that failed treatment with CPAP or BPAP during PSG, met diagnostic criteria (at the time) for CSA; ASV was objectively beneficial in these patients.¹⁷ ASV can be an especially useful modality to treat OSA in patients with CSA that either prevents tolerance of standard therapies or causes clinical consequences, presuming the patient does not also have HFrEF.¹⁸ The long-term outcomes of treatment with ASV therapy remain a matter of debate.

The SERVE-HF trial remains among the only studies that have assessed the mortality effects of CSA treatments, with unfavorable findings. Treatment of OSA has been associated with favorable chronic health benefits, though recent studies have questioned the degree of benefit attributable to OSA treatment.^19-24 Similar studies have not been done for comorbidities represented by our study cohort (ie, OSA with TECSA and/or comorbid CSA).

The lack of CSA diagnosis alone in our cohort may be partially attributable to changing practice patterns following the SERVE-HF trial, though it is not clear from these data why a higher baseline obstructive AHI was associated with adherence to ASV therapy. Our data in this regard are somewhat at odds with the preliminary results of the ADVENT-HF trial. In that study, adherence to ASV therapy in patients with predominantly OSA declined significantly more than in patients with predominantly CSA.²⁵ Most of our patients were diagnosed with predominantly OSA, so a direct comparison with the CSA group is problematic; additionally, the primary brand and the pressure adjustments algorithm used in our study differed from the ADVENT-HF trial.

OSA and CSA may present with similar clinical symptoms, including sleep fragmentation, insomnia, and excessive daytime sleepiness; however, the degree of symptomatology, especially daytime sleepiness, and the response to treatment, may be less in CSA.^2,26 Both the subjective report of symptoms (ESS) and PSG measures of sleep fragmentation were similar in our patients, again likely explained by the predominance of obstructive events.

The pathophysiology of CSA is more varied than OSA, which is probably relevant in this case. ASV was originally designed for the management of CSA with CSR, accomplishing this goal by stabilizing the periods of central apnea and hyperpnea characteristic of CSR.²⁷ Although other forms of CSA demonstrate breathing patterns distinct from CSR, ASV has become an accepted treatment for most of these. It is plausible that the long-term subjective benefit and tolerance of ASV in CSA without CSR is less than for CSA with CSR or OSA. None of the patients in our study had CSA with CSR.

Ultimately, it may be the objective treatment effect that lends to adherence, as has been shown previously in OSA patients; our group of adherent patients showed a greater improvement in AHI, relative to baseline, than the nonadherent patients did.²⁸ The technology behind ASV therapy can greatly reduce the frequencies of central apneas, yet this same treatment effectively splints the upper airway and even more effectively eliminates obstructive apneas and hypopneas. Variable adjusting EPAP devices would plausibly provide even more benefit in these patients, as has been shown in prior studies.²⁹ To the contrary, our small sample of patients with TESCA showed a nonsignificant trend toward adherence with fixed EPAP ASV.

Opioid use was substantial in our population, without significant differences between the groups. CPAP therapy is ineffective in improving opioid-associated CSA. In a recent study, 20 patients on opioid therapy with CSA were treated with CPAP therapy; after several weeks, the average therapeutic use was 4 to 5 hours per night and CPAP was abandoned in favor of ASV therapy due to persistent central apnea. ASV treatment was associated with a considerable reduction in central apnea index, AHI, arousal index, and oxygen desaturations in a remarkable improvement over CPAP.³⁰

 

Limitations and Future Directions

This retrospective, single-center study may have limited applicability to other populations. Adherence was used as a surrogate for subjective benefit from treatment, though benefit was not confirmed by the patients directly. Only patients seen in follow-up for documentation of the ASV download were identified for inclusion and data analysis. As a single center, we risk homogeneity in the treatment algorithms, though sleep medicine treatments are often decided at the time of the sleep studies. Studies and treatment recommendations were made at a variety of sites, including our sleep center, other US Department of Veterans Affairs hospitals, in the community network, and at US Department of Defense centers. Our population was homogenous in some ways; notably, 100% of our group was male, which is substantially higher than both the veteran population and the general population. Risk factors for OSA and CSA are more common in male patients, which may partially explain this anomaly. Lastly, with our small sample size, there is increased risk that the results seen occurred by chance.

There are several areas for further study. A larger multicenter study may permit these results to be generalized to the population and should include subjective measures of benefit. Patients with primarily CSA were largely absent in our group and may be the focus of future studies; data on predictors of treatment adherence in CSA are lacking. With the availability of consistent older adherence data, comparisons may be made between the efficacies of clinical practice habits, including treatment efficacy, before and after the results of the SERVE-HF trial became known.

Conclusions

In selected patients with preserved LVEF, ASV therapy appears especially effective in patients with OSA combined with CSA. Adherence to ASV treatment was associated with higher obstructive AHI during the baseline PSG and with a greater reduction in the AHI. This understanding may help guide sleep specialists in personalizing treatments for sleep-disordered breathing. Because objective efficacy appears to be important for therapy adherence, clinicians should be able to consistently determine the obstructive and central components of the residual AHI, thus taking all information into account when optimizing the treatment. Additionally, both OSA and CSA pressure requirements should be considered when developing ASV devices.

Acknowledgments

We thank Martha Harper, RRT, of Hampton Veterans Affairs Medical Center (HVAMC) for helping to identify our patients and assisting with data collection. This material is the result of work supported with resources and the use of HVAMC facilities.

Sleep apnea is a heterogeneous group of conditions that may be attributable to a wide array of underlying conditions, with varying contributions of obstructive or central sleep-disordered breathing. The spectrum from obstructive sleep apnea (OSA) to central sleep apnea (CSA) includes mixed sleep apnea, treatment-emergent CSA (TECSA), and Cheyne-Stokes respiration (CSR).¹ The pathophysiologic causes of CSA can be attributed to delayed cardiopulmonary circulation in heart failure, decreased brainstem ventilatory response due to stroke, blunting of central chemoreceptors in chronic opioid use, and/or stimulation of the Hering-Breuer reflex from activation of pulmonary stretch receptors after initiating positive airway pressure (PAP) for treatment of OSA.^2,3 Medications are commonly implicated in many forms of sleep-disordered breathing; importantly, opioids and benzodiazepines may blunt the respiratory drive, leading to CSA, and/or impair upper airway patency, resulting in or worsening OSA.

Continuous positive airway pressure (CPAP) therapy is largely ineffective in correcting CSA or improving outcomes and is often poorly tolerated in these patients.⁴ Adaptive servo-ventilation (ASV) is a form of bilevel PAP (BPAP) therapy that delivers variable adjusting pressure support, primarily to treat CSA. PAP also may relieve upper airway obstructions, thereby effectively treating any comorbid obstructive component. ASV has been well documented to improve sleep-related disorders and improve apnea-hypopnea index (AHI) in patients with CSA. However, longitudinal data have demonstrated increased mortality in patients with heart failure with reduced ejection fraction (HFrEF) who were treated with ASV.⁵ Since the SERVE-HF trial results came to light in 2015, there has been no consensus regarding the optimal use, if any, of ASV therapy.^6-8 This is partly related to the inability to fully explain the study’s major findings, which were unexpected at the time, and partly due to the absence of similar relevant mortality data in patients with CSA but without HFrEF.

TECSA may present in some patients with OSA who are new to PAP therapy. These events are frequently seen during PAP titration sleep studies, though patients can also experience significant TECSA shortly after initiating home PAP therapy. TECSA is felt to result from a combination of stimulating pulmonary stretch receptors and lowering arterial carbon dioxide below the apneic threshold. Chemoreceptors located in the medulla respond by attenuating the respiratory drive.⁹ Previous studies have shown most cases of mild TECSA resolve over time with CPAP treatment. However, in patients with persistent or worsening TECSA, ASV may be considered as an alternative to CPAP.

The prevalence of OSA in the veteran population is estimated to be as high as 60%, considerably higher than the general population estimation.¹⁰ Patients with more significant comorbidities may also experience a higher frequency of central events. Patients with CSA have also been shown to have a higher risk for cardiac-related hospital admissions, providing plausible justification for correcting CSA.¹⁰

In the current study, we aim to characterize the group of patients using ASV therapy in the modern era. We will assess the objective efficacy and adherence of ASV therapy in patients with primarily CSA compared with those having primarily OSA (ie, TECSA). Secondarily, we aim to identify baseline clinical and polysomnographic features that may be predictive of ASV adherence, as a surrogate for subjective benefit.¹¹ In the wake of the SERVE-HF study, the sleep medicine community has paused prescribing ASV therapy for CSA. We hope to provide more perspective on the treatment of veterans with CSA and identify the patient groups that would benefit most from ASV therapy.

 

Methods

This retrospective chart review examined patients prescribed ASV therapy at the Hampton Veterans Affairs Medical Center (HVAMC) in Virginia who had therapy data between January 1, 2015, and April 30, 2020. The start date was chosen to approximate the phase-in of wireless PAP devices at HVAMC and to correspond with the release of preliminary results from the SERVE-HF trial.

Patients were initially identified through a query into commercial wireless PAP management databases and cross-referenced with HVAMC patients. Adherence and efficacy data were obtained from the most recent clinical PAP data, which allowed for the evaluation of patients who discontinued therapy for reasons other than intolerance. Clinical, demographic, and polysomnography (PSG) data were obtained from the electronic health record. One patient, identified through the database query but not found in the electronic health record, was excluded. In cases of missing PSG data, especially AHI or similar values, all attempts were made to calculate the data with other provided values. This study was determined to be exempt by the HVAMC Institutional Review Board (protocol #20-01).

Statistics

Statistical analyses were designed to compare clinical characteristics and adherence to therapy of those with primarily CSA on PSG and those with primarily OSA. Because it was not currently known how many patients would fit into each of these categories, we also planned secondary comparisons of the clinical and PSG characteristics of those patients who were adherent with therapy and those who were not. Adherence with ASV therapy was defined as device use for ≥ 4 hours for ≥ 70% of nights.

Comparisons between the means of 2 normally distributed groups were performed with an unpaired t test. Comparisons between 2 nonnormally distributed groups and groups of dates were done with the Mann-Whitney U test. The normality of a group distribution was determined using D’Agostino-Pearson omnibus normality test. Two groups of dichotomous variables were compared with the Fisher exact test. P value < .05 was considered statistically significant.

Results

Thirty-one patients were prescribed ASV therapy and had follow-up at HVAMC since 2015. All patients were male. The mean (SD) age was 67.2 (11.4) years, mean body mass index (BMI) was 34.0 (5.9), and the mean (SD) Epworth Sleepiness Scale (ESS) score was 10.9 (5.8). Patient comorbidities included 30 (97%) with hypertension, 17 (55%) with diabetes mellitus, 16 (52%) with coronary artery disease, and 11 (35%) with congestive heart failure. Three patients had no echocardiogram or other documentation of left ventricular ejection fraction (LVEF). One of these patients had voluntarily stopped using PAP therapy, another had been erroneously started on ASV (ordered for fixed BPAP), and the third had since been retitrated to CPAP. In the 28 patients with documented LVEF, the mean (SD) LVEF was 61.8% (6.9). Ten patients (32%) had opioids documented on their medication lists and 6 (19%) had benzodiazepines.

The median date of diagnostic sleep testing was January 9, 2015, and testing was completed after the release of the initial field safety notice regarding the SERVE-HF trial preliminary findings May 13, 2015, for 14 patients (45%).¹² On diagnostic sleep testing, the mean (SD) AHI was 47.3 (25.6) events/h and the median (IQR) oxygen saturation (SpO₂) nadir was 82% (78-84). Three patients (10%) were initially diagnosed with CSA, 19 (61%) with OSA, and 9 (29%) with both. Sixteen patients (52%) had ASV with fixed expiratory PAP (EPAP), and 15 (48%) had variable adjusting EPAP. Mean (SD) usage of ASV was 6.5 (2.6) hours and 66.0% (34.2) of nights for ≥ 4 hours. Mean (SD) titrated EPAP (set or 90th/95th percentile autotitrated) was 10.1 (3.4) cm H₂O and inspiratory PAP (IPAP) (90th/95th percentile) was 17.1 (3.3) cm H₂O. The median (IQR) residual AHI on ASV was 2.7 events/h (1.1-5.1), apnea index (AI) was 0.4 (0.1-1.0), and hypopnea index (HI) was 1.4 (1.0-3.2); the residual central and obstructive events were not available in most cases.

 

Adherence

Seventeen patients (55%) met the minimum adherence criteria of ≥ 4 hours of usage for ≥ 70% of the nights. There were no significant differences in age, BMI, sex, race, comorbidities, medications, or ESS when comparing patients who were adherent to ASC and those who were not (Table 1). The date of diagnostic sleep testing and sleep architecture, including sleep latency, total sleep time, sleep efficiency, wake after sleep onset, arousal index, and percentage of rapid eye movement stage sleep were similar between the adherent patients and nonadherent patients (Table 2).

The overall AHI mean (SD) on diagnostic PSG were also similar between the adherent group (52.3 [24.8] events/h) and nonadherent group (45.1 [27.0] events/h) (P = .47). The mean (SD) for obstructive AHI (obstructive apneas, mixed apneas, obstructive hypopneas, and undifferentiated hypopneas per hour of sleep) were higher in the adherent group as a percentage of the total AHI: 81.5% (27.9) in the adherent group vs 46.7% (38.4%) in the nonadherent group; P = .02) (Figure 1).

This difference was primarily driven by a difference in mean (SD) HI: 29.7 (16.5) in the adherent group vs. 15.3 (12.1) in the nonadherent group (P = .04).

There were no significant differences between the proportions of patients on ASV with set EPAP or the titrated EPAP and IPAP. The median (IQR) residual AHI was lower in the adherent group compared with the nonadherent group, both in absolute values (1.7 [0.9-3.2] events/h vs 4.7 [2.4-10.3] events/h, respectively [P = .004]), and as a percentage of the pretreatment AHI (3.1% [2.5-6.0] vs 10.2% [5.3-34.4], respectively; P = .002) (Figure 2).

Primarily Obstructive Sleep Apnea

Sleep apnea was a mixed picture of obstructive and central events in many patients. Only 3 patients had “pure” CSA. Thus, we were unable to define discrete comparison groups based on the sleep-disordered breathing phenotype. We identified 19 patients with primarily OSA (ie, initially diagnosed with OSA, OSA with TECSA, or complex sleep apnea). The mean (SD) age was 66.1 (12.8) years, BMI was 36.2 (4.7), and ESS was 11.4 (5.6). The mean (SD) baseline AHI was 46.9 (29.5), obstructive AHI was 40.5 (30.4), and central AHI was 0.4 (1.2); the median (IQR) SpO₂ nadir was 81% (78%-84%). The mean (SD) titrated EPAP was 10.2 (3.5) cm H₂O, and the 90th/95th percentile IPAP was 17.9 (3.5) cm H₂O. The mean (SD) usage of ASV was 7.9 (5.3) hours with 11 patients (58%) meeting the minimum standard for adherence to ASV therapy.

No significant differences were seen between the adherent and nonadherent groups in clinical or demographic characteristics or date of diagnostic sleep testing (eAppendix, available online at doi:10.12788/fp.0374). In baseline sleep studies the mean (SD) HI was 32.3 (15.8) in the adherent group compared with 14.7 (8.8) in the nonadherent group (P = .049). In contrast, obstructive AHI was not significantly lower in the adherent group: 51.9 (30.9) in the adherent group compared with 22.2 (20.6) in the nonadherent group (P = .09). The median (IQR) residual AHI on ASV as a percentage of the pretreatment AHI was 3.0% (2.4%-6.5%) in the adherent group compared with 11.3% (5.4%-89.1%) in the nonadherent group, a statistically significant difference (P = .01). No other significant differences were seen between the groups.

 

Discussion

This study describes a real-world cohort of patients using ASV therapy and the characteristics associated with benefit from therapy. The patients that were prescribed and started ASV therapy most often had a significant degree of obstructive component to sleep-disordered breathing, whether primary OSA with TECSA or comorbid OSA and CSA. Moreover, we found that a higher obstructive AHI on the baseline PSG was associated with adherence to ASV therapy. Another important finding was that a lower residual AHI on ASV as a proportion of the baseline was associated with PAP adherence. Adherent patients had similar clinical characteristics as the nonadherent patients, including comorbidities, severity of sleep-disordered breathing, and obesity.

Though the results of the SERVE-HF trial have dampened the enthusiasm somewhat, ASV therapy has long been considered an effective and well-tolerated treatment for many types of CSA.¹³ In fact, treatments that can eliminate the central AHI are fairly limited.^4,14 Our data suggest that ASV is also effective and tolerated in OSA with TECSA and/or comorbid CSA. Recent studies suggest that CSA resolves spontaneously in a majority of TECSA patients within 2 to 3 months of regular CPAP use.¹⁵ Other estimates suggest that persistent TESCA may be present in 2% of patients with OSA on treatment.¹⁶

Given the high and rising prevalence of OSA, many people are at risk for clinically significant TESCA. Another retrospective case series found that 72% of patients that failed treatment with CPAP or BPAP during PSG, met diagnostic criteria (at the time) for CSA; ASV was objectively beneficial in these patients.¹⁷ ASV can be an especially useful modality to treat OSA in patients with CSA that either prevents tolerance of standard therapies or causes clinical consequences, presuming the patient does not also have HFrEF.¹⁸ The long-term outcomes of treatment with ASV therapy remain a matter of debate.

The SERVE-HF trial remains among the only studies that have assessed the mortality effects of CSA treatments, with unfavorable findings. Treatment of OSA has been associated with favorable chronic health benefits, though recent studies have questioned the degree of benefit attributable to OSA treatment.^19-24 Similar studies have not been done for comorbidities represented by our study cohort (ie, OSA with TECSA and/or comorbid CSA).

The lack of CSA diagnosis alone in our cohort may be partially attributable to changing practice patterns following the SERVE-HF trial, though it is not clear from these data why a higher baseline obstructive AHI was associated with adherence to ASV therapy. Our data in this regard are somewhat at odds with the preliminary results of the ADVENT-HF trial. In that study, adherence to ASV therapy in patients with predominantly OSA declined significantly more than in patients with predominantly CSA.²⁵ Most of our patients were diagnosed with predominantly OSA, so a direct comparison with the CSA group is problematic; additionally, the primary brand and the pressure adjustments algorithm used in our study differed from the ADVENT-HF trial.

OSA and CSA may present with similar clinical symptoms, including sleep fragmentation, insomnia, and excessive daytime sleepiness; however, the degree of symptomatology, especially daytime sleepiness, and the response to treatment, may be less in CSA.^2,26 Both the subjective report of symptoms (ESS) and PSG measures of sleep fragmentation were similar in our patients, again likely explained by the predominance of obstructive events.

The pathophysiology of CSA is more varied than OSA, which is probably relevant in this case. ASV was originally designed for the management of CSA with CSR, accomplishing this goal by stabilizing the periods of central apnea and hyperpnea characteristic of CSR.²⁷ Although other forms of CSA demonstrate breathing patterns distinct from CSR, ASV has become an accepted treatment for most of these. It is plausible that the long-term subjective benefit and tolerance of ASV in CSA without CSR is less than for CSA with CSR or OSA. None of the patients in our study had CSA with CSR.

Ultimately, it may be the objective treatment effect that lends to adherence, as has been shown previously in OSA patients; our group of adherent patients showed a greater improvement in AHI, relative to baseline, than the nonadherent patients did.²⁸ The technology behind ASV therapy can greatly reduce the frequencies of central apneas, yet this same treatment effectively splints the upper airway and even more effectively eliminates obstructive apneas and hypopneas. Variable adjusting EPAP devices would plausibly provide even more benefit in these patients, as has been shown in prior studies.²⁹ To the contrary, our small sample of patients with TESCA showed a nonsignificant trend toward adherence with fixed EPAP ASV.

Opioid use was substantial in our population, without significant differences between the groups. CPAP therapy is ineffective in improving opioid-associated CSA. In a recent study, 20 patients on opioid therapy with CSA were treated with CPAP therapy; after several weeks, the average therapeutic use was 4 to 5 hours per night and CPAP was abandoned in favor of ASV therapy due to persistent central apnea. ASV treatment was associated with a considerable reduction in central apnea index, AHI, arousal index, and oxygen desaturations in a remarkable improvement over CPAP.³⁰

 

Limitations and Future Directions

This retrospective, single-center study may have limited applicability to other populations. Adherence was used as a surrogate for subjective benefit from treatment, though benefit was not confirmed by the patients directly. Only patients seen in follow-up for documentation of the ASV download were identified for inclusion and data analysis. As a single center, we risk homogeneity in the treatment algorithms, though sleep medicine treatments are often decided at the time of the sleep studies. Studies and treatment recommendations were made at a variety of sites, including our sleep center, other US Department of Veterans Affairs hospitals, in the community network, and at US Department of Defense centers. Our population was homogenous in some ways; notably, 100% of our group was male, which is substantially higher than both the veteran population and the general population. Risk factors for OSA and CSA are more common in male patients, which may partially explain this anomaly. Lastly, with our small sample size, there is increased risk that the results seen occurred by chance.

There are several areas for further study. A larger multicenter study may permit these results to be generalized to the population and should include subjective measures of benefit. Patients with primarily CSA were largely absent in our group and may be the focus of future studies; data on predictors of treatment adherence in CSA are lacking. With the availability of consistent older adherence data, comparisons may be made between the efficacies of clinical practice habits, including treatment efficacy, before and after the results of the SERVE-HF trial became known.

Conclusions

In selected patients with preserved LVEF, ASV therapy appears especially effective in patients with OSA combined with CSA. Adherence to ASV treatment was associated with higher obstructive AHI during the baseline PSG and with a greater reduction in the AHI. This understanding may help guide sleep specialists in personalizing treatments for sleep-disordered breathing. Because objective efficacy appears to be important for therapy adherence, clinicians should be able to consistently determine the obstructive and central components of the residual AHI, thus taking all information into account when optimizing the treatment. Additionally, both OSA and CSA pressure requirements should be considered when developing ASV devices.

Acknowledgments

We thank Martha Harper, RRT, of Hampton Veterans Affairs Medical Center (HVAMC) for helping to identify our patients and assisting with data collection. This material is the result of work supported with resources and the use of HVAMC facilities.

The inappropriate use of antibiotics is associated with an increased risk of antibiotic resistance, health care costs, and risk of adverse drug reactions.¹ According to the Centers for Disease Control and Prevention (CDC), a 10% decrease in overall antibiotic use across different wards was associated with a 34% decrease in Clostridioides difficile (C difficile) infections.² In addition, antimicrobial resistance accounts for > 2.8 million infections and > 35,000 deaths each year.³ The estimated total economic costs of antibiotic resistance to the US economy have ranged as high as $20 billion in excess direct health care costs.⁴ A main goal of an antimicrobial stewardship program (ASP) is to optimize antibiotic use to prevent the adverse consequences of inappropriate antibiotic prescribing.

During the COVID-19 pandemic, increased use of empiric antibiotic therapy has been observed. According to the CDC, almost 80% of patients hospitalized with COVID-19 received an antibiotic from March 2020 to October 2020.⁵ Studies were conducted to investigate the prevalence of bacterial coinfection in patients with COVID-19 and whether antibiotics were indicated in this patient population. A United Kingdom multicenter, prospective cohort study showed a high proportion of patients hospitalized with COVID-19 received antimicrobials despite microbiologically confirmed bacterial infections being rare and more likely to be secondary infections.⁶

Many other studies have reported similar findings. Langord and colleagues found the prevalence of bacterial coinfection in patients with COVID-19 was 3.5% but that 71.9% received antibiotics.⁷ Coenen and colleagues identified 12.4% of the patients with possible and 1.1% of patients with probable bacterial coinfection, while 81% of the study population and 78% of patients were classified as unlikely bacterial coinfection received antibiotics.⁸

At Veterans Affairs Southern Nevada Healthcare System (VASNHS), an ASP team consisting of an infectious disease (ID) physician and 2 pharmacists provide daily prospective audits with intervention and feedback along with other interventions, such as providing restricted order menus, institutional treatment guidelines, and staff education to help improve antibiotic prescribing. The ASP pharmacists have a scope of practice to make changes to anti-infective therapies. The purpose of this study was to describe antibiotic prescribing in patients hospitalized with COVID-19 from November 1, 2020, to January 31, 2021, in an ASP setting led by pharmacists.

 

Methods

This retrospective descriptive study included patients who were hospitalized for the treatment of laboratory-confirmed COVID-19 infection. The Theradoc clinical surveillance system was used to retrieve a list of patients who were admitted to VASNHS from November 1, 2020, to January 31, 2021, and tested positive for COVID-19. Patients with incidental positive COVID-19 test results or those who received antibiotics for extrapulmonary indications on hospital admission were excluded.

Each patient chart was reviewed and data, including clinical presentations, procalcitonin (PCT), the requirement of supplemental oxygen, vital signs, imaging findings, antibiotic orders on admission, ASP interventions such as discontinuation or changes to antibiotic therapy during the first 72 hours of hospital admission, clinical outcomes, culture results, and readmission rate, defined as any hospital admission related to COVID-19 or respiratory tract infection within 30 days from the previous discharge, were collected.

The primary objective of the study was to describe antibiotic prescribing in patients hospitalized with COVID-19. The secondary outcomes included the prevalence of bacterial coinfection and nosocomial bacterial infection in patients hospitalized with COVID-19.

Results

A total of 199 patients were admitted to the hospital for laboratory-confirmed COVID-19 infection from November 1, 2020, to January 31, 2021. Sixty-one patients (31%) received at least 1 antibiotic on hospital admission. Among those patients who received empiric antibiotic treatment, 29 patients (48%) met the Systemic Inflammatory Response Syndrome (SIRS) criteria. Fifty-six patients (92%) had ≥ 1 PCT level obtained, and 26 of those (46%) presented with elevated PCT levels (PCT > 0.25). Fifty patients (82%) required oxygen supplement and 49 (80%) presented with remarkable imaging findings. Of 138 patients who did not receive empiric antibiotic therapy within 72 hours of hospital admission, 56 (41%) met the SIRS criteria, 31 (29%) had elevated PCT levels, 100 (72%) required oxygen supplement, and 79 (59%) presented with remarkable imaging findings.

Antibiotic Prescribing

Forty-six of 61 patients (75%) received antibiotic treatment for community-acquired pneumonia (CAP) that included ceftriaxone and azithromycin. Three patients (5%) received ≥ 1 broad-spectrum antibiotic (4th generation cephalosporin [cefepime] or piperacillin-tazobactam), 2 (3%) received vancomycin, and 1 (2%) received a fluoroquinolone (levofloxacin) on admission.

Among 61 patients who received empiric antibiotics, the readmission rate was 6%. The mortality rate was 20%, and the mean (SD) duration of hospital stay was 13.1 (12.5) days. There were 47 patients (77%) who had antibiotics discontinued by the ASP within 72 hours of admission. Among these patients, the readmission rate was 8% and the mortality rate was 15%. The mean (SD) duration of hospital stay was 12.7 (13.3) days. In the group of patients without empiric antibiotic therapy, the readmission rate was 6%, the mortality rate was 10%, and the mean (SD) duration of hospital stay was 10.1 (9.5) days.

Six of 199 patients (3%) had microbiologically confirmed bacterial coinfection on hospital admission: 3 were Pseudomonas aeruginosa (P aeruginosa) and 2 were Klebsiella oxytoca (Table 1).

A total of 16 patients (8%) developed a nosocomial infection, and 10 had suspected hospital-acquired or ventilator-acquired pneumonia (Table 2).

Eight patients had positive respiratory cultures, and P aeruginosa was the most common. Five patients had bacteremia, including 1 vancomycin-resistant Enterococcus, 1 methicillin-resistant Staphylococcus aureus, Enterococcus spp, 1 culture grew both Escherichia coli and methicillin-susceptible Staphylococcus aureus (MSSA), and 1 culture grew MSSA and Streptococcus salivarius. Five patients had candidemia during their hospital stay: 3 Candida albicans, 1 Candida tropicalis, and 1 Candida glabrata isolate (eAppendix, available online at doi:10.12788/fp.0380).

 

Discussion

Prospective audit and feedback and preauthorization are recommended in guidelines as “core components of any stewardship program.”⁹ At VASNHS, the ASP performs daily prospective audits with intervention and feedback. Efforts have been made to maintain daily ASP activities during the pandemic. This study aimed to describe antibiotic prescribing for patients hospitalized with COVID-19 in a pharmacist-led ASP setting. It was found that up to 31% of the patients received ≥ 1 antibiotic on admission for empiric treatment of bacterial coinfection. About half of these patients met the SIRS criteria. Most of these patients received ceftriaxone and azithromycin for concern of CAP. ASP discontinued antibiotics within 72 hours in most of the patients. Chart review and discussion with ID physicians and/or hospitalists determined the probability of bacterial coinfection as well as any potential complication or patient-specific risk factor. It is important to note that most patients who received antibiotics on admission had ≥ 1 PCT level and up to 46% of them had a PCT level > 0.25. However, according to Relph and colleagues, PCT may not be a reliable indicator of bacterial infection in severe viral diseases with raised interleukin-6 levels.¹⁰ An elevated PCT level should not be the sole indicator for empiric antibiotic treatment.

Study findings confirmed the low prevalence of bacterial coinfection in patients hospitalized with COVID-19. The overuse of empiric antibiotics in a patient population unlikely to present with bacterial coinfection is concerning. It is essential to continue promoting antimicrobial stewardship during the COVID-19 pandemic to ensure appropriate and responsible antimicrobial prescribing. A thorough clinical assessment consisting of comorbidities, clinical symptoms, radiologic and microbiologic findings, as well as other relevant workup or biomarker results is crucial to determine whether the antibiotic is strongly indicated in patients hospitalized with COVID-19. Empiric antibiotic therapy should be considered only in patients with clinical findings suggestive of bacterial coinfection.

Limitations

Limitations of our study included the study design (single-center, retrospective review, lack of comparative group) and small sample size with a 3-month study period. In addition, respiratory cultures are not commonly obtained in patients who present with mild-to-moderate CAP. Using culture results solely to confirm bacterial coinfection in patients with COVID-19 could have underestimated the prevalence of bacterial infection. Developing diagnostic criteria that include clinical signs and symptoms, imaging findings, and laboratory results as well as culture results would help to better assess the presence of bacterial coinfection in this patient population.

Conclusions

The study findings showed that up to 30% of patients hospitalized for COVID-19 infection received empiric antibiotic treatment for concern of bacterial coinfection. A pharmacist-led ASP provided interventions, including early discontinuation of antibiotics in 77% of these patients.

A low prevalence of bacterial coinfection (3%) in patients hospitalized with COVID-19 also was reported. A thorough clinical workup to determine the risk of bacterial coinfection in patients with COVID-19 is important before starting empiric antibiotic therapy. Continuing to promote the ASP during the COVID-19 pandemic to ensure responsible antibiotic use and prevent antimicrobial resistance is essential.

The inappropriate use of antibiotics is associated with an increased risk of antibiotic resistance, health care costs, and risk of adverse drug reactions.¹ According to the Centers for Disease Control and Prevention (CDC), a 10% decrease in overall antibiotic use across different wards was associated with a 34% decrease in Clostridioides difficile (C difficile) infections.² In addition, antimicrobial resistance accounts for > 2.8 million infections and > 35,000 deaths each year.³ The estimated total economic costs of antibiotic resistance to the US economy have ranged as high as $20 billion in excess direct health care costs.⁴ A main goal of an antimicrobial stewardship program (ASP) is to optimize antibiotic use to prevent the adverse consequences of inappropriate antibiotic prescribing.

During the COVID-19 pandemic, increased use of empiric antibiotic therapy has been observed. According to the CDC, almost 80% of patients hospitalized with COVID-19 received an antibiotic from March 2020 to October 2020.⁵ Studies were conducted to investigate the prevalence of bacterial coinfection in patients with COVID-19 and whether antibiotics were indicated in this patient population. A United Kingdom multicenter, prospective cohort study showed a high proportion of patients hospitalized with COVID-19 received antimicrobials despite microbiologically confirmed bacterial infections being rare and more likely to be secondary infections.⁶

Many other studies have reported similar findings. Langord and colleagues found the prevalence of bacterial coinfection in patients with COVID-19 was 3.5% but that 71.9% received antibiotics.⁷ Coenen and colleagues identified 12.4% of the patients with possible and 1.1% of patients with probable bacterial coinfection, while 81% of the study population and 78% of patients were classified as unlikely bacterial coinfection received antibiotics.⁸

At Veterans Affairs Southern Nevada Healthcare System (VASNHS), an ASP team consisting of an infectious disease (ID) physician and 2 pharmacists provide daily prospective audits with intervention and feedback along with other interventions, such as providing restricted order menus, institutional treatment guidelines, and staff education to help improve antibiotic prescribing. The ASP pharmacists have a scope of practice to make changes to anti-infective therapies. The purpose of this study was to describe antibiotic prescribing in patients hospitalized with COVID-19 from November 1, 2020, to January 31, 2021, in an ASP setting led by pharmacists.

 

Methods

This retrospective descriptive study included patients who were hospitalized for the treatment of laboratory-confirmed COVID-19 infection. The Theradoc clinical surveillance system was used to retrieve a list of patients who were admitted to VASNHS from November 1, 2020, to January 31, 2021, and tested positive for COVID-19. Patients with incidental positive COVID-19 test results or those who received antibiotics for extrapulmonary indications on hospital admission were excluded.

Each patient chart was reviewed and data, including clinical presentations, procalcitonin (PCT), the requirement of supplemental oxygen, vital signs, imaging findings, antibiotic orders on admission, ASP interventions such as discontinuation or changes to antibiotic therapy during the first 72 hours of hospital admission, clinical outcomes, culture results, and readmission rate, defined as any hospital admission related to COVID-19 or respiratory tract infection within 30 days from the previous discharge, were collected.

The primary objective of the study was to describe antibiotic prescribing in patients hospitalized with COVID-19. The secondary outcomes included the prevalence of bacterial coinfection and nosocomial bacterial infection in patients hospitalized with COVID-19.

Results

A total of 199 patients were admitted to the hospital for laboratory-confirmed COVID-19 infection from November 1, 2020, to January 31, 2021. Sixty-one patients (31%) received at least 1 antibiotic on hospital admission. Among those patients who received empiric antibiotic treatment, 29 patients (48%) met the Systemic Inflammatory Response Syndrome (SIRS) criteria. Fifty-six patients (92%) had ≥ 1 PCT level obtained, and 26 of those (46%) presented with elevated PCT levels (PCT > 0.25). Fifty patients (82%) required oxygen supplement and 49 (80%) presented with remarkable imaging findings. Of 138 patients who did not receive empiric antibiotic therapy within 72 hours of hospital admission, 56 (41%) met the SIRS criteria, 31 (29%) had elevated PCT levels, 100 (72%) required oxygen supplement, and 79 (59%) presented with remarkable imaging findings.

Antibiotic Prescribing

Forty-six of 61 patients (75%) received antibiotic treatment for community-acquired pneumonia (CAP) that included ceftriaxone and azithromycin. Three patients (5%) received ≥ 1 broad-spectrum antibiotic (4th generation cephalosporin [cefepime] or piperacillin-tazobactam), 2 (3%) received vancomycin, and 1 (2%) received a fluoroquinolone (levofloxacin) on admission.

Among 61 patients who received empiric antibiotics, the readmission rate was 6%. The mortality rate was 20%, and the mean (SD) duration of hospital stay was 13.1 (12.5) days. There were 47 patients (77%) who had antibiotics discontinued by the ASP within 72 hours of admission. Among these patients, the readmission rate was 8% and the mortality rate was 15%. The mean (SD) duration of hospital stay was 12.7 (13.3) days. In the group of patients without empiric antibiotic therapy, the readmission rate was 6%, the mortality rate was 10%, and the mean (SD) duration of hospital stay was 10.1 (9.5) days.

Six of 199 patients (3%) had microbiologically confirmed bacterial coinfection on hospital admission: 3 were Pseudomonas aeruginosa (P aeruginosa) and 2 were Klebsiella oxytoca (Table 1).

A total of 16 patients (8%) developed a nosocomial infection, and 10 had suspected hospital-acquired or ventilator-acquired pneumonia (Table 2).

Eight patients had positive respiratory cultures, and P aeruginosa was the most common. Five patients had bacteremia, including 1 vancomycin-resistant Enterococcus, 1 methicillin-resistant Staphylococcus aureus, Enterococcus spp, 1 culture grew both Escherichia coli and methicillin-susceptible Staphylococcus aureus (MSSA), and 1 culture grew MSSA and Streptococcus salivarius. Five patients had candidemia during their hospital stay: 3 Candida albicans, 1 Candida tropicalis, and 1 Candida glabrata isolate (eAppendix, available online at doi:10.12788/fp.0380).

 

Discussion

Prospective audit and feedback and preauthorization are recommended in guidelines as “core components of any stewardship program.”⁹ At VASNHS, the ASP performs daily prospective audits with intervention and feedback. Efforts have been made to maintain daily ASP activities during the pandemic. This study aimed to describe antibiotic prescribing for patients hospitalized with COVID-19 in a pharmacist-led ASP setting. It was found that up to 31% of the patients received ≥ 1 antibiotic on admission for empiric treatment of bacterial coinfection. About half of these patients met the SIRS criteria. Most of these patients received ceftriaxone and azithromycin for concern of CAP. ASP discontinued antibiotics within 72 hours in most of the patients. Chart review and discussion with ID physicians and/or hospitalists determined the probability of bacterial coinfection as well as any potential complication or patient-specific risk factor. It is important to note that most patients who received antibiotics on admission had ≥ 1 PCT level and up to 46% of them had a PCT level > 0.25. However, according to Relph and colleagues, PCT may not be a reliable indicator of bacterial infection in severe viral diseases with raised interleukin-6 levels.¹⁰ An elevated PCT level should not be the sole indicator for empiric antibiotic treatment.

Study findings confirmed the low prevalence of bacterial coinfection in patients hospitalized with COVID-19. The overuse of empiric antibiotics in a patient population unlikely to present with bacterial coinfection is concerning. It is essential to continue promoting antimicrobial stewardship during the COVID-19 pandemic to ensure appropriate and responsible antimicrobial prescribing. A thorough clinical assessment consisting of comorbidities, clinical symptoms, radiologic and microbiologic findings, as well as other relevant workup or biomarker results is crucial to determine whether the antibiotic is strongly indicated in patients hospitalized with COVID-19. Empiric antibiotic therapy should be considered only in patients with clinical findings suggestive of bacterial coinfection.

Limitations

Limitations of our study included the study design (single-center, retrospective review, lack of comparative group) and small sample size with a 3-month study period. In addition, respiratory cultures are not commonly obtained in patients who present with mild-to-moderate CAP. Using culture results solely to confirm bacterial coinfection in patients with COVID-19 could have underestimated the prevalence of bacterial infection. Developing diagnostic criteria that include clinical signs and symptoms, imaging findings, and laboratory results as well as culture results would help to better assess the presence of bacterial coinfection in this patient population.

Conclusions

The study findings showed that up to 30% of patients hospitalized for COVID-19 infection received empiric antibiotic treatment for concern of bacterial coinfection. A pharmacist-led ASP provided interventions, including early discontinuation of antibiotics in 77% of these patients.

A low prevalence of bacterial coinfection (3%) in patients hospitalized with COVID-19 also was reported. A thorough clinical workup to determine the risk of bacterial coinfection in patients with COVID-19 is important before starting empiric antibiotic therapy. Continuing to promote the ASP during the COVID-19 pandemic to ensure responsible antibiotic use and prevent antimicrobial resistance is essential.

The inappropriate use of antibiotics is associated with an increased risk of antibiotic resistance, health care costs, and risk of adverse drug reactions.¹ According to the Centers for Disease Control and Prevention (CDC), a 10% decrease in overall antibiotic use across different wards was associated with a 34% decrease in Clostridioides difficile (C difficile) infections.² In addition, antimicrobial resistance accounts for > 2.8 million infections and > 35,000 deaths each year.³ The estimated total economic costs of antibiotic resistance to the US economy have ranged as high as $20 billion in excess direct health care costs.⁴ A main goal of an antimicrobial stewardship program (ASP) is to optimize antibiotic use to prevent the adverse consequences of inappropriate antibiotic prescribing.

During the COVID-19 pandemic, increased use of empiric antibiotic therapy has been observed. According to the CDC, almost 80% of patients hospitalized with COVID-19 received an antibiotic from March 2020 to October 2020.⁵ Studies were conducted to investigate the prevalence of bacterial coinfection in patients with COVID-19 and whether antibiotics were indicated in this patient population. A United Kingdom multicenter, prospective cohort study showed a high proportion of patients hospitalized with COVID-19 received antimicrobials despite microbiologically confirmed bacterial infections being rare and more likely to be secondary infections.⁶

Many other studies have reported similar findings. Langord and colleagues found the prevalence of bacterial coinfection in patients with COVID-19 was 3.5% but that 71.9% received antibiotics.⁷ Coenen and colleagues identified 12.4% of the patients with possible and 1.1% of patients with probable bacterial coinfection, while 81% of the study population and 78% of patients were classified as unlikely bacterial coinfection received antibiotics.⁸

At Veterans Affairs Southern Nevada Healthcare System (VASNHS), an ASP team consisting of an infectious disease (ID) physician and 2 pharmacists provide daily prospective audits with intervention and feedback along with other interventions, such as providing restricted order menus, institutional treatment guidelines, and staff education to help improve antibiotic prescribing. The ASP pharmacists have a scope of practice to make changes to anti-infective therapies. The purpose of this study was to describe antibiotic prescribing in patients hospitalized with COVID-19 from November 1, 2020, to January 31, 2021, in an ASP setting led by pharmacists.

 

Methods

This retrospective descriptive study included patients who were hospitalized for the treatment of laboratory-confirmed COVID-19 infection. The Theradoc clinical surveillance system was used to retrieve a list of patients who were admitted to VASNHS from November 1, 2020, to January 31, 2021, and tested positive for COVID-19. Patients with incidental positive COVID-19 test results or those who received antibiotics for extrapulmonary indications on hospital admission were excluded.

Each patient chart was reviewed and data, including clinical presentations, procalcitonin (PCT), the requirement of supplemental oxygen, vital signs, imaging findings, antibiotic orders on admission, ASP interventions such as discontinuation or changes to antibiotic therapy during the first 72 hours of hospital admission, clinical outcomes, culture results, and readmission rate, defined as any hospital admission related to COVID-19 or respiratory tract infection within 30 days from the previous discharge, were collected.

The primary objective of the study was to describe antibiotic prescribing in patients hospitalized with COVID-19. The secondary outcomes included the prevalence of bacterial coinfection and nosocomial bacterial infection in patients hospitalized with COVID-19.

Results

A total of 199 patients were admitted to the hospital for laboratory-confirmed COVID-19 infection from November 1, 2020, to January 31, 2021. Sixty-one patients (31%) received at least 1 antibiotic on hospital admission. Among those patients who received empiric antibiotic treatment, 29 patients (48%) met the Systemic Inflammatory Response Syndrome (SIRS) criteria. Fifty-six patients (92%) had ≥ 1 PCT level obtained, and 26 of those (46%) presented with elevated PCT levels (PCT > 0.25). Fifty patients (82%) required oxygen supplement and 49 (80%) presented with remarkable imaging findings. Of 138 patients who did not receive empiric antibiotic therapy within 72 hours of hospital admission, 56 (41%) met the SIRS criteria, 31 (29%) had elevated PCT levels, 100 (72%) required oxygen supplement, and 79 (59%) presented with remarkable imaging findings.

Antibiotic Prescribing

Forty-six of 61 patients (75%) received antibiotic treatment for community-acquired pneumonia (CAP) that included ceftriaxone and azithromycin. Three patients (5%) received ≥ 1 broad-spectrum antibiotic (4th generation cephalosporin [cefepime] or piperacillin-tazobactam), 2 (3%) received vancomycin, and 1 (2%) received a fluoroquinolone (levofloxacin) on admission.

Among 61 patients who received empiric antibiotics, the readmission rate was 6%. The mortality rate was 20%, and the mean (SD) duration of hospital stay was 13.1 (12.5) days. There were 47 patients (77%) who had antibiotics discontinued by the ASP within 72 hours of admission. Among these patients, the readmission rate was 8% and the mortality rate was 15%. The mean (SD) duration of hospital stay was 12.7 (13.3) days. In the group of patients without empiric antibiotic therapy, the readmission rate was 6%, the mortality rate was 10%, and the mean (SD) duration of hospital stay was 10.1 (9.5) days.

Six of 199 patients (3%) had microbiologically confirmed bacterial coinfection on hospital admission: 3 were Pseudomonas aeruginosa (P aeruginosa) and 2 were Klebsiella oxytoca (Table 1).

A total of 16 patients (8%) developed a nosocomial infection, and 10 had suspected hospital-acquired or ventilator-acquired pneumonia (Table 2).

Eight patients had positive respiratory cultures, and P aeruginosa was the most common. Five patients had bacteremia, including 1 vancomycin-resistant Enterococcus, 1 methicillin-resistant Staphylococcus aureus, Enterococcus spp, 1 culture grew both Escherichia coli and methicillin-susceptible Staphylococcus aureus (MSSA), and 1 culture grew MSSA and Streptococcus salivarius. Five patients had candidemia during their hospital stay: 3 Candida albicans, 1 Candida tropicalis, and 1 Candida glabrata isolate (eAppendix, available online at doi:10.12788/fp.0380).

 

Discussion

Prospective audit and feedback and preauthorization are recommended in guidelines as “core components of any stewardship program.”⁹ At VASNHS, the ASP performs daily prospective audits with intervention and feedback. Efforts have been made to maintain daily ASP activities during the pandemic. This study aimed to describe antibiotic prescribing for patients hospitalized with COVID-19 in a pharmacist-led ASP setting. It was found that up to 31% of the patients received ≥ 1 antibiotic on admission for empiric treatment of bacterial coinfection. About half of these patients met the SIRS criteria. Most of these patients received ceftriaxone and azithromycin for concern of CAP. ASP discontinued antibiotics within 72 hours in most of the patients. Chart review and discussion with ID physicians and/or hospitalists determined the probability of bacterial coinfection as well as any potential complication or patient-specific risk factor. It is important to note that most patients who received antibiotics on admission had ≥ 1 PCT level and up to 46% of them had a PCT level > 0.25. However, according to Relph and colleagues, PCT may not be a reliable indicator of bacterial infection in severe viral diseases with raised interleukin-6 levels.¹⁰ An elevated PCT level should not be the sole indicator for empiric antibiotic treatment.

Study findings confirmed the low prevalence of bacterial coinfection in patients hospitalized with COVID-19. The overuse of empiric antibiotics in a patient population unlikely to present with bacterial coinfection is concerning. It is essential to continue promoting antimicrobial stewardship during the COVID-19 pandemic to ensure appropriate and responsible antimicrobial prescribing. A thorough clinical assessment consisting of comorbidities, clinical symptoms, radiologic and microbiologic findings, as well as other relevant workup or biomarker results is crucial to determine whether the antibiotic is strongly indicated in patients hospitalized with COVID-19. Empiric antibiotic therapy should be considered only in patients with clinical findings suggestive of bacterial coinfection.

Limitations

Limitations of our study included the study design (single-center, retrospective review, lack of comparative group) and small sample size with a 3-month study period. In addition, respiratory cultures are not commonly obtained in patients who present with mild-to-moderate CAP. Using culture results solely to confirm bacterial coinfection in patients with COVID-19 could have underestimated the prevalence of bacterial infection. Developing diagnostic criteria that include clinical signs and symptoms, imaging findings, and laboratory results as well as culture results would help to better assess the presence of bacterial coinfection in this patient population.

Conclusions

The study findings showed that up to 30% of patients hospitalized for COVID-19 infection received empiric antibiotic treatment for concern of bacterial coinfection. A pharmacist-led ASP provided interventions, including early discontinuation of antibiotics in 77% of these patients.

A low prevalence of bacterial coinfection (3%) in patients hospitalized with COVID-19 also was reported. A thorough clinical workup to determine the risk of bacterial coinfection in patients with COVID-19 is important before starting empiric antibiotic therapy. Continuing to promote the ASP during the COVID-19 pandemic to ensure responsible antibiotic use and prevent antimicrobial resistance is essential.