Hospital Medicine

Purpose

Ensuring that patients/families are engaged as partners in their health care is an effective way to measure the quality of patient care. Self-reported patient data, such as symptom burden, provides an accurate and effective way to measure patient-reported outcomes. Our team reviewed 20 patient charts, randomly, to assess for documentation of at least 3 or more domains of toxicities of immune checkpoint inhibitors. The baseline comprehensive documentation rate was 50%. Our goal is to improve the documentation rate to 80% for our first process improvement cycle.

Aim Statement

Increase documentation of 3 or more toxicities immune checkpoint inhibitors to a goal rate of 80%.

Methods

A free online patient monitoring checklist tool was printed and provided to patients receiving immune checkpoint inhibitors (on their infusion day) during the check-in process. The patients were instructed to complete the tool prior to the provider clinic visit, while in the waiting area. The completed tool was given to the provider on the day of their visit. Prior to the start of this Plan-Do-Study-Act (PDSA) cycle, all providers were “reminded”/ instructed to ensure documentation of 3 or more toxicities immune checkpoint inhibitors. The cycle lasted for 3 weeks. At the end of the 3 weeks, our team reviewed the charts of those patients.

Results

Documentation rate of 3 or more toxicities increased from 50% to 90%.

Conclusions

When completed patient monitoring tools were provided to the providers during the clinic visit, the providers increased their documentation rate of the toxicities. There is literature supporting improving patient satisfaction using self-reported symptoms monitoring tools. Also, given the burden of documentation and shorter visit times, providers found this to be an easy way to address patient symptoms. While electronic patient-reported outcome (e-PRO) tools are ideal for ongoing symptom monitoring, this is a simple way to address the same in low-resourced communities. For our next cycle, we plan on using patient feedback to improve the documentation form incorporating larger fonts for patients with low vision.

Purpose

Ensuring that patients/families are engaged as partners in their health care is an effective way to measure the quality of patient care. Self-reported patient data, such as symptom burden, provides an accurate and effective way to measure patient-reported outcomes. Our team reviewed 20 patient charts, randomly, to assess for documentation of at least 3 or more domains of toxicities of immune checkpoint inhibitors. The baseline comprehensive documentation rate was 50%. Our goal is to improve the documentation rate to 80% for our first process improvement cycle.

Aim Statement

Increase documentation of 3 or more toxicities immune checkpoint inhibitors to a goal rate of 80%.

Methods

A free online patient monitoring checklist tool was printed and provided to patients receiving immune checkpoint inhibitors (on their infusion day) during the check-in process. The patients were instructed to complete the tool prior to the provider clinic visit, while in the waiting area. The completed tool was given to the provider on the day of their visit. Prior to the start of this Plan-Do-Study-Act (PDSA) cycle, all providers were “reminded”/ instructed to ensure documentation of 3 or more toxicities immune checkpoint inhibitors. The cycle lasted for 3 weeks. At the end of the 3 weeks, our team reviewed the charts of those patients.

Results

Documentation rate of 3 or more toxicities increased from 50% to 90%.

Conclusions

When completed patient monitoring tools were provided to the providers during the clinic visit, the providers increased their documentation rate of the toxicities. There is literature supporting improving patient satisfaction using self-reported symptoms monitoring tools. Also, given the burden of documentation and shorter visit times, providers found this to be an easy way to address patient symptoms. While electronic patient-reported outcome (e-PRO) tools are ideal for ongoing symptom monitoring, this is a simple way to address the same in low-resourced communities. For our next cycle, we plan on using patient feedback to improve the documentation form incorporating larger fonts for patients with low vision.

Purpose

Ensuring that patients/families are engaged as partners in their health care is an effective way to measure the quality of patient care. Self-reported patient data, such as symptom burden, provides an accurate and effective way to measure patient-reported outcomes. Our team reviewed 20 patient charts, randomly, to assess for documentation of at least 3 or more domains of toxicities of immune checkpoint inhibitors. The baseline comprehensive documentation rate was 50%. Our goal is to improve the documentation rate to 80% for our first process improvement cycle.

Aim Statement

Increase documentation of 3 or more toxicities immune checkpoint inhibitors to a goal rate of 80%.

Methods

A free online patient monitoring checklist tool was printed and provided to patients receiving immune checkpoint inhibitors (on their infusion day) during the check-in process. The patients were instructed to complete the tool prior to the provider clinic visit, while in the waiting area. The completed tool was given to the provider on the day of their visit. Prior to the start of this Plan-Do-Study-Act (PDSA) cycle, all providers were “reminded”/ instructed to ensure documentation of 3 or more toxicities immune checkpoint inhibitors. The cycle lasted for 3 weeks. At the end of the 3 weeks, our team reviewed the charts of those patients.

Results

Documentation rate of 3 or more toxicities increased from 50% to 90%.

Conclusions

When completed patient monitoring tools were provided to the providers during the clinic visit, the providers increased their documentation rate of the toxicities. There is literature supporting improving patient satisfaction using self-reported symptoms monitoring tools. Also, given the burden of documentation and shorter visit times, providers found this to be an easy way to address patient symptoms. While electronic patient-reported outcome (e-PRO) tools are ideal for ongoing symptom monitoring, this is a simple way to address the same in low-resourced communities. For our next cycle, we plan on using patient feedback to improve the documentation form incorporating larger fonts for patients with low vision.

Veterans’ access to medical care was expanded outside of US Department of Veterans Affairs (VA) facilities with the inception of the 2014 Veterans Access, Choice, and Accountability Act (Choice Act).¹ This legislation aimed to remove barriers some veterans were experiencing, specifically access to health care. In subsequent years, approximately 17% of veterans receiving care from the VA did so under the Choice Act.² The Choice Act positively impacted medical care access for veterans but presented new challenges for VA pharmacies processing community care (CC) prescriptions, including limited access to outside health records, lack of interface between CC prescribers and the VA order entry system, and limited awareness of the VA national formulary.^3,4 These factors made it difficult for VA pharmacies to assess prescriptions for clinical appropriateness, evaluate patient safety parameters, and manage expenditures.

In 2019, the Maintaining Internal Systems and Strengthening Integrated Outside Networks (MISSION) Act, which expanded CC support and better defined which veterans are able to receive care outside the VA, updated the Choice Act.^4,5 However, VA pharmacies faced challenges in managing pharmacy drug costs and ensuring clinical appropriateness of prescription drug therapy. As a result, VA pharmacy departments have adjusted how they allocate workload, time, and funds.⁵

Pharmacists improve clinical outcomes and reduce health care costs by decreasing medication errors, unnecessary prescribing, and adverse drug events.^6-12 Pharmacist-driven formulary management through evaluation of prior authorization drug requests (PADRs) has shown economic value.^13,14 VA pharmacy review of community care PADRs is important because outside health care professionals (HCPs) might not be familiar with the VA formulary. This could lead to high volume of PADRs that do not meet criteria and could result in increased potential for medication misuse, adverse drug events, medication errors, and cost to the health system. It is imperative that CC orders are evaluated as critically as traditional orders.

The value of a centralized CC pharmacy team has not been assessed in the literature. The primary objective of this study was to assess the direct cost savings achieved through a centralized CC PADR process. Secondary objectives were to characterize the CC PADRs submitted to the site, including approval rate, reason for nonapproval, which medications were requested and by whom, and to compare CC prescriptions with other high-complexity (1a) VA facilities.

Community Care Pharmacy

VA health systems are stratified according to complexity, which reflects size, patient population, and services offered. This study was conducted at the Durham Veterans Affairs Health Care System (DVAHCS), North Carolina, a high-complexity, 251-bed, tertiary care referral, teaching, and research system. DVAHCS provides general and specialty medical, surgical, inpatient psychiatric, and ambulatory services, and serves as a major referral center.

DVAHCS created a centralized pharmacy team for processing CC prescriptions and managing customer service. This team’s goal is to increase CC prescription processing efficiency and transparency, ensure accountability of the health care team, and promote veteran-centric customer service. The pharmacy team includes a pharmacist program manager and a dedicated CC pharmacist with administrative support from a health benefits assistant and 4 pharmacy technicians. The CC pharmacy team assesses every new prescription to ensure the veteran is authorized to receive care in the community. Once eligibility is verified, a pharmacy technician or pharmacist evaluates the prescription to ensure it contains all required information, then contacts the prescriber for any missing data. If clinically appropriate, the pharmacist processes the prescription.

In 2020, the CC pharmacy team implemented a new process for reviewing and documenting CC prescriptions that require a PADR. The closed national VA formulary is set up so that all nonformulary medications and some formulary medications, including those that are restricted because of safety and/or cost, require a PADR.¹⁵ After a CC pharmacy technician confirms a veteran’s eligibility, the technician assesses whether the requested medication requires submitting a PADR to the VA internal electronic health record. The PADR is then adjudicated by a formulary management pharmacist, CC program manager, or CC pharmacist who reviews health records to determine whether the CC prescription meets VA medication use policy requirements.

If additional information is needed or an alternate medication is suggested, the pharmacist comments back on the PADR and a CC pharmacy technician contacts the prescriber. The PADR is canceled administratively then resubmitted once all information is obtained. While waiting for a response from the prescriber, the CC pharmacy technician contacts that veteran to give an update on the prescription status, as appropriate. Once there is sufficient information to adjudicate the PADR, the outcome is documented, and if approved, the order is processed.

Methods

The DVAHCS Institutional Review Board approved this retrospective review of CC PADRs submitted from June 1, 2020, through November 30, 2020. CC PADRs were excluded if they were duplicates or were reactivated administratively but had an initial submission date before the study period. Local data were collected for nonapproved CC PADRs including drug requested, dosage and directions, medication specialty, alternative drug recommended, drug acquisition cost, PADR submission date, PADR completion date, PADR nonapproval rationale, and documented time spent per PADR. Additional data was obtained for CC prescriptions at all 42 high-complexity VA facilities from the VA national CC prescription database for the study time interval and included total PADRs, PADR approval status, total CC prescription cost, and total CC fills.

Direct cost savings were calculated by assessing the cost of requested therapy that was not approved minus the cost of recommended therapy and cost to review all PADRs, as described by Britt and colleagues.¹³ The cost of the requested and recommended therapy was calculated based on VA drug acquisition cost at time of data collection and multiplied by the expected duration of therapy up to 1 year. For each CC prescription, duration of therapy was based on the duration limit in the prescription or annualized if no duration limit was documented. Cost of PADR review was calculated based on the total time pharmacists and pharmacy technicians documented for each step of the review process for a representative sample of 100 nonapproved PADRs and then multiplied by the salary plus benefits of an entry-level pharmacist and pharmacy technician.¹⁶ The eAppendix  describes specific equations used for determining direct cost savings. Descriptive statistics were used to evaluate study results.

Results

During the 6-month study period, 611 CC PADRs were submitted to the pharmacy and 526 met inclusion criteria (Figure 1). Of those, 243 (46.2%) were approved and 283 (53.8%) were not approved. The cost of requested therapies for nonapproved CC PADRs totaled $584,565.48 and the cost of all recommended therapies was $57,473.59. The mean time per CC PADR was 24 minutes; 16 minutes for pharmacists and 8 minutes for pharmacy technicians. Given an hourly wage (plus benefits) of $67.25 for a pharmacist and $25.53 for a pharmacy technician, the total cost of review per CC PADR was $21.33. After subtracting the costs of all recommended therapies and review of all included CC PADRs, the process generated $515,872.31 in direct cost savings. After factoring in administrative lag time, such as HCP communication, an average of 8 calendar days was needed to complete a nonapproved PADR.

The most common rationale for PADR nonapproval was that the formulary alternative was not exhausted. Ondansetron orally disintegrating tablets was the most commonly nonapproved medication and azelastine was the most commonly approved medication. Dulaglutide was the most expensive nonapproved and tafamidis was the most expensive approved PADR (Table 1). Gastroenterology, endocrinology, and neurology were the top specialties for nonapproved PADRs while neurology, pulmonology, and endocrinology were the top specialties for approved PADRs (Table 2).

Several high-complexity VA facilities had no reported data; we used the median for the analysis to account for these outliers (Figure 2). The median (IQR) adjudicated CC PADRs for all facilities was 97 (20-175), median (IQR) CC PADR approval rate was 80.9% (63.7%-96.8%), median (IQR) total CC prescriptions was 8440 (2464-14,466), and median (IQR) cost per fill was $136.05 ($76.27-$221.28).

Discussion

This study demonstrated direct cost savings of $515,872.31 over 6 months with theadjudication of CC PADRs by a centralized CC pharmacy team. This could result in > $1,000,000 of cost savings per fiscal year.

The CC PADRs observed at DVAHCS had a 46.2% approval rate; almost one-half the approval rate of 84.1% of all PADRs submitted to the study site by VA HCPs captured by Britt and colleagues.¹³ Results from this study showed that coordination of care for nonapproved CC PADRs between the VA pharmacy and non-VA prescriber took an average of 8 calendar days. The noted CC PADR approval rate and administrative burden might be because of lack of familiarity of non-VA providers regarding the VA national formulary. The National VA Pharmacy Benefits Management determines the formulary using cost-effectiveness criteria that considers the medical literature and VA-specific contract pricing and prepares extensive guidance for restricted medications via relevant criteria for use.¹⁵ HCPs outside the VA might not know this information is available online. Because gastroenterology, endocrinology, and neurology specialty medications were among the most frequently nonapproved PADRs, VA formulary education could begin with CC HCPs in these practice areas.

This study showed that the CC PADR process was not solely driven by cost, but also included patient safety. Nonapproval rationale for some requests included submission without an indication, submission by a prescriber that did not have the authority to prescribe a type of medication, or contraindication based on patient-specific factors.

Limitations

CC PADRs were assessed during the COVID-19 pandemic, which might have resulted in lower-than-normal CC prescription and PADR volumes, therefore underestimating the potential for direct cost savings. Entry-level salary was used to demonstrate cost savings potential from the perspective of a newly hired CC team; however, the cost savings might have been less if the actual salaries of site personnel were higher. National contract pricing data were gathered at the time of data collection and might have been different than at the time of PADR submission. Chronic medication prescriptions were annualized, which could overestimate cost savings if the medication was discontinued or changed to an alternative therapy within that time period.

The study’s exclusion criteria could only be applied locally and did not include data received from the VA CC prescription database. This can be seen by the discrepancy in CC PADR approval rates from the local and national data (46.2% vs 44.2%, respectively) and CC PADR volume. High-complexity VA facility data were captured without assessing the CC prescription process at each site. As a result, definitive conclusions cannot be made regarding the impact of a centralized CC pharmacy team compared with other facilities.

Conclusions

Adjudication of CC PADRs by a centralized CC pharmacy team over a 6-month period provided > $500,000 in direct cost savings to a VA health care system. Considering the CC PADR approval rate seen in this study, the VA could allocate resources to educate CC providers about the VA formulary to increase the PADR approval rate and reduce administrative burden for VA pharmacies and prescribers. Future research should evaluate CC prescription handling practices at other VA facilities to compare the effectiveness among varying approaches and develop recommendations for a nationally standardized process.

Acknowledgments

Concept and design (AJJ, JNB, RBB, LAM, MD, MGH); acquisition of data (AJJ, MGH); analysis and interpretation of data (AJJ, JNB, RBB, LAM, MD, MGH); drafting of the manuscript (AJJ); critical revision of the manuscript for important intellectual content (AJJ, JNB, RBB, LAM, MD, MGH); statistical analysis (AJJ); administrative, technical, or logistic support (LAM, MGH); and supervision (MGH).

Veterans’ access to medical care was expanded outside of US Department of Veterans Affairs (VA) facilities with the inception of the 2014 Veterans Access, Choice, and Accountability Act (Choice Act).¹ This legislation aimed to remove barriers some veterans were experiencing, specifically access to health care. In subsequent years, approximately 17% of veterans receiving care from the VA did so under the Choice Act.² The Choice Act positively impacted medical care access for veterans but presented new challenges for VA pharmacies processing community care (CC) prescriptions, including limited access to outside health records, lack of interface between CC prescribers and the VA order entry system, and limited awareness of the VA national formulary.^3,4 These factors made it difficult for VA pharmacies to assess prescriptions for clinical appropriateness, evaluate patient safety parameters, and manage expenditures.

In 2019, the Maintaining Internal Systems and Strengthening Integrated Outside Networks (MISSION) Act, which expanded CC support and better defined which veterans are able to receive care outside the VA, updated the Choice Act.^4,5 However, VA pharmacies faced challenges in managing pharmacy drug costs and ensuring clinical appropriateness of prescription drug therapy. As a result, VA pharmacy departments have adjusted how they allocate workload, time, and funds.⁵

Pharmacists improve clinical outcomes and reduce health care costs by decreasing medication errors, unnecessary prescribing, and adverse drug events.^6-12 Pharmacist-driven formulary management through evaluation of prior authorization drug requests (PADRs) has shown economic value.^13,14 VA pharmacy review of community care PADRs is important because outside health care professionals (HCPs) might not be familiar with the VA formulary. This could lead to high volume of PADRs that do not meet criteria and could result in increased potential for medication misuse, adverse drug events, medication errors, and cost to the health system. It is imperative that CC orders are evaluated as critically as traditional orders.

The value of a centralized CC pharmacy team has not been assessed in the literature. The primary objective of this study was to assess the direct cost savings achieved through a centralized CC PADR process. Secondary objectives were to characterize the CC PADRs submitted to the site, including approval rate, reason for nonapproval, which medications were requested and by whom, and to compare CC prescriptions with other high-complexity (1a) VA facilities.

Community Care Pharmacy

VA health systems are stratified according to complexity, which reflects size, patient population, and services offered. This study was conducted at the Durham Veterans Affairs Health Care System (DVAHCS), North Carolina, a high-complexity, 251-bed, tertiary care referral, teaching, and research system. DVAHCS provides general and specialty medical, surgical, inpatient psychiatric, and ambulatory services, and serves as a major referral center.

DVAHCS created a centralized pharmacy team for processing CC prescriptions and managing customer service. This team’s goal is to increase CC prescription processing efficiency and transparency, ensure accountability of the health care team, and promote veteran-centric customer service. The pharmacy team includes a pharmacist program manager and a dedicated CC pharmacist with administrative support from a health benefits assistant and 4 pharmacy technicians. The CC pharmacy team assesses every new prescription to ensure the veteran is authorized to receive care in the community. Once eligibility is verified, a pharmacy technician or pharmacist evaluates the prescription to ensure it contains all required information, then contacts the prescriber for any missing data. If clinically appropriate, the pharmacist processes the prescription.

In 2020, the CC pharmacy team implemented a new process for reviewing and documenting CC prescriptions that require a PADR. The closed national VA formulary is set up so that all nonformulary medications and some formulary medications, including those that are restricted because of safety and/or cost, require a PADR.¹⁵ After a CC pharmacy technician confirms a veteran’s eligibility, the technician assesses whether the requested medication requires submitting a PADR to the VA internal electronic health record. The PADR is then adjudicated by a formulary management pharmacist, CC program manager, or CC pharmacist who reviews health records to determine whether the CC prescription meets VA medication use policy requirements.

If additional information is needed or an alternate medication is suggested, the pharmacist comments back on the PADR and a CC pharmacy technician contacts the prescriber. The PADR is canceled administratively then resubmitted once all information is obtained. While waiting for a response from the prescriber, the CC pharmacy technician contacts that veteran to give an update on the prescription status, as appropriate. Once there is sufficient information to adjudicate the PADR, the outcome is documented, and if approved, the order is processed.

Methods

The DVAHCS Institutional Review Board approved this retrospective review of CC PADRs submitted from June 1, 2020, through November 30, 2020. CC PADRs were excluded if they were duplicates or were reactivated administratively but had an initial submission date before the study period. Local data were collected for nonapproved CC PADRs including drug requested, dosage and directions, medication specialty, alternative drug recommended, drug acquisition cost, PADR submission date, PADR completion date, PADR nonapproval rationale, and documented time spent per PADR. Additional data was obtained for CC prescriptions at all 42 high-complexity VA facilities from the VA national CC prescription database for the study time interval and included total PADRs, PADR approval status, total CC prescription cost, and total CC fills.

Direct cost savings were calculated by assessing the cost of requested therapy that was not approved minus the cost of recommended therapy and cost to review all PADRs, as described by Britt and colleagues.¹³ The cost of the requested and recommended therapy was calculated based on VA drug acquisition cost at time of data collection and multiplied by the expected duration of therapy up to 1 year. For each CC prescription, duration of therapy was based on the duration limit in the prescription or annualized if no duration limit was documented. Cost of PADR review was calculated based on the total time pharmacists and pharmacy technicians documented for each step of the review process for a representative sample of 100 nonapproved PADRs and then multiplied by the salary plus benefits of an entry-level pharmacist and pharmacy technician.¹⁶ The eAppendix  describes specific equations used for determining direct cost savings. Descriptive statistics were used to evaluate study results.

Results

During the 6-month study period, 611 CC PADRs were submitted to the pharmacy and 526 met inclusion criteria (Figure 1). Of those, 243 (46.2%) were approved and 283 (53.8%) were not approved. The cost of requested therapies for nonapproved CC PADRs totaled $584,565.48 and the cost of all recommended therapies was $57,473.59. The mean time per CC PADR was 24 minutes; 16 minutes for pharmacists and 8 minutes for pharmacy technicians. Given an hourly wage (plus benefits) of $67.25 for a pharmacist and $25.53 for a pharmacy technician, the total cost of review per CC PADR was $21.33. After subtracting the costs of all recommended therapies and review of all included CC PADRs, the process generated $515,872.31 in direct cost savings. After factoring in administrative lag time, such as HCP communication, an average of 8 calendar days was needed to complete a nonapproved PADR.

The most common rationale for PADR nonapproval was that the formulary alternative was not exhausted. Ondansetron orally disintegrating tablets was the most commonly nonapproved medication and azelastine was the most commonly approved medication. Dulaglutide was the most expensive nonapproved and tafamidis was the most expensive approved PADR (Table 1). Gastroenterology, endocrinology, and neurology were the top specialties for nonapproved PADRs while neurology, pulmonology, and endocrinology were the top specialties for approved PADRs (Table 2).

Several high-complexity VA facilities had no reported data; we used the median for the analysis to account for these outliers (Figure 2). The median (IQR) adjudicated CC PADRs for all facilities was 97 (20-175), median (IQR) CC PADR approval rate was 80.9% (63.7%-96.8%), median (IQR) total CC prescriptions was 8440 (2464-14,466), and median (IQR) cost per fill was $136.05 ($76.27-$221.28).

Discussion

This study demonstrated direct cost savings of $515,872.31 over 6 months with theadjudication of CC PADRs by a centralized CC pharmacy team. This could result in > $1,000,000 of cost savings per fiscal year.

The CC PADRs observed at DVAHCS had a 46.2% approval rate; almost one-half the approval rate of 84.1% of all PADRs submitted to the study site by VA HCPs captured by Britt and colleagues.¹³ Results from this study showed that coordination of care for nonapproved CC PADRs between the VA pharmacy and non-VA prescriber took an average of 8 calendar days. The noted CC PADR approval rate and administrative burden might be because of lack of familiarity of non-VA providers regarding the VA national formulary. The National VA Pharmacy Benefits Management determines the formulary using cost-effectiveness criteria that considers the medical literature and VA-specific contract pricing and prepares extensive guidance for restricted medications via relevant criteria for use.¹⁵ HCPs outside the VA might not know this information is available online. Because gastroenterology, endocrinology, and neurology specialty medications were among the most frequently nonapproved PADRs, VA formulary education could begin with CC HCPs in these practice areas.

This study showed that the CC PADR process was not solely driven by cost, but also included patient safety. Nonapproval rationale for some requests included submission without an indication, submission by a prescriber that did not have the authority to prescribe a type of medication, or contraindication based on patient-specific factors.

Limitations

CC PADRs were assessed during the COVID-19 pandemic, which might have resulted in lower-than-normal CC prescription and PADR volumes, therefore underestimating the potential for direct cost savings. Entry-level salary was used to demonstrate cost savings potential from the perspective of a newly hired CC team; however, the cost savings might have been less if the actual salaries of site personnel were higher. National contract pricing data were gathered at the time of data collection and might have been different than at the time of PADR submission. Chronic medication prescriptions were annualized, which could overestimate cost savings if the medication was discontinued or changed to an alternative therapy within that time period.

The study’s exclusion criteria could only be applied locally and did not include data received from the VA CC prescription database. This can be seen by the discrepancy in CC PADR approval rates from the local and national data (46.2% vs 44.2%, respectively) and CC PADR volume. High-complexity VA facility data were captured without assessing the CC prescription process at each site. As a result, definitive conclusions cannot be made regarding the impact of a centralized CC pharmacy team compared with other facilities.

Conclusions

Adjudication of CC PADRs by a centralized CC pharmacy team over a 6-month period provided > $500,000 in direct cost savings to a VA health care system. Considering the CC PADR approval rate seen in this study, the VA could allocate resources to educate CC providers about the VA formulary to increase the PADR approval rate and reduce administrative burden for VA pharmacies and prescribers. Future research should evaluate CC prescription handling practices at other VA facilities to compare the effectiveness among varying approaches and develop recommendations for a nationally standardized process.

Acknowledgments

Concept and design (AJJ, JNB, RBB, LAM, MD, MGH); acquisition of data (AJJ, MGH); analysis and interpretation of data (AJJ, JNB, RBB, LAM, MD, MGH); drafting of the manuscript (AJJ); critical revision of the manuscript for important intellectual content (AJJ, JNB, RBB, LAM, MD, MGH); statistical analysis (AJJ); administrative, technical, or logistic support (LAM, MGH); and supervision (MGH).

Veterans’ access to medical care was expanded outside of US Department of Veterans Affairs (VA) facilities with the inception of the 2014 Veterans Access, Choice, and Accountability Act (Choice Act).¹ This legislation aimed to remove barriers some veterans were experiencing, specifically access to health care. In subsequent years, approximately 17% of veterans receiving care from the VA did so under the Choice Act.² The Choice Act positively impacted medical care access for veterans but presented new challenges for VA pharmacies processing community care (CC) prescriptions, including limited access to outside health records, lack of interface between CC prescribers and the VA order entry system, and limited awareness of the VA national formulary.^3,4 These factors made it difficult for VA pharmacies to assess prescriptions for clinical appropriateness, evaluate patient safety parameters, and manage expenditures.

In 2019, the Maintaining Internal Systems and Strengthening Integrated Outside Networks (MISSION) Act, which expanded CC support and better defined which veterans are able to receive care outside the VA, updated the Choice Act.^4,5 However, VA pharmacies faced challenges in managing pharmacy drug costs and ensuring clinical appropriateness of prescription drug therapy. As a result, VA pharmacy departments have adjusted how they allocate workload, time, and funds.⁵

Pharmacists improve clinical outcomes and reduce health care costs by decreasing medication errors, unnecessary prescribing, and adverse drug events.^6-12 Pharmacist-driven formulary management through evaluation of prior authorization drug requests (PADRs) has shown economic value.^13,14 VA pharmacy review of community care PADRs is important because outside health care professionals (HCPs) might not be familiar with the VA formulary. This could lead to high volume of PADRs that do not meet criteria and could result in increased potential for medication misuse, adverse drug events, medication errors, and cost to the health system. It is imperative that CC orders are evaluated as critically as traditional orders.

The value of a centralized CC pharmacy team has not been assessed in the literature. The primary objective of this study was to assess the direct cost savings achieved through a centralized CC PADR process. Secondary objectives were to characterize the CC PADRs submitted to the site, including approval rate, reason for nonapproval, which medications were requested and by whom, and to compare CC prescriptions with other high-complexity (1a) VA facilities.

Community Care Pharmacy

VA health systems are stratified according to complexity, which reflects size, patient population, and services offered. This study was conducted at the Durham Veterans Affairs Health Care System (DVAHCS), North Carolina, a high-complexity, 251-bed, tertiary care referral, teaching, and research system. DVAHCS provides general and specialty medical, surgical, inpatient psychiatric, and ambulatory services, and serves as a major referral center.

DVAHCS created a centralized pharmacy team for processing CC prescriptions and managing customer service. This team’s goal is to increase CC prescription processing efficiency and transparency, ensure accountability of the health care team, and promote veteran-centric customer service. The pharmacy team includes a pharmacist program manager and a dedicated CC pharmacist with administrative support from a health benefits assistant and 4 pharmacy technicians. The CC pharmacy team assesses every new prescription to ensure the veteran is authorized to receive care in the community. Once eligibility is verified, a pharmacy technician or pharmacist evaluates the prescription to ensure it contains all required information, then contacts the prescriber for any missing data. If clinically appropriate, the pharmacist processes the prescription.

In 2020, the CC pharmacy team implemented a new process for reviewing and documenting CC prescriptions that require a PADR. The closed national VA formulary is set up so that all nonformulary medications and some formulary medications, including those that are restricted because of safety and/or cost, require a PADR.¹⁵ After a CC pharmacy technician confirms a veteran’s eligibility, the technician assesses whether the requested medication requires submitting a PADR to the VA internal electronic health record. The PADR is then adjudicated by a formulary management pharmacist, CC program manager, or CC pharmacist who reviews health records to determine whether the CC prescription meets VA medication use policy requirements.

If additional information is needed or an alternate medication is suggested, the pharmacist comments back on the PADR and a CC pharmacy technician contacts the prescriber. The PADR is canceled administratively then resubmitted once all information is obtained. While waiting for a response from the prescriber, the CC pharmacy technician contacts that veteran to give an update on the prescription status, as appropriate. Once there is sufficient information to adjudicate the PADR, the outcome is documented, and if approved, the order is processed.

Methods

The DVAHCS Institutional Review Board approved this retrospective review of CC PADRs submitted from June 1, 2020, through November 30, 2020. CC PADRs were excluded if they were duplicates or were reactivated administratively but had an initial submission date before the study period. Local data were collected for nonapproved CC PADRs including drug requested, dosage and directions, medication specialty, alternative drug recommended, drug acquisition cost, PADR submission date, PADR completion date, PADR nonapproval rationale, and documented time spent per PADR. Additional data was obtained for CC prescriptions at all 42 high-complexity VA facilities from the VA national CC prescription database for the study time interval and included total PADRs, PADR approval status, total CC prescription cost, and total CC fills.

Direct cost savings were calculated by assessing the cost of requested therapy that was not approved minus the cost of recommended therapy and cost to review all PADRs, as described by Britt and colleagues.¹³ The cost of the requested and recommended therapy was calculated based on VA drug acquisition cost at time of data collection and multiplied by the expected duration of therapy up to 1 year. For each CC prescription, duration of therapy was based on the duration limit in the prescription or annualized if no duration limit was documented. Cost of PADR review was calculated based on the total time pharmacists and pharmacy technicians documented for each step of the review process for a representative sample of 100 nonapproved PADRs and then multiplied by the salary plus benefits of an entry-level pharmacist and pharmacy technician.¹⁶ The eAppendix  describes specific equations used for determining direct cost savings. Descriptive statistics were used to evaluate study results.

Results

During the 6-month study period, 611 CC PADRs were submitted to the pharmacy and 526 met inclusion criteria (Figure 1). Of those, 243 (46.2%) were approved and 283 (53.8%) were not approved. The cost of requested therapies for nonapproved CC PADRs totaled $584,565.48 and the cost of all recommended therapies was $57,473.59. The mean time per CC PADR was 24 minutes; 16 minutes for pharmacists and 8 minutes for pharmacy technicians. Given an hourly wage (plus benefits) of $67.25 for a pharmacist and $25.53 for a pharmacy technician, the total cost of review per CC PADR was $21.33. After subtracting the costs of all recommended therapies and review of all included CC PADRs, the process generated $515,872.31 in direct cost savings. After factoring in administrative lag time, such as HCP communication, an average of 8 calendar days was needed to complete a nonapproved PADR.

The most common rationale for PADR nonapproval was that the formulary alternative was not exhausted. Ondansetron orally disintegrating tablets was the most commonly nonapproved medication and azelastine was the most commonly approved medication. Dulaglutide was the most expensive nonapproved and tafamidis was the most expensive approved PADR (Table 1). Gastroenterology, endocrinology, and neurology were the top specialties for nonapproved PADRs while neurology, pulmonology, and endocrinology were the top specialties for approved PADRs (Table 2).

Several high-complexity VA facilities had no reported data; we used the median for the analysis to account for these outliers (Figure 2). The median (IQR) adjudicated CC PADRs for all facilities was 97 (20-175), median (IQR) CC PADR approval rate was 80.9% (63.7%-96.8%), median (IQR) total CC prescriptions was 8440 (2464-14,466), and median (IQR) cost per fill was $136.05 ($76.27-$221.28).

Discussion

This study demonstrated direct cost savings of $515,872.31 over 6 months with theadjudication of CC PADRs by a centralized CC pharmacy team. This could result in > $1,000,000 of cost savings per fiscal year.

The CC PADRs observed at DVAHCS had a 46.2% approval rate; almost one-half the approval rate of 84.1% of all PADRs submitted to the study site by VA HCPs captured by Britt and colleagues.¹³ Results from this study showed that coordination of care for nonapproved CC PADRs between the VA pharmacy and non-VA prescriber took an average of 8 calendar days. The noted CC PADR approval rate and administrative burden might be because of lack of familiarity of non-VA providers regarding the VA national formulary. The National VA Pharmacy Benefits Management determines the formulary using cost-effectiveness criteria that considers the medical literature and VA-specific contract pricing and prepares extensive guidance for restricted medications via relevant criteria for use.¹⁵ HCPs outside the VA might not know this information is available online. Because gastroenterology, endocrinology, and neurology specialty medications were among the most frequently nonapproved PADRs, VA formulary education could begin with CC HCPs in these practice areas.

This study showed that the CC PADR process was not solely driven by cost, but also included patient safety. Nonapproval rationale for some requests included submission without an indication, submission by a prescriber that did not have the authority to prescribe a type of medication, or contraindication based on patient-specific factors.

Limitations

CC PADRs were assessed during the COVID-19 pandemic, which might have resulted in lower-than-normal CC prescription and PADR volumes, therefore underestimating the potential for direct cost savings. Entry-level salary was used to demonstrate cost savings potential from the perspective of a newly hired CC team; however, the cost savings might have been less if the actual salaries of site personnel were higher. National contract pricing data were gathered at the time of data collection and might have been different than at the time of PADR submission. Chronic medication prescriptions were annualized, which could overestimate cost savings if the medication was discontinued or changed to an alternative therapy within that time period.

The study’s exclusion criteria could only be applied locally and did not include data received from the VA CC prescription database. This can be seen by the discrepancy in CC PADR approval rates from the local and national data (46.2% vs 44.2%, respectively) and CC PADR volume. High-complexity VA facility data were captured without assessing the CC prescription process at each site. As a result, definitive conclusions cannot be made regarding the impact of a centralized CC pharmacy team compared with other facilities.

Conclusions

Adjudication of CC PADRs by a centralized CC pharmacy team over a 6-month period provided > $500,000 in direct cost savings to a VA health care system. Considering the CC PADR approval rate seen in this study, the VA could allocate resources to educate CC providers about the VA formulary to increase the PADR approval rate and reduce administrative burden for VA pharmacies and prescribers. Future research should evaluate CC prescription handling practices at other VA facilities to compare the effectiveness among varying approaches and develop recommendations for a nationally standardized process.

Acknowledgments

Concept and design (AJJ, JNB, RBB, LAM, MD, MGH); acquisition of data (AJJ, MGH); analysis and interpretation of data (AJJ, JNB, RBB, LAM, MD, MGH); drafting of the manuscript (AJJ); critical revision of the manuscript for important intellectual content (AJJ, JNB, RBB, LAM, MD, MGH); statistical analysis (AJJ); administrative, technical, or logistic support (LAM, MGH); and supervision (MGH).

The negative impact of the unnecessary prescribing of antibiotic is well known. Consequences include exposing patients to antibiotic adverse effects, risk of overgrowth of pathogenetic organisms such as clostridial species, unnecessary cost of drugs, and development of selection of antibiotic-resistant organisms in the populace at large. Acute viral respiratory infections are among the leading causes of inappropriate antibiotic usage.¹ In a study of 1000 adults with respiratory tract infections in an outpatient setting, 77% of patients were prescribed antibiotics, and the treatment was inappropriate in 64% of those who received prescriptions.² Patient expectations and clinician perceptions of these expectations play a role. One study showed that 54% of clinicians felt their patients expected to receive antibiotics for a visit due to an acute respiratory infection (ARI), such as a cough or cold; 26% of patients did in fact have this expectation.³

The US Department of Veterans Affairs (VA) Central Ohio Health Care System is a large ambulatory care facility, with 4 associated community-based outpatient clinics, serving more than 43,000 central Ohio veterans and completing more than 500,000 medical appointments annually. An antimicrobial stewardship program has been in place since 2013. In May 2018, the clinical pharmacist assigned to the program alerted medical leadership that, of 67 patients seen in primary care for ARIs between April 16, 2018, and May 15, 2018, 42 (63%) had been prescribed an antibiotic. Based on this finding, clinical leadership designed a process improvement program aimed at reducing inappropriate antibiotic usage for the treatment of uncomplicated ARls likely due to viral pathogens. Key components were clinician and patient education and the substitution of a symptomatic treatment kit in place of an antibiotic prescription.

Methods

Facility clinical leadership, assisted by Volunteer Services, developed a Viral Illness Support Pack to be dispensed by primary care practitioners (PCPs) to patients presenting with symptoms of viral ARIs. The contents of this support pack are shown in the Figure. Patients were provided with tissues, throat lozenges, lip balm, acetaminophen, hand sanitizer, a surgical mask, patient instructions, and the Antibiotics Aren’t Always the Answer pamphlet.⁴ The contents of the viral support pack were purchased through Volunteer Services using donated funds. In total, 460 packs were distributed to the primary care patient aligned care teams (PACTs), including the community-based outpatient clinics.

Clinicians and care teams received academic detailing prior to distribution of the viral support packs, stressing the importance of avoiding antibiotics to treat viral illnesses. Viral illness support packs were available for distribution from December 1, 2018, through March 31, 2019. The frequency of antibiotic dispensing to patients coded for ARI during this period was compared with that of the same time period in the previous year. All charts were reviewed for coding accuracy. Patients with illnesses requiring antibiotic treatment, such as pneumonia, exacerbations of chronic obstructive pulmonary disease and chronic bronchitis, and streptococcal pharyngitis, were excluded from the study. Statistical significance was determined using the unpaired t test.

Results

From December 1, 2018, to March 31, 2019, 357 viral support packs were distributed to patients (Table). For the historical control period from December 1, 2017, through March 31, 2018, 508 patients were treated for ARIs. Of these, 295 (58%) received clinically inappropriate antibiotics. In contrast, of the 627 patients treated for ARIs during the study period from December 1, 2018, through March 31, 2019, 310 (49%) received clinically inappropriate antibiotics. The 9% decrease during the period when viral support packs were distributed, compared with the prior year, was statistically significant (P = .02).

Discussion

The decrease in antibiotic prescriptions for ARIs was statistically significant. The success of this project can be attributed to 3 factors: clinician education, patient education, and the option for PCPs to provide symptomatic treatment for these patients rather than prescribe an antibiotic.

The importance of antibiotic stewardship has been emphasized to all PCPs at the VA Central Ohio Health Care System. Antibiotic stewardship has been the subject of grand rounds. Prior to distribution of the viral support pack, the chief of specialty medicine, the project’s champion, spoke to all PCPs. Adequate numbers of viral support packs were distributed to all primary care teams.

In addition to direct clinician-to-patient education at the time of the patients’ visits, educational materials were included in the viral support pack. The Antibiotics Aren’t Always the Answer pamphlet is available from the Centers for Disease Control and Prevention. It covers the importance of antibiotic awareness, discusses what antibiotics do and do not treat, how to stay healthy, and causes of antibiotic resistance. The pamphlet contains the clear message that antibiotics are not only ineffective against viral illness, but also can cause significant undesirable outcomes.

The pamphlet Viral Illness Support Pack Traffic Light Card (eAppendix available online at doi:10.12788/fp.0302) provides important clinical information to the patients about their illness. Patients are instructed to contact their primary care team if they are worse after 3 days of illness; symptoms are not improving after 10 days; or they experience blood in respiratory secretions, chills or generalized aching, and localized pain that is one-sided or significantly worsening. Patients are clearly informed to seek further care if not improving with symptomatic treatment.

The ability to provide patients with symptomatic relief, including throat lozenges, lip balm, and acetaminophen, was felt to be important in the success of the project. Furthermore, this eliminated an extra step of the patient needing to visit the pharmacy.

Limitations

Limitations of the study included starting distribution of the support packs somewhat after the onset of the viral illness season, failure to reach all prescribers for academic detailing at the start of the program, and several instances of temporary unavailability of the support packs in some areas.

Conclusions

Patients with ARIs are often significantly symptomatic and frequently believe that they require an antibiotic for treatment. Clinicians may adjust their behavior in response to their patients’ expectations, stated or unstated. The results of this project demonstrate that the combination of patient education and the ready availability of a nonantibiotic symptomatic treatment option can significantly decrease the unnecessary prescribing of antibiotics for viral illnesses.

Acknowledgments

The authors are grateful to Ms. Traci Washington for assistance in sourcing materials; to Karen Corr, PhD, and Anthony Restuccio, MD, for advice on methods; to Mr. Daniel Pignatelli for assistance with data interpretation; and to Mr. Keith Skidmore, Ms. Crystal Conley, and Ms. Megan Harris for assistance with assembling the Viral Illness Support Packs.

The negative impact of the unnecessary prescribing of antibiotic is well known. Consequences include exposing patients to antibiotic adverse effects, risk of overgrowth of pathogenetic organisms such as clostridial species, unnecessary cost of drugs, and development of selection of antibiotic-resistant organisms in the populace at large. Acute viral respiratory infections are among the leading causes of inappropriate antibiotic usage.¹ In a study of 1000 adults with respiratory tract infections in an outpatient setting, 77% of patients were prescribed antibiotics, and the treatment was inappropriate in 64% of those who received prescriptions.² Patient expectations and clinician perceptions of these expectations play a role. One study showed that 54% of clinicians felt their patients expected to receive antibiotics for a visit due to an acute respiratory infection (ARI), such as a cough or cold; 26% of patients did in fact have this expectation.³

The US Department of Veterans Affairs (VA) Central Ohio Health Care System is a large ambulatory care facility, with 4 associated community-based outpatient clinics, serving more than 43,000 central Ohio veterans and completing more than 500,000 medical appointments annually. An antimicrobial stewardship program has been in place since 2013. In May 2018, the clinical pharmacist assigned to the program alerted medical leadership that, of 67 patients seen in primary care for ARIs between April 16, 2018, and May 15, 2018, 42 (63%) had been prescribed an antibiotic. Based on this finding, clinical leadership designed a process improvement program aimed at reducing inappropriate antibiotic usage for the treatment of uncomplicated ARls likely due to viral pathogens. Key components were clinician and patient education and the substitution of a symptomatic treatment kit in place of an antibiotic prescription.

Methods

Facility clinical leadership, assisted by Volunteer Services, developed a Viral Illness Support Pack to be dispensed by primary care practitioners (PCPs) to patients presenting with symptoms of viral ARIs. The contents of this support pack are shown in the Figure. Patients were provided with tissues, throat lozenges, lip balm, acetaminophen, hand sanitizer, a surgical mask, patient instructions, and the Antibiotics Aren’t Always the Answer pamphlet.⁴ The contents of the viral support pack were purchased through Volunteer Services using donated funds. In total, 460 packs were distributed to the primary care patient aligned care teams (PACTs), including the community-based outpatient clinics.

Clinicians and care teams received academic detailing prior to distribution of the viral support packs, stressing the importance of avoiding antibiotics to treat viral illnesses. Viral illness support packs were available for distribution from December 1, 2018, through March 31, 2019. The frequency of antibiotic dispensing to patients coded for ARI during this period was compared with that of the same time period in the previous year. All charts were reviewed for coding accuracy. Patients with illnesses requiring antibiotic treatment, such as pneumonia, exacerbations of chronic obstructive pulmonary disease and chronic bronchitis, and streptococcal pharyngitis, were excluded from the study. Statistical significance was determined using the unpaired t test.

Results

From December 1, 2018, to March 31, 2019, 357 viral support packs were distributed to patients (Table). For the historical control period from December 1, 2017, through March 31, 2018, 508 patients were treated for ARIs. Of these, 295 (58%) received clinically inappropriate antibiotics. In contrast, of the 627 patients treated for ARIs during the study period from December 1, 2018, through March 31, 2019, 310 (49%) received clinically inappropriate antibiotics. The 9% decrease during the period when viral support packs were distributed, compared with the prior year, was statistically significant (P = .02).

Discussion

The decrease in antibiotic prescriptions for ARIs was statistically significant. The success of this project can be attributed to 3 factors: clinician education, patient education, and the option for PCPs to provide symptomatic treatment for these patients rather than prescribe an antibiotic.

The importance of antibiotic stewardship has been emphasized to all PCPs at the VA Central Ohio Health Care System. Antibiotic stewardship has been the subject of grand rounds. Prior to distribution of the viral support pack, the chief of specialty medicine, the project’s champion, spoke to all PCPs. Adequate numbers of viral support packs were distributed to all primary care teams.

In addition to direct clinician-to-patient education at the time of the patients’ visits, educational materials were included in the viral support pack. The Antibiotics Aren’t Always the Answer pamphlet is available from the Centers for Disease Control and Prevention. It covers the importance of antibiotic awareness, discusses what antibiotics do and do not treat, how to stay healthy, and causes of antibiotic resistance. The pamphlet contains the clear message that antibiotics are not only ineffective against viral illness, but also can cause significant undesirable outcomes.

The pamphlet Viral Illness Support Pack Traffic Light Card (eAppendix available online at doi:10.12788/fp.0302) provides important clinical information to the patients about their illness. Patients are instructed to contact their primary care team if they are worse after 3 days of illness; symptoms are not improving after 10 days; or they experience blood in respiratory secretions, chills or generalized aching, and localized pain that is one-sided or significantly worsening. Patients are clearly informed to seek further care if not improving with symptomatic treatment.

The ability to provide patients with symptomatic relief, including throat lozenges, lip balm, and acetaminophen, was felt to be important in the success of the project. Furthermore, this eliminated an extra step of the patient needing to visit the pharmacy.

Limitations

Limitations of the study included starting distribution of the support packs somewhat after the onset of the viral illness season, failure to reach all prescribers for academic detailing at the start of the program, and several instances of temporary unavailability of the support packs in some areas.

Conclusions

Patients with ARIs are often significantly symptomatic and frequently believe that they require an antibiotic for treatment. Clinicians may adjust their behavior in response to their patients’ expectations, stated or unstated. The results of this project demonstrate that the combination of patient education and the ready availability of a nonantibiotic symptomatic treatment option can significantly decrease the unnecessary prescribing of antibiotics for viral illnesses.

Acknowledgments

The authors are grateful to Ms. Traci Washington for assistance in sourcing materials; to Karen Corr, PhD, and Anthony Restuccio, MD, for advice on methods; to Mr. Daniel Pignatelli for assistance with data interpretation; and to Mr. Keith Skidmore, Ms. Crystal Conley, and Ms. Megan Harris for assistance with assembling the Viral Illness Support Packs.

The negative impact of the unnecessary prescribing of antibiotic is well known. Consequences include exposing patients to antibiotic adverse effects, risk of overgrowth of pathogenetic organisms such as clostridial species, unnecessary cost of drugs, and development of selection of antibiotic-resistant organisms in the populace at large. Acute viral respiratory infections are among the leading causes of inappropriate antibiotic usage.¹ In a study of 1000 adults with respiratory tract infections in an outpatient setting, 77% of patients were prescribed antibiotics, and the treatment was inappropriate in 64% of those who received prescriptions.² Patient expectations and clinician perceptions of these expectations play a role. One study showed that 54% of clinicians felt their patients expected to receive antibiotics for a visit due to an acute respiratory infection (ARI), such as a cough or cold; 26% of patients did in fact have this expectation.³

The US Department of Veterans Affairs (VA) Central Ohio Health Care System is a large ambulatory care facility, with 4 associated community-based outpatient clinics, serving more than 43,000 central Ohio veterans and completing more than 500,000 medical appointments annually. An antimicrobial stewardship program has been in place since 2013. In May 2018, the clinical pharmacist assigned to the program alerted medical leadership that, of 67 patients seen in primary care for ARIs between April 16, 2018, and May 15, 2018, 42 (63%) had been prescribed an antibiotic. Based on this finding, clinical leadership designed a process improvement program aimed at reducing inappropriate antibiotic usage for the treatment of uncomplicated ARls likely due to viral pathogens. Key components were clinician and patient education and the substitution of a symptomatic treatment kit in place of an antibiotic prescription.

Methods

Facility clinical leadership, assisted by Volunteer Services, developed a Viral Illness Support Pack to be dispensed by primary care practitioners (PCPs) to patients presenting with symptoms of viral ARIs. The contents of this support pack are shown in the Figure. Patients were provided with tissues, throat lozenges, lip balm, acetaminophen, hand sanitizer, a surgical mask, patient instructions, and the Antibiotics Aren’t Always the Answer pamphlet.⁴ The contents of the viral support pack were purchased through Volunteer Services using donated funds. In total, 460 packs were distributed to the primary care patient aligned care teams (PACTs), including the community-based outpatient clinics.

Clinicians and care teams received academic detailing prior to distribution of the viral support packs, stressing the importance of avoiding antibiotics to treat viral illnesses. Viral illness support packs were available for distribution from December 1, 2018, through March 31, 2019. The frequency of antibiotic dispensing to patients coded for ARI during this period was compared with that of the same time period in the previous year. All charts were reviewed for coding accuracy. Patients with illnesses requiring antibiotic treatment, such as pneumonia, exacerbations of chronic obstructive pulmonary disease and chronic bronchitis, and streptococcal pharyngitis, were excluded from the study. Statistical significance was determined using the unpaired t test.

Results

From December 1, 2018, to March 31, 2019, 357 viral support packs were distributed to patients (Table). For the historical control period from December 1, 2017, through March 31, 2018, 508 patients were treated for ARIs. Of these, 295 (58%) received clinically inappropriate antibiotics. In contrast, of the 627 patients treated for ARIs during the study period from December 1, 2018, through March 31, 2019, 310 (49%) received clinically inappropriate antibiotics. The 9% decrease during the period when viral support packs were distributed, compared with the prior year, was statistically significant (P = .02).

Discussion

The decrease in antibiotic prescriptions for ARIs was statistically significant. The success of this project can be attributed to 3 factors: clinician education, patient education, and the option for PCPs to provide symptomatic treatment for these patients rather than prescribe an antibiotic.

The importance of antibiotic stewardship has been emphasized to all PCPs at the VA Central Ohio Health Care System. Antibiotic stewardship has been the subject of grand rounds. Prior to distribution of the viral support pack, the chief of specialty medicine, the project’s champion, spoke to all PCPs. Adequate numbers of viral support packs were distributed to all primary care teams.

In addition to direct clinician-to-patient education at the time of the patients’ visits, educational materials were included in the viral support pack. The Antibiotics Aren’t Always the Answer pamphlet is available from the Centers for Disease Control and Prevention. It covers the importance of antibiotic awareness, discusses what antibiotics do and do not treat, how to stay healthy, and causes of antibiotic resistance. The pamphlet contains the clear message that antibiotics are not only ineffective against viral illness, but also can cause significant undesirable outcomes.

The pamphlet Viral Illness Support Pack Traffic Light Card (eAppendix available online at doi:10.12788/fp.0302) provides important clinical information to the patients about their illness. Patients are instructed to contact their primary care team if they are worse after 3 days of illness; symptoms are not improving after 10 days; or they experience blood in respiratory secretions, chills or generalized aching, and localized pain that is one-sided or significantly worsening. Patients are clearly informed to seek further care if not improving with symptomatic treatment.

The ability to provide patients with symptomatic relief, including throat lozenges, lip balm, and acetaminophen, was felt to be important in the success of the project. Furthermore, this eliminated an extra step of the patient needing to visit the pharmacy.

Limitations

Limitations of the study included starting distribution of the support packs somewhat after the onset of the viral illness season, failure to reach all prescribers for academic detailing at the start of the program, and several instances of temporary unavailability of the support packs in some areas.

Conclusions

Patients with ARIs are often significantly symptomatic and frequently believe that they require an antibiotic for treatment. Clinicians may adjust their behavior in response to their patients’ expectations, stated or unstated. The results of this project demonstrate that the combination of patient education and the ready availability of a nonantibiotic symptomatic treatment option can significantly decrease the unnecessary prescribing of antibiotics for viral illnesses.

Acknowledgments

The authors are grateful to Ms. Traci Washington for assistance in sourcing materials; to Karen Corr, PhD, and Anthony Restuccio, MD, for advice on methods; to Mr. Daniel Pignatelli for assistance with data interpretation; and to Mr. Keith Skidmore, Ms. Crystal Conley, and Ms. Megan Harris for assistance with assembling the Viral Illness Support Packs.

In the past 10 years, artificial intelligence (AI) applications have exploded in numerous fields, including medicine. Myriad publications report that the use of AI in health care is increasing, and AI has shown utility in many medical specialties, eg, pathology, radiology, and oncology.^1,2

In cancer pathology, AI was able not only to detect various cancers, but also to subtype and grade them. In addition, AI could predict survival, the success of therapeutic response, and underlying mutations from histopathologic images.³ In other medical fields, AI applications are as notable. For example, in imaging specialties like radiology, ophthalmology, dermatology, and gastroenterology, AI is being used for image recognition, enhancement, and segmentation. In addition, AI is beneficial for predicting disease progression, survival, and response to therapy in other medical specialties. Finally, AI may help with administrative tasks like scheduling.

However, many obstacles to successfully implementing AI programs in the clinical setting exist, including clinical data limitations and ethical use of data, trust in the AI models, regulatory barriers, and lack of clinical buy-in due to insufficient basic AI understanding.² To address these barriers to successful clinical AI implementation, we decided to create a formal governing body at James A. Haley Veterans’ Hospital in Tampa, Florida. Accordingly, the hospital AI committee charter was officially approved on July 22, 2021. Our model could be used by both US Department of Veterans Affairs (VA) and non-VA hospitals throughout the country.

AI Committee

The vision of the AI committee is to improve outcomes and experiences for our veterans by developing trustworthy AI capabilities to support the VA mission. The mission is to build robust capacity in AI to create and apply innovative AI solutions and transform the VA by facilitating a learning environment that supports the delivery of world-class benefits and services to our veterans. Our vision and mission are aligned with the VA National AI Institute. ⁴

The AI Committee comprises 7 subcommittees: ethics, AI clinical product evaluation, education, data sharing and acquisition, research, 3D printing, and improvement and innovation. The role of the ethics subcommittee is to ensure the ethical and equitable implementation of clinical AI. We created the ethics subcommittee guidelines based on the World Health Organization ethics and governance of AI for health documents.⁵ They include 6 basic principles: protecting human autonomy; promoting human well-being and safety and the public interest; ensuring transparency, explainability, and intelligibility; fostering responsibility and accountability; ensuring inclusiveness and equity; and promoting AI that is responsive and sustainable (Table 1).

Implementations

After a comprehensive evaluation, we implemented 2 ClearRead (Riverain Technologies) AI radiology solutions. ClearRead CT Vessel Suppress produces a secondary series of computed tomography (CT) images, suppressing vessels and other normal structures within the lungs to improve nodule detectability, and ClearRead Xray Bone Suppress, which increases the visibility of soft tissue in standard chest X-rays by suppressing the bone on the digital image without the need for 2 exposures.

The role of the education subcommittee is to educate the staff about AI and how it can improve patient care. Every Friday, we email an AI article of the week to our practitioners. In addition, we publish a newsletter, and we organize an annual AI conference. The first conference in 2022 included speakers from the National AI Institute, Moffitt Cancer Center, the University of South Florida, and our facility.

As the name indicates, the data sharing and acquisition subcommittee oversees preparing data for our clinical and research projects. The role of the research subcommittee is to coordinate and promote AI research with the ultimate goal of improving patient care.

Other Technologies

Although 3D printing does not fall under the umbrella of AI, we have decided to include it in our future-oriented AI committee. We created an online 3D printing course to promote the technology throughout the VA. We 3D print organ models to help surgeons prepare for complicated operations. In addition, together with our colleagues from the University of Florida, we used 3D printing to address the shortage of swabs for COVID-19 testing. The VA Sunshine Healthcare Network (Veterans Integrated Services Network 8) has an active Innovation and Improvement Committee. ⁹ Our improvement and innovation subcommittee serves as a coordinating body with the network committee .

Conclusions

Through the hospital AI committee, we believe that we may overcome many obstacles to successfully implementing AI applications in the clinical setting, including the ethical use of data, trust in the AI models, regulatory barriers, and lack of clinical buy-in due to insufficient basic AI knowledge.

Acknowledgments

This material is the result of work supported with resources and the use of facilities at the James A. Haley Veterans’ Hospital.

In the past 10 years, artificial intelligence (AI) applications have exploded in numerous fields, including medicine. Myriad publications report that the use of AI in health care is increasing, and AI has shown utility in many medical specialties, eg, pathology, radiology, and oncology.^1,2

In cancer pathology, AI was able not only to detect various cancers, but also to subtype and grade them. In addition, AI could predict survival, the success of therapeutic response, and underlying mutations from histopathologic images.³ In other medical fields, AI applications are as notable. For example, in imaging specialties like radiology, ophthalmology, dermatology, and gastroenterology, AI is being used for image recognition, enhancement, and segmentation. In addition, AI is beneficial for predicting disease progression, survival, and response to therapy in other medical specialties. Finally, AI may help with administrative tasks like scheduling.

However, many obstacles to successfully implementing AI programs in the clinical setting exist, including clinical data limitations and ethical use of data, trust in the AI models, regulatory barriers, and lack of clinical buy-in due to insufficient basic AI understanding.² To address these barriers to successful clinical AI implementation, we decided to create a formal governing body at James A. Haley Veterans’ Hospital in Tampa, Florida. Accordingly, the hospital AI committee charter was officially approved on July 22, 2021. Our model could be used by both US Department of Veterans Affairs (VA) and non-VA hospitals throughout the country.

AI Committee

The vision of the AI committee is to improve outcomes and experiences for our veterans by developing trustworthy AI capabilities to support the VA mission. The mission is to build robust capacity in AI to create and apply innovative AI solutions and transform the VA by facilitating a learning environment that supports the delivery of world-class benefits and services to our veterans. Our vision and mission are aligned with the VA National AI Institute. ⁴

The AI Committee comprises 7 subcommittees: ethics, AI clinical product evaluation, education, data sharing and acquisition, research, 3D printing, and improvement and innovation. The role of the ethics subcommittee is to ensure the ethical and equitable implementation of clinical AI. We created the ethics subcommittee guidelines based on the World Health Organization ethics and governance of AI for health documents.⁵ They include 6 basic principles: protecting human autonomy; promoting human well-being and safety and the public interest; ensuring transparency, explainability, and intelligibility; fostering responsibility and accountability; ensuring inclusiveness and equity; and promoting AI that is responsive and sustainable (Table 1).

Implementations

After a comprehensive evaluation, we implemented 2 ClearRead (Riverain Technologies) AI radiology solutions. ClearRead CT Vessel Suppress produces a secondary series of computed tomography (CT) images, suppressing vessels and other normal structures within the lungs to improve nodule detectability, and ClearRead Xray Bone Suppress, which increases the visibility of soft tissue in standard chest X-rays by suppressing the bone on the digital image without the need for 2 exposures.

The role of the education subcommittee is to educate the staff about AI and how it can improve patient care. Every Friday, we email an AI article of the week to our practitioners. In addition, we publish a newsletter, and we organize an annual AI conference. The first conference in 2022 included speakers from the National AI Institute, Moffitt Cancer Center, the University of South Florida, and our facility.

As the name indicates, the data sharing and acquisition subcommittee oversees preparing data for our clinical and research projects. The role of the research subcommittee is to coordinate and promote AI research with the ultimate goal of improving patient care.

Other Technologies

Although 3D printing does not fall under the umbrella of AI, we have decided to include it in our future-oriented AI committee. We created an online 3D printing course to promote the technology throughout the VA. We 3D print organ models to help surgeons prepare for complicated operations. In addition, together with our colleagues from the University of Florida, we used 3D printing to address the shortage of swabs for COVID-19 testing. The VA Sunshine Healthcare Network (Veterans Integrated Services Network 8) has an active Innovation and Improvement Committee. ⁹ Our improvement and innovation subcommittee serves as a coordinating body with the network committee .

Conclusions

Through the hospital AI committee, we believe that we may overcome many obstacles to successfully implementing AI applications in the clinical setting, including the ethical use of data, trust in the AI models, regulatory barriers, and lack of clinical buy-in due to insufficient basic AI knowledge.

Acknowledgments

This material is the result of work supported with resources and the use of facilities at the James A. Haley Veterans’ Hospital.

In the past 10 years, artificial intelligence (AI) applications have exploded in numerous fields, including medicine. Myriad publications report that the use of AI in health care is increasing, and AI has shown utility in many medical specialties, eg, pathology, radiology, and oncology.^1,2

In cancer pathology, AI was able not only to detect various cancers, but also to subtype and grade them. In addition, AI could predict survival, the success of therapeutic response, and underlying mutations from histopathologic images.³ In other medical fields, AI applications are as notable. For example, in imaging specialties like radiology, ophthalmology, dermatology, and gastroenterology, AI is being used for image recognition, enhancement, and segmentation. In addition, AI is beneficial for predicting disease progression, survival, and response to therapy in other medical specialties. Finally, AI may help with administrative tasks like scheduling.

However, many obstacles to successfully implementing AI programs in the clinical setting exist, including clinical data limitations and ethical use of data, trust in the AI models, regulatory barriers, and lack of clinical buy-in due to insufficient basic AI understanding.² To address these barriers to successful clinical AI implementation, we decided to create a formal governing body at James A. Haley Veterans’ Hospital in Tampa, Florida. Accordingly, the hospital AI committee charter was officially approved on July 22, 2021. Our model could be used by both US Department of Veterans Affairs (VA) and non-VA hospitals throughout the country.

AI Committee

The vision of the AI committee is to improve outcomes and experiences for our veterans by developing trustworthy AI capabilities to support the VA mission. The mission is to build robust capacity in AI to create and apply innovative AI solutions and transform the VA by facilitating a learning environment that supports the delivery of world-class benefits and services to our veterans. Our vision and mission are aligned with the VA National AI Institute. ⁴

The AI Committee comprises 7 subcommittees: ethics, AI clinical product evaluation, education, data sharing and acquisition, research, 3D printing, and improvement and innovation. The role of the ethics subcommittee is to ensure the ethical and equitable implementation of clinical AI. We created the ethics subcommittee guidelines based on the World Health Organization ethics and governance of AI for health documents.⁵ They include 6 basic principles: protecting human autonomy; promoting human well-being and safety and the public interest; ensuring transparency, explainability, and intelligibility; fostering responsibility and accountability; ensuring inclusiveness and equity; and promoting AI that is responsive and sustainable (Table 1).

Implementations

After a comprehensive evaluation, we implemented 2 ClearRead (Riverain Technologies) AI radiology solutions. ClearRead CT Vessel Suppress produces a secondary series of computed tomography (CT) images, suppressing vessels and other normal structures within the lungs to improve nodule detectability, and ClearRead Xray Bone Suppress, which increases the visibility of soft tissue in standard chest X-rays by suppressing the bone on the digital image without the need for 2 exposures.

The role of the education subcommittee is to educate the staff about AI and how it can improve patient care. Every Friday, we email an AI article of the week to our practitioners. In addition, we publish a newsletter, and we organize an annual AI conference. The first conference in 2022 included speakers from the National AI Institute, Moffitt Cancer Center, the University of South Florida, and our facility.

As the name indicates, the data sharing and acquisition subcommittee oversees preparing data for our clinical and research projects. The role of the research subcommittee is to coordinate and promote AI research with the ultimate goal of improving patient care.

Other Technologies

Although 3D printing does not fall under the umbrella of AI, we have decided to include it in our future-oriented AI committee. We created an online 3D printing course to promote the technology throughout the VA. We 3D print organ models to help surgeons prepare for complicated operations. In addition, together with our colleagues from the University of Florida, we used 3D printing to address the shortage of swabs for COVID-19 testing. The VA Sunshine Healthcare Network (Veterans Integrated Services Network 8) has an active Innovation and Improvement Committee. ⁹ Our improvement and innovation subcommittee serves as a coordinating body with the network committee .

Conclusions

Through the hospital AI committee, we believe that we may overcome many obstacles to successfully implementing AI applications in the clinical setting, including the ethical use of data, trust in the AI models, regulatory barriers, and lack of clinical buy-in due to insufficient basic AI knowledge.

Acknowledgments

This material is the result of work supported with resources and the use of facilities at the James A. Haley Veterans’ Hospital.

Dr. George Sakoulas is an infectious diseases clinician at Sharp Memorial Hospital in San Diego and professor of pediatrics at the University of California, San Diego School of Medicine. He was the lead investigator in a study published in the May/June 2022 issue of JCOM that found that, when allocated to the appropriate patient type, intravenous immunoglobulin can reduce hospital costs for COVID-19 care. ¹ He joined JCOM’s Editor-in-Chief, Dr. Ebrahim Barkoudah, to discuss the study’s background and highlight its main findings.

The following has been edited for length and clarity.

Dr. Barkoudah Dr. Sakoulas is an investigator and a clinician, bridging both worlds to bring the best evidence to our patients. We’re discussing his new article regarding intravenous immunoglobulin in treating nonventilated COVID-19 patients with moderate-to-severe hypoxia. Dr. Sakoulas, could you please share with our readers the clinical question your study addressed and what your work around COVID-19 management means for clinical practice?

Dr. Sakoulas Thank you. I’m an infectious disease physician. I’ve been treating patients with viral acute respiratory distress syndrome for almost 20 years as an ID doctor. Most of these cases are due to influenza or other viruses. And from time to time, anecdotally and supported by some literature, we’ve been using IVIG, or intravenous immunoglobulin, in some of these cases. And again, I can report anecdotal success with that over the years.

So when COVID emerged in March of 2020, we deployed IVIG in a couple of patients early who were heading downhill. Remember, in March of 2020, we didn’t have the knowledge of steroids helping, patients being ventilated very promptly, and we saw some patients who made a turnaround after treatment with IVIG. We were able to get some support from an industry sponsor and perform and publish a pilot study, enrolling patients early in the pandemic. That study actually showed benefits, which then led the sponsor to fund a phase 3 multicenter clinical trial. Unfortunately, a couple of things happened. First, the trial was designed with the knowledge we had in April of 2020, and again, this is before steroids, before we incorporated proning patients in the ICU, or started ventilating people early. So there were some management changes and evolutions and improvements that happened. And second, the trial was enrolling a very broad repertoire of patients. There were no age limitations, and the trial, ultimately a phase 3 multicenter trial, failed to meet its endpoint.

There were some trends for benefit in younger patients, and as the trial was ongoing, we continued to evolve our knowledge, and we really honed it down to seeing a benefit of using IVIG in patients with COVID with specific criteria in mind. They had to be relatively younger patients, under 65, and not have any major comorbidities. In other words, they weren’t dialysis patients or end-stage disease patients, heart failure patients, cancer or malignancy patients. So, you know, we’re looking at the patients under 65 with obesity, diabetes, and hypertension, who are rapidly declining, going from room air to BiPAP or high-flow oxygen in a short amount of time. And we learned that when using IVIG early, we actually saw patients improve and turn around.

What this article in JCOM highlighted was, number one, incorporating that outcome or that patient type and then looking at the cost of hospitalization of patients who received IVIG versus those that did not. There were 2 groups that were studied. One was the group of patients in that original pilot trial that I discussed who were randomized to receive 1 or the other prospectively; it was an unblinded randomized study. And the second group was a matched case-control study where we had patients treated with IVIG matched by age and comorbidity status and level of hypoxia to patients that did not receive IVIG. We saw a financial benefit in shortening or reducing hospitalizations, really coming down to getting rid of that 20% tail of patients that wound up going to the ICU, getting intubated, and using a high amount of hospital resources that would ramp up the cost of hospitalization. We saw great mitigation of that with IVIG, and even with a small subset of patients, we were able to show a benefit.

Dr. Barkoudah Any thoughts on where we can implement the new findings from your article in our practice at the moment, knowing we now have practice guidelines and protocols to treat COVID-19? There was a tangible benefit in treating the patients the way you approached it in your important work. Could you share with us what would be implementable at the moment?

Dr. Sakoulas I think, fortunately, with the increasing host immunity in the population and decreased virulence of the virus, perhaps we won’t see as many patients of the type that were in these trials going forward, but I suspect we will perhaps in the unvaccinated patients that remain. I believe one-third of the United States is not vaccinated. So there is certainly a vulnerable group of people out there. Potentially, an unvaccinated patient who winds up getting very sick, the patient who is relatively young—what I’m looking at is the 30- to 65-year-old obese, hypertensive, or diabetic patient who comes in and, despite the steroids and the antivirals, rapidly deteriorates into requiring high-flow oxygen. I think implementing IVIG in that patient type would be helpful. I don’t think it’s going to be as helpful in patients who are very elderly, because I think the mechanism of the disease is different in an 80-year-old versus a 50-year-old patient. So again, hopefully, it will not amount to a lot of patients, but I still suspect hospitals are going to see, perhaps in the fall, when they’re expecting a greater number of cases, a trickling of patients that do meet the criteria that I described.

Dr. Barkoudah JCOM’s audience are the QI implementers and hospital leadership. And what caught my eye in your article is your perspective on the pharmacoeconomics of treating COVID-19, and I really appreciate your looking at the cost aspect. Would you talk about the economics of inpatient care, the total care that we provide now that we’re in the age of tocilizumab, and the current state of multiple layers of therapy?

Dr. Sakoulas The reason to look at the economics of it is because IVIG—which is actually not a drug, it’s a blood product—is very expensive. So, we received a considerable amount of administrative pushback implementing this treatment at the beginning outside of the clinical trial setting because it hadn’t been studied on a large scale and because the cost was so high, even though, as a clinician at the bedside, I was seeing a benefit in patients. This study came out of my trying to demonstrate to the folks that are keeping the economics of medicine in mind that, in fact, investing several thousand dollars of treatment in IVIG will save you cost of care, the cost of an ICU bed, the cost of a ventilator, and the cost even of ECMO, which is hugely expensive.

If you look at the numbers in the study, for two-thirds or three-quarters of the patients, your cost of care is actually greater than the controls because you’re giving them IVIG, and it’s increasing the cost of their care, even though three-quarters of the patients are going to do just as well without it. It’s that 20% to 25% of patients that really are going to benefit from it, where you’re reducing your cost of care so much, and you’re getting rid of that very, very expensive 20%, that there’s a cost savings across the board per patient. So, it’s hard to understand when you say you’re losing money on three-quarters of the patients, you’re only saving money on a quarter of the patients, but that cost of saving on that small subset is so substantial it’s really impacting all numbers.

Also, abandoning the outlier principle is sort of an underlying theme in how we think of things. We tend to ignore outliers, not consider them, but I think we really have to pay attention to the more extreme cases because those patients are the ones that drive not just the financial cost of care. Remember, if you’re down to 1 ventilator and you can cut down the use of scarce ICU resources, the cost is sort of even beyond the cost of money. It’s the cost of resources that may become scarce in some settings. So, I think it speaks to that as well.

A lot of the drugs that we use, for example, tocilizumab, were able to be studied in thousands of patients. If you look at the absolute numbers, the benefit of tocilizumab from a magnitude standpoint—low to mid twenties to high twenties—you know, reducing mortality from 29% to 24%. I mean, just take a step back and think about that. Even though it’s statistically significant, try telling a patient, “Well, I’m going to give you this treatment that’s going to reduce mortality from 29% to 24%.” You know, that doesn’t really change anything from a clinical significance standpoint. But they have a P value less than .05, which is our standard, and they were able to do a study with thousands of patients. We didn’t have that luxury with IVIG. No one studied thousands of patients, only retrospectively, and those retrospective studies don’t get the attention because they’re considered biased with all their limitations. But I think one of the difficulties we have here is the balance between statistical and clinical significance. For example, in our pilot study, our ventilation rate was 58% with the non-IVIG patients versus 14% for IVIG patients. So you might say, magnitude-wise, that’s a big number, but the statistical significance of it is borderline because of small numbers.

Anyway, that’s a challenge that we have as clinicians trying to incorporate what’s published—the balancing of statistics, absolute numbers, and practicalities of delivering care. And I think this study highlights some of the nuances that go into that incorporation and those clinical decisions.

Dr. Barkoudah Would you mind sharing with our audience how we can make the connection between the medical outcomes and pharmacoeconomics findings from your article and link it to the bedside and treatment of our patients?

Dr. Sakoulas One of the points this article brings out is the importance of bringing together not just level 1A data, but also small studies with data such as this, where the magnitude of the effect is pretty big but you lose the statistics because of the small numbers. And then also the patients’ aspects of things. I think, as a bedside clinician, you appreciate things, the nuances, much sooner than what percolates out from a level 1A study. Case in point, in the sponsored phase 3 study that we did, and in some other studies that were prospectively done as well, these studies of IVIG simply had an enrollment of patients that was very broad, and not every patient benefits from the same therapy. A great example of this is the sepsis trials with Xigris and those types of agents that failed. You know, there are clinicians to this day who believe that there is a subset of patients that benefit from agents like this. The IVIG story falls a little bit into that category. It comes down to trying to identify the subset of patients that might benefit. And I think we’ve outlined this subset pretty well in our study: the younger, obese diabetic or hypertensive patient who’s rapidly declining.

It really brings together the need to not necessarily toss out these smaller studies, but kind of summarize everything together, and clinicians who are bedside, who are more in tune with the nuances of individual decisions at the individual patient level, might better appreciate these kinds of data. But I think we all have to put it together. IVIG does not make treatment guidelines at national levels and so forth. It’s not even listed in many of them. But there are patients out there who, if you ask them specifically how they felt, including a friend of mine who received the medication, there’s no question from their end, how they felt about this treatment option. Now, some people will get it and will not benefit. We just have to be really tuned into the fact that the same drug does not have the same result for every patient. And just to consider this in the high-risk patients that we talked about in our study.

Dr. Barkoudah While we were prepping for this interview, you made an analogy regarding clinical evidence along the lines of, “Do we need randomized clinical trials to do a parachute-type of experiment,” and we chatted about clinical wisdom. Would you mind sharing with our readers your thoughts on that?

Dr. Sakoulas Sometimes, we try a treatment and it’s very obvious for that particular patient that it helped them. Then you study the treatment in a large trial setting and it doesn’t work. For us bedside clinicians, there are some interventions sometimes that do appear as beneficial as a parachute would be, but yet, there has never been a randomized clinical trial proving that parachutes work. Again, a part of the challenge we have is patients are so different, their immunology is different, the pathogen infecting them is different, the time they present is different. Some present early, some present late. There are just so many moving parts to treating an infection that only a subset of people are going to benefit. And sometimes as clinicians, we’re so nuanced, that we identify a specific subset of patients where we know we can help them. And it’s so obvious for us, like a parachute would be, but to people who are looking at the world from 30,000 feet, they don’t necessarily grasp that because, when you look at all comers, it doesn’t show a benefit.

So the problem is that now those treatments that might help a subset of patients are being denied, and the subset of patients that are going to benefit never get the treatment. Now we have to balance that with a lot of stuff that went on during the pandemic with, you know, ivermectin, hydroxychloroquine, and people pushing those things. Someone asked me once what I thought about hydroxychloroquine, and I said, “Well, somebody in the lab probably showed that it was beneficial, analogous to lighting tissue paper on fire on a plate and taking a cup of water and putting the fire out. Well, now, if you take that cup of water to the Caldor fire that’s burning in California on thousands of acres, you’re not going to be able to put the fire out with that cup of water.” So while it might work in the lab, it’s truly not going to work in a clinical setting. We have to balance individualizing care for patients with some information people are pushing out there that may not be necessarily translatable to the clinical setting.

I think there’s nothing better than being at the bedside, though, and being able to implement something and seeing what works. And really, experience goes a long way in being able to individually treat a patient optimally.

Dr. Barkoudah Thank you for everything you do at the bedside and your work on improving the treatment we have and how we can leverage knowledge to treat our patients. Thank you very much for your time and your scholarly contribution. We appreciate it and I hope the work will continue. We will keep working on treating COVID-19 patients with the best knowledge we have.

Q&A participants: George Sakoulas, MD, Sharp Rees-Stealy Medical Group, La Jolla, CA, and University of California San Diego School of Medicine, San Diego, CA; and Ebrahim Barkoudah, MD, MPH, Department of Medicine, Brigham and Women’s Hospital, Boston, MA.

Disclosures: None reported.

Dr. George Sakoulas is an infectious diseases clinician at Sharp Memorial Hospital in San Diego and professor of pediatrics at the University of California, San Diego School of Medicine. He was the lead investigator in a study published in the May/June 2022 issue of JCOM that found that, when allocated to the appropriate patient type, intravenous immunoglobulin can reduce hospital costs for COVID-19 care. ¹ He joined JCOM’s Editor-in-Chief, Dr. Ebrahim Barkoudah, to discuss the study’s background and highlight its main findings.

The following has been edited for length and clarity.

Dr. Barkoudah Dr. Sakoulas is an investigator and a clinician, bridging both worlds to bring the best evidence to our patients. We’re discussing his new article regarding intravenous immunoglobulin in treating nonventilated COVID-19 patients with moderate-to-severe hypoxia. Dr. Sakoulas, could you please share with our readers the clinical question your study addressed and what your work around COVID-19 management means for clinical practice?

Dr. Sakoulas Thank you. I’m an infectious disease physician. I’ve been treating patients with viral acute respiratory distress syndrome for almost 20 years as an ID doctor. Most of these cases are due to influenza or other viruses. And from time to time, anecdotally and supported by some literature, we’ve been using IVIG, or intravenous immunoglobulin, in some of these cases. And again, I can report anecdotal success with that over the years.

So when COVID emerged in March of 2020, we deployed IVIG in a couple of patients early who were heading downhill. Remember, in March of 2020, we didn’t have the knowledge of steroids helping, patients being ventilated very promptly, and we saw some patients who made a turnaround after treatment with IVIG. We were able to get some support from an industry sponsor and perform and publish a pilot study, enrolling patients early in the pandemic. That study actually showed benefits, which then led the sponsor to fund a phase 3 multicenter clinical trial. Unfortunately, a couple of things happened. First, the trial was designed with the knowledge we had in April of 2020, and again, this is before steroids, before we incorporated proning patients in the ICU, or started ventilating people early. So there were some management changes and evolutions and improvements that happened. And second, the trial was enrolling a very broad repertoire of patients. There were no age limitations, and the trial, ultimately a phase 3 multicenter trial, failed to meet its endpoint.

There were some trends for benefit in younger patients, and as the trial was ongoing, we continued to evolve our knowledge, and we really honed it down to seeing a benefit of using IVIG in patients with COVID with specific criteria in mind. They had to be relatively younger patients, under 65, and not have any major comorbidities. In other words, they weren’t dialysis patients or end-stage disease patients, heart failure patients, cancer or malignancy patients. So, you know, we’re looking at the patients under 65 with obesity, diabetes, and hypertension, who are rapidly declining, going from room air to BiPAP or high-flow oxygen in a short amount of time. And we learned that when using IVIG early, we actually saw patients improve and turn around.

What this article in JCOM highlighted was, number one, incorporating that outcome or that patient type and then looking at the cost of hospitalization of patients who received IVIG versus those that did not. There were 2 groups that were studied. One was the group of patients in that original pilot trial that I discussed who were randomized to receive 1 or the other prospectively; it was an unblinded randomized study. And the second group was a matched case-control study where we had patients treated with IVIG matched by age and comorbidity status and level of hypoxia to patients that did not receive IVIG. We saw a financial benefit in shortening or reducing hospitalizations, really coming down to getting rid of that 20% tail of patients that wound up going to the ICU, getting intubated, and using a high amount of hospital resources that would ramp up the cost of hospitalization. We saw great mitigation of that with IVIG, and even with a small subset of patients, we were able to show a benefit.

Dr. Barkoudah Any thoughts on where we can implement the new findings from your article in our practice at the moment, knowing we now have practice guidelines and protocols to treat COVID-19? There was a tangible benefit in treating the patients the way you approached it in your important work. Could you share with us what would be implementable at the moment?

Dr. Sakoulas I think, fortunately, with the increasing host immunity in the population and decreased virulence of the virus, perhaps we won’t see as many patients of the type that were in these trials going forward, but I suspect we will perhaps in the unvaccinated patients that remain. I believe one-third of the United States is not vaccinated. So there is certainly a vulnerable group of people out there. Potentially, an unvaccinated patient who winds up getting very sick, the patient who is relatively young—what I’m looking at is the 30- to 65-year-old obese, hypertensive, or diabetic patient who comes in and, despite the steroids and the antivirals, rapidly deteriorates into requiring high-flow oxygen. I think implementing IVIG in that patient type would be helpful. I don’t think it’s going to be as helpful in patients who are very elderly, because I think the mechanism of the disease is different in an 80-year-old versus a 50-year-old patient. So again, hopefully, it will not amount to a lot of patients, but I still suspect hospitals are going to see, perhaps in the fall, when they’re expecting a greater number of cases, a trickling of patients that do meet the criteria that I described.

Dr. Barkoudah JCOM’s audience are the QI implementers and hospital leadership. And what caught my eye in your article is your perspective on the pharmacoeconomics of treating COVID-19, and I really appreciate your looking at the cost aspect. Would you talk about the economics of inpatient care, the total care that we provide now that we’re in the age of tocilizumab, and the current state of multiple layers of therapy?

Dr. Sakoulas The reason to look at the economics of it is because IVIG—which is actually not a drug, it’s a blood product—is very expensive. So, we received a considerable amount of administrative pushback implementing this treatment at the beginning outside of the clinical trial setting because it hadn’t been studied on a large scale and because the cost was so high, even though, as a clinician at the bedside, I was seeing a benefit in patients. This study came out of my trying to demonstrate to the folks that are keeping the economics of medicine in mind that, in fact, investing several thousand dollars of treatment in IVIG will save you cost of care, the cost of an ICU bed, the cost of a ventilator, and the cost even of ECMO, which is hugely expensive.

If you look at the numbers in the study, for two-thirds or three-quarters of the patients, your cost of care is actually greater than the controls because you’re giving them IVIG, and it’s increasing the cost of their care, even though three-quarters of the patients are going to do just as well without it. It’s that 20% to 25% of patients that really are going to benefit from it, where you’re reducing your cost of care so much, and you’re getting rid of that very, very expensive 20%, that there’s a cost savings across the board per patient. So, it’s hard to understand when you say you’re losing money on three-quarters of the patients, you’re only saving money on a quarter of the patients, but that cost of saving on that small subset is so substantial it’s really impacting all numbers.

Also, abandoning the outlier principle is sort of an underlying theme in how we think of things. We tend to ignore outliers, not consider them, but I think we really have to pay attention to the more extreme cases because those patients are the ones that drive not just the financial cost of care. Remember, if you’re down to 1 ventilator and you can cut down the use of scarce ICU resources, the cost is sort of even beyond the cost of money. It’s the cost of resources that may become scarce in some settings. So, I think it speaks to that as well.

A lot of the drugs that we use, for example, tocilizumab, were able to be studied in thousands of patients. If you look at the absolute numbers, the benefit of tocilizumab from a magnitude standpoint—low to mid twenties to high twenties—you know, reducing mortality from 29% to 24%. I mean, just take a step back and think about that. Even though it’s statistically significant, try telling a patient, “Well, I’m going to give you this treatment that’s going to reduce mortality from 29% to 24%.” You know, that doesn’t really change anything from a clinical significance standpoint. But they have a P value less than .05, which is our standard, and they were able to do a study with thousands of patients. We didn’t have that luxury with IVIG. No one studied thousands of patients, only retrospectively, and those retrospective studies don’t get the attention because they’re considered biased with all their limitations. But I think one of the difficulties we have here is the balance between statistical and clinical significance. For example, in our pilot study, our ventilation rate was 58% with the non-IVIG patients versus 14% for IVIG patients. So you might say, magnitude-wise, that’s a big number, but the statistical significance of it is borderline because of small numbers.

Anyway, that’s a challenge that we have as clinicians trying to incorporate what’s published—the balancing of statistics, absolute numbers, and practicalities of delivering care. And I think this study highlights some of the nuances that go into that incorporation and those clinical decisions.

Dr. Barkoudah Would you mind sharing with our audience how we can make the connection between the medical outcomes and pharmacoeconomics findings from your article and link it to the bedside and treatment of our patients?

Dr. Sakoulas One of the points this article brings out is the importance of bringing together not just level 1A data, but also small studies with data such as this, where the magnitude of the effect is pretty big but you lose the statistics because of the small numbers. And then also the patients’ aspects of things. I think, as a bedside clinician, you appreciate things, the nuances, much sooner than what percolates out from a level 1A study. Case in point, in the sponsored phase 3 study that we did, and in some other studies that were prospectively done as well, these studies of IVIG simply had an enrollment of patients that was very broad, and not every patient benefits from the same therapy. A great example of this is the sepsis trials with Xigris and those types of agents that failed. You know, there are clinicians to this day who believe that there is a subset of patients that benefit from agents like this. The IVIG story falls a little bit into that category. It comes down to trying to identify the subset of patients that might benefit. And I think we’ve outlined this subset pretty well in our study: the younger, obese diabetic or hypertensive patient who’s rapidly declining.

It really brings together the need to not necessarily toss out these smaller studies, but kind of summarize everything together, and clinicians who are bedside, who are more in tune with the nuances of individual decisions at the individual patient level, might better appreciate these kinds of data. But I think we all have to put it together. IVIG does not make treatment guidelines at national levels and so forth. It’s not even listed in many of them. But there are patients out there who, if you ask them specifically how they felt, including a friend of mine who received the medication, there’s no question from their end, how they felt about this treatment option. Now, some people will get it and will not benefit. We just have to be really tuned into the fact that the same drug does not have the same result for every patient. And just to consider this in the high-risk patients that we talked about in our study.

Dr. Barkoudah While we were prepping for this interview, you made an analogy regarding clinical evidence along the lines of, “Do we need randomized clinical trials to do a parachute-type of experiment,” and we chatted about clinical wisdom. Would you mind sharing with our readers your thoughts on that?

Dr. Sakoulas Sometimes, we try a treatment and it’s very obvious for that particular patient that it helped them. Then you study the treatment in a large trial setting and it doesn’t work. For us bedside clinicians, there are some interventions sometimes that do appear as beneficial as a parachute would be, but yet, there has never been a randomized clinical trial proving that parachutes work. Again, a part of the challenge we have is patients are so different, their immunology is different, the pathogen infecting them is different, the time they present is different. Some present early, some present late. There are just so many moving parts to treating an infection that only a subset of people are going to benefit. And sometimes as clinicians, we’re so nuanced, that we identify a specific subset of patients where we know we can help them. And it’s so obvious for us, like a parachute would be, but to people who are looking at the world from 30,000 feet, they don’t necessarily grasp that because, when you look at all comers, it doesn’t show a benefit.

So the problem is that now those treatments that might help a subset of patients are being denied, and the subset of patients that are going to benefit never get the treatment. Now we have to balance that with a lot of stuff that went on during the pandemic with, you know, ivermectin, hydroxychloroquine, and people pushing those things. Someone asked me once what I thought about hydroxychloroquine, and I said, “Well, somebody in the lab probably showed that it was beneficial, analogous to lighting tissue paper on fire on a plate and taking a cup of water and putting the fire out. Well, now, if you take that cup of water to the Caldor fire that’s burning in California on thousands of acres, you’re not going to be able to put the fire out with that cup of water.” So while it might work in the lab, it’s truly not going to work in a clinical setting. We have to balance individualizing care for patients with some information people are pushing out there that may not be necessarily translatable to the clinical setting.

I think there’s nothing better than being at the bedside, though, and being able to implement something and seeing what works. And really, experience goes a long way in being able to individually treat a patient optimally.

Dr. Barkoudah Thank you for everything you do at the bedside and your work on improving the treatment we have and how we can leverage knowledge to treat our patients. Thank you very much for your time and your scholarly contribution. We appreciate it and I hope the work will continue. We will keep working on treating COVID-19 patients with the best knowledge we have.

Q&A participants: George Sakoulas, MD, Sharp Rees-Stealy Medical Group, La Jolla, CA, and University of California San Diego School of Medicine, San Diego, CA; and Ebrahim Barkoudah, MD, MPH, Department of Medicine, Brigham and Women’s Hospital, Boston, MA.

Disclosures: None reported.

Dr. George Sakoulas is an infectious diseases clinician at Sharp Memorial Hospital in San Diego and professor of pediatrics at the University of California, San Diego School of Medicine. He was the lead investigator in a study published in the May/June 2022 issue of JCOM that found that, when allocated to the appropriate patient type, intravenous immunoglobulin can reduce hospital costs for COVID-19 care. ¹ He joined JCOM’s Editor-in-Chief, Dr. Ebrahim Barkoudah, to discuss the study’s background and highlight its main findings.

The following has been edited for length and clarity.

Dr. Barkoudah Dr. Sakoulas is an investigator and a clinician, bridging both worlds to bring the best evidence to our patients. We’re discussing his new article regarding intravenous immunoglobulin in treating nonventilated COVID-19 patients with moderate-to-severe hypoxia. Dr. Sakoulas, could you please share with our readers the clinical question your study addressed and what your work around COVID-19 management means for clinical practice?

Dr. Sakoulas Thank you. I’m an infectious disease physician. I’ve been treating patients with viral acute respiratory distress syndrome for almost 20 years as an ID doctor. Most of these cases are due to influenza or other viruses. And from time to time, anecdotally and supported by some literature, we’ve been using IVIG, or intravenous immunoglobulin, in some of these cases. And again, I can report anecdotal success with that over the years.

So when COVID emerged in March of 2020, we deployed IVIG in a couple of patients early who were heading downhill. Remember, in March of 2020, we didn’t have the knowledge of steroids helping, patients being ventilated very promptly, and we saw some patients who made a turnaround after treatment with IVIG. We were able to get some support from an industry sponsor and perform and publish a pilot study, enrolling patients early in the pandemic. That study actually showed benefits, which then led the sponsor to fund a phase 3 multicenter clinical trial. Unfortunately, a couple of things happened. First, the trial was designed with the knowledge we had in April of 2020, and again, this is before steroids, before we incorporated proning patients in the ICU, or started ventilating people early. So there were some management changes and evolutions and improvements that happened. And second, the trial was enrolling a very broad repertoire of patients. There were no age limitations, and the trial, ultimately a phase 3 multicenter trial, failed to meet its endpoint.

There were some trends for benefit in younger patients, and as the trial was ongoing, we continued to evolve our knowledge, and we really honed it down to seeing a benefit of using IVIG in patients with COVID with specific criteria in mind. They had to be relatively younger patients, under 65, and not have any major comorbidities. In other words, they weren’t dialysis patients or end-stage disease patients, heart failure patients, cancer or malignancy patients. So, you know, we’re looking at the patients under 65 with obesity, diabetes, and hypertension, who are rapidly declining, going from room air to BiPAP or high-flow oxygen in a short amount of time. And we learned that when using IVIG early, we actually saw patients improve and turn around.

What this article in JCOM highlighted was, number one, incorporating that outcome or that patient type and then looking at the cost of hospitalization of patients who received IVIG versus those that did not. There were 2 groups that were studied. One was the group of patients in that original pilot trial that I discussed who were randomized to receive 1 or the other prospectively; it was an unblinded randomized study. And the second group was a matched case-control study where we had patients treated with IVIG matched by age and comorbidity status and level of hypoxia to patients that did not receive IVIG. We saw a financial benefit in shortening or reducing hospitalizations, really coming down to getting rid of that 20% tail of patients that wound up going to the ICU, getting intubated, and using a high amount of hospital resources that would ramp up the cost of hospitalization. We saw great mitigation of that with IVIG, and even with a small subset of patients, we were able to show a benefit.

Dr. Barkoudah Any thoughts on where we can implement the new findings from your article in our practice at the moment, knowing we now have practice guidelines and protocols to treat COVID-19? There was a tangible benefit in treating the patients the way you approached it in your important work. Could you share with us what would be implementable at the moment?

Dr. Sakoulas I think, fortunately, with the increasing host immunity in the population and decreased virulence of the virus, perhaps we won’t see as many patients of the type that were in these trials going forward, but I suspect we will perhaps in the unvaccinated patients that remain. I believe one-third of the United States is not vaccinated. So there is certainly a vulnerable group of people out there. Potentially, an unvaccinated patient who winds up getting very sick, the patient who is relatively young—what I’m looking at is the 30- to 65-year-old obese, hypertensive, or diabetic patient who comes in and, despite the steroids and the antivirals, rapidly deteriorates into requiring high-flow oxygen. I think implementing IVIG in that patient type would be helpful. I don’t think it’s going to be as helpful in patients who are very elderly, because I think the mechanism of the disease is different in an 80-year-old versus a 50-year-old patient. So again, hopefully, it will not amount to a lot of patients, but I still suspect hospitals are going to see, perhaps in the fall, when they’re expecting a greater number of cases, a trickling of patients that do meet the criteria that I described.

Dr. Barkoudah JCOM’s audience are the QI implementers and hospital leadership. And what caught my eye in your article is your perspective on the pharmacoeconomics of treating COVID-19, and I really appreciate your looking at the cost aspect. Would you talk about the economics of inpatient care, the total care that we provide now that we’re in the age of tocilizumab, and the current state of multiple layers of therapy?

Dr. Sakoulas The reason to look at the economics of it is because IVIG—which is actually not a drug, it’s a blood product—is very expensive. So, we received a considerable amount of administrative pushback implementing this treatment at the beginning outside of the clinical trial setting because it hadn’t been studied on a large scale and because the cost was so high, even though, as a clinician at the bedside, I was seeing a benefit in patients. This study came out of my trying to demonstrate to the folks that are keeping the economics of medicine in mind that, in fact, investing several thousand dollars of treatment in IVIG will save you cost of care, the cost of an ICU bed, the cost of a ventilator, and the cost even of ECMO, which is hugely expensive.

If you look at the numbers in the study, for two-thirds or three-quarters of the patients, your cost of care is actually greater than the controls because you’re giving them IVIG, and it’s increasing the cost of their care, even though three-quarters of the patients are going to do just as well without it. It’s that 20% to 25% of patients that really are going to benefit from it, where you’re reducing your cost of care so much, and you’re getting rid of that very, very expensive 20%, that there’s a cost savings across the board per patient. So, it’s hard to understand when you say you’re losing money on three-quarters of the patients, you’re only saving money on a quarter of the patients, but that cost of saving on that small subset is so substantial it’s really impacting all numbers.

Also, abandoning the outlier principle is sort of an underlying theme in how we think of things. We tend to ignore outliers, not consider them, but I think we really have to pay attention to the more extreme cases because those patients are the ones that drive not just the financial cost of care. Remember, if you’re down to 1 ventilator and you can cut down the use of scarce ICU resources, the cost is sort of even beyond the cost of money. It’s the cost of resources that may become scarce in some settings. So, I think it speaks to that as well.

A lot of the drugs that we use, for example, tocilizumab, were able to be studied in thousands of patients. If you look at the absolute numbers, the benefit of tocilizumab from a magnitude standpoint—low to mid twenties to high twenties—you know, reducing mortality from 29% to 24%. I mean, just take a step back and think about that. Even though it’s statistically significant, try telling a patient, “Well, I’m going to give you this treatment that’s going to reduce mortality from 29% to 24%.” You know, that doesn’t really change anything from a clinical significance standpoint. But they have a P value less than .05, which is our standard, and they were able to do a study with thousands of patients. We didn’t have that luxury with IVIG. No one studied thousands of patients, only retrospectively, and those retrospective studies don’t get the attention because they’re considered biased with all their limitations. But I think one of the difficulties we have here is the balance between statistical and clinical significance. For example, in our pilot study, our ventilation rate was 58% with the non-IVIG patients versus 14% for IVIG patients. So you might say, magnitude-wise, that’s a big number, but the statistical significance of it is borderline because of small numbers.

Anyway, that’s a challenge that we have as clinicians trying to incorporate what’s published—the balancing of statistics, absolute numbers, and practicalities of delivering care. And I think this study highlights some of the nuances that go into that incorporation and those clinical decisions.

Dr. Barkoudah Would you mind sharing with our audience how we can make the connection between the medical outcomes and pharmacoeconomics findings from your article and link it to the bedside and treatment of our patients?

Dr. Sakoulas One of the points this article brings out is the importance of bringing together not just level 1A data, but also small studies with data such as this, where the magnitude of the effect is pretty big but you lose the statistics because of the small numbers. And then also the patients’ aspects of things. I think, as a bedside clinician, you appreciate things, the nuances, much sooner than what percolates out from a level 1A study. Case in point, in the sponsored phase 3 study that we did, and in some other studies that were prospectively done as well, these studies of IVIG simply had an enrollment of patients that was very broad, and not every patient benefits from the same therapy. A great example of this is the sepsis trials with Xigris and those types of agents that failed. You know, there are clinicians to this day who believe that there is a subset of patients that benefit from agents like this. The IVIG story falls a little bit into that category. It comes down to trying to identify the subset of patients that might benefit. And I think we’ve outlined this subset pretty well in our study: the younger, obese diabetic or hypertensive patient who’s rapidly declining.

It really brings together the need to not necessarily toss out these smaller studies, but kind of summarize everything together, and clinicians who are bedside, who are more in tune with the nuances of individual decisions at the individual patient level, might better appreciate these kinds of data. But I think we all have to put it together. IVIG does not make treatment guidelines at national levels and so forth. It’s not even listed in many of them. But there are patients out there who, if you ask them specifically how they felt, including a friend of mine who received the medication, there’s no question from their end, how they felt about this treatment option. Now, some people will get it and will not benefit. We just have to be really tuned into the fact that the same drug does not have the same result for every patient. And just to consider this in the high-risk patients that we talked about in our study.

Dr. Barkoudah While we were prepping for this interview, you made an analogy regarding clinical evidence along the lines of, “Do we need randomized clinical trials to do a parachute-type of experiment,” and we chatted about clinical wisdom. Would you mind sharing with our readers your thoughts on that?

Dr. Sakoulas Sometimes, we try a treatment and it’s very obvious for that particular patient that it helped them. Then you study the treatment in a large trial setting and it doesn’t work. For us bedside clinicians, there are some interventions sometimes that do appear as beneficial as a parachute would be, but yet, there has never been a randomized clinical trial proving that parachutes work. Again, a part of the challenge we have is patients are so different, their immunology is different, the pathogen infecting them is different, the time they present is different. Some present early, some present late. There are just so many moving parts to treating an infection that only a subset of people are going to benefit. And sometimes as clinicians, we’re so nuanced, that we identify a specific subset of patients where we know we can help them. And it’s so obvious for us, like a parachute would be, but to people who are looking at the world from 30,000 feet, they don’t necessarily grasp that because, when you look at all comers, it doesn’t show a benefit.

So the problem is that now those treatments that might help a subset of patients are being denied, and the subset of patients that are going to benefit never get the treatment. Now we have to balance that with a lot of stuff that went on during the pandemic with, you know, ivermectin, hydroxychloroquine, and people pushing those things. Someone asked me once what I thought about hydroxychloroquine, and I said, “Well, somebody in the lab probably showed that it was beneficial, analogous to lighting tissue paper on fire on a plate and taking a cup of water and putting the fire out. Well, now, if you take that cup of water to the Caldor fire that’s burning in California on thousands of acres, you’re not going to be able to put the fire out with that cup of water.” So while it might work in the lab, it’s truly not going to work in a clinical setting. We have to balance individualizing care for patients with some information people are pushing out there that may not be necessarily translatable to the clinical setting.

I think there’s nothing better than being at the bedside, though, and being able to implement something and seeing what works. And really, experience goes a long way in being able to individually treat a patient optimally.

Dr. Barkoudah Thank you for everything you do at the bedside and your work on improving the treatment we have and how we can leverage knowledge to treat our patients. Thank you very much for your time and your scholarly contribution. We appreciate it and I hope the work will continue. We will keep working on treating COVID-19 patients with the best knowledge we have.

Q&A participants: George Sakoulas, MD, Sharp Rees-Stealy Medical Group, La Jolla, CA, and University of California San Diego School of Medicine, San Diego, CA; and Ebrahim Barkoudah, MD, MPH, Department of Medicine, Brigham and Women’s Hospital, Boston, MA.

Disclosures: None reported.

From the University of Chicago Medical Center, Chicago, IL.

Abstract

Background: Epistaxis is a common chief complaint addressed by otolaryngologists. A review of the literature showed that there is a deficit in epistaxis education within the nursing community. Conversations with our nursing colleagues confirmed this unmet demand.

Objective: This quality improvement project aimed to increase general epistaxis knowledge, perceived comfort level managing nosebleeds, and perceived ability to stop nosebleeds among our nursing staff.

Methods: Data were collected through a survey administered before and after our intervention. The survey tested general epistaxis knowledge and assessed comfort and confidence in stopping epistaxis. Our intervention was an educational session covering pertinent epistaxis etiology and management. Quality improvement principles were used to optimize delivery of the intervention.

Results: A total of 51 nurses participated in the project. After participating in the in-service educational session, nurses answered significantly more epistaxis general knowledge questions correctly (mean [SD] difference, 2.07 [1.10] questions; 95% CI, 1.74-2.39; P < .001). There was no statistically significant difference in additional correct questions when stratified by clinical experience or clinical setting (P = .128 and P = 0.446, respectively). Nurses also reported feeling significantly more comfortable and significantly more confident in managing nosebleeds after the in-service (P = .007 and P < 0.001, respectively); 74.46% of nurses had an improvement in comfort level in managing epistaxis and 43.90% of nurses had an improvement in confidence in stopping epistaxis. After we moved the educational session from mid-shift to shift change, the nursing staff reported more satisfaction while maintaining similar improvements in knowledge and confidence.

Conclusion: We were able to significantly increase epistaxis knowledge, improve comfort levels managing epistaxis, and improve confidence in successful epistaxis management. Nurses of varying clinical experience and different clinical settings benefitted equally from our intervention.

Keywords: nosebleed; in-service; quality improvement.

Epistaxis, or nosebleed, is estimated to be the chief complaint in 1 in 200 emergency department visits in the United States.¹ Additionally, it represents up to one-third of otolaryngology-related emergency room admissions.² There is no existing literature, to our best knowledge, specifically investigating the incidence of epistaxis after a patient is admitted. Anecdotally, inpatients who develop epistaxis account for an appreciable number of consults to otolaryngology (ENT). Epistaxis is a cross-disciplinary issue, occurring in a range of clinical settings. For example, patients with epistaxis can present to the emergency department or to an outpatient primary care clinic before being referred to ENT. Additionally, inpatients on many different services can develop spontaneous epistaxis due to a variety of environmental and iatrogenic factors, such as dry air, use of nasal cannula, and initiation of anticoagulation. Based on the experience of our ENT providers and discussions with our nursing colleagues, we concluded that there was an interest in epistaxis management training among our nursing workforce.

The presence of unmet demand for epistaxis education among our nursing colleagues was supported by our literature review. A study performed in England surveyed emergency department nurses on first aid measures for management of epistaxis, including ideal head positioning, location of pressure application, and duration of pressure application.³ Overall, only 12% to 14% of the nursing staff answered all 3 questions correctly.³ Additionally, 73% to 78% of the nursing staff felt that their training in epistaxis management was inadequate, and 88% desired further training in epistaxis management.³ If generalized, this study confirms the demand for further epistaxis education among nurses.

In-services have previously been shown to be effective educational tools within the nursing community. A study in Ethiopia that evaluated pain management knowledge and attitudes before and after an in-service found a significant improvement in mean rank score of nurses’ knowledge and attitudes regarding pain management after they participated in the in-service.⁴ Scores on the knowledge survey improved from 41.4% before the intervention to 63.0% post intervention.⁴ A study in Connecticut evaluated nurses’ confidence in discussing suicidal ideation with patients and knowledge surrounding suicide precautions.⁵ After participating in an in-service, nurses were significantly more confident in discussing suicidal ideation with patients; application of appropriate suicide precautions also increased after the in-service.⁵

Our aim was for nurses to have an improvement in overall epistaxis knowledge, perceived comfort level managing nosebleeds, and perceived ability to stop nosebleeds after attending our in-service. Additionally, an overarching priority was to provide high-quality epistaxis education based on the literature and best practice guidelines.

Methods

Setting

This study was carried out at an 811-bed quaternary care center located in Chicago, Illinois. In fiscal year 2021, there were 91 643 emergency department visits and 33 805 hospital admissions. At our flagship hospital, 2658 patients were diagnosed with epistaxis during fiscal year 2021. The emergency department saw 533 patients with epistaxis, with 342 requiring admission and 191 being discharged. Separately, 566 inpatients received a diagnosis of epistaxis during their admission. The remainder of the patients with epistaxis were seen on an outpatient basis.

Data Collection

Data were collected from nurses on 5 different inpatient units. An email with information about the in-service was sent to the nurse managers of the inpatient units. These 5 units were included because the nurse managers responded to the email and facilitated delivery of the in-service. Data collection took place from August to December 2020.

Intervention

A quality improvement team composed of a resident physician champion, nurse educators, and nurse managers was formed. The physician champion was a senior otolaryngology resident who was responsible for designing and administering the pre-test, in-service, and post test. The nurse educators and nurse managers helped coordinate times for the in-service and promoted the in-service for their staff.

Our intervention was an educational in-service, a technique that is commonly used at our institution for nurse education. In-services typically involve delivering a lecture on a clinically relevant topic to a group of nurses on a unit. In developing the in-service, a top priority was to present high-quality evidence-based material. There is an abundance of information in the literature surrounding epistaxis management. The clinical practice guideline published by the American Academy of Otolaryngology lists nasal compression, application of vasoconstrictors, nasal packing, and nasal cautery as first-line treatments for the management of epistaxis.⁶ Nasal packing and nasal cautery tend to be perceived as interventions that require a certain level of expertise and specialized supplies. As such, these interventions are not often performed by floor nurses. In contrast, nasal compression and application of vasoconstrictors require only a few easily accessible supplies, and the risks are relatively minimal. When performing nasal compression, the clinical practice guidelines recommend firm, sustained compression to the lower third of the nose for 5 minutes or longer.⁶ Topical vasoconstrictors are generally underutilized in epistaxis management. In a study looking at a random sample of all US emergency department visits from 1992 to 2001, only 18% of visits used an epistaxis-related medication.² Oxymetazoline hydrochloride is a topical vasoconstrictor that is commonly used as a nasal decongestant. However, its vasoconstrictor properties also make it a useful tool for controlling epistaxis. In a study looking at emergency department visits at the University of Texas Health Science Center, 65% of patients had resolution of nosebleed with application of oxymetazoline hydrochloride as the only intervention, with another 18% experiencing resolution of nosebleed with a combination of oxymetazoline hydrochloride and silver nitrate cautery.⁷ Based on review of the literature, nasal compression and application of vasoconstrictors seemed to be low-resource interventions with minimal morbidity. Therefore, management centered around nasal compression and use of topical vasoconstrictors seemed appropriate for our nursing staff.

The in-service included information about the etiology and management of epistaxis. Particular emphasis was placed on addressing and debunking common misconceptions about nosebleed management. With regards to management, our presentation focused on the use of topical vasoconstrictors and firm pressure to the lower third of the nose for at least 5 minutes. Nasal packing and nasal cautery were presented as procedures that ENT would perform. After the in-service, questions from the nurses were answered as time permitted.

Testing and Outcomes

A pre-test was administered before each in-service. The pre-test components comprised a knowledge survey and a descriptive survey. The general epistaxis knowledge questions on the pre-test included the location of blood vessels most commonly responsible for nosebleeds, the ideal positioning of a patient during a nosebleed, the appropriate location to hold pressure during a nosebleed, and the appropriate duration to hold pressure during a nosebleed. The descriptive survey portion asked nurses to rate whether they felt “very comfortable,” “comfortable,” “uncomfortable,” or “very uncomfortable” managing nosebleeds. It also asked whether nurses thought they would be able to “always,” “usually,” “rarely,” or “never” stop nosebleeds on the floor. We collected demographic information, including gender identity, years of clinical experience, and primary clinical environment.

The post test asked the same questions as the pre-test and was administered immediately after the in-service in order to assess its impact. We also established an ongoing dialogue with our nursing colleagues to obtain feedback on the sessions.

Primary outcomes of interest were the difference in general epistaxis knowledge questions answered correctly between the pre-test and the post test; the difference in comfort level in managing epistaxis before and after the in-service; and the difference in confidence to stop nosebleeds before and after the in-service. A secondary outcome was determining the audience for the in-service. Specifically, we wanted to determine whether there were different outcomes based on clinical setting or years of clinical experience. If nurses in a certain clinical environment or beyond a certain experience level did not show significant improvement from pre-test to post test, we would not target them for the in-service. Another secondary outcome was determining optimal timing for delivery of the in-service. We wanted to determine if there was a nursing preference for delivering the in-service at mid-shift vs shift change.

Analysis

Statistical calculations were performed using Stata 15 (StataCorp LLC). A P value < .05 was considered to be statistically significant. Where applicable, 95% confidence intervals (CI) were calculated. T-test was used to determine whether there was a statistically significant difference between pre-test and post-test epistaxis knowledge question scores. T-test was also used to determine whether there was a statistically significant difference in test scores between nurses receiving the in-service at mid-shift vs shift change. Pearson chi-squared tests were used to determine if there was a statistically significant difference between pre-test and post-test perceptions of epistaxis management, and to investigate outcomes between different subsets of nurses.

SQUIRE 2.0 guidelines were utilized to provide a framework for this project and to structure the manuscript.⁸ This study met criteria for exemption from institutional review board approval.

Results

Fifty-one nurses took part in this project (Table). The majority of participants identified as female (88.24%), and just over half worked on medical floors (52.94%), with most of the remainder working in intensive care (25.49%) and surgical (15.69%) settings. There was a wide range of clinical experience, with 1.96% reporting 0 to 1 years of experience, 29.41% reporting 2 to 5 years, 23.53% reporting 5 to 10 years, 25.49% reporting 10 to 20 years, and 17.65% reporting more than 20 years.

There were unanswered questions on both the pre-test and post test. There was no consistently unanswered question. Omitted answers on the epistaxis knowledge questions were recorded as an “incorrect” answer. Omitted answers on the perception questions were considered null values and not considered in final analysis.

Primary Measures

General epistaxis knowledge (Figure, part A) improved from the pre-test, where out of 4 questions, the mean (SD) score was 1.74 (1.02) correct questions, to the post-test, where out of 4 questions, the mean score was 3.80 (0.40) correct questions. After participating in the in-service, nurses answered significantly more questions about epistaxis general knowledge correctly (mean difference, 2.07 [1.10]; 95% CI, 1.74-2.39; P < .001), and 80.43% of them got a perfect score on the epistaxis knowledge questions.

The second primary measure was the difference in comfort level in managing nosebleed. After participating in the in-service, nurses felt significantly more comfortable in managing nosebleeds (Figure, part B; P = .007), with 74.46% of nurses having an improved comfort level managing nosebleeds. Before the in-service, 12.76% of nurses felt “very comfortable” in managing nosebleeds vs more than three-quarters (76.59%) after the in-service. Of those who answered that they felt “comfortable” managing nosebleeds on the pre-test, 82.35% improved to feeling “very comfortable” in managing nosebleeds. Before the in-service, 14.89% of nurses felt “uncomfortable” or “very uncomfortable” in managing nosebleeds, and this decreased to 0 post intervention. After the in-service, 100.00% of nurses felt “comfortable” or “very comfortable” in managing nosebleeds.

After receiving the in-service, nurses felt significantly more confident in stopping nosebleeds (Figure, part C; P < .001), with 43.90% of them having an improvement in confidence in stopping epistaxis. Before the in-service, 7.31% of nurses felt that they would “always” be able to stop a nose-bleed, and this increased to 41.46% after the in-service. Of those who answered that they felt that they would “usually” be able to stop a nosebleed on the pre-test, 36.67% changed their answer to state that they would “always” be able to stop a nosebleed on the post test. Before the in-service, 19.51% of nurses felt that they would “rarely” or “never” be able to stop a nosebleed, and this decreased to 2.44% after the in-service.

Secondary Measures

All of the nurses who participated either “strongly agreed” or “agreed” that they learned something new from the in-service. However, to determine whether there was a population who would benefit most from the in-service, we stratified the data by years of clinical experience. There was no statistically significant difference in whether nurses with varying clinical experience learned something new (P = .148): 100% of nurses with 0-1 years of experience, 80.00% of nurses with 2-5 years of experience, 100% of nurses with 5-10 years of experience, 69.23% of nurses with 10-20 years of experience, and 100% of nurses with >20 years of experience “strongly agreed” that they learned something new from this in-service. There was no statistically significant difference on the post test compared to the pre-test in additional correct questions when stratified by clinical experience (P = .128). Second, when we stratified by clinical setting, we did not find a statistically significant difference in whether nurses in different clinical settings learned something new (P = .929): 88.89% of nurses in the medical setting, 87.50% of nurses in the surgical setting, and 84.62% of nurses in the intensive care setting “strongly agreed” that they learned something new from this presentation. On investigating additional questions correct on the post test compared to the pre-test, there was no statistically significant difference in additional correct questions when stratified by clinical setting (P = .446).

Optimal timing of the in-service was another important outcome. Initially, the in-service was administered at mid-shift, with 9 nurses participating at mid-shift, but our nursing colleagues gave unanimous feedback that this was a suboptimal time for delivery of an in-service. We changed the timing of the in-service to shift change; 42 nurses received the in-service at shift-change. There was no statistically significant difference in scores on the epistaxis knowledge questions between these two groups (P = .123). This indicated to us that changing the timing of the delivery resulted in similarly improved outcomes while having the added benefit of being preferred by our nursing colleagues.

Discussion

In undertaking this project, our primary aims were to improve epistaxis knowledge and perceived management in our nursing staff. Among our nursing staff, we were able to significantly increase epistaxis knowledge, improve comfort levels managing epistaxis, and improve confidence in successful epistaxis management. We also found that nurses of varying clinical experience and different clinical settings benefited equally from our intervention. Using quality improvement principles, we optimized our delivery. Our in-service focused on educating nurses to use epistaxis management techniques that were resource-efficient and low risk.

After participating in the in-service, nurses answered significantly more questions about epistaxis general knowledge correctly (Figure, part A; mean difference, 2.07 questions [1.10]; 95% CI, 1.74-2.39; P < .001), felt significantly more comfortable in managing nosebleeds (Figure, part B; P = .007), and felt significantly more confident in stopping nosebleeds (Figure, part C; P < .001). Based on these results, we successfully achieved our primary aims.

Our secondary aim was to determine the audience that would benefit the most from the in-service. All of the nurses who participated either “strongly agreed” or “agreed” that they learned something new from the in-service. There was no statistically significant difference in whether nurses of varying clinical experience learned something new (P = .148) or in additional correct questions when stratified by clinical experience (P =.128). Also, there was no statistically significant difference in whether nurses in different clinical settings learned something new (P = .929) or in additional correct questions when stratified by clinical setting (P = .446). These results indicated to us that all participants learned something new and that there was no specific target audience, but rather that all participants benefitted from our session.

Our nursing colleagues gave us feedback that the timing of the in-service during mid-shift was not ideal. It was difficult to gather nurses mid-shift due to pressing patient-care duties. Nurses also found it difficult to give their full attention at this time. Nurses, nurse educators, and nurse managers suggested that we conduct the in-service at shift change in order to capture a larger population and take advantage of time relatively free of clinical duties. Giving the in-service at a time with relatively fewer clinical responsibilities allowed for a more robust question-and-answer session. It also allowed our nursing colleagues to pay full attention to the in-service. There was no statistically significant difference in epistaxis general knowledge questions answered correctly; this indicates that the quality of the education session did not vary greatly. However, our nursing colleagues strongly preferred the in-service at shift change. By making this modification to our intervention, we were able to optimize our intervention.

The previously mentioned study in England reported that only 12% to 14% of their nursing staff got a perfect score on epistaxis knowledge questions. Prior to our study, there was no literature investigating the impact of an in-service on epistaxis knowledge. After our intervention, 80.43% of our nurses got a perfect score on the epistaxis knowledge questions. We believe that this is a fair comparison because our post-test questions were identical to the survey questions used in the previously mentioned study in England, with the addition of one question.³ Further, the findings of our study are consistent with other studies regarding the positive effect of in-service education on knowledge and attitudes surrounding clinical topics. Similar to the study in Ethiopia investigating nurses’ knowledge surrounding pain management, our study noted a significant improvement in nurses’ knowledge after participating in the in-service.⁴ Also, when comparing our study to the study performed in Connecticut investigating nurses’ confidence surrounding suicide precautions, we found a similar significant improvement in confidence in management after participating in the in-service.⁵

Given our reliance on a survey as a tool to collect information, our study was subject to nonresponse bias. For each main outcome question, there was a handful of nonresponders. While this likely indicated either overlooking a question or deferring to answer due to clinical inexperience or nonapplicable clinical role, it is possible that this may have represented a respondent who did not benefit from the in-service. Another source of possible bias is sampling bias. Attempts were made to capture a wide range of nurses at the in-service. However, if a nurse was not interested in the topic material, whether due to abundant clinical experience or disinterest, it is possible that they may not have attended. Additionally, the cohort was selected purely based on responses from nursing managers to the initial email. It is possible that nonresponding units may have benefitted differently from this in-service.

There were several limitations within our analysis. We did not collect data assessing the long-term retention of epistaxis knowledge and management techniques. It is possible that epistaxis knowledge, comfort in managing nosebleeds, and perceived confidence in stopping nosebleeds decreased back to baseline several months after the in-service. Ideally, we would have been able to collect this data to assess retention of the in-service information. Unfortunately, a significant number of nurses who initially participated in the project became lost to follow-up, making such data collection impossible. Additionally, there was no assessment of actual ability to stop nosebleeds before vs after this in-service. Perceived management of epistaxis vs actual management of epistaxis are 2 vastly different things. However, this data would have been difficult to collect, and it likely would not have been in the best interest of patients, especially before the in-service was administered. As an improvement to this project, we could have assessed how many nosebleeds nurses had seen and successfully stopped after the in-service. As previously mentioned, this was not possible due to losing a significant number of nurses to follow-up. Finally, we did not collect objective data on preference for administration of in-service at mid-shift vs shift change. We relied on subjective data from conversations with our colleagues. By collecting objective data, we could have supported this change to our intervention with data.

The primary challenge to sustainability for this intervention is nursing turnover. With each wave of departing nurses and new nursing hires, the difficulty of ensuring a consistent knowledge base and management standards within our nursing workforce became clearer. After optimizing our intervention, our solution was to provide a hospital-wide in-service, which was recorded and uploaded to an institution-wide in-service library. In this way, a nurse with the desire to learn about epistaxis management could access the material at any point in time. Another solution would have been to appoint champions for epistaxis management within each major department to deliver the epistaxis in-service to new hires and new rotators within the department. However, given the turnover witnessed in our study cohort, this may not be sustainable long term.

Conclusion

Epistaxis is a chief complaint that can present in many different clinical settings and situations. Therefore, the ability to stop epistaxis in a timely and effective fashion is valuable. Our study demonstrated that in-services can improve epistaxis knowledge and improve perceived epistaxis management. Ideally, this intervention will lead to improved patient care. Given that epistaxis is a ubiquitous issue, this study may benefit other institutions who want to improve care for patients with epistaxis.

Next steps for this intervention include utilizing in-services for epistaxis education at other institutions and collecting long-term data within our own institution. Collecting long-term data would allow us to assess the retention of epistaxis knowledge from our in-service.

Acknowledgments: The author thanks the nurse managers, nurse educators, and staff nurses involved in this project, as well as Dr. Louis Portugal for providing mentorship throughout this process and Dr. Dara Adams for assisting with statistical analysis.

Corresponding author: Avery Nelson, MD, University of Chicago Medical Center, 5841 S Maryland Ave, MC 1035, Chicago, IL 60637; avery.nelson@uchospitals.edu

Disclosures: None reported.

From the University of Chicago Medical Center, Chicago, IL.

Abstract

Background: Epistaxis is a common chief complaint addressed by otolaryngologists. A review of the literature showed that there is a deficit in epistaxis education within the nursing community. Conversations with our nursing colleagues confirmed this unmet demand.

Objective: This quality improvement project aimed to increase general epistaxis knowledge, perceived comfort level managing nosebleeds, and perceived ability to stop nosebleeds among our nursing staff.

Methods: Data were collected through a survey administered before and after our intervention. The survey tested general epistaxis knowledge and assessed comfort and confidence in stopping epistaxis. Our intervention was an educational session covering pertinent epistaxis etiology and management. Quality improvement principles were used to optimize delivery of the intervention.

Results: A total of 51 nurses participated in the project. After participating in the in-service educational session, nurses answered significantly more epistaxis general knowledge questions correctly (mean [SD] difference, 2.07 [1.10] questions; 95% CI, 1.74-2.39; P < .001). There was no statistically significant difference in additional correct questions when stratified by clinical experience or clinical setting (P = .128 and P = 0.446, respectively). Nurses also reported feeling significantly more comfortable and significantly more confident in managing nosebleeds after the in-service (P = .007 and P < 0.001, respectively); 74.46% of nurses had an improvement in comfort level in managing epistaxis and 43.90% of nurses had an improvement in confidence in stopping epistaxis. After we moved the educational session from mid-shift to shift change, the nursing staff reported more satisfaction while maintaining similar improvements in knowledge and confidence.

Conclusion: We were able to significantly increase epistaxis knowledge, improve comfort levels managing epistaxis, and improve confidence in successful epistaxis management. Nurses of varying clinical experience and different clinical settings benefitted equally from our intervention.

Keywords: nosebleed; in-service; quality improvement.

Epistaxis, or nosebleed, is estimated to be the chief complaint in 1 in 200 emergency department visits in the United States.¹ Additionally, it represents up to one-third of otolaryngology-related emergency room admissions.² There is no existing literature, to our best knowledge, specifically investigating the incidence of epistaxis after a patient is admitted. Anecdotally, inpatients who develop epistaxis account for an appreciable number of consults to otolaryngology (ENT). Epistaxis is a cross-disciplinary issue, occurring in a range of clinical settings. For example, patients with epistaxis can present to the emergency department or to an outpatient primary care clinic before being referred to ENT. Additionally, inpatients on many different services can develop spontaneous epistaxis due to a variety of environmental and iatrogenic factors, such as dry air, use of nasal cannula, and initiation of anticoagulation. Based on the experience of our ENT providers and discussions with our nursing colleagues, we concluded that there was an interest in epistaxis management training among our nursing workforce.

The presence of unmet demand for epistaxis education among our nursing colleagues was supported by our literature review. A study performed in England surveyed emergency department nurses on first aid measures for management of epistaxis, including ideal head positioning, location of pressure application, and duration of pressure application.³ Overall, only 12% to 14% of the nursing staff answered all 3 questions correctly.³ Additionally, 73% to 78% of the nursing staff felt that their training in epistaxis management was inadequate, and 88% desired further training in epistaxis management.³ If generalized, this study confirms the demand for further epistaxis education among nurses.

In-services have previously been shown to be effective educational tools within the nursing community. A study in Ethiopia that evaluated pain management knowledge and attitudes before and after an in-service found a significant improvement in mean rank score of nurses’ knowledge and attitudes regarding pain management after they participated in the in-service.⁴ Scores on the knowledge survey improved from 41.4% before the intervention to 63.0% post intervention.⁴ A study in Connecticut evaluated nurses’ confidence in discussing suicidal ideation with patients and knowledge surrounding suicide precautions.⁵ After participating in an in-service, nurses were significantly more confident in discussing suicidal ideation with patients; application of appropriate suicide precautions also increased after the in-service.⁵

Our aim was for nurses to have an improvement in overall epistaxis knowledge, perceived comfort level managing nosebleeds, and perceived ability to stop nosebleeds after attending our in-service. Additionally, an overarching priority was to provide high-quality epistaxis education based on the literature and best practice guidelines.

Methods

Setting

This study was carried out at an 811-bed quaternary care center located in Chicago, Illinois. In fiscal year 2021, there were 91 643 emergency department visits and 33 805 hospital admissions. At our flagship hospital, 2658 patients were diagnosed with epistaxis during fiscal year 2021. The emergency department saw 533 patients with epistaxis, with 342 requiring admission and 191 being discharged. Separately, 566 inpatients received a diagnosis of epistaxis during their admission. The remainder of the patients with epistaxis were seen on an outpatient basis.

Data Collection

Data were collected from nurses on 5 different inpatient units. An email with information about the in-service was sent to the nurse managers of the inpatient units. These 5 units were included because the nurse managers responded to the email and facilitated delivery of the in-service. Data collection took place from August to December 2020.

Intervention

A quality improvement team composed of a resident physician champion, nurse educators, and nurse managers was formed. The physician champion was a senior otolaryngology resident who was responsible for designing and administering the pre-test, in-service, and post test. The nurse educators and nurse managers helped coordinate times for the in-service and promoted the in-service for their staff.

Our intervention was an educational in-service, a technique that is commonly used at our institution for nurse education. In-services typically involve delivering a lecture on a clinically relevant topic to a group of nurses on a unit. In developing the in-service, a top priority was to present high-quality evidence-based material. There is an abundance of information in the literature surrounding epistaxis management. The clinical practice guideline published by the American Academy of Otolaryngology lists nasal compression, application of vasoconstrictors, nasal packing, and nasal cautery as first-line treatments for the management of epistaxis.⁶ Nasal packing and nasal cautery tend to be perceived as interventions that require a certain level of expertise and specialized supplies. As such, these interventions are not often performed by floor nurses. In contrast, nasal compression and application of vasoconstrictors require only a few easily accessible supplies, and the risks are relatively minimal. When performing nasal compression, the clinical practice guidelines recommend firm, sustained compression to the lower third of the nose for 5 minutes or longer.⁶ Topical vasoconstrictors are generally underutilized in epistaxis management. In a study looking at a random sample of all US emergency department visits from 1992 to 2001, only 18% of visits used an epistaxis-related medication.² Oxymetazoline hydrochloride is a topical vasoconstrictor that is commonly used as a nasal decongestant. However, its vasoconstrictor properties also make it a useful tool for controlling epistaxis. In a study looking at emergency department visits at the University of Texas Health Science Center, 65% of patients had resolution of nosebleed with application of oxymetazoline hydrochloride as the only intervention, with another 18% experiencing resolution of nosebleed with a combination of oxymetazoline hydrochloride and silver nitrate cautery.⁷ Based on review of the literature, nasal compression and application of vasoconstrictors seemed to be low-resource interventions with minimal morbidity. Therefore, management centered around nasal compression and use of topical vasoconstrictors seemed appropriate for our nursing staff.

The in-service included information about the etiology and management of epistaxis. Particular emphasis was placed on addressing and debunking common misconceptions about nosebleed management. With regards to management, our presentation focused on the use of topical vasoconstrictors and firm pressure to the lower third of the nose for at least 5 minutes. Nasal packing and nasal cautery were presented as procedures that ENT would perform. After the in-service, questions from the nurses were answered as time permitted.

Testing and Outcomes

A pre-test was administered before each in-service. The pre-test components comprised a knowledge survey and a descriptive survey. The general epistaxis knowledge questions on the pre-test included the location of blood vessels most commonly responsible for nosebleeds, the ideal positioning of a patient during a nosebleed, the appropriate location to hold pressure during a nosebleed, and the appropriate duration to hold pressure during a nosebleed. The descriptive survey portion asked nurses to rate whether they felt “very comfortable,” “comfortable,” “uncomfortable,” or “very uncomfortable” managing nosebleeds. It also asked whether nurses thought they would be able to “always,” “usually,” “rarely,” or “never” stop nosebleeds on the floor. We collected demographic information, including gender identity, years of clinical experience, and primary clinical environment.

The post test asked the same questions as the pre-test and was administered immediately after the in-service in order to assess its impact. We also established an ongoing dialogue with our nursing colleagues to obtain feedback on the sessions.

Primary outcomes of interest were the difference in general epistaxis knowledge questions answered correctly between the pre-test and the post test; the difference in comfort level in managing epistaxis before and after the in-service; and the difference in confidence to stop nosebleeds before and after the in-service. A secondary outcome was determining the audience for the in-service. Specifically, we wanted to determine whether there were different outcomes based on clinical setting or years of clinical experience. If nurses in a certain clinical environment or beyond a certain experience level did not show significant improvement from pre-test to post test, we would not target them for the in-service. Another secondary outcome was determining optimal timing for delivery of the in-service. We wanted to determine if there was a nursing preference for delivering the in-service at mid-shift vs shift change.

Analysis

Statistical calculations were performed using Stata 15 (StataCorp LLC). A P value < .05 was considered to be statistically significant. Where applicable, 95% confidence intervals (CI) were calculated. T-test was used to determine whether there was a statistically significant difference between pre-test and post-test epistaxis knowledge question scores. T-test was also used to determine whether there was a statistically significant difference in test scores between nurses receiving the in-service at mid-shift vs shift change. Pearson chi-squared tests were used to determine if there was a statistically significant difference between pre-test and post-test perceptions of epistaxis management, and to investigate outcomes between different subsets of nurses.

SQUIRE 2.0 guidelines were utilized to provide a framework for this project and to structure the manuscript.⁸ This study met criteria for exemption from institutional review board approval.

Results

Fifty-one nurses took part in this project (Table). The majority of participants identified as female (88.24%), and just over half worked on medical floors (52.94%), with most of the remainder working in intensive care (25.49%) and surgical (15.69%) settings. There was a wide range of clinical experience, with 1.96% reporting 0 to 1 years of experience, 29.41% reporting 2 to 5 years, 23.53% reporting 5 to 10 years, 25.49% reporting 10 to 20 years, and 17.65% reporting more than 20 years.

There were unanswered questions on both the pre-test and post test. There was no consistently unanswered question. Omitted answers on the epistaxis knowledge questions were recorded as an “incorrect” answer. Omitted answers on the perception questions were considered null values and not considered in final analysis.

Primary Measures

General epistaxis knowledge (Figure, part A) improved from the pre-test, where out of 4 questions, the mean (SD) score was 1.74 (1.02) correct questions, to the post-test, where out of 4 questions, the mean score was 3.80 (0.40) correct questions. After participating in the in-service, nurses answered significantly more questions about epistaxis general knowledge correctly (mean difference, 2.07 [1.10]; 95% CI, 1.74-2.39; P < .001), and 80.43% of them got a perfect score on the epistaxis knowledge questions.

The second primary measure was the difference in comfort level in managing nosebleed. After participating in the in-service, nurses felt significantly more comfortable in managing nosebleeds (Figure, part B; P = .007), with 74.46% of nurses having an improved comfort level managing nosebleeds. Before the in-service, 12.76% of nurses felt “very comfortable” in managing nosebleeds vs more than three-quarters (76.59%) after the in-service. Of those who answered that they felt “comfortable” managing nosebleeds on the pre-test, 82.35% improved to feeling “very comfortable” in managing nosebleeds. Before the in-service, 14.89% of nurses felt “uncomfortable” or “very uncomfortable” in managing nosebleeds, and this decreased to 0 post intervention. After the in-service, 100.00% of nurses felt “comfortable” or “very comfortable” in managing nosebleeds.

After receiving the in-service, nurses felt significantly more confident in stopping nosebleeds (Figure, part C; P < .001), with 43.90% of them having an improvement in confidence in stopping epistaxis. Before the in-service, 7.31% of nurses felt that they would “always” be able to stop a nose-bleed, and this increased to 41.46% after the in-service. Of those who answered that they felt that they would “usually” be able to stop a nosebleed on the pre-test, 36.67% changed their answer to state that they would “always” be able to stop a nosebleed on the post test. Before the in-service, 19.51% of nurses felt that they would “rarely” or “never” be able to stop a nosebleed, and this decreased to 2.44% after the in-service.

Secondary Measures

All of the nurses who participated either “strongly agreed” or “agreed” that they learned something new from the in-service. However, to determine whether there was a population who would benefit most from the in-service, we stratified the data by years of clinical experience. There was no statistically significant difference in whether nurses with varying clinical experience learned something new (P = .148): 100% of nurses with 0-1 years of experience, 80.00% of nurses with 2-5 years of experience, 100% of nurses with 5-10 years of experience, 69.23% of nurses with 10-20 years of experience, and 100% of nurses with >20 years of experience “strongly agreed” that they learned something new from this in-service. There was no statistically significant difference on the post test compared to the pre-test in additional correct questions when stratified by clinical experience (P = .128). Second, when we stratified by clinical setting, we did not find a statistically significant difference in whether nurses in different clinical settings learned something new (P = .929): 88.89% of nurses in the medical setting, 87.50% of nurses in the surgical setting, and 84.62% of nurses in the intensive care setting “strongly agreed” that they learned something new from this presentation. On investigating additional questions correct on the post test compared to the pre-test, there was no statistically significant difference in additional correct questions when stratified by clinical setting (P = .446).

Optimal timing of the in-service was another important outcome. Initially, the in-service was administered at mid-shift, with 9 nurses participating at mid-shift, but our nursing colleagues gave unanimous feedback that this was a suboptimal time for delivery of an in-service. We changed the timing of the in-service to shift change; 42 nurses received the in-service at shift-change. There was no statistically significant difference in scores on the epistaxis knowledge questions between these two groups (P = .123). This indicated to us that changing the timing of the delivery resulted in similarly improved outcomes while having the added benefit of being preferred by our nursing colleagues.

Discussion

In undertaking this project, our primary aims were to improve epistaxis knowledge and perceived management in our nursing staff. Among our nursing staff, we were able to significantly increase epistaxis knowledge, improve comfort levels managing epistaxis, and improve confidence in successful epistaxis management. We also found that nurses of varying clinical experience and different clinical settings benefited equally from our intervention. Using quality improvement principles, we optimized our delivery. Our in-service focused on educating nurses to use epistaxis management techniques that were resource-efficient and low risk.

After participating in the in-service, nurses answered significantly more questions about epistaxis general knowledge correctly (Figure, part A; mean difference, 2.07 questions [1.10]; 95% CI, 1.74-2.39; P < .001), felt significantly more comfortable in managing nosebleeds (Figure, part B; P = .007), and felt significantly more confident in stopping nosebleeds (Figure, part C; P < .001). Based on these results, we successfully achieved our primary aims.

Our secondary aim was to determine the audience that would benefit the most from the in-service. All of the nurses who participated either “strongly agreed” or “agreed” that they learned something new from the in-service. There was no statistically significant difference in whether nurses of varying clinical experience learned something new (P = .148) or in additional correct questions when stratified by clinical experience (P =.128). Also, there was no statistically significant difference in whether nurses in different clinical settings learned something new (P = .929) or in additional correct questions when stratified by clinical setting (P = .446). These results indicated to us that all participants learned something new and that there was no specific target audience, but rather that all participants benefitted from our session.

Our nursing colleagues gave us feedback that the timing of the in-service during mid-shift was not ideal. It was difficult to gather nurses mid-shift due to pressing patient-care duties. Nurses also found it difficult to give their full attention at this time. Nurses, nurse educators, and nurse managers suggested that we conduct the in-service at shift change in order to capture a larger population and take advantage of time relatively free of clinical duties. Giving the in-service at a time with relatively fewer clinical responsibilities allowed for a more robust question-and-answer session. It also allowed our nursing colleagues to pay full attention to the in-service. There was no statistically significant difference in epistaxis general knowledge questions answered correctly; this indicates that the quality of the education session did not vary greatly. However, our nursing colleagues strongly preferred the in-service at shift change. By making this modification to our intervention, we were able to optimize our intervention.

The previously mentioned study in England reported that only 12% to 14% of their nursing staff got a perfect score on epistaxis knowledge questions. Prior to our study, there was no literature investigating the impact of an in-service on epistaxis knowledge. After our intervention, 80.43% of our nurses got a perfect score on the epistaxis knowledge questions. We believe that this is a fair comparison because our post-test questions were identical to the survey questions used in the previously mentioned study in England, with the addition of one question.³ Further, the findings of our study are consistent with other studies regarding the positive effect of in-service education on knowledge and attitudes surrounding clinical topics. Similar to the study in Ethiopia investigating nurses’ knowledge surrounding pain management, our study noted a significant improvement in nurses’ knowledge after participating in the in-service.⁴ Also, when comparing our study to the study performed in Connecticut investigating nurses’ confidence surrounding suicide precautions, we found a similar significant improvement in confidence in management after participating in the in-service.⁵

Given our reliance on a survey as a tool to collect information, our study was subject to nonresponse bias. For each main outcome question, there was a handful of nonresponders. While this likely indicated either overlooking a question or deferring to answer due to clinical inexperience or nonapplicable clinical role, it is possible that this may have represented a respondent who did not benefit from the in-service. Another source of possible bias is sampling bias. Attempts were made to capture a wide range of nurses at the in-service. However, if a nurse was not interested in the topic material, whether due to abundant clinical experience or disinterest, it is possible that they may not have attended. Additionally, the cohort was selected purely based on responses from nursing managers to the initial email. It is possible that nonresponding units may have benefitted differently from this in-service.

There were several limitations within our analysis. We did not collect data assessing the long-term retention of epistaxis knowledge and management techniques. It is possible that epistaxis knowledge, comfort in managing nosebleeds, and perceived confidence in stopping nosebleeds decreased back to baseline several months after the in-service. Ideally, we would have been able to collect this data to assess retention of the in-service information. Unfortunately, a significant number of nurses who initially participated in the project became lost to follow-up, making such data collection impossible. Additionally, there was no assessment of actual ability to stop nosebleeds before vs after this in-service. Perceived management of epistaxis vs actual management of epistaxis are 2 vastly different things. However, this data would have been difficult to collect, and it likely would not have been in the best interest of patients, especially before the in-service was administered. As an improvement to this project, we could have assessed how many nosebleeds nurses had seen and successfully stopped after the in-service. As previously mentioned, this was not possible due to losing a significant number of nurses to follow-up. Finally, we did not collect objective data on preference for administration of in-service at mid-shift vs shift change. We relied on subjective data from conversations with our colleagues. By collecting objective data, we could have supported this change to our intervention with data.

The primary challenge to sustainability for this intervention is nursing turnover. With each wave of departing nurses and new nursing hires, the difficulty of ensuring a consistent knowledge base and management standards within our nursing workforce became clearer. After optimizing our intervention, our solution was to provide a hospital-wide in-service, which was recorded and uploaded to an institution-wide in-service library. In this way, a nurse with the desire to learn about epistaxis management could access the material at any point in time. Another solution would have been to appoint champions for epistaxis management within each major department to deliver the epistaxis in-service to new hires and new rotators within the department. However, given the turnover witnessed in our study cohort, this may not be sustainable long term.

Conclusion

Epistaxis is a chief complaint that can present in many different clinical settings and situations. Therefore, the ability to stop epistaxis in a timely and effective fashion is valuable. Our study demonstrated that in-services can improve epistaxis knowledge and improve perceived epistaxis management. Ideally, this intervention will lead to improved patient care. Given that epistaxis is a ubiquitous issue, this study may benefit other institutions who want to improve care for patients with epistaxis.

Next steps for this intervention include utilizing in-services for epistaxis education at other institutions and collecting long-term data within our own institution. Collecting long-term data would allow us to assess the retention of epistaxis knowledge from our in-service.

Acknowledgments: The author thanks the nurse managers, nurse educators, and staff nurses involved in this project, as well as Dr. Louis Portugal for providing mentorship throughout this process and Dr. Dara Adams for assisting with statistical analysis.

Corresponding author: Avery Nelson, MD, University of Chicago Medical Center, 5841 S Maryland Ave, MC 1035, Chicago, IL 60637; avery.nelson@uchospitals.edu

Disclosures: None reported.

From the University of Chicago Medical Center, Chicago, IL.

Abstract

Background: Epistaxis is a common chief complaint addressed by otolaryngologists. A review of the literature showed that there is a deficit in epistaxis education within the nursing community. Conversations with our nursing colleagues confirmed this unmet demand.

Objective: This quality improvement project aimed to increase general epistaxis knowledge, perceived comfort level managing nosebleeds, and perceived ability to stop nosebleeds among our nursing staff.

Methods: Data were collected through a survey administered before and after our intervention. The survey tested general epistaxis knowledge and assessed comfort and confidence in stopping epistaxis. Our intervention was an educational session covering pertinent epistaxis etiology and management. Quality improvement principles were used to optimize delivery of the intervention.

Results: A total of 51 nurses participated in the project. After participating in the in-service educational session, nurses answered significantly more epistaxis general knowledge questions correctly (mean [SD] difference, 2.07 [1.10] questions; 95% CI, 1.74-2.39; P < .001). There was no statistically significant difference in additional correct questions when stratified by clinical experience or clinical setting (P = .128 and P = 0.446, respectively). Nurses also reported feeling significantly more comfortable and significantly more confident in managing nosebleeds after the in-service (P = .007 and P < 0.001, respectively); 74.46% of nurses had an improvement in comfort level in managing epistaxis and 43.90% of nurses had an improvement in confidence in stopping epistaxis. After we moved the educational session from mid-shift to shift change, the nursing staff reported more satisfaction while maintaining similar improvements in knowledge and confidence.

Conclusion: We were able to significantly increase epistaxis knowledge, improve comfort levels managing epistaxis, and improve confidence in successful epistaxis management. Nurses of varying clinical experience and different clinical settings benefitted equally from our intervention.

Keywords: nosebleed; in-service; quality improvement.

Epistaxis, or nosebleed, is estimated to be the chief complaint in 1 in 200 emergency department visits in the United States.¹ Additionally, it represents up to one-third of otolaryngology-related emergency room admissions.² There is no existing literature, to our best knowledge, specifically investigating the incidence of epistaxis after a patient is admitted. Anecdotally, inpatients who develop epistaxis account for an appreciable number of consults to otolaryngology (ENT). Epistaxis is a cross-disciplinary issue, occurring in a range of clinical settings. For example, patients with epistaxis can present to the emergency department or to an outpatient primary care clinic before being referred to ENT. Additionally, inpatients on many different services can develop spontaneous epistaxis due to a variety of environmental and iatrogenic factors, such as dry air, use of nasal cannula, and initiation of anticoagulation. Based on the experience of our ENT providers and discussions with our nursing colleagues, we concluded that there was an interest in epistaxis management training among our nursing workforce.

The presence of unmet demand for epistaxis education among our nursing colleagues was supported by our literature review. A study performed in England surveyed emergency department nurses on first aid measures for management of epistaxis, including ideal head positioning, location of pressure application, and duration of pressure application.³ Overall, only 12% to 14% of the nursing staff answered all 3 questions correctly.³ Additionally, 73% to 78% of the nursing staff felt that their training in epistaxis management was inadequate, and 88% desired further training in epistaxis management.³ If generalized, this study confirms the demand for further epistaxis education among nurses.

In-services have previously been shown to be effective educational tools within the nursing community. A study in Ethiopia that evaluated pain management knowledge and attitudes before and after an in-service found a significant improvement in mean rank score of nurses’ knowledge and attitudes regarding pain management after they participated in the in-service.⁴ Scores on the knowledge survey improved from 41.4% before the intervention to 63.0% post intervention.⁴ A study in Connecticut evaluated nurses’ confidence in discussing suicidal ideation with patients and knowledge surrounding suicide precautions.⁵ After participating in an in-service, nurses were significantly more confident in discussing suicidal ideation with patients; application of appropriate suicide precautions also increased after the in-service.⁵

Our aim was for nurses to have an improvement in overall epistaxis knowledge, perceived comfort level managing nosebleeds, and perceived ability to stop nosebleeds after attending our in-service. Additionally, an overarching priority was to provide high-quality epistaxis education based on the literature and best practice guidelines.

Methods

Setting

This study was carried out at an 811-bed quaternary care center located in Chicago, Illinois. In fiscal year 2021, there were 91 643 emergency department visits and 33 805 hospital admissions. At our flagship hospital, 2658 patients were diagnosed with epistaxis during fiscal year 2021. The emergency department saw 533 patients with epistaxis, with 342 requiring admission and 191 being discharged. Separately, 566 inpatients received a diagnosis of epistaxis during their admission. The remainder of the patients with epistaxis were seen on an outpatient basis.

Data Collection

Data were collected from nurses on 5 different inpatient units. An email with information about the in-service was sent to the nurse managers of the inpatient units. These 5 units were included because the nurse managers responded to the email and facilitated delivery of the in-service. Data collection took place from August to December 2020.

Intervention

A quality improvement team composed of a resident physician champion, nurse educators, and nurse managers was formed. The physician champion was a senior otolaryngology resident who was responsible for designing and administering the pre-test, in-service, and post test. The nurse educators and nurse managers helped coordinate times for the in-service and promoted the in-service for their staff.

Our intervention was an educational in-service, a technique that is commonly used at our institution for nurse education. In-services typically involve delivering a lecture on a clinically relevant topic to a group of nurses on a unit. In developing the in-service, a top priority was to present high-quality evidence-based material. There is an abundance of information in the literature surrounding epistaxis management. The clinical practice guideline published by the American Academy of Otolaryngology lists nasal compression, application of vasoconstrictors, nasal packing, and nasal cautery as first-line treatments for the management of epistaxis.⁶ Nasal packing and nasal cautery tend to be perceived as interventions that require a certain level of expertise and specialized supplies. As such, these interventions are not often performed by floor nurses. In contrast, nasal compression and application of vasoconstrictors require only a few easily accessible supplies, and the risks are relatively minimal. When performing nasal compression, the clinical practice guidelines recommend firm, sustained compression to the lower third of the nose for 5 minutes or longer.⁶ Topical vasoconstrictors are generally underutilized in epistaxis management. In a study looking at a random sample of all US emergency department visits from 1992 to 2001, only 18% of visits used an epistaxis-related medication.² Oxymetazoline hydrochloride is a topical vasoconstrictor that is commonly used as a nasal decongestant. However, its vasoconstrictor properties also make it a useful tool for controlling epistaxis. In a study looking at emergency department visits at the University of Texas Health Science Center, 65% of patients had resolution of nosebleed with application of oxymetazoline hydrochloride as the only intervention, with another 18% experiencing resolution of nosebleed with a combination of oxymetazoline hydrochloride and silver nitrate cautery.⁷ Based on review of the literature, nasal compression and application of vasoconstrictors seemed to be low-resource interventions with minimal morbidity. Therefore, management centered around nasal compression and use of topical vasoconstrictors seemed appropriate for our nursing staff.

The in-service included information about the etiology and management of epistaxis. Particular emphasis was placed on addressing and debunking common misconceptions about nosebleed management. With regards to management, our presentation focused on the use of topical vasoconstrictors and firm pressure to the lower third of the nose for at least 5 minutes. Nasal packing and nasal cautery were presented as procedures that ENT would perform. After the in-service, questions from the nurses were answered as time permitted.

Testing and Outcomes

A pre-test was administered before each in-service. The pre-test components comprised a knowledge survey and a descriptive survey. The general epistaxis knowledge questions on the pre-test included the location of blood vessels most commonly responsible for nosebleeds, the ideal positioning of a patient during a nosebleed, the appropriate location to hold pressure during a nosebleed, and the appropriate duration to hold pressure during a nosebleed. The descriptive survey portion asked nurses to rate whether they felt “very comfortable,” “comfortable,” “uncomfortable,” or “very uncomfortable” managing nosebleeds. It also asked whether nurses thought they would be able to “always,” “usually,” “rarely,” or “never” stop nosebleeds on the floor. We collected demographic information, including gender identity, years of clinical experience, and primary clinical environment.

The post test asked the same questions as the pre-test and was administered immediately after the in-service in order to assess its impact. We also established an ongoing dialogue with our nursing colleagues to obtain feedback on the sessions.

Primary outcomes of interest were the difference in general epistaxis knowledge questions answered correctly between the pre-test and the post test; the difference in comfort level in managing epistaxis before and after the in-service; and the difference in confidence to stop nosebleeds before and after the in-service. A secondary outcome was determining the audience for the in-service. Specifically, we wanted to determine whether there were different outcomes based on clinical setting or years of clinical experience. If nurses in a certain clinical environment or beyond a certain experience level did not show significant improvement from pre-test to post test, we would not target them for the in-service. Another secondary outcome was determining optimal timing for delivery of the in-service. We wanted to determine if there was a nursing preference for delivering the in-service at mid-shift vs shift change.

Analysis

Statistical calculations were performed using Stata 15 (StataCorp LLC). A P value < .05 was considered to be statistically significant. Where applicable, 95% confidence intervals (CI) were calculated. T-test was used to determine whether there was a statistically significant difference between pre-test and post-test epistaxis knowledge question scores. T-test was also used to determine whether there was a statistically significant difference in test scores between nurses receiving the in-service at mid-shift vs shift change. Pearson chi-squared tests were used to determine if there was a statistically significant difference between pre-test and post-test perceptions of epistaxis management, and to investigate outcomes between different subsets of nurses.

SQUIRE 2.0 guidelines were utilized to provide a framework for this project and to structure the manuscript.⁸ This study met criteria for exemption from institutional review board approval.

Results

Fifty-one nurses took part in this project (Table). The majority of participants identified as female (88.24%), and just over half worked on medical floors (52.94%), with most of the remainder working in intensive care (25.49%) and surgical (15.69%) settings. There was a wide range of clinical experience, with 1.96% reporting 0 to 1 years of experience, 29.41% reporting 2 to 5 years, 23.53% reporting 5 to 10 years, 25.49% reporting 10 to 20 years, and 17.65% reporting more than 20 years.

There were unanswered questions on both the pre-test and post test. There was no consistently unanswered question. Omitted answers on the epistaxis knowledge questions were recorded as an “incorrect” answer. Omitted answers on the perception questions were considered null values and not considered in final analysis.

Primary Measures

General epistaxis knowledge (Figure, part A) improved from the pre-test, where out of 4 questions, the mean (SD) score was 1.74 (1.02) correct questions, to the post-test, where out of 4 questions, the mean score was 3.80 (0.40) correct questions. After participating in the in-service, nurses answered significantly more questions about epistaxis general knowledge correctly (mean difference, 2.07 [1.10]; 95% CI, 1.74-2.39; P < .001), and 80.43% of them got a perfect score on the epistaxis knowledge questions.

The second primary measure was the difference in comfort level in managing nosebleed. After participating in the in-service, nurses felt significantly more comfortable in managing nosebleeds (Figure, part B; P = .007), with 74.46% of nurses having an improved comfort level managing nosebleeds. Before the in-service, 12.76% of nurses felt “very comfortable” in managing nosebleeds vs more than three-quarters (76.59%) after the in-service. Of those who answered that they felt “comfortable” managing nosebleeds on the pre-test, 82.35% improved to feeling “very comfortable” in managing nosebleeds. Before the in-service, 14.89% of nurses felt “uncomfortable” or “very uncomfortable” in managing nosebleeds, and this decreased to 0 post intervention. After the in-service, 100.00% of nurses felt “comfortable” or “very comfortable” in managing nosebleeds.

After receiving the in-service, nurses felt significantly more confident in stopping nosebleeds (Figure, part C; P < .001), with 43.90% of them having an improvement in confidence in stopping epistaxis. Before the in-service, 7.31% of nurses felt that they would “always” be able to stop a nose-bleed, and this increased to 41.46% after the in-service. Of those who answered that they felt that they would “usually” be able to stop a nosebleed on the pre-test, 36.67% changed their answer to state that they would “always” be able to stop a nosebleed on the post test. Before the in-service, 19.51% of nurses felt that they would “rarely” or “never” be able to stop a nosebleed, and this decreased to 2.44% after the in-service.

Secondary Measures

All of the nurses who participated either “strongly agreed” or “agreed” that they learned something new from the in-service. However, to determine whether there was a population who would benefit most from the in-service, we stratified the data by years of clinical experience. There was no statistically significant difference in whether nurses with varying clinical experience learned something new (P = .148): 100% of nurses with 0-1 years of experience, 80.00% of nurses with 2-5 years of experience, 100% of nurses with 5-10 years of experience, 69.23% of nurses with 10-20 years of experience, and 100% of nurses with >20 years of experience “strongly agreed” that they learned something new from this in-service. There was no statistically significant difference on the post test compared to the pre-test in additional correct questions when stratified by clinical experience (P = .128). Second, when we stratified by clinical setting, we did not find a statistically significant difference in whether nurses in different clinical settings learned something new (P = .929): 88.89% of nurses in the medical setting, 87.50% of nurses in the surgical setting, and 84.62% of nurses in the intensive care setting “strongly agreed” that they learned something new from this presentation. On investigating additional questions correct on the post test compared to the pre-test, there was no statistically significant difference in additional correct questions when stratified by clinical setting (P = .446).

Optimal timing of the in-service was another important outcome. Initially, the in-service was administered at mid-shift, with 9 nurses participating at mid-shift, but our nursing colleagues gave unanimous feedback that this was a suboptimal time for delivery of an in-service. We changed the timing of the in-service to shift change; 42 nurses received the in-service at shift-change. There was no statistically significant difference in scores on the epistaxis knowledge questions between these two groups (P = .123). This indicated to us that changing the timing of the delivery resulted in similarly improved outcomes while having the added benefit of being preferred by our nursing colleagues.

Discussion

In undertaking this project, our primary aims were to improve epistaxis knowledge and perceived management in our nursing staff. Among our nursing staff, we were able to significantly increase epistaxis knowledge, improve comfort levels managing epistaxis, and improve confidence in successful epistaxis management. We also found that nurses of varying clinical experience and different clinical settings benefited equally from our intervention. Using quality improvement principles, we optimized our delivery. Our in-service focused on educating nurses to use epistaxis management techniques that were resource-efficient and low risk.

After participating in the in-service, nurses answered significantly more questions about epistaxis general knowledge correctly (Figure, part A; mean difference, 2.07 questions [1.10]; 95% CI, 1.74-2.39; P < .001), felt significantly more comfortable in managing nosebleeds (Figure, part B; P = .007), and felt significantly more confident in stopping nosebleeds (Figure, part C; P < .001). Based on these results, we successfully achieved our primary aims.

Our secondary aim was to determine the audience that would benefit the most from the in-service. All of the nurses who participated either “strongly agreed” or “agreed” that they learned something new from the in-service. There was no statistically significant difference in whether nurses of varying clinical experience learned something new (P = .148) or in additional correct questions when stratified by clinical experience (P =.128). Also, there was no statistically significant difference in whether nurses in different clinical settings learned something new (P = .929) or in additional correct questions when stratified by clinical setting (P = .446). These results indicated to us that all participants learned something new and that there was no specific target audience, but rather that all participants benefitted from our session.

Our nursing colleagues gave us feedback that the timing of the in-service during mid-shift was not ideal. It was difficult to gather nurses mid-shift due to pressing patient-care duties. Nurses also found it difficult to give their full attention at this time. Nurses, nurse educators, and nurse managers suggested that we conduct the in-service at shift change in order to capture a larger population and take advantage of time relatively free of clinical duties. Giving the in-service at a time with relatively fewer clinical responsibilities allowed for a more robust question-and-answer session. It also allowed our nursing colleagues to pay full attention to the in-service. There was no statistically significant difference in epistaxis general knowledge questions answered correctly; this indicates that the quality of the education session did not vary greatly. However, our nursing colleagues strongly preferred the in-service at shift change. By making this modification to our intervention, we were able to optimize our intervention.

The previously mentioned study in England reported that only 12% to 14% of their nursing staff got a perfect score on epistaxis knowledge questions. Prior to our study, there was no literature investigating the impact of an in-service on epistaxis knowledge. After our intervention, 80.43% of our nurses got a perfect score on the epistaxis knowledge questions. We believe that this is a fair comparison because our post-test questions were identical to the survey questions used in the previously mentioned study in England, with the addition of one question.³ Further, the findings of our study are consistent with other studies regarding the positive effect of in-service education on knowledge and attitudes surrounding clinical topics. Similar to the study in Ethiopia investigating nurses’ knowledge surrounding pain management, our study noted a significant improvement in nurses’ knowledge after participating in the in-service.⁴ Also, when comparing our study to the study performed in Connecticut investigating nurses’ confidence surrounding suicide precautions, we found a similar significant improvement in confidence in management after participating in the in-service.⁵

Given our reliance on a survey as a tool to collect information, our study was subject to nonresponse bias. For each main outcome question, there was a handful of nonresponders. While this likely indicated either overlooking a question or deferring to answer due to clinical inexperience or nonapplicable clinical role, it is possible that this may have represented a respondent who did not benefit from the in-service. Another source of possible bias is sampling bias. Attempts were made to capture a wide range of nurses at the in-service. However, if a nurse was not interested in the topic material, whether due to abundant clinical experience or disinterest, it is possible that they may not have attended. Additionally, the cohort was selected purely based on responses from nursing managers to the initial email. It is possible that nonresponding units may have benefitted differently from this in-service.

There were several limitations within our analysis. We did not collect data assessing the long-term retention of epistaxis knowledge and management techniques. It is possible that epistaxis knowledge, comfort in managing nosebleeds, and perceived confidence in stopping nosebleeds decreased back to baseline several months after the in-service. Ideally, we would have been able to collect this data to assess retention of the in-service information. Unfortunately, a significant number of nurses who initially participated in the project became lost to follow-up, making such data collection impossible. Additionally, there was no assessment of actual ability to stop nosebleeds before vs after this in-service. Perceived management of epistaxis vs actual management of epistaxis are 2 vastly different things. However, this data would have been difficult to collect, and it likely would not have been in the best interest of patients, especially before the in-service was administered. As an improvement to this project, we could have assessed how many nosebleeds nurses had seen and successfully stopped after the in-service. As previously mentioned, this was not possible due to losing a significant number of nurses to follow-up. Finally, we did not collect objective data on preference for administration of in-service at mid-shift vs shift change. We relied on subjective data from conversations with our colleagues. By collecting objective data, we could have supported this change to our intervention with data.

The primary challenge to sustainability for this intervention is nursing turnover. With each wave of departing nurses and new nursing hires, the difficulty of ensuring a consistent knowledge base and management standards within our nursing workforce became clearer. After optimizing our intervention, our solution was to provide a hospital-wide in-service, which was recorded and uploaded to an institution-wide in-service library. In this way, a nurse with the desire to learn about epistaxis management could access the material at any point in time. Another solution would have been to appoint champions for epistaxis management within each major department to deliver the epistaxis in-service to new hires and new rotators within the department. However, given the turnover witnessed in our study cohort, this may not be sustainable long term.

Conclusion

Epistaxis is a chief complaint that can present in many different clinical settings and situations. Therefore, the ability to stop epistaxis in a timely and effective fashion is valuable. Our study demonstrated that in-services can improve epistaxis knowledge and improve perceived epistaxis management. Ideally, this intervention will lead to improved patient care. Given that epistaxis is a ubiquitous issue, this study may benefit other institutions who want to improve care for patients with epistaxis.

Next steps for this intervention include utilizing in-services for epistaxis education at other institutions and collecting long-term data within our own institution. Collecting long-term data would allow us to assess the retention of epistaxis knowledge from our in-service.

Acknowledgments: The author thanks the nurse managers, nurse educators, and staff nurses involved in this project, as well as Dr. Louis Portugal for providing mentorship throughout this process and Dr. Dara Adams for assisting with statistical analysis.

Corresponding author: Avery Nelson, MD, University of Chicago Medical Center, 5841 S Maryland Ave, MC 1035, Chicago, IL 60637; avery.nelson@uchospitals.edu

Disclosures: None reported.

Authors' Response

We agree with the valid comments made by Dr. Kerguelen and will respond to each set of questions in order.

Regarding the first set of questions on how we knew that our CMI was low and our patient acuity was under- represented, the University of Miami Health System is a designated cancer center with a Prospective Payment System exempt model (PPS exempt), and is one of 11 hospitals in the United States excluded for payment under the Inpatient Prospective Payment System. We know, therefore, that we care for a very complex patient population. Additionally, we benchmark ourselves against other academic medical centers (AMCs) with similarly complex patients and had noted that our patients appeared “less complex.” Specifically, our baseline CMI was 1.77 in early 2018 compared with an overall higher CMI for the AMC cohort; also, the total number of diagnoses we captured was lower than that in other AMCs. These 2 facts together alerted us that we likely had coding and clinical documentation improvement (CDI) opportunities. We recognized that our complexity was not being captured both because the clinical information was not documented in a manner readily translatable to ICD-10 codes and codes were missed when the documentation did exist. To remedy these problems, we implemented multiple immediate “fixes,” which included revamping our CDI efforts, re-education, and enhancements to our electronic health record for providers, CDIs, and coders. Since publication of our article, our CMI has continued to increase month over month, up to 2.57 most recently in May 2022, as we have continued to focus on several additional initiatives to impact both better documentation and coding.

The second set of questions asked whether the perceived low CMI was causing problems with payers and about the risk of artificially increasing the CMI through overdiagnosis as well as audit mechanisms to avoid this, and changes in expected mortality and observed mortality. To our knowledge, the lower CMI did not cause any problems with payers, but this is something we are currently tracking. Coding and documentation are constantly audited both internally (by our quality department) and externally (using Inter-Rater Reliability audits and validation), with no noted trend or targeted opportunities. We only include comorbidities that are current, actively monitored/managed, and pertinent to the care of our patients. We have not noted a change in denials, which gives us confidence we are not now overdiagnosing.

Our observed mortality has also increased. We, like all institutions, experienced the confounding factor of the COVID-19 pandemic, which coincided with the higher observed mortality over the course of the past 2 years. While the observed mortality (indicating sicker patients assuming no worsening of care processes) may partly explain our increased coding complexity, our decreasing mortality index (observed:expected mortality) suggests that our efforts to improve documentation and coding likely reflect improved capture of missed complexity (Figure).

We understand the concerns raised by Dr. Kerguelen about potential mis(over)coding. As part of this quality initiative, therefore, we plan long-term evaluations of our processes and metrics to better determine and guide our understanding of the impact of what we have already implemented and future interventions. In fact, we are in the process of analyzing additional interventions and hope to share results from these evaluations soon.

Marie Anne Sosa, MD
Tanira Ferreira, MD
Hayley Gershengorn, MD
Melissa Soto
Estin Kelly
Ameena Shrestha
Julianne Burgos
Sandeep Devabhaktuni
Dipen Parekh, MD
Maritza Suarez, MD
University of Miami Hospital and Clinics, Miami, FL
mxs2157@med.miami.edu

Disclosures: None reported.

Authors' Response

We agree with the valid comments made by Dr. Kerguelen and will respond to each set of questions in order.

Regarding the first set of questions on how we knew that our CMI was low and our patient acuity was under- represented, the University of Miami Health System is a designated cancer center with a Prospective Payment System exempt model (PPS exempt), and is one of 11 hospitals in the United States excluded for payment under the Inpatient Prospective Payment System. We know, therefore, that we care for a very complex patient population. Additionally, we benchmark ourselves against other academic medical centers (AMCs) with similarly complex patients and had noted that our patients appeared “less complex.” Specifically, our baseline CMI was 1.77 in early 2018 compared with an overall higher CMI for the AMC cohort; also, the total number of diagnoses we captured was lower than that in other AMCs. These 2 facts together alerted us that we likely had coding and clinical documentation improvement (CDI) opportunities. We recognized that our complexity was not being captured both because the clinical information was not documented in a manner readily translatable to ICD-10 codes and codes were missed when the documentation did exist. To remedy these problems, we implemented multiple immediate “fixes,” which included revamping our CDI efforts, re-education, and enhancements to our electronic health record for providers, CDIs, and coders. Since publication of our article, our CMI has continued to increase month over month, up to 2.57 most recently in May 2022, as we have continued to focus on several additional initiatives to impact both better documentation and coding.

The second set of questions asked whether the perceived low CMI was causing problems with payers and about the risk of artificially increasing the CMI through overdiagnosis as well as audit mechanisms to avoid this, and changes in expected mortality and observed mortality. To our knowledge, the lower CMI did not cause any problems with payers, but this is something we are currently tracking. Coding and documentation are constantly audited both internally (by our quality department) and externally (using Inter-Rater Reliability audits and validation), with no noted trend or targeted opportunities. We only include comorbidities that are current, actively monitored/managed, and pertinent to the care of our patients. We have not noted a change in denials, which gives us confidence we are not now overdiagnosing.

Our observed mortality has also increased. We, like all institutions, experienced the confounding factor of the COVID-19 pandemic, which coincided with the higher observed mortality over the course of the past 2 years. While the observed mortality (indicating sicker patients assuming no worsening of care processes) may partly explain our increased coding complexity, our decreasing mortality index (observed:expected mortality) suggests that our efforts to improve documentation and coding likely reflect improved capture of missed complexity (Figure).

We understand the concerns raised by Dr. Kerguelen about potential mis(over)coding. As part of this quality initiative, therefore, we plan long-term evaluations of our processes and metrics to better determine and guide our understanding of the impact of what we have already implemented and future interventions. In fact, we are in the process of analyzing additional interventions and hope to share results from these evaluations soon.

Marie Anne Sosa, MD
Tanira Ferreira, MD
Hayley Gershengorn, MD
Melissa Soto
Estin Kelly
Ameena Shrestha
Julianne Burgos
Sandeep Devabhaktuni
Dipen Parekh, MD
Maritza Suarez, MD
University of Miami Hospital and Clinics, Miami, FL
mxs2157@med.miami.edu

Disclosures: None reported.

Authors' Response

We agree with the valid comments made by Dr. Kerguelen and will respond to each set of questions in order.

Regarding the first set of questions on how we knew that our CMI was low and our patient acuity was under- represented, the University of Miami Health System is a designated cancer center with a Prospective Payment System exempt model (PPS exempt), and is one of 11 hospitals in the United States excluded for payment under the Inpatient Prospective Payment System. We know, therefore, that we care for a very complex patient population. Additionally, we benchmark ourselves against other academic medical centers (AMCs) with similarly complex patients and had noted that our patients appeared “less complex.” Specifically, our baseline CMI was 1.77 in early 2018 compared with an overall higher CMI for the AMC cohort; also, the total number of diagnoses we captured was lower than that in other AMCs. These 2 facts together alerted us that we likely had coding and clinical documentation improvement (CDI) opportunities. We recognized that our complexity was not being captured both because the clinical information was not documented in a manner readily translatable to ICD-10 codes and codes were missed when the documentation did exist. To remedy these problems, we implemented multiple immediate “fixes,” which included revamping our CDI efforts, re-education, and enhancements to our electronic health record for providers, CDIs, and coders. Since publication of our article, our CMI has continued to increase month over month, up to 2.57 most recently in May 2022, as we have continued to focus on several additional initiatives to impact both better documentation and coding.

The second set of questions asked whether the perceived low CMI was causing problems with payers and about the risk of artificially increasing the CMI through overdiagnosis as well as audit mechanisms to avoid this, and changes in expected mortality and observed mortality. To our knowledge, the lower CMI did not cause any problems with payers, but this is something we are currently tracking. Coding and documentation are constantly audited both internally (by our quality department) and externally (using Inter-Rater Reliability audits and validation), with no noted trend or targeted opportunities. We only include comorbidities that are current, actively monitored/managed, and pertinent to the care of our patients. We have not noted a change in denials, which gives us confidence we are not now overdiagnosing.

Our observed mortality has also increased. We, like all institutions, experienced the confounding factor of the COVID-19 pandemic, which coincided with the higher observed mortality over the course of the past 2 years. While the observed mortality (indicating sicker patients assuming no worsening of care processes) may partly explain our increased coding complexity, our decreasing mortality index (observed:expected mortality) suggests that our efforts to improve documentation and coding likely reflect improved capture of missed complexity (Figure).

We understand the concerns raised by Dr. Kerguelen about potential mis(over)coding. As part of this quality initiative, therefore, we plan long-term evaluations of our processes and metrics to better determine and guide our understanding of the impact of what we have already implemented and future interventions. In fact, we are in the process of analyzing additional interventions and hope to share results from these evaluations soon.

Marie Anne Sosa, MD
Tanira Ferreira, MD
Hayley Gershengorn, MD
Melissa Soto
Estin Kelly
Ameena Shrestha
Julianne Burgos
Sandeep Devabhaktuni
Dipen Parekh, MD
Maritza Suarez, MD
University of Miami Hospital and Clinics, Miami, FL
mxs2157@med.miami.edu

Disclosures: None reported.

To the Editor:

I read with interest the article by Sosa and colleagues¹ in which they present some stimulating analyses pertaining to a topic that we have been discussing at my institution for several years. Part of this discussion deals with the complexity of our hospital and how complexity is affected by comorbidity coding.

In 2013, we implemented the International Refined-DRGs (IR-DRGs) system to measure complexity at our hospital in Bogotá, Colombia. Our perception at that time was that the case mix index (CMI) was very low (0.7566), even for a general hospital with a high volume of pathologies with low relative weight (RW). Two medical auditors were assigned to review the medical records in order to improve the quality, quantity, and order of diagnoses. Emphasis was placed on patients with stays longer than 5 days and with only 1 diagnosis coded at admission. Additionally, International Classification of Diseases 10th Revision (World Health Organization version) diagnoses from chapters R (Symptoms and Signs Not Elsewhere Classified) and V through Y (External Causes) were blocked in the electronic health record. With these measures, our CMI increased 74%, reaching 1.3151 by the end of 2021, with a maximum peak of 1.6743 in May 2021, which coincided with the third peak of COVID-19 in Colombia.

However, the article by Sosa and colleagues draws my attention to the following: why do the authors state that their CMI is low and the patient acuity was under-represented? Is this due to a comparison with similar hospitals, or to a recommendation from a regulatory agency? We have found our CMI remains low because of a high volume of nonsurgical care (60%), deliveries, and digestive, respiratory, and urinary pathologies of low RW.

Also, was the perceived low CMI causing problems with payers? And further, how did the authors avoid the risk of artificially increasing the CMI through overdiagnosis of patients, and were there audit mechanisms to avoid this? While there was a clear change in expected mortality, did the observed mortality also change with the strategies implemented? This last question is relevant because, if the observed mortality were maintained, this would provide evidence that a coding problem was the cause of their hospital’s low CMI.

I reiterate my congratulations to the authors for presenting analyses that are very useful to other providers and researchers worldwide interested in addressing management issues related to the correct identification and classification of patients.

Carlos Kerguelen, MD, MA
Fundacion Santa Fe de Bogotá, Bogotá, Colombia
carlos.kerguelen@fsfb.org.co

Disclosures: None reported.

To the Editor:

I read with interest the article by Sosa and colleagues¹ in which they present some stimulating analyses pertaining to a topic that we have been discussing at my institution for several years. Part of this discussion deals with the complexity of our hospital and how complexity is affected by comorbidity coding.

In 2013, we implemented the International Refined-DRGs (IR-DRGs) system to measure complexity at our hospital in Bogotá, Colombia. Our perception at that time was that the case mix index (CMI) was very low (0.7566), even for a general hospital with a high volume of pathologies with low relative weight (RW). Two medical auditors were assigned to review the medical records in order to improve the quality, quantity, and order of diagnoses. Emphasis was placed on patients with stays longer than 5 days and with only 1 diagnosis coded at admission. Additionally, International Classification of Diseases 10th Revision (World Health Organization version) diagnoses from chapters R (Symptoms and Signs Not Elsewhere Classified) and V through Y (External Causes) were blocked in the electronic health record. With these measures, our CMI increased 74%, reaching 1.3151 by the end of 2021, with a maximum peak of 1.6743 in May 2021, which coincided with the third peak of COVID-19 in Colombia.

However, the article by Sosa and colleagues draws my attention to the following: why do the authors state that their CMI is low and the patient acuity was under-represented? Is this due to a comparison with similar hospitals, or to a recommendation from a regulatory agency? We have found our CMI remains low because of a high volume of nonsurgical care (60%), deliveries, and digestive, respiratory, and urinary pathologies of low RW.

Also, was the perceived low CMI causing problems with payers? And further, how did the authors avoid the risk of artificially increasing the CMI through overdiagnosis of patients, and were there audit mechanisms to avoid this? While there was a clear change in expected mortality, did the observed mortality also change with the strategies implemented? This last question is relevant because, if the observed mortality were maintained, this would provide evidence that a coding problem was the cause of their hospital’s low CMI.

I reiterate my congratulations to the authors for presenting analyses that are very useful to other providers and researchers worldwide interested in addressing management issues related to the correct identification and classification of patients.

Carlos Kerguelen, MD, MA
Fundacion Santa Fe de Bogotá, Bogotá, Colombia
carlos.kerguelen@fsfb.org.co

Disclosures: None reported.

To the Editor:

I read with interest the article by Sosa and colleagues¹ in which they present some stimulating analyses pertaining to a topic that we have been discussing at my institution for several years. Part of this discussion deals with the complexity of our hospital and how complexity is affected by comorbidity coding.

In 2013, we implemented the International Refined-DRGs (IR-DRGs) system to measure complexity at our hospital in Bogotá, Colombia. Our perception at that time was that the case mix index (CMI) was very low (0.7566), even for a general hospital with a high volume of pathologies with low relative weight (RW). Two medical auditors were assigned to review the medical records in order to improve the quality, quantity, and order of diagnoses. Emphasis was placed on patients with stays longer than 5 days and with only 1 diagnosis coded at admission. Additionally, International Classification of Diseases 10th Revision (World Health Organization version) diagnoses from chapters R (Symptoms and Signs Not Elsewhere Classified) and V through Y (External Causes) were blocked in the electronic health record. With these measures, our CMI increased 74%, reaching 1.3151 by the end of 2021, with a maximum peak of 1.6743 in May 2021, which coincided with the third peak of COVID-19 in Colombia.

However, the article by Sosa and colleagues draws my attention to the following: why do the authors state that their CMI is low and the patient acuity was under-represented? Is this due to a comparison with similar hospitals, or to a recommendation from a regulatory agency? We have found our CMI remains low because of a high volume of nonsurgical care (60%), deliveries, and digestive, respiratory, and urinary pathologies of low RW.

Also, was the perceived low CMI causing problems with payers? And further, how did the authors avoid the risk of artificially increasing the CMI through overdiagnosis of patients, and were there audit mechanisms to avoid this? While there was a clear change in expected mortality, did the observed mortality also change with the strategies implemented? This last question is relevant because, if the observed mortality were maintained, this would provide evidence that a coding problem was the cause of their hospital’s low CMI.

I reiterate my congratulations to the authors for presenting analyses that are very useful to other providers and researchers worldwide interested in addressing management issues related to the correct identification and classification of patients.

Carlos Kerguelen, MD, MA
Fundacion Santa Fe de Bogotá, Bogotá, Colombia
carlos.kerguelen@fsfb.org.co

Disclosures: None reported.