Evidence-based medicine: How it becomes a 4-letter word

One day a residency program decided to put its evidence-based medicine (EBM) curriculum to good use. A group of faculty and residents conducted a thorough review of the evidence regarding liquid-based cytoloutilizegy vs conventional Pap smears. They identified the key national recommendations and reviewed the supporting evidence behind each recommendation, tracing back to the individual studies themselves.^1-3

Based on the review, the group concluded that there was insufficient evidence to recommend one method of screening over another, but that there were situations in which one method might be preferred. They presented the evidence and their conclusions to the majority of faculty and residents at grand rounds. Following the presentation, the larger group discussed the relative merits of each screening method and decided the elements of evidence that supported the liquid test were more relevant to the practice than the conventional Pap smear. As a result, a decision was made by the group to stop carrying supplies for the conventional Pap smear. While the decision seemed reasonable on the level of an individual practitioner, several faculty and residents were unhappy with the “evidence-based” decision.

KAP theory and EBM

KAP theory identifies Knowledge, Attitudes, and Practice beliefs as key elements that drive healthcare providers’ decisions about medical care. In a sense EBM represents knowledge.⁴ There is a collective body of medical knowledge in the form of research, which represents “the evidence.” And there is what the healthcare provider himself “knows.” A major purpose of healthcare recommendations, point of care information systems, and best practice guidelines is to help the healthcare provider’s individual medical knowledge reflect the collective body of evidence.

For the purposes of this example, evidence will be considered absolute, inadequate, conditional, or relative. Absolute evidence occurs when there is clearly a correct answer. For example, the net benefits of aspirin for the treatment of myocardial infarction are clear. However, for most topics the evidence is not absolute; rather, it is inconclusive.⁵ The evidence may be inconclusive because it is inadequate—eg, insufficient research, conflicting studies, or research on peripheral topics. As an example, studies have demonstrated that aspirin decreases colorectal polyps, which may or may not be peripheral to the question of whether aspirin prevents colorectal cancer.

The evidence can also be conditional, meaning that in some defined instances the net benefit is clear. However, extending this net benefit beyond these instances is less clear. For example, patients at high risk for cardiovascular disease have a clear net benefit in taking aspirin for myocardial infarction prevention. Finally, the evidence may be relative, with a balance of known benefits and known risks.⁶ Using the aspirin example for cardiovascular disease prevention, patients at moderate risk receive benefit from aspirin in preventing myocardial infarction but at a risk cost of increased bleeding.

When the evidence is inconclusive, the second and third aspects of KAP theory—attitudes and practice beliefs—become very important. Healthcare providers and patients may arrive at different conclusions based on different viewpoints. On an individual level, healthcare practitioners use tools such as shared decision-making and patient-centered care to reach decisions.^6,7 However, inconclusive evidence provides a unique challenge when trying to develop local, regional, or national standards.

Evidence heresy

EBM frequently has negative connotations. In a room full of healthcare providers, some will believe that EBM should revolutionize the practice of medicine,⁸ and some that EBM has limited utility.⁹ How does this happen? The above scenario serves as a useful example, highlighting 3 misuses of the term “evidence” that frequently give EBM a bad name.

First, inconclusive evidence should not be stated in absolute terms; rather, it is more helpful to explicitly state what we know and the limits of what we know. Shaughnessy and Slawson wrote, “Absolute certainty is absolutely impossible, and we do not have to wait for that, of course.”¹⁰ This reflects the paucity of topics with certain evidence and highlights the need for clinicians to act on the available information. Every clinician necessarily utilizes this skill on a daily basis. The clinician has to become an Information Master¹¹ and know not only the end result of what the evidence indicates but also the facts supporting the end results and how those facts apply to the care of an unique individual.¹² However, taking this a step further and stating that one answer or option is absolutely correct in all cases ventures into dangerous ground. During the residency’s discussion of cervical cancer screening tests, the group recognized the merits of both options verbally but, the act of removing all conventional Pap smear supplies implied the nonverbal judgment that liquid-based technology was an absolute correct answer.

Second, while attitudes and practice beliefs can be used to weigh elements of evidence to reach a final conclusion, the conclusion should not negate other perspectives. An important skill of an adept clinician is the ability to interweave the healthcare provider’s and the patient’s attitudes and practice beliefs into the body of existing evidence to determine the appropriate intervention.¹³ However, attitudes and practice beliefs vary from individual to individual and from community to community. When these factors play a critical role in defining the appropriate action based on the evidence, how attitudes and practice beliefs are used should be explicitly stated. In the Pap smear example, the pivotal issue of contention was the belief about whether individual practitioners should act as stewards of limited healthcare resources. Proponents of using solely the liquid-based Pap smear felt the cost problem was a national issue and that the actions of the individual clinician had little impact on global healthcare costs. Others felt their local actions affected insurance premiums, leading directly to decreased healthcare access.¹¹ For cervical cancer, the key impact on mortality is getting any form of screening.¹ Using the liquid-based method for low-risk women may increase cervical cancer mortality by increasing costs and decreasing healthcare access. Removing conventional Pap smears disempowered the latter group of clinicians from implementing their practice beliefs and attitudes.

Finally, a conclusion should not be labeled as “evidence-based” when it is really made on other grounds such as economics, law, ethics, convenience, social values, or policy. Certainly, reviewing medical evidence is an important step in making decisions. However, the process for making decisions on these factors should be held to the same standards as making medical evidence decisions. This includes defining the process and explicitly stating the basis by which final decision will be made. The US is very conflicted when it comes to dealing with these non-evidence issues. We have no national standard for incorporating costs into healthcare decisions.^14,15 With respect to healthcare delivery, we have a wide range of social values that are sometimes disproportionate to logical expectations.¹⁶ Few effective systems are in place to incorporate these elements in healthcare decisions and, as a result, “evidence” is often used as a code word to focus on other issues.

For the Pap smear example, the decision factors were really economics, law, and systems of care. Proponents of the liquid-based method cited the community standard of care, fear of malpractice, patient expectations of receiving the latest technology, and the ease of adopting one screening method for the entire office. Others felt these issues, although important, were secondary to the lack of evidence supporting a liquid-based system as a sole screening method. For lowrisk women, adopting the liquid-based method only makes economic sense if screening is done every 2 or 3 years.¹⁷ However, many low-risk women still favor performing a Pap smear annually.¹⁸ As a result a decision-making process other than the strict EBM method, focusing on other factors would be necessary to change the practice standard.

Conclusions: Recognize the limitations of EBM

Cervical cancer screening serves as a common example of a difficult decision healthcare providers are faced with on a daily basis—what to do when evidence, based on patient oriented outcomes, is inconclusive. Providers do not have the luxury of merely stating the evidence is inconclusive; they must act. Frequently decisions are based on attitudes and practice beliefs in a broader context of unique economic, legal, and practice environments.

EBM is one tool in the decision-making armamentarium. It is a very powerful tool and has had a very positive impact on healthcare. Its methods have been well defined and explicitly stated. However, failing to recognize its limitations and making a decision under the rubric of EBM, when other variables are clearly playing a role, perpetuates the perception of its limited utility. Advocates of EBM need to wield this instrument carefully and judiciously.

CORRESPONDENCE
Alex Krist, MD, 3825 Charles Stewart Drive, Fairfax VA 22033. E-mail: ahkrist@vcu.edu

One day a residency program decided to put its evidence-based medicine (EBM) curriculum to good use. A group of faculty and residents conducted a thorough review of the evidence regarding liquid-based cytoloutilizegy vs conventional Pap smears. They identified the key national recommendations and reviewed the supporting evidence behind each recommendation, tracing back to the individual studies themselves.^1-3

Based on the review, the group concluded that there was insufficient evidence to recommend one method of screening over another, but that there were situations in which one method might be preferred. They presented the evidence and their conclusions to the majority of faculty and residents at grand rounds. Following the presentation, the larger group discussed the relative merits of each screening method and decided the elements of evidence that supported the liquid test were more relevant to the practice than the conventional Pap smear. As a result, a decision was made by the group to stop carrying supplies for the conventional Pap smear. While the decision seemed reasonable on the level of an individual practitioner, several faculty and residents were unhappy with the “evidence-based” decision.

KAP theory and EBM

KAP theory identifies Knowledge, Attitudes, and Practice beliefs as key elements that drive healthcare providers’ decisions about medical care. In a sense EBM represents knowledge.⁴ There is a collective body of medical knowledge in the form of research, which represents “the evidence.” And there is what the healthcare provider himself “knows.” A major purpose of healthcare recommendations, point of care information systems, and best practice guidelines is to help the healthcare provider’s individual medical knowledge reflect the collective body of evidence.

For the purposes of this example, evidence will be considered absolute, inadequate, conditional, or relative. Absolute evidence occurs when there is clearly a correct answer. For example, the net benefits of aspirin for the treatment of myocardial infarction are clear. However, for most topics the evidence is not absolute; rather, it is inconclusive.⁵ The evidence may be inconclusive because it is inadequate—eg, insufficient research, conflicting studies, or research on peripheral topics. As an example, studies have demonstrated that aspirin decreases colorectal polyps, which may or may not be peripheral to the question of whether aspirin prevents colorectal cancer.

The evidence can also be conditional, meaning that in some defined instances the net benefit is clear. However, extending this net benefit beyond these instances is less clear. For example, patients at high risk for cardiovascular disease have a clear net benefit in taking aspirin for myocardial infarction prevention. Finally, the evidence may be relative, with a balance of known benefits and known risks.⁶ Using the aspirin example for cardiovascular disease prevention, patients at moderate risk receive benefit from aspirin in preventing myocardial infarction but at a risk cost of increased bleeding.

When the evidence is inconclusive, the second and third aspects of KAP theory—attitudes and practice beliefs—become very important. Healthcare providers and patients may arrive at different conclusions based on different viewpoints. On an individual level, healthcare practitioners use tools such as shared decision-making and patient-centered care to reach decisions.^6,7 However, inconclusive evidence provides a unique challenge when trying to develop local, regional, or national standards.

Evidence heresy

EBM frequently has negative connotations. In a room full of healthcare providers, some will believe that EBM should revolutionize the practice of medicine,⁸ and some that EBM has limited utility.⁹ How does this happen? The above scenario serves as a useful example, highlighting 3 misuses of the term “evidence” that frequently give EBM a bad name.

First, inconclusive evidence should not be stated in absolute terms; rather, it is more helpful to explicitly state what we know and the limits of what we know. Shaughnessy and Slawson wrote, “Absolute certainty is absolutely impossible, and we do not have to wait for that, of course.”¹⁰ This reflects the paucity of topics with certain evidence and highlights the need for clinicians to act on the available information. Every clinician necessarily utilizes this skill on a daily basis. The clinician has to become an Information Master¹¹ and know not only the end result of what the evidence indicates but also the facts supporting the end results and how those facts apply to the care of an unique individual.¹² However, taking this a step further and stating that one answer or option is absolutely correct in all cases ventures into dangerous ground. During the residency’s discussion of cervical cancer screening tests, the group recognized the merits of both options verbally but, the act of removing all conventional Pap smear supplies implied the nonverbal judgment that liquid-based technology was an absolute correct answer.

Second, while attitudes and practice beliefs can be used to weigh elements of evidence to reach a final conclusion, the conclusion should not negate other perspectives. An important skill of an adept clinician is the ability to interweave the healthcare provider’s and the patient’s attitudes and practice beliefs into the body of existing evidence to determine the appropriate intervention.¹³ However, attitudes and practice beliefs vary from individual to individual and from community to community. When these factors play a critical role in defining the appropriate action based on the evidence, how attitudes and practice beliefs are used should be explicitly stated. In the Pap smear example, the pivotal issue of contention was the belief about whether individual practitioners should act as stewards of limited healthcare resources. Proponents of using solely the liquid-based Pap smear felt the cost problem was a national issue and that the actions of the individual clinician had little impact on global healthcare costs. Others felt their local actions affected insurance premiums, leading directly to decreased healthcare access.¹¹ For cervical cancer, the key impact on mortality is getting any form of screening.¹ Using the liquid-based method for low-risk women may increase cervical cancer mortality by increasing costs and decreasing healthcare access. Removing conventional Pap smears disempowered the latter group of clinicians from implementing their practice beliefs and attitudes.

Finally, a conclusion should not be labeled as “evidence-based” when it is really made on other grounds such as economics, law, ethics, convenience, social values, or policy. Certainly, reviewing medical evidence is an important step in making decisions. However, the process for making decisions on these factors should be held to the same standards as making medical evidence decisions. This includes defining the process and explicitly stating the basis by which final decision will be made. The US is very conflicted when it comes to dealing with these non-evidence issues. We have no national standard for incorporating costs into healthcare decisions.^14,15 With respect to healthcare delivery, we have a wide range of social values that are sometimes disproportionate to logical expectations.¹⁶ Few effective systems are in place to incorporate these elements in healthcare decisions and, as a result, “evidence” is often used as a code word to focus on other issues.

For the Pap smear example, the decision factors were really economics, law, and systems of care. Proponents of the liquid-based method cited the community standard of care, fear of malpractice, patient expectations of receiving the latest technology, and the ease of adopting one screening method for the entire office. Others felt these issues, although important, were secondary to the lack of evidence supporting a liquid-based system as a sole screening method. For lowrisk women, adopting the liquid-based method only makes economic sense if screening is done every 2 or 3 years.¹⁷ However, many low-risk women still favor performing a Pap smear annually.¹⁸ As a result a decision-making process other than the strict EBM method, focusing on other factors would be necessary to change the practice standard.

Conclusions: Recognize the limitations of EBM

Cervical cancer screening serves as a common example of a difficult decision healthcare providers are faced with on a daily basis—what to do when evidence, based on patient oriented outcomes, is inconclusive. Providers do not have the luxury of merely stating the evidence is inconclusive; they must act. Frequently decisions are based on attitudes and practice beliefs in a broader context of unique economic, legal, and practice environments.

EBM is one tool in the decision-making armamentarium. It is a very powerful tool and has had a very positive impact on healthcare. Its methods have been well defined and explicitly stated. However, failing to recognize its limitations and making a decision under the rubric of EBM, when other variables are clearly playing a role, perpetuates the perception of its limited utility. Advocates of EBM need to wield this instrument carefully and judiciously.

CORRESPONDENCE
Alex Krist, MD, 3825 Charles Stewart Drive, Fairfax VA 22033. E-mail: ahkrist@vcu.edu

One day a residency program decided to put its evidence-based medicine (EBM) curriculum to good use. A group of faculty and residents conducted a thorough review of the evidence regarding liquid-based cytoloutilizegy vs conventional Pap smears. They identified the key national recommendations and reviewed the supporting evidence behind each recommendation, tracing back to the individual studies themselves.^1-3

Based on the review, the group concluded that there was insufficient evidence to recommend one method of screening over another, but that there were situations in which one method might be preferred. They presented the evidence and their conclusions to the majority of faculty and residents at grand rounds. Following the presentation, the larger group discussed the relative merits of each screening method and decided the elements of evidence that supported the liquid test were more relevant to the practice than the conventional Pap smear. As a result, a decision was made by the group to stop carrying supplies for the conventional Pap smear. While the decision seemed reasonable on the level of an individual practitioner, several faculty and residents were unhappy with the “evidence-based” decision.

KAP theory and EBM

KAP theory identifies Knowledge, Attitudes, and Practice beliefs as key elements that drive healthcare providers’ decisions about medical care. In a sense EBM represents knowledge.⁴ There is a collective body of medical knowledge in the form of research, which represents “the evidence.” And there is what the healthcare provider himself “knows.” A major purpose of healthcare recommendations, point of care information systems, and best practice guidelines is to help the healthcare provider’s individual medical knowledge reflect the collective body of evidence.

For the purposes of this example, evidence will be considered absolute, inadequate, conditional, or relative. Absolute evidence occurs when there is clearly a correct answer. For example, the net benefits of aspirin for the treatment of myocardial infarction are clear. However, for most topics the evidence is not absolute; rather, it is inconclusive.⁵ The evidence may be inconclusive because it is inadequate—eg, insufficient research, conflicting studies, or research on peripheral topics. As an example, studies have demonstrated that aspirin decreases colorectal polyps, which may or may not be peripheral to the question of whether aspirin prevents colorectal cancer.

The evidence can also be conditional, meaning that in some defined instances the net benefit is clear. However, extending this net benefit beyond these instances is less clear. For example, patients at high risk for cardiovascular disease have a clear net benefit in taking aspirin for myocardial infarction prevention. Finally, the evidence may be relative, with a balance of known benefits and known risks.⁶ Using the aspirin example for cardiovascular disease prevention, patients at moderate risk receive benefit from aspirin in preventing myocardial infarction but at a risk cost of increased bleeding.

When the evidence is inconclusive, the second and third aspects of KAP theory—attitudes and practice beliefs—become very important. Healthcare providers and patients may arrive at different conclusions based on different viewpoints. On an individual level, healthcare practitioners use tools such as shared decision-making and patient-centered care to reach decisions.^6,7 However, inconclusive evidence provides a unique challenge when trying to develop local, regional, or national standards.

Evidence heresy

EBM frequently has negative connotations. In a room full of healthcare providers, some will believe that EBM should revolutionize the practice of medicine,⁸ and some that EBM has limited utility.⁹ How does this happen? The above scenario serves as a useful example, highlighting 3 misuses of the term “evidence” that frequently give EBM a bad name.

First, inconclusive evidence should not be stated in absolute terms; rather, it is more helpful to explicitly state what we know and the limits of what we know. Shaughnessy and Slawson wrote, “Absolute certainty is absolutely impossible, and we do not have to wait for that, of course.”¹⁰ This reflects the paucity of topics with certain evidence and highlights the need for clinicians to act on the available information. Every clinician necessarily utilizes this skill on a daily basis. The clinician has to become an Information Master¹¹ and know not only the end result of what the evidence indicates but also the facts supporting the end results and how those facts apply to the care of an unique individual.¹² However, taking this a step further and stating that one answer or option is absolutely correct in all cases ventures into dangerous ground. During the residency’s discussion of cervical cancer screening tests, the group recognized the merits of both options verbally but, the act of removing all conventional Pap smear supplies implied the nonverbal judgment that liquid-based technology was an absolute correct answer.

Second, while attitudes and practice beliefs can be used to weigh elements of evidence to reach a final conclusion, the conclusion should not negate other perspectives. An important skill of an adept clinician is the ability to interweave the healthcare provider’s and the patient’s attitudes and practice beliefs into the body of existing evidence to determine the appropriate intervention.¹³ However, attitudes and practice beliefs vary from individual to individual and from community to community. When these factors play a critical role in defining the appropriate action based on the evidence, how attitudes and practice beliefs are used should be explicitly stated. In the Pap smear example, the pivotal issue of contention was the belief about whether individual practitioners should act as stewards of limited healthcare resources. Proponents of using solely the liquid-based Pap smear felt the cost problem was a national issue and that the actions of the individual clinician had little impact on global healthcare costs. Others felt their local actions affected insurance premiums, leading directly to decreased healthcare access.¹¹ For cervical cancer, the key impact on mortality is getting any form of screening.¹ Using the liquid-based method for low-risk women may increase cervical cancer mortality by increasing costs and decreasing healthcare access. Removing conventional Pap smears disempowered the latter group of clinicians from implementing their practice beliefs and attitudes.

Finally, a conclusion should not be labeled as “evidence-based” when it is really made on other grounds such as economics, law, ethics, convenience, social values, or policy. Certainly, reviewing medical evidence is an important step in making decisions. However, the process for making decisions on these factors should be held to the same standards as making medical evidence decisions. This includes defining the process and explicitly stating the basis by which final decision will be made. The US is very conflicted when it comes to dealing with these non-evidence issues. We have no national standard for incorporating costs into healthcare decisions.^14,15 With respect to healthcare delivery, we have a wide range of social values that are sometimes disproportionate to logical expectations.¹⁶ Few effective systems are in place to incorporate these elements in healthcare decisions and, as a result, “evidence” is often used as a code word to focus on other issues.

For the Pap smear example, the decision factors were really economics, law, and systems of care. Proponents of the liquid-based method cited the community standard of care, fear of malpractice, patient expectations of receiving the latest technology, and the ease of adopting one screening method for the entire office. Others felt these issues, although important, were secondary to the lack of evidence supporting a liquid-based system as a sole screening method. For lowrisk women, adopting the liquid-based method only makes economic sense if screening is done every 2 or 3 years.¹⁷ However, many low-risk women still favor performing a Pap smear annually.¹⁸ As a result a decision-making process other than the strict EBM method, focusing on other factors would be necessary to change the practice standard.

Conclusions: Recognize the limitations of EBM

Cervical cancer screening serves as a common example of a difficult decision healthcare providers are faced with on a daily basis—what to do when evidence, based on patient oriented outcomes, is inconclusive. Providers do not have the luxury of merely stating the evidence is inconclusive; they must act. Frequently decisions are based on attitudes and practice beliefs in a broader context of unique economic, legal, and practice environments.

EBM is one tool in the decision-making armamentarium. It is a very powerful tool and has had a very positive impact on healthcare. Its methods have been well defined and explicitly stated. However, failing to recognize its limitations and making a decision under the rubric of EBM, when other variables are clearly playing a role, perpetuates the perception of its limited utility. Advocates of EBM need to wield this instrument carefully and judiciously.

CORRESPONDENCE
Alex Krist, MD, 3825 Charles Stewart Drive, Fairfax VA 22033. E-mail: ahkrist@vcu.edu