User login
, according to new research conducted and funded by the board.
“In this study comparing the percentages of 186 categories of medical conditions seen by general internists in office visits and hospital stays with the percentages of 3,461 questions on IM-MOC examinations from 2010 to 2013, 69% of examination questions were concordant with conditions seen,” Bradley Gray, PhD, ABIM senior health services researcher, Philadelphia, and his colleagues said in a report published in JAMA (2017 Jun 13;317[22]:2317-24).
Questions’ discordance with conditions seen specifically in the office outpatient setting and in the hospital was greater. Comparing questions and office-based practice, 58% of questions (2,010) involving 145 conditions were categorized as concordant. Comparing questions and hospital stays only, 42% of questions (1,456) involving 122 conditions were categorized as concordant.
The study did not evaluate the importance of the conditions that were found in questions that were in discordance with what physicians see in practice.
“Most of the discordant conditions where the frequency of questions on the exam was greater than the frequency of conditions seen were conditions that may have been uncommon but were rated as extremely important to patient care by internists responding to a survey involving review of exam blueprints,” Marianne Green, MD, study coauthor and ABIM board member, said in a statement. “An example of this is diagnosis of vasculitis, a rare but painful condition that can slow vital blood supply to tissues and organs. Based on physicians’ input, these conditions continue to be included on the exam.”
Conversely, there are questions that appear infrequently despite the fact that they cover conditions commonly seen in practice.
“For example, the question percentage for hypertension was judged to be concordant because it was similar to hospital stay percentage (1.91% vs. 1.84%), even though the question percentage was much lower than office visit percentage for this condition (1.91% vs. 13.87%),” the authors stated in the report.
ABIM researchers added that certain common conditions do not require more questions “because care guidelines are widely disseminated and more questions in these areas may be repetitive in terms of content and, therefore, do not contribute significantly to the assessment of a physician’s clinical judgment, especially when limited testing time is available.”
The overall percentage was a reasonable range, Andrea Paul, MD, chief medical officer of BoardVitals, a New York-based company that helps doctors prepare for MOC exams, said in an interview.
“Aiming for somewhere between 65% and 75% on commonly experienced patient conditions is reasonable,” Dr. Paul said. “The reason for that is that those common things that physicians see regularly in practice are just that – they are regularly practiced. So, they don’t necessarily need as much review and testing on topics that they are maintaining their own knowledge of on a daily basis.”
It is important for maintenance of certification exams to cover the rarer conditions, compared with what doctors see regularly, she added.
“I think the exams are aiming to cover those in greater detail because, if a rare condition were to walk into someone’s practice, there would be a great lapse in reviewing those obscure or rare conditions, and it might lead to a delay in diagnosing it or a delay in treating one of those conditions,” Dr. Paul said. “While people find it a nuisance to review something they don’t see regularly, that’s the reason that it’s important.”
The study’s findings have helped inform the review of the MOC testing blueprint that began in 2015, the ABIM noted, so the results may not match the current rate of concordance between questions asked and conditions that physicians are seeing. The report’s authors added that not enough new exams have been administered and not enough information on office visits has been made available to determine the latest concordance rates.
Dr. Paul suggested that, even in internal medicine, it might be better to offer examinations that are more specialized.
“Of people taking that exam, a great proportion of them don’t practice in general outpatient internal medicine,” she said. “They are either specialized in oncology or nephrology, or they practice in academia or are researchers. So, although you would think that everyone would have what they define as a general internal medicine practice, a great proportion doesn’t practice general internal medicine, especially outpatient internal medicine.
“The way to get that [concordance] number up would be to create different maintenance of certification exams based on people’s practice type,” Dr. Paul added. “To have an exam that tests what that person’s specific area is ... That would be a way that I could see to improve it.”
All of the researchers are either employed by or affiliated with ABIM. No additional conflicts of interest were disclosed.
In recent years, ABIM examinations have placed greater emphasis on evidence-based medicine. Yet, the ABIM has been too slow to develop rigorous evidence to support the validity of its recertification examination. There will never be a perfect examination, nor will there be a perfect way to assess physician knowledge, understanding, decision making, and clinical skills.
However, the study by Gray et al. represents an important effort to provide evidence on the validity of the recertification examination and its relevance to practicing internists.
The ABIM must strive for continuous improvement and is most relevant and useful when it asks for help and listens to its members, a process that it has been involved with for the past several years. Open, ongoing evaluation of the recertification examination will be essential, along with frank and productive discussion about how to ensure the continuous improvement, excellence, and relevance of the recertification process.
Adam B. Schwartz, MD, of New York University and J. Sanford Schwartz, MD, of the University of Pennsylvania, Philadelphia, made these comments in an editorial (JAMA. 2017 Jun 13;317[22]:2288-9).
In recent years, ABIM examinations have placed greater emphasis on evidence-based medicine. Yet, the ABIM has been too slow to develop rigorous evidence to support the validity of its recertification examination. There will never be a perfect examination, nor will there be a perfect way to assess physician knowledge, understanding, decision making, and clinical skills.
However, the study by Gray et al. represents an important effort to provide evidence on the validity of the recertification examination and its relevance to practicing internists.
The ABIM must strive for continuous improvement and is most relevant and useful when it asks for help and listens to its members, a process that it has been involved with for the past several years. Open, ongoing evaluation of the recertification examination will be essential, along with frank and productive discussion about how to ensure the continuous improvement, excellence, and relevance of the recertification process.
Adam B. Schwartz, MD, of New York University and J. Sanford Schwartz, MD, of the University of Pennsylvania, Philadelphia, made these comments in an editorial (JAMA. 2017 Jun 13;317[22]:2288-9).
In recent years, ABIM examinations have placed greater emphasis on evidence-based medicine. Yet, the ABIM has been too slow to develop rigorous evidence to support the validity of its recertification examination. There will never be a perfect examination, nor will there be a perfect way to assess physician knowledge, understanding, decision making, and clinical skills.
However, the study by Gray et al. represents an important effort to provide evidence on the validity of the recertification examination and its relevance to practicing internists.
The ABIM must strive for continuous improvement and is most relevant and useful when it asks for help and listens to its members, a process that it has been involved with for the past several years. Open, ongoing evaluation of the recertification examination will be essential, along with frank and productive discussion about how to ensure the continuous improvement, excellence, and relevance of the recertification process.
Adam B. Schwartz, MD, of New York University and J. Sanford Schwartz, MD, of the University of Pennsylvania, Philadelphia, made these comments in an editorial (JAMA. 2017 Jun 13;317[22]:2288-9).
, according to new research conducted and funded by the board.
“In this study comparing the percentages of 186 categories of medical conditions seen by general internists in office visits and hospital stays with the percentages of 3,461 questions on IM-MOC examinations from 2010 to 2013, 69% of examination questions were concordant with conditions seen,” Bradley Gray, PhD, ABIM senior health services researcher, Philadelphia, and his colleagues said in a report published in JAMA (2017 Jun 13;317[22]:2317-24).
Questions’ discordance with conditions seen specifically in the office outpatient setting and in the hospital was greater. Comparing questions and office-based practice, 58% of questions (2,010) involving 145 conditions were categorized as concordant. Comparing questions and hospital stays only, 42% of questions (1,456) involving 122 conditions were categorized as concordant.
The study did not evaluate the importance of the conditions that were found in questions that were in discordance with what physicians see in practice.
“Most of the discordant conditions where the frequency of questions on the exam was greater than the frequency of conditions seen were conditions that may have been uncommon but were rated as extremely important to patient care by internists responding to a survey involving review of exam blueprints,” Marianne Green, MD, study coauthor and ABIM board member, said in a statement. “An example of this is diagnosis of vasculitis, a rare but painful condition that can slow vital blood supply to tissues and organs. Based on physicians’ input, these conditions continue to be included on the exam.”
Conversely, there are questions that appear infrequently despite the fact that they cover conditions commonly seen in practice.
“For example, the question percentage for hypertension was judged to be concordant because it was similar to hospital stay percentage (1.91% vs. 1.84%), even though the question percentage was much lower than office visit percentage for this condition (1.91% vs. 13.87%),” the authors stated in the report.
ABIM researchers added that certain common conditions do not require more questions “because care guidelines are widely disseminated and more questions in these areas may be repetitive in terms of content and, therefore, do not contribute significantly to the assessment of a physician’s clinical judgment, especially when limited testing time is available.”
The overall percentage was a reasonable range, Andrea Paul, MD, chief medical officer of BoardVitals, a New York-based company that helps doctors prepare for MOC exams, said in an interview.
“Aiming for somewhere between 65% and 75% on commonly experienced patient conditions is reasonable,” Dr. Paul said. “The reason for that is that those common things that physicians see regularly in practice are just that – they are regularly practiced. So, they don’t necessarily need as much review and testing on topics that they are maintaining their own knowledge of on a daily basis.”
It is important for maintenance of certification exams to cover the rarer conditions, compared with what doctors see regularly, she added.
“I think the exams are aiming to cover those in greater detail because, if a rare condition were to walk into someone’s practice, there would be a great lapse in reviewing those obscure or rare conditions, and it might lead to a delay in diagnosing it or a delay in treating one of those conditions,” Dr. Paul said. “While people find it a nuisance to review something they don’t see regularly, that’s the reason that it’s important.”
The study’s findings have helped inform the review of the MOC testing blueprint that began in 2015, the ABIM noted, so the results may not match the current rate of concordance between questions asked and conditions that physicians are seeing. The report’s authors added that not enough new exams have been administered and not enough information on office visits has been made available to determine the latest concordance rates.
Dr. Paul suggested that, even in internal medicine, it might be better to offer examinations that are more specialized.
“Of people taking that exam, a great proportion of them don’t practice in general outpatient internal medicine,” she said. “They are either specialized in oncology or nephrology, or they practice in academia or are researchers. So, although you would think that everyone would have what they define as a general internal medicine practice, a great proportion doesn’t practice general internal medicine, especially outpatient internal medicine.
“The way to get that [concordance] number up would be to create different maintenance of certification exams based on people’s practice type,” Dr. Paul added. “To have an exam that tests what that person’s specific area is ... That would be a way that I could see to improve it.”
All of the researchers are either employed by or affiliated with ABIM. No additional conflicts of interest were disclosed.
, according to new research conducted and funded by the board.
“In this study comparing the percentages of 186 categories of medical conditions seen by general internists in office visits and hospital stays with the percentages of 3,461 questions on IM-MOC examinations from 2010 to 2013, 69% of examination questions were concordant with conditions seen,” Bradley Gray, PhD, ABIM senior health services researcher, Philadelphia, and his colleagues said in a report published in JAMA (2017 Jun 13;317[22]:2317-24).
Questions’ discordance with conditions seen specifically in the office outpatient setting and in the hospital was greater. Comparing questions and office-based practice, 58% of questions (2,010) involving 145 conditions were categorized as concordant. Comparing questions and hospital stays only, 42% of questions (1,456) involving 122 conditions were categorized as concordant.
The study did not evaluate the importance of the conditions that were found in questions that were in discordance with what physicians see in practice.
“Most of the discordant conditions where the frequency of questions on the exam was greater than the frequency of conditions seen were conditions that may have been uncommon but were rated as extremely important to patient care by internists responding to a survey involving review of exam blueprints,” Marianne Green, MD, study coauthor and ABIM board member, said in a statement. “An example of this is diagnosis of vasculitis, a rare but painful condition that can slow vital blood supply to tissues and organs. Based on physicians’ input, these conditions continue to be included on the exam.”
Conversely, there are questions that appear infrequently despite the fact that they cover conditions commonly seen in practice.
“For example, the question percentage for hypertension was judged to be concordant because it was similar to hospital stay percentage (1.91% vs. 1.84%), even though the question percentage was much lower than office visit percentage for this condition (1.91% vs. 13.87%),” the authors stated in the report.
ABIM researchers added that certain common conditions do not require more questions “because care guidelines are widely disseminated and more questions in these areas may be repetitive in terms of content and, therefore, do not contribute significantly to the assessment of a physician’s clinical judgment, especially when limited testing time is available.”
The overall percentage was a reasonable range, Andrea Paul, MD, chief medical officer of BoardVitals, a New York-based company that helps doctors prepare for MOC exams, said in an interview.
“Aiming for somewhere between 65% and 75% on commonly experienced patient conditions is reasonable,” Dr. Paul said. “The reason for that is that those common things that physicians see regularly in practice are just that – they are regularly practiced. So, they don’t necessarily need as much review and testing on topics that they are maintaining their own knowledge of on a daily basis.”
It is important for maintenance of certification exams to cover the rarer conditions, compared with what doctors see regularly, she added.
“I think the exams are aiming to cover those in greater detail because, if a rare condition were to walk into someone’s practice, there would be a great lapse in reviewing those obscure or rare conditions, and it might lead to a delay in diagnosing it or a delay in treating one of those conditions,” Dr. Paul said. “While people find it a nuisance to review something they don’t see regularly, that’s the reason that it’s important.”
The study’s findings have helped inform the review of the MOC testing blueprint that began in 2015, the ABIM noted, so the results may not match the current rate of concordance between questions asked and conditions that physicians are seeing. The report’s authors added that not enough new exams have been administered and not enough information on office visits has been made available to determine the latest concordance rates.
Dr. Paul suggested that, even in internal medicine, it might be better to offer examinations that are more specialized.
“Of people taking that exam, a great proportion of them don’t practice in general outpatient internal medicine,” she said. “They are either specialized in oncology or nephrology, or they practice in academia or are researchers. So, although you would think that everyone would have what they define as a general internal medicine practice, a great proportion doesn’t practice general internal medicine, especially outpatient internal medicine.
“The way to get that [concordance] number up would be to create different maintenance of certification exams based on people’s practice type,” Dr. Paul added. “To have an exam that tests what that person’s specific area is ... That would be a way that I could see to improve it.”
All of the researchers are either employed by or affiliated with ABIM. No additional conflicts of interest were disclosed.
FROM JAMA