Should pediatricians fret over their falling board scores?

Few pediatricians have warm, fuzzy memories about taking their initial board exam.

But many reacted strongly when they read a recent post on Twitter by Bryan Carmody, MD, who noted that the passing rate for first-time test takers had dipped to its lowest level in 5 years – hitting 81% in 2021, down from 91% 3 years earlier.

“It’s literally an awfully written exam,” replied one person who posted. Another asked: “At what point is the exam just not reflective of clinical practice?” And, inevitably, the question of the effect of COVID-19 surfaced: “Is any of this attributable to pulling early career physicians into the pandemic?”

But Dr. Carmody, an associate professor of pediatrics at the University of Eastern Virginia Medical School, Norfolk, isn’t buying that explanation. He researched board scores for internal medicine, general surgery, and family medicine for 2021. All were stable during the same period, he said, leading him to dismiss the idea that the pandemic drove the decline. “It’s not really clear to me why other specialties wouldn’t have seen similar drops,” Dr. Carmody said.

The slip has caught the attention of the American Board of Pediatrics, according to Judy Schaechter, MD, MBA, who was chair of the department of pediatrics at the University of Miami before taking her post as president and CEO of the American Board of Pediatrics in 2021.

“So, our first question was, was this within the range of what one might expect?” Dr. Schaechter said. “Were there other factors that might have come into play?”

The board performs an extensive analysis every year before releasing scores, and it didn’t uncover any changes in the difficulty or content of the test in 2021, nor did the score that was needed to pass increase. Dr. Schaechter pointed out that the passing rate that year was not unprecedented – in 2016, it also dipped to 81%.

Dr. Schaechter said COVID-19 might have affected test takers. “Remember, pediatrics was different from any other specialty during the pandemic,” she said. The census in pediatric wards around the country dropped dramatically in the first two winters of the pandemic, leaving residents with less hands-on experience with patients and mentorship from attendings – both of which can help test-takers pass the exam.

Eyal Ben-Isaac, MD, an associate professor of the department of pediatrics at the Keck School of Medicine at the University of Southern California, Los Angeles, said residents likely suffered during the pandemic, when noon lectures and grand rounds became virtual events.

“I’m sure that clearly affected a person’s ability to sit and listen and really learn the material, as opposed to either doing it hands on or learning the material from a faculty member face to face,” Dr. Ben-Isaac said.

But how much do the didactic experiences of residency programs contribute to residents’ readiness to take the boards? Thomas Welch, MD, professor and chair emeritus of the department of pediatrics at SUNY Upstate Medical University, Syracuse, credits his own success in advancing through college, medical school, pediatric residency, and nephrology fellowship to his skill as a test taker.

He confirmed his suspicions by conducting a study that evaluated correlations between residents’ performance on the United States Medical Licensing Exam (USMLE) taken during medical school and their board scores after completing residency.

Dr. Welch said he wasn’t surprised to find that “the best predictor of one’s passing the pediatric boards was not the training program in which one worked. It was their performance on Step 2 [taken during the fourth year of medical school] of the USMLE.”

Although Dr. Ben-Isaac felt that changes in residency training opportunities might have partially explained the drop in passing rates, he agreed that other factors contribute to success on boards. As director of the pediatric residency program at Children’s Hospital of Los Angeles from 1994 to 2019, one of his first goals was to increase the pass rate of graduates. He developed a board review course for residents, revising it over time on the basis of resident feedback and adding individual coaching for residents who wanted more help.

“Without a question, it raised our board pass rate to being one of the highest in the country,” he said.

Dr. Welch said that while “being up all night with a sick child teaches you a lot about medicine and certainly makes you a better doctor, it doesn’t do anything to improve your board scores.”

None of the pediatricians was too worried about a 1-year drop in scores, and the consensus was that supporting residents with review courses and coaching on how to take multiple choice tests would raise passing rates.

“There are definitely people who are amazing clinicians who did not pass the boards on their first attempt,” Dr. Ben-Isaac said.

But Dr. Schaechter defended the importance of the examination. “Our first obligation is really to the public,” she said. The ABP’s role is to ensure that pediatricians “provide the care that parents want their kids to have.”

As Dr. Welch put it, “Would I trust someone who didn’t pass the board exam to take care of my own kid? Probably not.”

A version of this article first appeared on Medscape.com.

Few pediatricians have warm, fuzzy memories about taking their initial board exam.

But many reacted strongly when they read a recent post on Twitter by Bryan Carmody, MD, who noted that the passing rate for first-time test takers had dipped to its lowest level in 5 years – hitting 81% in 2021, down from 91% 3 years earlier.

“It’s literally an awfully written exam,” replied one person who posted. Another asked: “At what point is the exam just not reflective of clinical practice?” And, inevitably, the question of the effect of COVID-19 surfaced: “Is any of this attributable to pulling early career physicians into the pandemic?”

But Dr. Carmody, an associate professor of pediatrics at the University of Eastern Virginia Medical School, Norfolk, isn’t buying that explanation. He researched board scores for internal medicine, general surgery, and family medicine for 2021. All were stable during the same period, he said, leading him to dismiss the idea that the pandemic drove the decline. “It’s not really clear to me why other specialties wouldn’t have seen similar drops,” Dr. Carmody said.

The slip has caught the attention of the American Board of Pediatrics, according to Judy Schaechter, MD, MBA, who was chair of the department of pediatrics at the University of Miami before taking her post as president and CEO of the American Board of Pediatrics in 2021.

“So, our first question was, was this within the range of what one might expect?” Dr. Schaechter said. “Were there other factors that might have come into play?”

The board performs an extensive analysis every year before releasing scores, and it didn’t uncover any changes in the difficulty or content of the test in 2021, nor did the score that was needed to pass increase. Dr. Schaechter pointed out that the passing rate that year was not unprecedented – in 2016, it also dipped to 81%.

Dr. Schaechter said COVID-19 might have affected test takers. “Remember, pediatrics was different from any other specialty during the pandemic,” she said. The census in pediatric wards around the country dropped dramatically in the first two winters of the pandemic, leaving residents with less hands-on experience with patients and mentorship from attendings – both of which can help test-takers pass the exam.

Eyal Ben-Isaac, MD, an associate professor of the department of pediatrics at the Keck School of Medicine at the University of Southern California, Los Angeles, said residents likely suffered during the pandemic, when noon lectures and grand rounds became virtual events.

“I’m sure that clearly affected a person’s ability to sit and listen and really learn the material, as opposed to either doing it hands on or learning the material from a faculty member face to face,” Dr. Ben-Isaac said.

But how much do the didactic experiences of residency programs contribute to residents’ readiness to take the boards? Thomas Welch, MD, professor and chair emeritus of the department of pediatrics at SUNY Upstate Medical University, Syracuse, credits his own success in advancing through college, medical school, pediatric residency, and nephrology fellowship to his skill as a test taker.

He confirmed his suspicions by conducting a study that evaluated correlations between residents’ performance on the United States Medical Licensing Exam (USMLE) taken during medical school and their board scores after completing residency.

Dr. Welch said he wasn’t surprised to find that “the best predictor of one’s passing the pediatric boards was not the training program in which one worked. It was their performance on Step 2 [taken during the fourth year of medical school] of the USMLE.”

Although Dr. Ben-Isaac felt that changes in residency training opportunities might have partially explained the drop in passing rates, he agreed that other factors contribute to success on boards. As director of the pediatric residency program at Children’s Hospital of Los Angeles from 1994 to 2019, one of his first goals was to increase the pass rate of graduates. He developed a board review course for residents, revising it over time on the basis of resident feedback and adding individual coaching for residents who wanted more help.

“Without a question, it raised our board pass rate to being one of the highest in the country,” he said.

Dr. Welch said that while “being up all night with a sick child teaches you a lot about medicine and certainly makes you a better doctor, it doesn’t do anything to improve your board scores.”

None of the pediatricians was too worried about a 1-year drop in scores, and the consensus was that supporting residents with review courses and coaching on how to take multiple choice tests would raise passing rates.

“There are definitely people who are amazing clinicians who did not pass the boards on their first attempt,” Dr. Ben-Isaac said.

But Dr. Schaechter defended the importance of the examination. “Our first obligation is really to the public,” she said. The ABP’s role is to ensure that pediatricians “provide the care that parents want their kids to have.”

As Dr. Welch put it, “Would I trust someone who didn’t pass the board exam to take care of my own kid? Probably not.”

A version of this article first appeared on Medscape.com.

Few pediatricians have warm, fuzzy memories about taking their initial board exam.

But many reacted strongly when they read a recent post on Twitter by Bryan Carmody, MD, who noted that the passing rate for first-time test takers had dipped to its lowest level in 5 years – hitting 81% in 2021, down from 91% 3 years earlier.

“It’s literally an awfully written exam,” replied one person who posted. Another asked: “At what point is the exam just not reflective of clinical practice?” And, inevitably, the question of the effect of COVID-19 surfaced: “Is any of this attributable to pulling early career physicians into the pandemic?”

But Dr. Carmody, an associate professor of pediatrics at the University of Eastern Virginia Medical School, Norfolk, isn’t buying that explanation. He researched board scores for internal medicine, general surgery, and family medicine for 2021. All were stable during the same period, he said, leading him to dismiss the idea that the pandemic drove the decline. “It’s not really clear to me why other specialties wouldn’t have seen similar drops,” Dr. Carmody said.

The slip has caught the attention of the American Board of Pediatrics, according to Judy Schaechter, MD, MBA, who was chair of the department of pediatrics at the University of Miami before taking her post as president and CEO of the American Board of Pediatrics in 2021.

“So, our first question was, was this within the range of what one might expect?” Dr. Schaechter said. “Were there other factors that might have come into play?”

The board performs an extensive analysis every year before releasing scores, and it didn’t uncover any changes in the difficulty or content of the test in 2021, nor did the score that was needed to pass increase. Dr. Schaechter pointed out that the passing rate that year was not unprecedented – in 2016, it also dipped to 81%.

Dr. Schaechter said COVID-19 might have affected test takers. “Remember, pediatrics was different from any other specialty during the pandemic,” she said. The census in pediatric wards around the country dropped dramatically in the first two winters of the pandemic, leaving residents with less hands-on experience with patients and mentorship from attendings – both of which can help test-takers pass the exam.

Eyal Ben-Isaac, MD, an associate professor of the department of pediatrics at the Keck School of Medicine at the University of Southern California, Los Angeles, said residents likely suffered during the pandemic, when noon lectures and grand rounds became virtual events.

“I’m sure that clearly affected a person’s ability to sit and listen and really learn the material, as opposed to either doing it hands on or learning the material from a faculty member face to face,” Dr. Ben-Isaac said.

But how much do the didactic experiences of residency programs contribute to residents’ readiness to take the boards? Thomas Welch, MD, professor and chair emeritus of the department of pediatrics at SUNY Upstate Medical University, Syracuse, credits his own success in advancing through college, medical school, pediatric residency, and nephrology fellowship to his skill as a test taker.

He confirmed his suspicions by conducting a study that evaluated correlations between residents’ performance on the United States Medical Licensing Exam (USMLE) taken during medical school and their board scores after completing residency.

Dr. Welch said he wasn’t surprised to find that “the best predictor of one’s passing the pediatric boards was not the training program in which one worked. It was their performance on Step 2 [taken during the fourth year of medical school] of the USMLE.”

Although Dr. Ben-Isaac felt that changes in residency training opportunities might have partially explained the drop in passing rates, he agreed that other factors contribute to success on boards. As director of the pediatric residency program at Children’s Hospital of Los Angeles from 1994 to 2019, one of his first goals was to increase the pass rate of graduates. He developed a board review course for residents, revising it over time on the basis of resident feedback and adding individual coaching for residents who wanted more help.

“Without a question, it raised our board pass rate to being one of the highest in the country,” he said.

Dr. Welch said that while “being up all night with a sick child teaches you a lot about medicine and certainly makes you a better doctor, it doesn’t do anything to improve your board scores.”

None of the pediatricians was too worried about a 1-year drop in scores, and the consensus was that supporting residents with review courses and coaching on how to take multiple choice tests would raise passing rates.

“There are definitely people who are amazing clinicians who did not pass the boards on their first attempt,” Dr. Ben-Isaac said.

But Dr. Schaechter defended the importance of the examination. “Our first obligation is really to the public,” she said. The ABP’s role is to ensure that pediatricians “provide the care that parents want their kids to have.”

As Dr. Welch put it, “Would I trust someone who didn’t pass the board exam to take care of my own kid? Probably not.”

A version of this article first appeared on Medscape.com.