Commercial board-review courses offer no bang for the buck

CHICAGO – A national survey of 1,246 residents found no evidence that taking a commercial board-review course provides any benefit in passing the American Board of Surgery certifying exam.

And none of the courses was significantly better than another, reported Dr. Mark Malangoni, an ACS Fellow and ABS associate executive director.

“Exam-review courses are now a multibillion-dollar industry … but there is little evidence to suggest they actually improve exam performance,” he said.

Dr. Malangoni and his ABS colleagues electronically surveyed 1,396 candidates who took the ABS certifying exam (CE) during the 2012-2013 academic year. Of the 1,246 who responded, 974 (78%) had taken a review course. Another 140 candidates declined to respond to the two-question survey, but their CE test results were available for comparison.

Among first-time examinees, the CE pass rate was 83.7% with any review course, 80.7% with no course, and 75.6% for the nonresponders, Dr. Malangoni said at the annual meeting of the Central Surgical Association.

Among those who had taken the CE before, pass rates were 77%, 69%, and 58%, respectively, with the intergroup difference statistically significant only for the nonresponders (P value = .01).

Despite the lack of benefit, however, most examinees (64%) believed that the review course improved their preparation for the CE, he said.

Those repeating the CE were significantly more likely than first-time examinees to take a review course (84.6% vs. 76.1%; P = .002).

Several candidates decided that multiple review courses might be better: 16% of the first-time examinees and 32% of the repeat examinees. Again, pass rates were not significantly different with multiple courses, Dr. Malangoni said.

The survey identified nine commercially available courses, but five were not analyzed because there were no enrollees or a low number (≤ 40) of enrollees. The remaining four courses were assigned a letter for anonymity. Enrollment in all courses ranged from 21 to 706 students.

In multivariate analysis, the only significant predictor for passing the initial CE was the qualifying examination scale score (Odds ratio, 1.09; P < .001). Course D trended toward a benefit, but the 99% confidence intervals overlapped and the P value did not meet the predetermined threshold of ≤ .01 (OR, 3.25; P = .02; 99% CI 0.86-12.22), he said.

For repeat examinees, no variables including the four review courses, gender, program size, or international medical graduation significantly predicted success on the CE exam.

“Given the time and effort, and expense, we feel that CE candidates should consider these results when assessing how best to prepare for this examination,” Dr. Malangoni concluded.

The study is not the first to question the benefit of review courses for already cash-strapped medical students and residents (N. Engl. J. Med. 2011:365:104-5), he noted.

Dr. Malangoni balked, however, at suggesting review courses may not be worth the investment for any candidate.

“Our results are applicable to a large group; however, it is not possible to determine when taking a review course would be beneficial for a specific candidate,” he said in an interview.

Dr. Malangoni reported having no financial disclosures.

pwendling@frontlinemedcom.com

On Twitter @pwendl

CHICAGO – A national survey of 1,246 residents found no evidence that taking a commercial board-review course provides any benefit in passing the American Board of Surgery certifying exam.

And none of the courses was significantly better than another, reported Dr. Mark Malangoni, an ACS Fellow and ABS associate executive director.

“Exam-review courses are now a multibillion-dollar industry … but there is little evidence to suggest they actually improve exam performance,” he said.

Dr. Malangoni and his ABS colleagues electronically surveyed 1,396 candidates who took the ABS certifying exam (CE) during the 2012-2013 academic year. Of the 1,246 who responded, 974 (78%) had taken a review course. Another 140 candidates declined to respond to the two-question survey, but their CE test results were available for comparison.

Among first-time examinees, the CE pass rate was 83.7% with any review course, 80.7% with no course, and 75.6% for the nonresponders, Dr. Malangoni said at the annual meeting of the Central Surgical Association.

Among those who had taken the CE before, pass rates were 77%, 69%, and 58%, respectively, with the intergroup difference statistically significant only for the nonresponders (P value = .01).

Despite the lack of benefit, however, most examinees (64%) believed that the review course improved their preparation for the CE, he said.

Those repeating the CE were significantly more likely than first-time examinees to take a review course (84.6% vs. 76.1%; P = .002).

Several candidates decided that multiple review courses might be better: 16% of the first-time examinees and 32% of the repeat examinees. Again, pass rates were not significantly different with multiple courses, Dr. Malangoni said.

The survey identified nine commercially available courses, but five were not analyzed because there were no enrollees or a low number (≤ 40) of enrollees. The remaining four courses were assigned a letter for anonymity. Enrollment in all courses ranged from 21 to 706 students.

In multivariate analysis, the only significant predictor for passing the initial CE was the qualifying examination scale score (Odds ratio, 1.09; P < .001). Course D trended toward a benefit, but the 99% confidence intervals overlapped and the P value did not meet the predetermined threshold of ≤ .01 (OR, 3.25; P = .02; 99% CI 0.86-12.22), he said.

For repeat examinees, no variables including the four review courses, gender, program size, or international medical graduation significantly predicted success on the CE exam.

“Given the time and effort, and expense, we feel that CE candidates should consider these results when assessing how best to prepare for this examination,” Dr. Malangoni concluded.

The study is not the first to question the benefit of review courses for already cash-strapped medical students and residents (N. Engl. J. Med. 2011:365:104-5), he noted.

Dr. Malangoni balked, however, at suggesting review courses may not be worth the investment for any candidate.

“Our results are applicable to a large group; however, it is not possible to determine when taking a review course would be beneficial for a specific candidate,” he said in an interview.

Dr. Malangoni reported having no financial disclosures.

pwendling@frontlinemedcom.com

On Twitter @pwendl

CHICAGO – A national survey of 1,246 residents found no evidence that taking a commercial board-review course provides any benefit in passing the American Board of Surgery certifying exam.

And none of the courses was significantly better than another, reported Dr. Mark Malangoni, an ACS Fellow and ABS associate executive director.

“Exam-review courses are now a multibillion-dollar industry … but there is little evidence to suggest they actually improve exam performance,” he said.

Dr. Malangoni and his ABS colleagues electronically surveyed 1,396 candidates who took the ABS certifying exam (CE) during the 2012-2013 academic year. Of the 1,246 who responded, 974 (78%) had taken a review course. Another 140 candidates declined to respond to the two-question survey, but their CE test results were available for comparison.

Among first-time examinees, the CE pass rate was 83.7% with any review course, 80.7% with no course, and 75.6% for the nonresponders, Dr. Malangoni said at the annual meeting of the Central Surgical Association.

Among those who had taken the CE before, pass rates were 77%, 69%, and 58%, respectively, with the intergroup difference statistically significant only for the nonresponders (P value = .01).

Despite the lack of benefit, however, most examinees (64%) believed that the review course improved their preparation for the CE, he said.

Those repeating the CE were significantly more likely than first-time examinees to take a review course (84.6% vs. 76.1%; P = .002).

Several candidates decided that multiple review courses might be better: 16% of the first-time examinees and 32% of the repeat examinees. Again, pass rates were not significantly different with multiple courses, Dr. Malangoni said.

The survey identified nine commercially available courses, but five were not analyzed because there were no enrollees or a low number (≤ 40) of enrollees. The remaining four courses were assigned a letter for anonymity. Enrollment in all courses ranged from 21 to 706 students.

In multivariate analysis, the only significant predictor for passing the initial CE was the qualifying examination scale score (Odds ratio, 1.09; P < .001). Course D trended toward a benefit, but the 99% confidence intervals overlapped and the P value did not meet the predetermined threshold of ≤ .01 (OR, 3.25; P = .02; 99% CI 0.86-12.22), he said.

For repeat examinees, no variables including the four review courses, gender, program size, or international medical graduation significantly predicted success on the CE exam.

“Given the time and effort, and expense, we feel that CE candidates should consider these results when assessing how best to prepare for this examination,” Dr. Malangoni concluded.

The study is not the first to question the benefit of review courses for already cash-strapped medical students and residents (N. Engl. J. Med. 2011:365:104-5), he noted.

Dr. Malangoni balked, however, at suggesting review courses may not be worth the investment for any candidate.

“Our results are applicable to a large group; however, it is not possible to determine when taking a review course would be beneficial for a specific candidate,” he said in an interview.

Dr. Malangoni reported having no financial disclosures.

pwendling@frontlinemedcom.com

On Twitter @pwendl