Accuracy of gene test for thyroid nodules questioned

BALTIMORE – Biopsy results from a commercially available genetic test for ruling out malignancy of thyroid nodules may not provide reliable answers to clinicians and patients.

When fine-needle aspiration biopsy of thyroid nodules comes back inconclusive, clinicians have increasingly utilized the Afirma gene expression classifier (GEC) to rule out malignancy, but a retrospective analysis of almost 200 patients with indeterminate biopsy results along with a pooled analysis of 11 previous studies has raised questions about the negative predictive value of the test.

©Sebastian Kaulitzki/Fotolia

“The Afirma GEC test has substantial variability in performance,” said Dr. Zaid Al-Qurayshi of Tulane University, New Orleans, who reported the results at the annual meeting of the American Association of Endocrine Surgeons. “This variability cannot be explained based on differences in prevalence alone, but may also be the result of intrinsic test properties.”

The Afirma GEC measures the expression of 167 genes to more precisely determine the cancer risk of an indeterminate biopsied thyroid nodule and avoid unnecessary surgery. The test costs approximately $4,800 per nodule.

The researchers undertook the study in light of an American Thyroid Association (ATA) statement last year that concluded that test results are predicated on the clinician knowing the prevalence of malignancy within each indeterminate cytologic category at his/her own institution. Without this information, the performance of the diagnostic tests may vary substantially (Thyroid. 2015;25:760-8).

The single-center, retrospective cohort analysis included 192 patients with 210 indeterminate biopsy results, 145 of whom had surgery with 154 thyroid nodules. With a malignancy prevalence of 45%, the expected negative predictive value (NPV) of the test was estimated to be 85%, Dr. Al-Qurayshi said. However, the actual observed NPV was 69%. “If the prevalence was assumed to be 25%, the expected NPV was estimated to be 94%, while the observed NPV would have been 85%,” Dr. Al-Qurayshi said.

The researchers calculated the expected NPV by adopting the sensitivity and specificity rates of the test as reported in previous studies, while they calculated the observed NPV based on the actual negative rate among the Tulane cohort, Dr. Al-Qurayshi said.

Dr. Al-Qurayshi and colleagues then compared their results with pooled data from 11 other studies of the Afirma GEC. The pooled data analysis included 1,303 patients and yielded a malignancy prevalence of 31.1%, with a range of 29%-35%, and a pooled NPV of 92%, with a range of 87%-96%, Dr. Al-Qurayshi said.

“A lot of previously published studies took the sensitivity and specificity that were previously reported for granted, and now we are showing this sensitivity is all over the place,” Dr. Al-Qurayshi said. “Now, we don’t know which is the true one, and we need a larger clinical trial first to determine the true properties. Then we can ask how the prevalence in one’s institution is affecting the performance of the test.”

In an interview, Dr. Emad Kandil, senior study coauthor, also of Tulane, said the 69% NPV of the Tulane cohort puts the diagnostic scenario “back to ground zero, which is similar to what we had prior to the use of the new commercially available genetic tests.” He added, “A larger, randomized trial of the Afirma GEC test should answer those questions.”

The seminal study for the Afirma GEC, authored by Dr. Erik Alexander of Brigham and Women’s Hospital, Boston, in 2012, reported a 92% NPV with the test (N Engl J Med. 2012;367:705-15).

“The first thought was that they had different results because their population was different,” Dr. Al-Qurayshi said. “The ATA statement noted that it is the clinician’s responsibility to determine if this test is appropriate for their population or not, but the performance of the test doesn’t just depend on the population property, but it also depends on the intrinsic testing properties.”

Dr. Kandil disclosed that he has been a primary investigator in the ENHANCE multicenter study of the Afirma GEC. The other coauthors had no financial disclosures.

BALTIMORE – Biopsy results from a commercially available genetic test for ruling out malignancy of thyroid nodules may not provide reliable answers to clinicians and patients.

When fine-needle aspiration biopsy of thyroid nodules comes back inconclusive, clinicians have increasingly utilized the Afirma gene expression classifier (GEC) to rule out malignancy, but a retrospective analysis of almost 200 patients with indeterminate biopsy results along with a pooled analysis of 11 previous studies has raised questions about the negative predictive value of the test.

©Sebastian Kaulitzki/Fotolia

“The Afirma GEC test has substantial variability in performance,” said Dr. Zaid Al-Qurayshi of Tulane University, New Orleans, who reported the results at the annual meeting of the American Association of Endocrine Surgeons. “This variability cannot be explained based on differences in prevalence alone, but may also be the result of intrinsic test properties.”

The Afirma GEC measures the expression of 167 genes to more precisely determine the cancer risk of an indeterminate biopsied thyroid nodule and avoid unnecessary surgery. The test costs approximately $4,800 per nodule.

The researchers undertook the study in light of an American Thyroid Association (ATA) statement last year that concluded that test results are predicated on the clinician knowing the prevalence of malignancy within each indeterminate cytologic category at his/her own institution. Without this information, the performance of the diagnostic tests may vary substantially (Thyroid. 2015;25:760-8).

The single-center, retrospective cohort analysis included 192 patients with 210 indeterminate biopsy results, 145 of whom had surgery with 154 thyroid nodules. With a malignancy prevalence of 45%, the expected negative predictive value (NPV) of the test was estimated to be 85%, Dr. Al-Qurayshi said. However, the actual observed NPV was 69%. “If the prevalence was assumed to be 25%, the expected NPV was estimated to be 94%, while the observed NPV would have been 85%,” Dr. Al-Qurayshi said.

The researchers calculated the expected NPV by adopting the sensitivity and specificity rates of the test as reported in previous studies, while they calculated the observed NPV based on the actual negative rate among the Tulane cohort, Dr. Al-Qurayshi said.

Dr. Al-Qurayshi and colleagues then compared their results with pooled data from 11 other studies of the Afirma GEC. The pooled data analysis included 1,303 patients and yielded a malignancy prevalence of 31.1%, with a range of 29%-35%, and a pooled NPV of 92%, with a range of 87%-96%, Dr. Al-Qurayshi said.

“A lot of previously published studies took the sensitivity and specificity that were previously reported for granted, and now we are showing this sensitivity is all over the place,” Dr. Al-Qurayshi said. “Now, we don’t know which is the true one, and we need a larger clinical trial first to determine the true properties. Then we can ask how the prevalence in one’s institution is affecting the performance of the test.”

In an interview, Dr. Emad Kandil, senior study coauthor, also of Tulane, said the 69% NPV of the Tulane cohort puts the diagnostic scenario “back to ground zero, which is similar to what we had prior to the use of the new commercially available genetic tests.” He added, “A larger, randomized trial of the Afirma GEC test should answer those questions.”

The seminal study for the Afirma GEC, authored by Dr. Erik Alexander of Brigham and Women’s Hospital, Boston, in 2012, reported a 92% NPV with the test (N Engl J Med. 2012;367:705-15).

“The first thought was that they had different results because their population was different,” Dr. Al-Qurayshi said. “The ATA statement noted that it is the clinician’s responsibility to determine if this test is appropriate for their population or not, but the performance of the test doesn’t just depend on the population property, but it also depends on the intrinsic testing properties.”

Dr. Kandil disclosed that he has been a primary investigator in the ENHANCE multicenter study of the Afirma GEC. The other coauthors had no financial disclosures.

BALTIMORE – Biopsy results from a commercially available genetic test for ruling out malignancy of thyroid nodules may not provide reliable answers to clinicians and patients.

When fine-needle aspiration biopsy of thyroid nodules comes back inconclusive, clinicians have increasingly utilized the Afirma gene expression classifier (GEC) to rule out malignancy, but a retrospective analysis of almost 200 patients with indeterminate biopsy results along with a pooled analysis of 11 previous studies has raised questions about the negative predictive value of the test.

©Sebastian Kaulitzki/Fotolia

“The Afirma GEC test has substantial variability in performance,” said Dr. Zaid Al-Qurayshi of Tulane University, New Orleans, who reported the results at the annual meeting of the American Association of Endocrine Surgeons. “This variability cannot be explained based on differences in prevalence alone, but may also be the result of intrinsic test properties.”

The Afirma GEC measures the expression of 167 genes to more precisely determine the cancer risk of an indeterminate biopsied thyroid nodule and avoid unnecessary surgery. The test costs approximately $4,800 per nodule.

The researchers undertook the study in light of an American Thyroid Association (ATA) statement last year that concluded that test results are predicated on the clinician knowing the prevalence of malignancy within each indeterminate cytologic category at his/her own institution. Without this information, the performance of the diagnostic tests may vary substantially (Thyroid. 2015;25:760-8).

The single-center, retrospective cohort analysis included 192 patients with 210 indeterminate biopsy results, 145 of whom had surgery with 154 thyroid nodules. With a malignancy prevalence of 45%, the expected negative predictive value (NPV) of the test was estimated to be 85%, Dr. Al-Qurayshi said. However, the actual observed NPV was 69%. “If the prevalence was assumed to be 25%, the expected NPV was estimated to be 94%, while the observed NPV would have been 85%,” Dr. Al-Qurayshi said.

The researchers calculated the expected NPV by adopting the sensitivity and specificity rates of the test as reported in previous studies, while they calculated the observed NPV based on the actual negative rate among the Tulane cohort, Dr. Al-Qurayshi said.

Dr. Al-Qurayshi and colleagues then compared their results with pooled data from 11 other studies of the Afirma GEC. The pooled data analysis included 1,303 patients and yielded a malignancy prevalence of 31.1%, with a range of 29%-35%, and a pooled NPV of 92%, with a range of 87%-96%, Dr. Al-Qurayshi said.

“A lot of previously published studies took the sensitivity and specificity that were previously reported for granted, and now we are showing this sensitivity is all over the place,” Dr. Al-Qurayshi said. “Now, we don’t know which is the true one, and we need a larger clinical trial first to determine the true properties. Then we can ask how the prevalence in one’s institution is affecting the performance of the test.”

In an interview, Dr. Emad Kandil, senior study coauthor, also of Tulane, said the 69% NPV of the Tulane cohort puts the diagnostic scenario “back to ground zero, which is similar to what we had prior to the use of the new commercially available genetic tests.” He added, “A larger, randomized trial of the Afirma GEC test should answer those questions.”

The seminal study for the Afirma GEC, authored by Dr. Erik Alexander of Brigham and Women’s Hospital, Boston, in 2012, reported a 92% NPV with the test (N Engl J Med. 2012;367:705-15).

“The first thought was that they had different results because their population was different,” Dr. Al-Qurayshi said. “The ATA statement noted that it is the clinician’s responsibility to determine if this test is appropriate for their population or not, but the performance of the test doesn’t just depend on the population property, but it also depends on the intrinsic testing properties.”

Dr. Kandil disclosed that he has been a primary investigator in the ENHANCE multicenter study of the Afirma GEC. The other coauthors had no financial disclosures.