User login
, while providing greater alignment with pathology-based surveillance intervals, based on a randomized controlled trial.
These findings suggest that autonomous AI may one day replace histologic assessment of diminutive polyps, reported lead author Roupen Djinbachian, MD, of the Montreal University Hospital Research Center, Montreal, Quebec, Canada, and colleagues.Optical diagnosis of diminutive colorectal polyps has been proposed as a cost-effective alternative to histologic diagnosis, but its implementation in general clinical practice has been hindered by endoscopists’ concerns about incorrect diagnoses, the investigators wrote in Gastroenterology.“AI-based systems (CADx) have been proposed as a solution to these barriers to implementation, with studies showing high adherence to Preservation and Incorporation of Valuable Endoscopic Innovations (PIVI) thresholds when using AI-H,” they wrote. “However, the efficacy and safety of autonomous AI-based diagnostic platforms have not yet been evaluated.”
To address this knowledge gap, Dr. Djinbachian and colleagues conducted a randomized controlled noninferiority trial involving 467 patients, all of whom underwent elective colonoscopies at a single academic institution.
Participants were randomly assigned to one of two groups. The first group received an optical diagnosis of diminutive (1-5 mm) colorectal polyps using an autonomous AI-based CADx system without any human input. The second group had diagnoses performed by endoscopists who used AI-H to make their optical diagnoses.
The primary outcome was the accuracy of optical diagnosis compared with the gold standard of histologic evaluation. Secondarily, the investigators explored associations between pathology-based surveillance intervals and various measures of accuracy, including sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV).
The results showed that the accuracy of optical diagnosis for diminutive polyps was similar between the two groups, supporting noninferiority. Autonomous AI achieved an accuracy rate of 77.2%, while the AI-H group had an accuracy of 72.1%, which was not statistically significant (P = .86).
But when it came to pathology-based surveillance intervals, autonomous AI showed a clear advantage; the autonomous AI system achieved a 91.5% agreement rate, compared with 82.1% for the AI-H group (P = .016).
“These findings indicate that autonomous AI not only matches but also surpasses AI-H in accuracy for determining surveillance intervals,” the investigators wrote, noting that this finding highlights the “complexities of human interaction with AI modules where human intervention could lead to worse outcomes.”
Further analysis revealed that the sensitivity of autonomous AI for identifying adenomas was 84.8%, slightly higher than the 83.6% sensitivity of the AI-H group. Specificity was 64.4% for autonomous AI vs 63.8% for AI-H. While PPV was higher in the autonomous AI group (85.6%), compared with the AI-H group (78.6%), NPV was lower for autonomous AI than AI-H (63.0% vs 71.0%).
Dr. Djinbachian and colleagues suggested that future research should focus on larger, multicenter trials to validate these findings and further explore the integration of autonomous AI systems in clinical practice. They also noted that improving AI algorithms to accurately diagnose sessile serrated lesions could enhance the overall effectiveness of AI-based optical diagnosis.
“The performance of autonomous AI in accurately diagnosing diminutive polyps and determining appropriate surveillance intervals suggests that it could play a crucial role in streamlining colorectal cancer screening processes, reducing the burden on pathologists, and potentially lowering healthcare costs,” the investigators concluded.The study was supported by Fujifilm, which had no role in the study design or data analysis. Dr. von Renteln reported additional research funding from Vantage and Fujifilm.
In the era of computer vision for endoscopy and colonoscopy, current paradigms rely on AI as a co-pilot or second observer, with the physician serving as the final arbiter in procedure-related decision-making. This study by Djinbachian and Haumesser et al brings up the interesting wrinkle of autonomous AI as a potentially superior (or noninferior) option in narrow, task-specific use cases.
In this study, human input from the endoscopist after CADx diagnosis led to lower agreement between the AI-predicted diagnosis and corresponding surveillance intervals; human oversight more often incorrectly changed the resultant diagnosis and led to shorter than recommended surveillance intervals.
This study offers a small but very important update to the growing body of literature on CADx in colonoscopy. So far, prospective validation of CADx compared with the human eye for in-situ diagnosis of polyps has provided mixed results. This study is one of the first to examine the potential role of “automatic” CADx without additional human input and sheds light on the importance of the AI-human hybrid in medical care. How do the ways in which humans interact with the user interface and output of AI lead to changes in outcome? How can we optimize the AI-human interaction in order to provide optimal results?
Jeremy R. Glissen Brown is an assistant professor in the Department of Internal Medicine and Division of Gastroenterology at Duke University Medical Center, Durham, North Carolina. He has served as a consultant for Medtronic and Olympus, and on the advisory board for Odin Vision.
In the era of computer vision for endoscopy and colonoscopy, current paradigms rely on AI as a co-pilot or second observer, with the physician serving as the final arbiter in procedure-related decision-making. This study by Djinbachian and Haumesser et al brings up the interesting wrinkle of autonomous AI as a potentially superior (or noninferior) option in narrow, task-specific use cases.
In this study, human input from the endoscopist after CADx diagnosis led to lower agreement between the AI-predicted diagnosis and corresponding surveillance intervals; human oversight more often incorrectly changed the resultant diagnosis and led to shorter than recommended surveillance intervals.
This study offers a small but very important update to the growing body of literature on CADx in colonoscopy. So far, prospective validation of CADx compared with the human eye for in-situ diagnosis of polyps has provided mixed results. This study is one of the first to examine the potential role of “automatic” CADx without additional human input and sheds light on the importance of the AI-human hybrid in medical care. How do the ways in which humans interact with the user interface and output of AI lead to changes in outcome? How can we optimize the AI-human interaction in order to provide optimal results?
Jeremy R. Glissen Brown is an assistant professor in the Department of Internal Medicine and Division of Gastroenterology at Duke University Medical Center, Durham, North Carolina. He has served as a consultant for Medtronic and Olympus, and on the advisory board for Odin Vision.
In the era of computer vision for endoscopy and colonoscopy, current paradigms rely on AI as a co-pilot or second observer, with the physician serving as the final arbiter in procedure-related decision-making. This study by Djinbachian and Haumesser et al brings up the interesting wrinkle of autonomous AI as a potentially superior (or noninferior) option in narrow, task-specific use cases.
In this study, human input from the endoscopist after CADx diagnosis led to lower agreement between the AI-predicted diagnosis and corresponding surveillance intervals; human oversight more often incorrectly changed the resultant diagnosis and led to shorter than recommended surveillance intervals.
This study offers a small but very important update to the growing body of literature on CADx in colonoscopy. So far, prospective validation of CADx compared with the human eye for in-situ diagnosis of polyps has provided mixed results. This study is one of the first to examine the potential role of “automatic” CADx without additional human input and sheds light on the importance of the AI-human hybrid in medical care. How do the ways in which humans interact with the user interface and output of AI lead to changes in outcome? How can we optimize the AI-human interaction in order to provide optimal results?
Jeremy R. Glissen Brown is an assistant professor in the Department of Internal Medicine and Division of Gastroenterology at Duke University Medical Center, Durham, North Carolina. He has served as a consultant for Medtronic and Olympus, and on the advisory board for Odin Vision.
, while providing greater alignment with pathology-based surveillance intervals, based on a randomized controlled trial.
These findings suggest that autonomous AI may one day replace histologic assessment of diminutive polyps, reported lead author Roupen Djinbachian, MD, of the Montreal University Hospital Research Center, Montreal, Quebec, Canada, and colleagues.Optical diagnosis of diminutive colorectal polyps has been proposed as a cost-effective alternative to histologic diagnosis, but its implementation in general clinical practice has been hindered by endoscopists’ concerns about incorrect diagnoses, the investigators wrote in Gastroenterology.“AI-based systems (CADx) have been proposed as a solution to these barriers to implementation, with studies showing high adherence to Preservation and Incorporation of Valuable Endoscopic Innovations (PIVI) thresholds when using AI-H,” they wrote. “However, the efficacy and safety of autonomous AI-based diagnostic platforms have not yet been evaluated.”
To address this knowledge gap, Dr. Djinbachian and colleagues conducted a randomized controlled noninferiority trial involving 467 patients, all of whom underwent elective colonoscopies at a single academic institution.
Participants were randomly assigned to one of two groups. The first group received an optical diagnosis of diminutive (1-5 mm) colorectal polyps using an autonomous AI-based CADx system without any human input. The second group had diagnoses performed by endoscopists who used AI-H to make their optical diagnoses.
The primary outcome was the accuracy of optical diagnosis compared with the gold standard of histologic evaluation. Secondarily, the investigators explored associations between pathology-based surveillance intervals and various measures of accuracy, including sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV).
The results showed that the accuracy of optical diagnosis for diminutive polyps was similar between the two groups, supporting noninferiority. Autonomous AI achieved an accuracy rate of 77.2%, while the AI-H group had an accuracy of 72.1%, which was not statistically significant (P = .86).
But when it came to pathology-based surveillance intervals, autonomous AI showed a clear advantage; the autonomous AI system achieved a 91.5% agreement rate, compared with 82.1% for the AI-H group (P = .016).
“These findings indicate that autonomous AI not only matches but also surpasses AI-H in accuracy for determining surveillance intervals,” the investigators wrote, noting that this finding highlights the “complexities of human interaction with AI modules where human intervention could lead to worse outcomes.”
Further analysis revealed that the sensitivity of autonomous AI for identifying adenomas was 84.8%, slightly higher than the 83.6% sensitivity of the AI-H group. Specificity was 64.4% for autonomous AI vs 63.8% for AI-H. While PPV was higher in the autonomous AI group (85.6%), compared with the AI-H group (78.6%), NPV was lower for autonomous AI than AI-H (63.0% vs 71.0%).
Dr. Djinbachian and colleagues suggested that future research should focus on larger, multicenter trials to validate these findings and further explore the integration of autonomous AI systems in clinical practice. They also noted that improving AI algorithms to accurately diagnose sessile serrated lesions could enhance the overall effectiveness of AI-based optical diagnosis.
“The performance of autonomous AI in accurately diagnosing diminutive polyps and determining appropriate surveillance intervals suggests that it could play a crucial role in streamlining colorectal cancer screening processes, reducing the burden on pathologists, and potentially lowering healthcare costs,” the investigators concluded.The study was supported by Fujifilm, which had no role in the study design or data analysis. Dr. von Renteln reported additional research funding from Vantage and Fujifilm.
, while providing greater alignment with pathology-based surveillance intervals, based on a randomized controlled trial.
These findings suggest that autonomous AI may one day replace histologic assessment of diminutive polyps, reported lead author Roupen Djinbachian, MD, of the Montreal University Hospital Research Center, Montreal, Quebec, Canada, and colleagues.Optical diagnosis of diminutive colorectal polyps has been proposed as a cost-effective alternative to histologic diagnosis, but its implementation in general clinical practice has been hindered by endoscopists’ concerns about incorrect diagnoses, the investigators wrote in Gastroenterology.“AI-based systems (CADx) have been proposed as a solution to these barriers to implementation, with studies showing high adherence to Preservation and Incorporation of Valuable Endoscopic Innovations (PIVI) thresholds when using AI-H,” they wrote. “However, the efficacy and safety of autonomous AI-based diagnostic platforms have not yet been evaluated.”
To address this knowledge gap, Dr. Djinbachian and colleagues conducted a randomized controlled noninferiority trial involving 467 patients, all of whom underwent elective colonoscopies at a single academic institution.
Participants were randomly assigned to one of two groups. The first group received an optical diagnosis of diminutive (1-5 mm) colorectal polyps using an autonomous AI-based CADx system without any human input. The second group had diagnoses performed by endoscopists who used AI-H to make their optical diagnoses.
The primary outcome was the accuracy of optical diagnosis compared with the gold standard of histologic evaluation. Secondarily, the investigators explored associations between pathology-based surveillance intervals and various measures of accuracy, including sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV).
The results showed that the accuracy of optical diagnosis for diminutive polyps was similar between the two groups, supporting noninferiority. Autonomous AI achieved an accuracy rate of 77.2%, while the AI-H group had an accuracy of 72.1%, which was not statistically significant (P = .86).
But when it came to pathology-based surveillance intervals, autonomous AI showed a clear advantage; the autonomous AI system achieved a 91.5% agreement rate, compared with 82.1% for the AI-H group (P = .016).
“These findings indicate that autonomous AI not only matches but also surpasses AI-H in accuracy for determining surveillance intervals,” the investigators wrote, noting that this finding highlights the “complexities of human interaction with AI modules where human intervention could lead to worse outcomes.”
Further analysis revealed that the sensitivity of autonomous AI for identifying adenomas was 84.8%, slightly higher than the 83.6% sensitivity of the AI-H group. Specificity was 64.4% for autonomous AI vs 63.8% for AI-H. While PPV was higher in the autonomous AI group (85.6%), compared with the AI-H group (78.6%), NPV was lower for autonomous AI than AI-H (63.0% vs 71.0%).
Dr. Djinbachian and colleagues suggested that future research should focus on larger, multicenter trials to validate these findings and further explore the integration of autonomous AI systems in clinical practice. They also noted that improving AI algorithms to accurately diagnose sessile serrated lesions could enhance the overall effectiveness of AI-based optical diagnosis.
“The performance of autonomous AI in accurately diagnosing diminutive polyps and determining appropriate surveillance intervals suggests that it could play a crucial role in streamlining colorectal cancer screening processes, reducing the burden on pathologists, and potentially lowering healthcare costs,” the investigators concluded.The study was supported by Fujifilm, which had no role in the study design or data analysis. Dr. von Renteln reported additional research funding from Vantage and Fujifilm.
FROM GASTROENTEROLOGY