Don't trust apps to spot skin cancer

Three out of four smartphone applications designed to assess melanoma risk misclassified at least 30% of melanomas as benign in a case-control diagnostic accuracy study.

Although these applications are intended for educational purposes rather than diagnostic purposes, they pose a potential risk to consumers, according to medical student Joel A. Wolf and his colleagues at the University of Pittsburgh.

"These applications are not subject to any sort of validation or regulatory oversight. Despite disclaimers that these applications are intended for educational purposes, they have the potential to harm users who may believe mistakenly that the evaluation given by such an application is a substitute for medical advice," the investigators wrote. The study was published online Jan. 16 in JAMA Dermatology.

The investigators used the four smartphone applications to evaluate digital images of pigmented cutaneous lesions, including 60 histologically confirmed melanomas and 128 benign control lesions. The applications deemed 85%-98% of the images evaluable.

Based only on the images considered evaluable, the overall sensitivity of the applications ranged from 7% to 98% and the overall specificity ranged from 30% to 94%. Positive predictive value ranged from 33% to 42% and negative predictive value ranged from 65% to 97%.

The highest sensitivity occurred with the only application that sent images to a board-certified dermatologist for analysis within 24 hours; the other three applications relied on automated algorithms for image analysis (JAMA Dermatol. 2013 Jan. 16 [doi:10.1001/jamadermatol.2013.3282]).

"Even the best of these three applications (that used automated algorithms) classified 18 of 60 melanomas (30%) in our study as benign," the investigators said.

The images used for this study were selected from a database of images captured routinely from patients before skin lesion removal. Only close-up images without identifiable features and without equivocal histology were used. The smartphone applications evaluated as part of the study were selected based on their ability to use existing images and on claims or suggestions of "an ability to assist users in determining whether a skin lesion may be malignant," Mr. Wolf and his associates said.

The applications were not identified because the purpose of the study was to determine the accuracy of such applications in general, rather than to make a statement about any particular application, the investigators noted.

Given the ubiquity of smartphone health care applications that are marketed to consumers (13,000 are available in the largest online application store alone, the investigators said), and given that the Food and Drug Administration has announced plans to regulate certain health-related smartphone applications, the findings underscore the need for physicians to be aware of the types of applications available to "aid us in protecting and educating our patients," the investigators noted.

These types of technologies have the potential to improve mortality from melanoma, but extreme care must be taken to prevent potential harm, they added.

"Despite disclaimers presented by each of these applications that they were designed for educational purposes rather than actual diagnosis and that they should not substitute for standard medical care, releasing a tool to the public requires some thought as to how it could be misused. This potential is of particular concern in times of economic hardship, when uninsured and even insured patients, deterred by the cost of copayments for medical visits, may turn to these applications as alternatives to physician evaluation," Mr. Wolf and his associates said.

This study was supported by grants from the National Institutes of Health. Study author Dr. Laura K. Ferris disclosed having served as an investigator and consultant for MELA Sciences Inc. The remaining authors had no disclosures to report.

Three out of four smartphone applications designed to assess melanoma risk misclassified at least 30% of melanomas as benign in a case-control diagnostic accuracy study.

Although these applications are intended for educational purposes rather than diagnostic purposes, they pose a potential risk to consumers, according to medical student Joel A. Wolf and his colleagues at the University of Pittsburgh.

"These applications are not subject to any sort of validation or regulatory oversight. Despite disclaimers that these applications are intended for educational purposes, they have the potential to harm users who may believe mistakenly that the evaluation given by such an application is a substitute for medical advice," the investigators wrote. The study was published online Jan. 16 in JAMA Dermatology.

The investigators used the four smartphone applications to evaluate digital images of pigmented cutaneous lesions, including 60 histologically confirmed melanomas and 128 benign control lesions. The applications deemed 85%-98% of the images evaluable.

Based only on the images considered evaluable, the overall sensitivity of the applications ranged from 7% to 98% and the overall specificity ranged from 30% to 94%. Positive predictive value ranged from 33% to 42% and negative predictive value ranged from 65% to 97%.

The highest sensitivity occurred with the only application that sent images to a board-certified dermatologist for analysis within 24 hours; the other three applications relied on automated algorithms for image analysis (JAMA Dermatol. 2013 Jan. 16 [doi:10.1001/jamadermatol.2013.3282]).

"Even the best of these three applications (that used automated algorithms) classified 18 of 60 melanomas (30%) in our study as benign," the investigators said.

The images used for this study were selected from a database of images captured routinely from patients before skin lesion removal. Only close-up images without identifiable features and without equivocal histology were used. The smartphone applications evaluated as part of the study were selected based on their ability to use existing images and on claims or suggestions of "an ability to assist users in determining whether a skin lesion may be malignant," Mr. Wolf and his associates said.

The applications were not identified because the purpose of the study was to determine the accuracy of such applications in general, rather than to make a statement about any particular application, the investigators noted.

Given the ubiquity of smartphone health care applications that are marketed to consumers (13,000 are available in the largest online application store alone, the investigators said), and given that the Food and Drug Administration has announced plans to regulate certain health-related smartphone applications, the findings underscore the need for physicians to be aware of the types of applications available to "aid us in protecting and educating our patients," the investigators noted.

These types of technologies have the potential to improve mortality from melanoma, but extreme care must be taken to prevent potential harm, they added.

"Despite disclaimers presented by each of these applications that they were designed for educational purposes rather than actual diagnosis and that they should not substitute for standard medical care, releasing a tool to the public requires some thought as to how it could be misused. This potential is of particular concern in times of economic hardship, when uninsured and even insured patients, deterred by the cost of copayments for medical visits, may turn to these applications as alternatives to physician evaluation," Mr. Wolf and his associates said.

This study was supported by grants from the National Institutes of Health. Study author Dr. Laura K. Ferris disclosed having served as an investigator and consultant for MELA Sciences Inc. The remaining authors had no disclosures to report.

Three out of four smartphone applications designed to assess melanoma risk misclassified at least 30% of melanomas as benign in a case-control diagnostic accuracy study.

Although these applications are intended for educational purposes rather than diagnostic purposes, they pose a potential risk to consumers, according to medical student Joel A. Wolf and his colleagues at the University of Pittsburgh.

"These applications are not subject to any sort of validation or regulatory oversight. Despite disclaimers that these applications are intended for educational purposes, they have the potential to harm users who may believe mistakenly that the evaluation given by such an application is a substitute for medical advice," the investigators wrote. The study was published online Jan. 16 in JAMA Dermatology.

The investigators used the four smartphone applications to evaluate digital images of pigmented cutaneous lesions, including 60 histologically confirmed melanomas and 128 benign control lesions. The applications deemed 85%-98% of the images evaluable.

Based only on the images considered evaluable, the overall sensitivity of the applications ranged from 7% to 98% and the overall specificity ranged from 30% to 94%. Positive predictive value ranged from 33% to 42% and negative predictive value ranged from 65% to 97%.

The highest sensitivity occurred with the only application that sent images to a board-certified dermatologist for analysis within 24 hours; the other three applications relied on automated algorithms for image analysis (JAMA Dermatol. 2013 Jan. 16 [doi:10.1001/jamadermatol.2013.3282]).

"Even the best of these three applications (that used automated algorithms) classified 18 of 60 melanomas (30%) in our study as benign," the investigators said.

The images used for this study were selected from a database of images captured routinely from patients before skin lesion removal. Only close-up images without identifiable features and without equivocal histology were used. The smartphone applications evaluated as part of the study were selected based on their ability to use existing images and on claims or suggestions of "an ability to assist users in determining whether a skin lesion may be malignant," Mr. Wolf and his associates said.

The applications were not identified because the purpose of the study was to determine the accuracy of such applications in general, rather than to make a statement about any particular application, the investigators noted.

Given the ubiquity of smartphone health care applications that are marketed to consumers (13,000 are available in the largest online application store alone, the investigators said), and given that the Food and Drug Administration has announced plans to regulate certain health-related smartphone applications, the findings underscore the need for physicians to be aware of the types of applications available to "aid us in protecting and educating our patients," the investigators noted.

These types of technologies have the potential to improve mortality from melanoma, but extreme care must be taken to prevent potential harm, they added.

"Despite disclaimers presented by each of these applications that they were designed for educational purposes rather than actual diagnosis and that they should not substitute for standard medical care, releasing a tool to the public requires some thought as to how it could be misused. This potential is of particular concern in times of economic hardship, when uninsured and even insured patients, deterred by the cost of copayments for medical visits, may turn to these applications as alternatives to physician evaluation," Mr. Wolf and his associates said.

This study was supported by grants from the National Institutes of Health. Study author Dr. Laura K. Ferris disclosed having served as an investigator and consultant for MELA Sciences Inc. The remaining authors had no disclosures to report.