Smartphone applications intended help people evaluate the potential risk of skin lesions can fail to alert users reliably enough to catch deadly melanomas in time for treatment, a new study finds.

University of Pittsburgh Medical Center researchers putting four smartphone dermatology apps to the test found that three of them incorrectly classified 30 percent or more of melanomas as “not concerning.”

“Although such applications have the potential to improve patient awareness and physician-patient communication,” say the authors in JAMA Dermatology, “applications that provide any type of medical advice might result in harm to the patient if that advice is incorrect or misleading.”

The authors worry that without established regulatory oversight of mobile medical apps, economically disadvantaged and uninsured patients, who “may believe mistakenly that the evaluation given by such an application is a substitute for medical advice,” will fail to seek appropriate care.

The researchers measured the sensitivity, specificity, and positive and negative predictive values of four smartphone applications designed to aid non-clinician users in determining whether their skin lesion is benign or malignant. They provided each of the apps with 188 pictures of skin lesions determined to have good image quality and a clear diagnosis already rendered by a board-certified dermatopathologist.

Of the four applications tested, three used algorithmic evaluation to analyze the images and report their results. The fourth and most accurate app, the one that missed just 1 of the 53 melanomas it was provided for evaluation, essentially functioned as a tool for store-and-forward teledermatology tool, with each smartphone photo being evaluated by a board-certified dermatologist for evaluation.

Medical imaging assessments have proven to be a tricky area, and not just in dermatology. Apps used to interpret radiological images on mobile devices garnered special attention when the U.S. Food and Drug Administration released its Draft Guidance on Mobile Medical Apps in July 2011. The agency said then that such apps would be subject to review because readings could be hampered by the devices’ smaller screen sizes, lower contrast ratios, and uncontrolled ambient lighting during use. Furthermore, even under ideal conditions, image interpretation can vary between clinicians.

What remains to be seen is if the proliferation of unaided and automated diagnosis apps can overcome current limitations and, by doing so, prove to be a net positive rather than a risk.