Pitt researchers develop AI method to flag uncertain cancer diagnoses

Researchers at the University of Pittsburgh have developed a programming technique that allows artificial intelligence models to estimate the reliability of their own predictions in medical imaging. The method, known as uncertainty-weighted learning, is designed to improve the accuracy of AI systems used for clinical tasks, specifically the diagnosis of breast cancer from mammography images.

The study, led by Shandong Wu, a professor of radiology, involved training AI models to recognize when they are likely to return incorrect results. Wu and his team, which includes Jiren Li and Dooman Arefan, used a public dataset called CBIS-DDSM to test these models. The dataset contains mammography images that allow researchers to evaluate how well an AI can distinguish between benign and malignant tissue.

Traditional AI models in medicine often provide a prediction without indicating how confident the system is in that result. Wu explained that this lack of transparency is a major concern in clinical settings where decisions have high stakes. By implementing uncertainty quantification, the models can now generate an estimate of how uncertain they are about each individual prediction, providing a layer of safety for medical professionals.

The research team followed a three-step process to refine the models. First, they incorporated uncertainty quantification into a standard image classification AI model. This allowed the system to assign an uncertainty score to every diagnosis it made on the mammography data. Next, the researchers developed an evaluation method to check whether these uncertainty scores actually matched the risk of the model making an error.

In the final step, the team applied the uncertainty-weighted learning approach to the training process. This technique guides the AI to focus more heavily on data samples where the model is uncertain, rather than treating all images equally. Wu noted that this approach improves the alignment between the estimated uncertainty and the actual prediction errors observed in the dataset.

The results showed that the uncertainty-weighted learning method produced more reliable uncertainty estimates compared to baseline models trained without the technique. At the individual patient level, the method enabled more accurate estimates of the likelihood that a specific AI prediction was incorrect. Wu stated that the study demonstrated that AI models should be evaluated not just on accuracy, but also on their ability to identify potentially incorrect predictions.

Wu highlighted that the approach allows uncertainty to be incorporated directly into the model training process. This shifts the role of uncertainty from a post-hoc estimate to a fundamental part of how the model learns. The research suggests that this method can serve as a benchmark for the field, providing a standard way to compare the reliability of different AI models in clinical applications.

Na Du, an assistant professor in the department of informatics and networked systems, commented on the broader trend of AI research at Pitt. Du leads the Human-Autonomy Teaming Lab and noted that AI is becoming a central part of interdisciplinary research, connecting computing with fields like psychology, engineering, and biomedical informatics. Du emphasized that the value of AI depends on how effectively people can work with these systems.

Du pointed out that while AI development is expanding rapidly, it is not inherently negative. The key is whether the technology addresses meaningful problems and demonstrates benefits through careful research. Du stressed the need for guardrails regarding privacy, fairness, and the risk of over-reliance on AI recommendations. The goal, according to Du, is to establish appropriate trust where users understand when a system is useful and when they need to intervene.