A new artificial intelligence system ItpCtrl-AI promises to greatly improve chest X-ray diagnostics by offering both interpretability and controllability – addressing the long-standing challenge of AI transparency in medical imaging. Developed by researchers at the University of Arkansas in collaboration with MD Anderson Cancer Center, ItpCtrl-AI models radiologists’ gaze patterns to ensure its decision-making process aligns with human expertise.
AI-driven diagnostic tools have demonstrated remarkable accuracy in detecting medical abnormalities, such as fluid accumulation in the lungs, enlarged hearts, and early signs of cancer. However, many of these AI models function as “black boxes,” making it difficult for medical professionals to understand how conclusions are reached.
According to Ngan Le, assistant professor of computer science and computer engineering at the University of Arkansas, transparency is critical for the adoption of AI in medicine. “When people understand the reasoning process and limitations behind AI decisions, they are more likely to trust and embrace the technology,” Le said.
ItpCtrl-AI, short for interpretable and controllable artificial intelligence, was designed to bridge this gap by replicating how radiologists analyze chest X-rays. Unlike conventional AI systems that simply predict diagnoses, ItpCtrl-AI generates gaze heatmaps – visual representations of the areas radiologists focus on during their examination. These heatmaps provide a transparent view into the AI’s decision-making process, enhancing both trust and interpretability.
To develop this AI model, researchers tracked the eye movements of radiologists as they reviewed chest X-ray images. They recorded not only where experts looked but also how long they focused on specific areas before reaching a diagnosis. The collected data was then used to train ItpCtrl-AI, enabling it to generate attention heatmaps that highlight key diagnostic regions within an image.
By leveraging these gaze-based insights, the AI system filters out irrelevant areas before making a diagnostic prediction, ensuring that it only considers meaningful information – just as a human radiologist would. This attention-based decision-making approach makes ItpCtrl-AI significantly more interpretable than traditional AI models.
To support the development of ItpCtrl-AI, researchers created DiagnosedGaze++, a first-of-its-kind dataset that aligns medical findings with radiologists’ eye gaze data. Unlike existing datasets, DiagnosedGaze++ provides detailed anatomical attention maps, setting a new standard for AI-driven diagnostic transparency.
Using a semi-automated approach, the research team filtered and structured radiologists’ eye-tracking data, ensuring that each heatmap accurately corresponded to medical abnormalities. This dataset not only improves AI interpretability but also paves the way for future advancements in medical imaging AI.
ItpCtrl-AI is not the only AI-driven system advancing medical imaging transparency. At QuData, we also employ Grad-CAM (Gradient-weighted Class Activation Mapping) to generate heatmaps for mammogram analysis.
At its core, Grad-CAM highlights the most influential regions of an image that contribute to the AI model’s decision, allowing radiologists to pinpoint areas of interest with greater precision. This technique ensures that AI-assisted breast cancer detection remains explainable and aligned with medical expertise. By integrating heatmap-based visual explanations, both ItpCtrl-AI and QuData’s AI-powered solutions enhance trust and usability in clinical settings.
Transparency in AI-assisted diagnosis is not just a technical advancement – it is an ethical necessity. The ability to explain AI decisions is crucial for ensuring fairness, mitigating bias, and maintaining accountability in healthcare. With legal and ethical concerns surrounding AI in medicine, ItpCtrl-AI offers a model that allows doctors to take responsibility for AI-assisted diagnosis.
The research team is now working to enhance ItpCtrl-AI to analyze three-dimensional CT scans, which require even more complex decision-making processes. By incorporating depth information and broader anatomical structures, the AI system could further improve diagnostic precision in critical medical applications.
To encourage further research and adoption, the project’s source code, models, and annotated dataset will be made publicly available. This initiative aims to set a new benchmark for AI-driven transparency and accountability in medical imaging.