Traditional Diagnostic Systems Excel Over Generative AI in Disease Diagnosis

Fri 30th May, 2025

Research Findings Highlight the Continued Relevance of Traditional Diagnostic Tools

Recent investigations reveal that traditional diagnostic decision support systems (DDSS) continue to outperform generative artificial intelligence (AI) in the realm of disease diagnosis. The study, conducted by computer scientists at Massachusetts General Hospital (MGH), focused on comparing the efficacy of MGH's long-established DDSS, known as DXplain, against modern generative AI models such as ChatGPT and Gemini.

DXplain, developed in 1984, utilizes an extensive database of disease profiles and clinical findings to assist healthcare professionals in generating and prioritizing potential diagnoses. In light of the rise of generative AI tools in the medical field, researchers aimed to evaluate how these newer technologies stack up against established systems like DXplain.

The research team assessed the diagnostic performance of DXplain, ChatGPT, and Gemini across 36 varied patient cases, which encompassed different racial, ethnic, age, and gender demographics. Each diagnostic system was tasked with providing potential diagnoses for these cases, both with and without accompanying laboratory data.

Results indicated that when lab data was available, DXplain achieved a correct diagnosis rate of 72%, while ChatGPT reached 64% and Gemini 58%. Notably, in scenarios lacking lab data, DXplain still led the pack with a 56% accuracy rate, outperforming ChatGPT at 42% and Gemini at 39%. While these performance differences were not statistically significant, they underscore the reliability of traditional systems in diagnostic contexts.

The study's co-author emphasized the importance of recognizing the capabilities of traditional expert systems like DXplain, which can enhance clinical decision-making by recalling critical information that might be overlooked in high-pressure situations. Furthermore, the researchers suggest that the integration of generative AI with established diagnostic systems could yield even better results. By utilizing the linguistic strengths of large language models in conjunction with the data-driven approach of systems like DXplain, the potential for improved diagnostic accuracy and patient outcomes is significant.

In summary, the findings highlight the strengths of traditional DDSS in delivering reliable diagnoses and suggest a promising future where the strengths of both traditional and generative AI systems can be combined for optimal clinical decision support.


More Quick Read Articles »