AI doctor failure rate can be 80 percent high in differential diagnosis, Harvard study shows

AI chatbots' differential diagnostic failure rate reaches 80 percent due to incomplete data, including in ChatGPT, DeepSeek, Gemini, and Claude, according to Harvard Medical School research.

A study evaluated over 20 state-of-the-art AI large language models and found that errors occurred in differential diagnoses based solely on the patient’s initial symptoms and signs.

The research further showed that when patients provided additional test results, the final diagnosis failure rate fell to around 40 percent. 

Researchers noted that AI chatbots require sufficiently detailed patient information to deliver more accurate diagnoses; therefore, when patients cannot provide thorough health data, the results generated by AI are deemed to be unreliable.

They also emphasized that using AI to make diagnostic decisions without medical professionals’ intervention is premature.