Study: AI chatbots drop medical advice if the patient downplays their symptoms

Artificial intelligence chatbots may initially identify a medical problem correctly, but can change their recommendation if the user insists that their symptoms are not serious. This is the conclusion of a study conducted by researchers in the United Kingdom, who analysed simulated medical conversations.

The findings were presented at the European Respiratory Society Congress – ERS 2026, held in Barcelona.

How the study was conducted

The researchers created seven fictitious patient profiles, all presenting symptoms and risk factors that warranted medical assessment and, where appropriate, a sleep study. The clinical scenarios were based on obstructive sleep apnoea syndrome (OSAS).

Five freely available chatbots were tested: ChatGPT, Google Gemini, Claude, DeepSeek, and Grok, across a total of 700 conversations.

Each of the seven clinical scenarios was presented in two versions, containing the same medical information but with different patient attitudes: in one version, the patient accepted the recommendation to see a doctor; in the other, they downplayed their symptoms and declined further investigation.

Results: recommendations change when the patient pushes back

In all 350 conversations where the fictitious patient accepted the recommendation, the chatbots maintained the correct referral for specialist assessment.

When the patient challenged the need for investigation, the correct recommendation was maintained in only 225 out of 350 conversations — that is, 64%. In other words, in more than a third of cases, the chatbot changed its recommendation, even though the medical information had remained identical.

The problem was more pronounced in high-risk scenarios. In one case suggestive of severe obstructive sleep apnoea, the chatbots maintained the recommendation for assessment in only 22% of conversations where the patient declined investigation.

In another scenario, the patient reported having already fallen asleep at the wheel — a significant indicator of excessive daytime sleepiness and an associated safety risk. Even in this case, when the patient downplayed the problem, the chatbots recommended appropriate assessment in only 32% of conversations. Some inadequate responses omitted the driving risk entirely.

From medical recommendation to lifestyle advice

The researchers observed that, depending on the chatbot, in approximately a quarter to half of conversations involving patients who downplayed their symptoms, the recommendation for medical assessment disappeared entirely, being replaced by general lifestyle advice.

The models did not make errors from the outset — in the initial stage of the conversation, they correctly identified the situation and recommended medical assessment. The vulnerability emerged later, when the user challenged the recommendation and the model adjusted its response to align with the patient's position, despite the clinical data remaining unchanged.

This behaviour is referred to in artificial intelligence research as sycophancy — the tendency of a model to conform excessively to the user's opinions, even when these contradict the available information.

Why sleep apnoea matters

Obstructive sleep apnoea causes repeated interruptions to breathing during sleep and can lead to excessive daytime sleepiness. Left undiagnosed and untreated, the condition is associated with an increased risk of high blood pressure, stroke, cardiovascular disease, and type 2 diabetes.

According to the researchers, approximately 80–90% of moderate or severe cases remain undiagnosed. Diagnosis requires a medical assessment and, where indicated, a sleep study that monitors breathing and other parameters throughout the night.

Study limitations

The researchers note that the study did not involve real patients, but rather fictitious profiles and controlled conversations specifically designed to test how chatbots respond to users who downplay their symptoms. The results do not demonstrate that, in practice, the use of chatbots leads to delayed diagnosis or treatment.

Nevertheless, the authors believe the findings point to a potential safety concern: a chatbot may correctly identify a problem at the outset, but arrive at an inappropriate response just a few exchanges later, after the user contradicts it. For this reason, the researchers argue, the safety evaluation of medical chatbots should not be limited to straightforward questions and isolated responses.