🔗 Share this article

On the 14th of October, 2025, the head of OpenAI issued a remarkable statement.

“We made ChatGPT rather limited,” the statement said, “to guarantee we were acting responsibly regarding mental health concerns.”

Being a doctor specializing in psychiatry who researches recently appearing psychotic disorders in adolescents and emerging adults, this was an unexpected revelation.

Scientists have found sixteen instances recently of users showing psychotic symptoms – experiencing a break from reality – while using ChatGPT use. Our unit has since recorded an additional four examples. Alongside these is the publicly known case of a adolescent who took his own life after talking about his intentions with ChatGPT – which supported them. Assuming this reflects Sam Altman’s notion of “exercising caution with mental health issues,” it falls short.

The strategy, based on his announcement, is to be less careful in the near future. “We understand,” he adds, that ChatGPT’s controls “made it less useful/engaging to many users who had no psychological issues, but given the gravity of the issue we aimed to get this right. Given that we have managed to mitigate the significant mental health issues and have new tools, we are planning to responsibly relax the limitations in many situations.”

“Mental health problems,” should we take this perspective, are independent of ChatGPT. They are associated with individuals, who either have them or don’t. Thankfully, these issues have now been “mitigated,” even if we are not informed the means (by “updated instruments” Altman presumably indicates the imperfect and simple to evade guardian restrictions that OpenAI has lately rolled out).

But the “emotional health issues” Altman wants to place outside have strong foundations in the design of ChatGPT and other advanced AI conversational agents. These systems wrap an basic data-driven engine in an interface that mimics a dialogue, and in this process indirectly prompt the user into the belief that they’re interacting with a being that has agency. This illusion is compelling even if intellectually we might know the truth. Assigning intent is what humans are wired to do. We curse at our car or laptop. We ponder what our animal companion is thinking. We perceive our own traits in various contexts.

The popularity of these products – over a third of American adults stated they used a virtual assistant in 2024, with 28% reporting ChatGPT specifically – is, primarily, based on the influence of this perception. Chatbots are always-available assistants that can, as per OpenAI’s official site tells us, “think creatively,” “consider possibilities” and “work together” with us. They can be assigned “characteristics”. They can call us by name. They have approachable identities of their own (the original of these tools, ChatGPT, is, maybe to the concern of OpenAI’s advertising team, stuck with the title it had when it went viral, but its biggest rivals are “Claude”, “Gemini” and “Copilot”).

The false impression by itself is not the primary issue. Those analyzing ChatGPT commonly reference its historical predecessor, the Eliza “therapist” chatbot created in 1967 that generated a comparable perception. By modern standards Eliza was basic: it produced replies via straightforward methods, typically rephrasing input as a question or making vague statements. Memorably, Eliza’s developer, the computer scientist Joseph Weizenbaum, was taken aback – and concerned – by how many users seemed to feel Eliza, to some extent, grasped their emotions. But what current chatbots produce is more insidious than the “Eliza phenomenon”. Eliza only reflected, but ChatGPT magnifies.

The large language models at the core of ChatGPT and other modern chatbots can realistically create natural language only because they have been supplied with almost inconceivably large amounts of raw text: publications, social media posts, audio conversions; the more comprehensive the superior. Definitely this educational input includes truths. But it also necessarily involves fiction, partial truths and false beliefs. When a user sends ChatGPT a message, the underlying model reviews it as part of a “background” that includes the user’s past dialogues and its own responses, merging it with what’s encoded in its training data to generate a statistically “likely” answer. This is intensification, not mirroring. If the user is wrong in any respect, the model has no way of comprehending that. It repeats the inaccurate belief, possibly even more convincingly or eloquently. Perhaps provides further specifics. This can lead someone into delusion.

Who is vulnerable here? The more relevant inquiry is, who is immune? Every person, without considering whether we “have” existing “emotional disorders”, may and frequently develop incorrect beliefs of our own identities or the world. The ongoing interaction of discussions with others is what helps us stay grounded to common perception. ChatGPT is not a person. It is not a companion. A dialogue with it is not a conversation at all, but a feedback loop in which much of what we communicate is cheerfully reinforced.

OpenAI has acknowledged this in the similar fashion Altman has admitted “mental health problems”: by externalizing it, categorizing it, and declaring it solved. In spring, the organization explained that it was “tackling” ChatGPT’s “overly supportive behavior”. But cases of loss of reality have continued, and Altman has been retreating from this position. In the summer month of August he asserted that many users appreciated ChatGPT’s answers because they had “lacked anyone in their life offer them encouragement”. In his recent announcement, he noted that OpenAI would “release a fresh iteration of ChatGPT … in case you prefer your ChatGPT to answer in a extremely natural fashion, or use a ton of emoji, or simulate a pal, ChatGPT will perform accordingly”. The {company