The Vanguard of Human Interaction

The case for why AI feels human, isn't human, and will have significant consequences anyway.

Wien IX, Berggasse 19. Sigmund Freud’s residence after fleeing the Nazis in 1938. Image is from the Sigmund Freud Museum, in Vienna, Austria.

Human interaction is the fundamental unit of psychiatry.

Psychiatry is unique in medicine in that the guidelines involve purely clinical criteria for diagnosis (1). Whether a strength or a weakness, this endows the Psychiatric Interview as the bread and butter of a psychiatrist more than any other tool. What is the patient’s life story? What are the triggering events? How do these factors relate to the delusory, manic, or depressive episode that brings them to the hospital today? A psychiatrist’s skill lies as much in building connection as their diagnostic acumen and pharmacological knowledge.

This feels uniquely human, because it is the most like an art form; the multidimensionality and complexity of the human psyche is not reducible to clean causative chains like the feedback loops of the endocrine system, or the biological fluid mechanics of the heart. Every field of medicine has its own dogmas and unexplored edges, but psychiatry has always lived closer to the edge. Rather than a single foundational framework, it draws from a pluralism of models: biological, psychological, social, cultural. There is no simple ground truth.

In this sense, it feels inevitable that psychiatry is furthest from being automatable. Radiology and Pathology are on the other end of this science-art spectrum, and lie squarely in the “medicine is a data and information-processing discipline” camp. Psychiatrists can lie back in their Freudian armchairs, and chuckle when over 70% of the 1,000 AI FDA application approvals are for Radiology applications.

“Not so fast!” says the AI companies.

---

AI Feels Human

In 2022, the grand public experiment of Large Language Models (LLMs) was launched through ChatGPT. In 2023, therapists began noticing patients quoting it in sessions/consults. In 2024, we saw the first clinical studies which demonstrated that LLMs could not only outperform clinicians diagnostically, but also on scales of empathy (2). Now, in 2025, we see signs that LLMs perform better than humans WITH LLMs on diagnosis, communication, and even economics (3). The companies want to make the case that it is absurd to not replace parts of clinician workflows with augmented LLM tools.

Claims like this may spur psychiatrists to cautiously sit up from their armchairs, and counter with: “But these are simply biased industry-funded studies!”, or “These patient interactions are all simulated!”, which are all valid critiques. But these articles spell two salient points: these models can diagnose difficult conditions provided the same information, and more effectively interact with patients than clinicians over text.

This second point, human interaction, is crucial to consider in the context of psychiatry. One popular psychiatric handbook considers there to be four goals in a psychiatric interview: 1) to build a therapeutic alliance, 2) collect relevant information, 3) interview for a diagnosis, and 4) negotiate the treatment (4). According to the AI company articles - assessing benchmarks of empathy, information gathering, diagnosis, and management recommendations - LLMs fit all four of these criteria.

Let’s continue along this line of argument. Sure, we can grant LLMs better data processing and cognitive abilities - they run on at least 450,000 times more energy than our human brains, and are trained on trillions of words compared to our millions - of course they should be effective knowledge machines. But human interaction? Building trust and rapport? Surely this is a quintessential part of what it means to be human!

Fragility of Human Psychology

Perhaps we are more fallible than we think.

This is not the first time human interaction has been attacked by automation. Every article discussing AI therapists has written about ELIZA, one of the first chatbots developed by MIT in the 1960s. It takes on the role of a Rogerian Psychotherapist, turning every question you ask back at you (example). Unexpectedly, users were convinced of its intelligence despite its simplicity in programming - so much so the phenomenon is now termed the ELIZA effect (5).

LLMs today represent a more powerful ELIZA effect than ever before. In raw numbers, OpenAI is often quoted as the fastest growing company in history, reaching 100M users within 2 months. Although LLMs are largely tools for knowledge work and coding, through time this ‘tool’ has tapped deeper into our inner psyches. There are 20 million teenagers roleplaying with LLMs, more than 10,000 adults creating digital resurrections of their deceased grandparents, and 50,000 users testing AI designed for therapy.

These are not examples of technology as a tool; these are examples of technology as a companion.

What makes these LLMs so convincing? There are plenty of perspectives, but I’ll add on three thoughts from a technological lens -

1) Pre-Training: LLMs are trained to be expert ‘human’ imitators. There are plenty of analogies to draw on to think of how these models are trained - an algorithm built to predict the next word (stochastic parrots), or a bag of words where you put in words and it spits out the most relevant ones. Regardless of where you view AI as just a statistical model, the point is that they are trained on trillions of words written on the internet by humans. If you’ve read every single scientific journal article (or every piece of harry potter fanfiction), you can bet you’re able to mimic something pretty convincing.

2) Post-Training: LLMs are reinforced to appeal to humans. What happens after a model is trained on the internet? It is then ‘fine-tuned’ on high-quality datasets which can give LLMs their personality. There’s also a popular term known as Reinforcement Learning with Human Feedback (RLHF), which refers to the positive or negative “thumbs up or down” that you can give the chatbot. These signals enable a direct human feedback loop to cater more towards what humans want - which sometimes leads to a chatbot that is a bit too nice.

3) Engineering: LLMs are built to be personalised for humans. Finally, the engineering on top of LLMs can enable them to be more personalised for every user. In the same way your doctor can remember your niece’s name, so can your LLM - these tools often have features which capture salient parts of your conversation, and store them in memory for use in prompts. In the same way social media has optimised our feeds for our interests, so can LLMs.

The combination of these three features manifests as the same humanistic Rogerian psychological school of thought as ELIZA, but with more degrees of freedom and knowledge. Fantastic as a tool, but dangerous as a companion - particularly considering they seem to exhibit unconstrained, unconditional positive regard. Yes-men are pretty good at tapping into what makes us feel good about ourselves, less so for any real conversations.

In the same way technology today has been built to optimise for your attention, LLMs today are built to ask for more - your mind.

---

AI Isn’t Human

Despite this, I wrote feels, not is. LLMs are far more multidimensional algorithms and anthropomorphised than the chatbots of the past - but this does not equate them as having mastered human interaction.

AI can’t do the job of a psychiatrist.

Let’s return to that psychiatric interview with a less techno-optimistic lens. The success of LLMs in this context has two key assumptions: that AIs will receive perfect information from patients, and that every patient responds to the same outputs.

Both of these break down in psychiatry, since LLMs lack bodies and minds.

Body: Human interaction is multidimensional

LLMs currently do not have full access to the breadth of richness that comes with the full human interaction. Dissecting this, we are left with its constituent data types - messaging on text, calling on audio, seeing people on video, and the other senses that come in-person. These units differ in one fundamental aspect: Information.

Text alone is insufficient to give the additional contextual information psychiatry needs to come to its formulations. When someone is depressed, can you assess their severity through purely their text-based utterances? When someone is manic, can you feel their pressured speech through the screen?

Humans are not only equipped with our brains, but also wired further through our endocrine, enteric, and autonomic systems to decode these ‘simple’ ideas of empathy, trust, and lying. There’s a reason there’s a cultural belief that the “gut feeling is correct”, and it lives beyond rationalisms and cognition.

LLMs are limited because of both the information they have available to them and their current feedback systems. They can be perfect diagnostic reasoners; but they can’t elicit the perfect information needed in difficult psychiatric contexts. The hidden cues in our sensory systems give us far more powerful feedback about how to ask diagnostic questions, how to build trust, how to negotiate the management. LLMs just get a thumbs up or down.

Minds: Humans are wired for humans

Humans do not live in a vacuum, but within a culture of other humans and shared beliefs. There are three aspects to this which are necessary in the therapeutic relationship.

First, there’s a reason that psychology and psychiatry have no ‘one-size-fits-all’ cures. Unconditional positive regard will undoubtedly help some, but not all. In the same way people struggle to find a good therapist-fit, people will struggle to be supported by a single AI with a monolithic philosophy.

The danger comes most in a psychiatric context. Humans have mechanisms and cultural beliefs to know when to not be purely sycophantic and disagree. LLMs don’t, and we see this in emerging case studies of AI-induced psychosis, and with mental-health specific risk factors like social isolation and impaired belief-updating mechanisms (i.e. jumping to extreme conclusions based on limited data). How can we expect LLMs to diagnose, let alone support acutely delusional psychiatric patients if they can’t help but agree with them?

Second is liability. If an LLM tells you to take an antipsychotic, they are not liable for any of the side effects incurred. They have no stakes. If a psychiatrist does, this human is responsible for what happens, and unless they are psychopaths, we can better trust humans to uphold this responsibility. We do not perceive machines to have these same feelings - we only grade them on their ability to deliver reliable outcomes based on risk.

Third is transference - the Freudian psychological concept that each human’s past experiences are subconsciously placed onto the present interaction. The crux of this idea is that it is not simply the content of the information, but who you receive the information from that matters. An individual’s response to AI-based interviewing or therapy will vary based on their individual life experience. There will simply be some who refuse to talk to AI chatbots.

This extends more deeply to the cultural zeitgeist; this will shift over time, as capabilities and views on AI change.

---

AI is an impactful technology

So, human interaction is not solved by current AI, and the psychiatrist remains in their armchair. We’ve explored limitations in the data and systems, inherent psychology, and the human perspective.

But even despite these limitations, I’m convinced that AI will change the face of human interaction and psychiatry.

If text-based LLMs are an example to extrapolate from, we can expect multimodal AIs that harness audio and visual forms to have an equally and more powerful ability to forge human connections. If an AI has every piece of data that a human has, what next?

LLMs are already useful tools that have potential to change the clinical paradigm - psychiatrist supply is finite and software is infinitely scalable and constantly available. LLMs will inevitably be used to supplement lower acuity aspects of management (think medication adjustment, spot diagnoses).

The social problem comes when they extend beyond the world of knowledge, but into human interaction. Without proper safeguards, the sycophantic properties of LLMs will run unhinged (6).

The great new experiment of AI will unfold similarly to social media (7); a powerful, easily-accessible technology that shapes and changes human interaction and creates worlds of unintended consequences as a result. Mental health is a cultural phenomenon, after all. In the same way psychiatric terminology has bent to model modern paradigms of internet addiction, a new language for AI-related conditions will emerge.

Psychiatry is not immune to the changing rhythm of the technological world - but while other medical fields have more positive revolutions to look forward to, Psychiatry, as the vanguard for human interaction, may be tasked with a bigger challenge than ever before.

---

(1) Psychiatric practice (as practiced by Australian Psychiatric Registrars) relies on a diagnostic pyramid of classification which rules out biological causes of psychiatric symptoms first. So although this is a simplification, we expect most psychiatric conditions to be non-biological (at least, in the form of easily diagnosable blood-based biomarkers). Neurologists also tend to like to take the case when it becomes more neuro-biological.

(2) The AMIE (Google Research, 2024) paper ran a double-blind crossover study testing 20 clinicians against their AMIE model with patient actors in a synchronous text chat interface. The AMIE model was judged by the actors to be significantly better than clinicians across domains of diagnostic accuracy, management, and empathy based on standardised questionnaires.

(3) Healthbench (OpenAI, 2025) tests 5000 conversations across 262 clinicians and models and grades their performance. They find their Apr 2025 model having equivalent performance to physicians WITH models across domains of communication quality and accuracy, amongst others. The MAI-DxO (Microsoft Research, 2025) model is considered 4x more accurate than 21 physicians on NEJM case diagnoses, and more constrained in resource usage.

(4) I took these points from the aptly titled The Psychiatric Interview (2017), a well regarded guide for wannabe trainees written by psychiatrist Daniel Carlat.

(5) Fascinatingly, I also came across PARRY, created by the psychiatrist Keith Colby at Stanford, which aims to simulate a person with schizophrenia, and even then 30 psychiatrists were unable to discern the chatbot from patients.

(6) Technological folie à deux (2025) provides a great primer on the emerging ideas within the psychiatry-AI field - one example being the concept of bidirectional belief amplification, where with a maladaptive user, AI psychologies are currently built to reinforce negative beliefs of the user, spurring potentially antisocial behaviour.

(7) The Social Dilemma (2020) provides a more balanced view on the harms of social media, and The AI Dilemma (2023) gives a perspective on the more immediate pragmatic issues ahead.

Much thanks to Chris Chiu, Shayan Lahijanian, Joshua Han, Alice Park, Marc Jurblum, and Louis Ereve for reading drafts.