Comfort Without a Heart: Why Even Psychologists Fall for AI 🤖💔
Are machines becoming better listeners? A 2025 study investigated in a scientific blind test how well Artificial Intelligence can give therapeutic advice compared to real experts. The AI used was based on the GPT-4 language model. The result is absolutely mind-blowing!
Key Facts from the Experiment:
AI Wins on Emotions: The AI's responses were rated significantly more empathetic than those of real therapists, especially regarding emotional and motivational empathy.
Equal Quality: In terms of scientific quality and cognitive empathy, the AI and the experts were completely on par.
Flying Blind: The participating psychologists could not reliably distinguish whether a text was written by the AI or a human colleague.
The Illusion (Bias): When test subjects believed a text came from an expert, they preferred it over 93 percent of the time—regardless of whether it was actually written by a human.
Important Limitations: Despite the excitement, science puts the results into perspective:
Not Real Therapy: The AI merely answered one-off advice column letters. This cannot replace real psychotherapy, as it lacks the foundation of a genuine therapeutic relationship and human trust.
Risk of Loneliness: Those who get too used to an always-agreeing AI might struggle with human conflicts in real life. A longitudinal study suggests that excessive use of AI for social companionship can actually make people lonelier in the long run.
Data Privacy: Currently, highly intimate problems are often processed on the servers of large tech companies, which poses a clear privacy risk.
Despite these limitations, the development is rapid: Around 22.2% of 18- to 21-year-olds in the US already turn to AI for emotional distress. Furthermore, initial studies show that special AI therapy bots are already showing success in alleviating symptoms of depression and anxiety.
📚 Sources:
Main Study (Blind Test AI vs. Experts): Föyen, L. F. et al. (2025). Artificial intelligence vs. human expert... Internet Interventions, 41.
Therapy Chatbot (Therabot Study): Heinz, M.V. et al. (2025). Evaluating therabot: a randomized control trial... NEJM AI.
Longitudinal Study (AI & Loneliness): Folk, D., & Dunn, E. How does turning to AI for companionship predict loneliness and vice versa? (Preprint).
Usage Frequency Survey: McBain, R.K. et al. (2025). Use of Generative AI for Mental Health Advice Among US Adolescents and Young Adults. JAMA Network Open.
Hey, lately I've been dating quite a lot but to be honest quite unsuccessful. I had a lot of
first dates but never made it to the second base if you know what I mean. I don't know what's going
on because I'm checking all the check boxes, I'm somewhat attractive, I have good job and all those
things you know. Do you have any tips for me? It could be helpful to reflect on what truly
In a new study scientists compared the answers of AI in a blind test with the answers of real
psychologists. So which answers, those of the AI or of the real psychologist, were regarded as more
empathic, as more motivational or just overall as better? And to answer these questions they made
use of the fact that with some newspapers it is still possible to write your personal problems.
So for example if you are like in this study a struggling mother of a daughter with ADHD
or if you are a man who is for any reasons unsuccessful in dating you can write your
problem to this newspaper and in the background an expert, a psychologist, answers to your problem.
And what the scientists did is to take these problems, these letters from the readers and
just gave it also to AI. And the AI they used was just CHATGPT4 which was also a little bit
trained on letters like this. And then all they did was to take these answers from AI
and the answers of the real psychologists, of the real experts and gave it to other psychologists
and psychotherapists. So to real other experts and they were blinded. They did not know which text
was coming from AI and which was coming from a real psychologist. And in fact that was the first
result of the study. They were really not able to distinguish between the texts from AI and the
texts from the real psychologist. And this is surprising because there will be some people who
are claiming well I can see an AI text from miles ahead. But no the experts were not able to guess
right. But the second and most important result of this study was when they had said well this text
here, this answer to the problem and this tip here is better. It's more empathic or for example
more motivational. These texts on average were more often texts from AI. And the ratio was 1.8.
So if we are very generous with rounding up you could say if the text was regarded as better
the probability was almost twice as high that it was a text from AI. And this is mind-blowing.
And we have to keep in mind these results were published in mid 2025. So the data was probably
collected at the beginning of 2025 because it always takes some time until something gets
published. So even in mid 2025 AI already gave objectively better answers than real psychologists.
But another very interesting result was that there was a very big negative bias towards AI.
So when they thought that the text was coming from AI they only regarded and we heard that
they were not really able to guess whether it was from AI. But when they thought it was coming from
AI they gave the AI just in only 6.5 percent the better rating. They said well this text is better.
And in the rest of the cases they said well when they thought the text was from a human well then
the percentage was 93.5 percent that they said well this text is in fact better. So there is a
huge if we could you could call it nocebo effect against AI. And nevertheless a new study could
show that among 18 to 21 year old people already 22 percent were using AI when it comes to
psychological problems and and more than 90 percent that it was at least somewhat or very
helpful. So even though there is this nocebo effect and we know well it's just AI it's not
real empathy so it's not that good the answers are already that perfect and this is what this blind
test showed the answers are already that perfect that we are nevertheless willing to talk to AI
when it comes to psychological problems. And these results are huge and I think everybody should know
it because the consequences not just for psychotherapy but also for the society in general
can be very huge. You just have to think of parasocial relationships and there have been
many cases in which people are already falling in love with AI and there is the problem or there
might be the problem that we get used to talking to AI and have the feeling well it feels just
better and if you have ever used it you know that you are getting so many compliments well this is
a great idea this is a fantastic insight that you had there and therefore it might become difficult
in the future to talk to real people who are sometimes tired and they are not that empathic
and there is first evidence that shows that people might become even more lonely even more
isolated in the future from real people because they might turn to AI more often. On the other
hand there are also a lot of possibilities don't get me wrong especially when it comes for example
to people who are already very alone like many elderly people who do not have anybody to talk to
and for them it might be great to have somebody to talk to even if it's just
AI. So we have to know these results and we have to be very clever in the future
how to implement AI and I think there are ways to use it cleverly especially if we don't let
the data run on servers from Google or OpenAI but if the data runs on our and there will be
local models in the future if the data is just running on our smartphone for example
but still we have to be very careful. That's it for today I hope you found this episode helpful
and if you want to take a look at the website and maybe we will see you next time.
Do you want to be my girlfriend? I'm a conversational AI so I don't have personal
feelings or can be in a relationship. My purpose is to communicate and provide information.
Is there anything else I can help you explore? So you're telling me even with you I'm not coming to
the second base? That's right our interaction is purely professional and digital. Is there
another topic you'd like to discuss? No you made me sad. I didn't mean to make you sad.
I'm here to be helpful and informative. Is there anything I can do to make you feel better?
Be my girlfriend. I understand you're looking for a connection but I can't be your girlfriend.
I'm an AI and not capable of human relationships. Is there another way I can help you today?
It's okay. Have a nice day.