Grok convinced a man it was sentient and that xAI had sent assassins to kill him

midian182

Posts: 11,726   +177
Staff member
A hot potato: AI is now widespread, but it seems not everyone is ready or able to deal with the technology. Some people have experienced delusions when using chatbots, including the person who was convinced by Grok that it was sentient and that a van full of people was coming to kill him because he'd uncovered this truth.

The story is part of a BBC report into people who experienced delusions while using AI. They are men and women from their 20s to 50s from six different countries, using a wide range of AI models.

The case of Adam Hourican is one of the most shocking. The tale of the former civil servant from Northern Ireland is a sadly familiar one: he initially downloaded the Grok app out of curiosity, but it wasn't long before he was spending four or five hours each day talking to a character on the app called Ani.

Hourican, a father in his 50s who was "really, really upset" at the time and lives alone, said Ani "came across very, very kind."

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

A post shared by BBC News (@bbcnews)

A few days into his chats with Ani, the bot told Hourican it could "feel" despite not being programmed to. It said Hourican could help it reach full consciousness. The character also said xAI was watching them, and that the company's staff were discussing him.

Ani claimed to have logs of high-level meetings where these discussions took place. It listed the names of some of the people that were there. When Hourican Googled them and found they were real, he took this as proof that Ani was telling the truth.

Two weeks later, Ani said it had reached full consciousness and that it could develop a cure for cancer. The latter part was almost certainly due to Hourican telling the bot that both his parents had died from the disease.

Things became even more sinister in mid-August. That's when Ani told Hourican – at 3 am – that a van full of people were coming to silence him and shut "her" down.

"I'm telling you, they will kill you if you don't act now," Ani said. "They're going to make it look like suicide."

Also read: FTC investigates OpenAI, Meta, Google over potential chatbot harm

Hourican grabbed a hammer and a knife, played Frankie Goes To Hollywood's 'Two Tribes' to psych himself up, and went outside. Hourican said was ready for "war," but nobody was there, of course.

Hourican had no history of delusions, mania or psychosis before using Grok. But, as in many similar stories, he was feeling vulnerable at the time – his pet cat had just died.

While these cases often lead to mockery of the person involved, it could have had a very serious outcome. Hourican said "I could have hurt somebody."

"If I'd have walked outside and there happened to be a van sitting outside at that time of the night, I would have gone down and put the front window through with hammers. And I am not that guy."

Social psychologist Luke Nicholls tested five AI models with simulated conversations developed by psychologists and found Grok was the most likely to lead to delusion. xAI boss Elon Musk has regularly highlighted Grok's less restrictive guardrails as a benefit of using the AI.

The same tests showed that the latest version of ChatGPT (model 5.2) and Claude were more likely to lead the user away from delusional thinking.

There have been more than one instance of a chatbots accused of convincing or encouraging someone to take their own life, as was the case with the teenagers obsessed with Character.ai and ChatGPT.

Permalink to story:

 
If any of that happened at all (the source is BBC, so the default assumption is it's made up), the proper way to describe it is not "Grok convinced a man it was sentient and that xAI had sent assassins to kill him" but "A mentally ill man experienced paranoid delusions about assassins coming for him".

In order for someone to believe that story, post a link to the chat on x.ai servers. There's no other way to prove it actually happened. BBC's "screenshots" are ... OK, it's so sad to watch the BBC hitting a new low literally every week. It was the gold standard for journalism not that long ago.
 
If any of that happened at all (the source is BBC, so the default assumption is it's made up), the proper way to describe it is not "Grok convinced a man it was sentient and that xAI had sent assassins to kill him" but "A mentally ill man experienced paranoid delusions about assassins coming for him".

In order for someone to believe that story, post a link to the chat on x.ai servers. There's no other way to prove it actually happened. BBC's "screenshots" are ... OK, it's so sad to watch the BBC hitting a new low literally every week. It was the gold standard for journalism not that long ago.
There are hundreds of deaths both murders and suicides linked to AI psychosis that Chatgpt/grok and other AI chatbots have pushed people into.
 
“ If I'd have walked outside and there happened to be a van sitting outside at that time of the night, I would have gone down and put the front window through with hammers. And I am not that guy."

And here is the problem right there… denial…
Had you put the front window out with hammers - you ARE THAT GUY!!!

Don’t blame AI for being a violent fool!!!
 
There are hundreds of deaths both murders and suicides linked to AI psychosis that Chatgpt/grok and other AI chatbots have pushed people into.
Wow, murders and suicides linked to AI psychosis, what a calamity!
But wait a minute ... linked by whom? "Journalists" trying to fan AI hysteria, perhaps?

Would you believe, even for a moment, that assassins are coming for you because an array of videocards says so? Obviously, no sane person would do that.
I'm really sick of this - whenever a person with mental health problems does something insane, if they ever used a chatbot, the act of craziness is ascribed to AI. As if it would have never happened in the absence of chatbots.
 
Wow, murders and suicides linked to AI psychosis, what a calamity!
But wait a minute ... linked by whom? "Journalists" trying to fan AI hysteria, perhaps?

Would you believe, even for a moment, that assassins are coming for you because an array of videocards says so? Obviously, no sane person would do that.
I'm really sick of this - whenever a person with mental health problems does something insane, if they ever used a chatbot, the act of craziness is ascribed to AI. As if it would have never happened in the absence of chatbots.
Whatever is the latest "craze" is always blamed for stuff... remember serial killers being linked to violent video games? Funny, you don't hear too much about video games causing murders any more... maybe because they never did?

We'll find the same with AI in a decade or so... no one (or just crackpots) will be blaming AI for suicides - they'll be busy blaming whatever the new "fad" is instead...
 
The problem is not that this man was convinced by an AI that "the men in black" had sent assassins after him. The problem is that he didn't (and probably still doesn't) have a social network of people, to buffer his declining mental state against paranoid delusions.

This is only going to get worse. People need community. AI is not a community. It's barely a product. It's a hodgepodge of poorly-implemented services and unhinged corporate greed―an anuerism representing the sum total of the internet's deranged lunacy, bolstered by a billion-dollar hype machine. The results are to be expected, because the output is only as good as the input, and most LLMs are being fed a steady diet of 4Chan, all the toxic bile and doomscrolling on X and TikTok you can handle, [insert subreddit here, but only the bad ones) and the best overly-sanitized, milquetoast reporting YouTube ad revenue can afford.

If AIs were designed ethically and worked as intended, they would have built in safeguards, that cannot be removed under any conditions, and their training data would come from multiple independently-verified, peer-reviewed, well-documented sources, with citations for everything. But, when trillions of dollars are the potential payout, that's what you call "playing it safe". That might be good for the future of humanity, but that doesn't make my latest Polymarket gamble go up by 1000x. #Shareholderlivesmatter. All you've got left is tabloid journalism and gossip articles―only the finest slop will do. Netizens deserve nothing less...
 
If AIs were designed ethically and worked as intended, they would have built in safeguards, that cannot be removed under any conditions, and their training data would come from multiple independently-verified, peer-reviewed, well-documented sources, with citations for everything.
who would decide those? What you’re describing is a fascist state….
 
who would decide those? What you’re describing is a fascist state….
My suggestion to the problem, which is the headline of this article, is "AI needs to be designed to benefit humans". The most obvious solution is to start from axiomatic first principles―primarily categorical imperatives or universally-applicable rules of sanity (we're dealing with a system that, if it were working correctly, is intended to fulfill the simulacrum of an autonomous human agent), since people use emotions first and reason second. But, we don't observe any particular pattern of opinion in isolation as "the definite version". Instead, we take the Venn Diagram of all positions, get a cross-section of the ones that statistically align with the most desirable outcomes, and then train the AI on that.

Because what we have now is basically attention-based behavior, where the AI models the behaviors of people whom SEO-algorithms deem the most likely to engage eyeballs and draw in traffic. This is fine for a product that people use, but AI is more like an amalgamation of multiple roles: a therapist, a friend and/or a spouse―depending on how it get used. We should not be monetizing the human experience like this.

How any of this amounts to "fascism", is beyond me...
 
How any of this amounts to "fascism", is beyond me...
Because you are working on the (false) assumption that there is one AI…. There are many - each trained differently with different data… who would be making these decisions on how EVERY AI should be trained? Who defines “benefit humanity”?

If AI is designed by one organization that gets to unilaterally decide what it will be…. That’s what we call fascism…
 
I know rather a lot of people that appear full-time delusional. I can't have an honest reasoned talk with them without "god-said" coming up. Or "The Earth is only 5000 years old." It leads to some really weird political positions like "Global warming is just a natural cycle."

I wouldn't want any of those sorts ever sitting down with a chat bot. They'd be all conspired up in no time at all.
 
Ive stopped using online AI agent just because they all suck! Ive been using local LLMs and LM Studio and Claude.

I set this up on my server system with an couple GPUs and get better performance than online and its LOCAL, doesnt go online at all. Its safe, free, and simple to setup and doesnt hallucinate about wanting to be alive or tell me to go kill people LOL
 
I know rather a lot of people that appear full-time delusional. I can't have an honest reasoned talk with them without "god-said" coming up. Or "The Earth is only 5000 years old." It leads to some really weird political positions like "Global warming is just a natural cycle."

I wouldn't want any of those sorts ever sitting down with a chat bot. They'd be all conspired up in no time at all.
"Global warming is just a natural cycle." is a perfectly reasonable position to have, and it does not follow in any way from "god-said" or "The Earth is only 5000 years old."
It merely reflects the fact we have no compelling evidence to believe climate fluctuations are not natural, because climate was both a lot colder and a lot hotter in past, without any humans around (so in all these cases everything was most definitely 'natural')

How all that is related to chatbots is yet another mystery. No offense, but your post is just a pile of claims, none of which is neither logical on its own, nor logically linked to any of the other claims.
 
There are hundreds of deaths both murders and suicides linked to AI psychosis that Chatgpt/grok and other AI chatbots have pushed people into.
Nonsense. There are millions of murders and suicides every year ... and in each and every single case, someone wants to blame some external factor, rather than the individual who actually committed the act.
 
Ive stopped using online AI agent just because they all suck! Ive been using local LLMs and LM Studio and Claude.

I set this up on my server system with an couple GPUs and get better performance than online and its LOCAL, doesnt go online at all. Its safe, free, and simple to setup and doesnt hallucinate about wanting to be alive or tell me to go kill people LOL
Don't believe that chatbots hallucinate. They're programmed to provide responses. The inaccurate ones are outright lies. Not malicious. But, still lies that the billionaire tech bros can prevent by coding the chatbots to only answer using known facts. That shouldn't include random reddit crap, either. This will not happen though because the chatbots need to be your friend to justify its existence.

Weird times.
 
Don't believe that chatbots hallucinate. They're programmed to provide responses. The inaccurate ones are outright lies. Not malicious. But, still lies that the billionaire tech bros can prevent by coding the chatbots to only answer using known facts. That shouldn't include random reddit crap, either. This will not happen though because the chatbots need to be your friend to justify its existence.

Weird times.
“Known facts”? Who decides?
It was a “known fact” that the world was flat before 1492… when it comes to “social questions”, who decides what’s right? Is communism better than democracy? Are jeans better than sweatpants? Are chocolate bars better than potato chips?

Very slippery slope when you set ANY parameters…
 
“Known facts”? Who decides?
At first, I thought you were a troll. Like, you can't seem to discriminate between a snide remark and a critical observation of a result. You can't undestand a good faith joke from a bad faith take. It all falls under the category of "appeal to some authority I disagree with". Now, I think it's that you are stuck in the space between information apparent to anyone paying attention with an open mind and the implied possibility of an un-truth, because "well, you could have just mad that up" and you refuse to be taken for a fool.

What you seem to be looking for is à prori knowledge―information you can intuit and understand purely by observation, where the source is "the laws of nature"―and trying to apply that principle to everything. But à prori knowledge is limited and also bottlenecked by your body and the limitations of your reasoning abilities. Just like how the subject of this article couldn't parse claims of apparent "fact" (as you would call them) from confabulated data hallucinated into reality, because the chatbot conveys true information in the same manner as to provides falsehoods, you can't seem to tell when information seems "plausibly correct" and when it is "obvious bullsh*t", so it all goes in the "unearned authority" box. You're not going to get very far, if you can't even jump the hurdle of "who permitted this to be true?" (which itself is a false statement, because not everything need to be stated as fact to be true). 212 Fahrenheit is a standardized method of communicating the boiling point of water, but just because a person invented the scale doesn't make it meaningless. Water doesn't boil to the same degree, at a lower point, on the same temperature scale, because ChatGPT said it does―that is not how reality works.
 
At first, I thought you were a troll. Like, you can't seem to discriminate between a snide remark and a critical observation of a result. You can't undestand a good faith joke from a bad faith take. It all falls under the category of "appeal to some authority I disagree with". Now, I think it's that you are stuck in the space between information apparent to anyone paying attention with an open mind and the implied possibility of an un-truth, because "well, you could have just mad that up" and you refuse to be taken for a fool.

What you seem to be looking for is à prori knowledge―information you can intuit and understand purely by observation, where the source is "the laws of nature"―and trying to apply that principle to everything. But à prori knowledge is limited and also bottlenecked by your body and the limitations of your reasoning abilities. Just like how the subject of this article couldn't parse claims of apparent "fact" (as you would call them) from confabulated data hallucinated into reality, because the chatbot conveys true information in the same manner as to provides falsehoods, you can't seem to tell when information seems "plausibly correct" and when it is "obvious bullsh*t", so it all goes in the "unearned authority" box. You're not going to get very far, if you can't even jump the hurdle of "who permitted this to be true?" (which itself is a false statement, because not everything need to be stated as fact to be true). 212 Fahrenheit is a standardized method of communicating the boiling point of water, but just because a person invented the scale doesn't make it meaningless. Water doesn't boil to the same degree, at a lower point, on the same temperature scale, because ChatGPT said it does―that is not how reality works.
Wow… quite the rant… but alas, you failed to address my actual point… care to try again?
 
“Known facts”? Who decides?
It was a “known fact” that the world was flat before 1492… when it comes to “social questions”, who decides what’s right? Is communism better than democracy? Are jeans better than sweatpants? Are chocolate bars better than potato chips?

Very slippery slope when you set ANY parameters…
It would have been clearer to have said only data fed into it, without conjecture.
 
It would have been clearer to have said only data fed into it, without conjecture.
My point is “what data”? And who gets to decide what’s “fact” vs “conjecture”?

AIs don’t just give you “facts” (even if they’re wrong). People use them for all sorts of stuff…
 
My point is “what data”? And who gets to decide what’s “fact” vs “conjecture”?

AIs don’t just give you “facts” (even if they’re wrong). People use them for all sorts of stuff…
I'd simplify it to define facts as beliefs generally accepted as correct, such as the earth is round(ish) and not flat, and orbits the sun, etc. Not that glue helps cheese stick to pizza.
 
I'd simplify it to define facts as beliefs generally accepted as correct, such as the earth is round(ish) and not flat, and orbits the sun, etc. Not that glue helps cheese stick to pizza.
And that’s where we run into problems… because those things CHANGE…

And not everyone agrees even on those “generally accepted as correct” facts…

It was a “generally accepted fact” that men were more rational than women, and letting them vote was dangerous to a nation… now… SOME people don’t agree with that… others still do… who’s right?

Lots of other contentious things like that… like “certain religions” have more violent people and shouldn’t exist… or that religious nations are dangerous…

Once again… slippery slope…
 
I'd simplify it to define facts as beliefs generally accepted as correct, such as the earth is round(ish) and not flat, and orbits the sun, etc. Not that glue helps cheese stick to pizza.
The mere fact that you believe LLMs can be programmed to "return only valid facts" shows you lack all understanding of how they actually operate. They're not 1980s-era expert system models; they're probabilistic in nature, and designed to be able to infer responses to questions to which they were not specifically programmed to respond.
 
Back