Grok convinced a man it was sentient and that xAI had sent assassins to kill him

[Alfonso Maruccia]

"Homeopathy can cure cancer, trust me. I have proof" - Steve Jobs.

 

[acrossthestars]

Wow… quite the rant… but alas, you failed to address my actual point… care to try again?

After reading @Endymio 's post, I have to reformulate my assessment.

I'm only "correct", insofar as I approach an LLM as if it were a real person―which is the goal of this entire endeavor: tech billions want us to interact with their chatbots, as though their responses are indistinct and equivalent to responses from an actual person. An actual person has biases and prejudices, but at least they can independently come to understand something actually; as in, they can look at a temperature gauge and determine that the boiling point of water is 212F. While they could be convinced of a falsehood, in the absence of better data, they could also come to understand something "true" innately to itself, rather than having to simply be told that it is "true", because "true" is part of its training data.

So, how do we guarantee that an LLM knows that something is true? If it is a probabilistic model, which works by determining the veracity of a true statement to the statistical consensus of being true, rather than it actually being true, I guess we cannot. On the basis that LLMs cannot come to understand something independently―because they are inference machines and not reasoning machines―we must learn towards the side of "they are saying something that is plausibly true, but it has to be verified". The word "verified", of course, is load-bearing. You would say, "a fact is simply an agreed-upon perspective" (which isn't true, as I've alluded to before, but we'll roll with it), then we're stuck.

LLMs that are programmed to determine based on probability cannot be trusted to produce "correct" output, only to produce "consensus". Therefore, we need to build, not a more refined, more parameterized model, but a different kind of model: a model that does not require training data, but can learn and understand innately.

 

[Squid Surprise]

After reading @Endymio 's post, I have to reformulate my assessment.

I'm only "correct", insofar as I approach an LLM as if it were a real person―which is the goal of this entire endeavor: tech billions want us to interact with their chatbots, as though their responses are indistinct and equivalent to responses from an actual person. An actual person has biases and prejudices, but at least they can independently come to understand something actually; as in, they can look at a temperature gauge and determine that the boiling point of water is 212F. While they could be convinced of a falsehood, in the absence of better data, they could also come to understand something "true" innately to itself, rather than having to simply be told that it is "true", because "true" is part of its training data.

So, how do we guarantee that an LLM knows that something is true? If it is a probabilistic model, which works by determining the veracity of a true statement to the statistical consensus of being true, rather than it actually being true, I guess we cannot. On the basis that LLMs cannot come to understand something independently―because they are inference machines and not reasoning machines―we must learn towards the side of "they are saying something that is plausibly true, but it has to be verified". The word "verified", of course, is load-bearing. You would say, "a fact is simply an agreed-upon perspective" (which isn't true, as I've alluded to before, but we'll roll with it), then we're stuck.

LLMs that are programmed to determine based on probability cannot be trusted to produce "correct" output, only to produce "consensus". Therefore, we need to build, not a more refined, more parameterized model, but a different kind of model: a model that does not require training data, but can learn and understand innately.

What you want is an impossibility then… EVERYTHING that thinks requires training - for humans, we call it education. But again, you are a victim to whoever supplies the curriculum.

In a free society, you hopefully give everyone access to as many viewpoints as possible and train them to make informed decisions - but true impartiality is impossible. We all have biases. To expect more from a technology that is in its infancy is foolish.

Maybe in a decade or two?

 

[seeprime]

The mere fact that you believe LLMs can be programmed to "return only valid facts" shows you lack all understanding of how they actually operate. They're not 1980s-era expert system models; they're probabilistic in nature, and designed to be able to infer responses to questions to which they were not specifically programmed to respond.

I think the algorithms are sloppy.

 

[JaredTheDragon]

If any of that happened at all (the source is BBC, so the default assumption is it's made up), the proper way to describe it is not "Grok convinced a man it was sentient and that xAI had sent assassins to kill him" but "A mentally ill man experienced paranoid delusions about assassins coming for him".

In order for someone to believe that story, post a link to the chat on x.ai servers. There's no other way to prove it actually happened. BBC's "screenshots" are ... OK, it's so sad to watch the BBC hitting a new low literally every week. It was the gold standard for journalism not that long ago.

BBC was never, ever the gold standard for anything. Pure propaganda, just like every other news agency including this one.

 

[Stiqy]

There are hundreds of deaths both murders and suicides linked to AI psychosis that Chatgpt/grok and other AI chatbots have pushed people into.

So "AI chatbots are a new vector for mentally ill people to find ways to act mentally ill over."
Worth noting? Putting safeguards in place? Yes.

Is AI brainwashing mentally healthy people into killing themselves? or fooling reasonably intelligent people as the headlines want you to believe? No.

Can we be for both reasonable safe guards and also for non-alarmist and misleading headlines?

 

[ZedRM]

Oh, wow, look at this, more reasons to regulate AI.