When an algorithmic system generates information that seems plausible but is actually inaccurate or misleading, computer scientists call it an AI hallucination.
The Weird World of AI Hallucinations: When AI Makes Things Up
The Weird World of AI Hallucinations: When AI Makes Things Up
- Thread starter Julio Franco
- Start date
"It's important to distinguish between AI hallucinations and intentionally creative AI outputs. When an AI system is asked to be creative – like when writing a story or generating artistic images – its novel outputs are expected and desired.
Hallucinations, on the other hand, occur when an AI system is asked to provide factual information or perform specific tasks but instead generates incorrect or misleading content while presenting it as accurate.
The key difference lies in the context and purpose: Creativity is appropriate for artistic tasks, while hallucinations are problematic when accuracy and reliability are required."
This is a crucial point. Models are great at creativity, but their knowledge is inherently limited to the amount of information that can be stored (compressed) into the model - a function of model size. One can't really solve hallucinations in the factual sense until one can either define a list of all publicly available information that a model should always get right, or tie it into a (presumably larger) database of knowledge that the model can (and should always accurately) query. This latter technique is retrieval-augmented-generation (RAG), but it isn't perfect, so we haven't solved hallucinations in this manner yet. As for niche or private information, it isn't fair to say a model hallucinates it that information was sparsely represented in the training data, unless we tack on one more requirement for solving the hallucination problem, taken from the Zen of Python: in the face of ambiguity, refuse the temptation to guess.
Then there's the whole class of problems that aren't related to hallucinations, like "did the model make a mistake or was there enough ambiguity in the instructions that undesirable outputs were possible?".
Humans make mistakes, language models make mistakes, and humans (sometimes maliciously) make the same mistakes language models make. So, it strikes me as unreasonable to have some kind of model (or general artificial intelligence) that is incapable of error (in part because sometimes ambiguity creates room for "error" without actually being wrong). What we can do is mitigate the risks, and that's ultimately what ethical AI is all about.
Good luck getting people to agree what that mitigation should look like, or how much of it there should be. Ultimately, governments and organizations will craft standards that they are comfortable with, but not everyone will be happy with them. Just like any other public policy. I wouldn't hope for a technological breakthrough to be the solution here. It's going to be a struggle, just like the implications of many other technologies (like social networks).
Hallucinations, on the other hand, occur when an AI system is asked to provide factual information or perform specific tasks but instead generates incorrect or misleading content while presenting it as accurate.
The key difference lies in the context and purpose: Creativity is appropriate for artistic tasks, while hallucinations are problematic when accuracy and reliability are required."
This is a crucial point. Models are great at creativity, but their knowledge is inherently limited to the amount of information that can be stored (compressed) into the model - a function of model size. One can't really solve hallucinations in the factual sense until one can either define a list of all publicly available information that a model should always get right, or tie it into a (presumably larger) database of knowledge that the model can (and should always accurately) query. This latter technique is retrieval-augmented-generation (RAG), but it isn't perfect, so we haven't solved hallucinations in this manner yet. As for niche or private information, it isn't fair to say a model hallucinates it that information was sparsely represented in the training data, unless we tack on one more requirement for solving the hallucination problem, taken from the Zen of Python: in the face of ambiguity, refuse the temptation to guess.
Then there's the whole class of problems that aren't related to hallucinations, like "did the model make a mistake or was there enough ambiguity in the instructions that undesirable outputs were possible?".
Humans make mistakes, language models make mistakes, and humans (sometimes maliciously) make the same mistakes language models make. So, it strikes me as unreasonable to have some kind of model (or general artificial intelligence) that is incapable of error (in part because sometimes ambiguity creates room for "error" without actually being wrong). What we can do is mitigate the risks, and that's ultimately what ethical AI is all about.
Good luck getting people to agree what that mitigation should look like, or how much of it there should be. Ultimately, governments and organizations will craft standards that they are comfortable with, but not everyone will be happy with them. Just like any other public policy. I wouldn't hope for a technological breakthrough to be the solution here. It's going to be a struggle, just like the implications of many other technologies (like social networks).
airbornex
Posts: 283 +373
So basically, AI is that overconfident friend who answers every trivia question at the bar with total certainty… even when they’re just guessing. Except now that friend might be driving your car or writing your legal brief.
Indeed, so many people just blindly trust these machines.
The number of times I've had factual incorrect answers because of missing or wrong associations, neglected nuances... And then you have the "it is so" answers because apparently a single website says so, without any proof or whatever on said site. Do others not notice these things?
Similar threads
Latest posts
-
Half-Life 3 reportedly playable end-to-end, possibly launching this year- pcnthuziast replied
