Google removes AI answers from some medical searches after experts warn of risks to users' health

[midian182]

What just happened? Don't be surprised to find some of Google's AI Overviews conspicuously absent from certain medical searches. The company has removed several of these AI summaries following a report that found inaccurate information was being shown to users that could put their health at risk.

The Guardian reported at the start of January that AI Overviews, which are supposed to provide AI-generated summaries of search results, were putting people at risk by offering misleading health advice.

One example was the AI Overviews advising people with pancreatic cancer to avoid high-fat foods. Experts say this is exactly the opposite of what should be recommended, and may increase the patient's risk of death.

The summaries also showed incorrect information about crucial liver function tests, which could leave people with serious liver disease wrongly thinking they are healthy. Answers for women's cancer tests were also showing the wrong information, which experts say could cause people to dismiss serious symptoms.

Google claimed that the incorrect examples they were shown linked to well-known, reputable sources. But the company has now removed AI Overviews for several health queries.

Don't put your faith in AI Overviews

Questions that no longer surface Google's AI feature include "what is the normal range for liver blood tests?" and "what is the normal range for liver function tests?"

A Google spokesperson said: "We do not comment on individual removals within Search. In cases where AI Overviews miss some context, we work to make broad improvements, and we also take action under our policies where appropriate."

While medical experts have welcomed the move, there are warnings that rephrasing these questions could still lead to misleading AI Overviews being generated. The Guardian found that only slight variations to the queries prompted the AI Overviews to appear.

It's also noted that several incorrect or misleading AI Overviews that appeared in the original Guardian report have not been removed, including summaries related to cancer and mental health. Google reiterated that these had not been removed because they link to well-known and reputable sources and informed people when it was important to seek out expert advice.

The AI Overviews feature has a history of generating information that ranges from false or misleading to utterly nonsensical. It famously told people to put glue on pizza to help the cheese stick, and to eat rocks, neither of which are particularly good for your health or teeth.

A report from September found that over 10% of Google AI Overviews cite AI-generated content.

Permalink to story:

Google removes AI answers from some medical searches after experts warn of risks to users' health

 

[ScottSoapbox]

I blame Google and AI…

BUT I also blame people too lazy to read the actual article on how to treat their serious illness.

 

[deathspeed]

It's a step; now let's get rid of all AI overviews. If you ask Google if a 2003 Taurus has the same wheel bolt pattern as a 2005 Stratus coupe, AI overview confidently states yes, but roadside experience after a double blowout says in fact no, the spares are not interchangeable.
People rely on AI overviews too much because of the confident tone and placement at the top of search results, and some people are not even aware that they should dig deeper on important topics.

 

[JimF1]

This AI crap is already bad news and it's going to get worse.

 

[yRaz]

I blame Google and AI…

BUT I also blame people too lazy to read the actual article on how to treat their serous illness.

I never learned how to read

 

[brucek]

I'd rather they keep improving than give up. It's not like over-confident, under-informed or out-of-date information is a new problem created just by the AI era. You can get plenty of wrong information from traditional web searches, social media, friends of friends, the "expert" on the "news", etc.

Even human professionals are not immune from making mistakes, not being up-to-the-minute, disagreeing with each other, or stating accurate information in a hurried or confusing manner so that it's easy for the client to misinterpret it.

And then there's a whole other set of concerns where sometimes optimal human results intentionally come from less-than-precise information - I.e., a fully accurate assessment of your toddler's piano playing would probably put it at less than masterful, while most people will recognize that that's not actually what the toddler needs to hear at that moment. "You're doing great, keep going" may be the correct feedback even when it's not accurate feedback.

 

[unoficialoficial]

If the primary concern of doctors were patient health, they would advocate for Google to enhance the accuracy of its information, given that it serves as a fast and accessible resource where time is critical in numerous health issues. Instead, they perceive it as a competitor whose goal is elimination rather than improvement. This clearly suggests that they prioritize financial gain over patient well-being.

 

[ScottSoapbox]

I'd rather they keep improving than give up. It's not like over-confident, under-informed or out-of-date information is a new problem created just by the AI era. You can get plenty of wrong information from traditional web searches, social media, friends of friends, the "expert" on the "news", etc.

Even human professionals are not immune from making mistakes, not being up-to-the-minute, disagreeing with each other, or stating accurate information in a hurried or confusing manner so that it's easy for the client to misinterpret it.

And then there's a whole other set of concerns where sometimes optimal human results intentionally come from less-than-precise information - I.e., a fully accurate assessment of your toddler's piano playing would probably put it at less than masterful, while most people will recognize that that's not actually what the toddler needs to hear at that moment. "You're doing great, keep going" may be the correct feedback even when it's not accurate feedback.

I do think the panic about AI trustworthiness is somewhat overblown since we were often trusting a random reddit poster before AI.

Obviously, some things (like medical advice) are really important, but most of what people search for online doesn't really matter if it's wrong 5% of the time.

 

[SRB]

I'd rather they keep improving than give up. It's not like over-confident, under-informed or out-of-date information is a new problem created just by the AI era. You can get plenty of wrong information from traditional web searches, social media, friends of friends, the "expert" on the "news", etc.

Even human professionals are not immune from making mistakes, not being up-to-the-minute, disagreeing with each other, or stating accurate information in a hurried or confusing manner so that it's easy for the client to misinterpret it.

And then there's a whole other set of concerns where sometimes optimal human results intentionally come from less-than-precise information - I.e., a fully accurate assessment of your toddler's piano playing would probably put it at less than masterful, while most people will recognize that that's not actually what the toddler needs to hear at that moment. "You're doing great, keep going" may be the correct feedback even when it's not accurate feedback.

How about like almost every other technology that's achieved mass market status, Why don't we get it working BEFORE we drop it on an unsuspecting public. Why are we the guinea pigs?

 

[airbornex]

This isn’t even an AI hallucinating out of nowhere... it’s confidently remixing real sources into advice that’s actively harmful. Turns out summarizing medicine is harder than summarizing movie plots.

 

[scoffer]

But....but....but.....AI...AI...AI..........

Yours sincerely.
Emperor J.Wang.