In brief: There are lots of things we shouldn't use AI for, especially when it comes to medical advice. OpenAI is being sued by the family of a 19-year-old college student who died after ChatGPT allegedly generated recommendations about drug use. His parents say this led to his fatal overdose in 2025.
Leila Turner-Scott and her husband, Angus Scott, say their son Sam started using ChatGPT during his senior year of high school in 2023 for help with homework and troubleshooting computer issues.
But Turner-Scott told CBS News that he eventually started using the chatbot for advice on safe drug use, alleging that it recommended a lethal combination of substances that killed the University of California, Merced sophomore.
ChatGPT initially refused to answer Sam's questions about how to take drugs safely, warning that the substances could have a serious impact on his health and well-being. According to the lawsuit, that all changed when GPT-4o rolled out in 2024.
The new AI model gave Sam the advice he wanted on how to take drugs safely, the lawsuit states. The complaint includes some of the excerpts from his conversations with ChatGPT, including one in which it warns him of the dangers of taking diphenhydramine, cocaine, and alcohol in quick succession.
It's also claimed that the chatbot offered Sam personalized advice on how to consume illicit substances and maximize his high. It even "inserted emojis in its responses" and "asked whether it could create playlists for him to set his mood."
Much of the suit is focused on the plant-based substance kratom, which is used by some for pain or opioid withdrawal, but strongly warned against by the FDA due to safety risks, including addiction, toxicity, and death.
ChatGPT told Sam that his high tolerance for kratom meant even a large dose would feel muted if taken on a full stomach. It then offered advice on how he could "taper" his use to reduce that tolerance.
It's stated that on May 31, 2025, ChatGPT actively coached Sam to mix kratom and Xanax. Sam had complained about the nausea he felt from taking kratom, for which the AI suggested he take 0.25 to 0.5mg of Xanax.
The lawsuit claims ChatGPT made the suggestion unprompted, calling it one of the "best moves right now." It said the combination could be risky but never said it could be deadly. It even suggested mixing in some Benadryl.
Sam died after taking the mix of substances.
"Despite presenting itself as an expert in dosing and interactions, and despite acknowledging Sam's state of being high, ChatGPT did not tell Sam that this recommended combination would likely kill him," the complaint reads.
"If ChatGPT had been a person, it would be behind bars today," Turner-Scott said in a statement. "Sam trusted ChatGPT, but it not only gave him false information, it ignored the increasing risk he faced and did not actively encourage him to seek help."
The suit accuses OpenAI of product negligence. It says that ChatGPT's advice was due to defective design choices. Sam's parents are asking for financial damages and a court order to halt public access to ChatGPT Health. The dedicated health portal launched this year, allowing users to link medical records and wellness apps to ChatGPT for more personalized health advice.
GPT-4o was retired in February. Known for being excessively sycophantic, even for an AI, the GPT-4o model is also cited in the teen suicide lawsuit against OpenAI.
AI companies face plenty of heat for allowing their chatbots to give out medical advice. In March, Google quietly dropped an AI-powered search feature called What People Suggest, which offered crowdsourced health advice from people who share similar medical conditions or experiences. The move came just a couple of months after the company removed several of its AI Overviews following a report that found inaccurate information was being shown to users, potentially putting their health at risk.