Farmer trusted AI after it gave good advice, then it helped wipe out 25 acres of his crops

[midian182]

Facepalm: Most readers of this site know not to blindly trust AI on important matters, even if it got things right before. A farmer in Chuzhou, China, discovered this the hard way when he took weed and pest control advice from a chatbot. The result was the death of 25 acres of crops.

The 67-year-old farmer was recommended by the AI to use high-efficiency flupyrimethalin and flusulfasulfaether to kill weeds and mix it with thiamethoxazine and methyl salt. As reported by Taiwanese Chinese-language news website CTWANT (via Tom's Hardware), he followed the instructions blindly, failing to check that what he was doing was safe.

It wasn't until the next morning that the farmer realized he shouldn't have been so hasty. Around 24.7 acres of his sesame seedlings were dead.

The presumably upset farmer returned to the AI to ask why this had happened. The reason, apparently, was the use of flusulfasulfaether. The herbicide is mainly used to kill broadleaf weeds in soybean fields, but as sesame is classified as a broadleaf plant, it's affected by the chemical.

The AI also confirmed that flusulfasulfaether is supposed to be sprayed directly onto affected areas only, rather than spreading it across an entire field, which is how the farmer applied it, unfortunately.

The angry farmer said that the AI never warned him of the dangers of following its advice but, as any chatbot will tell you itself, it can make mistakes, misunderstand details, or rely on incomplete information. It's why ChatGPT has a disclaimer at the bottom warning that it can make mistakes and that users should check important information.

What's especially ironic in this instance is that the farmer initially didn't trust the AI app, but he changed his mind after it proved helpful. Given the end result, he probably should have trusted his gut.

There's a long list of people regretting putting their trust in AI-generated answers and regretting it. One of the worst cases happened last year when a 60-year-old man, after reading reports on the negative impact excessive amounts of sodium chloride (common table salt) can have on the body, decided to remove it from his diet.

When the man asked ChatGPT about substances that could replace salt, one of its suggestions was sodium bromide, which can be true for certain applications, such as cleaning, but certainly not for food. As the man discovered, his new diet gave him bromism, a.k.a. chronic bromide poisoning, a psychiatric disorder not commonly seen since the 19th century.

Permalink to story:

Farmer trusted AI after it gave good advice, then it helped wipe out 25 acres of his crops

 

[RudyBob]

This was no farmer that did this. Farmer's are a lot more intelligent. Unless they live in Hooterville.

 

[user556]

This was no farmer that did this. Farmer's are a lot more intelligent. Unless they live in Hooterville.

Even though LLMs do not have intent, they still speak with confidence. That alone can trick people.

 

[bviktor]

Yep. Definitely taking all our jobs.

 

[DrSuess]

I think CoPilot is still labeled "For Entertainment Purposes Only". They did this because they didn't want to get sued and people seem to believe everything they see and hear from the internet is true. They didn't anyone drinking bleach because CoPilot told them it was a good way to get rid of viruses.

 

[JMGuy]

Even though LLMs do not have intent, they still speak with confidence. That alone can trick people.

No. AI is a computer program, not a scapegoat for human laziness.

Feeling “tricked” by AI is a signal—a crystal clear symptom of an inability to judge good output. It’s an “inner voice” revealing a person's ignorance; AKA: naively accepting a spoon-fed answer for something a person is ill-equipped to verify due to lack of domain knowledge.

It’s actually OK to discover you have something more to learn. That is not a failure. We learn what we learn whenever we learn it. Nobody is born with knowledge. Stupidity, however, is the inability to recognize lessons and adjust.

AI doesn't “trick” people; it magnifies capability or lack of capability. I.e. it enhances productivity for those who know a field, how to verify results, and correct for deviation, and multiplies incompetence for those who don't. Use it with caution, verify the data, and blame your own lack of due diligence if you get it wrong.

 

[Axeia]

Before doing anything that could potentially cost you big. At the very least check with some of the other models.

I've had it plenty of times where before blindly trusting Gemini's output I run it by Claude it'll have a different opinion or useful supplemental information. People really need to get it in their hands that AI can get things very, very wrong no matter how will written the response is and how often it's been right before.

You live and you learn I guess, even at the farmers age of 67.

 

[NumberNine]

Well, at least it's marginally more interesting than reading recycled Google analytics reports about what people are searching this month. A little critical thinking goes a long ways.

 

[user556]

No. AI is a computer program, not a scapegoat for human laziness.

It's irrelevant what the narrator is. Being tricked is about confidence. Once the confidence is gained the speaker is treated as truthful.

Sure, the dude let his guard down but everyone does at some point. You, me, everyone is susceptible. Why do you think Trump suddenly thought it was a good idea to start bombing Iran when he'd previously complained so much about throwing away money on war?

 

[user556]

The best defense is to have a variety of heads reviewing. But that requires a continuous team effort, which farmers often don't have. Especially small holdings.

Writing the plan down, as you imagine it, can be fantastic for going over the plan in logical steps. Adding notes and proposals and solutions. In the organising process the flaws begin to stand out.

 

[wiyosaya]

IMO, this just goes to show that literally everything AI "says" needs to be verified. AI is not an explicitly trustable savior of humanity even though it has been marketed as such and people are dumb enough to believe that marketing.