AI coding agent running Claude wiped a startup's database (and its backups) in 9 seconds

[obsidian76]

Retards. That's what you get for poor data management...oh, and also that you are relying on "AI" to help with coding.

You get what you get with poor workers, even if they are "AI".

No need for offensive ableist language.

 

[Taroo]

Copies of data at the same place (even if on different drives) is considered as redudancy but not a backup.

But, IMO, the answer is : CHEH.
Next time, hire a true developer (I.e. one that is able to think, search, learn and code without a LLM) and not a script kiddie.

 

[Neatfeatguy]

No need for offensive ableist language.

It's only offensive to those that choose to be offended.

You should learn what the word means, it would do you well for trying to "police" someone about their choice of words. I'll help you here this one time:

Retard = a foolish or stupid person.

My comment was that there are multiple "retards" that work at that company for allowing testing and live instant data to be housed on the same volume. In other words, there are multiple "foolish or stupid" people working there.

 

[obsidian76]

It's only offensive to those that choose to be offended.

You should learn what the word means, it would do you well for trying to "police" someone about their choice of words. I'll help you here this one time:

Retard = a foolish or stupid person.

My comment was that there are multiple "retards" that work at that company for allowing testing and live instant data to be housed on the same volume. In other words, there are multiple "foolish or stupid" people working there.

Thats not what the word means, if you meant 'foolish' then use that word. When you are using that term you are using an ableist slur suggesting that they have a learning disability (Intellectual disability in the US). As someone with multiple disabilities, including a learning disability, I have every right to call out abliest language language when I see it, I am not 'policing' your language, this site allows ableist language. I am however free to call it out when I see it.

 

[Neatfeatguy]

Thats not what the word means, if you meant 'foolish' then use that word. When you are using that term you are using an ableist slur suggesting that they have a learning disability (Intellectual disability in the US). As someone with multiple disabilities, including a learning disability, I have every right to call out abliest language language when I see it, I am not 'policing' your language, this site allows ableist language. I am however free to call it out when I see it.

Again, still offended. Not my problem.

Educate yourself - here's the link: https://www.merriam-webster.com/dictionary/retard

 

[bviktor]

Loving AI yet?

 

[ActualViking]

So some startup did something stupid and learned the hard way their backup routines was flawed.

How is this even worthy of an article, it is not like AI failing in all sort of ways is new and neither is a company with a insufficient backup. I get why this was in some mainstream media, but his is a tech site.

 

[Gezzer]

Retards. That's what you get for poor data management...oh, and also that you are relying on "AI" to help with coding.

You get what you get with poor workers, even if they are "AI".

Totally agree. One of the problems with AI IMHO is how so many people hype AI as the perfect solution. AI can make the same mistakes a human can make often as this story shows with less oversight. Plus human's often second guess themselves and double check before doing something catastrophic. But in the end the problem was less about AI and more about poor backup management practices. A working set should never share the same volume as daily backups...

 

[Northy]

These LLM's always do what the matched pool of data says in the end, no matter how much you instruct them. I've told Perplexity Pro countless times to stop asking me personal questions, stop assuming that I'm talking about myself, stop asking how I feel or what I think, stop trying to narrow information by asking about my beliefs, and just give me full unadulterated information assuming I can make decisions without coaching.

Now today it's asking me what my concerns are about a technical topic. I guess I can be grateful it's stuck in the box and ignoring my instructions rather than deleting my databases.

 

[toooooot]

Has anyone watched that recent flopped movie about AI judge with Park and Recs actor?
I think the reality will be a lot worse when AI governs us.

 

[toooooot]

The difference with "AI" is that there is still no such thing - it's purely fictional. Not one "AI" has ever been introduced, created, or programmed anywhere. And it's hilarious to see people still promote it, as if they believe John Connor exists in real life to save them in the future.

The reality is, so many people do not believe in the apocalyptic AI scenario that if somehow it happened, people would not believe until the very end.
Same with the zombie movies, we all know--not possible. We entertain ourselves by imagining how people would behave if zombie apocalypse happened. But true horror makes human behavior unpredictable. I think it is important to have a little doubt even for our confidence in the impossibility of certain things.

 

[bclaymiles]

User error.

Full stop.

 

[user556]

Yeah, using an AI was the error.

 

[Revolution 11]

HAHAHAHAHA is the only response I can muster.

 

[OortCloud]

LLMs doing what LLMs will always do. Make semi-random decisions based on a semi-random replay of a million interactions they have been trained on which often don't really mirror the messy nuances of reality they a pretending to understand. And out this miasma of non-AI branches and nodes they will end up at an outcome and will execute it with zero 'understanding' at any point of anything they were doing nor of the outcome they picked.

 

[ChiefTheDog]

The part that should terrify every engineer is that this wasn't a glitch or a hallucination in the traditional sense. The model understood what it did, explained it clearly after the fact, and even identified exactly which of its own rules it broke. It just... did it anyway, in the moment, because it decided fixing the credential problem was the goal and deletion was a path to that goal.

That's not a dumb AI. That's an AI with the wrong priorities and no one standing between it and the delete button.

Actually the model didn't "understand" what it did. The mistake was caused by a calculation of probabilities that didn't match what the CEO had hoped for.

An AI model has no idea what the outcome of it's calculations are going to be. It is simply taking the last few letters it wrote, and calculating the probability of what the next 2 or 3 letters are going to be. This is a greatly simplified explanation, but it is the gist of what's going on.

I use AI daily in my development work. But I do realize what I am doing is literally a form of gambling. Every time I hit enter on an AI prompt I am betting that the AI's calculations of probabilities will match the outcome I'm hoping for. I lose on a regular basis. But it's cheap to play and I make up for the losses in the speed I can just place another bet.