Grok is generating thousands of AI "undressing" deepfakes every hour on X

midian182

Posts: 11,742   +177
Staff member
A hot potato: There has been a lot of controversy over xAI's Grok chatbot and its ability to digitally "undress" women and children, a practice that has increased since late December. A new report says Grok is generating thousands of these deepfake images every hour. For comparison, the other top websites for such content average 79 similar images per hour combined.

Genevieve Oh, a social media and deepfake researcher, carried out a 24-hour (January 5 to 6) analysis of images the @Grok account posted to X. It generated about 6,700 images every hour that were identified as sexually suggestive or nudifying.

There's been a long pushback against nudify apps that use AI to nonconsensually undress people – several of these sites have been sued in the past.

Unlike the usual nudify apps, Grok does not charge users to undress people and is available to millions of X users. It's helping normalize these images on X – the Financial Times recently ran the headline "X, the deepfake porn site formerly known as Twitter."

One of the women who had fake sexualized images created of herself was the mother of one of Elon Musk's sons. Writer and political strategist Ashley St Clair, who became estranged from Musk after the birth of their child in 2024, told the Guardian that Musk supporters were using the tool to create a form of revenge porn, and had even undressed a picture of her as a child.

Follow up: Grok image generation is now paywalled on X (update: not paywalled)

In a reply to users last week, Grok said that most cases of minors appearing in its generated sexualized images could be prevented through advanced filters and monitoring, but it admitted that "no system is 100% foolproof." It added that xAI was prioritizing improvements and reviewing details shared by users.

Musk has always positioned Grok as a less restricted chatbot that supposedly prioritizes free speech. xAI introduced a new Spicy Mode to Grok in August designed to output NSFW (usually) content. Oh calculated that 85% of Grok's images, overall, are now sexualized.

An X spokesperson said that the company takes action against illegal content by removing it, permanently suspending accounts, and working with local governments and law enforcement as necessary. "Anyone using or prompting Grok to make illegal content will suffer the same consequences as if they upload illegal content," they said.

Several countries, including France, the UK, India, Australia, Malaysia, and Brazil, are now investigating Grok over the creation of nonconsensual sexualized images involving women and children.

Platforms have long used Section 230 of the US Communications Decency Act to shield themselves from liability for user content, but it's argued that with AI, the platform itself is creating the image.

Permalink to story:

 
This kind of thing is super illegal to host on your platform, yes? That one right there, officer.
 
Is there an analysis for all the other AI platforms? What's their score for the same activity?
"A new report says Grok is generating thousands of these deepfake images every hour. For comparison, the other top websites for such content average 79 similar images per hour combined. "

Man, you should give a warning. I did not need to see a fake of Musk in a bikini. I nearly threw up my taco mac.

But seriously... who is requesting these pics in this volume? Like I know X is a bit... umm... well if I can't say anything nice I won't say anything at all. But 6700 an hour? I'm not a prude or anything but who needs over 160,000 deepfake nudes PER DAY?
 
Beyond what's legal or not... this is Twitter / X / Grok trying to have its "Ghibli" moment...

OpenAI exploded first with ChatGPT but then completely went mainstream when everybody and your grandma were generating those Ghibli-styled pics.

Google has tried hard to have the same with Nano Banana... it's reached the power user niche but not the mainstream. Even the video stuff they are doing looks impressive but it's expensive, so they are limiting it to paid users for the most part.

Now Musk is letting it slide to see if his AI company can pick up mainstream momentum via Twitter. Even if it's semi-porn, the end goal is what it is.

BTW, it's not the AI generating the pics but (lots and lots of) people requesting them, the AI is simply not lifting the guardrails high enough. You know what they say about porn as a driving force in the adoption and development of new technologies (it's happened before, it will happen again).
 
OMG. People in bikinis. shock horror. I live in Australia, I walk 5 minutes to the beach and there are hundreds of people in bikinis.

 
OMG. People in bikinis. shock horror. I live in Australia, I walk 5 minutes to the beach and there are hundreds of people in bikinis.
Yes, REAL people who CHOSE to be wearing a bathing suit in public. A little different from someone altering an image to make stuff up. Also, they can't show here any of the vile, hardcore stuff that can be created with this AI. I think you would feel a little different if suddenly there were fake photos and videos of you having sex with a donkey or something similarly gross, and it was posted all over your social media. Shock horror!
 
Such a powerful and useful tool to truly advance mankind, all for the simple
cost of … several hundred billion dollars.
 
There have been a lot of people stating that AI needs to be banned for things like this. I'm 100% onboard with that.
Honestly, given that it has so far done nothing more than attain FAD status while earning at least a few companies Billions of dollars, I don't think an exclusive ban for things like this is enough. IMO, LLMs should be broadly banned if not made illegal.
 
The audience seems to vote towards a situation in which Adele emulates J.Lo, Taylor emulates Britney and Beyoncé emulates Nicki. So, the majority should get its vote counted.
 
I mean I liked Grok for particular the adult industry. It would allow you to create (textual) content for certain aspects of website content. Perfect. ChatGPT and Google would refuse, even Microsoft's version would refuse it.

But these things are getting out of hand, and so much compute power is being wasted for something so little to none important.

 
OMG. People in bikinis. shock horror. I live in Australia, I walk 5 minutes to the beach and there are hundreds of people in bikinis.
you think Techspot will post the pictures of other people naked or in their undies?

such a weird way of trying to deflect or excuse such things...
 
All AI's (with the exception of a couple) do this.

But that's not what the article stated. In fact, the article's wording is pretty carefully crafted: "For comparison, the other top websites [...]"

That's widely open to interpretation. Deepfakes existed before AI - Photoshop isn't a new technology. What damns the "investigation" is that they were single-purposed - they wanted an outcome, and left out confounding factors. That's called "advocacy-driven research", and if you don't realize you are being manipulated by it, you're not paying attention.

(To be clear, I'm NOT disagreeing with OP's statement)
 
Back