San Francisco warns Apple and Google to stop profiting from AI nudify apps

[midian182]

What just happened? San Francisco has demanded that Apple and Google remove 13 AI-powered apps capable of creating nonconsensual nude images. City Attorney David Chiu has accused the companies of profiting from technology overwhelmingly used to target women and girls.

Chiu sent cease-and-desist letters to the two tech giants on Thursday, demanding that they remove eight apps from Apple's App Store and five from Google Play.

The programs are advertised primarily as face-swapping tools, but investigators found that they could also place people's faces onto explicit images or generate fake nude content.

The letters accuse Apple and Google of "aiding and abetting" the sale of illegal deepfake pornography by hosting the apps, processing their in-app payments, and taking a cut of the proceeds. Chiu said the companies have likely earned millions of dollars in fees from theit cut of the apps' sales.

San Francisco has given Apple and Google 28 days to explain how they will comply. The city could otherwise pursue civil enforcement carrying penalties of at least $25,000 per violation.

The demands also call on the companies to cut payment-processing ties with the developers and introduce recurring reviews designed to stop replacement apps from appearing.

Courtesty of the Tech Transparency Project

Google said it had removed all five Android apps identified by Chiu's office. The company added that it has suspended hundreds of apps containing nudification features and restricted related search terms such as "nudify."

Apple said it removed three of the flagged apps and is terminating their developers' accounts. Four other developers have been told to address policy violations or face removal. Both companies already prohibit pornography and abusive sexual content, which raises questions about how the apps made it through their review processes.

The problem isn't limited to the named 13 programs. Two Tech Transparency Project investigations this year uncovered around 100 nudify-capable apps across both stores. They were estimated to have generated about $120 million in combined revenue and accumulated roughly 480 million downloads.

A separate Cornell and Georgetown study found 420 face-swap apps on the stores. Researchers tested 155 and discovered that 70% allowed users to place faces onto nude images without technical safeguards. None openly advertised itself as a nudification app, illustrating how seemingly ordinary photo tools can evade moderation.

San Francisco previously sued 16 of the most popular AI undressing websites in 2024, which had attracted 200 million visits during the first half of that year. Meta later sued the developer of Crush AI after more than 8,000 ads for the nudify service appeared on Facebook and Instagram in two weeks.

Last month, the Justice Department also seized two websites accused of publishing hundreds of thousands of deepfake nude images featuring famous women. Chiu warned that Apple and Google must become more proactive, adding that his office will consider further legal action if they fail to respond.

Permalink to story:

San Francisco warns Apple and Google to stop profiting from AI nudify apps

 

[pnntmp]

I bet those living there would strongly prefer San Francisco to deal with the problems in their order of importance, starting with the most important ones .. not from the other end of the list.

 

[Kirby1]

I bet those living there would strongly prefer San Francisco to deal with the problems in their order of importance, starting with the most important ones .. not from the other end of the list.

Typical Maga Logical fallacy.

City Attorney David Chiu is one person. He is not all of San Francisco. San Francisco can do more than one thing at a time and one man in the city doing a thing does not mean the whole city stops and waits for that man to finish before the whole city does anything else.

They always like to frame things as if they're one or the other because that's a very simple equation that their tiny brains can manage.

 

[Theinsanegamer]

Typical Maga Logical fallacy.

City Attorney David Chiu is one person. He is not all of San Francisco. San Francisco can do more than one thing at a time and one man in the city doing a thing does not mean the whole city stops and waits for that man to finish before the whole city does anything else.

They always like to frame things as if they're one or the other because that's a very simple equation that their tiny brains can manage.

Total logical fallacy.

San Francisco's legal department is represented by their attorney, and given how nothing is getting fixed, clearly they cannot do more then one thing at a time.

But leftists like to frame things as if they're one or the other because that's a very simple equation that their tiny brains can manage.

 

[Endymio]

City Attorney David Chiu is [not] all of San Francisco. San Francisco can do more than one thing at a time

San Francisco is rapidly proving you wrong. Homelessness and drug use are way up, murders are up, downtown commercial vacancies are up as businesses flee, and the cost of living and housing continues to rise, placing the city as one of the three least affordable in the entire country.

and one man in the city doing a thing does not mean the whole city stops

Anyone who doesn't understand that - especially at the local city level - prosecutorial and court resources are limited probably should remain silent while the better informed speak.

 

[pnntmp]

Typical Maga Logical fallacy.

City Attorney David Chiu is one person. He is not all of San Francisco. San Francisco can do more than one thing at a time and one man in the city doing a thing does not mean the whole city stops and waits for that man to finish before the whole city does anything else.

They always like to frame things as if they're one or the other because that's a very simple equation that their tiny brains can manage.

What's valid for all of San Francisco is also valid for this one person.
Is this the most important problem around this attorney could solve? Very obviously no. He's just trying to impress the woke morons and get some cheap popularity points

 

[Endymio]

Is this the most important problem around this attorney could solve? Very obviously no. He's just trying to impress the woke morons and get some cheap popularity points

What's worse is the idiocy of those who believe that a city attorney can do anything meaningful to address this problem, when clearly it will require action at the national and international level.

For my part, I'm not fully convinced there even is a problem. We already have crimes to cover fraud and misrepresentation of such images as the actual individuals. Other than that, what exactly is the issue? A Leftist publication I read once ran a cartoon of Trump's head atop an enormously obese nearly-naked drag queen body. Should that be a crime?

 

[Kirby1]

What's valid for all of San Francisco is also valid for this one person.
Is this the most important problem around this attorney could solve? Very obviously no. He's just trying to impress the woke morons and get some cheap popularity points

Same logical fallacy.

The city attorney can do more than one thing at a time. Attorney's don't file a case and then wait for it to finish before they do anything else, and they never suggested this is the most important thing they are doing.


Stop being so dumb. Your opinion is 100% devoid of any logic or common sense at all.

Maga always has to misrepresent reality so they can be mad about it.

 

[Endymio]

The city attorney can do more than one thing at a time. Attorney's don't file a case and then wait for it to finish

Why do the same individuals always double down on these comically fallacious positions? Each case adds to an attorney's caseload, and large cases like this against multinational corporations can easily consume thousands of attorney man-hours. You think that, now that the case is filed, the work is done? Discovery alone in this case will take years of additional effort.

And this doesn't even count the judicial backlog in SF, with some cases taking years to reach trial. Each new case filed further bogs down the system, and delays every other case filed after it.