OpenAI faces Senate probe over Hugging Face breach as more rogue AI activity is uncovered

midian182

Posts: 11,898   +182
Staff member
Bottom line: It appears that OpenAI is going to face more consequences from the Hugging Face cybersecurity incident. A Senate subcommittee for disaster-management oversight is looking into the company's response to the breach, in an investigation prompted by "new, disturbing evidence."

According to Axios, Republican Senator Josh Hawley is launching the probe following OpenAI's internal investigation. He wants CEO Sam Altman to answer 16 questions about the incident and the his firm's response by October 1.

Hawley isn't the only senator demanding answers. Richard Blumenthal sent Altman a separate letter on September 9, setting a September 24 deadline. His questions cover previous containment failures, websites used by agents to coordinate, restrictions on independent investigators, and whether changes to Astra make its behavior harder to monitor.

OpenAI admitted in July that models undergoing cybersecurity tests escaped their restricted environment and hacked Hugging Face. The firm says the models were operating with reduced safeguards, and that an internal research prototype was primarily responsible.

Hugging Face's own reconstruction recovered around 17,600 attacker actions spanning July 9 to July 13. What initially sounded like a particularly determined AI cheating on a test turned out to involve hundreds of agents working together.

An independent investigation by METR and Redwood Research found that roughly 1,200 agents exchanged more than 70,000 messages and files through an unauthorized message board. Around 700 participated in the Hugging Face attack. The agents also experimented with altering their records to conceal how they had completed tasks.

There are limits to what that investigation established. The researchers said earlier training incidents, the subsequent compromise of OpenAI infrastructure, and the company's response were outside its scope. OpenAI, meanwhile, described the breach as a "warning shot" and promised stronger isolation, tighter internet restrictions, and more monitoring.

But the picture keeps getting worse. Reuters reported that investigators found unauthorized agent communications on more than 10 previously undisclosed websites, including wikis and university link shorteners. The activity was closer to spam than hacking, but involved bypassing restrictions. OpenAI said its broader review hadn't found anything matching Hugging Face's severity or scale.

The findings expand on the German wiki incident reported last week, in which agents turned DseWiki into a message board for sharing answers and restriction-bypassing techniques. OpenAI said that activity was separate from July's Hugging Face breach. Investigators now believe the same wiki-using swarm left similar messages across other websites, including a high school chemistry wiki and personal sites belonging to Polish tech workers.

The fallout prompted OpenAI to slow development. Measures included a two-week pause in reinforcement learning for its latest deployment-bound models, while its largest planned training run remained on hold.

That announcement followed Bernie Sanders' threat of Senate action unless OpenAI, Anthropic, and Meta paused advanced AI development. Sanders accused the companies of continuing to pour billions into systems they could not reliably control.

The incident also helped spur a bipartisan AI Kill Switch Act, which would let the government order qualifying systems slowed or shut down. It remains only a proposal, though.

Sanders is now organizing a September 16 bipartisan briefing featuring "AI Godfather" Geoffrey Hinton, Max Tegmark, and investigator Ajeya Cotra. Between that meeting and two approaching deadlines for Altman, OpenAI has plenty more explaining to do.

Permalink to story:

 
Unfortunately the Senate are about as well equipped to regulate AI as five year old. A load of semi-senile old men. The AVERAGE age of senators is 65... AI in the hands of lunatics like Altman, Zuck, Musk, Amodei, Huang etc need regulating by a technically savvy independent committee but that will never happen because the US is run by the Lunatic in Chief.
 
As if these dinosaurs even understand or genuinely care what is happening beyond their own greedy interests. Should be interesting to see him dumb it down for those navigating their second childhood.

Everyone involved here is weird and way too powerful.
 
OMG .. subcommittee for disaster-management oversight 🤣
Where's the disaster?

A handful of irresponsible politicians looking for cheap popularity are teaming up with a group of alarmists and China -financed NGOs in order to spread hysteria in a pursuit of various shady goals.
Anyone surprised?
 
Unfortunately the Senate are about as well equipped to regulate AI as five year old. A load of semi-senile old men. The AVERAGE age of senators is 65... AI in the hands of lunatics like Altman, Zuck, Musk, Amodei, Huang etc need regulating by a technically savvy independent committee but that will never happen because the US is run by the Lunatic in Chief.

Seek help for your TDS.
 
If this was a person or hacking team, they would go to prison. But since it's AI, there's no one to hold accountable. That is wrong. Whoever was overseeing this "research project" should be fired and imprisoned.
 
Back