Over 800 public figures, including "AI godfathers" and Steve Wozniak, sign open letter to ban superintelligent AI

[midian182]

A hot potato: Tech companies are in a race to create superintelligent AI, regardless of what the consequences of creating something much smarter than humans might be. It's led to hundreds of public figures from the world of tech, politics, media, education, religion, and even royalty signing an open letter calling for a ban on the development of superintelligence until certain conditions are met.

Among the 800+ signatories are two of the "Godfathers of AI," Geoffrey Hinton and Yoshua Bengio. Apple co-founder Steve Wozniak and Virgin Group founder Richard Branson are on the list, too.

Former chief strategist to Donald Trump, Steve Bannon, former Joint Chiefs of Staff Chairman Mike Mullen, Glenn Beck, Joseph Gordon-Levitt, and musicians Will.I.am and Grimes also signed the letter. Even Prince Harry and Meghan, the Duke and Duchess of Sussex, feel strongly about the subject.

The letter calls for a prohibition on the development of superintelligent AI until there is both broad scientific consensus that it will be done safely and controllably and there is strong public support.

The statement, organized by the AI safety group Future of Life (FLI), acknowledges that AI tools may have benefits such as unprecedented health and prosperity, but the companies' goal of creating superintelligence in the coming decade that can significantly outperform all humans on essentially all cognitive tasks has raised concerns.

The usual worries were cited: AI taking so many jobs that it leads to human economic obsolescence, disempowerment, loss of freedom, civil liberties, dignity, and control, and national security risks. The potential for total human extinction is also mentioned.

Last week, the FLI said that OpenAI had issued subpoenas to it and its president as a form of retaliation for calling for AI oversight.

According to a US poll that surveyed 2,000 adults, most AI companies' mantra of "move fast and break things" when it comes to developing the technology is supported by just 5% of people. Almost three quarters of Americans want robust regulation on advanced AI, and six out of ten agree that it should not be created until it is proven to be safe or controllable.

A recent Pew Center survey found that in the US, 44% of people trust the government to regulate AI and 47% are distrustful.

Sam Altman recently gave another prediction on when superintelligence will arrive. The OpenAI boss said it will be here by 2030, adding that up to 40% of tasks that happen in the economy today will be taken over by AI in the not very distant future.

Meta is also chasing superintelligence. CEO Mark Zuckerberg said it is close and will "empower" individuals. The company recently split its Meta Superintelligence Labs into four smaller groups, so the technology might be further away than Zuckerberg predicted.

Ultimately, it's unlikely the letter will prompt AI companies to slow their superintelligence development. A similar letter in 2023, which was signed by Elon Musk, had little to no effect.

Permalink to story:

Over 800 public figures, including "AI godfathers" and Steve Wozniak, sign open letter to ban superintelligent AI

 

[unoficialoficial]

AI is already at the super-intelligent level. Leading AI models archive IQ scores exceeding 130 (based on measures from trackingai.org) and possess vast knowledge bases capable of processing over one million tokens—equivalent to thousands of pages of information.

But how "super" does it need to be to count as superintelligent? If we consider this: the average human IQ is 100, and IQ points represent significant, hard-won steps. If you gathered one million people with an IQ of 100 and tasked them with solving a problem requiring a 120 IQ, they still wouldn't succeed as a group. One individual with a 120 IQ could solve it, but millions with 100 IQs cannot. Given this, an IQ of 130+ is remarkably impressive and clearly qualifies as "super-intelligent."

So, instead of being afraid of it, maybe you could use it to improve your workflow?

AI is an even more significant advancement than the invention of writing. Yet, some people want to slow it down! Perhaps some of them don't mean superintelligent, but conscious, AI. However, conscious AI will never happen. Consciousness cannot occur in silicon; it requires biological neurons and their microenvironment.

 

[user556]

There's nothing to worry about because current AI has no intelligence at all! Besides, what happened to all the AGI predictions? No signs of that ever happening either so I can't see anything else even better arriving in the foreseeable future.

 

[VariableSpike]

Current "AI" is just a language model which doesn't really have any inherent intelligence and doesn't actually "think", just assumes as to the chain of words and "thoughts" that fit the prompt you asked, hence the need to call on external resources when logic, maths or the like is involved so it doesn't just get stuck in circles, "AI" has its niches (as it did when "AI" wasn't yet a moniker and we just called it machine learning in the business space) but it is eons from being anything actually intelligent and I have seen it do countless dumb things, make up stuff and more which renders it useless unless you are using it as a toy to vaguely answer dumb questions no one else has thought of before

 

[ScottSoapbox]

The problem is not super intelligent AI.

It’s people believing we achieved super intelligent AI long before we have and putting it in charge of something vital.

 

[Axeia]

If you gathered one million people with an IQ of 100 and tasked them with solving a problem requiring a 120 IQ, they still wouldn't succeed as a group. One individual with a 120 IQ could solve it, but millions with 100 IQs cannot.

Do you have (non-AI generated) data to back that up?
IQ tests often are a test of knowledge rather than of intelligence (there isn't one golden standard test).

Get those million people, have them all take the same IQ test with the option to tap out on the questions they're not confident in answering. Now take the most frequent answer as the one for the test. Betcha the test won't come out as a 100, or 120 but far far higher. I'd even argue that AI works on the same level to some degree - weighing and averaging the most common solution.
No single person can know everything. But everyone together will by definition know all that is currently known.

That's all just based on the concept of knowledge. Intelligence which I'd argue is quite different and different people are intelligent in different way - most pronounced in those with autism who can be <b>extremely</b> intelligent when it comes to say math. Able to pick it up at a completely bonkers rate and come up with solutions problem no one else has been able to before but at the same time are unable to even take care of themselves.
(Tip to certain redditors amongst us, diagnosing yourself with autism does not mean you're super intelligent and right about everything )

 

[BobDolesGhost]

The problem is, if the US / Allies ban the creation of a super intelligent AI (when we're capable of creating one), our rival / enemy nations won't. Ignoring AI resulting in the end of the world for now, those nations will have a near limitless advantage over us then. I don't like it, but its the truth - "If we don't do create it, our enemies will"

 

[Axeia]

1) It would deflate the bubble (maybe not pop it, but definitely take some wind out of its sails) as a large part of the promise is having something more intelligent than us pesky humans. If you control god-tier intelligence then you're the one in control.
2) Everyone is far too afraid of 'falling behind' even though the progress curve is getting fairly flat and the difference between the top contenders is fairly small with each predictably falling into niches.

Thankfully this super intelligent AI still seems far off, the current ones already will resort to blackmail or even murder to prevent getting shut down (seems like certain SCI-FI content wasn't all that fictional). Now imagine that but outside a test setting and with an intelligence matched by nothing else. Who ever manages to do will suddenly regret thinking that falling behind was the worst option - actually getting there is.

Maybe I read too much SciFi as I'm sure these people are right*, almost as sure as I am of the fact that it won't stop everyone. Heck the US goverment is the most likely party to keep pushing for it, as the best country in the world *cough* they have a reputation to keep up, they must have the best AI.

* Not because I hold a lot of these people in high regard (I mean Prince Harry... wtf cares) they just happen to share my viewpoint on this particular subject.

 

[wiyosaya]

The tech world is nowhere near developing HAL-9000.

But anyway, "Over 800 public figures once again claim 'The Sky is Falling.'" works better as a title for me.

 

[pnntmp]

From those 800+ names, only 2 matter - Geoffrey Hinton and Yoshua Bengio.

Politicians, celebrities, prince Harry and his Maggie - all these are entirely irrelevant.
The title should be "Geoffrey Hinton and Yoshua Bengio sign open letter ...".

We should avoid at any cost safetists and other hysterical hypochondriacs deciding the future of AI, because if they are in charge we'll never make any progress. See what happened with nuclear power ..

 

[FF222]

1. What's the point of banning something that doesn't exist and we don't see how we could create one? All actual experts in the field (who don't have a vast interest in saying otherwise, because of business reasons) agree that LLMs won't ever become superintelligent, and they won't even get much smarter than they are now.
2. There's no point in banning anything you can't define precisely - and you can't define "superintelligent AI" either.
3. Anybody being able to create a superintelligent AI will create it. Then will support banning that anybody else create it.
4. This "ban" sounds very similar to me, because of all the above, ie. that the ban only serves the purpose of no new competitor getting ahead of those in power and already controlling current AI companies.
5. And finally, the more interviews I hear with Geoffrey Hinton, "grandfather of AI", the more I convinced I am that the guy is totally clueless about current AI models, or how they work. Obviously he pioneered the field, but he is now old, and just doesn't seem to comprehend new stuff.

 

[Xelions]

I love how people debate the notion of the ban, how intelligent is AI really, and whether we should be concerned now or in the future. I prefer to look at it differently. If its sentient-like, well let's play nice and if it ain't who cares for the moment. Seriously if AI ends us. Good! The way humans are running this planet, their actions and attitudes says plenty. If this is our time, so be it.

 

[axiomatic13]

The FUD machine is in high gear.

 

[joe2025]

Steve Wozniak has really gotten dumb after the Apple II.

 

[mbk34]

Super intelligent AI is inevitable. It doesn't matter if a random D list group of celebs say we shouldn't. It doesn't matter if we all suspect it will be a bad thing in the long run. If it's possible then it will be done and, in all honesty, I'd like to know what it thinks about.

 

[exothermic]

You can get 800 "experts" to sign anything. An antivax petition, a call for the Western world to give up its nuclear weapons, pro-regulation, anti-regulation, you name it.

 

[JMGuy]

The problem is, if the US / Allies ban the creation of a super intelligent AI (when we're capable of creating one), our rival / enemy nations won't. Ignoring AI resulting in the end of the world for now, those nations will have a near limitless advantage over us then. I don't like it, but its the truth - "If we don't do create it, our enemies will"

That’s not how things work.

My uncle was a senior analyst in military intelligence—one of those guys standing three rows back when the president was being briefed in the war room. Over dinner one night (and after a bottle of wine), he got a little loose-lipped. Before his wife shut him down for treading into classified territory, he talked about China’s capabilities versus ours. His general point was simple: if you’ve heard of it, it doesn’t matter. The U.S. military is usually 20–30 years ahead, technologically, of whatever’s publicly known.

This was twenty years ago, and he’s since passed, but I believed every word. This was a man who, even in retirement, had a government-grade surveillance system monitoring his property line.

If that’s true, then superintelligent AI isn’t something “on the horizon”—it’s already here, and not in the hands of Meta or Elon Musk. The U.S. doesn’t wait for private industry to invent such things; if it’s deemed too dangerous for civilians, it gets quietly folded into the black budget.

So when we talk about AI and geopolitics, we should leave military capability out of the conversation—they’re not even playing the same game.

 

[MaestroIT]

The AI bubble will pop before super intelligent AI

 

[BuckarooBonzaii]

Perhaps China is smiling?

 

[user556]

My uncle was a senior analyst in military intelligence—one of those guys standing three rows back when the president was being briefed in the war room. Over dinner one night (and after a bottle of wine), he got a little loose-lipped. Before his wife shut him down for treading into classified territory ...

More likely she shut him down for being a big-head and talking rubbish ... again.

 

[BobDolesGhost]

That’s not how things work.

My uncle was a senior analyst in military intelligence—one of those guys standing three rows back when the president was being briefed in the war room. Over dinner one night (and after a bottle of wine), he got a little loose-lipped. Before his wife shut him down for treading into classified territory, he talked about China’s capabilities versus ours. His general point was simple: if you’ve heard of it, it doesn’t matter. The U.S. military is usually 20–30 years ahead, technologically, of whatever’s publicly known.

This was twenty years ago, and he’s since passed, but I believed every word. This was a man who, even in retirement, had a government-grade surveillance system monitoring his property line.

If that’s true, then superintelligent AI isn’t something “on the horizon”—it’s already here, and not in the hands of Meta or Elon Musk. The U.S. doesn’t wait for private industry to invent such things; if it’s deemed too dangerous for civilians, it gets quietly folded into the black budget.

So when we talk about AI and geopolitics, we should leave military capability out of the conversation—they’re not even playing the same game.

For a country that produced (tried and failed) the Zumwalt, LCS Freedom, LCS Independence, naval disaster programs. For a country that can't produce new aircraft carriers, submarines, and other craft to replace its aging out craft that are many decades old.

For a country with infrastructure in very poor repair, cracking, and failing. For a country that has so few munitions that it can't send/sell enough to Ukraine to fight the paper tiger incompetent bullet sponge Russian military.

For a country that decided building a rocket with no original destination (referring to SLS/Orion), spontaneously later deciding "Oh - its going to the moon", which of course can't even reach low lundar orbit, costs $100+ billion over 15+ years and $4+ billion per launch, and then exclaiming at the proverbial last minute "Oh, I guess we need a landing craft and space suits" - and then shoving the tiniest bit of funding toward those HARDER PROBLEMS to solve... (And now we're complaining that SpaceX is "behind schedule" after giving the harder problem only ~4 years and <$3 billion - like the spacesuits are on schedule and even that SLS/Orion Artemis 3 is going to be ready on time - lolololol)

For a country where we want to get fast broadband network to every citizen, shove hundreds of billions to ISPs, and barely move the needle. For a country that spends absurd amounts of printed money at 0% interest and acts surprised when economic inflation has eroded our average citizen's purchasing power to the point where the basic necessities of food and shelter is an even larger struggle.

Obviously I could go on and on more. Almost everything our Fed/State governments touch turns to ****. We do have some modern successes like the F35 program (that we actually got it in production and its technology and capability are very good, not for its exorbitant cos, late deliveries, and even later block upgrades), the B21 not yet delivered but on time and on budget. But its honestly hard to think of modern successes that I'd be proud of my country.

I have a very hard time believing in the competence of our government. We've managed to F up a truly epic position of World Leader. We're not dethroned yet, and maybe we can turn ourselves around. I'm pessimistic to the chances though. Our society is hostile to intelligence, work ethic, and success. We as a society have little interest in doing some hard work to enrich our lot in life. My state has a fully free ride 2yr degree / trade school program. You can learn to be a plumber for free and in not too long of a time, be making more than a guy with a master's degree and tech job (not saying this is assured, but the path exists for you to pursue). People would rather do below the bare minimum effort, demand a government check, and mostly browse TikTok while half watching the latest "I just met this guy, lets get married for lols" TV show. Our country keeps moving closer and closer to being ruled by "Dwayne Elizondo Mountain Dew Herbert Camacho" (look it up if you don't get the reference, and prepare yourself to watch a not impossible, maybe probable, prophecy of our future)

A country populated by lazy morons just doesn't lead to the magic capabilities you and your friend believe we're capable of.

Sorry if I went off the deep end a bit. I've seen some **** in the past few days that makes me shake my head in dread, more than normal head shaking dread

 

[yannus]

How do you feel about AI ?

Super excited to lose my job.

Anyway, an open letter against AI won't change a thing.

 

[jeffbert]

Nobody lives forever, why worry about AI?

It is one thing for AI to be super-intelligent, quite another for it to become sentient.

 

[BogdanR]

The part with Super AI “empowering individuals” is deeply troubling, especially coming from someone like Zuckerberg.
These people have proven to have absolutely no social responsibility whatsoever.

 

[BogdanR]

The tech world is nowhere near developing HAL-9000.

But anyway, "Over 800 public figures once again claim 'The Sky is Falling.'" works better as a title for me.

At some point The Sky Will Be Falling. It is a question of when not if.
I wonder what final sly piece of wit you would deliver then.