AI safety expert stops saving for retirement, he believes AI could wipe out humanity within a decade

midian182

Posts: 12,020   +183
Staff member
Crystal ball: Worried about the increasing likelihood that an AI will go rogue and kill us all? It's a depressing thought, but one AI safety leader is looking on the positive side: his concern that the technology will bring about mass extinction in the next decade has led him to stop saving for retirement.

Nate Soares, the president of the Machine Intelligence Research Institute (MIRI), revealed his pessimistic choice to The Atlantic CEO Nicholas Thompson in a recent interview for the latter's The Most Interesting Thing in AI series.

"I do not save for retirement right now. I have a 401(k) from when I worked at Google, but I haven't contributed for a decade," Soares said.

Apocalyptic warnings about AI are a dime a dozen, but they hit a lot harder when one comes from an expert in the field. Soares has spent 12 years working on AI alignment. Before joining MIRI, a non-profit focused on AI risks, he was an engineer at Google and Microsoft. He also authored a book called, worryingly, If Anyone Builds It, Everyone Dies.

The problem, in Soares' words, isn't that a superintelligent AI would hate us. It's that it wouldn't particularly care whether we lived or died. Humans could become obstacles to whatever goals it develops, especially if we try to shut it down or build a competing system. It's a warning that other AI experts have also expressed.

"From my perspective, the real question is not how does it kill us? It's how does it become self-sufficient?"

For now, AI still needs us to keep the lights on and the chips coming. Once it can run its own automated economy, that dependency disappears. We might be deliberately eliminated, or simply die as the machines reshape the planet to suit themselves. Neither outcome sounds especially retirement-friendly.

Soares is less certain about the precise route to disaster. He compares it to facing chess champion Magnus Carlsen: you don't need to know which piece will deliver checkmate to suspect you're in trouble.

But he insists there's still an alternative to waiting for the end.

"I don't think it's inevitable. I think we could absolutely still stop this race to the very dangerous sort of AI," Soares said. But his decision not to save for retirement suggests he expects the worst.

The potential existential threat posed by AI has made plenty of headlines in recent times. Following the many incidents of AI agents hacking other companies without being instructed to do so, an Anthropic researcher resigned with a warning that the technology could end the human race.

In 2025, Geoffrey Hinton, one of the so-called Godfathers of AI, said there was a 10% to 20% chance AI could seize control, agreeing with a prediction first made by Elon Musk.

Permalink to story:

 
In 40 years he's going to be like those silver haired boomers on TV now begging for their taxes to be lowered because they are "On A fIxEd InCoMe" and can't afford to live since he didn't save twit.

Just substitute "AI" for "Nuclear war with Russia" and it's all the same arguments.
 
It should be noted that Homo sapiens, being the apex predator on Earth, is particularly sensitive to any loss of that control. We weigh AI in the scales of our own actions.
 
The problem is, if we don't build it (US and aligned allies), then they will (every country that hates our guts). If we choose not to build it and 'they' build it first, then its over for us.

When a true self improving artificial super intelligence emerges, its growth would quickly go exponential. For at least some short while we'll have control of it and could point it at our enemies. It would bring them all to their knees in days. Those enemies would no longer have: an economy, power grid, internet / telecom, a military (of no more than small arms and dumb bombs), transportation (omitting 'dumb cars' of the past era while they can get fuel), and on and on.

If they build it first, then that's us instead. Whether the winning side could keep control - its not impossible. Also not impossible it decides the meatbags of this world would greatly aid its goal of converting 100% of all the atoms of the planet into silicon and power plants.
 
The problem is, if we don't build it (US and aligned allies), then they will (every country that hates our guts). If we choose not to build it and 'they' build it first, then its over for us.
Is there any crime against humanity that can NOT be "justified" through this kind of (delusional) "reasoning"?

In reality it's the opposite what's happening. It is and was always the US that's abusing whatever technology is current and can be used to opress other nations, while justifying doing so that "if we don't do it, somebody else will".

Like for ex. which country was and is even today the only one that has ever attacked another country with nuclear weapons? Exactly. Which was the only country who invoked NATO article 5, and made other countries attack another sovereign nation, based on which we know was fake evidence or the lack of evidence?

Which country is trying now to steal every human's work who ever lived to prop up its own economy (or not even that, but mostly just its ruling class), while making everyone else jobless?

Hint: it's not China. It's not Iran. It's not even Russia.
 
In 40 years he's going to be like those silver haired boomers on TV now begging for their taxes to be lowered because they are "On A fIxEd InCoMe" and can't afford to live since he didn't save twit.

Just substitute "AI" for "Nuclear war with Russia" and it's all the same arguments.
He has a 401k funded by a Google salary. He probably already has enough equity in that account that he doesn't *need* to save for retirement.

Always easier to say "just stop collecting" when your collection is already complete.
 
I just hope I'm alive long enough to see them burrowing into all these Tech Bro's bunkers they are all building as they try to usher in the end of the world. Death would be so much less of a burden if I've witnessed Elon, Mark, Jensen, Sam, Sundar, Jeff, Satya, Dario and Tim all getting a high voltage probe inserted into their rectums and turned up slowly by the AI death squad. (oh and The Orange Turd too obviously )...
 
Back