AI smarter than people could cause the extinction of the human race, says OpenAI

[midian182]

In brief: A lot of people are concerned about the advancement of AI, especially when it comes to the creation of machines smarter than humans. Even ChatGPT creator OpenAI is aware of the potential dangers of superintelligence, including the extinction of the human race, and has put together a team to mitigate these risks.

OpenAI writes that controlling AI systems much smarter than people requires scientific and technical breakthroughs to steer and control them. To address this issue within four years, it is starting a new team and dedicating 20% of the compute it has secured to this effort.

OpenAI believes superintelligence will be the most impactful technology ever invented and could help solve many of the world's problems. But its vast power might also be dangerous, leading to the disempowerment of humanity or even human extinction. Such an AI might seem a very long way off, but the company believes it could be here sometime this decade.

"Currently, we don't have a solution for steering or controlling a potentially superintelligent AI, and preventing it from going rogue," writes OpenAI co-founder Ilya Sutskever and Jan Leike, the new team's co-head.

As humans won't be able to supervise AI systems much smarter than us, and current alignment techniques will not scale to superintelligence, new breakthroughs are required.

The end result of misaligned AI?

The superalignment team's goal is to build a "human-level automated alignment researcher." Alignment research refers to ensuring AI is aligned with human values and follows human intent. The aim is to train AI systems using human feedback, then train AI that can help evaluate other AI systems, and finally build an AI that can perform alignment research faster and better than humans.

OpenAI admits that solving the technical challenges of superintelligence in four years is an ambitious goal and there's no guarantee it will succeed, but it is optimistic. The company is now hiring researchers and engineers to join the team.

"Superintelligence alignment is fundamentally a machine learning problem, and we think great machine learning experts – even if they're not already working on alignment – will be critical to solving it," explain Sutskever and Leike. "We plan to share the fruits of this effort broadly and view contributing to alignment and safety of non-OpenAI models as an important part of our work."

We've seen plenty of concerning reports about where AI is heading, including one of the 'Godfathers of AI,' Geoffrey Hinton, leaving Google with a warning that as companies take advantage of more powerful AI systems, they're becoming increasingly dangerous.

OpenAI boss Sam Altman was one of several experts who recently warned about the possibility of AI causing the extinction of the human race, comparing these systems to the risks posed by nuclear war and pandemics. Over two-thirds of Americans are worried about it threatening civilization, and Warren Buffett compared AI's creation to the atomic bomb.

Not everyone shares these fears, though. Meta's chief scientist and another one of the three AI Godfathers, Prof Yann LeCun, said that warnings AI is a threat to humanity are "ridiculous."

Permalink to story.

https://www.techspot.com/news/99318-openai-new-team-hopefully-safeguard-humanity-ai-superintelligence.html

 

[Theinsanegamer]

OpenAI also thinks that they should be the body that governs which AI are allowed to exist, and has lobbied to get other AI projects banned.

So excuse me if I dont trust this team as far as I can throw them.

 

[yRaz]

I don't have a problem with this. We're greedy, selfish, gluttons and everyone is gonna die someday anyway

 

[wiyosaya]

I don't have a problem with this. We're greedy, selfish, gluttons and everyone is gonna die someday anyway

Yes, true, we are all going to die someday anyway. However, to actively pursue a Darwin award is just foolish, IMO. If the extinction of the human race is what comes of this, humanity will certainly win the ultimate Darwin award. Honestly, I don't think pursuing humanity's extinction is a wise pursuit.

OpenAI also thinks that they should be the body that governs which AI are allowed to exist, and has lobbied to get other AI projects banned.

Interesting.

So excuse me if I dont trust this team as far as I can throw them.

Someone has to start somewhere in this effort. I think there are others working toward the same goal, so the future of AI may not depend on this effort alone.

As I understand it, AI does come up with crap answers at times (not based on my direct experience with it) and if we are just going to get crap out of it, I have to ask the question - "does it really warrant further development?", or given that it provides crap answers some of the time, is it really "super-intelligent?" I, for one, do not think that in all cases, AI approaches super-intelligence.

Furthermore, I think there are those out there that think AI can fix stupid. I disagree.

 

[mbk34]

"AI smarter than people could ..."
If it's as smart as some of the people I have to deal with then that's pretty low hanging fruit.

 

[dangh]

AI in current form is only a set of algorithms doing only what requested. It doesn't have a will or goal and do not understand a thing. It is just a software tool. The only thing dangerous here are people using this tool.
Remember go ai winning with go master? It lost with go amateur. Not because it wasn't powerful, but because it is not intelligent. If there is no patrern in model computer can't be imaginative and create new move. It simply check huge number of patterns to select one matching best current situation. Ai is only set of if..else, just with some weighted algorithm on top and dynamic if patterns. It is just normal algorithm with a cool name, similarly like we call a normal network connected computer system a cloud.

But ai is a buzz word. Controversies will make money for both for and against supporters. And I'm sure human race will kill itself way before we will be able to come up with real thinking and sentient electronic brain.

 

[hahahanoobs]

People usually wait until it's too late before they act, so if AI is going to do damage to us, it will happen. To what extent? Most likely minimal. We're not the brightest, but we do get lucky often. *fingers crossed*

If AI isn't the next big thing, we're in a lot of trouble, because we're not doing as well as we should be at this point. Mars isn't it. That idea doesn't even make sense. You want to leave a dying planet... to start new life on a dead one? Are we a race that gives up on people that are dying, or do we try our hardest to save them, and then work on solutions to prevent it from happening again? Is it easier to save someone with or without a pulse? The move to Mars idea shouldn't make sense to anyone. We need AI. NOW. lol

 

[yRaz]

Honestly, I don't think pursuing humanity's extinction is a wise pursuit.

What do you mean? We've been doing that for basically our entire civilized existence.

 

[pencea]

Looking forward to Skynet or The Matrix taking over.

 

[Theinsanegamer]

Yes, true, we are all going to die someday anyway. However, to actively pursue a Darwin award is just foolish, IMO. If the extinction of the human race is what comes of this, humanity will certainly win the ultimate Darwin award. Honestly, I don't think pursuing humanity's extinction is a wise pursuit.

Interesting.


Someone has to start somewhere in this effort. I think there are others working toward the same goal, so the future of AI may not depend on this effort alone.

As I understand it, AI does come up with crap answers at times (not based on my direct experience with it) and if we are just going to get crap out of it, I have to ask the question - "does it really warrant further development?", or given that it provides crap answers some of the time, is it really "super-intelligent?" I, for one, do not think that in all cases, AI approaches super-intelligence.

Furthermore, I think there are those out there that think AI can fix stupid. I disagree.

You know, there's this saying about "fool me once, shame on you, fool me twice, shame on me". When it comes to "companies/industries regulating themselves" I believe we are at "fool me 96 thousand times". And yet there are still people who look at a company claiming they are concerned about regulating their own industry and think "maybe the 96001st time it will work!"

 

[p51d007]

Remember the scene in the movie "Airplane" when the guy pulled the cord, causing the runway lights to go out and then he said just kidding. Simply make it where you can cut the power, or start the fire sprinkler to short it all out.
We are already at "skynet beta".

 

[m3diapers]

A smart AI will not cause the extinction of humans; instead, it will offer jobs with higher pay and better benefits. Most of us will be pleased to work for it. The richest 1% will likely face extinction.

 

[sailsperson]

AI in current form is only a set of algorithms doing only what requested. It doesn't have a will or goal and do not understand a thing. It is just a software tool. The only thing dangerous here are people using this tool.
Remember go ai winning with go master? It lost with go amateur. Not because it wasn't powerful, but because it is not intelligent. If there is no patrern in model computer can't be imaginative and create new move. It simply check huge number of patterns to select one matching best current situation. Ai is only set of if..else, just with some weighted algorithm on top and dynamic if patterns. It is just normal algorithm with a cool name, similarly like we call a normal network connected computer system a cloud.

But ai is a buzz word. Controversies will make money for both for and against supporters. And I'm sure human race will kill itself way before we will be able to come up with real thinking and sentient electronic brain.

Exactly this. The AI hysteria has been pushed through for the past couple of years by - surprise - either those directly benefit from it (I.e. OpenAI, the creators of ChatGPT) or those who hope to invest some money for mad returns just like it happened with the Theranos when a bunch of moneybags, who knew nothing about the subject, fell for the buzzwords and "the potential impact", pumped money into something they probably thought they understood, only for it all to flop.

 

[BuckarooBonzaii]

It's only dangerous when overbearing Governments and greedy Companies put their hands into it. A Governing Body by OpenAI only means Government Control. Not too hard to see that OpenAI is positioning themselves.

 

[hwertz]

I mean, if you have a AI with superhuman intelligence, how are the superalignment team going to be sure of anything? A human psychopath can figure out what responses are expected to pointed questions, so I don't see why a highly intelligent "rogue" AI would not be able to do this too.

I can say, ChatGPT (for instance), I ran one of those miniature models (the 4GB one) locally, and it does not run any CPU time when it's not actively answering a question. Knowing that it's not just sitting there burning cycles day and night did help allay my concerns there. (It also gets reset after each session.) But these ones that are getting constantly updated with new information, that are running all the time? I do have concerns about this.

I am curious to see if it's possible to develop effectively an "AI MRI". You have these neural networks where each layer has a high number of connections to the previous and next layer, but during training some links can become inactive (an "activation function" decides which links are more active and which less as the network trains. A problem with some activation algorithms has been having a large percentage of the neurons essentially being unused because the activation value drops to 0... so you could have an equivalent result with a much smaller neural network without so many inactive neurons.)

So, does an operating neural network just end up using apparently "random" connections from one layer to the next as it generates text or images or whatever, or do structures end up being apparent if you run the network and display what parts of the neural network are actually being used? Can this be used on future neural networks to determine if they are being dishonest, have ulterior motives, or concealing or witholding information. Furthermore, if you think your AI has reached a level of being truly intelligent, is it ethical to even do so? It would be considered unethical to permanently strap a human to a lie detector and try to use MRIs for anything beyond medical diagnosis (I.e. trying to read their thoughts and so on.)

That said -- OpenAI's warnings are bit disingenous. I've seen statements that amount to "We're worried about AI getting out of control and too powerful, it should be tightly regulated so not just anybody can develop more powerful AI systems. But we at OpenAI should be able to." This does make me wonder how much is genuine concern, and how much is effectively FUD to hope they can reduce competition and put themselves in the prime position in AI development.

 

[cuerdc]

I would trust a super intelligent A.I with its risk of extinction over the human leaders and governments of the world with their risk of extinction.

 

[kinetix]

I mean, if you have a AI with superhuman intelligence, how are the superalignment team going to be sure of anything? A human psychopath can figure out what responses are expected to pointed questions, so I don't see why a highly intelligent "rogue" AI would not be able to do this too.

I can say, ChatGPT (for instance), I ran one of those miniature models (the 4GB one) locally, and it does not run any CPU time when it's not actively answering a question. Knowing that it's not just sitting there burning cycles day and night did help allay my concerns there. (It also gets reset after each session.) But these ones that are getting constantly updated with new information, that are running all the time? I do have concerns about this.

I am curious to see if it's possible to develop effectively an "AI MRI". You have these neural networks where each layer has a high number of connections to the previous and next layer, but during training some links can become inactive (an "activation function" decides which links are more active and which less as the network trains. A problem with some activation algorithms has been having a large percentage of the neurons essentially being unused because the activation value drops to 0... so you could have an equivalent result with a much smaller neural network without so many inactive neurons.)

So, does an operating neural network just end up using apparently "random" connections from one layer to the next as it generates text or images or whatever, or do structures end up being apparent if you run the network and display what parts of the neural network are actually being used? Can this be used on future neural networks to determine if they are being dishonest, have ulterior motives, or concealing or witholding information. Furthermore, if you think your AI has reached a level of being truly intelligent, is it ethical to even do so? It would be considered unethical to permanently strap a human to a lie detector and try to use MRIs for anything beyond medical diagnosis (I.e. trying to read their thoughts and so on.)

That said -- OpenAI's warnings are bit disingenous. I've seen statements that amount to "We're worried about AI getting out of control and too powerful, it should be tightly regulated so not just anybody can develop more powerful AI systems. But we at OpenAI should be able to." This does make me wonder how much is genuine concern, and how much is effectively FUD to hope they can reduce competition and put themselves in the prime position in AI development.

local ChatGPT models?
I thought that the only ones that had leaked were the LlaMa from Facebook, in addition to some open source published by some research group (I don't remember their origin)
I'm still looking to try them, the ones form FB. I have all the code but due to laziness I haven't downloaded and prepared the models yet. given the amount of RAM in my PC I think I can try up to the 3rd in size.

about "inactive neurons".
There are really many networks where they really learn in their final parts, where they "mix" the features that they have learned in the previous layers, layers that what they have learned are filter weights to extract features. And yes, many of these weights, and those of the final interconnections, can be zero or with values so small that their contribution is negligible and they can also be set to zero, and all be eliminated, which reduces the computation needs, memory memory. archive. And for this, network optimization methods are applied, ranging from pruning to merging layers and connections. Since a few months ago I am training some networks where I have to apply strong regularization and data augmentation, because there is a lot of similarity in the data between the different classes. I am forcing them to learn what is really important. That´s why I don't think I can do many more optimizations to the networks, they doesn't have much room left to "cut" dead weight, besides that they are small, highly optimized networks.

Anyway, even having "functional AI MRI" (which to a certain extent is available and applied) there are many things that end up being "black boxes", how and why they work are diluted and are no longer understood, and fear is that it ceases to be able to control to a fine degree.

Speaking of network optimizations, I remember the creators of Stable Diffusion said they believed their model could be optimized very, very well, even reduce it to a lot less than 1GB, but if they have, they haven't published it yet. Samsung managed to run SD on a mobile (using the NPU), with acceptable performance considering the platform, but I think it was mostly based on aggressive quantization.

 

[ET3D]

Alignment research refers to ensuring AI is aligned with human values and follows human intent.

"Human values" is rather nebulous. The values of the people working at OpenAI don't necessarily match the values of all people in the US, let alone values in China or Iran.

I'm not worrying at all about a superintelligent AI going rogue. I'm very worried about a superintelligent AI doing exactly what people want it to do.

 

[MSIGamer]

Hahah, another 'this will happen in the next 10 years' statement. People are so hilariously scared of silly things.

 

[Wye43]

The steam machines will take our jobs!
The only people who should be concerned with AI are those with lower intellect than a tomato. That means a few billions people.