Nvidia shows what games could be like with fully AI-controlled NPCs

[Cal Jeffrey]

In context: Monday kicked off 2024's Game Developer's Conference, opening with companies showcasing their AI-driven game creations. Unity teased a collaboration with a conversational AI startup, Convai, and Nvidia announced a slew of projects that utilize its MetaHuman and other AI technologies.

During his GDC 2024 keynote address, Nvidia CEO Jensen Huang played a brief trailer/explainer of Covert Protocol, a collaborative tech demo with AI SDK developer Inworld AI. Covert Protocol is a code name for an AI-enhanced development platform that Inworld is building. It utilizes Unreal Engine 5 and leverages several of Nvidia's AI technologies, including Riva Automatic Speech Recognition and Audio2Face, to provide realistic characters that can speak, react, and interact with their environment in real time.

Covert Protocol is similar to Convair/Unity's Project Neural Nexus tech demo, also featured at GDC. The main difference is that Covert Protocol looks much better than Neural Nexus. Graphically, it appears as one would expect from UE5 and Epic's MetaHuman technology, which is stunning. Neural Nexus uses Unity and appears drab and washed out.

The Covert Protocol models look a little more plastic than past MetaHuman Creator demos. However, one must consider that when Epic showcased the highly detailed, nearly photorealistic models, there were no complex backgrounds to chew up cycles. So developers could use full resources to make MetaHuman faces look as detailed as possible. Covert Protocol is closer to what the models can look like in a finished game, which is still pretty good.

The demo's gameplay is more of a conversational mystery adventure rather than the FPS Convai showcased. So we didn't get to see any heavy action, but it did demonstrate bits of conversations, and the facial animations and voice intonation of the AI were much better. The speech still doesn't compare to human voice acting, but it's also not completely devoid of emotion like Convai's models. There is some hints of emotion in there.

While the demonstration was undeniably impressive, it raises valid questions about the future of gaming, just as Project Neural Nexus did. Namely, do gamers truly desire the ability to engage in open-ended and sometimes inane conversations with virtual Chat-GTP avatars? Or will they reject these AI-driven ideas in favor of traditional voice acting and scripted dialogs?

It's hard to say without a finished product. However, we can already see AI fatigue setting in as companies inundate consumers with AI-based product ideas daily. If the current trend continues, AI could soon control everything, from gaming to daily tasks. Games developed without thought and soul poured into them - that rely too heavily on AI mechanics - are likely to annoy gamers rather than convert them to sales.

Permalink to story.

https://www.techspot.com/news/102322-nvidia-reveals-what-games-fully-ai-controlled-npcs.html

 

[return]

Can't wait to shoot their limbs off.

 

[toooooot]

Interesting. but somehow I feel that their intellect will be of the quality those who making this feature.
And if it comes with other software like unity or unreal, it wont be cheap

 

[Eldrach]

I think generative AI will do wonders for open world games where npc’s are usually limited to a very tiny range of responses.
With AI you could go on an actual adventure with some random street sweeper and make the game «your own»

 

[Fearghast]

Well, compared to Ubisoft / Besheda writing, even hallucinating AI can do better.

 

[OortCloud]

Does this require internet access and make external API calls in order to run the AI? If so it's a huge no-no. A whole generation of games which will cease to work when nVidia pulls the plug on the backend.

 

[takaozo]

If this is the only usage they came up with for AI in games, I'm affraid the new games will be oriented to this. No more story driven action.
Just sit and talk with AI... But wait don't we already have virtual assistants already?

 

[Tom Yum]

Starfield showed the danger of relying on procedurally generated content that give players the opportunity to explore 100's of worlds, but no reason to actually want to. Now developers will create NPCs that players can talk to for hours, but again give no reason to do so because none of the NPCs AI-generated responses matter (because developers won't write all the background content and world lore needed to train an AI that could provide a meaningful interactive experience in-game). Meanwhile, developers who actually pay for good writers will continue to make immersive games, with or without AI.

 

[Theinsanegamer]

I think generative AI will do wonders for open world games where npc’s are usually limited to a very tiny range of responses.
With AI you could go on an actual adventure with some random street sweeper and make the game «your own»

If I wanted to talk to a virtual assistant, I would just do that.

The last thing we need is the game industry adopting a "make your own story" philosophy. 99% of these "adventures" would be the NPC walking in a circle until it hallucinates or giving you some RNG fetch quest. Games are supposed to be well crafted experiences with interesting characters, plot, and setting, not some AI slop slathered onto your face.

 

[stewi0001]

Does this require internet access and make external API calls in order to run the AI? If so it's a huge no-no. A whole generation of games which will cease to work when nVidia pulls the plug on the backend.

I was thinking the same thing. I am assuming not based on the video, but who knows. If it not dependent on an external service, then how much additional storage and memory is going to require of a user's PC?

 

[Dr Roboto]

What a bunch of marketing hype and talking heads in that video. The tiny demo was pointless and showed off nothing. The rest of the video was just "trust us" we are an amazing company and our tech is amazing.

 

[wiyosaya]

The "lip sync" is just as crappy with this as it is with the Unity version.

 

[human7]

I think generative AI will do wonders for open world games where npc’s are usually limited to a very tiny range of responses.
With AI you could go on an actual adventure with some random street sweeper and make the game «your own»

Yep. Someone's going to develop a No Man's Sky equivalent for NPC conversations sooner or later. The question is will it feel vast but empty/shallow like the No Man's Sky originally did (or full of buggy/silly conversations), or will it actually be quite good?

 

[wiyosaya]

What a bunch of marketing hype and talking heads in that video. The tiny demo was pointless and showed off nothing. The rest of the video was just "trust us" we are an amazing company and our tech is amazing.

How dare you profane the Marketing Gods.

 

[human7]

I was thinking the same thing. I am assuming not based on the video, but who knows. If it not dependent on an external service, then how much additional storage and memory is going to require of a user's PC?

Unlikely external. But it could be, depends on the developer I guess, and either way Nvidia wins.

The memory question is the interesting one. Will Nvidia fix the memory lineup in the next generation of GPUs? They have strong incentives to do so, but they also have incentives to not give their flagship product too much memory for fear of cannibalizing some of their AI products. Only time will tell. If I were them I'd just put the extra memory on there because the AI products will need to get more memory to stay competitive with AMD, anyways.