Nvidia imagines a future "DLSS 10" could allow neural networks to render entire game worlds

[Alfonso Maruccia]

In a nutshell: Deep Learning Super Sampling (DLSS) technology popularized the use of neural networks to upscale and enhance real-time 3D graphics, delivering significant improvements in both performance and quality. Competing technologies from AMD (FSR) and Intel (XeSS) have also embraced a similar approach to graphics rendering. However, Nvidia appears to be working on a revolutionary integration of AI algorithms into its RTX-exclusive technology.

Cyberpunk 2077 2.0 recently introduced DLSS 3.5 as the latest iteration of Nvidia's technology, aimed at enhancing performance while maintaining high graphics quality in graphically demanding games. DLSS 3.5 incorporates ray reconstruction using a single AI denoising model and is exclusively available on RTX 40-series GPUs, similar to the previously introduced DLSS 3.0 in September 2022.

Since its initial release in 2019, DLSS has undergone gradual updates, integrating new AI-based upscaling techniques. According to Nvidia's Bryan Catanzaro, the technology will continue to evolve with each new release for the foreseeable future. In a few years, DLSS is expected to become sophisticated enough to handle the entire rendering of a video game world through its neural network-based algorithms.

Catanzaro participated in a recent roundtable focused on DLSS 3.5 and the Cyberpunk 2077 expansion, Phantom Liberty, organized by Digital Foundry. During the discussion, Catanzaro hinted at the potential capabilities of a hypothetical "DLSS 10," suggesting that future iterations of the technology would likely achieve remarkable advancements in real-time graphics.

Nvidia's representative reminded the Digital Foundry panel that the company had already demonstrated real-time rendering of a 3D world using a neural network back in 2018 at that year's NeurIPS conference. In the demonstration, Unreal Engine 4 provided data about the 3D scene and objects to render, while Nvidia's neural network algorithm handled the on-screen graphics rendering.

Catanzaro noted that the final results were relatively basic and certainly "not anything close to Cyberpunk 2077." However, the landscape for generative AI has changed significantly since then. Neural networks have made significant advancements, although AI algorithms remain somewhat unpredictable and prone to generating hallucinations.

Looking ahead, Catanzaro suggested that in a few years, DLSS would evolve into a "completely neural rendering system." Such a system has the potential to revolutionize the traditional rendering approach that relies on raster graphics. Between now and the development of DLSS 10, Nvidia's VP stated that the GeForce-exclusive technology would continue to undergo gradual improvements.

The ultimate goal of DLSS, as Catanzaro explained, is to empower developers to create games that are "more immersive and more beautiful" than what we can currently imagine.

Permalink to story.

https://www.techspot.com/news/100281-nvidia-believes-dlss-10-could-allow-neural-networks.html

 

[Dimitriid]

Curious to read what will the usual Nvidia enthusiasts will say about this one: You still think it's a good idea to praise their innovation when DLSS 10 is only working for the 12090ti that's 5000 USD and there's literally no other Nvidia card sold on that generation?

Because that's their end game and vision of the future.

But perhaps more important: If you think it's fine that Nvidia intentionally segments the PC gaming market with DLSS versions, how would you think any AAA publisher would make a return if the number of PC gamers that can actually run DLSS 10 dropped to like less than 1% of what it is today?

In a world where Nvidia decides nobody should consider any other API at least not without the latest DLSS that can render the entire game on it's own and they continue to be in control of all the hardware that can even run DLSS 10 how could anybody seriously do any business under such extreme conditions?

There is no gaming future in Nvidia and DLSS it's designed to collapse into just an ML product Nvidia sells to power some quick and dirty mobile gacha games remotely.

 

[VcoreLeone]

Since this piece of news cointains no actual information, we're much more interested in you going off on your rants by youself here.

 

[Theinsanegamer]

Curious to read what will the usual Nvidia enthusiasts will say about this one: You still think it's a good idea to praise their innovation when DLSS 10 is only working for the 12090ti that's 5000 USD and there's literally no other Nvidia card sold on that generation?

Because that's their end game and vision of the future.

But perhaps more important: If you think it's fine that Nvidia intentionally segments the PC gaming market with DLSS versions, how would you think any AAA publisher would make a return if the number of PC gamers that can actually run DLSS 10 dropped to like less than 1% of what it is today?

In a world where Nvidia decides nobody should consider any other API at least not without the latest DLSS that can render the entire game on it's own and they continue to be in control of all the hardware that can even run DLSS 10 how could anybody seriously do any business under such extreme conditions?

There is no gaming future in Nvidia and DLSS it's designed to collapse into just an ML product Nvidia sells to power some quick and dirty mobile gacha games remotely.

So in the future AMD and intel make up 99% of GPU and 100% of CPU sales for PCs, with nvidia this weird sudo brand only owned by oddballs?

 

[MSIGamer]

Please no. I have enough of this AI generated garbage. Let game developers be smart again and optimize their games, so we can have crisp graphics the way the artists intended and good performance out of the box. Not a glitch fest that doesn't run without some form of upscaling that basically every triple A game requires these days

 

[yRaz]

although AI algorithms remain somewhat unpredictable and prone to generating hallucinations

that's the real appeal of LSD, though, isn't it? It's also cheaper than gaming! With $100 games that require features from $1500 graphics cards to run and microtransactions out to wazoo I'm starting to think that a healthy drug habit is the way to go.

I play like 5 games on a regular basis and 3 of them are over 20 years old. Today's games are trash. There are maybe 2 releases a year worth playing. I bought BG3 and Starfield. Starfield is trash, 60 hours in I feel like someone mixed Mass Effect 3 with Minecraft while taking too much Adderall. It's not even "fun bad" like other Bethesda games where the bugs are more entertaining than the game itself.

Gaming is dying and I couldn't be happier because the future of gaming hardware is unacceptable. nVidia wants to own all the gaming feature so no one else can sell cards while also not making enough graphics cards for people to buy. AMD is doing an OKAY job at making performance affordable and Intel is basically the retarded step child that people say "oh, that's cute, look at you!" to at holiday dinners then promptly go back to what they were doing.

I'm exhausted with this industry at this point. AMD, nVidia, Developers, it's all nonsense. I can have more fun for less with some LSD and a bag of Doritos. I completely unironically and unsarcastically mean that. Anytime people try to talk about the state of the industry all the fanboys derail the discussion to start fighting with each other.

 

[kiwigraeme]

Still find it funny about people trying to put Pandora back in her box

"I would not use a product with AI generated" etc

You probably have for years without knowing about it

This is a tool - to be used wisely or not

Remember when any Tom, Jane, or kelly could now make a techo track on their PC - and most sounded samey
130 beats per minute or whatever was the flavour of the day - yet electronical music from 60s , 70s could sound kind different with 4 bit thingy mahjigs

All these people ranting - I would 100% prefer judicious AI use, creative use - than asset spamming games we can see now - oh look you are in a new building - slightly bigger , longer than - but same as 33% of all other buildings and those in our last 10 games - we changed the colour and added some motifs - you are in a whole new country

You want diversity , realism done cheaply - you will need AI to help

I want this done independent of Nvidia on an open platform - or at least Epic /Unity etc

 

[torchyear]

In a few years, DLSS is expected to become sophisticated enough to handle the entire rendering of a video game world through its neural network-based algorithms.

Were they talking about local or server based neural networks? Perhaps with cloud game streaming, but I can’t see local compute delivering this kind of hypothetical output in the near future.

 

[Jarenien]

AI is the wave of the future for gaming, and really the only way it will ever continue to improve. The reason games take six years to create isn't because of the code, it's making the 4K textures. Now imagine they need to make 8K, 16K, 32K, etc..., it's going to get ridiculous to try to create something with that much resolution in a timely matter without the use of AI. You will create textures in 720p/1080p, and AI will upscale it. Frame Generation right now is effing amazing (those that say it isn't are delusional and/or jealous because they can't afford it and/or are just plain morons) because it is literally the best thing to happen to computers since the SSD, it's that huge, and the tech will only continue to improve and get better.

 

[yRaz]

jealous because they can't afford it

found the 4090 owner

 

[SalmanIsHere]

Most people commenting here sadly have no idea about Transistor Advancement Limit.
Nvidia figured it out before everyone that we are about to hit the wall.
Which resulted them in inventing Tensor cores in 2017 with Volta.
Now, whole world wants those Tensor Cores.

Amd only got 25% jump from 7nm to 5nm 6950xt to 7900xtx- Techspot - Shader Core/RT Core
Nvidia gpu 75% jump from 8nm aka 10nm to 5nm - 3090 to 4090 - Techspot - Shader Core

But Nvidia also got 200%+ jump vs 3090 thanks to Tensor Cores and Optical Flow Accelarator.

So, Future is AI.

 

[VcoreLeone]

I'd agree with everything that people say about nvidia going in the wrong direction, if only amd or anyone else were able to prove them wrong and build a faster, or at least a matching card, and at the same time prove that without dlss, nvidia would need to build 30-40% faster cards. cause it seems to me that ada still wins on performance, and we'd all be losing a really great feature, that for me allows playing at 4K quality on my 1440p screen.
the real problem is competition lagging behind them, when nvidia can release a ad103 die that's not much more than half (52%) of full ad102 cuda count, and still about match full n31 in rasterization, and blow them to pieces in RT/PT

that said, I don't really like anything that nvidia offers under 4090, and will likely purchase nothing from 40 series, unless there'll be refresh coming with an ad102 4080 Super around 7900xtx price and 90% of 4090 performance.

 

[Jarenien]

found the 4090 owner

Nope, got a 4070 that cost me only $200 after I sold my 3070.

 

[yRaz]

Nope, got a 4070 that cost me only $200 after I sold my 3070.

Either you got a great deal for selling your 3070, a great deal on the 4070 or you're lying about how much you spent

 

[VcoreLeone]

Either you got a great deal for selling your 3070, a great deal on the 4070 or you're lying about how much you spent

sold for 400, bought for 600. I think that's what he means. Coming form 3070, it was just necessary.
I did the same with my 3060ti, sold for 280 eur, bought a used 6800 for 400. Card is great, but I really missed using dldsr+dlss, so when I found a used rtx3080 for 389 I just got it, and now have two really nice cards.

 

[pcnthuziast]

Tfw 240p upscaled to 4k...

 

[CourtJustCourt]

They need to imagine implementing frame generation on all their cards instead of money grab lockouts to push their newer cards. Hope AMD's methods take over the gaming world. OPEN SOURCE LIFE!