Nvidia announces an AI-driven green screen for streamers

[Cal Jeffrey]

Bottom line: With TwitchCon only a day away, Nvidia has announced the RTX Broadcast Engine. The new RTX-accelerated SDKs utilize the RTX GPU’s "dedicated AI processors called Tensor Cores" to add various effects to live streams.

The most exciting and useful of these for the serious streamer is RTX Greenscreen. Chroma key compositing (colloquially Green Screen) is the method of filming a subject on a green backdrop and then substituting the background with another image. This is how many streamers insert themselves into other content like gameplay.

However, using chroma keying can be a hassle. The background screen takes up a lot of studio space. The lighting must be just right, and the streamer has to avoid wearing anything green for the effect to work.

Nvidia notes, RTX Greenscreen uses artificial intelligence to identify the part of the image that is human and then remove the background. It works in real-time and appears to be just as effective as the traditional method. Sufficient lighting is probably still essential.

A second SDK called RTX AR detects the user’s face. It can track facial features and expressions, even creating a model on which other 3D effects can be applied and changed in real-time. One application would be to use your face to control an animated model much the same way Apple’s Memojis work.

The third application is RTX Style Filters. These filters “use an AI technique called style transfer to transform the look and feel of a webcam feed based on the style of another image.” Streamers can use this to give their feed a sketched look or other strange-looking effects.

Keep in mind that right now, these features are only available in the RTX Broadcast Engine SDKs. However, Nvidia mentioned that is has been working with OBS, a popular live-streaming application developer, on integrating RTX Greenscreen into OBS Studio. It will demo it at TwitchCon this weekend, and should roll out to users in about a month.

The company said Streamlabs and XSplit are also working on adding the effects to their apps. It had no word on when those companies would be ready with updates.

Permalink to story.

https://www.techspot.com/news/82080-nvidia-announces-ai-driven-green-screen-streamers.html

 

[Uncle Al]

It has come such a long way! Back in the day when I was in college the original "blue screen" was introduced and while it worked (usually) it had many, many drawbacks and just didn't last very long. Of course that is also when video tape was 6" wide, all editing was done on the fly and live broadcasts were still the norm rather than the exception and "remote" broadcasts took a small army and several trucks full of equipment.
Green screen was a VAST improvement that truly revolutionized the telecommunications industry and now, bringing all that power that would set a station back a cool $1M can run on a PC ..... makes we wish I was 30 years younger so I'd be around to see what else they come up with!

 

[QuantumPhysics]

I bought all the equipment I'd need to stream games including an Elgato, but:

#1 Windows Game Bar recorder has been less headache.
#2 Streaming to my Youtube is more hassle than simply recording the gameplay and uploading it.

I get high latency (over 100) which makes it difficult to play a multiplayer game. I tried with Quake Champions and went from murdering everyone to getting my asskicked.

I doubt I'd want to put myself on my streams. I'd rather the gameplay be the focus.


I just wish to God I could have recorded half my online sessions.

 

[Teko03]

Can you live stream games with your avatar instead of...the actual you?

 

[ThrakazogZ]

So......they couldn't sell enough of their ridiculously overpriced RTX cards with Ray Tracing and a small performance increase. Why not make a streaming feature that could work on any card in the past few generations, but make it exclusive to the overpriced RTX cards to get those things moving.

 

[texasrattler]

So......they couldn't sell enough of their ridiculously overpriced RTX cards with Ray Tracing and a small performance increase. Why not make a streaming feature that could work on any card in the past few generations, but make it exclusive to the overpriced RTX cards to get those things moving.

It's called marketing, genius. Why would a company focus on older gen tech cards when you have brand new RTX cards you want to push. Think about that as you run a company.
There are other software that can work with other/all cards, this one is designed by nvidia for their latest cards. In time if they choose to, they may open to all, may not. For now, it's RTX.

Also they are working with Streamlabs which is going to be interesting as that company is getting ready to be bought by Logitech.

 

[Markoni35]

So......they couldn't sell enough of their ridiculously overpriced RTX cards with Ray Tracing and a small performance increase. Why not make a streaming feature that could work on any card in the past few generations, but make it exclusive to the overpriced RTX cards to get those things moving.

Exactly so. I haven't seen anything revolutionary here that other cards couldn't do. You can record the background behind you and then if you put anything else in front it of it, proper software (even without using a GPU) can make a diff between the background and the new object.

But okay, there's plenty of people who don't know that. It will be good for sales. Economy is always based on the ignorance of customers.

 

[braybray]

So......they couldn't sell enough of their ridiculously overpriced RTX cards with Ray Tracing and a small performance increase. Why not make a streaming feature that could work on any card in the past few generations, but make it exclusive to the overpriced RTX cards to get those things moving.

Exactly so. I haven't seen anything revolutionary here that other cards couldn't do. You can record the background behind you and then if you put anything else in front it of it, proper software (even without using a GPU) can make a diff between the background and the new object.

But okay, there's plenty of people who don't know that. It will be good for sales. Economy is always based on the ignorance of customers.

Tell me if any company has done this successfully?
Logitech has tried it, but failed . just be greatful that the AI has came to give us some use...