Nvidia's new AI model makes complex 3D geometry from 2D video clips

[Daniel Sims]

Forward-looking: As the ongoing AI boom propels Nvidia into the trillion-dollar club alongside the biggest tech companies on Earth, it continues to unveil new hardware and software tools to leverage AI. One of them could become a new method to facilitate 3D art development for entertainment and enterprise purposes.

Nvidia has unveiled an AI model that can examine 2D video clips of objects and scenes to reproduce them in digital 3D. The technology, called Neuralangelo, could become helpful in game development, industrial digital twins, and robotics.

Suppose a designer uses a smartphone to record a brief video of a truck from multiple angles. In that case, Neuralangelo can select the best frames to build a 3D outline of the vehicle before optimizing a more detailed model. The user can then import the model into a CAD program. The tool can do something similar to an entire structure or interior using video from a drone.

Unlike prior models, Nvidia claims Neuralangelo can capture minute details like glass panels, smooth marble, or roof shingles more accurately. The company's whitepaper on the method shows Neuralangelo's higher level of granularity compared to competing models like NeuS and NeuralWarp when reproducing small objects or large buildings.

Click to enlarge

The company will showcase Neuralangelo and almost 30 other research projects on related subjects at the Vancouver Conference on Computer Vision and Pattern Recognition (CVPR) in late June. Neuralangelo is one of the multiple technologies leveraging AI Nvidia has recently presented.

At Computex in Taipei, the company demonstrated a service utilizing generative AI to give video game NPCs a more comprehensive array of dialogue. It can also react dynamically to what players say. Nvidia also unveiled the DGX GH200 supercomputing platform, which will provide powerful chips to help clients like Microsoft, Google, Meta, and SoftBank with AI-based applications.

Despite disappointment from Nvidia's recent consumer GPUs, the company is seeing explosive growth from enterprise hardware. Generative AI models like ChatGPT, which OpenAI built upon thousands of Nvidia graphics cards, have helped propel a boom in the sector, lifting Nvidia's valuation to over $1 trillion, making it the first chip company to join the ranks of Apple, Amazon, and Microsoft.

Permalink to story.

https://www.techspot.com/news/98919-nvidia-new-ai-model-makes-complex-3d-geometry.html

 

[daffy duck]

Why not use structured light to create the 3D image. We developed a device for $50 that could be used to create very detailed 3D images from the 2D video's and this was 12 years ago. The device was super simple and you could use multiple unit's or move a single camera and device around the subject.

With todays AI smarts I can only wonder how good this would be now and what directions we could have taken. Alas or overlords at Canon Japan told us it was garbage and shelved it after wasting 2 years of our life insisting we must do it.

 

[kinetix]

Why not use structured light to create the 3D image. We developed a device for $50 that could be used to create very detailed 3D images from the 2D video's and this was 12 years ago. The device was super simple and you could use multiple unit's or move a single camera and device around the subject.

With todays AI smarts I can only wonder how good this would be now and what directions we could have taken. Alas or overlords at Canon Japan told us it was garbage and shelved it after wasting 2 years of our life insisting we must do it.

certainly. a structured lighting system is very simple and if done well it gives good results. A few years ago I wanted to build a setup of 2-axis structured light + stereo-photography + structure from motion, I tried to integrate the best of each one, but I did not complete it, I ended up dedicating myself to other topics,

 

[NetNomad]

This was possible to do from reality capture for almost 3 years now

 

[Hodor]

NASA might release the Apollo 11 landing in higher quality with some extra behind-the-scene clips. In full 3D, ready for VR.

 

[Dr Roboto]

Structure from motion and Multi-View Stereo techniques have been around for ever (I was using it back in the 2010 era). Microsoft used to have a free website called Photosynth that you could upload pictures to and it returned you a 3d model. I guess nvidia made some enhancements for better detail and automatic image selection from a video. It is neat for sure, but I guess that nvidia is the darling of the tech world now (except for gamers) and anything that has AI in it must be a big deal.