Nvidia is using deep learning to reconstruct missing parts of photos (and it's very accurate)

[Greg S]

Retouching images is often synonymous with Adobe Photoshop but Nvidia holds the hardware required for advanced manipulation techniques. Using an enhanced deep learning technique, "image inpainting" allows for sections of images to be removed and filled with realistic software-generated infill.

Although Photoshop's Content-Aware Fill feature that appeared back in CS5 may come to mind, Nvidia's research project goes well beyond sampling surrounding areas and edge detection. A neural network was built and trained using over 55,000 images containing random holes and streaks with an additional 25,000 images generated to test its abilities.

Six different categories of masks were created with random patterns to help improve the accuracy of image reconstruction efforts. Tesla V100 GPUs were used with PyTorch deep learning framework to accelerate the neural network training process.

Previous attempts at image inpainting are at a disadvantage compared to Nvidia's new methodology because there is a strong dependency on the input to algorithms that calculate the values of missing pixels. This dependency causes artifacts and blurriness. To remedy the problem, Nvidia researchers use a partial convolution layer that repeatedly checks for how well the output pixels match the receiving area.

Another key advantage of Nvidia's solution is that holes needing to be filled can be any shape. Traditional attempts at image reconstruction have struggled to work with non-rectangular data sets. “To the best of our knowledge, we are the first to demonstrate the efficacy of deep learning image inpainting models on irregularly shaped holes,” said an Nvidia researcher.

The research team will be looking at how partial convolution techniques can be applied to super-resolution applications. As 4K and higher resolutions are slowly becoming more affordable for the masses, the ability to upscale lower resolution content will become even more important.

Permalink to story.

https://www.techspot.com/news/74316-nvidia-using-deep-learning-reconstruct-missing-parts-photos.html

 

[stewi0001]

Based on the video (and its current training), it looks like it does a good job except with faces. The old man was looking better until they did his eyes! lol

 

[VitalyT]

A good approximation that creates a deceptive sense of accuracy that's not really there, which becomes obvious when applied to lost/missing text.

 

[Amet Monegro]

More marketing than reality xD


greetings

 

[antiproduct]

So... how long till people start using this to create nudes? Hopefully they planned that part out and they'll just create a bathing suit for them.

 

[JaredTheDragon]

If those results are a "success", I'd love to see what a "failure" would have been.

 

[Evernessince]

A good approximation that creates a deceptive sense of accuracy that's not really there, which becomes obvious when applied to lost/missing text.

Essentially. All it's doing is guessing what should be there based on other pictures it's "seen". I wouldn't want this used in a courtroom.

 

[JaredTheDragon]

Essentially. All it's doing is guessing what should be there based on other pictures it's "seen". I wouldn't want this used in a courtroom.

Let's keep in mind that calling this "AI" at all is a misnomer. It's artificial stupid, at best. Any decent attorney would rip something like this to shreds in court, as the margin of error alone is beyond belief.

 

[trgz]

Based on the video (and its current training), it looks like it does a good job except with faces. The old man was looking better until they did his eyes! lol

It seemed to me that he ended up with the same eyes as the girl did.

 

[Solar Flair]

"The research team will be looking at how partial convolution techniques can be applied to super-resolution applications. As 4K and higher resolutions are slowly becoming more affordable for the masses, the ability to upscale lower resolution content will become even more important."

You are telling me I should keep my low-res happy hour video collections?
Good...

 

[wiyosaya]

This is nothing new. Over 10-years ago, I heard of a research project that could coax real data out of the noise in an image and it was said to be highly successful.

The difference in this case is that the data is, apparently, completely missing and that the system has to look for other images to compare against. I have to ask what about the woman? Did it simply find another image of her in the public domain and use that to reconstruct the final image? To a certain extent, this does not seem all that impressive.

 

[pmshah]

A good approximation that creates a deceptive sense of accuracy that's not really there, which becomes obvious when applied to lost/missing text.

Essentially. All it's doing is guessing what should be there based on other pictures it's "seen". I wouldn't want this used in a courtroom.

This may be the chance for Polaroid to make a comeback. That probably is the only imaging technology which cannot be manipulated.

 

[Knot Schure]

Wow, lots of negative comments.

I was highly impressed. It made more sense after watching the video, but it was an insight into what we can expect of computers in the near future.

 

[pmshah]

Wow, lots of negative comments.

I was highly impressed. It made more sense after watching the video, but it was an insight into what we can expect of computers in the near future.

That is the scary part. This AI and machine learning thingy can create audio files which are 100 identical to real life people. Now the NSA can put any kind of words in anyone's mouth and just lock them up in Guantanamo. All because some oil executive or a pharma CEO did not like his face!!