Nvidia is using deep learning to reconstruct missing parts of photos (and it's very accurate)

By Greg S · 13 replies
Apr 25, 2018
Post New Reply
  1. Retouching images is often synonymous with Adobe Photoshop but Nvidia holds the hardware required for advanced manipulation techniques. Using an enhanced deep learning technique, "image inpainting" allows for sections of images to be removed and filled with realistic software-generated infill.

    Although Photoshop's Content-Aware Fill feature that appeared back in CS5 may come to mind, Nvidia's research project goes well beyond sampling surrounding areas and edge detection. A neural network was built and trained using over 55,000 images containing random holes and streaks with an additional 25,000 images generated to test its abilities.

    Six different categories of masks were created with random patterns to help improve the accuracy of image reconstruction efforts. Tesla V100 GPUs were used with PyTorch deep learning framework to accelerate the neural network training process.

    Previous attempts at image inpainting are at a disadvantage compared to Nvidia's new methodology because there is a strong dependency on the input to algorithms that calculate the values of missing pixels. This dependency causes artifacts and blurriness. To remedy the problem, Nvidia researchers use a partial convolution layer that repeatedly checks for how well the output pixels match the receiving area.

    Another key advantage of Nvidia's solution is that holes needing to be filled can be any shape. Traditional attempts at image reconstruction have struggled to work with non-rectangular data sets. “To the best of our knowledge, we are the first to demonstrate the efficacy of deep learning image inpainting models on irregularly shaped holes,” said an Nvidia researcher.

    The research team will be looking at how partial convolution techniques can be applied to super-resolution applications. As 4K and higher resolutions are slowly becoming more affordable for the masses, the ability to upscale lower resolution content will become even more important.

    Permalink to story.

     
  2. stewi0001

    stewi0001 TS Evangelist Posts: 1,888   +1,274

    Based on the video (and its current training), it looks like it does a good job except with faces. The old man was looking better until they did his eyes! lol
     
    Reehahs and Amet Monegro like this.
  3. VitalyT

    VitalyT Russ-Puss Posts: 3,938   +2,301

    A good approximation that creates a deceptive sense of accuracy that's not really there, which becomes obvious when applied to lost/missing text.
     
    Evernessince and Amet Monegro like this.
  4. Amet Monegro

    Amet Monegro TS Enthusiast Posts: 71   +21

    More marketing than reality xD


    greetings
     
    trgz and JaredTheDragon like this.
  5. antiproduct

    antiproduct TS Booster Posts: 73   +45

    So... how long till people start using this to create nudes? Hopefully they planned that part out and they'll just create a bathing suit for them.
     
    FPSChris likes this.
  6. JaredTheDragon

    JaredTheDragon TS Guru Posts: 387   +250

    If those results are a "success", I'd love to see what a "failure" would have been.
     
    trgz likes this.
  7. Evernessince

    Evernessince TS Evangelist Posts: 2,788   +1,920

    Essentially. All it's doing is guessing what should be there based on other pictures it's "seen". I wouldn't want this used in a courtroom.
     
    trgz and JaredTheDragon like this.
  8. JaredTheDragon

    JaredTheDragon TS Guru Posts: 387   +250

    Let's keep in mind that calling this "AI" at all is a misnomer. It's artificial stupid, at best. Any decent attorney would rip something like this to shreds in court, as the margin of error alone is beyond belief.
     
    Evernessince likes this.
  9. trgz

    trgz TS Addict Posts: 238   +55

    It seemed to me that he ended up with the same eyes as the girl did.
     
  10. Solar Flair

    Solar Flair TS Enthusiast Posts: 33   +24

    "The research team will be looking at how partial convolution techniques can be applied to super-resolution applications. As 4K and higher resolutions are slowly becoming more affordable for the masses, the ability to upscale lower resolution content will become even more important."

    You are telling me I should keep my low-res happy hour video collections?
    Good...
     
  11. wiyosaya

    wiyosaya TS Evangelist Posts: 2,747   +1,310

    This is nothing new. Over 10-years ago, I heard of a research project that could coax real data out of the noise in an image and it was said to be highly successful.

    The difference in this case is that the data is, apparently, completely missing and that the system has to look for other images to compare against. I have to ask what about the woman? Did it simply find another image of her in the public domain and use that to reconstruct the final image? To a certain extent, this does not seem all that impressive.
     
  12. pmshah

    pmshah TS Booster Posts: 109

    This may be the chance for Polaroid to make a comeback. That probably is the only imaging technology which cannot be manipulated.
     
  13. Knot Schure

    Knot Schure TS Member Posts: 67   +20

    Wow, lots of negative comments.

    I was highly impressed. It made more sense after watching the video, but it was an insight into what we can expect of computers in the near future.
     
  14. pmshah

    pmshah TS Booster Posts: 109

    That is the scary part. This AI and machine learning thingy can create audio files which are 100 identical to real life people. Now the NSA can put any kind of words in anyone's mouth and just lock them up in Guantanamo. All because some oil executive or a pharma CEO did not like his face!!
     

Similar Topics

Add your comment to this article

You need to be a member to leave a comment. Join thousands of tech enthusiasts and participate.
TechSpot Account You may also...