Why it matters: Google Research is working on new AI image noise reduction technology that could drastically change low-light photography. The tool is capable of reconstructing a dark scene with powerful denoising and minimal artifacts, comfortably outperforming existing denoise tools.
Computational photography has come a long way, and nowadays it is prevalent in smartphones and post-processing software. Noise reduction is arguably one of the most valued tools, as even the best camera sensors are not exempt from image noise, especially when used in a darker environment. Google Research unveiled an exciting new technology that uses artificial intelligence to eliminate image noise from darker scenes, effectively allowing photographers to "see in the dark."
They are calling this new tool RawNeRF and it forms part of their open source project known as MultiNeRF. RawNeRF is capable of assisting photographers when capturing darker scenes specifically. It makes use of AI that has unprecedented denoising power, and what's really impressive is that the denoising seems to happen with a minimal loss in quality and far fewer artifacts than comparable tools.
NeRF (Neural Radiance Fields) is a view synthesizer that can scan a collection of images and reconstruct an accurate 3D render. Ben Mildenhall, a Google researcher, explains that RawNeRF "combines images taken from many different camera viewpoints to jointly denoise and reconstruct a scene." So it's not just a denoiser, but it be used to vary the camera position and view the scene from different angles. Scenes are reconstructed in a linear HDR color space, which means it can work out details like varying the exposure, use tone mapping, and changing the focus.
As shown in the video above, Mildenhall uses a smartphone photo of a candlelit table to demonstrate the power of RawNeRF. He applies minimal post-processing and brightening, and while the resulting picture is more detailed, it has a significant amount of sensor noise. He shows that running the picture through a cutting edge deep denoiser leaves unsightly artifacts, but with RawNeRF the results are truly staggering, especially with reference to the image quality and lack of artifacts. The reason it performs so well is because the AI is trained on raw image data rather than post-processed JPEGs.
It's exciting to think that we could see this technology integrated into our cameras and smartphones very soon, and it'll likely become a game changer for professional photographers and hobbyists. There is no existing noise reduction tool that even comes close to matching these results.