Historians are rejecting the work of YouTubers that upscale old video footage to 4K

nanoguy

Posts: 585   +8
Staff member
The big picture: YouTubers are using neural networks and other software tools to repair, upscale, and colorize old video footage, so that it can be experienced in a different way on modern hardware. However, academics believe that could open a can of worms as the original footage is changed to a degree that removes the context of how it was captured.

Earlier this year, we saw how a YouTuber was able to achieve remarkable results in upscaling old, low-resolution footage from 1896 to 4K video at 60 frames per second. This raised the question if "Enhance" would soon become more than a television trope, and whether AI tech could be used with all the old video and audio recordings that are public domain.

Since then, Denis Shiryaev's channel has exploded in popularity as he's kept adding more upscaled videos and colorized black and white footage that attempts to recreate the various nuances of the 19th and 20th century. But this is merely a showcase for a larger project, a Poland-based company called Neural Love whose mission is to use neural networks and algorithms to clean up, upscale, stabilize, and colorize historic images and video.

For many people, the resulting content has captured their attention and transported them back through time. However, it turns out not everyone likes the new trend. Some historians of art believe these modernisation efforts are problematic and offer a distorted image of what was actually captured in the original content.

Emily Mark-FitzGerald, who is Associate Professor at University College Dublin’s School of Art History and Cultural Policy, notes that "The problem with colourisation is it leads people to just think about photographs as a kind of uncomplicated window onto the past, and that's not what photographs are."

Neural Love's Elizabeth Peck told Wired the company does explain to its clients the way it approaches "the restoration aspect and the enhancement aspect." For instance, the removal of noise and imperfections is seen by the company as merely the process of "returning the film to its original state."

Academics disagree, but they worry more about how colorization and frame rate smoothing in particular add in details that were never captured on the original film. For Luke McKernan, lead curator of news and moving images at the British Library, these techniques turn out subjective interpretations of the original content, which isn't acceptable as it removes the authenticity and the accuracy of the source footage.

McKernan argues that "colourisation does not bring us closer to the past; it increases the gap between now and then. It does not enable immediacy; it creates difference."

In any case, Neural Love knows the resulting videos are not historically accurate, and they offer the option for clients to add a watermark so that they won't be taken at face value. Whether or not that's enough, only time will tell -- but people scrolling on, say, a Twitter feed might not take the time to search for the original or even realize it is not meant to be historically accurate.

A somewhat similar debate could spark around video game remasters. Take the recently released Crysis Remastered -- it comes with 8K textures, software ray tracing, voxel global illumination and a host of other changes. However, for many it lacks the kind of attention to detail that made the original a great game that's still talked about 13 years later. It still melts video cards, but it was never just about that -- Crysis was a seminal point in the history of video game graphics, and the original is still the definitive version that will offer you a window into that era.

Permalink to story.

 

Uncle Al

Posts: 7,485   +5,992
I would love to see a very complete analysis of what exactly is being "changed" and to what extent. There is something to be said for keeping the original product for it's historical significance, but there is no reason now to have the upgraded product as long as nothing is significantly altered.

We live in a time where there is a great deal of effort to alter history for personal reasons rather than preserve it, analyse it, criticize it when appropriate, but to destroy it based upon biased opinions, regardless of how many, is unfair and unjust. If, for example, it is a statue of questionable value, rather than destroy it, surround it with statues that present "the rest of the story". Children will learn a more complete history this way, and have a meaningful understanding of the evolution of time and how it affects and is affected by those in the present day.
 
  • Like
Reactions: RebelFlag and kimo1

Lew Zealand

Posts: 1,591   +1,624
TechSpot Elite
Those grumpy crusty rusties, aren't they? That's just pure ignorance toward progress.

Later on, nobody is gonna give 2 cent about what they thought, people will just gaze at the footage and admire the restoration ingenuity.
Yeah, people don't care about any sort of accuracy, just how good it looks. Instead of whitewashing history, we can now colorwash it with fake 4K/60. That's why so many gamers love DLSS. Why have real when you can have fake?
 
  • Like
Reactions: mgwerner

SNGX1275

Posts: 10,592   +453
Sorry the so-called upscaled version looks worse. The colorization is woeful and the improved detail is at best miniscule. Should have just left it at B&W.
Yeah, wtf? They went through all that trouble to make the vid happen, and the colorization faded from crappy colorized to black and white at a few hz. However, I did think the enhanced detail was good. But to get back to complaining, the source vid wasn't shot at a constant FPS, likely a hand crank, but the re-imagination was done as if it was a constant, so the movements looked unnatural. If you are gonna fake up an old video, at least fake it enough to make motion natural.
 

ET3D

Posts: 1,717   +363
I haven't seen any discussion of what's ruined by upscaling and colorisation. Emily Mark-FitzGerald's full opinion isn't linked, but I read Luke McKernan's argument, and that's bullshit. How can one argue that a black and white grainy image is more true to life than a recreated image? Was the world black and white then, or people's perception?

A recreated work may be off when showing real colour, but it would still be less off than a black and white video. If, as McKernan says, early film even caught the blacks and whites wrong, then it's even further from 'reality', and a proper recreation (considering the film's characteristics) would be even closer to life by comparison.

It's possible that adding detail that isn't available from the original is problematic, but there's need to detail specific potential problems. Just saying it's not historically accurate is bullshit. B&W isn't historically accurate except for the very narrow field of photography equipment history.
 

Dennis83

Posts: 8   +3
How can one argue that a black and white grainy image is more true to life than a recreated image?
Since this technology is just in its infancy by the looks of it, we should wait and compare again after it has been brought to perfection. More likely they will start to accept that the reinterpretation is more true to life than the original in black and white.
--
Yes it is not original anymore and restoring removes the possibility for the the viewer to make up our personal interpretation and perception of what we see in the original. Maybe that's what bother's then. Maybe that's what they actually mean by "true to life" ?
 
  • Like
Reactions: nanoguy

winjer

Posts: 82   +241
A LOT of "historians" are full of self-important baloney.
Actually, they are right. These old movies don't just show pictures.
The defects, the lack of color, the inconsistent frame rate are all part of the history.
They show the limitations of the technology of the time and are a look into the past just as the images in itself.
Besides, this kind of AI work is based on interpretation from machine learning, not from real detail captured in the original film.
Now, watching these upscale movies are very interesting, but we must be aware of the it's limitations as it represents history.
 

tellmewhy

Posts: 31   +8
If I was that person who had shoot that film I would like to watch it enhanced.

That natural human intention is the force which have lead the innovation which have produce as result the today's better technology.

If the people was been afraid to choose the more enhanced against the less enhanced the humanity would have stuck in the caves.
 

bexwhitt

Posts: 504   +205
The choice of what was filmed in the first place is subjective, making old films watchable and relatable for modern viewing is laudable. Looks like some academics have sticks placed somewhere they shouldn't
 

stewi0001

Posts: 2,439   +1,941
TechSpot Elite
The colorization is kind of neat and it's not like the original footage gets destroyed or anything in the process of making this so why can't we have both? I don't care for the added sound effects though.
 

Gomos

Posts: 32   +59
This up-scaling, colorizing, high-framerate conversion technology is a cool AI exercise, but that's were it should end. There is no need to make thousands of these fake videos. I personally find them very unpleasant to watch.

A company with the purpose of doing this is just ridiculous.
 

Damocles

Posts: 39   +32
I like the enhancements. For those who want to watch the original scratched, jerky, monochrome originals, that's fine. Let them. I think the enhancements lend a greater air of realism more than the originals could ever achieve and, in my opinion, they are a great improvement.