Data waste: According to study, a significant portion of stored server data is never accessed

[Daniel Sims]

Why it matters: Making cloud servers and data centers more efficient is a crucial step in the push to increase sustainability and reduce carbon emissions. However, one company has started drawing attention to what it calls "data wastage" – the retention of large amounts of data no one accesses – and the factors making it difficult to cut back.

Up to 70 or 80 percent of the data that some companies store goes unused, according to enterprise data infrastructure provider NetApp. The company has recently started bringing attention to the issue to help clients meet sustainability goals.

NetApp Chief Technology Evangelist Matt Watts recently told The Register that storage comprises 15 to 20 percent of data center power consumption. Furthermore, a national survey indicates that cloud usage of the UK's power grid could grow from 2.5 percent to six percent by the end of this decade.

NetApp provides data tools for clients like Microsoft, Amazon, and Google. Last April, it published a report on the difficulties of tackling data wastage. Watts initially reported that around 41 percent of stored data is never accessed, but he has since revised the number to as much as twice that in certain areas.

Most leading IT figures believe that cutting back on unwanted data could help reduce carbon emissions, but many companies lack the resources. Some don't have enough time or find differentiating between valuable and unwanted data too daunting.

Obviously, companies that handle clients' data also don't want to cause trouble by deleting information someone might need. Watts highlighted disagreement and confusion in some organizations about whether IT departments are the owners or simply the caretakers of the data they manage, which can present an additional roadblock.

NetApp's BlueXP classification tool, part of a service that gives client companies unified control over diverse data infrastructures, was a significant factor in revealing the scale of data wastage. It retrieved metadata from its data center clients, showing who owned their stored files and when customers last accessed their information.

The issue comes amid NetApp's controversial decision to remove BlueXP's support for services like Google Cloud, Amazon S3, and OneDrive. Watts said that NetApp intends to focus on its internal storage systems instead so the company could stand out.

Permalink to story:

Data waste: According to study, a significant portion of stored server data is never accessed

 

[yRaz]

I wonder how much internet porn contributes to carbon emissions.

 

[NumberNine]

I wonder how much internet porn contributes to carbon emissions.

Online sources estimate 300 million tons, or roughly 1% of all global carbon emissions are directly caused by streaming internet pornography, but this is based on numbers from 2019 - pre machine learning craze - so these figures have likely increased.

You monsters.

 

[umeng2002]

Carbon emissions... oh no...

 

[ScottSoapbox]

Articles handwringing about increased carbon emissions, increase carbon emissions.

 

[McRuff]

How much of that documents that are not accessed are kept for tax or other regulatory reasons. I have to keep 7 years worth of business records that I will never look at unless I get audited.

 

[thehacker]

Oh! How dare you tell me that my 1,084,239 selfies I uploaded to Facebook and 1,093 videos on Instagram are wastes and increasing carbon emission? Wouldn't lizard boy know better than you? huh

 

[Burty117]

I’ve done a fair few SharPoint migrations in my time, or NAS migrations, I file server migrations.

It is incredible how much rubbish companies keep, NetApp are spot on here, and the issue really is the lack of employees who are willing to own any data and label it as “not needed”.

For some reason, some companies think it’s the IT Departments job to delete old stuff, which it can be if agreed on by the board and processes put in place, but generally speaking, especially smaller companies with outsourced IT, they just refuse to own their own data a lot of the time.

 

[FaTaL]

No such thing as wasted data... just waste product people who don't know how to utilize it...

in a world where digital footprint touches almost every facet of life, scrapping data is asinine.

 

[wiak]

Says the company that uses nonstandard hard drives in their storage appliances and makes their appliance a paperweight when you buy it second hand

 

[adamada72]

I’ve done a fair few SharPoint migrations in my time, or NAS migrations, I file server migrations.

It is incredible how much rubbish companies keep, NetApp are spot on here, and the issue really is the lack of employees who are willing to own any data and label it as “not needed”.

For some reason, some companies think it’s the IT Departments job to delete old stuff, which it can be if agreed on by the board and processes put in place, but generally speaking, especially smaller companies with outsourced IT, they just refuse to own their own data a lot of the time.

Having been in the corporate world for some time. You don't get fired for storing too much data. Even if the company has to pay for something it doesn't need by A LOT your job is safe and nobody is putting there job on the line to fix the problem. No department is going to say they have too little of a budget, they always need more and no department knows the truth. It is what it is . Old adage from the 70's 80's, nobody ever got fired for buying intel fits here.

 

[Burty117]

Having been in the corporate world for some time. You don't get fired for storing too much data. Even if the company has to pay for something it doesn't need by A LOT your job is safe and nobody is putting there job on the line to fix the problem. No department is going to say they have too little of a budget, they always need more and no department knows the truth. It is what it is . Old adage from the 70's 80's, nobody ever got fired for buying intel fits here.

Indeed, I have seen companies do something about it, they will keep all data for 7 years, then cold storage backup any data older than 7 years and delete it from hot storage, cold storage they keep for 20 years then bin it.

But that was all agreed with the board of director and is mostly automated, just kept an eye on by IT.

It is possible, just as you say, most companies still run by 70-80’s rules.

 

[ET3D]

What a deliberate obfuscation of actual power consumption figures. In it's original report NetApp said:

"So, how does this data wastage translate for the average U.K. business in terms of energy cost?
Given that the average organization spends a whopping £300,000 each year on storing data, that
equates to £120,000 of wasted investment—£10,000 per month.' (That's based on the ~40% figure.)

Interesting how the goal moved from energy cost to investment. (And even on the cost side, it probably doesn't work that way.)

In the article, according to NetApp's Chief Technology Evangelist, Matt Watts:

"Watts estimates that 15 to 20 percent of power consumed by datacenters is storage, with fluctuations depending on what the hard drives are doing."

Assuming that the data still needs to be accessible, so is on a hard drive, then data that's not used can be on a hard drive that's spun down. That should reduce power use significantly compared to drives which are used. If the drive isn't spun down even if data is never accessed, then NetApp is the one wasting money, not the companies.

My guess is that NetApp is worried more about space and cost of drives than energy cost.