Power outage causes Toshiba and Western Digital to lose 6 exabytes of NAND

[Shawn Knight]

Bottom line: Given the current oversupply of the NAND industry, it seems like the market is in a decent position to buffer the shutdown versus what happened to hard drive production back in 2011 thanks to devastating flooding in Thailand. Heck, it might even turn out to be a "good accident" in terms of stabilizing the market.

Western Digital recently announced that an unexpected power outage in the Yokkaichi region of Japan affected flash fabrication facilities operated by joint venture partner Toshiba Memory Corporation. The outage, which occurred on June 15, impacted both the facilities where NAND is made and the tools used to make the storage.

The main facility hasn’t been fully operational since the outage and won’t resume full production until sometime in mid-July according to Reuters.

Western Digital said it is continuing to assess the impact of the outage but expects flash wafer production to be cut by approximately 6 exabytes, most of which will be contained in the first quarter of fiscal year 2020.

There are roughly 1,000 terabytes in a petabyte and 1,000 petabytes in an exabyte, so yeah, we’re dealing with a lot of wasted storage. According to ExtremeTech, 6EB represents roughly half of Western Digital’s quarterly NAND production.

The bigger question has to do with how this will affect overall global NAND production. ExtremeTech said the Yokkaichi Operations facility where the incident took place turns out around 35 percent of the world’s NAND.

Masthead credit: NAND flash and circuit board by KenSoftTH

Permalink to story.

https://www.techspot.com/news/80746-power-outage-causes-toshiba-western-digital-lose-6.html

 

[Morris Minor]

Yeah we can't let the price keep dropping!!!

 

[fps4ever]

Very convenient. It's not like companies with that kind of electrical need and delicate infrastructure spending billions on those factories would have any sort of back electrical systems for such occasions.

 

[JimEs]

Welp guys, buy a SSD before the end of price drops.

 

[Uncle Al]

Woops

 

[antiproduct]

So.... they have these expensive machines that need power to run and a power outage could really mess things up... and they just plug it into the wall and hope for the best? No uninterruptible power supply crossed anyone's mind?

Yeah, seems like something they'd do to reduce supply so they could raise the price.

 

[Burty117]

Fabricated story to bring prices up right before they get some large orders in from Microsoft or Sony (PS5 / Project Scarlett). Seems far to convenient and its been mentioned above, these facilities have backup power to stop stuff like this happening.

 

[Vrmithrax]

CEO: "So, you're saying that if we 'accidentally' blow out the power grid, we'll eat a bunch of wasted wafers in production, but that could cut supply and stabilize our prices without making us look like greedy devils?"

Technician: "Well, I suppose that's one possible outcome if that happ..."

CEO: Shoves tech out of the way, and starts throwing toasters into bathtubs while giggling maniacally.

 

[Evernessince]

Doesn't make any sense, why didn't they have extensive backup power systems in place?

 

[Puiu]

Doesn't make any sense, why didn't they have extensive backup power systems in place?

Unless they have a massive power plant just as a backup, you won't be able to avoid a big an power outage from stopping the production and ruining everything that was in the machines that require constant power.

 

[Burty117]

Unless they have a massive power plant just as a backup, you won't be able to avoid a big an power outage from stopping the production and ruining everything that was in the machines that require constant power.

I've been to places like this, they have a room or two full of battery's that can run the place for just long enough (usually around 15 minutes) for the generators to kick in, they are massive things that can usually run for a day or two depending on what Diesel reserves they have and even then, they normally have a contract with a local Diesel supplier to get trucks there within hours to replenish stock.

 

[ThrakazogZ]

Every other comment here is pretty spot on. There's a NAND factory "accident" a minimum of once a year. It also "magically" rotates through the list of NAND suppliers, so that it doesn't look too suspicious. Amazing how it's never (or maybe rarely.....it's tough to keep count of all the NAND factory accidents) the same major manufacturer twice in a row.

EDIT #1-
In 2017, Toshiba had to suspend NAND production due to viruses on all their machines.............

I forget which company had the 2018 "accident".

EDIT #2-
Samsung had the NAND factory accident in 2018, due to.........wait for it........power outage.

 

[fps4ever]

Unless they have a massive power plant just as a backup, you won't be able to avoid a big an power outage from stopping the production and ruining everything that was in the machines that require constant power.

That is simply not true...whatsoever. They keep huge data centers operating just fine with backup power via outages. Ask Google, MS, etc...

 

[PEnnn]

"Heck, it might even turn out to be a "good accident" in terms of stabilizing the market."

And there you have it!
PS: That poor ba$tard who caused this huge problem is getting a huge raise.

 

[Kashim]

Can someone explain to me how a power outage can take a massive production line like this offline for months? Sounds pretty fishy to me. This convenient "accident" happens right when SSD prices are cratering. Hmm.

 

[Puiu]

That is simply not true...whatsoever. They keep huge data centers operating just fine with backup power via outages. Ask Google, MS, etc...

Data centers have redundancies, factories don't. And we did get data centers problems in the past, like how in early 2018 AWS had some issues with some servers in the US.

From an article written by smart-energy.com
"
According to a study conducted by Uptime Institute, global data center outages have increased by 6% since 2017.

• Power failures have been the primary cause of the outages and the cost per outage has risen by 50% from 2010 to 2016
• The cost of unplanned outages has risen to of $8,851 per minute
• 33% of the survey participants experienced outages due to power failures
• Respondents who experienced IT downtime increased from 25% in 2017 to 31% in 2018
• The average duration of an outage ranged between 95 and 97 minutes
• The US recorded 3,526 outages which impacted 36.7 million people in 2017

"

 

[fps4ever]

Data centers have redundancies, factories don't. And we did get data centers problems in the past, like how in early 2018 AWS had some issues with some servers in the US.

From an article written by smart-energy.com
"
According to a study conducted by Uptime Institute, global data center outages have increased by 6% since 2017.

• Power failures have been the primary cause of the outages and the cost per outage has risen by 50% from 2010 to 2016
• The cost of unplanned outages has risen to of $8,851 per minute
• 33% of the survey participants experienced outages due to power failures
• Respondents who experienced IT downtime increased from 25% in 2017 to 31% in 2018
• The average duration of an outage ranged between 95 and 97 minutes
• The US recorded 3,526 outages which impacted 36.7 million people in 2017

"

You are correct in the fact that I picked some terrible examples due to data redundancy. Here is where you are wrong (mybad for generalizing my comment as well) using general power outages and the average company. Not all power outages cause problems or data loss due to outages. There may be a loss of productivity and or connectivity for a certain amount of time. Yes dollars and hours lost there.

But for an example, I work for a mid size TV/ISP and sitting outside my data room is a diesel generator the size or a semi trailer but a little taller. In the basement we have rows of huge battery's for any such power outages. We can keep phone, Internet and cable TV up indefinitely until power is restored. Switch over is practically instantaneous, I.e., computers, server racks and phone switches stay up during the auto cut-over. We have a few million dollar building and any company that needs/requires constant power will have the same. Especially multi billion dollar engineering facilitates. You also need to remember you don't have to supply power to every part of the plant, just the critical equipment that needs it, that is if you want to save money for backup power. Do you think a company that size could lose that much NAND in a power outage isn't going to take extra special care to NOT have that happen?

Sure anything possible but highly unlikely given the circumstances and timing of such things in these factories. That would be very naive to just chalk it up to a plain old power outage.

PS: Just to add we have 2 or 3 factories in my city, the paper mill that has it's own power substation because you can't let million dollar machines 300 feet long shut down with a power loss. We have a food processing factory that has it's own backup power. You can't just stop processing and let food rot until power is back on, etc....etc.

 

[Markoni35]

Welp guys, buy a SSD before the end of price drops.

Guys I'm so proud of you. More and more of you understand how the world works. As opposed to tons of naive people I see posting here.

 

[Baratom]

Well someone just got promoted.
No doubt about it ...

 

[Markoni35]

Yeah we can't let the price keep dropping!!!

Exactly so. First we made the RAM prohibitively expensive, then we made the GPUs sick expensive, now we're doing the same with SSDs. Not unlike those who burn smaller diamonds to reduce the supply, to keep the prices artificially high.

 

[DelJo63]

We just L O V E conspiracies don't we!