Google data center hit by four successive lightning strikes, some data permanently lostBy Shawn Knight 14 comments
There's an old idiom that suggests lighting never strikes the same place twice. Apparently it's one that Google never queried as one of its data centers in Belgium recently lost data following four successive lightning strikes courtesy of Mother Nature.
The incident in question took place last week, causing a brief loss of power to storage systems that host capacity for Google Compute Engine (GCE) instances for the region. The search giant said in an incident report that automatic auxiliary systems quickly restored power and although the systems were also on battery backup, some storage systems were attached to battery backups that had experienced extended or repeated battery drain.
Google said manual intervention was required to restore the systems to their normal state and unfortunately, a small amount of data was permanently lost.
The company said it takes full responsibility for the data loss and is using the incident to remind users that GCE instances and persistent disks within a zone exist in a single data center. As such, they are unavoidably vulnerable to data center-scale disasters such as this.
Google said it is also in the process of upgrading to storage hardware that is less susceptible to the power failure mode that triggered the incident.
The unfortunate episode serves as a reminder to back up your important data in an offline location. It's also not a bad idea to unplug your computer from its power source during a thunderstorm or other dangerous weather event that could induce an electrical surge.
Image courtesy Wikipedia