What just happened? In a classic case of never thinking things can get any worse, under-pressure Facebook and its suite of apps have just had their worst outage since 2019. It took almost six hours for the issues to be resolved, during which time the internet was flooded with memes, and Mark Zuckerberg lost over $6 billion.
The problems started on Monday just before noon ET when people began reporting that they couldn't access Facebook, Instagram, Messenger, or WhatsApp—even the Oculus app and website were impacted. It would be almost six hours before regular service resumed.
When Instagram & Facebook are down. pic.twitter.com/mVFlVOOCOC— Netflix (@netflix) October 4, 2021
The problems didn't only affect Facebook's apps. The company said it also impacted many of its internal tools and communication systems. There were reports of employees unable to get into Facebook's offices because their passcards weren't working. Several told The Verge that they had to resort to using their Outlook email accounts to communicate with colleagues. Those who tried to log into work tools such as Google Docs using their email addresses also found they were locked out. The problems meant diagnosing and fixing the problem took even longer.
Congratulations, you made it to the next round pic.twitter.com/hviCZfG8q9— drey (@Dreymwangi) October 4, 2021
Facebook has now apologized for the outage and explained what went wrong. It says a configuration change to the backbone routers that coordinate network traffic between the company's data centers was to blame. Cloudflare senior vice president Dane Knecht tweeted that Facebook's border gateway protocol (BGP) routes, which find the best path to deliver traffic, were suddenly "withdrawn from the internet."
Security reporter Brian Krebs described it as "a routine BGP update gone wrong."
"By not having BGP announcements for your DNS name servers, DNS falls apart = nobody can find you on the internet. Same with WhatsApp btw. Facebook have basically deplatformed themselves from their own platform," tweeted Kevin Beaumont, Microsoft's former Head of Security Operations Centre.
Facebook has apologized for the outage and said there is no evidence user data was impacted.
The outage couldn't have come at a worse time for Facebook. It is currently dealing with the fallout from a Wall Street Journal report that revealed the company knows how damaging Instagram is to the mental health of teenage girls. There's also the whistleblower set to testify to US senators that Facebook prioritizes making money over doing what is good for the public.
According to Bloomberg, the (latest) bad publicity and outage led to a Facebook stock selloff that dropped its price 4.9% on Monday, wiping more than $6 billion off Mark Zuckerberg's personal wealth and sending him below fourth-place Bill Gates on the Billionaires Index.