#Facebookdown: How one router issue can break the internet

Menlo Park, California - The internet got a little crazy during #facebookdown and the reason for the outage is both simple and complicated.

Do not unplug anything, otherwise you might just unplug the whole internet.
Do not unplug anything, otherwise you might just unplug the whole internet.  © IMAGO/photothek/xThomasxKoehlerx

Facebook's VP of Infrastructure, Santosh Janardhan, announced that the services were back up after hours of outage for Facebook and all of its products.

According to Janardhan's update, the issue was a configuration change to the routers that Facebook relies on to let the company's data centers communicate with each other.

That communication effectively stopped because of the router settings.

It was like someone unplugged Facebook from the internet, because no one anywhere could access Facebook for six hours.

Even though the service is back and Facebook's data centers are talking to each other again, many are wondering how the biggest social media company in the world could just dip off the internet.

One way to think about the disruption is to imagine a map that tells your information how to go from A to B through the internet.

That map is called a Border Gateway Protocol (BGP) and it sketches out the network of networks that make up the internet. It also figures out what the best destinations and routes are for information that you send, so that your WhatsApp message can get to your friend's phone.

The BGP updates itself, too, based on changes in different networks which can be owned by Internet Service Providers (ISPs), or by companies, such as Facebook.

According to Cloudflare, a network service company, Facebook's BGP changes made it so that the whole network was removed from the global internet. That part of the "map" was completely erased.

Six hours was inconvenient, but BGP issues can get bad

Rows of servers in the Data Hall at the Facebook Data Center in Los Lunas, New Mexico.
Rows of servers in the Data Hall at the Facebook Data Center in Los Lunas, New Mexico.  © IMAGO/ZUMA Wire/xJimxThompsonx

The whole incident is a reminder that the internet is fragile, and everything being interconnected has a flip side.

In their technical breakdown of the map to the internet, Cloudflare explained what can go wrong with BGP updates.

A great example is the internet just breaking in 2004, when a Turkish service provider, TTNet, updated the map for its neighboring networks to say that TTNet was the best destination for all data on the internet.

The BGP updates spread from network to network, and eventually caused the internet's traffic to go to TTNet, which disrupted things so badly that many people had no internet access for a whole day.

Another instance was an internet service provider in Pakistan trying to block the country's users from YouTube in 2008. Because of how it updated its BGP routes, YouTube was down for the whole world for a few hours. Just like in 2004, the updates to one network spread to other networks, which led to all of them blocking access to YouTube.

BGP updates can also be used to cause harm on purpose.

In 2018, bad actors redirected traffic that was meant for Amazon, and used faulty BGP routing to steal $100,000 of cryptocurrency.

BGP is a useful tool, even though it has been purposefully and accidentally misused in the past. It is a self-updating map, and using it to help your information navigate the internet is vital for everyday life.

Facebook's outage on October 4 just goes to show you how complex the internet is. One simple change can be enough to knock out an entire network – and affect billions of people.

Cover photo: IMAGO/photothek/xThomasxKoehlerx

More on the topic Internet: