CenturyLink outage led to a 3.5% drop in global web traffic | ZDNet

CenturyLink incident takes down Cloudflare, Reddit, Hulu, AWS, Blizzard, Steam, Xbox Live, Discord, and dozens more.

By

Catalin Cimpanu

| August 30, 2020 — 23:25 GMT (00:25 BST)

| Topic: Telcos

US internet service provider CenturyLink has suffered a major technical outage on Sunday after a misconfiguration in one of its data centers created havoc all over the internet.

Due to the technical nature of the outage — involving both firewall and BGP routing — the error spread outward from CenturyLink’s network and also impacted other internet service providers, ending up causing connectivity problems for many more other companies.

The list of tech giants who had services go down today because of the CenturyLink outage includes big names like Amazon, Twitter, Microsoft (Xbox Live), EA, Blizzard, Steam, Discord, Reddit, Hulu, Duo Security, Imperva, NameCheap, OpenDNS, and many more.

Cloudflare, which was also severely impacted today, said CenturyLink’s outward-propagating issue led to a 3.5% drop in global internet traffic, which would make this one of the biggest internet outages ever recorded.

According to a CenturyLink status page, the issue originated from CenturyLink’s data center in Mississauga, a city near Ontario, Canada.

Flowspec is an extension for the BGP protocol that allows companies to use BGP routes to distribute firewall rules across their network. Flowspec announcements are usually used when dealing with security incidents, such as BGP hijacks or DDoS attacks, as it allows companies to change their entire network to react and mitigate attacks within seconds.

However, today, CenturyLink said that its Mississauga data center sent out an incorrect Flowspec announcement that effectively prevented the company’s BGP routes from taking root.

Cloudflare, which observed the incident from afar, believes CenturyLink effectively but its entire network into a loop by announcing a brand new set of BGP routes and then accidentally dropping all routes via the misconfigured Flowspec rule.

BGP routes are the glue that keeps the internet up. They are a type of message that internet companies relay between each other. BGP routes tell each internet provider which chunk of IP addresses is available on its network.

However, as CenturyLink’s incorrect Flowspec command brought down some of the routers inside its network, some of those routers also began to announce incorrect BGP routes to other « Tier 1 » neighboring internet service.

CenturyLink fixed the issue by taking the rare step of telling all other Tier 1 internet providers to de-peer, and ignore any traffic coming from its network. Companies rarely take these kinds of decisions, as this results in full connectivity loss for all its customers.

wow wow, that must have been one of the biggest Internet wide outages in a while.. @CenturyLink asking other « tier1″s to de-peer.. that shows how bad it must have been, inability to recover. Customers dropping their peering with 3356, but routes not being withdrawn.. #ouch

On L3/CTL’s request, we’ve disabled all peering sessions until the situation is under control. Great to see industry-wide cooperation at what is undoubtably a hard time for AS3356. https://t.co/lbr38IHhyi

All in all, CenturyLink had to reset all equipment and start with clean BGP routing tables, a process that took almost seven hours to complete, from around 12:13 UTC to 18:58 UTC, the company said.

« This was a significant global Internet outage, » said Matthew Prince, co-founder & CEO of Cloudflare, in his analysis of the outage.

By

Catalin Cimpanu

| August 30, 2020 — 23:25 GMT (00:25 BST)

| Topic: Telcos

By registering, you agree to the Terms of Use and acknowledge the data practices outlined in the Privacy Policy.

You will also receive a complimentary subscription to the ZDNet’s Tech Update Today and ZDNet Announcement newsletters. You may unsubscribe from these newsletters at any time.

You agree to receive updates, alerts, and promotions from the CBS family of companies – including ZDNet’s Tech Update Today and ZDNet Announcement newsletters. You may unsubscribe at any time.

By signing up, you agree to receive the selected newsletter(s) which you may unsubscribe from at any time. You also agree to the Terms of Use and acknowledge the data collection and usage practices outlined in our Privacy Policy.

Opensignal says the usage and launch of 5G on low-band spectrum improves range, but hinders the speed measurements.

Equipping firefighters with these technologies will help them better understand and respond more quickly to bushfires, the report found.

On a standalone basis, profits from the former Vodafone Hutchinson Australia were once again wiped out by financing costs.

South Island continues to get the raw end of the deal, with almost twice the drop in speeds to those seen in the North Island.

© 2020 CBS Interactive. All rights reserved.
Privacy Policy |
Cookies |
Ad Choice |
Advertise |
Terms of Use |
Mobile User Agreement

SOURCE: https://www.w24news.com