One Customer Made a Change. Then, Fastly's 'Severe' Outage

Fastly says software bug was behind global disruption that affected world's top websites
By Newser Editors and Wire Services
Posted Jun 9, 2021 8:54 AM CDT
One Customer Made a Change. Then, Fastly's 'Severe' Outage
The Fastly homepage is seen on Tuesday in Los Angeles.   (AP Photo/Marcio Jose Sanchez)

Fastly, the company hit by a major outage that caused many of the world's top websites to go offline briefly this week, blamed the problem on a software bug that was triggered when a customer changed a setting. The problem at Fastly meant internet users couldn't connect to a host of popular websites early Tuesday, including the New York Times, the Guardian, Twitch, Reddit, and the British government's homepage. "We experienced a global outage due to an undiscovered software bug that surfaced on June 8 when it was triggered by a valid customer configuration change," Nick Rockwell, Fastly's senior vice president of engineering and infrastructure, said in a blog post late Tuesday, per the AP. He said the outage was "broad and severe," but that the company quickly IDed, isolated, and disabled the problem, and after 49 minutes, most of its network was up and running again.

The bug had been included in a software update that was rolled out in May, and Rockwell said the company is trying to figure out why it wasn't detected during testing. "Even though there were specific conditions that triggered this outage, we should have anticipated it," Rockwell said. San Francisco-based Fastly provides what's called a content delivery network—an arrangement that allows customer websites to store data such as images and videos on various mirror servers across 26 countries. Keeping the data closer to users means it shows up faster. But the incident highlighted how the much of the global internet is dependent on a handful of behind-the-scenes companies like Fastly that provide vital infrastructure, and it amplified concerns about how vulnerable they are to more serious disruption.

(More Fastly stories.)

Get the news faster.
Tap to install our app.
Install the Newser News app
in two easy steps:
1. Tap in your navigation bar.
2. Tap to Add to Home Screen.