Life was much different a little more than a decade ago, wasn’t it? Every morning did not begin with a glance at the notifications on your cell phone, every place you visited did not have to be checked-in on Facebook, every opinion did not lead to an argument on Twitter, and every photo did not need validation on Instagram. However, fast forward to now, and you will see that smart phones have become a necessary gadget for simply surviving in this fast-paced digital world. I wonder how many of us could get through a day without suffering from an anxiety attack if we were barred from using our phones the entire time!

Well, I’m not sure who wished for such a ‘technological-detox’, but subscribers of the largest mobile networks in the United States were in such a situation on Friday, the 31st of January.

Network down. Everywhere!

Reports of major outages at all four of the big US wireless carriers, (as well as some smaller carriers) kept pouring in, with Down Detector showing downtime especially on the East Coast. Users were panicking (obviously!) and taking to social media to lodge complaints, and that’s when they realized they (or their network) weren’t alone.

Graph showing a spike in customer complaints for all major carriers in the U.S.
Source: Down Detector

Looking at the complaints, it was evident that the majority of users were reporting that they couldn’t make phone calls or send texts, with plenty of people also saying that mobile data wasn’t working. Many said on Twitter that it took them several attempts to make a call and that phones kept cycling between full service and no service at all. Outages were being reported from all over the country, making it difficult to pin down exactly what went wrong.

Is there a @verizon outage in the Triad?

Anyone else having issues?

— Christian Morgan (@CMorganWX) January 31, 2020

Sprint, Verizon and T-Mobile having major cell phone outages. Any news on this someone fill me in?

— ~ Yours Truly~ (@kandigurl9581) January 31, 2020

Map showing the affected areas in the country.
Source: Down Detector

The culprit – A change in network configuration

It was only a few hours (and probably thousands of complaints) later that Verizon figured out that the outage was the result of making a change to its network that impacted a number of cell sites. What made things worse was that carriers weren’t able to provide an estimated time for when the service would be fully restored.

“Verizon uses Charter Communications fiber lines to carry information between its cell sites and the core network in your area. This morning Charter made a change to its network which impacted a number of our cell sites. Charter is working quickly to fix this issue.” reported VZWSupport in response to customer complaints.

Clearly, customers were left fuming, but with not much choice during this entire episode, which took almost a day to fix. While the whole thing can be seen in a light manner if you look at how challenged customers feel when they’re unable to use their phones for a few hours, there is also a pretty serious angle to it. Apparently, emergency services in counties across the state were warning customers via Twitter that the outage could prohibit people from reaching aid via 911. Now, in a country like the United States where there is heavy dependency on such services, an outage like this can have undesirable consequences in emergency situations and could even cost lives! Luckily for everyone, there were no such instances reported from anywhere.

We are aware of a widespread Verizon Wireless outage. Customers calls may not connect, including to 911.

Verizon is working on the problem.

— Emergency Services (@RandolphCoES) January 31, 2020

Let us make cellular networks great again!

This was not the first outage that carriers have had in recent times, and it is certainly not going to be the last. With the increasing complexities of telecom networks, configuration errors are going to happen, and carriers need to put systems in place to help them foresee and prevent such incidents.

Usually, in cases such as Friday’s, the solution seems very simple, but only in hindsight. In reality, the ‘simple’ change that Charter had made to its network wouldn’t have been so simple to detect until it started affecting thousands of subscribers nationwide.

However, what could have saved the day for the carriers is - having a real-time anomaly detection system in place that could monitor the network and cell site data for unusual patterns and behavior to alert the respective teams, before it started impacting subscribers. Ability to detect anomalies on network infrastructure enables better visibility and control, facilitating timely action by Network Operations teams to enhance efficiency and customer experience, and in some cases, even saving lives!

Were you one of the affected subscribers in last week’s network outage? If yes, let us know for how long, and which network in the comments section.

Manage your Network operations and revenue with Augmented Analytics

Download the whitepaper

Rakshit Raviprakash

Rakshit is a Product Marketing professional with close to 10 years of experience and comes with strong expertise in Integrated Marketing, product positioning, messaging, content creation, and demand generation for IT products and solutions. Rakshit holds a management degree from M.P. Birla Institute of Management, specializing in Marketing and Information Systems and a bachelor’s degree in Mechanical Engineering.