The goof behind losing 911 service in May’s big outage
Glenwood Springs Post Independent
Why did half of the Roaring Fork Valley lose 911 service for more than 16 hours in May when a contractor cut a CenturyLink fiber-optic line south of Glenwood Springs?
It turns out that the backup circuit was routed through the same bundle of cable, the Post Independent learned through an open records request.
That was “a virtual engineering and planning fiasco,” an engineer who lives in the valley complained to the Colorado Public Utilities Commission.
A “911 Failure/Outage Report” filed with the PUC obtained by the PI through the open records request explained it this way: “The contingency plan could not be implemented because the alternate number is provided by another carrier, who did not have a dial tone at the time of the event.”
The backup carrier was not alone — the damage cut cell phone service from just south of Glenwood Springs to Aspen, disrupted most Internet access and, critically, knocked out 911 between Carbondale and Basalt even if people could find a working land line.
CenturyLink told the PI on Wednesday that it now has backup 911 circuits for Carbondale and Basalt.
“Since the outage in May, we have been working on ways to increase the capacity of our redundant circuits and paths in this difficult-to-serve but important area,” Mark Soltes, CenturyLink director of state legislative affairs for Colorado, said in a statement. “We have recently completed work to reroute 911 circuits as backup for Basalt and Carbondale so all four communities now have redundant service for 911 calls in case of future outages.”
Soltes added that Aspen and Snowmass Village 911 service was backed up by radio, or microwave, redundancy.
Residents likely never will know what contractor caused the damage, and any financial settlement apparently will be between CenturyLink and the other companies.
“Our company policy is that we don’t release that information,” regarding who caused the damage, CenturyLink spokeswoman Sara Spaulding said.
The third-party contractor and any settlement are beyond the purview of the Public Utilities Commission, said Lynn Notarianni, chief of the commission’s telecom section.
In fact, the initial incident report, emailed by CenturyLink to the PUC at 12:39 a.m. May 27 was completely blacked out, or redacted, in material the commission sent in response to the PI’s request under the Colorado Open Records Act. CenturyLink had deemed it proprietary information and the PUC was barred by law from disclosing it, said Assistant Attorney General Erin L. McLauthlin, who represents the PUC.
The commission received a complaint from a valley resident who was stunned by the lack of redundancy.
“This is to request, in no uncertain terms, a full investigation into the 16+ hour outage that occurred May 26-27 in the Roaring Fork Valley south of Glenwood Springs,” wrote the resident, whose name was redacted from documents sent in response to the PI’s request.
“As an engineer, I’m just astonished. Virtually all the phones in the entire middle and upper Roaring Fork Valley — land lines and 911, as well as (at least) Verizon and AT&T cell phones — were disabled.
“To have no redundancy whatsoever in that large a telecommunications system is simply unacceptable, a virtual engineering and planning fiasco,” the complaint said.
Notarianni said the PUC is investigating the incident as well as studying rulemaking to address the problem of 911 redundancy in rural areas.
“As you get more rural, challenges are greater and (solutions) more limited and complex,” she said. In cities, which have much more cable, “there are more opportunities to connect. Not everyplace has redundancy,” which the commission is assessing.
State Rep. Bob Rankin, R-Carbondale, expressed concern about the incident.
“We need to be assured that these kind of issues are under control,” he said.
Reliability of 911 systems is a matter of national concern. The Federal Communications Commission, for example, is examining an April 9, 2014, outage that affected 11 million people in seven states for six hours. In that instance, “a 911 call-routing facility in Englewood, Colorado, stopped directing emergency calls to eighty-one 911 call centers in … California, Florida, Minnesota, North Carolina, Pennsylvania, South Carolina and Washington. Although, fortunately, it appears that no one died as a result, the incident — and the flaws it revealed — is simply unacceptable,” an FCC report on the incident says.
“What is most troubling is that this is not an isolated incident or an act of nature,” the report says. “So-called ‘sunny day’ outages are on the rise. That’s because, as 911 has evolved into a system that is more technologically advanced, the interaction of new and old systems is introducing fragility into the communications system that is more important in times of dire need.”
While the issues in the Roaring Fork Valley may be slightly different, Carbondale and Rural Fire Protection District Chief Ron Leach summed up the May outage similarly: “We’re damn lucky something didn’t happen” while 911 was down, he said. “If it had, it could have been really bad here.”
Start a dialogue, stay on topic and be civil.
If you don't follow the rules, your comment may be deleted.