Subscribe
West coast Internet users, particularly those around San Francisco and Sacramento, experienced major disruptions to the Internet last evening into this morning, affecting everything from cable television and phone service to popular online destinations including Amazon.com (and websites hosted by its AWS data service), Tinder, and Netflix.
The range of disruptions led to early media speculation a “coordinated attack” on the Internet was underway on the west coast, but a statement from the Sacramento field office of the Federal Bureau of Investigation this morning clarified it was investigating only a single case of alleged intentional vandalism in the San Francisco area today.
The FBI suspects someone climbed down a manhole in Livermore early this morning and intentionally cut a high traffic fiber line owned by Level 3 and Zayo. This is not the first case of suspected vandalism. At least 10 other fiber line cuts in Fremont, Berkeley, San Jose, Alamo, and Walnut Creek have occurred in the Bay Area over the last year.
[flv]http://www.phillipdampier.com/video/USA Today FBI investigating 11 attacks on San Francisco-area Internet lines 7-1-15.flv[/flv]
USA Today reports the FBI is now investigating the 11th intentional fiber cut in the San Francisco Bay area in 12 months. (1:18)
The hardest hit ISP was Wave Broadband in West Sacramento, Calif. The fiber outage wiped out cable, phone and broadband service for customers across Sacramento, Rocklin, and surrounding communities including Dixon.
A broader issue yesterday evening also affected customers beyond northern California. Amazon.com and websites using its AWS platform suddenly stopped responding between 5:24pm-6:10pm PT last night. But that issue was later determined to be an unrelated “route leak” from Axcelx, a data center provider in Boston.
Thousand Eyes reports that problem “affected a wide range of services including consumer internet sites like Yelp, Netflix and Match; SaaS services such as HipChat and Jobvite; and financial firms such as Experian and Zions Bank.”
Any report of fiber vandalism concerns security experts, who suggest terrorists could target the highly visible data cables and create massive telecommunications disruptions in the United States.
“When it’s situations that are scattered all in one geography, that raises the possibility that they are testing out capabilities, response times and impact,” JJ Thompson, CEO of Rook Security, told USA Today. “That is a security person’s nightmare.”
[flv]http://www.phillipdampier.com/video/KCRA Sacramento Wave Broadband service restored after deliberate act 7-1-15.mp4[/flv]
KCRA in Sacramento said the telecommunications outages in Sacramento were frustrating for businesses, residents, and local government — all affected by the fiber cut in San Francisco. (2:20)
Fiber cables are also often readily identifiable by their bright orange insulation as well as from warning signs alerting construction crews and others to their presence underground.
DownDetector clearly identifies the impact of the fiber outage affecting Wave Broadband in the Sacramento area.
“There are flags and signs indicating to somebody who wants to do damage: This is where it is folks,” said Richard Doherty, research director of The Envisioneering Group, a technology assessment and market research firm. “You often have fiber from several companies sometimes going down the same street or the same trench. One attacker can dig one hole and wipe out service from three companies.”
The FBI is asking for the public’s help in identifying the vandal in the Bay Area. In addition to this morning’s attack, anyone who may have seen anything suspicious in these earlier attacks should contact them at 415-553-7400.
- July 6, 2014, 9:44 p.m. near 7th and Grayson St. in Berkeley
- July 6, 2014, 11:39 p.m. near Niles Canyon Blvd. and Mission Blvd. in Fremont
- July 7, 2014, 12:24 a.m. near Jones Road and Iron Horse Trail in Walnut Creek
- July 7, 2014, 12:51 a.m. near Niles Canyon Blvd. and Alameda Creek in Fremont
- July 7, 2014, 2:13 a.m. near Stockton Ave. and University Ave. in San Jose
- February 24, 2014, 11:30 p.m. near Niles Canyon Blvd. and Mission Blvd. in Fremont
- February 24, 2014, 11:30 p.m. near Niles Canyon Blvd. and Alameda Creek in Fremont
- June 8, 2015, 11:00 p.m. near Danville Blvd. and Rudgear Road in Alamo
- June 8, 2015, 11:40 p.m. near Overacker Ave and Mowry Ave in Fremont
- June 9, 2015, 1:38 p.m. near Jones Road and Parkside Dr. in Walnut Creek
[flv]http://www.phillipdampier.com/video/KXTV Sacramento FBI Sacramento area internet outage result of vandalism 7-1-15.flv[/flv]
KXTV in Sacramento reports the fiber cuts have immediate security and public safety implications for public officials. But network planners say no fiber cut should have disrupted so many customers and suggest better planning could have spared many from the service outage. (2:23)
Outdoor fiber cable doesn’t have bright orange insulation. Every buried or aerial fiber I’ve ever encountered was black, with a thick outer sheath, sometimes with a metallic layer under that for extra armor. It is possible for fiber to be in an orange conduit but, that conduit is almost completely buried until it intersects with a manhole or handhole. The fact that someone is climbing into a vault or opening a handhole means they have the tools and know how to do so. They have probably worked in the industry at some level. OR, they are just crazed addicts trying… Read more »
We have plenty of DoT fiber around here that is black on the pole but is orange running down the pole or at the point it descends underground or into a cabinet. It is very obvious on many poles which cables are fiber and which are copper. I have not explored whether the orange is the sheath or some sort of conduit. I’ve even seen some fiber cables tagged on the pole as fiber. I agree with you that the culprit is probably a disgruntled ex-employee or someone with industry experience, probably as an engineer or technician. The Oxy/Meth freaks… Read more »
Uh, the culprit has been committing dozens of acts, across many states, and nearly simultaneously in widely separated areas. The culprit has been excavating fiber in very remote areas from under tons of dirt and rock, and using power tools to cut through the outer jackets of the cables. Then, the cables are just left laying there. The culprit could be a group of ex-employees spending a lot of personal time and going to a lot of risk for completely unknown reasons. Consider that this might be a startlingly well-informed but curiously unpublicized activist group, or corporate sabotage connected to… Read more »