Tens of thousands of rural New York families were hopeful after Gov. Andrew Cuomo announced in 2015 his intention to bring true broadband to every corner of the state by the year 2018. At the time, it was the largest and most ambitious broadband investment of any state in the country, putting $670 million in lawsuit settlement money and rural broadband funds from the FCC on the table to build out rural broadband service other states only talk about.

But for many rural New Yorkers, Gov. Cuomo’s program was a failure that could lock in substandard internet service (or no service at all) for years. What began as a 100% broadband commitment later evolved into 99.9% (then 98% in another estimate) after state officials learned $670 million was not enough to convince providers to share the cost of extending their networks to the most rural of the rural as well as those unlucky enough to live just a little too far down the road to make extending cable broadband worthwhile. But the governor proclaimed mission accomplished, and as far as the Cuomo Administration is concerned, the rural broadband issue has been resolved.

“There were a lot of tax dollars that were flipped and the governor has said, ‘Internet for everybody. Everybody will have internet.’ Well, that’s not the case. We’re not seeing that and those were his promises, not mine, but I voted for that money. A lot of other members did too,” Sen. Rob Ortt (R-North Tonawanda) told WBFO radio last year.

Ortt wants to know where the money is going and who exactly is getting it, and proposed legislation requiring annual reports from the Empire State Development Corporation detailing expenditures and disclosing the formula used to determine who gets true broadband service, and who does not.

For those not getting high-speed wireless or wired connections, the state has either offered nothing or dreaded satellite internet service, paying HughesNet $14,888,249 to supply discounted satellite equipment Hughes itself routinely discounts as a marketing promotion on their own dime.

For rural residents learning HughesNet was their designated future provider, many experienced with satellite internet over the last decade and hating nearly every minute of it, it was “thanks for nothing.”

“The governor pulled the rug right out from under us,” Ann told Stop the Cap! from her home near Middle Granville in Washington County, just minutes away from the Vermont border. “I have kids that require internet access to finish research and send in homework assignments. Internet service is not an option, and my kids’ grades are suffering because they have to complete homework assignments in the car or in a fast food restaurant or coffee shop that has Wi-Fi.”

Ann used HughesNet before, and canceled it because service went out whenever snow arrived in town.

“I thought the governor promised 100 Mbps service and HughesNet can’t even provide 25 Mbps,” she claims. “If you get 5 Mbps on a clear summer’s day, you are doing okay. In winter, reading email is the only thing that won’t frustrate you. It’s slow, slow, slow.”

Gov. Andrew Cuomo announcing rural broadband initiatives in New York.

Nick D’Agostino brought his family to a new home an hour northeast of Syracuse when he got a new job. He was counting on the governor’s commitment to bring wired internet access to a home that used to have Verizon DSL, but no longer does after Verizon’s wired infrastructure deteriorated to the point where the company stopped offering the service to new customers like him arriving in the neighborhood. D’Agostino had to spend hours researching the state’s Broadband Program Office website to find out which provider was going to be supplying his census block (neighborhood) with 100 Mbps internet. He found HughesNet instead.

“It’s a kick in the pants because we have a lot of experience with HughesNet and Exede and neither came close to meeting their advertising claims,” he told Stop the Cap! “Exede was often unusable and a horrible company to deal with. HughesNet has a new ‘Gen 5’ service that is capable of DSL speeds, but comes with a low data cap and speed throttling.”

D’Agostino warns that New York made a terrible choice relying on satellite internet, even though HughesNet’s latest fleet of satellites has offered improvement over HughesNet a decade ago.

“The problem is HughesNet customers in a geographic area all share the same spot beam — a regionally targeted satellite signal that serves a specific state or region,” D’Agostino said. “When we lived in North Carolina, the population growth in rural areas meant a lot more satellite customers were sharing the same spot beam, and speeds plummeted, especially after Netflix, Hulu, and cord cutting took off. Nothing eats bandwidth like streaming video, which is why you can subscribe to their 50 GB allowance package and be over that limit after a single week.”

D’Agostino fears that tens of thousands of additional satellite users will dramatically slow down HughesNet across upstate New York unless the company finds a way to get more shared bandwidth to serve the state’s rural broadband leftovers.

“That usually means, ‘wait until the next generation of satellites are launched,’ something nobody should have to wait for,” D’Agostino said.

The obvious solution for D’Agostino is to convince Charter Spectrum, the nearest cable provider, to extend its lines down his street. The cable company agreed, if he paid an $88,000 engineering, pole, and installation fee.

“That is not going to happen, even if we got the dozen or so neighbors in our position to split the cost,” he said. “This is why Cuomo’s program is a flop. It turns out close to $700 million is not enough, and they probably always knew there would be people they could never economically serve because they are miles and miles from the nearest DSL or cable connection. But if the electric and phone companies are compelled to offer service, the same should be true for internet access.”

D’Agostino believes rural New Yorkers left behind need to organize and make their voices heard.

“They keep saying we are .1% of New York, but I’ve seen plenty of rural town supervisors and other local officials across upstate New York complain they have all been left behind, and that decision will cost their towns good education, jobs, competitive agribusiness, and services online that everyone assumes people can easily access,” he said. “Clearly the state is not telling the truth about how many are being internet-orphaned. There have been three rounds of broadband funding in New York. It is time for a fourth round, finding either tax breaks or funding to get existing providers to reach more areas like mine that are less than a mile from a Spectrum customer.”

Ann shares that sentiment, and adds that Vermont is looking for ways to get internet to its rural residents as well.

“We’re at the point where companies or co-ops already offering service are probably the quickest and easiest option to solve the rural internet crisis, but they are not going to pay to do it if they are not required to,” she said. “We have taxes and surcharges on our phone bill now that are supposed to pay for internet expansion, but the amounts are too small to get the job done I guess. Perhaps it is time to revisit this, because 99.995% is better than 99.9% and satellite internet should be the last resort for people living in a cottage miles from anyone else, not for people who can be in town in less than a five-minute drive.”

A familiar story for any rural resident trying to get internet access to their rural home. But there is a small silver lining. HughesNet’s newest generation of satellites has provided a modest improvement that is often better than rural DSL. (10:19)

What do an emergency operations center in Cochise County, Ariz., Colorado hospitals, the Idaho Bureau of Corrections, and many 911 call centers across Massachusetts have in common? They were all brought down by a two-day nationwide CenturyLink outage from Dec. 27-28 that also resulted in internet outages for tens of thousands of CenturyLink’s residential customers. The cause? CenturyLink blamed a single, faulty third-party network management card in Denver for disrupting services for CenturyLink and other phone companies, notably Verizon, from Alaska to Florida.

Hours after outage began, two days after Christmas, CenturyLink issued a general statement:

“CenturyLink experienced a network event on one of our six transport networks beginning on December 27 that impacted voice, IP, and transport services for some of our customers. The event also impacted CenturyLink’s visibility into our network management system, impairing our ability to troubleshoot and prolonging the duration of the outage.”

That “network event” caused serious disruptions to critical services in 37 states, including 911, according to Brian Kyes, president of the Massachusetts Major City Chiefs of Police Association.

“This is affecting 911 (wireline & wireless) delivery to most of Massachusetts,” Kyes said in a statement during the outage to the Boston Herald. “We have heard from MEMA that this issue may also affect some landlines but I have not heard of any specific situations or communities that have been impacted. We are advising all police and fire chiefs to test their local 911 systems and notify their residents of potential issues by reverse 911, social media or any other means that they have at their disposal. The interruption in service may depend on a particular phone carrier and the information that we have is that it may be intermittent.”

CenturyLink outages on Dec. 27, 2018. (Image: Downdetector.com)

The disruptions affected much of Massachusetts — a state served primarily by Verizon Communications, because CenturyLink is a major commercial services vendor inside and outside of its local landline service areas and supplies some connectivity services to Verizon, mostly for wireless customers.

ATM networks also went down in certain parts of the country. CenturyLink is one of many vendors providing data connectivity between the cashpoint machines and several banking institutions.

Also impacted, the Idaho Department of Corrections, including inmate phone systems, and the Idaho Department of Education, which lost the ability to make or receive calls.

Consumers also noticed their internet connections were often down or sporadic in some locations, primarily because CenturyLink’s backbone network became saturated with rogue packets.

The Denver Post presented a more detailed technical explanation about the outage:

CenturyLink said the [defective] card was propagating “invalid frame packets” that were sent out over its secondary network, which controlled the flow of data traffic.

“Once on the secondary communication channel, the invalid frame packets multiplied, forming loops and replicating high volumes of traffic across the network, which congested controller card CPUs (central processing unit) network-wide, causing functionality issues and rendering many nodes unreachable,” the company said in a statement.

Once the syndrome gets going, it can be difficult to trace back to its original source and to stop, a big reason networks are designed to isolate failures early and contain them.

“We have learned through experience about these different types of failure modes. We build our systems to try and localize those failures,” said Craig Partridge, chair of the computer science department at Colorado State University in Fort Collins and a member of the Internet Hall of Fame. “I would hope that what is going on is that CenturyLink is trying to understand why a relatively well-known failure mode has bit them.”

The Federal Communications Commission also expects answers to some questions, opening another investigation of the phone company. In 2015, CenturyLink agreed to pay a $16 million settlement to the federal agency after a seven-state outage in April 2014.

Pai

FCC Chairman Ajit Pai said the agency would once again take a look at CenturyLink, focusing on disruptions to emergency services.

“When an emergency strikes, it’s critical that Americans are able to use 911 to reach those who can help,” Pai said in a statement. “This inquiry will include an examination of the effect that CenturyLink’s outage appears to have had on other providers’ 911 services.”

A retired manager at Qwest, a former Baby Bell now owned by CenturyLink, strongly criticized CenturyLink’s lack of communications with customers and an apparent lack of network redundancy.

“For a company in the communication business, they sure failed on this,” said Albuquerque resident Sam Martin. “I participated on the Qwest Disaster Recovery teams, and I do not recall ever having the network down for this kind of time and certainly never the 911 network. The 911 network should never have been down. The lack of this network can contribute to delays in rescue and fire saving lives.”

Martin is dubious about CenturyLink’s explanation for the network outage, suggesting a defective network card may be only a part of the problem.

“The explanations given so far are not valid,” Martin said. “The public may not be aware of it, but the communication network has redundancy and for essential services like inter-office trunking and 911 calls, there are duplicate fiber optic feeds – “rings” that duplicate the main circuit in another path – and switching equipment to these locations so that they may be switched electronically and automatically upon failure to a back-up network ring. When these systems are operating properly, the customer is unaware a failure occurred. If the automatic switching does not take place, employees involved with disaster recovery can intervene and manually switch the affected network to another fiber ring or electronic hub and service is restored until the actual damage is fixed.”

None of those things appeared to happen in this case, and the outage persisted for 48 hours before all services were restored.

“CenturyLink has to have a disaster recovery plan with redundancies in place for electrical, inbound and outbound local and toll-free carriers, as well as network and hardware component redundancies. CenturyLink should be able to switch between multiple fiber optic rings or central offices in case entire networks of phones go down. They would then locate and repair, or replace, defective telecommunication components without the customer ever knowing. The fact that this did not happen is discouraging and scary for the consumer. The fact that it happened nationwide is even more surprising and disturbing. Hopefully the truth will come out soon.”

A critical editorial in the Albuquerque Journal added:

We need answers from CenturyLink beyond the cryptic “a network element” caused the outage. We need to know how many CenturyLink and Verizon customers were affected. And we need to know what they – and other internet and phone providers – are doing to prevent similar outages or worse from happening in the future. Because if the outage showed nothing else, it’s that like an old-time string of Christmas lights, we are living in an interconnected world.

And when one light goes out, they can all go out.

KTVB in Boise, Idaho reported on CenturyLink’s massive outage on Dec. 27-28 and how it impacted local businesses and government services. (3:09)

Windstream is relying on the Federal Communications Commission’s Connect America Fund to double the areas where it will offer 100 Mbps broadband service, expected to reach 30% of the company’s 18-state local service area by the end of the first quarter of 2019.

“Windstream understands that premium internet speeds are critical to families and businesses in rural America, and we are systematically enhancing our network to meet that urgent demand,” said Jeff Small, president of consumer and small and medium-sized business services. “Network upgrades are expensive, especially in rural areas where there are relatively few customers, so Windstream is using a combination of its own capital and crucial support from the FCC’s Connect America Fund to make faster speeds more widely available. Without support from the Connect America Fund, many of these projects simply would not be economically feasible.”

Thomas told attendees at the Citi 2019 TMT West Conference Windstream’s legacy copper wire telephone network is not up to the job of handling the kinds of internet speeds more modern technologies can manage.

In urban and larger service areas, Windstream is most likely to deploy fiber to the home service in new housing developments and select gentrified neighborhoods where a business case exists to invest in fiber upgrades. The company also typically replaces its copper wireline infrastructure with fiber where road construction projects or damage forces the company to replace or relocate its lines. Suburban and more densely populated rural areas are likely to receive an upgraded version of Windstream’s DSL service that can manage up to 50 or 100 Mbps. In Windstream’s significant rural service area, the phone company is increasingly turning to fixed wireless technology, especially in flat midwestern states like Nebraska and Iowa where it plans to offer a combination of 3.5 GHz “CBRS” and 5G millimeter wave fixed wireless broadband capable of delivering up to 1,000 Mbps.

Windstream’s service area

“[We are deploying wireless internet] probably at a larger scale than a lot of the larger wireless companies,” Thomas said, especially in flatter areas where wireless signals go a long way.

Because most current broadband expansion fund programs require companies to commit to at least 25/3 Mbps service, simply expanding basic ADSL technology has proven inadequate to meet the government’s speed requirements. But wiring fiber to the home service to get faster speeds in rural areas does not meet the Return On Investment requirements Windstream’s shareholders demand. Windstream claims fixed wireless can solve both problems.

“You can get 100 Mbps out there very cost-effectively,” Thomas claimed. “You are really blowing away copper infrastructure and making it irrelevant because you’re embracing this 100 Mbps technology.”

As of early 2019, Windstream claims that 60% of its customers can get at least 25 Mbps service, 40% can receive at least 50 Mbps service. By the end of March, 30% will be able to receive 100 Mbps service.

A satisfied Windstream customer talks about his upgrade to 50/8 Mbps, which replaces his old 6 Mbps DSL service. (6:03)

A corporate-funded business advocacy group backed by the telecom industry and the Koch Brothers is pursuing a lawsuit asking the D.C. Court of Appeals to toss pro-consumer deal conditions imposed by the Federal Communications Commission in return for granting its 2016 approval of the acquisition of Time Warner Cable and Bright House Networks by Charter Communications.

The Competitive Enterprise Institute filed an initial petition with the FCC asking the agency to rescind its own deal conditions shortly after the merger was completed. CEI argued the agency imposed “harmful merger conditions on Charter that had nothing to do with the merger itself,” and that the FCC did not have the authority to put corporate merger deal conditions in place.

CEI specifically targeted its objections to the FCC’s seven-year ban on Charter Spectrum data caps and consumption billing, arguing the ban raised broadband pricing for all Spectrum customers and prevented the cable company from offering discounts to low usage customers. It also claimed that Charter had to increase pricing for all customers because the FCC required Spectrum to raise broadband speeds, introduce a discounted internet program for low-income customers, and expand service to at least two million new households not presently served by Spectrum.

The FCC ultimately rejected CEI’s petition in 2018, claiming the group had no standing to challenge the merger transaction or deal conditions. The group called the FCC’s decision wrong, claiming consumers will “have to foot the bill for an overreaching federal agency” and that “the FCC has no authority to micromanage the internet at the public’s expense.”

This week, it filed an opening brief appealing the FCC’s decision to the D.C. Court of Appeals, which oversees the legality of the FCC’s regulatory decisions.

The 101-page filing maintains the FCC overreached by imposing any deal conditions on the 2016 multi-billion dollar merger deal, especially those that might require the merged company to spend money to improve service to customers. CEI argued such conditions were “arbitrary and capricious” and had no place as part of approving a business merger transaction.

The group submitted evidence from four individuals who attested to their belief that the deal conditions “probably contributed” to price increases after customers abandoned their legacy Bright House and Time Warner Cable plans in favor of Spectrum plans and pricing. The customers reported rate hikes ranging from $4 a month to $20 a month “for the same services,” but did not attach copies of their bills allowing a court to ascertain whether those rate increases involved cable television or broadband service or both.

No evidence was provided to prove CEI’s assertion that rate increases were directly tied to merger conditions other than a declaration from Robert W. Crandall, an economist and nonresident senior fellow at the Technology Policy Institute in Washington, D.C. Crandall argued any deal conditions requiring a cable company to spend money to expand, improve, or discount services would likely impact subscriber rates.

No disclosure was made regarding any fees paid to Crandall to conduct research on behalf of CEI. The Technology Policy Institute is financially backed almost entirely by the Koch Brothers and corporate interests including AT&T, Charter Communications, Comcast, and Verizon.

CEI’s legal brief depends on assertions made by then-minority Republican members of the FCC, notably then-Commissioners Ajit Pai and Michael O’Rielly, who objected to the FCC’s merger conditions. CEI ignored the views of the then-Democratic majority on the Commission, who voted to approve the merger with deal conditions. Then Chairman Thomas Wheeler and Commissioners Mignon Clyburn and Jessica Rosenworcel were not mentioned anywhere in CEI’s brief. Today the Commission has a Republican majority, with Pai now serving as chairman.

The FCC in 2016 (from left to right): Commissioners Ajit Pai, Mignon Clyburn, Chairman Tom Wheeler, and Commissioners Jessica Rosenworcel and Michael O’Rielly

CEI’s argument follows a similar pattern to arguments made against net neutrality — namely, the FCC has no authority to regulate broadband services or the pricing and policies of the companies providing it, as companies offer different services including health therapy, if you want to offer this you could check this hypnotherapist certification online just for this. Charter Communications has occasionally argued the same point with the New York State Public Service Commission, which imposed deal conditions of its own in return for approval of the merger.

Charter has consistently reserved the right to object to deal conditions requiring it to build out service to rural areas, as well as any deal conditions that go beyond the authority of state regulators to oversee broadband service. In Charter’s view, state regulators have no such authority. In the state’s view, the PSC has the right to consider a myriad of factors because its regulatory mandate  requires approving or rejecting a merger based on the public interest. Its 2016 merger order found the transaction was not in the public interest unless the parties agreed to certain deal conditions, which closely resembled those required by the FCC. When Charter allegedly failed to meet the conditions it agreed to, the New York regulator could not directly compel Charter Spectrum into compliance, but it could and did decertify the merger itself.

Should the D.C. Court of Appeals find in favor of CEI, the deal conditions imposed by the FCC would be revoked, although Charter could continue to honor those conditions voluntarily. Separate legal cases would have to be brought in state courts to invalidate deal conditions imposed by state regulators.