Santa Broadband: Ajit Pai’s magical world of broadband

God bless deregulation and your local phone and cable companies for making American Broadband Great Again.

That’s the message FCC Chairman Ajit Pai hopes will be the take away in the forthcoming 2019 Broadband Deployment Report — a highly dubious and over optimistic assessment of America’s rural broadband landscape.

“For the past two years, closing the digital divide has been the FCC’s top priority,” Chairman Pai said. “We’ve been tackling this problem by removing barriers to infrastructure investment, promoting competition, and providing efficient, effective support for rural broadband expansion through our Connect America Fund. This report shows that our approach is working. But we won’t rest until all Americans can have access to broadband and the 21st century opportunities it provides to communities everywhere.”

Except closing the rural-urban broadband gap has been a FCC priority for more than two years, and was a particularly high priority for the previous administration, which devoted a large amount of controversial stimulus funding after the Great Recession to internet expansion during the Obama Administration. In fact, Chairman Pai repeatedly claimed credit for broadband expansion projects that were funded by the previous administration, while at the same time criticizing the FCC under former Chairman Thomas Wheeler for harming investment in broadband with the enforcement of net neutrality.

The FCC continues to rely on dubious and flawed data to produce its reports — unverified data typically volunteered by the country’s phone and cable companies. The FCC has been frequently criticized for relying on inaccurate broadband availability maps, taking providers at their word on broadband speeds that fail to materialize in the real world, and reporting expansion projects that do not directly benefit consumers.

Pai’s office this week released a press release attempting to conflate broadband gains to his deregulatory policies and the banishment of net neutrality.

“The private sector has responded to FCC reforms by deploying fiber to 5.9 million new homes in 2018, the largest number ever recorded. And overall, capital expenditures by broadband providers increased in 2017, reversing declines that occurred in both 2015 and 2016.”

But Pai does not offer any evidence to back up those claims. In fact, as Stop the Cap! has reported, many of the country’s largest telecom companies have been cutting capital expenditures, many initiated as part of system upgrades to convert to digital cable television or to increase the amount of fiber optics to increase cable system reliability — neither relevant to the debate about net neutrality. This year, Charter Communications has announced a dramatic drop in spending (despite the repeal of net neutrality) because their long-planned system upgrades surrounding the retirement of analog cable television are now complete. Charter also had its merger agreement with Time Warner Cable revoked in New York for failing to meet its rural broadband commitments in that state.

Comcast cut spending by 3% because it bought fewer set-top TV boxes in light of cord-cutting customer losses. Verizon, which has been aggressively promoting its forthcoming 5G millimeter wave wireless network, slashed spending from $17.2 billion in 2017 to between $16.6-17 billion last year, and a significant sum of that money was earmarked for 5G buildouts in urban areas, not expanding rural internet. AT&T’s capital expenditures for 2019 are not expected to move much, placed in the $23 billion range for 2019, just a little more than last year. But AT&T is expecting to be reimbursed $1.6 billion by the federal government for AT&T’s FirstNet public safety network buildout, and much of its other spending is targeting its wireless business, including a plan to launch 5G services in 19 cities this year. That means less money for AT&T’s wireline network, including fiber broadband for homes and businesses.

Pai’s claims about the increased availability of broadband, at higher speeds, comes largely at similar incremental rates to progress under the Obama Administration. In New York, which is seeking to approach near universal broadband coverage, what moved the needle the most was a large sum of funding available to subsidize rural broadband expansion. The availability of substantial financial assistance from the state government, which some described as corporate welfare, appeared to be the most effective broadband expansion motivator for an industry Pai praised in his press release, not deregulation or the repeal of net neutrality.

Spending less, charging more in 2019.

Despite repeated claims from some in Washington that eliminating net neutrality would stimulate U.S. telecommunications companies to invest more in their networks, Charter Communications has announced a dramatic $1.9 billion cut in capital expenditures (CapEx) spending on its Spectrum cable systems for 2019.

Charter posted 2018 revenue of $43.6 billion (up 4.9 percent over 2017), with especially healthy returns for its internet service, which grew 7.1%. Charter earned $11.2 billion in revenue, up 5.9% in the fourth quarter of 2018 alone, partly from rate increases, reduced costs, and additional broadband customers.

Republican FCC commissioners have repeatedly argued that deregulating the internet by sweeping away net neutrality would stimulate companies to invest more in their networks. But it now appears the reverse is true. In 2017, Charter spent $8.7 billion on network investments; in 2018 the company spent $9.1 billion. But this year, with net neutrality no longer the law of the land, the cable company is planning to dramatically cut investments in 2019 to just $7 billion. The combined company, which now includes Time Warner Cable (TWC) and Bright House Networks (BH), has never spent this little on capital expenditures. The 2016 merger between Charter and TWC and BH forced a 189.4% spike in spending after the deal was completed, as Charter began a cable system overhaul and upgrade.

Charter is expecting it can distribute more of its revenue to shareholders, share buybacks, and debt payments as a result of the completion of its all-digital conversion project, which eliminated analog television signals from cable systems to make more room for revenue-enhancing internet service. The company also gets to lease more set-top boxes to customers seeking to view digital television signals on older analog TV sets.

Charter also reports it has successfully completed its DOCSIS 3.1 internet upgrade to more than 99% of its cable systems, allowing the introduction of premium-priced gigabit internet speed.

Charter executives signaled investors earlier this month Charter expects to post greater revenue and profits as a result of the spending reductions, but these new-found gains will have no effect on the company’s ongoing plans to continue mildly aggressive rate increases in 2019.

Charter has not disclosed how much it plans to spend on its new mobile business in 2019. The company is marketing its mobile phone service more aggressively this year as it prepares to accept customers bringing existing phones to its cellular service, powered by Charter’s in-home and in-business Wi-Fi and Verizon Wireless’ 4G LTE network.

Solving America’s rural broadband crisis will take a lot more than demonstration projects, token grants, and press releases.

Since 2008, Stop the Cap! has witnessed media coverage that breathlessly promises rural broadband is right around the corner, evidenced by a new state or federal grant to build what later turns out to be a middle mile or institutional fiber optic network that is strictly off-limits to homes and businesses. Politicians who participate in these press events tend to favor publicity over performance, often misleading reporters and constituents about just how significant a particular project will be towards resolving a community’s broadband challenges. Much of the time, these projects turn out to serve a very limited number of people or only fund part of a broadband initiative.

Officials last week said they are hoping to make North Carolina “the first ‘giga-state,’ with broadband access for all its residents.” But to realistically achieve that goal, nothing short of an expenditure of hundreds of millions of dollars will be required to realistically achieve that goal statewide.

A decade ago, rural broadband progressed in North Carolina, as communities transitioned away from traditional tobacco and textile businesses to the information and technology economy. To assure a foundation for that economic shift, several communities identified their local substandard (or lacking) broadband as a major problem. The state’s phone and cable companies at the time — notably Time Warner Cable, AT&T, and CenturyLink, often proved to be obstacles by refusing to upgrade networks in the state’s smaller communities. Some cities decided to stop relying on what the broadband companies were willing to offer and chose to construct their own modern, publicly owned broadband network alternatives, open to residents and businesses. A handful of cities in North Carolina went a different direction and acquired a dilapidated and bankrupt cable system and invested in upgrades, hoping cable broadband improvements would help protect their communities’ competitiveness to attract digital economy jobs.

That progress largely stalled after Republicans took control of the state legislature in 2011 and passed a draconian municipal broadband law that effectively banned public broadband expansion. Most of those backing the measure took lucrative campaign contributions from the state’s dominant phone and cable companies. One, Sen. Marilyn Avila, worked so closely with Time Warner Cable’s lobbyists, the resulting bill was effectively drafted by the state’s largest cable company. For that effort, she was later wined and dined by cable lobbyists at a celebration dinner in Asheville.

To be fair, some North Carolina cities are experiencing a broadband renaissance. Charlotte, Raleigh, Greensboro, Cary, Durham, Winston-Salem, and Chapel Hill will have a choice of providers for gigabit service. Google has installed fiber in some of these cities while AT&T and Charter lay down more fiber optics and introduce upgrades to support gigabit speeds.

Things are considerably worse outside of large cities.

In North Carolina, 585,000 people live in areas where their wired connections cannot reach the FCC’s speed definition of broadband — 25 Mbps, and another 145,000 live without any notion of broadband at all.

Bringing all of North Carolina up to at least the nation’s minimum standard for broadband will not be cheap, and politicians and public policy groups must be realistic about the real cost to once and for all resolve North Carolina’s rural broadband challenges. Where the money comes from is a question that will be left to state and local officials and their constituents. Some advocate only tax credit-inspired private funding, others support a public-private partnership to share costs, while still others believe public money should be only spent on publicly owned, locally developed broadband networks. Regardless of which model is proposed, without a specific and realistic budget proposal to move forward, the public will likely be disappointed with the results.

The Facts of Broadband Life

There is a reason rural areas are underserved or unserved. America’s broadband providers are primarily for-profit, investor-owned companies. They are not public servants and they respond first to the interests of their shareholders. Customers might come in second. When a publicly owned utility or co-op is created, in most cases it is the result of years of frustration trying to get a commercial provider to serve a rural or high cost area. Public projects are usually designed to serve almost everyone, even though it will likely take years for construction costs to be recovered. Investor-owned companies are not nearly as patient, and usually demand a Return On Investment formula that offers a much shorter window to recover costs. For broadband, adequately populated areas that can be reached affordably and attract enough new customers to recover the initial investment will get service, while those areas that cannot are left behind. The two populations most likely to fail the ROI test are the urban poor that may not be able to afford to subscribe and rural residents a company claims it cannot afford to serve. Many early cable TV franchise agreements insisted on ROI formulas that allowed companies only to skip areas with inadequate population density, not inadequate income, which explains why cable service is available in even the poorest city neighborhoods, while a wealthy resident in a rural area goes unserved.

Today, most cable and phone companies install fiber optic infrastructure most commonly in new housing developments or previously unwired business parks, while allowing existing copper wire infrastructure already in place elsewhere to remain in service. Some companies, including AT&T and Verizon, have made an effort in some areas to replace copper infrastructure with fiber optics, but in most cases, their rural service areas remain served by copper wiring installed decades ago. As a result, most rural residents end up with DSL internet from the phone company, often at speeds of 5 Mbps or less, or no internet service at all. Neither of these phone companies, much less independent telcos like CenturyLink and Frontier, have shown much interest in scrapping copper wiring for fiber optics in rural service areas. There is simply no economic case that shareholders will accept for costly upgrades that will deliver little, if any, short-term benefits to a company’s bottom line. That reality has led some communities to try incentivizing commercial providers to make an investment anyway, usually with a package of tax breaks and cost sharing. But many communities have achieved better results even faster by launching their own fiber broadband services that the public can access.

Some states with large rural areas have recognized that solving the rural broadband problem will be costly — almost always more costly than first thought. Such projects often take longer than one hoped, and will require some form of taxpayer matching funds, municipal bonds, public buy-in, or a miraculous sudden investment from a generous cable or phone company. In states with municipal broadband bans, like North Carolina, politicians who support such restrictions believe the cable and phone companies will spontaneously solve the rural internet problem on their own. Such beliefs have stalled rural broadband improvements in many of the states ensnared by such laws, usually tailored to protect a duopoly of the same phone and cable companies that have historically refused to offer adequate broadband service to their rural customers.

Challenges for North Carolina

(Courtesy: North Carolina League of Municipalities – Click image for more information)

North Carolina is a growing state, so a small part of the rural broadband problem may work itself out as population densities increase to a level that crosses that critical ROI threshold. But most rural communities will be waiting years for that to happen. Intransigent phone and cable companies are unlikely to respond positively to local officials seeking better service if it requires a substantial investment. As industry lobbyists will tell you, it is not the job of government to dictate the services of privately owned companies. The Republican majority in North Carolina’s legislature underlines that principle regularly in the form of legislation that reduces regulation and oversight. Many, but not all of those Republicans have also taken a strong strand against the idea of municipalities stepping up to resolve their local broadband challenges by working around problematic cable and phone companies. The ideology that government should never be in the business of competing against private businesses usually takes precedence.

Almost a decade ago, the cable and phone companies of North Carolina made three failed attempts to enshrine this principle in a new statewide law that would limit municipal broadband encroachment to such an extent it made future projects unviable. They succeeded in passing a law on their fourth attempt in 2011, the same year Republicans took control in the state legislature.

Today, Republicans still control the legislature with a Democratic governor providing some checks and balances. Why is this important? Because for North Carolina to achieve its goal, it will realistically need a combination of bipartisan support for rural broadband funding and an end to the municipal broadband ban.

Where is the money?

Although North Carolina wants to be America’s first “gigabit” state, New York is the first to at least claim full broadband coverage across the entire state. That did not and could not happen without a multi-year spending program. Recently, North Carolina’s Department of Information Technology launched a $10 million GREAT Grant program to provide last-mile connectivity to the most economically distressed counties in the state. While a noble effort, and one no doubt limited by the availability of funds to spend on broadband expansion, it is a drop in a bucket of water thrown into a barely filled pool.

To put this problem in better context, New York’s goal of full broadband coverage (which in our view remains incomplete) required not only $500 million acquired from settlement proceeds won by the state after suing Wall Street banks for causing the Great Recession, but another $170 million in federal broadband expansion funds that were expected to be forfeited because Verizon — the state’s largest phone company — was not interested in the money or upgrading their DSL service upstate.

Big Money: New York’s rural broadband funding initiative spent hundreds of millions to attack the rural broadband problem. Gov. Andrew Cuomo outlines funding for just one of several rounds of broadband funding.

Last year, Gov. Andrew Cuomo detailed success for his Broadband for All program by pointing out the state spent $670 million to upgrade or introduce broadband service to 2.42 million locations in rural New York, giving the state 99.9% coverage. That amounts to an average grant of $277 per household or business. In turn, award recipients — largely incumbent phone and cable companies, had to commit to matching private investments. For that state money, the provider had to typically offer at least 100 Mbps service, except in the most rural parts of the state, where a lower speed was acceptable.

North Carolina has 585,000 underserved or unserved locations. Just by using New York’s average $277 grant, North Carolina will have to spend approximately $202 million with similar matching funds from private companies to reach those locations. In fact, it is assuredly more than that because North Carolina’s goal is gigabit speed, not 100 Mbps. Also, New York declared ‘mission accomplished’ while stranding tens of thousands of expensive or difficult to reach residents with subsidized satellite internet access. That offers nothing close to gigabit speed. A more realistic figure for North Carolina in 2019 could be as high as $250-300 million in taxpayer dollars — combined with similar private matching funds to convince AT&T, Charter, CenturyLink and others that the time is right to expand into more rural areas. But as New York discovered, there will be areas in the state no company will bid to serve because the money provided is inadequate.

If the thought of handing over tax dollars to big phone and cable companies bothers you, the alternative is helping communities start and run their own networks in the public interest. Except private providers routinely retaliate with well-funded campaigns of fear, uncertainty and doubt over those projects, and they become political footballs to everyone except their customers, who generally appreciate the service and local accountability.

If North Carolina’s state government relies on a series of $10 million appropriations for grants, it will likely take at least 20 years to wire the state. Stop the Cap! agrees with the goals North Carolina has set to deliver ubiquitous, gigabit-fast broadband. But those goals will be difficult to reach in the present political climate. Republicans in the state legislature approved reductions in the corporate income tax rate to 2.5 percent, down from 3 percent last year, and the personal income tax rate drops to 5.25 percent from 5.499 percent. North Carolina’s latest budget sets aside $13.8 billion for education, $3.8 billion for Medicaid, $3 billion in new debt for road maintenance, and $31 million in grants to attract the film industry to shoot their projects in the state.

It is likely any appropriation significant enough to actually deliver on the commitment to provide total broadband coverage will have to be spread out over several years, unless another funding mechanism can be identified. That assumes the Republicans in the state legislature will be receptive to the idea, which remains an open question.

What do an emergency operations center in Cochise County, Ariz., Colorado hospitals, the Idaho Bureau of Corrections, and many 911 call centers across Massachusetts have in common? They were all brought down by a two-day nationwide CenturyLink outage from Dec. 27-28 that also resulted in internet outages for tens of thousands of CenturyLink’s residential customers. The cause? CenturyLink blamed a single, faulty third-party network management card in Denver for disrupting services for CenturyLink and other phone companies, notably Verizon, from Alaska to Florida.

Hours after outage began, two days after Christmas, CenturyLink issued a general statement:

“CenturyLink experienced a network event on one of our six transport networks beginning on December 27 that impacted voice, IP, and transport services for some of our customers. The event also impacted CenturyLink’s visibility into our network management system, impairing our ability to troubleshoot and prolonging the duration of the outage.”

That “network event” caused serious disruptions to critical services in 37 states, including 911, according to Brian Kyes, president of the Massachusetts Major City Chiefs of Police Association.

“This is affecting 911 (wireline & wireless) delivery to most of Massachusetts,” Kyes said in a statement during the outage to the Boston Herald. “We have heard from MEMA that this issue may also affect some landlines but I have not heard of any specific situations or communities that have been impacted. We are advising all police and fire chiefs to test their local 911 systems and notify their residents of potential issues by reverse 911, social media or any other means that they have at their disposal. The interruption in service may depend on a particular phone carrier and the information that we have is that it may be intermittent.”

CenturyLink outages on Dec. 27, 2018. (Image: Downdetector.com)

The disruptions affected much of Massachusetts — a state served primarily by Verizon Communications, because CenturyLink is a major commercial services vendor inside and outside of its local landline service areas and supplies some connectivity services to Verizon, mostly for wireless customers.

ATM networks also went down in certain parts of the country. CenturyLink is one of many vendors providing data connectivity between the cashpoint machines and several banking institutions.

Also impacted, the Idaho Department of Corrections, including inmate phone systems, and the Idaho Department of Education, which lost the ability to make or receive calls.

Consumers also noticed their internet connections were often down or sporadic in some locations, primarily because CenturyLink’s backbone network became saturated with rogue packets.

The Denver Post presented a more detailed technical explanation about the outage:

CenturyLink said the [defective] card was propagating “invalid frame packets” that were sent out over its secondary network, which controlled the flow of data traffic.

“Once on the secondary communication channel, the invalid frame packets multiplied, forming loops and replicating high volumes of traffic across the network, which congested controller card CPUs (central processing unit) network-wide, causing functionality issues and rendering many nodes unreachable,” the company said in a statement.

Once the syndrome gets going, it can be difficult to trace back to its original source and to stop, a big reason networks are designed to isolate failures early and contain them.

“We have learned through experience about these different types of failure modes. We build our systems to try and localize those failures,” said Craig Partridge, chair of the computer science department at Colorado State University in Fort Collins and a member of the Internet Hall of Fame. “I would hope that what is going on is that CenturyLink is trying to understand why a relatively well-known failure mode has bit them.”

The Federal Communications Commission also expects answers to some questions, opening another investigation of the phone company. In 2015, CenturyLink agreed to pay a $16 million settlement to the federal agency after a seven-state outage in April 2014.

Pai

FCC Chairman Ajit Pai said the agency would once again take a look at CenturyLink, focusing on disruptions to emergency services.

“When an emergency strikes, it’s critical that Americans are able to use 911 to reach those who can help,” Pai said in a statement. “This inquiry will include an examination of the effect that CenturyLink’s outage appears to have had on other providers’ 911 services.”

A retired manager at Qwest, a former Baby Bell now owned by CenturyLink, strongly criticized CenturyLink’s lack of communications with customers and an apparent lack of network redundancy.

“For a company in the communication business, they sure failed on this,” said Albuquerque resident Sam Martin. “I participated on the Qwest Disaster Recovery teams, and I do not recall ever having the network down for this kind of time and certainly never the 911 network. The 911 network should never have been down. The lack of this network can contribute to delays in rescue and fire saving lives.”

Martin is dubious about CenturyLink’s explanation for the network outage, suggesting a defective network card may be only a part of the problem.

“The explanations given so far are not valid,” Martin said. “The public may not be aware of it, but the communication network has redundancy and for essential services like inter-office trunking and 911 calls, there are duplicate fiber optic feeds – “rings” that duplicate the main circuit in another path – and switching equipment to these locations so that they may be switched electronically and automatically upon failure to a back-up network ring. When these systems are operating properly, the customer is unaware a failure occurred. If the automatic switching does not take place, employees involved with disaster recovery can intervene and manually switch the affected network to another fiber ring or electronic hub and service is restored until the actual damage is fixed.”

None of those things appeared to happen in this case, and the outage persisted for 48 hours before all services were restored.

“CenturyLink has to have a disaster recovery plan with redundancies in place for electrical, inbound and outbound local and toll-free carriers, as well as network and hardware component redundancies. CenturyLink should be able to switch between multiple fiber optic rings or central offices in case entire networks of phones go down. They would then locate and repair, or replace, defective telecommunication components without the customer ever knowing. The fact that this did not happen is discouraging and scary for the consumer. The fact that it happened nationwide is even more surprising and disturbing. Hopefully the truth will come out soon.”

A critical editorial in the Albuquerque Journal added:

We need answers from CenturyLink beyond the cryptic “a network element” caused the outage. We need to know how many CenturyLink and Verizon customers were affected. And we need to know what they – and other internet and phone providers – are doing to prevent similar outages or worse from happening in the future. Because if the outage showed nothing else, it’s that like an old-time string of Christmas lights, we are living in an interconnected world.

And when one light goes out, they can all go out.

KTVB in Boise, Idaho reported on CenturyLink’s massive outage on Dec. 27-28 and how it impacted local businesses and government services. (3:09)

Windstream is relying on the Federal Communications Commission’s Connect America Fund to double the areas where it will offer 100 Mbps broadband service, expected to reach 30% of the company’s 18-state local service area by the end of the first quarter of 2019.

“Windstream understands that premium internet speeds are critical to families and businesses in rural America, and we are systematically enhancing our network to meet that urgent demand,” said Jeff Small, president of consumer and small and medium-sized business services. “Network upgrades are expensive, especially in rural areas where there are relatively few customers, so Windstream is using a combination of its own capital and crucial support from the FCC’s Connect America Fund to make faster speeds more widely available. Without support from the Connect America Fund, many of these projects simply would not be economically feasible.”

Thomas told attendees at the Citi 2019 TMT West Conference Windstream’s legacy copper wire telephone network is not up to the job of handling the kinds of internet speeds more modern technologies can manage.

In urban and larger service areas, Windstream is most likely to deploy fiber to the home service in new housing developments and select gentrified neighborhoods where a business case exists to invest in fiber upgrades. The company also typically replaces its copper wireline infrastructure with fiber where road construction projects or damage forces the company to replace or relocate its lines. Suburban and more densely populated rural areas are likely to receive an upgraded version of Windstream’s DSL service that can manage up to 50 or 100 Mbps. In Windstream’s significant rural service area, the phone company is increasingly turning to fixed wireless technology, especially in flat midwestern states like Nebraska and Iowa where it plans to offer a combination of 3.5 GHz “CBRS” and 5G millimeter wave fixed wireless broadband capable of delivering up to 1,000 Mbps.

Windstream’s service area

“[We are deploying wireless internet] probably at a larger scale than a lot of the larger wireless companies,” Thomas said, especially in flatter areas where wireless signals go a long way.

Because most current broadband expansion fund programs require companies to commit to at least 25/3 Mbps service, simply expanding basic ADSL technology has proven inadequate to meet the government’s speed requirements. But wiring fiber to the home service to get faster speeds in rural areas does not meet the Return On Investment requirements Windstream’s shareholders demand. Windstream claims fixed wireless can solve both problems.

“You can get 100 Mbps out there very cost-effectively,” Thomas claimed. “You are really blowing away copper infrastructure and making it irrelevant because you’re embracing this 100 Mbps technology.”

As of early 2019, Windstream claims that 60% of its customers can get at least 25 Mbps service, 40% can receive at least 50 Mbps service. By the end of March, 30% will be able to receive 100 Mbps service.

A satisfied Windstream customer talks about his upgrade to 50/8 Mbps, which replaces his old 6 Mbps DSL service. (6:03)