Some Comcast customers are receiving notifications their service may be briefly interrupted as a result of ongoing network enhancements:

As you can imagine, customers are using our services more than usual, and in certain neighborhoods we are adding capacity to improve our network and your service now. Your area has been identified as a location that will benefit from network improvements, which is why it’s necessary for us to complete the work at this time, so you’ll continue to connect and enjoy the services you expect.

We are installing more fiber throughout our network by replacing some coaxial cable with fiber lines. This will include some construction in your neighborhood and will cause some intermittent interruptions to your service during the day.

In the future, we’ll be able to offer you even more reliable service, greater network capacity and more gig speeds to more homes down the line. This means more downloading, more streaming and more gaming.

Comcast is currently preparing its network for full duplex DOCSIS 4.0 service by adopting “node plus zero” network architecture, which in plainer terms means removing signal amplifiers from existing lines and replacing a significant percentage of its copper coaxial cable infrastructure with fiber optics. The fiber network upgrades are reportedly bringing more consistent and faster internet speed on a more reliable network as soon as the switch is complete. Most customers won’t see fiber cables replacing the current cable line coming into their homes, however. The upgrade is mostly taking place on utility poles and, in some places, underground as Comcast replaces copper wiring with optical fiber cables.

Sometime next year, Comcast is expected to start adopting Full Duplex DOCSIS (FDX) 4.0, the newest cable broadband standard, capable of bringing identical upload and download speeds to customers. It is the only major American cable operator to favor FDX, which will place upstream and downstream traffic on the same block of cable spectrum. Other cable operators including Charter Spectrum, Cox, and Mediacom will support Extended Spectrum DOCSIS (ESD) 4.0, which is expected to be cheaper to deploy and will continue to keep downstream and upstream traffic confined to separate frequency bands, albeit larger ones capable of delivering much faster speeds.

With many cable operators reporting a need to double network capacity every 18-24 months to keep up with customer traffic demands, the industry is spending time and money contemplating how to meet future needs while also finding ways to cut costs and make networks more efficient.

Top technology executives from five major cable operators answered questions (sub. req’d.) from Multichannel News about their current broadband networks and their plans for the future. Some, like Mediacom, are aggressively adopting DOCSIS 3.1 cable broadband upgrades for their customers while companies like Cox and Comcast are deploying multiple solutions that use both traditional hybrid fiber-coax network technology and, on occasion, fiber-to-the-home to boost speed and performance. But at least one cable company — Charter Communications — thinks it can continue operating its existing DOCSIS 3 network without major upgrades for several years to come.

Cable Broadband Traffic Can Be Handled

“We’ve been on a pretty steady path of doubling our network capacity every 18-24 months for several years, and I don’t see anything that makes me think that will change,” said Tony Werner, president of technology and product at Comcast. “We’ve been strategically extending fiber further into our network to meet customer demand, and that effort, combined with our commitment to deploying DOCSIS 3.1 has given us a network that’s powerful, flexible, and ready for what’s next.”

J.R. Walden, senior vice president of technology at Mediacom was more aggressive.

“We have completed the removal of all the analog channels. That was the big step one,” Walden said. “Step two was to start transitioning high-speed data over to DOCSIS 3.1, so we’re not adding any more 3.0 channels, and reuse spectrum for 3.1, which is a bit more efficient. The whole company is 3.1, all the modems we’re buying since June have been 3.1, so we’ve begun that next transition.”

Walden added Mediacom is also trying to improve broadband performance by reducing the number of customers sharing the same connection.

“We average about 285 homes to 290 homes per node as an average,” he said.

Mediacom is also scrapping older technology on the TV side to open new bandwidth. The cable company is getting rid of MPEG-2-only set-top boxes so the company can transition its video lineup to MPEG-4. But even that won’t last long. Walden admits the company will then quickly start moving less-viewed channels and some premium networks to IP delivery.

Traditional cable broadband service relies on a hybrid fiber-coax network.

In its European markets, Liberty Global has adopted Converged Cable Access Platform (CCAP) equipment across its footprint. CCAP technology saves cable operators space and operates more efficiently, and supports future convergence of technologies that cable operators want to adopt in the future. CCAP has helped Liberty Global deal with its 45% traffic growth by making upgrades easier. The company is also using advanced features of CCAP to better balance how many customers are sharing a connection. The next step is adopting DOCSIS 3.1.

“Seventy to 80% of our plant will be DOCSIS 3.1 ready by the end of next year, giving us a path to even greater capacity expansion allowing us to continue to increase the available capacity across our access network, upstream and downstream,” said Dan Hennessy, chief architect of network architecture for Liberty.

Charter is prioritizing maximizing performance on the network it already has.

“Our priority is to constantly balance capacity against demand. It’s a never-ending quest,” said Jay Rolls, Charter’s chief technology officer. “We watch it very closely, and we’re very pragmatic about it — the volume of tools, metrics and ways to see what’s really happening, and invest accordingly, is really deepening in ways that matter.”

Is Fiber-to-the-Home in Your Future?

While some cable operators like Altice’s Cablevision are scrapping their existing hybrid fiber-coax networks in favor of fiber-to-the-home (FTTH), America’s largest cable operators are not in any hurry to follow Altice.

Comcast has expanded its fiber network closer to customers in the last few years, but sees no need to convert customers to FTTH service.

“I feel pretty strongly that the best path ahead is to leverage the existing coaxial network and DOCSIS resources to the fullest, then inch towards FTTH, over time Why? Because we can. We don’t have to build an entire network just to turn up one customer.”

The next generation of cable broadband service may depend on CCAP – technology that will cut operator costs and lay the foundation for changing the way video and other services are delivered to customers.

Cox has a 10-year Network 2.0 plan that will bring fiber closer to customers, but not directly to every home. More important to Cox is having the option to support symmetrical speeds, which means delivering upload speeds as fast as download speeds. In the meantime, network cabling Greensboro can improve your current connectivity and reliability, preparing your network for high-speed internet.

“We’re also thinking about the fiber investment and fiber deep as it relates to our wireless strategy, enabling some of our customers with a small cell strategy but also positioning ourselves to take advantage of that in the future, as well as thinking about fiber deep to benefit both residential and our commercial customers simultaneously,” said Kevin Hart, Cox’s executive vice president and chief product and technology officer.

Liberty/Virgin Media’s Project Lightning is bringing cable broadband and TV service to places in the UK that never had cable service before.

In Europe, Liberty Global’s “Project Lightning” network expansion initiative is building out traditional cable service in the United Kingdom. Most of the UK never adopted cable service, favoring small satellite dish service instead. Now Liberty Global is putting cable expansion on its priority list. But decades after most North Americans got cable service for the first time, today’s new buildouts are based largely on fiber optics — either fiber to the home or fiber to the neighborhood, where coaxial cable completes the journey to a customer’s home.

Charter admits the technology it will use in the future partly depends on what the competition is offering. Rolls says the company can eventually roll out DOCSIS 3.1, take fiber deeper, or offer symmetrical download/upload speeds presumably targeted towards its commercial customers. But he also suggested Charter’s existing network can continue to deliver acceptable levels of service without spending a lot on major upgrades.

“It’s a rational approach, where we’re trying to balance the needs, the available technologies, and the costs,” Rolls said. But he also suggested DOCSIS 3.1 isn’t always the answer to upgrades. “DOCSIS 3.1 has some pretty remarkable capabilities, but it’s not necessarily a hard-and-fast reason to not take fiber deeper, for instance [allowing for additional DOCSIS 3 node splits]. Different situations drive different capacity decisions.”

Walden agreed, and Mediacom customers should not expect more than DOCSIS 3.1 upgrades for the near future.

“[Fiber deep] is a bit further out, at least as a large-scale type of project,” Walden told Multichannel News. “I think fiber deep for multi-dwelling units, high-density areas and some planned higher end communities doing deeper fiber or fiber-to-the-home [is happening]. But as a wholesale [change] and going to node+0 kind of architecture, I don’t see that in the next two years.”

Are Symmetrical Speeds Important for Customers?

Verizon’s fiber to the home service FiOS uses symmetrical broadband speeds to its advantage in the marketplace.

Many fiber to the home networks offer customers identical upload and download speeds, but cable broadband was designed to favor downstream speeds over upstream. That decision was based on the premise the majority of users will receive much more traffic than they send. But as the internet evolves, some are wondering if cable broadband’s asymmetric design is now outdated and some competitors like Verizon’s FiOS fiber to the home service now use its symmetrical speed advantage as a selling point.

Cox Communications does not think most customers care, even though its network upgrades are laying the foundation to deliver symmetrical speeds.

“It’s a little but further out on the horizon,” said Hart. “The upstream growth rate is ticking up a couple of notches, but not to the tune that we would need significant additional capacity and/or a complementary need for symmetrical bandwidth. [A]t this stage, the symmetrical is a nice-to-have for residential and definitely will be a good option for our commercial customers.”

Rolls isn’t sure if symmetrical speeds are important to customers either and Charter has no specific plans to move towards upload speed upgrades.

“The world of applications and services continues to evolve, obviously, but so far we’ve been able to meet those needs with an asymmetrical topology,” Rolls said. “That said, things like real-time gaming, augmented and virtual reality, and the Internet of Things — some of those will likely drive more symmetry in the network. It remains to be seen.”

Verizon Wireless, facing scrutiny from FCC chairman Thomas Wheeler, today announced it has canceled plans to introduce a new “network optimization” policy that would have significantly throttled down speeds for heavy users still on grandfathered, unlimited use data plans.

Stop the Cap! received a statement from Verizon Wireless this afternoon announcing a sudden change of heart:

Verizon is committed to providing its customers with an unparalleled mobile network experience.  At a time of ever-increasing mobile broadband data usage, we not only take pride in the way we manage our network resources, but also take seriously our responsibility to deliver exceptional mobile service to every customer.  We’ve greatly valued the ongoing dialogue over the past several months concerning network optimization and we’ve decided not to move forward with the planned implementation of network optimization for 4G LTE customers on unlimited plans.  Exceptional network service will always be our priority and we remain committed to working closely with industry stakeholders to manage broadband issues so that American consumers get the world-class mobile service they expect and value.

Chairman Wheeler questioned Verizon’s strategy almost immediately after the company announced its “network optimization” strategy in July.

Wheeler

“‘Reasonable network management’ concerns the technical management of your network; it is not a loophole designed to enhance your revenue streams,” Wheeler wrote in a July 30 letter to Verizon Wireless CEO Dan Mead. “It is disturbing to me that Verizon Wireless would base its ‘network management’ on distinctions among its customers’ data plans, rather than on network architecture or technology.”

Wheeler reminded Mead the FCC defined network management practices to be reasonable “if it is appropriate and tailored to achieving a legitimate network management purpose, taking into account the particular network architecture and technology of the broadband Internet access service.”

Wheeler told Mead Verizon’s plans didn’t qualify.

“I know of no past FCC statement that would treat as ‘reasonable network management’ a decision to slow traffic to a user who has paid, after all, for ‘unlimited” service,'” Wheeler wrote.

Wheeler also questioned how Verizon could justify its planned speed throttling under the conditions it agreed to after winning the 700MHz “C Block.” That spectrum was accompanied by a special FCC mandate – open platform rules which prohibits Verizon Wireless from denying, limiting, or restricting the ability of end users to download and use applications of their choosing on the C Block networks. A speed throttle would make using some applications impossible.

In August, Wheeler hammered home his opposition to Verizon’s plans at a news conference.

“My concern in this instance–and it’s not just with Verizon, by the way, we’ve written to all the carriers–is that [network management] is moving from a technology and engineering issue to a business issue, such as choosing between different subscribers based on your economic relationship with them.”

Wheeler has expressed irritation that Verizon’s justification for congestion management only applied to its unlimited customers, while those paying on a per-gigabyte basis could use (and spend) as much as they like.

Verizon responded that other providers — notably AT&T — already have a similar network management policy in place, throttling speeds of grandfathered unlimited customers who consume more than 3GB of wireless traffic on its 3G network or 5GB on its 4G network a month.

“‘All the kids do it’ was never something that worked with me when I was growing up and didn’t work with my kids,” Wheeler responded, noting Verizon was trying to reframe the issue instead of justifying the need for speed throttles for some customers, while giving others unlimited access as long as they pay.

Level 3’s global network: Orange lines represent Level 3-owned infrastructure, yellow lines show leased or co-owned connections.

Five of the largest Internet Service Providers in the country are intentionally sabotaging your use of the Internet by allowing their network connections to degrade unless they receive extra compensation from content companies they often directly compete with.

Mark Taylor, vice president of content and media for Level 3, wrote a lengthy primer on how Internet providers exchange traffic with each other across a vast global network. While clients of Level 3 are likely to have few problems exchanging traffic back and forth across Level 3’s global network, vital interconnections with other providers that make sure everyone can communicate with everyone else on the Internet are occasional trouble spots.

Every provider has different options to reach other providers, but favor those offering the most direct route possible to minimize “hops” between networks, which slow down the connection and increase the risk of service interruptions. These connections are often arranged through peering agreements. Level 3 has 51 peers, minimized in number to keep traffic moving as efficiently as possible.

This oversaturated port in Dallas cannot handle all the traffic trying to pass through it, so Internet packets are often dropped and traffic speeds are slowed.

Taylor writes most peering arrangements were informal agreements between engineers and did not involve any money changing hands. Today, 48 of the 51 Level 3 peering agreements don’t involve compensation. In fact, Level 3 refuses to pay “arbitrary charges to add interconnection capacity.” Taylor feels such upgrades are a matter of routine and are not costly for either party.

Peering agreements have been a very successful part of the Internet experience, even if end users remain completely in the dark about how Internet traffic moves around the world. In the view of many, customers don’t need to know and shouldn’t care, because their monthly Internet bill more than covers the cost of transporting data back and forth.

Because of ongoing upgrades the average utilization of Level 3’s connections is around 36 percent of capacity — busy enough to justify keeping the connection and providing spare capacity for days when Internet traffic explodes during breaking news or over the holidays.

However, Taylor says more than a year ago, something suddenly changed at five U.S. Internet Service Providers. They stopped periodic upgrades and allowed some of their connections to become increasingly busy with traffic. Today, six of Level 3’s 51 peer connections are now 90 percent saturated with traffic for several hours a day, which causes traffic to degrade or get lost.

“[The] congestion [has become] permanent, has been in place for well over a year and […] our peer refuses to augment capacity,” Taylor wrote. “They are deliberately harming the service they deliver to their paying customers. They are not allowing us to fulfill the requests their customers make for content.”

Taylor adds all but one of the affected connections are U.S. consumer broadband networks with a dominant or exclusive market share. Where competition exists, no provider allows their Internet connections to degrade, said Taylor.

Taylor won’t directly name the offenders, but he left an easy-to-follow trail:

“The companies with the congested peering interconnects also happen to rank dead last in customer satisfaction across all industries in the U.S.,” Taylor wrote. “Not only dead last, but by a massive statistical margin of almost three standard deviations.”

Taylor footnotes the source for his rankings, the American Consumer Satisfaction Index. The five worse providers listed for consumer satisfaction:

• Comcast
• Time Warner Cable
• Charter Communications
• Cox Communications
• Verizon

AT&T has also made noises about insisting on compensation for its own network upgrades, blaming Netflix traffic.

In fact, Netflix traffic seems to be a common point of contention among Internet Service Providers that also sell their own television packages. They now insist the streaming video provider establish direct, paid connections with their networks. Level 3 is affected because it carries a substantial amount of traffic on behalf of Netflix.

Ultimately, the debate is about who pays for network upgrades to keep up with traffic growth. Taylor says Level 3’s cost to add an extra 10Gbps port would be between $10-20 thousand dollars, spare change for multi-billion dollar Americans cable and phone companies. Normally, competition would never allow a traffic dispute like this interfere with a customer’s usage experience. Angry customers would simply switch providers. But the lack of competition prevents this from happening in the United States, leaving customers in the middle.

This leaves Taylor with a question: “Shouldn’t a broadband consumer network with near monopoly control over their customers be expected, if not obligated, to deliver a better experience than this?”

AT&T says it plans to adopt fiber to the home service in cities around the United States as part of an expansion of its U-verse GigaPower service.

CEO Randall Stephenson told investors at the Morgan Stanley Technology, Telecom, and Media Conference the “cost dynamics” of fiber optics have become “really encouraging” in its targeted fiber deployment in Austin, Tex.

“In fact I would tell you we are so encouraged that we want to begin taking this to other communities [where] we can get the terms and conditions like we have in Austin,” Stephenson said, referring to Austin’s red-tape cutting and clearing the way for fiber upgrades with eased permit requirements and pole attachment policies. “We are redirecting investment to fiber to the home deployment, and in fact we are going to launch the service in Dallas this summer.”

Stephenson added that where U-verse faces significant competition from a “new competitor,” AT&T will be “a little more aggressive and assertive in deploying that technology around the country.”

That most likely means AT&T will choose fiber to the home service in areas facing imminent competition from Google Fiber or another similar provider.