Home » Broadband "Shortage" »Broadband Speed »Comcast/Xfinity »Consumer News »Net Neutrality »Public Policy & Gov't »Video » Currently Reading:

Comcast Needed Help to Let Them Know Their Broadband Pipes Were Full

The country’s largest cable internet service provider needed help from an app developer in Portland, Ore. to let it know its broadband pipes were full and to do something about it.

Comcast customers were complaining about slow downloads from the Panic website and the company’s own workers were saying largely the same thing when attempting to remotely connect to the company’s servers from home.

Because Panic’s web servers have just a single connection to the internet via Cogent, it would be a simple matter to track down where the traffic bottleneck was occurring, assuming there was one. The company asked for volunteers to run a test transferring 20MB of data first from Panic’s server and then again from a control server hosted with Linode, a popular and well-respected hosting company.

The results were pretty stunning.

With speeds often around only 356.3kbps for Comcast customers connecting to Panic, something was definitely up. It also explained why employees had a rough time connecting to the company’s server as well — Panic’s workers are based in Portland, Ore., where Comcast is used by almost every employee.

The slowdowns were not related to the time of day and because the problem persisted for weeks, it wasn’t a temporary technical fault. Panic’s blog picks up the story about what is behind all this:

Peering.

Major internet pipes, like Cogent, have peering agreements with network providers, like Comcast. These companies need each other — Cogent can’t exist if their network doesn’t go all the way to the end user, and Comcast can’t exist if they can’t send their customer’s data all over the world. One core tenet of peering is that it is “settlement-free” — neither party pays the other party to exchange their traffic. Instead, each party generates revenue from their customers. Cogent generates revenue from us. Comcast generates revenue from us at home. Everyone wins, right?

After a quick Google session, I learned that Cogent and Comcast have quite a storied history. This history started when Cogent started delivering a great deal of video content to Comcast customers… content from Netflix. and suddenly, the “peering pipe” that connects Cogent and Comcast filled up and slowed dramatically down.

Normally when these peering pipes “fill up”, more capacity is added between the two companies. But, if you believe Cogent’s side of the story, Comcast simply decided not to play ball — and refused to add any additional bandwidth unless Cogent paid them. In other words, Comcast didn’t like being paid nothing to deliver Netflix traffic, which competes with its own TV and streaming offerings. This Ars Technica article covers it well. (How did Netflix solve this problem in 2014? Netflix entered into a business agreement to pay Comcast directly. And suddenly, more peering bandwidth opened up between Comcast and Cogent, like magic.)

We felt certain history was repeating itself: the peering connection between Comcast and Cogent was once again saturated. Cogent said their hands were tied. What now?

In addition to giving the internet public policy community new evidence that peering fights leaving customers stuck in the middle might be heating up once again. It also suggests if Comcast was unaware of the problem, it does not reflect well on the cable company to wait weeks until a customer reports such a serious slowdown before fixing it.

The folks at Panic took a chance and reported the problem to Comcast, bypassing the usual customer support route in favor of a corporate contact who listed a direct email address on the company’s website. Comcast took the request seriously and eventually responded, “give us one to two weeks, and if you re-run your test I think you’ll be happy with the results.”

Indeed, the problem was fixed. The folks at Panic say according to Comcast, two primary changes were made:

  1. Comcast added more capacity for Cogent traffic. (As suspected, the pipe was full.)
  2. Cogent made some unspecified changes to their traffic engineering.

The folks at Panic and their users are happy that the problem is fixed, but some questions remain:

  1. Is Comcast intentionally throttling web traffic in an attempt to extract a more favorable peering agreement with Cogent?
  2. How could Comcast not know this particular connection was hopelessly over-capacity for several weeks, leaving customers to deal with heavily throttled traffic.

“While this story amazingly had a happy ending, I’m not looking forward to the next time we’re stuck in the middle of a peering dispute between two companies,” wrote Cabel. “It feels absolutely inevitable, all the more so now that net neutrality is gone. Here’s hoping the next time it happens, the responsible party is as responsive as Comcast was this time.”

Panic explains internet slowdowns resulting from peering disputes in this (3:30) video.







Search This Site:

Contributions:

Recent Comments:

  • Doug Stoffa: Digital takes up way less space than old analog feeds - agreed. In a given 6 MHz block, the cable company can send down 1 NTSC analog station, 2-4 HD...
  • Phillip Dampier: Digital video TV channels occupy next to nothing as far as bandwidth goes. Just look at the huge number of premium international channels loading up o...
  • Doug Stoffa: It's a bit more complicated than that. Television stations (and the networks that provide them programming) have increased their retransmission fees ...
  • Alex sandro: Most of the companies offer their services with contracts but Spectrum cable company offer contract free offers for initial year which is a very good ...
  • John: I live in of the effected counties, believe it or not our village is twenty three miles from WSKG Tower, approxiamately eighty miles from Syracuse, WS...
  • Wilhelm: I'm in the Finger Lakes where Spectrum removed WROC-8 last Fall, but we still get other Rochester channels, WHAM-13, WHEC-10 and WXXI-21. I have to wo...
  • dhkjsalhf: "Another classic case of businesses being much smarter than governments." I don't know whether this was sarcastic or not, but I feel it's a sentiment...
  • New Yorker: It makes no sense. I wonder sometimes if raising the limits on how much money rich people giving to candidates could make it more expensive to buy of...
  • New Yorker: Will New York go through with the threat? As an upstater I have seen infrastructure projects drag on in cost and time (eg. 1.5 yrs to repair a tiny b...
  • Matthew H Mosher: Another classic case of businesses being much smarter than governments....
  • Matthew H Mosher: Doesn't matter. Rural NY will remain left behind....
  • Dylan: Hopefully this does not happen as I would like to see Charter continue with its current plans of upgrades in NY, like the 200mbps upgrade. Maybe Chart...

Your Account: