Home » Broadband "Shortage" »Broadband Speed »Comcast/Xfinity »Consumer News »Net Neutrality »Public Policy & Gov't »Video » Currently Reading:

Comcast Needed Help to Let Them Know Their Broadband Pipes Were Full

The country’s largest cable internet service provider needed help from an app developer in Portland, Ore. to let it know its broadband pipes were full and to do something about it.

Comcast customers were complaining about slow downloads from the Panic website and the company’s own workers were saying largely the same thing when attempting to remotely connect to the company’s servers from home.

Because Panic’s web servers have just a single connection to the internet via Cogent, it would be a simple matter to track down where the traffic bottleneck was occurring, assuming there was one. The company asked for volunteers to run a test transferring 20MB of data first from Panic’s server and then again from a control server hosted with Linode, a popular and well-respected hosting company.

The results were pretty stunning.

With speeds often around only 356.3kbps for Comcast customers connecting to Panic, something was definitely up. It also explained why employees had a rough time connecting to the company’s server as well — Panic’s workers are based in Portland, Ore., where Comcast is used by almost every employee.

The slowdowns were not related to the time of day and because the problem persisted for weeks, it wasn’t a temporary technical fault. Panic’s blog picks up the story about what is behind all this:

Peering.

Major internet pipes, like Cogent, have peering agreements with network providers, like Comcast. These companies need each other — Cogent can’t exist if their network doesn’t go all the way to the end user, and Comcast can’t exist if they can’t send their customer’s data all over the world. One core tenet of peering is that it is “settlement-free” — neither party pays the other party to exchange their traffic. Instead, each party generates revenue from their customers. Cogent generates revenue from us. Comcast generates revenue from us at home. Everyone wins, right?

After a quick Google session, I learned that Cogent and Comcast have quite a storied history. This history started when Cogent started delivering a great deal of video content to Comcast customers… content from Netflix. and suddenly, the “peering pipe” that connects Cogent and Comcast filled up and slowed dramatically down.

Normally when these peering pipes “fill up”, more capacity is added between the two companies. But, if you believe Cogent’s side of the story, Comcast simply decided not to play ball — and refused to add any additional bandwidth unless Cogent paid them. In other words, Comcast didn’t like being paid nothing to deliver Netflix traffic, which competes with its own TV and streaming offerings. This Ars Technica article covers it well. (How did Netflix solve this problem in 2014? Netflix entered into a business agreement to pay Comcast directly. And suddenly, more peering bandwidth opened up between Comcast and Cogent, like magic.)

We felt certain history was repeating itself: the peering connection between Comcast and Cogent was once again saturated. Cogent said their hands were tied. What now?

In addition to giving the internet public policy community new evidence that peering fights leaving customers stuck in the middle might be heating up once again. It also suggests if Comcast was unaware of the problem, it does not reflect well on the cable company to wait weeks until a customer reports such a serious slowdown before fixing it.

The folks at Panic took a chance and reported the problem to Comcast, bypassing the usual customer support route in favor of a corporate contact who listed a direct email address on the company’s website. Comcast took the request seriously and eventually responded, “give us one to two weeks, and if you re-run your test I think you’ll be happy with the results.”

Indeed, the problem was fixed. The folks at Panic say according to Comcast, two primary changes were made:

  1. Comcast added more capacity for Cogent traffic. (As suspected, the pipe was full.)
  2. Cogent made some unspecified changes to their traffic engineering.

The folks at Panic and their users are happy that the problem is fixed, but some questions remain:

  1. Is Comcast intentionally throttling web traffic in an attempt to extract a more favorable peering agreement with Cogent?
  2. How could Comcast not know this particular connection was hopelessly over-capacity for several weeks, leaving customers to deal with heavily throttled traffic.

“While this story amazingly had a happy ending, I’m not looking forward to the next time we’re stuck in the middle of a peering dispute between two companies,” wrote Cabel. “It feels absolutely inevitable, all the more so now that net neutrality is gone. Here’s hoping the next time it happens, the responsible party is as responsive as Comcast was this time.”

Panic explains internet slowdowns resulting from peering disputes in this (3:30) video.







Search This Site:

Contributions:

Recent Comments:

  • Bob: I also got $1 increase for my 100/10 internet from Spectrum. A rep said it's for the speed increase that's coming in 2019. I complained that I was pro...
  • EJ: It makes sense to focus on wireless considering the government contract they have. The strange thing is they referenced fixed wireless in this article...
  • nick: Interesting how they conveniently leave out (Spectrum TV Choice) streaming service which is also $30/mo ($25/mo for the first 2 years)....
  • krichnercom: That's not the only way they are "fleecing the customers" if you leave them they will make it very difficult to return the equipment if your in a rura...
  • Mike: I have used Converter boxes since 2009 changeover. Yes I have gone through a few as some brands have heat issues, etc. but used boxes are found at Goo...
  • Dylan: They got to pay for that 5G somehow without demonstrating to Wallstreet that it will cost more than what they want it to cost. So you got to cut costs...
  • Aardvark: Is the loss of channels mostly in more rural areas? Anecdotally that would appear to be the case. Here in the NYC metro ...
  • Mick Allen: My first thought was that the folks who have lost reception should rescan their converter boxes. After giving it a little thought I decided to ask a ...
  • David: I'm in Temple City, CA (91780) & my rate went up $1 with my October statement. I only have their internet service (100 mbps). They never sent noti...
  • ADubb: I completely agree with the comment above. I go over my limit each month, have a few Nest cams and cloud back up devices on my home network. Something...
  • Paul Houle: I think AT&T is more interested in fixed wireless using low band spectrum in exurban areas. At my location Unlimitedville (AT&T) was able to ...
  • EJ: How about investing in some CAPEX. Identify and fix problem areas. O wait Spectrum don't need to do that. If I had to guess my bottom dollar I would b...

Your Account: