Phillip “Speaking as a Customer” Dampier

Two defenders of large Internet Service Providers are coming to the defense of the broadband industry by questioning a Guardian article that reported major Internet Service Providers were intentionally allowing a degradation in performance of Content Delivery Networks and other high volume Internet traffic in a dispute over money.

Richard Bennett and Dan Rayburn today both published articles attempting to discredit Battle for the Net’s effort highlighting the impact interconnection disputes can have on consumers.

Rayburn:

On Monday The Guardian ran a story with a headline stating that major Internet providers are slowing traffic speeds for thousands of consumers in North America. While that’s a title that’s going to get a lot of people’s attention, it’s not accurate. Even worse, other news outlets like Network World picked up on the story, re-hashed everything The Guardian said, but then mentioned they could not find the “study” that The Guardian is talking about. The reason they can’t find the report is because it does not exist.

[…] Even if The Guardian article was trying to use data collected via the BattlefortheNet website, they don’t understand what data is actually being collected. That data is specific to problems at interconnection points, not inside the last mile networks. So if there isn’t enough capacity at an interconnection point, saying ISPs are “slowing traffic speeds” is not accurate. No ISP is slowing down the speed of the consumers’ connection to the Internet as that all takes place inside the last mile, which is outside of the interconnection points. Even the Free Press isn’t quoted as saying ISPs are “slowing” down access speed, but rather access to enough capacity at connection points.

Bennett:

In summary, it appears that Battle for the Net may have cooked up some dubious tests to support their predetermined conclusion that ISPs are engaging in evil, extortionate behavior.

It may well be the case that they want to, but AT&T, Verizon, Charter Cable, Time Warner Cable, Brighthouse, and several others have merger business and spectrum auction business pending before the FCC. If they were manipulating customer experience in such a malicious way during the pendency of the their critical business, that would constitute executive ineptitude on an enormous scale. The alleged behavior doesn’t make customers stick around either.

I doubt the ISPs are stupid enough to do what the Guardian says they’re doing, and a careful examination of the available test data says that Battle for the Net is actually cooking the books. There is no way a long haul bandwidth and latency test says a thing about CDN performance. Now it could be that Battle for the Net has as a secret test that actually measures CDNs, but if so it’s certainly a well-kept one. Stay tuned.

The higher line measures speeds received by Comcast customers connecting to websites handled by GTT in Atlanta. The lower line represents speeds endured by AT&T customers, as measured by MLab.

Stop the Cap! was peripherally mentioned in Rayburn’s piece because we originally referenced one of the affected providers as a Content Delivery Network (CDN). In fact, GTT is a Tier 1 IP Network, providing service to CDNs, among others — a point we made in a correction prompted by one of our readers yesterday.

Both Rayburn and Bennett scoff at Battle for the Net’s methodology, results, and conclusion your Internet Service Provider might care more about money than keeping customers satisfied with decent Internet speeds. Bennett alludes to the five groups backing the Battle for the Net campaign as “comrades” and Rayburn comes close to suggesting the Guardian piece represented journalistic malpractice.

Much was made of the missing “study” that the Guardian referenced in its original piece. Stop the Cap! told readers in our original story we did not have a copy to share either, but would update the story once it became available.

We published our own story because we were able to find, without much difficulty, plenty of raw data collected by MLab from consumers conducting voluntary Internet Health Tests, on which Battle for the Net drew its conclusions about network performance. A review of that data independently confirmed all the performance assertions made in the Guardian story, with or without a report. There are obvious and undeniable significant differences in performance between certain Internet Service Providers and traffic distribution networks like GTT.

So let’s take a closer look at the issues Rayburn and Bennett either dispute or attempt to explain away:

1. MLab today confirmed there is a measurable and clear problem with ISPs serving around 75% of Americans that apparently involves under-provisioned interconnection capacity. That means the connection your ISP has with some content distributors is inadequate to handle the amount of traffic requested by customers. Some very large content distributors like Netflix increasingly use their own Content Delivery Networks, while others rely on third-party distributors to move that content for them. But the problem affects more than just high traffic video websites. If Stop the Cap! happens to reach you through one of these congested traffic networks and your ISP won’t upgrade that connection without compensation, not only will video traffic suffer slowdowns and buffering, but so will traffic from every other website, including ours, that happens to be sent through that same connection.

MLab: “Customers of Comcast, Time Warner Cable, and Verizon all saw degraded performance [in NYC] during peak use hours when connecting across transit ISPs GTT and Tata. These patterns were most dramatic for customers of Comcast and Verizon when connecting to GTT, with a low-speed of near 1 Mbps during peak hours in May. None of the three experienced similar problems when connecting with other transit providers, such as Internap and Zayo, and Cablevision did not experience the same extent of problems.”

Our initial findings show persistent performance degradation experienced by customers of a number of major access ISPs across the United States during the first half of 2015. While the ISPs involved differ, the symptoms and patterns of degradation are similar to those detailed in last year’s Interconnections study: decreased download throughput, increased latency and increased packet loss compared to the performance through different access ISPs in the same region. In nearly all cases degradation was worse during peak use hours. In last year’s technical report, we found that peak-hour degradation was an indicator of under-provisioned interconnection capacity whose shortcomings are only felt when traffic grows beyond a certain threshold.

Patterns of degraded performance occurred across the United States, impacting customers of various access ISPs when connecting to measurement points hosted within a number of transit ISPs in Atlanta, Chicago, Los Angeles, New York, Seattle, and Washington, D.C. Many of these access-transit ISP pairs have not previously been available for study using M-Lab data. In September, 2014, several measurement points were added in transit networks across the United States, making it possible to measure more access-transit ISP interconnection points. It is important to note that while we are able to observe and record these episodes of performance degradation, nothing in the data allows us to draw conclusions about who is responsible for the performance degradation. We leave determining the underlying cause of the degradation to others, and focus solely on the data, which tells us about consumer conditions irrespective of cause.

Rayburn attempts to go to town highlighting MLab’s statement that the data does not allow it to draw conclusions about who is responsible for the traffic jam. But any effort to extend that to a broader conclusion the Guardian article is “bogus” is folly. MLab’s findings clearly state there is a problem affecting the consumer’s Internet experience. To be fair, Rayburn’s view generally accepts there are disputes involving interconnection agreements, but he defends the current system that requires IP networks sending more traffic than they return to pay the ISP for a better connection.

Rayburn’s website refers to him as “the voice of industry.”

1. Rayburn comes to the debate with a different perspective than ours. Rayburn’s website highlights the fact he is the “voice of the industry.” He also helped launch the industry trade group Streaming Video Alliance, which counts Comcast as one of its members. Anyone able to afford the dues for sponsor/founding member ($25,000 annually); full member ($12,500); or supporting member ($5,500) can join.

Stop the Cap! unreservedly speaks only for consumers. In these disputes, paying customers are the undeniable collateral damage when Internet slowdowns occur and more than a few are frequently inconvenienced by congestion-related slowdowns.

It is our view that allowing paying customers to be caught in the middle of these disputes is a symptom of the monopoly/duopoly marketplace broadband providers enjoy. In any industry where competition demands a provider deliver an excellent customer experience, few would ever allow these kinds of disputes to alienate customers. In Atlanta, Los Angeles, and Chicago, for example, AT&T has evidently made a business decision to allow its connections with GTT to degrade to just a fraction of the performance achieved by other providers. Nothing else explains consistent slowdowns that have affected AT&T U-verse and DSL customers for months on end that involve GTT while Comcast customers experience none of those problems.

We also know why this is happening because AT&T and GTT have both confirmed it to Ars Technica, which covered this specific slowdown back in March. As is always the case about these disputes, it’s all about the money:

AT&T is seeking money from network operators and won’t upgrade capacity until it gets paid. Under its peering policy, AT&T demands payment when a network sends more than twice as much traffic as it receives.

“Some providers are sending significantly more than twice as much traffic as they are receiving at specific interconnection points, which violates our peering policy that has been in place for years,” AT&T told Ars. “We are engaged in commercial-agreement discussions, as is typical in such situations, with several ISPs and Internet providers regarding this imbalanced traffic and possible solutions for augmenting capacity.”

Missing from this discussion are AT&T customers directly affected by slowdowns. AT&T’s attitude seems uninterested in the customer experience and the company feels safe stonewalling GTT until it gets a check in the mail. It matters less that AT&T customers have paid $40, 50, even 70 a month for high quality Internet service they are not getting.

In a more competitive marketplace, we believe no ISP would ever allow these disputes to impact paying subscribers, because a dissatisfied customer can cancel service and switch providers. That is much less likely if you are an AT&T DSL customer with no cable competition or if your only other choice cannot offer the Internet speed you need.

1. Consolidating the telecommunications industry will only guarantee these problems will get worse. If AT&T is allowed to merge with DirecTV and expand Internet service to more customers in rural areas where cable broadband does not reach, does that not strengthen AT&T’s ability to further stonewall content providers? Of course it does. In fact, even a company the size of Netflix eventually relented and wrote a check to Comcast to clear up major congestion problems experienced by Comcast customers in 2014. Comcast could have solved the problem itself for the benefit of its paying customers, but refused. The day Netflix’s check arrived, problems with Netflix magically disappeared.

More mergers and more consolidation does not enhance competition. It entrenches big ISPs to play more aggressive hardball with content providers at the expense of consumers.

Even Rayburn concedes these disputes are “not about ‘fairness,’ it’s business,” he writes. “Some pay based on various business terms, others might not. There is no law against it, no rule that prohibits it.”

Battle for the Net’s point may be that there should be.

The village of Ogden, Ill., no longer has an official website. Comcast deleted it without warning.

Like many of America’s smallest towns and villages, Ogden relies on web space from its Internet Service Provider to host the community government’s website.

Except, as Mayor Jack Reiner reports, Comcast no longer supports business or government websites hosted through a broadband account and it simply disappeared one day.

The village is now attempting to reconstruct the site and considering the cost of alternative providers.

Friday is the last day to submit your views on Net Neutrality with the Federal Communications Commission. Although there may be some future opportunities to comment, it’s important to make your voice heard with the FCC today. Almost 650,000 Americans have done so to date, and we need to see this number rise even higher to combat the influence and power of Big Telecom companies looking to turn the Internet into a corporate toll booth.

If you recall, FCC chairman Tom Wheeler is promoting a scheme where big ISPs like Verizon, AT&T and Comcast can divide up the Internet and introduce toll lanes allowing preferred paid traffic to travel on the Internet at faster speeds, usually at the expense of unpaid traffic that will get relegated to an Internet slow lane. It’s pay to play, and customers of these ISPs are already getting a preview of the new corporate road map for the net. Netflix viewers on ISPs that don’t have a paid agreement to handle video traffic suffer from rebuffering and lower quality video. But ISPs collecting tolls from Netflix don’t subject their customers to a degraded online video experience. Of course, before ISPs realized they could make money selling fast lanes, Netflix worked fine on virtually all of these providers.

Wheeler’s proposal would extend the two-tiered Internet to other websites and service providers, allowing big telecom companies to hand-pick winners and losers and discriminate in favor of their own Internet traffic. Comcast does that today with online video on certain game consoles. If that video comes from Comcast, it doesn’t count against any usage caps. If it doesn’t, it could get rough sticking to Comcast’s arbitrary usage allowance.

The FCC is in way over its head, unaware of the creative ways ISPs can find loopholes large enough to drive through any well-intentioned consumer protections. There is only once certain way to keep ISPs honest — reclassify them as what they should have been all along – a telecommunications service subject to common carrier rules. That would guarantee ISPs could not meddle with your Internet service for financial gain, could not artificially slow down “non-preferred” traffic to make room for paid traffic, and would guarantee that Internet applications of the future will succeed or fail on their merits, not on how much money they are willing to spend.

Since the FCC website is jammed today, we recommend e-mailing the Commission by this Friday at: [email protected]

Our friends at Free Press have published some sample comments they are getting, which may help you formulate yours. Here is ours as well:

Dear Chairman Wheeler,

Although we believe your intentions are good, your proposed Net Neutrality rules simply do not afford enough protection to preserve a free and open Internet. Troubling signs are already clear as providers test how much they can get away with meddling with Internet traffic. The wireless experience is replete with examples of selective speed throttling, usage caps, and traffic discrimination that allows some content to escape the usage meter and throttle while competitors cannot.

The Internet is a transformative experience for many Americans because for the first time in a long time, entrepreneurs can build online businesses that are judged on their merit, not on how much money they have to spend to achieve and maintain prominence. Anything that allows an ISP to collect additional funds for a “preferred” traffic lane will come at the detriment of others who have to share the same broadband pipe. This is especially evident in the wireless world, which escaped even the light touch regulatory framework of your predecessor. Providers promptly began creating new schemes to further monetize growing data traffic, bandwidth shortage or not. Almost none of these changes really benefit customers — they are simply new revenue-making schemes.

A foreshadowing of what is likely to happen under your proposal is also apparent with Comcast and Netflix. For several years subscribers had no trouble accessing online video. But when the issue of traffic compensation was reintroduced by Internet Service Providers, the upgrades to manage natural Internet growth largely stopped and the Netflix viewing experience on these ISPs deteriorated. Verizon, AT&T and Comcast all argue that a paid traffic deal would adequately compensate them to enhance the viewing experience customers already pay good money to receive with or without a paid peering arrangement with Netflix.

Money drives these debates. If an ISP properly managed their broadband infrastructure, there would be no incentive for any company to contract for a better online experience on a so-called “fast lane” because existing service would perform more than adequately. When a company cuts back on those upgrades, a market for paid prioritization appears. Customers will ultimately pay the price, primarily to ISPs that already enjoy an enormous margin selling broadband service at inflated prices.

A rising tide floats all boats, so your focus should not be as short-sighted as allowing ISPs to divide up the limited broadband highway. The FCC should instead focus on setting the conditions to hasten new competition and force existing providers to upgrade and maintain their networks for the benefit of all subscribers and content producers. The FCC must also move swiftly to cancel state bans on community broadband networks, eliminate regulations that deter broadband start-ups, and maintain enough oversight to guarantee a level playing field on which all can compete.

There is only one way to effectively accomplish all that. Reclassify broadband service the way it should have been classified all along: as a telecommunications service subject to common carrier regulations. Canada has been very successful requiring ISPs to open their last mile networks to competitors, which have allowed people to avoid compulsory usage caps. Customers have a choice of multiple providers from their local phone or cable company, giving rise to much-needed competition.

With strong Net Neutrality, consumers can reach the websites they want without interference. Ignore nonsense suggesting Net Neutrality is a government takeover or censors the Internet — two provably false assertions. In fact, Net Neutrality is the opposite.

I urge you to move with all speed towards reclassification, if only to prevent the inevitable legal challenges to any future policies built on the shakier ground of the current framework, which has not held up well under court scrutiny. I hope the voices of more than a half-million Americans contacting you on this issue will be more than enough to overcome industry objections. We are not asking for 1950s-style telephone regulations. We just want a legally affirmed platform that allows the Internet of today to continue being successful tomorrow.

Yours very truly,

Bad enough

A big company needs a big name, and so what if you can’t say it out loud, so long as your check reaches the cable cartel on time to avoid those inconvenient late fees.

The shock waves of the $45 billion dollar proposed merger of Comcast and Time Warner Cable (not to mention AT&T and DirecTV) have reached as far as Great Britain where appalled editorial writers in the British press are pondering whether Washington has lost its mind or just its integrity… or a combination of both, by actually contemplating the unthinkable rebirth of the American Robber Baron.

Only instead of railroads powering America’s early 20th century economy, today its broadband. Overseas, broadband is plentiful, fast, and cheap. Back home, cable operators are hard at work in a comfortable monopoly/duopoly working on excuses to justify Internet rationing with usage caps, outrageous equipment rental fees, rate hikes, and usage billing for a product about as cheap to offer as a phone call on one of those unlimited calling plans you probably already have.

From The Economist:

“On “OUTLAW”, a drama that aired on NBC, a Supreme Court justice leaves the bench to join a law firm. In real life he might have begun working for Comcast, America’s largest cable company, which owns NBC. Many of Washington’s top brass are on Comcast’s payroll, including Margaret Attwell Baker, a former commissioner of the Federal Communications Commission (FCC), America’s telecoms regulator, who in government had helped approve Comcast’s takeover of NBCUniversal in 2011. Even Barack Obama has Comcast ties. “I have been here so much, the only thing I haven’t done in this house is have seder dinner,” he quipped at a fundraiser hosted last year at the home of David Cohen, Comcast’s chief lobbyist.

“It helps to have influential friends, especially if you are seeking to expand your grip on America’s pay-TV and broadband markets.

“[…] The deal would create a Goliath far more fearsome than the latest ride at the Universal Studios theme park (also Comcast-owned). Comcast has said it would forfeit 3m subscribers, but even with that concession the combination of the two firms would have around 30m—more than 30% of all TV subscribers and around 33% of broadband customers. In the cable market alone (ie, not counting suppliers of satellite services such as DirecTV), Comcast has as much as 55% of all TV and broadband subscribers.

Worse

“Comcast will argue that its share of customers in any individual market is not increasing. That is true only because cable companies decided years ago not to compete head-to-head, and divided the country among themselves. More than three-quarters of households have no choice other than their local cable monopoly for high-speed, high-capacity internet.

“For consumers the deal would mean the union of two companies that are already reviled for their poor customer service and high prices. Greater size will fix neither problem. Mr Cohen has said, “We’re certainly not promising that customer bills are going to go down or even that they’re going to increase less rapidly.” Between 1995 and 2012 the average price of a cable subscription increased at a compound annual rate of more than 6%.”

Before blaming it all on President Obama’s close relationship with Comcast’s top executives, it was the Republicans in Washington that set this tragic monopolistic farce into motion. Michael Powell, President George W. Bush’s idea of the best man in America to protect the public interest at the FCC, represented the American people about as well as ‘Heckuva Job Brownie.’ Instead of promoting competition, Powell used his time to beef-up his résumé for a very cushy post-government job heading America’s top cable lobby – the National Cable & Telecommunications Association. Attwell-Baker was even more shameless, departing the FCC for her sweet new executive digs at Comcast just a short time after enthusiastically voting in favor of its NBCUniversal merger deal.

Powell and others made certain that Internet Service Providers would not be classified as “common carriers,” which would require them to rent their broadband pipes at a reasonable wholesale rate to competitors. The industry and their well-compensated friends in the House and Senate argued such a status would destroy investment in broadband expansion and innovation. Instead it destroyed the family budget as prices for mediocre service in uncompetitive markets soared. Today, consumers in common carrier countries including France and Britain pay a fraction of what Americans do for Internet access, and get faster speeds as well.

Letting Comcast grow even larger, The Economist argues, will allow one company to dominate not just your Internet experience, but also the content consumers access and at what speed.

“There is plenty for Mr Obama and Mr Cohen to discuss at their next dinner,” concludes the magazine. “But better yet, officials could keep their distance from Comcast, and reject a merger that would reduce competition, provide no benefit to consumers and sap the incentive to innovate.”

Considering the enormous sums of money Comcast has shown a willingness to spend on winning over supporters for its business agenda, restraint on the part of Washington will need voter vigilance, much the same way calling out non-profits who gush over Comcast while quietly cashing their contribution checks must also be fully exposed to regulators who will ultimately decide the fate of the merger.

[flv]http://www.phillipdampier.com/video/Antitrust Us.mp4[/flv]

Antitrust Us: Cartoonist Mark Fiore takes on the corporate idea that merging cable companies together creates more competition. (1:50)