More than two dozen independent broadband providers are busily wiring parts of the Republic of South Africa with fiber to the home service in a rush to relegate telephone company giant Telkom’s DSL offerings into the dustbin of irrelevance.

The pace of fiber broadband expansion is happening so rapidly, Telkom CEO Sipho Maseko has had to warn investors the phone company’s continued dependence on its copper infrastructure could threaten the company’s future. Consumers and businesses are demanding better broadband in a country that has languished under Telkom’s insistence on sticking with copper infrastructure that has delivered slow Internet speeds and stingy data caps for more than a decade.

The Sunday Times notes South Africa’s fiber revolution is delivering speeds up to 1,000Mbps on a network that literally sells itself. Fiber providers deliver speeds 250 times faster than ADSL and are helping make usage caps and usage-based billing a part of South Africa’s past. New fiber builds are announced in neighborhoods, towns, and cities almost weekly, many driven by residents in neighborhoods pooling together to attract competition. Independent contractors are winning a large share of the broadband deployment business, able to string fiber cables less expensively than Telkom and its bureaucracy.

VUMA is a fiber service provider in South Africa, following Google Fiber’s “fiberhood” example to expand service.

“The rate at which con­sumers are turn­ing to al­ter­na­tives to Telkom to build these net­works is re­mark­able,” the Times editorial states. “Un­til a year ago, [Telkom’s] ab­so­lute dom­i­nance over the ‘last mile’ into homes and busi­nesses seemed set to last for years. No more. Telkom’s core busi­ness is sud­denly threat­ened.”

Maseko

The projects are large and small. Sea Point in Capetown, Blair­gowrie in Jo­han­nes­burg, Kloof and Hill­crest in Dur­ban are all working with start-up providers instead of Telkom. Many are convinced Telkom management is either incompetent or has been more interested in the welfare of its executives than its customers, and more than a few are voting with their feet.

The most aggressive stampede to fiber broadband is occurring in rich suburbs and gated communities prevalent in affluent areas. These are the customers Telkom cannot afford to lose and many are unlikely to ever return to what used to be the state-owned telephone company. The Times argues the longer Telkom pretends it still has a monopoly, the worse things are going to be for a company in for a rude shock.

“For the first time, the lum­ber­ing in­cum­bent, which once held an ab­so­lute mo­nop­oly over fixed lines, is hav­ing to com­pete for con­sumers’ at­ten­tion with a range of nim­ble start-ups that prom­ise su­perb broad­band at de­cent prices, and of­ten on an ‘open ac­cess’ ba­sis — mean­ing con­sumers are free to choose Internet Service Providers, and ser­vice providers can get di­rect ac­cess to the infrastructure,” the newspaper writes.

The newspaper scoffed at Telkom’s wasted opportunities and poor management decisions that now threaten its future viability.

Among Telkom’s biggest failures was a $815 million investment beginning in 2007 on an “ill-fated adventure” in the Nigerian wireless marketplace. Telkom said it was “misled” by several Nigerian businessmen into bleeding billions of South African Rand into a wireless company that used CDMA technology in a country dominated by cheap GSM providers. A shaky network of cellular dealers incapable of attracting new customers only made things worse. The venture’s losses were so huge, it attracted the attention of South African legislators who questioned the wisdom of Telkom investing in Nigeria while allowing South African broadband to stagnate from inadequate investment.

When two dozen fiber to the home competitors began installing fiber to the home service in South Africa, Telkom grudgingly has started to compete with fiber builds of their own. They are likely to face two new national fiber competitors, in addition to the independents, within months.

A year earlier, Telkom also proved less than competent when it entered South Africa’s pay television business. In 2006, Telkom earmarked more than $600 million to be spent on a venture unlikely to win enough customers from dominant MultiChoice to be sustainable. By 2009, Telkom decided to sell most of its stake in the venture at fire sale prices and still found few interested buyers.

Telkom’s management has been accused of gross incompetence, particularly for spending resources on poorly researched business ventures where it lacked experience. The Times asked readers to ponder what South African telecommunications would look like today if Telkom instead spent its almost $2 billion dollars in Nigerian and pay television losses on fiber broadband upgrades inside the country. Since 2006, Telkom preferred to spend as little as possible on network upgrades while trying to convince South Africans to stick with copper-delivered DSL and its variant VDSL, available only in very limited areas. Telkom’s business decisions today still leave most of its customers with no better than 4Mbps DSL.

The question South African business observers are asking is whether Telkom’s new interest in fiber is too little, too late. Mobile operators Vodacom and MTN are planning to build their own competing national fiber to the home networks to compete with Telkom as well.

Charter Communications last week filed its 362 page redacted Public Interest Statement laying out its case to win approval of its acquisition of Time Warner Cable and Bright House Networks, to be run under the Charter banner.

“Charter may not be a household name for all Americans, but it has developed into an industry leader by implementing customer and Internet-friendly business practices,” its statement reads.

The sprawling document is effectively a sales pitch to federal regulators to accept Charter’s contention the merger is in the public interest, and the company promises a range of voluntary and committed service upgrades it says will improve the customer experience for those becoming a part of what will be America’s second largest cable operator.

Charter’s proposed upgrades fall under several categories of direct interest to consumers:

Broadband: Charter will commit to upgrade customers to 60Mbps broadband within 30 months (about 2.5 years) after the deal is approved. That could mean some Time Warner Cable customers will still be serviced with standard speeds of 15Mbps as late as 2018. Time Warner Cable’s Maxx upgrade program will be effectively frozen in place and will continue in only those areas “consistent with Time Warner Cable’s existing deployment plans.” That will leave out a large sections of the country not on the upgrade list. Charter has committed to impose no data caps, usage-based pricing or modem fees, but only for three years, after which it will be free to change those policies at will.

Wi-Fi: Charter promises to build on Time Warner’s 100,000 Wi-Fi hotspots, most in just a few cities, and Bright House’s denser network of 45,000 hotspots with a commitment to build at least 300,000 new hotspots across Charter’s expanded service area within four years. Charter will also evaluate deploying cable modems that also act as public Wi-Fi hotspots. Comcast already offers over 500,000 hotspots with plans for many more, making Charter’s wireless commitment less ambitious than what Comcast today offers customers.

Cable-TV: Charter has committed to moving all Time Warner and Bright House systems to all-digital service within 30 months. Customers will need to lease set-top boxes designed to handle Charter’s encryption system for all cable connected televisions. Among those boxes includes Charter’s new, IP-capable Worldbox CPE and cloud-based Spectrum Guide user interface system.

Video on the Go: Charter will adopt Time Warner Cable’s streaming platform and apps to provide 300 streaming television channels to customers watching from inside their homes (a small fraction of those channels are available while outside of the home). Customers will not be able to watch on-demand recorded DVR shows from portable devices, but can program their DVRs from apps or the website.

Discount Internet for the Poor: Charter references the fact its minimum entry-level broadband speed is 60Mbps so that does not bode well for Time Warner Cable’s Everyday Low Priced Internet $14.99 slow-speed Internet plan. Instead Charter will build upon Bright House Networks’ mysterious broadband program for low-income consumers.

Based on Charter’s initial proposal, Stop the Cap! will urge state and federal regulators to require changes of these terms before approving any merger. Among them:

1. All existing Time Warner Cable and Bright House service areas should be upgraded to meet or exceed the levels of service offered by Time Warner Cable’s Maxx program within 30 months. It is not acceptable to upgrade some customers while others are left with a much more modest upgrade program proposed by Charter;
2. Charter must commit to Net Neutrality principles without an expiration date;
3. Regardless of any usage-cap or usage-based pricing plans Charter may introduce after its three-year “no caps” commitment expires, Charter must permanently continue to offer unlimited, flat rate Internet service at a reasonable price as an alternative to usage-priced plans;
4. Customers must be given the option of opting out of any leased/provided-modem Wi-Fi hotspot plan that offers a wireless connection to outside users without the customer’s consent;
5. Charter must commit to a more specific Wi-Fi hotspot program that details towns and cities to be serviced and proposed pricing for non-customers;
6. Charter must allow customers to use their own set-top equipment (eg. Roku, Apple TV, etc.) to receive cable television service without compulsory equipment/rental fees. The company must also commit to offering discount alternatives such as DTAs for secondary televisions and provide an option for income-challenged customers compelled to accept new equipment to continue receiving cable television service;
7. Charter must retain Time Warner Cable’s Everyday Low Priced $14.99 Internet plan regardless of any other low-income discount program it offers. If it chooses to adopt Bright House’s program, it must broaden it to accept applications year-round, simplify the application process and eliminate any waiting periods;
8. Charter must commit to independent verification of customer quality and service standards and adhere to any regulatory guidelines imposed by state or federal regulators as a condition of approval.
9. Charter must commit to expansion of its cable network into a reasonable number of adjacent, unserved areas by committing a significant percentage (to be determined) of measurable financial benefits of the merger to the company or its executives towards this effort.

Stop the Cap! will closely monitor the proceedings and intends to participate on both the state (New York) and federal level to guarantee any merger provides consumers with an equitable share of the benefits. We will also be examining the impact of the merger on existing Time Warner Cable and Bright House employees and will promote merger conditions that protect jobs and limit outsourcing, especially overseas.

Bell Canada will invest $1.14 billion to bring gigabit fiber to the home service to more than one million homes and apartments in the Greater Toronto Area (GTA) over the next three years.

It will be the largest fiber build ever attempted in North America, and will serve every home and business in the GTA, beginning with 50,000 homes and businesses that will be upgraded to all-fiber service this summer.

“This is something that quite frankly none of us could have imagined just a few years ago,” Bell Canada president and CEO George Cope said at a press conference this morning. “This will be 20 times faster (than average Internet speeds) and it really is building for the consumer what large, large enterprise would have had just a few years ago for their corporations.”

Toronto will be the fastest broadband city in North, Central, and South America when Bell is finished laying 9,000 kilometers of fiber underground and on 80,000 Bell and Toronto Hydro utility poles. At least 27 Bell telephone exchanges will be fully upgraded to 100% fiber service, eliminating huge swaths of older copper wiring. At least 2,400 new jobs will be created, but Bell and Toronto city officials are convinced an all-fiber optic network will attract even more jobs and help broaden Toronto’s digital economy.

Bell’s project in Toronto will be vastly larger than AT&T U-verse with GigaPower, Comcast’s 2Gbps fiber service, and Google Fiber because:

• It will actually exist, unlike fiber to the press release announcements of phantom fiber upgrades from Comcast and AT&T that serve only a miniscule number of customers;
• Will not rely on “fiberhoods” and will deliver fiber service to every home and business and every neighborhood across the entire GTA.

No pricing has yet been announced but Bell promised it would be competitive with other gigabit broadband projects in North America. That likely means Toronto residents will pay between $70-100 a month for gigabit service. No details about usage caps or allowances were included in the announcement.

Bell is already upgrading some of its existing Fibe network in other cities to deliver gigabit speeds on a more limited basis in Atlantic Canada (Bell Aliant) and in select cities in Ontario and Quebec as part of a $20 billion network upgrade.

[flv]http://www.phillipdampier.com/video/CP24 Bell Gigabit Announcement 6-25-15.flv[/flv]

CP24 carried this morning’s press conference introducing Bell Gigabit Internet across Toronto. (19:51)

Phillip “Speaking as a Customer” Dampier

Two defenders of large Internet Service Providers are coming to the defense of the broadband industry by questioning a Guardian article that reported major Internet Service Providers were intentionally allowing a degradation in performance of Content Delivery Networks and other high volume Internet traffic in a dispute over money.

Richard Bennett and Dan Rayburn today both published articles attempting to discredit Battle for the Net’s effort highlighting the impact interconnection disputes can have on consumers.

Rayburn:

On Monday The Guardian ran a story with a headline stating that major Internet providers are slowing traffic speeds for thousands of consumers in North America. While that’s a title that’s going to get a lot of people’s attention, it’s not accurate. Even worse, other news outlets like Network World picked up on the story, re-hashed everything The Guardian said, but then mentioned they could not find the “study” that The Guardian is talking about. The reason they can’t find the report is because it does not exist.

[…] Even if The Guardian article was trying to use data collected via the BattlefortheNet website, they don’t understand what data is actually being collected. That data is specific to problems at interconnection points, not inside the last mile networks. So if there isn’t enough capacity at an interconnection point, saying ISPs are “slowing traffic speeds” is not accurate. No ISP is slowing down the speed of the consumers’ connection to the Internet as that all takes place inside the last mile, which is outside of the interconnection points. Even the Free Press isn’t quoted as saying ISPs are “slowing” down access speed, but rather access to enough capacity at connection points.

Bennett:

In summary, it appears that Battle for the Net may have cooked up some dubious tests to support their predetermined conclusion that ISPs are engaging in evil, extortionate behavior.

It may well be the case that they want to, but AT&T, Verizon, Charter Cable, Time Warner Cable, Brighthouse, and several others have merger business and spectrum auction business pending before the FCC. If they were manipulating customer experience in such a malicious way during the pendency of the their critical business, that would constitute executive ineptitude on an enormous scale. The alleged behavior doesn’t make customers stick around either.

I doubt the ISPs are stupid enough to do what the Guardian says they’re doing, and a careful examination of the available test data says that Battle for the Net is actually cooking the books. There is no way a long haul bandwidth and latency test says a thing about CDN performance. Now it could be that Battle for the Net has as a secret test that actually measures CDNs, but if so it’s certainly a well-kept one. Stay tuned.

The higher line measures speeds received by Comcast customers connecting to websites handled by GTT in Atlanta. The lower line represents speeds endured by AT&T customers, as measured by MLab.

Stop the Cap! was peripherally mentioned in Rayburn’s piece because we originally referenced one of the affected providers as a Content Delivery Network (CDN). In fact, GTT is a Tier 1 IP Network, providing service to CDNs, among others — a point we made in a correction prompted by one of our readers yesterday.

Both Rayburn and Bennett scoff at Battle for the Net’s methodology, results, and conclusion your Internet Service Provider might care more about money than keeping customers satisfied with decent Internet speeds. Bennett alludes to the five groups backing the Battle for the Net campaign as “comrades” and Rayburn comes close to suggesting the Guardian piece represented journalistic malpractice.

Much was made of the missing “study” that the Guardian referenced in its original piece. Stop the Cap! told readers in our original story we did not have a copy to share either, but would update the story once it became available.

We published our own story because we were able to find, without much difficulty, plenty of raw data collected by MLab from consumers conducting voluntary Internet Health Tests, on which Battle for the Net drew its conclusions about network performance. A review of that data independently confirmed all the performance assertions made in the Guardian story, with or without a report. There are obvious and undeniable significant differences in performance between certain Internet Service Providers and traffic distribution networks like GTT.

So let’s take a closer look at the issues Rayburn and Bennett either dispute or attempt to explain away:

1. MLab today confirmed there is a measurable and clear problem with ISPs serving around 75% of Americans that apparently involves under-provisioned interconnection capacity. That means the connection your ISP has with some content distributors is inadequate to handle the amount of traffic requested by customers. Some very large content distributors like Netflix increasingly use their own Content Delivery Networks, while others rely on third-party distributors to move that content for them. But the problem affects more than just high traffic video websites. If Stop the Cap! happens to reach you through one of these congested traffic networks and your ISP won’t upgrade that connection without compensation, not only will video traffic suffer slowdowns and buffering, but so will traffic from every other website, including ours, that happens to be sent through that same connection.

MLab: “Customers of Comcast, Time Warner Cable, and Verizon all saw degraded performance [in NYC] during peak use hours when connecting across transit ISPs GTT and Tata. These patterns were most dramatic for customers of Comcast and Verizon when connecting to GTT, with a low-speed of near 1 Mbps during peak hours in May. None of the three experienced similar problems when connecting with other transit providers, such as Internap and Zayo, and Cablevision did not experience the same extent of problems.”

Our initial findings show persistent performance degradation experienced by customers of a number of major access ISPs across the United States during the first half of 2015. While the ISPs involved differ, the symptoms and patterns of degradation are similar to those detailed in last year’s Interconnections study: decreased download throughput, increased latency and increased packet loss compared to the performance through different access ISPs in the same region. In nearly all cases degradation was worse during peak use hours. In last year’s technical report, we found that peak-hour degradation was an indicator of under-provisioned interconnection capacity whose shortcomings are only felt when traffic grows beyond a certain threshold.

Patterns of degraded performance occurred across the United States, impacting customers of various access ISPs when connecting to measurement points hosted within a number of transit ISPs in Atlanta, Chicago, Los Angeles, New York, Seattle, and Washington, D.C. Many of these access-transit ISP pairs have not previously been available for study using M-Lab data. In September, 2014, several measurement points were added in transit networks across the United States, making it possible to measure more access-transit ISP interconnection points. It is important to note that while we are able to observe and record these episodes of performance degradation, nothing in the data allows us to draw conclusions about who is responsible for the performance degradation. We leave determining the underlying cause of the degradation to others, and focus solely on the data, which tells us about consumer conditions irrespective of cause.

Rayburn attempts to go to town highlighting MLab’s statement that the data does not allow it to draw conclusions about who is responsible for the traffic jam. But any effort to extend that to a broader conclusion the Guardian article is “bogus” is folly. MLab’s findings clearly state there is a problem affecting the consumer’s Internet experience. To be fair, Rayburn’s view generally accepts there are disputes involving interconnection agreements, but he defends the current system that requires IP networks sending more traffic than they return to pay the ISP for a better connection.

Rayburn’s website refers to him as “the voice of industry.”

1. Rayburn comes to the debate with a different perspective than ours. Rayburn’s website highlights the fact he is the “voice of the industry.” He also helped launch the industry trade group Streaming Video Alliance, which counts Comcast as one of its members. Anyone able to afford the dues for sponsor/founding member ($25,000 annually); full member ($12,500); or supporting member ($5,500) can join.

Stop the Cap! unreservedly speaks only for consumers. In these disputes, paying customers are the undeniable collateral damage when Internet slowdowns occur and more than a few are frequently inconvenienced by congestion-related slowdowns.

It is our view that allowing paying customers to be caught in the middle of these disputes is a symptom of the monopoly/duopoly marketplace broadband providers enjoy. In any industry where competition demands a provider deliver an excellent customer experience, few would ever allow these kinds of disputes to alienate customers. In Atlanta, Los Angeles, and Chicago, for example, AT&T has evidently made a business decision to allow its connections with GTT to degrade to just a fraction of the performance achieved by other providers. Nothing else explains consistent slowdowns that have affected AT&T U-verse and DSL customers for months on end that involve GTT while Comcast customers experience none of those problems.

We also know why this is happening because AT&T and GTT have both confirmed it to Ars Technica, which covered this specific slowdown back in March. As is always the case about these disputes, it’s all about the money:

AT&T is seeking money from network operators and won’t upgrade capacity until it gets paid. Under its peering policy, AT&T demands payment when a network sends more than twice as much traffic as it receives.

“Some providers are sending significantly more than twice as much traffic as they are receiving at specific interconnection points, which violates our peering policy that has been in place for years,” AT&T told Ars. “We are engaged in commercial-agreement discussions, as is typical in such situations, with several ISPs and Internet providers regarding this imbalanced traffic and possible solutions for augmenting capacity.”

Missing from this discussion are AT&T customers directly affected by slowdowns. AT&T’s attitude seems uninterested in the customer experience and the company feels safe stonewalling GTT until it gets a check in the mail. It matters less that AT&T customers have paid $40, 50, even 70 a month for high quality Internet service they are not getting.

In a more competitive marketplace, we believe no ISP would ever allow these disputes to impact paying subscribers, because a dissatisfied customer can cancel service and switch providers. That is much less likely if you are an AT&T DSL customer with no cable competition or if your only other choice cannot offer the Internet speed you need.

1. Consolidating the telecommunications industry will only guarantee these problems will get worse. If AT&T is allowed to merge with DirecTV and expand Internet service to more customers in rural areas where cable broadband does not reach, does that not strengthen AT&T’s ability to further stonewall content providers? Of course it does. In fact, even a company the size of Netflix eventually relented and wrote a check to Comcast to clear up major congestion problems experienced by Comcast customers in 2014. Comcast could have solved the problem itself for the benefit of its paying customers, but refused. The day Netflix’s check arrived, problems with Netflix magically disappeared.

More mergers and more consolidation does not enhance competition. It entrenches big ISPs to play more aggressive hardball with content providers at the expense of consumers.

Even Rayburn concedes these disputes are “not about ‘fairness,’ it’s business,” he writes. “Some pay based on various business terms, others might not. There is no law against it, no rule that prohibits it.”

Battle for the Net’s point may be that there should be.

Customers paying $7.99 a month for what used to be called Hulu Plus will be able to add Showtime to their Hulu subscription for an extra $8.99 a month — two dollars less than what Showtime will charge Apple TV and other online video customers.

Showtime Networks’ online streaming service will launch in early July for $10.99 a month, $4 less than HBO Now, which charges $14.99. But Hulu customers will get an extra 18 percent discount if they bundle Showtime with Hulu’s premium option.

Hulu customers who subscribe to Showtime will have access to every Showtime original series ever produced along with Showtime’s full catalog of the same movies, documentaries, specials and sports programming available to cable television customers. Hulu will also carry the east and west coast feeds of Showtime’s primary channel for those who want to watch live events.

The partnership is designed to strengthen Hulu’s competitive position against Netflix and Amazon’s video services.

Showtime CEO Matt Blank doubts Showtime’s online streaming service will cannibalize its existing subscriber base, although most satellite and cable providers charge at least $5 more per month for the premium movie channel ($13.99-16.99 through most cable/telco/satellite providers).

[flv]http://www.phillipdampier.com/video/Bloomberg Showtime CEO Broadband-Only Customers Are an Opportunity 6-4-15.flv[/flv]

Showtime CEO Matt Blank explains to Bloomberg News why selling Showtime online for $10.99 a month ($8.99 for premium Hulu customers) will not hurt existing distributors like cable and satellite providers. (4:22)