The Wall Street Journal believes the majority of Americans are paying for internet speed they never use or need, but their investigation largely ignores the question of traffic bottlenecks and data caps that require many customers to upgrade to premium tiers to avoid punitive overlimit fees.

The newspaper’s examination was an attempt to test the marketing messages of large cable and phone companies that claim premium speeds of 250, 500, or 1,000 Mbps will enhance video streaming. A total of 53 journalists across the country performed video streaming tests over a period of months, working with researchers at Princeton University and the University of Chicago to determine how much of their available bandwidth was used while streaming videos from Netflix, Amazon Prime Video, YouTube and other popular streaming services.

Unsurprisingly, the newspaper found most only need a fraction of their available internet speed — often less than 10 Mbps — to watch high quality HD streaming video, even with up to seven video streams running concurrently. That is because video streaming services are designed to produce good results even with lower speed connections. Video resolution and buffering are dynamically adjusted by the streaming video player depending on the quality of one’s internet connection, with good results likely for anyone with a basic broadband connection of 10-25 Mbps. As 4K streams become more common, customers will probably get better performance with faster tiers, assuming the customer has an unshaped connection that does not throttle video streaming speeds as many mobile connections do and the streaming service offers a subscription tier offering 4K video. Netflix, for example, charges more for 4K streams. Some other services do not offer this option at all.

Image: WSJ

WSJ:

For most modern televisions, the highest picture clarity is the “full” high-definition standard, 1080p, followed by the slightly lower HD standard, 720p, then “standard resolution,” 480p. The Journal study found a household’s percentage of 1080p viewing had little to do with the speed it was paying for. In some cases, streaming services intentionally transmit in lower resolution to accommodate a device such as a mobile phone.

When all HD viewing is considered—1080p and 720p—there were some benefits to paying for the very highest broadband tiers, those 250 Mbps and above.

Streaming services compress their streams in smart ways, so they don’t require much bandwidth. We took a closer look at specific services by gathering data on our households’ viewing over a period of months. Unlike the “stress test,” this was regular viewing of shows and movies, one at a time.

Netflix streamed at under 4 Mbps, on average, over the course of a show or movie, with not much difference in the experience of someone who was paying for a 15 Mbps connection and someone with a one gigabit (1,000 Mbps) connection. The findings were similar for the other services.

There is a brief speed spike when a stream begins. Netflix reached the highest max speeds of the services we tested, but even those were a fraction of the available bandwidth.

Users watching YouTube might launch a video slightly faster than those watching Netflix, and at lower resolution, but this is a function of how those services work, not your broadband speed, the researchers said.

Whereas Netflix tries to load “nice high quality video” when you press play and hence has higher spikes, YouTube appears to “want to start as fast as possible,” said Paul Schmitt, one of the researchers.

A spokeswoman for Alphabet Inc.’s YouTube said the service chooses playback quality based on factors including type of device, network speed, user preferences and the resolution of the originally uploaded video. A Netflix Inc. spokeswoman said the company aims to deliver quality video with the least possible bandwidth. Amazon.com Inc. had no comment.

The Journal finds little advantage for consumers subscribing to premium speed tiers, if they did so hoping for improved streaming video. The unanswered question is why customers believe they need faster internet speeds to get those improvements in the first place.

The answer often lies in the quality of the connection between the streaming provider and the customer. There are multiple potential bottlenecks that can make a YouTube video stutter and buffer on even the fastest internet connection. Large providers have had high profile disputes with large streaming companies over interconnection agreements that bring Netflix and YouTube traffic to those internet service providers’ customers. Some ISPs want compensation to handle the increasing amount of incoming video traffic and have intentionally not allowed adequate upgrades to keep up with growing subscriber demand. This creates a traffic bottleneck, usually most noticeable at night, when even a small YouTube video can get stuck buffering. Other streaming videos can suffer from repeated pauses or deteriorate into lower resolution video quality, regardless of the speed of your connection.

Another common bottleneck comes from oversold service providers that have too much traffic and not enough capacity to manage it. DSL and satellite internet customers often complain about dramatic slowdowns in performance during peak usage times in the evenings and on weekends. In many cases, too many customers in a neighborhood are sharing the connection back to the phone company. Satellite customers only have a finite amount of bandwidth to work with and once used, all speeds slow. Some other providers do not pay for a large enough pipeline to the internet backbone, making some traffic slow to a crawl when that connection is full.

Customers are sold on speed upgrades by providers that tell them faster speeds will accommodate more video traffic, which is true but not the whole answer. No amount of speed will overcome intentional traffic shaping, an inadequate connection to the video streaming service, or an oversold network. Too bad the Journal did not investigate these conditions, which are more common than many people think.

Finally, some customers feel compelled to upgrade to premium tiers because their provider enforces data caps, and premium tiers offer larger usage allowances. Cable One, Suddenlink, and Mediacom customers, among others, get a larger usage allowance upgrading. Other providers offer a fixed cap, often 1 TB, which does not go away unless a customer pays an additional monthly fee or bundles video service.

Data caps are a concern for video streaming customers because the amount of data that can be consumed in a month is substantial. As video quality improves, data consumption increases. The Journal article does not address data caps.

Finally, the Journal investigation confined itself to video streaming, but internet users are also increasingly using other high traffic services, especially cloud backup and downloading, especially for extremely large video game updates. The next generation of high bandwidth internet applications will only be developed if high speed internet service is pervasive, so having fast internet speed is not a bad thing. In fact, providers have learned it is relatively cheap to increase customer speeds and use that as a justification to raise broadband prices. Other providers, like Charter Spectrum, have dropped lower speed budget plans to sell customers 100 or 200 Mbps service, with a relatively inexpensive upgrade to 400 Mbps also gaining in popularity.

Does the average consumer need a premium speed tier for their home internet connection? Probably not. But they do need affordable unlimited internet service free of bottlenecks and artificial slowdowns, especially at the prices providers charge these days. That is an investigation the Journal should conduct next.

Charter Spectrum’s broadband-only customers run up more than double the amount of broadband usage average customers subscribing to both cable TV and broadband use, and that consumption is growing fast.

“Data usage by residential internet customers is rising rapidly and monthly median data usage is over 200 GB per customer,” Charter CEO Thomas Rutledge said on a morning quarterly results conference call. “When you look at average monthly usage for customers that don’t subscribe to our traditional video product, usage climbs to over 400 GB per month.”

Last week, Comcast reported its average broadband customer also used over 200 GB a month, but did not break out the difference between those subscribing to cable TV and those who do not. If Comcast’s broadband-only customers are consuming a comparable amount of data, they could be nearing half of their monthly usage allowance (1 TB), in markets where Comcast caps its customers’ usage. But because that is only an average, it means many more Comcast customers are likely nearing or now exceeding Comcast’s data cap, exposing them to hefty overlimit penalties.

Spectrum does not impose any data allowances on its customers — all usage is unlimited.

Charter officials also reported their average mobile customers use “well under 10 GB a month.” The fact Charter did not get more specific about mobile usage is important because the new product is getting scrutiny from some on Wall Street concerned it will have a hard time becoming profitable because of its wholesale agreement with Verizon Wireless, which provides the 4G LTE service for Spectrum Mobile.

Subscribers have been primarily drawn to the $14/GB plan, which includes unlimited talk and texting, because it offers a very low entry price for a full-function wireless plan. But a customer only needs to use more than 3 GB of service per month to find their bill higher than what they would pay subscribing to Spectrum Mobile’s $45 unlimited usage plan. If Charter executives said the average mobile user consumed 5 GB of data, analysts could deduce what the average customer bill probably looked like. To maximize profits, Charter needs customers to select an unlimited data plan and keep data usage low to assure it can cover the wholesale costs Verizon Wireless charges the cable company for wireless connectivity.

Rutledge

Rutledge stressed he expects Spectrum Mobile to be profitable with the current Verizon Wireless MVNO contract in place — the service simply needs a larger user base to overcome its current losses.

Rutledge also announced Spectrum Mobile was testing dual SIM technology, which could allow it to eventually offload more of its 4G LTE traffic to its own (cheaper) network, which could eventually include mid-band wireless spectrum and the CBRS spectrum the company is already testing for fixed wireless service for rural areas. Spectrum could also follow Comcast with its own in-home network of publicly available Wi-Fi or innovate with unlicensed wireless mobile spectrum using small cells or external antennas.

Charter executives noted that customer data demands were pushing many to upgrade to higher speed internet products.

“Over 80% of our internet customers are now in packages that deliver 100 Mbps of speed or more and 30% of our customers are getting 200 Mbps or more,” Rutledge said. “We’re also seeing strong demand for our Ultra product, which delivers 400 Mbps, and we have gigabit service available everywhere.”

The costs to continue upgrading service for broadband customers are negligible on the company’s current platform, Rutledge admits. In the future, Charter Spectrum is considering offering 10 Gbps and 25 Gbps symmetrical service to customers, and it can scale up upgrades very quickly.

“For example, in only 14 months we launched DOCSIS 3.1, which took our speeds up to 1 Gbps across our entire footprint at a cost of just $9 per passing,” Rutledge said.

Nearly every wireless provider in the United States is intentionally slowing down your data service, detrimentally affecting smartphone apps and video streaming.

That is the conclusion of researchers at Northeastern University, University of Massachusetts — Amherst and Stony Brook University, studying the results of more than 100,000 Wehe app users that have run 719,417 tests in 135 countries verifying net neutrality compliance, before and after the open internet rules were repealed in the U.S. earlier this year.

The raw data collected from the app is used as part of a validated, peer-reviewed method of determining which ISPs are throttling their customers’ connections and what services are being targeted.

Nearly Every Mobile Provider Is Throttling Your Speed, Even on “Unlimited” Plans

The researchers concluded that nearly every wireless provider is throttling at least one streaming video service, some reducing speeds the most for customers on budget priced plans while higher value customers are throttled less. No ISP consistently throttled all online video, setting up an unfair playing field for companies that benefit from not being throttled against those that are. Few customers noticed much difference in the performance of streaming video  after the repeal of net neutrality in the U.S., largely because the wireless companies involved — AT&T, Verizon, T-Mobile, Sprint and others — were already quietly throttling video.

“Our data shows that all of the U.S. Cellular ISPs that throttled after June 11th were already throttling prior to this date,” the researchers wrote. “In short, it appears that U.S. Cellular ISPs were ignoring the [former FCC Chairman Thomas] Wheeler FCC rules pertaining to ‘no throttling’ while those rules were still in effect.”

Summary of Detected Throttling

For each ISP, the researchers included tests only where a user’s set of tests indicated differentiation (speed throttling of specific apps or services) for at least one app and did not detect differentiation for at least one other app. This helps to filter out many false positives. As a result, the number of tests in this table is substantially lower than the total number of tests Wehe users ran. The researchers sorted the Cellular ISPs based on the number of tests from users of each ISP. If they did not detect differentiation, researchers used the entry “Not detected.” The researchers claim that offers enough evidence that throttling is not happening. In some cases researchers do not have enough tests to confirm whether there is throttling, indicated by “No data.” 

The table has two column groups for the results: before the new FCC rules took effect on June 11th, and after. If behavior changed from after June 11th, it is highlighted in bold. 

Sprint’s Skype Throttle

The researchers found that video was not the only service impacted by speed throttles. Sprint (and its subsidiary, prepaid provider Boost), for example, is actively throttling Skype.

“This is interesting because Skype’s telephony service directly competes with the telephony service provided by Sprint,” the researchers wrote. But curiously, the throttle almost entirely impacts Android phone users, while iOS devices have less than a 4% chance of being speed throttled. But isolating the exact trigger for throttling remains elusive, the researchers claim.

“While we have strong evidence of Skype throttling from our users’ tests, we could not reproduce this throttling with a data plan that we purchased from Sprint earlier this year,” the researchers admit. “This is likely because it affects only certain subscription plans, but not the one that we purchased.”

When asked to comment, Sprint said: “Sprint does not single out Skype or any individual content provider in this way.” The test results indicate otherwise, suggest the researchers.

T-Mobile’s “Boosting” Throttle Can Mess Up Streaming Video

Some providers, like T-Mobile, attempt to sell their throttled speeds as pro-consumer. In return for reduced definition video, customers are free to watch more online content over their portable devices without it counting against a data cap. But T-Mobile’s video throttle is unique among providers as it initially allows a short burst of regular speed to buffer the first few seconds of a streamed video before quickly throttling video playback speed. Many video players do not expect to see initial robust speeds quickly and severely throttled. Consumers report video playback is often interrupted, sometimes several times, as the player gradually adapts to the low-speed, throttled connection. Consumers receive lower quality video as a consequence.

T-Mobile Plays Favorites

Through extensive testing, research found throttling begins after a certain number of bytes have been transferred, and it is not based strictly on time; below is a list of the detected byte limits for the “boosted” (i.e., unthrottled video streaming) period.

The impact of T-Mobile’s “boosting” speed throttle. Initial speeds of streaming video reach 25 Mbps before being throttled to a consistent 1.5 Mbps.

More concerning to the researchers is their finding that video apps are treated differently by T-Mobile.

“T-Mobile throttles YouTube without giving it a boosting period, while T-Mobile does not throttle Vimeo at all,” the researchers report. “Such behavior highlights the risks of content-based filtering: there is fundamentally no way to treat all video services the same (because not all video services can be identified), and any additional content-specific policies — such as boosting — can lead to unfair advantages for some providers, and poor network performance for others.”

The team of researchers had just one conclusion after reviewing the available data.

“Net neutrality violations are rampant, and have been since we launched Wehe,” the researchers report. “Further, the implementation of such throttling practices creates an unlevel playing field for video streaming providers while also imposing engineering challenges related to efficiently handling a variety of throttling rates and other behavior like boosting. Last, we find that video streaming is not the only type of application affected, as there is evidence of Skype throttling in our data. Taken together, our findings indicate that the openness and fairness properties that led to the Internet’s success are at risk in the U.S.”

The team “strongly encourages” policymakers to rely on fact-based data to make informed decisions about internet regulations, implying that provider-supplied data about net neutrality policies may not reveal the full impact of speed throttles and other traffic favoritism that is common where net neutrality protections do not exist.

“You’re giving it away… you are giving it all away!” — An unknown Wall Street analyst tossing and turning in the night.

America is simply not paying enough for wireless service. Thanks to dastardly competition introduced by T-Mobile and Sprint (potentially to be snuffed out in due course if their merger gets approved), wireless pricing is no longer a license to print money. Forced to offer one-size-fits-all affordable $40-50 unlimited plans, the prospects to grow Average Revenue Per User (ARPU) have never been worse because you can’t charge people for more service on an “unlimited plan” without admitting that plan is not exactly “unlimited.”

Wall Street analysts, already upset at the thought of carriers spending more than $100 billion on 5G network upgrades, are in a real tizzy about how companies are going to quickly recoup that investment. No matter that some wireless companies have profit margins in the 50% range and customers have paid providers for a service they were assured would keep up with the times and network demand. If there is to be a 5G revolution in the United States, some insist it must not come at the cost of reliable profits — so the industry must find a way to stick consumers with the bill.

It is not common for industry analysts to go public brainstorming higher prices and more customer gouging. After all, North Americans already pay some of the highest cell phone bills in the world, only mitigated (for now) by scrappy T-Mobile and Sprint. Mark Lowenstein, a leading industry analyst, consultant, and commentator, was willing to go public in the pages of Fierce Wireless, arguing “operators should be considering charging a premium price for what will hopefully be a premium service.” That is likely music to the ears of AT&T and Verizon, both frustrated their pricing power in the market has been reduced by credible competition from a significantly improved T-Mobile.

Lowenstein fears the prospects of a “race-to-the-bottom 5G price war” which could arrive if America’s wireless companies offer a credible home internet replacement that lets consumers tell the local phone or cable company to ‘take a hike.’ Since wireless operators will bundle significant discounts for those who subscribe to both home and mobile plans, telecommunications services may actually cost less than what Wall Street was banking on.

Something must be done. Lowenstein:

In mobile, there’s been premium pricing for premium phones. And Verizon Wireless, for a few years when it had a clear network lead, was sort of able to charge a higher price for its service (but not a premium price). But today, there isn’t really premium pricing for premium services. That should change when 5G really kicks into gear.

So how do you extract more cash from consumers’ wallets? Create artificial tiers that have no relationship to the actual cost of the network, but could potentially get people to willingly pay a lot more for something they will initially get for a simple, flat price:

One simple way would be a flat premium price, similar to the “tiers” of Netflix for a higher number of devices or 4K/Ultra HD.  So, perhaps $10 per line for 5G, or $25 for a family plan. Another approach would be more akin to broadband, where there are pricing tiers for different levels of service performance. So if the base 4G LTE plan is $50 per month today, for an average 100 Mbps service, 5G packages could be sold in gradations of $10 for higher speeds (i.e. $60 for 300 Mbps, $70 for 500, $80 for 1 Gbps, and so on). An interesting angle on this is that some of the higher-end 4G LTE services such as Gigabit LTE (and beyond) could get incorporated into this, so it becomes less of a 4G vs. 5G discussion and more of a tier of service discussion.

I would also like to see some flexibility with regard to how one can purchase 5G capabilities. For example, a user might only need those premium 5G features occasionally, and might only be prepared to pay that higher price when the service is being used. Here, we can borrow from the Wi-Fi model, where operators offer a “day pack” for 5G, or for a certain city, location, or 5G-centic app or experience. 5G is going to be hot-spotty for awhile anyway, so why not use a Wi-Fi type model for pricing?

Even better, now with net neutrality in the ash heap of history, courtesy of the Republican-dominated FCC, providers can extract even more of your money by artificially messing with wireless traffic!

Lowenstein sees a brand new world of “app-centric pricing” where wireless carriers can charge even more to assure a fast lane for those entertainment, gaming, and virtual reality apps of the future, designed to take full advantage of 5G. Early tests have shown millimeter wave 5G networks can deliver extremely low latency traffic to customers from day one. That kills the market for selling premium, low-latency add-ons for demanding apps before companies can even start counting the money. So assuming providers are willing to purposely impede network performance, there just could be a market selling sub-100ms assured latency for an extra fee.

The potential of a Money Party only 5G can deliver is coming, but time is short to get the foundation laid for surprise toll lanes and “premium traffic” enhancements made possible without net neutrality. But first, the wireless industry has to get consumers hooked on 5G at a tantalizingly reasonable price. Charge too much, too soon and consumers may decide 4G LTE is good enough for them. That is why Lowenstein recommends operators not get carried away when 5G first launches.

“We don’t want to be setting ourselves up for a WiMAX-like disappointment,” Lowenstein writes. “The next 12-18 months are largely going to be ‘5G Experimentation’ mode, with limited markets, coverage, and devices. Heck, it’s likely to be two years before there’s a 5G iPhone in the United States, where iOS still commands nearly half the market.”

The disappointment will eventually be all yours, dear readers, if Lowenstein’s recommendations are adopted — when “certain milestones” trigger “rate adjustment” letters some day in the future.

Lowenstein sees four signs to start the pillaging, and we’ve paraphrased them:

• Coverage: Wait until 30-40% of a city is covered with 5G, then jack up the price. As long as customers get something akin to 5G one-third of the time, they’ll moan about why their 5G footprint is so limited, but they will keep paying more for the scraps of coverage they get.
• Markets: Price the service differently in each market depending on how stingy customers are likely to be at different price points. Then hike those prices to a new “nationwide” standard plan when 5G is available in the top 20-30 cities in the country. Since there may not be much competition, customers can take it or leave it.
• Performance: AT&T and Verizon’s gotta gouge, but it’s hard to do it with a straight face if your 5G service is barely faster than 4G LTE. Lowenstein recommends waiting until speeds are reliably north of 100 Mbps, then you can let rip with those diamond-priced plans.
• Devices: It’s hard to extract another $50-100 a month from family plan accounts if there are an inadequate number of devices that support 5G. While your kids “languish” with 4G LTE smartphones and dad enjoys his 5G experience, mom may shut it all down when the bill comes. Wait until everyone in the family can get a 5G phone before delivering some good old-fashioned bill shock, just like companies did in the golden days of uncompetitive wireless.

These ideas can only be adopted if a lack of competition assures all players nobody is going to call them out for pickpocketing customers. Ajit Pai’s FCC won’t interfere, and is even subsidizing some of the operators’ costs with taxpayer dollars and slanted deregulation to let companies construct next generation 5G networks as cheaply as possible (claiming it is important to beat China, where 5G service will cost much less). Should actual competition remain in the wireless market, all the dreams of rate-hikes-because-we-can will never come true, as long as one carrier decides they can grow their business by charging reasonable prices at their competitors’ expense.

Morrow

Australia is learning a costly lesson finding ways to extend broadband service to rural areas in the country, choosing fixed wireless and satellite networks that will ultimately cost more than extending fiber optic broadband to rural customers.

Australia’s National Broadband Network (NBN) is tasked with supplying virtually all of Australia with internet access, using fiber/wired broadband in urban and suburban areas and fixed wireless and satellite internet access in the country’s most remote locations.

But just a few years after debuting satellite broadband and fixed LTE 4G wireless service in many parts of the country, demand has quickly begun to overwhelm capacity, forcing costly upgrades and punitive measures against so-called “heavy superusers.” The NBN has also scrapped plans to introduce higher-speed fixed wireless services, fearing it will only create additional demands on a network that was not envisioned to manage heavy broadband usage from video streaming.

NBN CEO Bill Morrow has elected to place most of the blame on his customers, specifically “superusers” that he characterized as “online gamers” who spend hours during the day and peak usage periods consuming large parts of the fixed wireless network’s available capacity.

“In the fixed wireless, there’s a large portion [of end users] that are using terabytes of data,” Morrow said. “We’re evaluating a form of fair use policy to say, ‘We would groom these extreme users.’ Now the grooming could be that, during the busy period of the day when these heavy users are impacting the majority, that they actually get throttled back to where they’re taking down what everybody else is taking down.”

Under the current NBN fair use policy, monthly downloads per household are capped at 400 GB, with maximum usage during peak usage periods limited to 150 GB a month, which is already significantly less than what most average American households consume each month. With expensive and unexpected early upgrades to more than 3,100 cell towers to manage rapidly growing usage, the cost of service is starting to rise substantially, even as usage limits and speed reductions make these networks less useful for consumers.

In areas where the NBN extends a fiber optic network, the fixed wholesale price for a 50/20 Mbps connection is $32.00 (U.S.) per month. (A 100/40 Mbps connection costs $46.25). For fixed wireless, prices are rising. A 50/20 Mbps fixed wireless connection (with usage cap) will now cost $46.25 a month.

Morrow took heat from members of Parliament over his claim that online gamers were chiefly responsible for slowing down the NBN’s fixed wireless network.

“With great respect to everything you said over the last 15 minutes, you have been saying to us the problem here is gamers,” said MP Stephen Jones (Whitlam).

Morrow clarified that online gamers were not the principal cause of congestion. The main issue is concurrency, which drags down network speeds when multiple family members unexpectedly use an internet connection at the same time. The worst congestion results when several family members launch internet video streams at the same time. Online video not only leads average users’ traffic, it can also quickly outstrip available cell tower capacity. High quality video streaming can quickly impact 4G LTE service during peak usage periods, driving speeds down for all users. The NBN now considers these newly revealed capacity constraints a limit on the feasibility of using wireless technology like LTE to supply internet access.

The current mitigation strategy includes limiting video bandwidth, discouraging video streaming with usage caps or speed throttles, capacity upgrades at cell towers, and public education requesting responsible usage during peak usage times. With capacity issues becoming more serious, Morrow canceled plans to upgrade fixed wireless to 100 Mbps speeds because of costs. The proposed upgrades would have cost “exponentially” more than wired internet access.

Hype vs. Reality: Most Australians reject fixed wireless and satellite internet as woefully inadequate. (Source: BIRRR)

Actual Fixed Wireless speeds

Actual Satellite Internet speeds

The concept of supplying fixed wireless or satellite internet access to rural areas may have made sense a decade ago, but there are growing questions about the suitability of this technology based on growth in consumer usage patterns, which increasingly includes streaming video. The cost to provide a sufficiently robust wireless network could easily rival or even outpace the costs of extending traditional fiber optic wired service to many rural properties currently considered cost prohibitive to serve. In Australia, fixed wireless and satellite has delivered sub-standard access for rural consumers, and requires the imposition of “fair usage” caps and speed throttles that inconvenience customers. For now, Morrow believes that is still the best solution, given that Australia’s national broadband plan relies heavily on wireless access in rural communities.

“[The benefit of a fair usage policy is] big enough to where if we did groom them during the busy time of the day, it would be a substantial [speed] lift for people,” he said. “I don’t think there’s a silver bullet in any of this – this is going to require us to think through a number of different areas.”

Better Internet for Rural, Regional and Rural Australia (a volunteer consumer group) shares horror stories about relying on satellite to solve rural broadband problems. (7:50)