More than 300,000 residents from Louisville to Lexington in Kentucky and north into Indiana were left with no cable service for more than nine hours today after an equipment failure at an Insight Communications office on Okolona Road wiped out analog cable.

Louisville cable viewers channel flipping up and down the dial found nothing but frozen pictures, a captured moment in time from 2:53am this morning.  What they found next when calling Insight was nothing but hours of busy signals.  Some customers in southern Indiana found some channels worked fine while others did not.  In all, the outage impacted at least some channels across most of Insight’s service area in Kentuckiana.

Louisville, Kentucky

Insight initially blamed the problem on a router failure that developed during routine overnight maintenance.  A backup router also failed at the same time.  Company officials originally anticipated service would be restored by six this morning, but that did not come to pass.

Because the equipment failure had never been seen before by Insight technicians, it took nearly nine hours to finally resolve the problem.  Local television stations were deluged with calls from viewers wondering what happened to their favorite shows, and Insight’s dropped ball was topic number one on most local talk radio programs today.

Insight subscribers were especially upset that they couldn’t reach the cable company for answers.  Customer service lines were left jammed through much of the day.

Insight spokesman Jason Keller apologized for the outage.

Subscribers either saw this message, or a frozen picture across their channel lineup this morning.

“Everything that we have has a redundancy built into it. This is no different, but unfortunately on this particular morning with this particular piece of equipment, both the main equipment and the backup did not function properly,” Keller said, calling the outage “highly unusual.”

Keller called today’s outage the largest that he has seen during the company’s 10 years of service in Louisville.

By 10:00am, Louisville customers had their HD channels back, with the remaining analog channels restored by 1:30 this afternoon.

Despite the severity of the outage and its widespread impact, Insight Communications is refusing to issue blanket refunds to affected customers.  Instead, individual customers have to call or contact the company and request a refund, which they characterize as an amount under $1.00.

Customers can cost Insight more than that just by availing themselves of that option, and registering their displeasure over today’s long-lasting outage.

Impacted customers can request a refund online or by phone at (502) 357-4400.

Most television newscasts in the Louisville area treated today’s outage as their top news story, with several issuing periodic updates throughout the morning into the afternoon about the service problems.

[flv width=”640″ height=”380″]http://www.phillipdampier.com/video/WAVE Louisville Equipment problems cause loss of service for Insight customers 5-4-10.flv[/flv]

WAVE-TV in Louisville told viewers “it’s not our fault” that Insight subscribers couldn’t watch the station for at least nine hours today.  (2 minutes)

[flv width=”640″ height=”380″]http://www.phillipdampier.com/video/WLKY Louisville Insight Experiences Cable Outages 5-4-10.flv[/flv]

WLKY-TV in Louisville said it received tons of calls from concerned viewers, one of whom was upset that the company “doesn’t want to give customers answers.”  (2 minutes)

[flv width=”432″ height=”260″]http://www.phillipdampier.com/video/WHAS Louisville Technical Problems at Insight 5-4-10.mp4[/flv]

WHAS-TV, also in Louisville spent time outside of WHAS Radio’s studios this afternoon covering angry reactions from Insight customers on local talk radio.  (2 minutes)

[flv width=”432″ height=”260″]http://www.phillipdampier.com/video/WHAS Louisville After Nine Hours Insight Restores Service 5-4-10.mp4[/flv]

WHAS followed up its earlier report with a wrap-up during its early evening newscast explaining what caused the nine hour outage.  (3 minutes)

[flv]http://www.phillipdampier.com/video/WDRB Louisville Insight Outage Ends 5-4-10.flv[/flv]

WDRB-TV in Louisville explained to its viewers how they could get their money back for a day’s worth of frozen pictures.  (2 minutes)

Windstream Communications customers in eastern Nebraska have spent much of today without access to emergency 911 services, and many were without their own landlines as well.  A switch failure in downtown Lincoln caused the outage impacting several counties starting at 7:45 Thursday morning.

Emergency services personnel were forced to rely on cell phones and amateur radio operators to process calls for emergency service, and several law enforcement personnel were staged in outage areas to assist with any calls for help.

Emergency dispatch centers were silent for much of today.  Those attempting to call 911 received a busy signal.

“You know there’s people out there that are going to need help at some point, there always is, and they’re unable to get that help,” Cass County dispatcher Deb Thiessen told KETV in Omaha.

“It’s very rare there’s an outage of this type,” said Cass County Chief Deputy Brad Lahm.

[flv]http://www.phillipdampier.com/video/KOLN-ABC Nebraska Windstream Suffers Major Outage 4-1-10.flv[/flv]

KOLN-TV and NTV report on today’s major Windstream outage.  (5 minutes)

[flv width=”640″ height=”380″]http://www.phillipdampier.com/video/KETV Omaha Windstream Failure No April Fools Joke 4-1-10.flv[/flv]

KETV-TV in Omaha also covered the outage in their viewing area. (2 minutes)

It's your fault our service doesn't work.

Central Floridians are angry and annoyed with a broadband provider that is more adept at randomly assigning blame than actually resolving serious service problems.  Bright House Networks customers in the Orlando area first noticed their Road Runner service began slowing down around December 23rd.  Web pages took minutes to render, if they finished at all.  Important e-mail was inaccessible at times for many accustomed to a much faster online experience than the bad old days of dial-up.

Problems worsened by Christmas Day, and despite complaints from across the entire region, Bright House technicians spent their time assigning blame elsewhere.  In a classic case of buck passing (Deluxe Goldman-Sachs Home Edition), the cable operator initially began blaming customers for the problems, claiming everything from virus infections to bad routers.

“The technician said he was certain it was either my router or my Windows XP had become hopelessly corrupted with viruses, and I might have to reformat my hard drive and start all over,” writes Stop the Cap! reader Kris.  “Two days before Christmas was the worst possible time for something like this to happen, and it was clear Bright House’s biggest priority was to get me off the phone as fast as they could.”

As customers abandoned all hope of using their broadband accounts on Christmas Day, calls continued to pour into Bright House customer support.  Even the media got involved, noting the cable company adopted a “mum’s the word” strategy on their website, saying nothing about the increasingly maddening service problems.

By then, company officials must have figured out blaming the customers wasn’t working too well, and they blamed Christmas instead.

“I was told heavy Christmas web traffic was responsible,” said Jed, a Stop the Cap! reader.  “They told me with everyone getting new computers and laptops and other electronics, it might be awhile before things got back to normal, perhaps even as late as next week when people returned to work.  Considering I was getting less than 56kbps service at this point, I wasn’t buying it.”

[flv width=”640″ height=”380″]http://www.phillipdampier.com/video/WFTV Orlando Internet Outages Frustrate Bright House Customers 12-27-09.flv[/flv]

WFTV-TV in Orlando is credited for being among the first in the media to shine a spotlight on Bright House Networks’ failure to address their ongoing Internet service problems (2 minutes)

As the weekend wore on, enterprising customers learned it was probable a DNS server or other connection point further up the Internet was probably causing all of the trouble.  Yet that theory was repeatedly denied by Bright House, who was forced to begin issuing statements to the local press, still blaming others for broadband woes.

“Some Bright House Networks Road Runner Internet customers are experiencing intermittent problems accessing various websites,” Bright House spokesman Brian Craven wrote. “The issue is a result of off-network congestion. BHN engineers are working to resolve the issue.”

Customers were also on the receiving end of that old chestnut ‘the exaflood,’ the theory that the Internet is being crushes by a global traffic flood worthy of Noah’s Ark.  As comments piled up on Orlando media’s online message boards, customers traded the excuses coming from Bright House, wondering why the company couldn’t spend as much effort actually fixing the problems with Road Runner on Xanax.

Finally, several days later, company officials admitted the problems were coming from a lot closer to home — theirs, not yours. Brian “It’s Congestion” Craven was back with a revised statement:

“A hardware problem experienced by a Bright House Networks vendor caused some Bright House Networks customers to experience intermittent problems accessing some Internet websites. The issue was resolved at 11 p.m. Sunday. Bright House Networks Internet service was never down.  The situation only affected some customers’ ability to access certain Internet sites.”

Some websites like Google, for instance.

So it wasn’t your fault after all.  It was one of their “vendors.”  Customers pondered when they would be able to receive service credit for several days of useless broadband.

The answer?  Never… tough luck:

“Customer credits will not be given because at no time was Internet service down. It was a latency issue in which some customers experienced intermittent problems accessing certain websites. The issue was caused by a hardware problem experienced by a Bright House Networks vendor,” Craven added.

Customers began lighting the torches.

[flv width=”640″ height=”500″]http://www.phillipdampier.com/video/WOFL Orlando Bright House Outage 12-29-09.flv[/flv]

WOFL-TV in Orlando reports on growing customer rage over the lousy customer service being provided by Bright House Networks. (1 minute)

News accounts noted some customers disappointed by the company’s callous response were returning the favor by unceremoniously dumping their cable modems on the counter at the nearest Bright House cable store, canceling service.  For those brave enough to stay, lessons were learned. As one Web Worker Daily contributor lamented, the most effective way to get Bright House off their collective butts was to embarrass them in the media:

The biggest help [came] when the media started reporting the problem. A local TV station and the Orlando Sentinel both picked up the story. Within only a couple hours, the problem that supposedly didn’t even exist was magically solved, after having dragged on for at least a week.

The lesson I came away with was that fighting as a group is more powerful than going it alone — and even better is having a reporter or two in that group.

The Internet… interrupted: Bright House Networks’ holiday gift to you.  A week of buck passing, liberal use of the “excuse-o-matic” that blames others for their own problems, and a complete unwillingness to do the right thing by customers.  When a service doesn’t work properly, customers don’t want to hear a finger-pointing blame game.  They want the service fixed… fast, and receive credit for the inconvenience they experienced while trying to use your service.  Anyone aware of good customer relations already recognizes these are not unreasonable requests.

Too bad Bright House spent most of its time creatively not fixing its problems until the media got interested.  They should stay on the company’s case until it provides the credit customers deserve.

[flv width=”640″ height=”380″]http://www.phillipdampier.com/video/WFTV Orlando Bright House Not Taking Blame For Outage 12-28-09.flv[/flv]

WFTV-TV in Orlando reports on the inevitable customer blowback that happens when a service provider treats their customers with disregard.  [Apologies for the audio sync problem.] (2 minutes)

[flv]http://www.phillipdampier.com/video/There’s a Map for That 1.flv[/flv]

Verizon’s “There’s a Map for That” Advertising Campaign: Spot 2 (pre-revision — includes “out of touch” language (30 seconds)

Verizon's advertising only displays network coverage of 3G service areas

AT&T Mobility has filed suit against Verizon Wireless in the Northern District Court of Georgia (Atlanta Division) demanding the court order Verizon to stop running ads that suggest AT&T has lousy wireless 3G data coverage.

The suit comes in response to a series of advertisements from Verizon that compare the coverage maps of both companies “3G” wireless data networks.  The term “3G” refers to the third generation (3G) of mobile telephony standards – IMT-2000.  In general terms, local wireless networks upgraded to provide 3G service can provide much faster wireless data speeds than those still operating under older standards like “2G.”

Verizon Wireless has aggressively deployed 3G upgrades across its service area, while AT&T has largely focused on more urban population centers for their 3G upgrades, something Verizon’s advertising calls out.

The crux of the suit is exactly how Verizon depicts the differences in coverage.

AT&T claims the ads leave viewers with the impression that those vast white areas depicted on the coverage map designated by Verizon as “AT&T,” are areas without any data coverage at all.  Most cell phone company coverage maps routinely depict “no service” areas in white, and AT&T claims Verizon underlined the impression in its ads, including one on radio, that included the phrase “out of touch” when speaking about non-3G AT&T service areas.  AT&T described the ad above as showing “a frustrated or sad AT&T customer sitting alone on a bench because she is not able to use her wireless device to meet up with her friends.”

AT&T Mobility’s own coverage map depicts data coverage in varying hues of blue, designating the different types of data service coverage available nationwide, but those different hues and service areas only become apparent after starting to zoom in on specific regions of the country.

AT&T's "Nationwide" Coverage Map for Data

AT&T's coverage map changes when you zoom in, depicting the different types of network standards used in different areas. This map of eastern Texas shows coverage ranging from 3G to woefully slow EDGE networks owned by "AT&T partner" companies

On AT&T’s maps, areas in white are labeled “no service available.”

On October 7th, AT&T Mobility contacted Verizon Wireless and demanded that they either cease the ads or modify them to make them, in AT&T’s words, “less misleading.”

In response, Verizon dropped the “out of touch” language from the ads and inserted a fine print disclaimer at the bottom indicating “Voice and data services available outside of 3G areas.”

AT&T considers the modifications inadequate and filed the lawsuit asking for a cessation of the ads and monetary damages from perceived ill-gotten profits from Verizon snatching away AT&T customers.

Verizon’s defense?  Accuracy.  Verizon Wireless’ ads never stop referring to “3G service” and both maps include specifically labeled “3G Coverage.”

AT&T argues that their network is actually more expansive than Verizon’s, when you also include AT&T’s more prevalent 2G and earlier wireless data standards.  But that’s arguing apples and oranges.  Verizon intends to promote and leverage benefits from upgrading its service areas, large and small, to 3G service.  AT&T has not done that, and in fact has been on the receiving end of criticism from customers frustrated at times with the poor performance of its network, including slow data speeds, dropped calls, and insufficient coverage in certain areas.

Verizon's ads clearly depict "3G Coverage" on their map comparison

The gadget enthusiast press has not been enthusiastic about AT&T’s lawsuit, wishing the company would be as enthusiastic with network upgrades as they are engaging their legal team to fight Verizon, or is little more than a whining villain that has been exposed for its inadequacies.

AT&T customers frustrated with their mobile experience are probably still better off than T-Mobile customers, some of whom spent much of yesterday with no service at all.  In the second nationwide outage in two months, T-Mobile claims about two million customers nationwide experienced voice and data service outages for much of the day, although anecdotal reports suggest a company estimate of “five percent of customers impacted” is low.  No explanation for the outage was given.  This comes after an embarrassing server failure in October which led to some T-Mobile Sidekick customers being without service for up to a month, as well as a loss of stored data which company officials have slowly tried to restore weeks after the system crashed.

One of the major consequences of having insufficient experience and resources running a telecommunications network FairPoint inherited from Verizon is that when something goes wrong, it often turns into a catastrophic service failure that leaves people without service for days on end.

As we continue to watch the teetering FairPoint Communications lurch towards either a “white knight” rescue or bankruptcy court, ponder being one of 12,000 Vermont residents who suffered through a DSL service outage that lasted nearly a week this past June.

“The first day I was mad, the next day I was angry, the third day I was begging for Internet service so I could continue on with day to day activities of running a business,” said Bret Knapp, co-owner of Hilltop RV Center in New Haven.

Knapp relies on his FairPoint DSL service to stay in contact with his customers.

Knapp spent hours on the phone with FairPoint customer service representatives in Texas trying to resolve the problem to no avail.  At one point, after 50-60 calls, a FairPoint representative hung up on him.

Beth Fastiggi, a FairPoint spokeswoman agreed the problems were unacceptable.

“We are making significant progress; internally, we still have a lot of work to do,” she told WPTZ news.

The state telecommunications regulator in Vermont told the station complaints regarding FairPoint arrive daily from across the state.

[flv width=”480″ height=”360″]http://www.phillipdampier.com/video/WPTZ Plattsburgh FairPoint Outage Affects 12,000 Vermonters 6-10-09 .flv[/flv]

WPTZ-TV Plattsburgh covers the FairPoint DSL outage that wiped out service for a week for 12,000 Vermont residents. [2 minutes]