PHUKET: As I continued my pursuit of a decent internet connection in Phuket for the house Iím moving into, a funny thing happened. Thatís funny as in strange, not funny as in hilarious.
Depending on where you lived in Thailand, at about 2am on Friday morning, February 15, if you had internet service from TOT your line went down. Again, depending on where you lived, the line came back up about 11am to 2pm. Thatís more-or-less 12 hours with absolutely no internet connection, anywhere in Thailand. And, as best I can tell, it happened to every single TOT customer Ė Phuket, Bangkok, Chiang Mai, Isaan.
What about hot swapping? What about redundant switches? What about alternate backup sites? I mean, this is the internet, for heavenís sake. It was designed and built to take a nuclear attack and keep on working. Sure, you get occasional outages for an hour if some idiot is asleep at the wheel and the system is set for manual routing (which should never be the case, but it happens). City blocks can get shut off if somebody digs with a backhoe and takes out the trunk line to the block. Servers can go down if theyíre taken out by viruses, or if the power goes out and the generator doesnít work Ė but any organization worth its salt has alternate sites that can be fired up within minutes.
But an entire country going offline for 12 hours? Thatís the kind of thing you expect if Iranís government pulls the plug, or if Syria suddenly decides its citizens shouldnít be able to communicate. Indeed, it happened last November in Syria, from November 29 to December 1. In this case, though, the cause doesnít appear to be politically motivated. Itís simply incompetence. I think.
The fact is that nobody knows why the internet went out. Iíve seen two plausible explanations, neither of which sounds encouraging. The first reason, reported by a local newspaper, was that TOTís "main server in Bangkok crashed." The second reason, reported on ThaiVisa
, was that two fiber optic cables in southern Thailand were severed concurrently Ė "one was cut by nibbling squirrels (or possibly mice), the other by someone planting palm oil trees."
The first explanation doesnít ring true, at least to me. TOTís main switch for domestic use, known as AS 9737, has a huge 120 Gbps connection to the TOT Local Network Internet Exchange switch identified as AS 37930 Ė yes, all of these big switches have numbers that identify them uniquely. But AS 9737 also has a backup connection at 22 Gbps to the CAT exchange AS4652 in Bangrak. CAT didnít go down, so there should have been backup internet access immediately available to TOT. How a failure in a "server" Ė most likely a switch, and probably AS 37930 Ė couldíve taken out all domestic internet baffles me.
As for the second proffered reason Ė remember, neither of these are official explanations Ė I just donít buy it. If all of TOTís internet traffic, domestic and international, relies on two fiber optic cables in southern Thailand, the people who set up the network topology for the TOT system should hang up their white lab coats. And if the people in charge of watching the lines saw one of them go down, they shouldíve had an army out fixing it immediately. I guess itís possible that 1) the TOT routing system was totally borked
and 2) a squirrel chewed through a line and 3) a palm oil tree digger cut a second line, all simultaneously. Somehow, I seriously doubt it.
Hereís what I know for sure: On the Phuket Gazette
website, reader "Annoyed" said, "I have not had an internet connection for 24 hours; this morning I was finally able to get back on line. The TOT customer service lady was impatient (obviously stressed out with so many callers having similar problems), her English was slurred, and she was at times not even polite. Having been unable to conduct business because of the TOT problem we really didnít need to add TOT rudeness."
Reader "Agogohome" said, "Did I report it? Nope! Itís not worth the phone call or visit, just to receive the usual surprised response, and their worthless promises."
And thereís the rub.
First, itís hard for me to accept that the largest internet service provider (ISP) in the country just went dead for more-or-less 12 (or 24) hours. It shouldnít even be possible.
Second, thereís no notification about system-wide outages. When internet services in other parts of the world go down, thereís a web site people can go to, to see up-to-the-minute information about what caused the problem, and how long itíll take to fix it.
Third, even the employees donít know whatís happening. I see that happening over and over again in Phuket: thereís a widespread internet problem, a customer calls the hotline, and theyíre told that theyíre the first ones to report it. If that doesnít fly because the customerís too savvy, the employee on the phone says that (choose your favorite nonsensical excuse) happened, and they have no idea when itíll be fixed. They may even offer to send out a technician, knowing full well that a hundred other people have reported the same problem.
Itís all about communication. Painfully, our ISPs donít seem to understand that their customers would appreciate some frank explanations. Rabid customers would also stop calling the support lines incessantly if they could just find some information they can trust.
Iím not picking on TOT Ė theyíre not the only offenders. Since February 11, Iíve had widespread reports of outages on the True DOCSIS 20 Mbps system. (DOCSIS is the technology that relies on coax (similar to a cable TV cable), thatís only available in select parts of the island. True calls it "Ultra.") While the "fast" DOCSIS 20 Mbps line was down, it looks like the "slow" DOCSIS 10 Mbps line was up and running fine. Weíve had complaints of bad lines at phuketinternetspeed.com, and Iíve seen similar complaints on the ThaiVisa.com forums, but thereís been nothing official.
So what really happened? Itís likely that weíll never know. Iíve seen exactly zero reports of the TOT outage in the mainstream Thai press Ė a phenomenal lack of concern.
Perhaps the outage originated as a failure of the ICT Ministryís website blocking system, which according to the Washington Post
allows the government to shut down all internet in Thailand in a few minutes. Cuba, Ethiopia, Lybia, North Korea, Syria, Tunisia, Turkmenistan, Yemen, and a few African countries have similar systems. I just donít know.
Unless the government announces an open inquiry, the outage will probably just be ignored. I wonít hold my breath.
Moral for you: the Second Law of Landlines in Phuket, as was espoused last week in this column.
Never get one landline internet connection. Get two, on two separate lines, from two different ISPs. That wonít protect you if the whole country goes internet-dark. But it should handle anything short of a catastrophic failure.
Our regular weekly computer clinic roundtables continue every Sunday morning, 10:00 at Sandwich Shoppe Chalong (in English) and 11:00 at Sandwich Shoppe Patong (in German). If you have a Windows problem that needs to be solved, or a question about internet service in Phuket, drop by and ask one of the assembled gurus. Itís always free. Sponsored by the
Phuket Gazette and Khun Woodyís Sandwich Shoppes.
Live Wire is Woody Leonhardís weekly snapshot of all things Internet in Phuket.
Follow him on Twitter, @PhuketLiveWire, and "like" the pages at facebook.com/SandwichShoppe, facebook.com/phuketgazette and now Google+, or send him mail at Woody@KhunWoody.com.