Hurricane Sandy hits New York City, power shut off in lower Manhattan, websites without redundant servers go down. Datagram network among those affected.
According to several news reports, Datagram’s data center and websites for the United Nations, Buzzfeed, Gawker, Gizmodo, the Huffington Post, Daily Kos, Bloomberg news and Livestream went down due to Hurricane Sandy. (As of this post, the UN and Gawker are still down – test these websites here.)
Among the worst affected was Datagram, a NYC-based Hosting and Internet Services Provider whose clients include: Gawker, Huffington Post, Activision and many others. Datagram provides network and data center based services including Dedicated Hosting, Internet Access, Colocation and Disaster Recovery Solutions.
ConEd, New York City’s electrical utility company, shut off power to lower Manhattan last night, which affected Datagram. Data centers are equipped with backup electrical generators, but the fuel pumps that supply Datagram’s generators were in the basement, which flooded.
According to Buzzfeed, one of Datagram’s clients, they were told that the Datagram basement was flooded with five feet of water. Datagram’s public statement read, “[We] are currently waiting for the Army Corps of Engineers to assist in pumping out water… Generators are unable pump fuel due to the flooding in the basements. Downed trees are causing fiber outages all across the northeast. We will provide updates as soon as possible. Thank you for your patience.”
Datagram is located on 33 Whitehall Street in Battery Park, the southernmost point of Manhattan and a declared evacuation zone. The area was among the worst-hit in the city.
Many sites are updating their Twitter stream and redirecting users to their Tumblr accounts to provide status updates. Gawker’s website was completely unreachable this morning. Their Twitter status read, “Gawker is temporarily down because the 57th Street Crane just flooded our servers with sea foam, or something. Back with you shortly.”
Many NYC-based website servers affected by Hurricane Sandy have redundant systems on the West Coast. However, redirecting website users from the East Coast to the West Coast adds latency.
The issue now for data centers and server rooms is latency issues due to DNS propagation. When one set of servers fails, it is possible to redirect traffic to another set of servers located elsewhere, but because of time-to-live settings for DNS records, this takes time (anywhere from minutes to hours to days depending the time-to-live time setting). Until the numerical location of alternate servers has fully spread through the network, many of the sites affected by Hurricane Sandy will remain offline for at least some of its users.
All this goes to show that even the best contingency plans can be thwarted by Mother Nature. While data centers always plan for power grid failure, flooding due to a hurricane in NYC is another story.
Here at Dotcom-Monitor we monitor, alert, and report on website speed, web application performance and server issues. We are offering a free trial of our web monitoring service, so you can stay up-to-date on performance issues regarding Hurricane Sandy and the outages that ensued. For more information click here.
