The Rack Report

Brant Bernet: Irene, Data Centers, and Camping

 Email Article  

Tweet  

  

  

  

  

“Be Prepared.” The ubiquitous Boy Scouts motto. As the father of two Eagle Scouts (I personally made it through half a year of Cub Scouts and ditched my little hat and dress blues in second grade), I have come to know these two words well. “Will it rain at the camp out?” Be prepared. “Will someone run out of water half way through the hike?” Be prepared. “Will I run out of matches just before I build my fire and the temperature drops to below freezing?” Be prepared. “How will I open this can of beans?” Be prepared.

The list is endless. So, when one of our monthly camp outs was scheduled at a beautiful ranch outside of Dallas on a beautiful Saturday night, I naturally made my list. Note that I was careful to make sure my boys were aware of my list-making prowess. In the days leading up to the camp out, I was vocal about my incredible preparedness and more than a bit ashamed of the fact that my offspring had no clue how to pack for a night on the open range.

When my wife saw the stockpile of camping gear that was causing a fire hazard in the front hall, she asked if I was finally cleaning out the garage. Not to be deterred by the domestic jeers, and the fact that they boys had taken exactly three minutes to stuff what they needed into their oversized backpacks, I loaded the car in a manner that would have made Leonardo da Vinci proud, and we headed out.

“Storm clouds? What?” I checked the weather and it said nothing about storm clouds, much less a flash flood and freak drop in temperature! I had my air mattress and enough wood to heat a Roman castle, but no rain fly for my tent—a rain fly that would have taken up about 4 square inches. The night was to be cool, but not cold, not toe-freezing, lose-your-nose-to-frostbite, cold. I was cold, I was wet, I was miserable. I did not sleep a wink.

Hell is thought of as a place of unspeakable heat and constant misery, I found its first cousin in West Texas that night.

I remember the first dad that I saw the next morning; he looked as bad as I felt. He, too, was unprepared for the weather. He told me that he had drifted off at around 11 p.m. into a long, deep sleep and woke up relaxed and alive, ready to face the day. Unfortunately, when he looked at his watch, he realized he had been sleeping for just over 45 minutes, and when he realized the weather had taken a turn for the worse, he never drifted off again.

Then, gasp!!!, I thought of my children, my poor boys, frozen like popsicles, I was sure. “Why didn’t you prepare them for the weather?” I could just hear their mother asking.

As I walked the 100 yards to the boy’s campsite, I heard the faint sound of laughter and I caught a whiff of bacon and eggs being cooked over a hardwood campfire. It was 7 o’clock in the morning and the boys had broken camp, packed their bags, and cooked breakfast. The first thing they said was, “Hey, Dad, you look awful. We hear it rained last night?”

So … did you survive Irene? For a data center user, Irene provided way too much drama over the weekend. If you build a data center in California, building code mandates that you plan to certain seismic specifications. If you build in Miami, you plan for the multiple annual hurricanes. In Oklahoma, tornados. But if you build a data center in New York, New Jersey, or Virginia, what do you prepare for? All of the above? None of the above? Given the recent events of nature, it is more the former than the latter.

You don’t think much about the 100-year (or 500-year) floodplain until Irene is breathing down your neck. Is your data center on high ground? Did you check the FEMA maps before you built or signed a long-term lease? We talk a lot about diverse paths for power and fiber, but what about diverse paths for employees? If a tree limb is down or flooding makes it impossible to get to your data center through the front drive, do your engineers, employees, and customers have a back-up plan?

Make sure you have thought through all forms of diverse path architecture. Many on the East Coast were forced to move to generator power. From the reports that I have read, most made the switch over without a hitch. But once you have depleted the fuel in your on-site tank, how will you replenish your supply? The massive generators are worthless without the diesel that makes them fire. Make sure your plan includes a safe (and dedicated) supply of fuel.

Finally, many IT departments are spending the additional funds necessary to set up an alternate, out-of-region data center. Disaster recovery plans include secondary sites that range from simply running a few critical applications to a complete, mirrored site. In any case, that is the next step in the evolution of data center preparedness.

I guess it is impossible to think of everything. Over the last 20 years many mistakes have been made, studied, corrected, and eliminated in the critical facility world. When making your list, don’t forget your “rain fly,” even if the weatherman says you won’t need it.

My boys knew exactly what to take to survive a cold, wet night; they were truly prepared. With camping, as with data centers, being prepared will allow you to sleep more comfortably. Did you remember your rain fly?

Brant Bernet is co-founder and managing director of Lincoln Rackhouse. Contact him at brant@rackhouse.com.