(The Hosting News) – As you have no doubt heard – probably every day for the past several weeks – oil and gas exploration company BP epxerienced a deep water oil rig explosion April 20, 2010 off the coast of Louisiana. The collapse of the rig killed 11 workers, and has released at least 6 million gallons of crude oil into the Gulf of Mexico, according to a Coast Guard and BP estimate of the rate of oil flow out of the broken drill pipe.
If you haven’t actually seen the effects of the oil on the Gulf coast environment, then these devastating photos of the effects of the oil spill might come as a bit of a shock. One of the clearest lessons emerging from this trajedy is this: it is not enough to think about a disaster that might hit at some unspecified time in the future in an unknown way. In order to be prepared for a major negative event, you have to prepare and have all of your actions and systems tested and ready to go at a moment’s notice – for a particular eventuality.
What Web Hosting Companies Can Learn
These lessons as applied to a web hosting enterprise can be summed up in the following list of thoughts to prepare you for the worst. My thanks to reseller hosting provider in the UK, 34SP.com for helping out with these concepts. I’m sure you can think up many more.
1. Assume that something really bad will go wrong and that it will happen next week. A key flaw in reacting to the oil spill was complacency built up over years of seeing no spills take place in the Gulf. That resulted in the erroneous assumption that there would be no incidents in the future. Web hosting companies should not make this mistake. Just because your servers and network have performed well for weeks or even months, doesn’t mean they can’t be disrupted suddenly. Spontaneous events such as power outages, loss of network connectivity, malicious activity, or hardware failure are facts of life in the hosting industry. To live in denial, assuming that this will never happen at your hosting firm is a recipe for disaster.
2. Make a comprehensive list of the major things that can go wrong. Start here: make a list of things that would really wreak havoc if they broke. Your main server components, main switching, network connections, backups failing, phones going out – whatever points of failure you can envision in your particular setup – write them down in a list.
3. Rank order your list from ‘worst’ to ‘least worst’. Review your list of your company’s unique potential points of failure, and then imagine that the item broke…badly. Now imagine the fallout on your business. As you imagine each of these scenarios in detail, arrange the list to reflect the scenarios that are the most catostrophic at the top of the list – and those that merely hurt, but are not devastating, at the bottom of the list. You now have a rank ordered list of potential problems and can address what to do if they actually occur.
4. Starting with item number 1, create an action plan to deal with
each potential disaster. Get as specific as you can about your response. For example – who exactly on your staff will be notified, and in what order? Which vendors will potentially be involved? Who are those emergency contacts? Document any hardware or software that may be required in an emergency to fix the problem – or better yet – consider any ways to make that system redundant, or consider stocking spare parts to be used in the event of an emergency.
5. Practice having a disaster. I live in Los Angeles. A few weeks ago the Coast Guard coducted a statewide series of drills to simulate an actual disaster scenario. According to information provided by a summary in the Los Angeles Times, ”The simulation will involve an actor pretending to be a gunman and a fake contamination of hazardous materials near Rainbow Harbor, officials said. The exercises are part of California’s annual two-day homeland security and disaster preparedness drills.” In other words, the emergency response teams imagined a scenario that would cause real destation and practiced addressing it, as those it were actually happening. This is exactly what will prepare your web hosting company for your imagined ‘worst case’ scenario. Prep your team and discuss what you’re trying to accomplish with the drill. Then conduct the drill as best you can and debrief with the participants. You can always refine your testing and drilling through repetition.
While these five steps are only a start on getting ready for a major negative event at your web hosting company, we can all learn from the Gulf oil disaster: be prepared for things to go wrong, and have a plan ready and practiced for when they do.
If you haven’t actually seen the effects of the oil on the Gulf coast environment, then these devastating photos of the effects of the oil spill might come as a bit of a shock. One of the clearest lessons emerging from this trajedy is this: it is not enough to think about a disaster that might hit at some unspecified time in the future in an unknown way. In order to be prepared for a major negative event, you have to prepare and have all of your actions and systems tested and ready to go at a moment’s notice – for a particular eventuality.
What Web Hosting Companies Can Learn
These lessons as applied to a web hosting enterprise can be summed up in the following list of thoughts to prepare you for the worst. My thanks to reseller hosting provider in the UK, 34SP.com for helping out with these concepts. I’m sure you can think up many more.
1. Assume that something really bad will go wrong and that it will happen next week. A key flaw in reacting to the oil spill was complacency built up over years of seeing no spills take place in the Gulf. That resulted in the erroneous assumption that there would be no incidents in the future. Web hosting companies should not make this mistake. Just because your servers and network have performed well for weeks or even months, doesn’t mean they can’t be disrupted suddenly. Spontaneous events such as power outages, loss of network connectivity, malicious activity, or hardware failure are facts of life in the hosting industry. To live in denial, assuming that this will never happen at your hosting firm is a recipe for disaster.
2. Make a comprehensive list of the major things that can go wrong. Start here: make a list of things that would really wreak havoc if they broke. Your main server components, main switching, network connections, backups failing, phones going out – whatever points of failure you can envision in your particular setup – write them down in a list.
3. Rank order your list from ‘worst’ to ‘least worst’. Review your list of your company’s unique potential points of failure, and then imagine that the item broke…badly. Now imagine the fallout on your business. As you imagine each of these scenarios in detail, arrange the list to reflect the scenarios that are the most catostrophic at the top of the list – and those that merely hurt, but are not devastating, at the bottom of the list. You now have a rank ordered list of potential problems and can address what to do if they actually occur.
4. Starting with item number 1, create an action plan to deal with
each potential disaster. Get as specific as you can about your response. For example – who exactly on your staff will be notified, and in what order? Which vendors will potentially be involved? Who are those emergency contacts? Document any hardware or software that may be required in an emergency to fix the problem – or better yet – consider any ways to make that system redundant, or consider stocking spare parts to be used in the event of an emergency.
5. Practice having a disaster. I live in Los Angeles. A few weeks ago the Coast Guard coducted a statewide series of drills to simulate an actual disaster scenario. According to information provided by a summary in the Los Angeles Times, ”The simulation will involve an actor pretending to be a gunman and a fake contamination of hazardous materials near Rainbow Harbor, officials said. The exercises are part of California’s annual two-day homeland security and disaster preparedness drills.” In other words, the emergency response teams imagined a scenario that would cause real destation and practiced addressing it, as those it were actually happening. This is exactly what will prepare your web hosting company for your imagined ‘worst case’ scenario. Prep your team and discuss what you’re trying to accomplish with the drill. Then conduct the drill as best you can and debrief with the participants. You can always refine your testing and drilling through repetition.
While these five steps are only a start on getting ready for a major negative event at your web hosting company, we can all learn from the Gulf oil disaster: be prepared for things to go wrong, and have a plan ready and practiced for when they do.
No comments:
Post a Comment