Fires in LA, floods in the Carolinas, storm surge in Florida – natural disasters seem to be more frequent than ever. And with data center construction entering a period of expansion – both in traditional areas as well as across the nation – data centers managers are paying attention to disaster preparedness once again and revisiting their disaster recovery (DR) plans.

But it isn’t just Mother Nature that facilities and IT managers need to worry about. Data loss, power outages, equipment damage, and human error can also lead to catastrophe. Hence, data centers should possess, regularly update, and test their DR plan so ensure business continuity. These are things both facilities and IT departments need to do together. 

Data Center Risks and Considerations

Data centers face various risks depending on their location and infrastructure. From floods to earthquakes and power outages to hurricanes, the challenges are diverse. Careful planning ensures data center disaster preparedness. Redundancy, emergency power systems, and having a location to move to during extreme events all play a part in minimizing the impact of a disaster. 

Depending on the type of challenge that might be faced, a variety of safeguards should be in place. Here is a brief summary of some key points to consider in disaster preparedness and recovery:

Floods: Flooding typically isn’t accounted for in many data centers until AFTER they experience their first flood. Even though it is a remote possibility, don’t risk it. Equipment should be positioned on a higher floor or on a raised platform to prevent damage. And in this age of increased liquid cooling, internal flooding is also a threat.

Earthquakes: In seismically active, regions structural damage to buildings and equipment must be considered, as should the potential for disruptions to connectivity or power supply.

Storms: Storms such as hurricanes, tornados, and other weather events can impact much of the country. Protective measures should address problems caused by such things as high winds, heavy rain, and flying debris, breaches in walls and roofing, loss of power and connectivity, and the inability of personnel to reach the data center.

Wildfires: Where wildfires are a danger, actions to take include clearing brush in the vicinity, having good sprinkler and external fire protection systems, backup power in case of a grid outage, and filtration systems that can minimize smoke damage (which is particularly important for facilities that utilize outside air for cooling).

Risk Mitigation: Every data center should identify the types of events that could impact their facility. Once identified, a natural disaster risk assessment should be done to prioritize risks and determine the most appropriate solutions. As every location is unique and every data center has its own specific design and location, all plans and strategies should be customized at a highly granular level. Avoid the temptation to brush off DR planning by downloading a generic plan and adding the name of your facility to tick off a compliance box. Although this article focuses on natural disasters, a comprehensive DR plan should also consider man made risks such as proximity to railroad lines, interstates, airports, chemical storage, battery storage, etc.

Sensors: Environmental monitoring is a wise action for data centers. All it takes is the installation of sensors to detect early signs of potential disasters. Monitor areas such as water levels, seismic activity, and power quality.

Insurance: Acquire insurance coverage to mitigate financial loss. Some plans only cover actual damage. Others might compensate for heavy disruption such as having to move to an alternate location and loss of revenue during an event.

Backup: Backup is one of those areas that can easily fall into neglect, but is essential. Yet all it takes is a change in personnel or a shift in software for it to drop out partially or completely. When inspected, some data centers are shocked to find that new applications and data stores implemented are not included in the current backup schedule. Testing should be a part of the backup and DR process. Stage a complete disaster and see if it is possible to recover all data and how long it takes.

Redundancy: As well as storing backups in geographically distant locations, redundant servers and other hardware play their part in avoiding outages and data loss. Redundant power supplies, emergency power systems, uninterruptible power supplies (UPS), and backup generators or power sources are key to maintaining operations during outages. Furthermore, it is important to know the current run time of all backup and redundant systems. This includes the duration for which the UPS can support the load if the engine-generators fail to start, the length of time the diesel tanks can keep the engine-generators running if fuel deliveries are delayed, and the amount of water storage available to sustain operations in the event of a water supply interruption.

Security: Security it too big a subject to polish off in a couple of sentences. It encompasses internal and external threats. On the external side, guard against intrusion, phishing campaigns, and credential sloppiness. Internally, insider threats are on the rise.

Cooling: Keeping the data center cool is, of course, an essential aspect of disaster prevention. Correct cooling ensures hot spots are addressed appropriately and overall temperatures are maintained at a safe level. To cool effectively and efficiently while keeping costs in check, implement standard solutions such as containment panels, hot aisle/cold aisle systems, correct use of floor tiles, and the use of grommets, sealing tape, and blanking panels to keep air where it should be and prevent mixing of cold and hot air. Some facilities may want to consider placing the cooling unit fans on UPS power to prevent IT equipment from overheating before the UPS power is depleted, in case the engine-generators fail to start.

Personnel: Staff considerations should also be accounted for, such as the availability of food and water on-site, sleeping arrangements, proximity to the facility, and their ability to reach the site during a natural disaster.

Standards: Refer to guidelines and standards from organizations such as the Uptime Institute, ISO 22301 for business continuity management, NIST 800, and others. They provide a more comprehensive rundown of all aspects of DR. (This article is not meant to be a comprehensive list of all the aspects of DR you should consider.)