C-SIGHTS

For IT leaders, the risk is real—and they are accountable for mitigating risk.

Traditionally, disaster recovery involved duplicating IT infrastructure and software systems in the event of a failure in the primary environment. All of the leaders we’ve listened to have shifted their focus from providing system availability to ensuring business continuation. They commented, “We can buy inexpensive hardware; we have duplicate services available through technology like backup data centers, hosted software, and [the] cloud; but we have to understand the business processes and priorities in order to provide the right solutions.”

Senior IT leaders have become better partners with the organizations they support. They perform research on the many potential threats (technical, environmental, cyber); they understand the negative impact of such events to the business, the customers, the investors, and the community; and they raise awareness of the necessity to create response plans that can be executed quickly and effectively.

According to these leaders, “The investment in business continuation is constant. It’s no longer just an IT line item. It’s an organizational investment in time and planning, developing real-time processes with contingencies, creating multi-step communication strategies, and performing pre-disaster exercises.”

A range of approaches with some common themes

While approaches to BCDR vary by organization type or market segment, there are some broad characteristics that define the two groups of organizations with whom we met:

1. State, Local, and Education (SLED) organizations mentioned being mission-oriented, and keeping the organization running is an expectation of their jobs. Two county CIOs and CISOs discussed the “all-hands-on-deck” approach, activating their plans and testing them against the new challenges introduced in California.

• “This year was different than other years. The threats were the same with the fires we experience, but the utility company began a series of rolling power outages. While we pay the telecom companies for priority service, that only helped us for a few hours. We had moved several of our systems to the cloud, but there was no internet available to access it. Additionally, our people lost power at home and had no way to access systems.”
• “We found flaws in our communications plans as employee cell phones lost power and most of our team members no longer have landlines. There were worse traffic issues as the signal lights were out, so even the people we were able to contact had a difficult time getting onsite.”
• “Everyone is expected to be available, and Public Safety is our number one goal. We will have to revisit several parts of our plan and adjust based on what we experienced.”

2. Private sector CIOs were mixed in their approaches. One described their culture as having a “best efforts” approach to keep operations moving, and others had strong product and operational safety risks associated with power outages.

• A financial services CIO said, “We knew our systems were up and our processes were running. We only had a few critical people involved in managing the business. Everyone else was told to focus on their families and making sure everyone was safe.”
• The CIO of a national poultry operation had major concerns and challenges. “We have an extensive investment in backup generators and water supplies to keep the animals healthy. We are able to pause processing if the power situation continues for extended periods of time. But our biggest concerns are trucking and refrigeration for processed chickens. We have USDA standards that must be followed and safe handling procedures. Anything that interrupts that can cause harm to our products and cause danger to consumers. We could lose entire shipments at great financial loss if we do not have appropriate safeguards in place.”
• The CIO for an extensive shipping operation described the impact of interruptions to their organization: “Our biggest concern is safety. We move tons of goods through daily. There are vessels and trucks coming and going. Everyone needs to know what is happening, where people are, ensure safe and effective loading and unloading, and track activities throughout the property. Corporations pay to have their goods shipped through us. If we cannot guarantee smooth operations, customers will lose trust with us and send their shipments to Canada or Mexico. We rely on continual operation to keep us in business. We have operations centers that run 24/7 to keep disruptions to a minimum."

3. Common themes emerged, spanning both segments:

• All senior IT leaders agreed that cyber threats are increasingly seen as a BCDR incident. From small phishing attempts to large ransomware events, each of the organizations have had to create a cyber response plan.
• Even the most prepared organizations did not anticipate the impact of the rolling power outages.
• Their communications plans all needed some adjustment. They must align with their business peers to create effective response plans.