Prepping for a ‘parade of horribles’

The state of Wisconsin is partnering with businesses to form a response plan for a cyber attack on the power grid, which could simultaneously bring down electric, water, and wastewater service, as well as the Internet and related computing and smartphone use.

From the pages of In Business magazine.

Are you familiar with the term “nightmare scenario?” That’s essentially what the State of Wisconsin is preparing for as it pieces together a cyber disruption response strategy in the event that a cyber attack on the electrical grid knocks out power for a prolonged period of time.

Imagine no water or sewer or — gulp! — Internet service for weeks or months. That’s not a remote possibility but a very real one as state government and utility officials develop the response strategy as part of a public-private partnership. The players include the state Department of Administration, electric utilities such as Alliant Energy and Madison Gas and Electric, private infrastructure owners such as AT&T and IBM Corp., and the Wisconsin National Guard.

As he surveys the landscape, Gary Wolter, chairman, president, and CEO of MGE, seems to hear about a successful cyber attack on a business or operation every month. If the next one takes out the electrical grid for weeks or longer, he knows the lack of a response plan will leave us all in a post-Katrina style world of hurt. “We’re treating this very seriously,” Wolter states. “We understand the electric grid is a high-value target. We’re planning to make sure that we are as prepared as we can be for that type of attack.”

State officials have no illusions about protecting every component of every network, so they are looking to apply risk-based mitigation strategies in order to bring the most critical services back on line as quickly as possible. Leading the strategy formation are Major General Donald P. Dunbar of the Wisconsin National Guard and David Cagigal, chief information officer for the State of Wisconsin. Dunbar, who serves as the adjutant general for the state and as a homeland security advisor to the governor, notes he is not technically proficient in the things that make cyber attacks dangerous. “I know just enough to be scared,” he says. 

To remove some of the fear factor, five teams have been working on the response strategy, including a private-sector team of utility representatives and a team of Wisconsin National Guard cyber experts. This calendar year, much of their focus will be on scenario and contingency planning and additional training.

“We can handle snowstorms, tornados, floods, or other chaotic situations very quickly and in a very responsive way,” Cagigal notes, “but on the cyber side, the day of an event, people are struggling with ‘What do we do next? Who does what, where, and when? Whose responsibility will it be? How do we get the show on the road and marshal the correct resources to be able to address this?’”

No loving this parade

With electric generation and distribution automated to a great extent, a successful cyber attack on the grid could simultaneously bring down electric, water, and wastewater service, and the Internet and related computing and smartphone use. This “parade of horribles,” as Dunbar describes it, is only the beginning. “Fresh water needs power,” Dunbar notes. “Sewage needs power. The only thing that can make a three- or four-month power outage worse is everyone trying to leave Milwaukee and Madison, or Chicago and coming north. Imagine millions of people leaving their homes because they can’t live there any more because conditions have gotten so bad.

“We’ll try to manage that, but I don’t think we can, so some things we have to mitigate, and that’s what we’re going through now, trying to think through what steps could we take. You can’t eliminate everything but we can eliminate some things. We’re trying to buy down that risk.”

While the strategy still is in an aspirational phase, possible solutions are emerging. One answer represents a back- to-the-future approach, but Cagigal cited “micro-generation,” which isolates power in smaller pockets. The concept involves generating power separately and fixing the entire grid later on.

“Go back to the 1910s and 1920s, when the power grid started with smaller micro-grid generation units,” Cagigal explains. “That’s a solution. A real game changer would be advances in energy storage technology, but that’s maybe 10 years away.”

When Craig Fenrick, head of operations for MGE, thinks about the electrical grid, he thinks of an interconnected, nationwide system, and he notes that recapturing “the baby steps” that existed 100 years ago has been done before. Restoring the power generation, transmission, and distribution systems involved what he called “an integration of effort” among utilities in various regions.

“Each of those, when we talk about the micro grid, and when we talk about restoring the grid from a larger-scale event, we often times talk about restoring it in smaller, more localized areas, balancing the generation with the load as you bring the system back together,” Fenrick says. “That’s something the utilities have experience with when you think about the northeast blackouts in 2003, that sort of restoration of bringing back parts of the system at a time. We actually, as utilities, do regular drills of responding to and restoring a system in that manner.”

Utilities also learned things from the integrated response to Hurricane Sandy — things that could apply to a widespread attack on the electrical grid. That response was a large-scale effort because the storm impacted a sizable population over multiple northeastern states, most of which experience hurricanes once in a blue moon. The response to Hurricane Sandy also illustrated the interdependency of critical services. “It was a large effort that involved help from other utilities from all across the country,” Fenrick notes. “The utilities have a system in place where they can call on fellow utilities that may not be affected by the areas that lost power. They get called on for help in restoring power in an incident like that.”

To explain the importance of a micro-grid approach, Dunbar recounts a story involving Melissa Hathaway, who was President Obama’s first cyber czar during the first months of his administration in 2009. She had selected a Christmas gift while shopping in a mall with her then 7-year-old son when the store’s network went down. Since she couldn’t pay for the gift she was just standing there waiting with an increasingly impatient son, who asked why she couldn’t just pay with her credit card. When she explained the computer system was down, he asked if she had cash money. She did have cash on hand but went on to explain how the store manages inventory through the computer network.

“She says it’s just a complicated system, and he finally says to her, ‘Mommy, we’re not very smart if we can’t go backwards,’” Dunbar recalls. “And I thought, ‘Out of the mouth of babes.’”

(Continued)

Advanced business continuity

What would Wisconsin businesses do in the event of a cyber disruption? Many of them already have a plan B for a temporary emergency that typically includes satellite phones, diesel-powered backup generators, remote data centers, and water delivery. However, a prolonged outage is a very different animal and additional contingency plans will be necessary.

Cagigal says he’s not in a position to direct what those corporate contingency plans should be, but he notes the state is asking businesses to think about what they would do in the event of a cyber attack that disrupts electrical service. “I can’t tell them what the solution is, but I think they are ignoring the possibility,” he states.

Noting how chaos reigned in the aftermath of Hurricanes Katrina and Sandy, Cagigal says many business continuity plans are predicated on the delivery of diesel fuel. Following a power grid failure, diesel fuel might not be delivered as quickly as existing contingency plans anticipate, so the state recommends that businesses prepare for unpredictable events.

“There is a plan today,” Cagigal notes. “Let’s call it Plan B, and as long as I’m getting diesel fuel delivered, I’m okay. The question is what happens if you don’t get the diesel fuel delivered? What if we have no power, no computer, no dispatch capability, and no GPS for the trucks because they don’t have power, either? How far is that fuel going to go from whatever you have in the tank to when you think you’re going to get the next delivery if we’re a month, two months, three months without power? We really have to think about what does plan C look like or what does plan D look like?”

MGE’s advice to businesses on planning for a prolonged power outage is to look at the consequences of a disruption. This is true regardless of whether it’s a cyber attack that takes out the grid or a natural event like the devastating tornado that ripped through Joplin, Mo., in 2011. “Regardless of the cause, the business has to ask what are the consequences of a prolonged outage?” Wolter states. “For example, right now, the local hospitals all have backup generation because of the consequences of not having power.”

Their task is defining what they most need to protect and then customizing their response based on the threats they are most concerned about. Factors to consider include whether they have data and operations decentralized in order to have a natural backup or a secondary base of operation. “Those types of questions are going to vary from business to business,” Wolter notes. “It’s a risk analysis of their own operation.”

The Wisconsin National Guard’s response to a long power outage will be to spackle the cracks, Dunbar notes. He says the National Guard does not have, nor should it have, a plan to run the entire country because it necessarily runs on private industry. “I don’t pretend for a second the state or the National Guard will come riding in on a horse to save the day,” he says. “This is a different kind of animal. We all have personal responsibility. We have corporate responsibility when it comes to cyber. What I’m looking for, as a senior state official, is where can we fill a gap or a seam and enable the state to help a business, an industry, or a critical infrastructure to recover if necessary?

“We are looking for gaps and seams, places where if the parade of horribles happens, what could we do? Where could we best mitigate the worst effects?”

Dunbar offers one example of how the National Guard could fill the gaps. For small neighborhood stores in a populated area, planners will have to make sure enough of them have power generation capability or at least backup capability. “Perhaps we could plan with them to make sure we have the ability to plug a generator in, not necessarily have a generator,” Dunbar explains, “and then if something happens, we could bring a certain number of key generators to certain key parts of the city, get the power on, and keep people fed while the broader recovery happens.”

One of Dunbar’s worries is substandard communication, which could easily become one of the “horribles” if the power goes down. According to him, even if the response is going well, if people don’t sense that it’s going well, panic can set in. That’s when people start thinking about getting out, and everybody who leaves their home or leaves their community can make the problem worse.

“If you tell me I’ve got seven or eight pieces of critical infrastructure, I’m pretty good at planning,” Dunbar states. “I’ll make a plan and I’ll protect that, not 24 hours a day but in a crisis. The governor can authorize that, so I view it as filling gaps and seams and marginal mitigation that would make the parade of horribles less horrible.”

A job that’s never done

Citing press reports that Russia launched a cyber attack that brought down the Ukraine’s electrical grid, Cagigal says there is no reason why that could not happen here. “That’s a complex attack,” he notes. “It’s a coordinated attack. That’s more than one person doing some damage, that’s a nation-state trying to make a statement. We need to be aware of how sophisticated this can get and how important it will be to our way of life, our economy, and the vitality of this state and any other state.”

Cagigal further notes that such an incident will not respect borders. If an incident happens in Illinois, “it will spill over into Wisconsin and vice versa,” he warns.

Dunbar characterized the unfolding response strategy as both foundational and aspirational because there is “no destination and no getting done” — just continual process improvement. “It’s 2016 and long after we have departed this earth, this will be a problem for people on this planet,” he notes. “This is not going away. It’s going to continue. Hopefully, we can come up with better solutions. Hopefully, we can come up with better strategies.”

(Continued)

Post Katrina: The public-private imperative

A woman and her child wait with hundreds of other flood survivors at the Convention Center in New Orleans, Thursday, Sept. 1, 2005. Thousands of storm refugees had been assembling outside for days, waiting for help to come.

Remember how the country watched in horror at the government’s response to Hurricane Katrina? The images of mothers and families stuck on a bridge days after the storm drove them from their homes, holding up their newborns to demonstrate that simple necessities like disposable diapers still hadn’t made their way to devastated areas were among the scenes that angered the public and made millions of people wonder why emergency planning had been so slipshod.

It’s not as though the Gulf Coast had never been struck by a major storm before, and when people watched as states like Florida prepared for and responded more effectively to hurricanes, they wondered why there was such a vast difference. Part of the answer lies in more experience with storms — Florida routinely examines its response to each storm in search of process improvements — and part of it lies in the value of public-private partnerships between entities like Florida Power & Light Co. and the government.

David Cagigal, a former chief information technology officer for Alliant Energy who now serves in virtually the same capacity for the state of Wisconsin, refers not just to the value of public-private partnerships but also the necessity of them. As the technology expert in charge of putting together Wisconsin’s cyber disruption response plan, he notes the statewide plan must be formed in collaboration between public and private entities in part because state government does not own the communication assets.

This is why the likes of Madison Gas and Electric, American Transmission Co., and various telecommunications providers are all part of the Badger State conversation.

“All the lines, specifically in Wisconsin, and all the services that we receive from the network are leased by us,” Cagigal notes. “They own the lines — Frontier Communications, CenturyLink, AT&T, and all of the mom and pops. All that wire in the ground is theirs. We [state government] have no right to control it. We have a right to use it for service and pay the bills, but we can’t control the resilience or the performance of those lines. We have to partner with them on a public-private basis.”

Click here to sign up for the free IB ezine — your twice-weekly resource for local business news, analysis, voices, and the names you need to know. If you are not already a subscriber to In Business magazine, be sure to sign up for our monthly print edition here.