There was the sound of breakers tripping in all seven of the grid’s low-voltage substation, and then, the station was plunged into darkness. It was the worst possible scenario: swaths of the country’s grid had already been offline for a month, exhausting battery backups at power plants and substations alike.
What would you do if you were in that utility command center? Turn up everything all at once? Turn up smaller pieces of the grid and put them into a protected environment to run cyberforensics and thus keep them from potentially spreading whatever malware was used in the attack?
Those are the kinds of questions that are typically confined to a lab setting. But earlier this month, on a small island 1.5 miles off the shore of Long Island, the Defense Advanced Research Projects Agency (DARPA) brought the dreaded scenario to life.
Plum Island – at 840 acres, it’s about the same size as Central Park, in Manhattan – is officially called the Plum Island Animal Disease Center. Currently run by the Department for Homeland Security (DHS), the federal facility comprises 70 mostly decrepit buildings.
The island has its own fire department, power plant, water treatment plant and security. The center was originally created in 1954, in response to outbreaks of foot-and-mouth disease in cattle. DHS took over control of Plum Island in 2003, due to the research center’s critical role in protecting the nation’s livestock from infectious animal diseases.
It’s a mixture of industrial infrastructure and isolated, unpeopled, wind-swept, undeveloped acreage with unparalleled views, as the government described in its sales listing when it tried to offload the property.
In short, you couldn’t ask for a better spot to stage an attack on the electric power grid, according to Stan Pietrowicz, a researcher at Perspecta Labs who’s working on a network analysis and threat detection tool that can be used in so-called “black-start” situations, when power has to be restored to a dead grid. Wired quotes him:
We had 18 substations, two utilities, two command centers, and we had two generation sources that we had to bring up a crank path and synchronize. It had a realism that you don’t really find in lab environments that made you rethink the approach.
A cranking path is a portion of the electric system that can be isolated and then energized to deliver electric power from a generation source to enable startup of other generating units.
The week-long exercise, dubbed “Liberty Eclipse,” was designed to throw everything imaginable at a group of DARPA-funded research projects known as Rapid Attack Detection, Isolation and Characterization Systems (RADICS). The aim of the three-year-old RADICS program is to ensure that US utilities can bounce back from a blackout brought on by a cyberattack.
And the aim of the Liberty Eclipse project was to uncover gaps in RADICS defenses under dire, black-start conditions, in which a cyberattack wrestles the power grid to its knees and forces operators to start from scratch.
Walter Weiss, a program manager for the exercise, told reporters that nobody has ever done this before.
As described by EE News – a news outlet focused on energy and the environment – this wasn’t just a simple staging of a cyberattack. The project planners tossed a variety of wrenches into the mix, including a steady onslaught of simulated cyber and physical attacks. For example, at one point, they introduced a data “wiper,” modeled on real-world cases of ransomware, which could send grid operators back to square one if they weren’t careful.
According to Wired, Plum Island’s weather also played a role. Rainy days and high winds made it difficult to take the ferry back and forth to the island and hampered physical work on the grid. The conditions also showed the limitations of one of the recovery tools being developed to survey the grid from above: balloons carrying lightweight electromagnetic radiation detectors that could be launched during a blackout to seek out simple indicators of live power, such as Wi-Fi hotspots from home routers and electromagnetic signals that could show where electrons are actually flowing.
The balloons couldn’t cut it, and the red-team hackers running the attacks never let up while those balloon-born sensors were being buffeted. Wired:
One day, the researchers were instructed to pack overnight bags in case they couldn’t come back from the island until morning. The balloons weren’t reliable in the bad weather, so some of the researchers tried flying the sensors on a kite instead. That proved impractical with the winds. And all the while, the so-called red team kept hacking away.
According to Weiss, DARPA is working on a public after-action report that will cover any major weaknesses found in the RADICS program and map out next steps. The Department of Energy (DOE) is also drafting its own set of takeaways: according to EE News, it completed a related tabletop exercise last month and joined in on the exercise at Plum Island. Others who trekked out to the island included dozens of representatives from major utilities and industry groups.
Successful cyberattacks are real
Real-world scenarios of power grids being crippled by hackers aren’t purely hypothetical: the Ukrainian power grid was attacked in December 2015, affecting 20 substations and leaving about 230,000 people without electricity for hours.
The SANS Institute categorized the outage as a coordinated cyberattack. Malware didn’t directly cause the outage, SANS said, but it did give the attackers a foothold into the grid’s command and control, and malware was also used to thwart recovery.
The Ukrainian power grid was attacked again in December 2016, when remote terminal units (RTUs) controlling circuit breakers at Ukrenergo‘s Pivnichna power substation near Kiev suddenly shut down.
The two attacks had striking similarities, including the same BlackEnergy 3 malware, initiated by malicious spear-phishing attachments that had reportedly bounced around inside state organizations for months.
What was particularly worrisome in the case of the Ukrainian outages was the prospect that the attackers could have been using Ukraine as a playground as much as a battlefield: after all, experts pointed out, the country uses the same equipment and security protections from the same vendors as everybody else around the world.
Marina Krotofil, a researcher from Honeywell Industrial Cyber Security Lab who worked on the investigation:
If the attackers learn how to go around those tools and appliances in Ukrainian infrastructures, they can then directly go to the west.
The fact that successful attacks have already been carried out makes testing out attacks in real-world settings vital: bring on the wind, the rain, and the darkness, and then take away the sensors that enable operators to figure out what the hell is going on. Pietrowicz:
Most of the exercise was really about trying to figure out what was going on and deal with the conditions. It wasn’t a hit and run – while we were cleaning things up the adversary was countering our moves. There was one instance on the third day of the exercise where we almost had the crank path fully established and the attacker took out one of our key substations. It was sort of a letdown and we had to just keep going and figure out our next viable path. Even that small victory got taken away from us.
The participants on two teams, each of which was struggling to start up a grid labelled as a top priority, succeeded in black-starting the grids. Overall, mission accomplished. But participants said that the true insights didn’t come from the successes. Rather, it was the setbacks along the way that gave the most valuable insights.
DARPA plans to run another, even more sophisticated version of the exercise on Plum Island in May, with potentially more of the same to come after that. RADICS’s Weiss told reporters that he hopes that ultimately, the DOE will take over the exercises and incorporate them into preparedness training for government workers and utilities.
SOURCE: https://nakedsecurity.sophos.com/2018/11/15/darpa-uses-a-remote-island-to-stage-a-cyberattack-on-the-us-power-grid/