• Author / Uploaded
• Lawrence Wong

Citation preview

On the Horns of a Dilemma Successful Patch Management Strategies in SCADA and Industrial Control Systems Eric Byres, P.Eng. CTO VP of Engineering CTO, Tofino Security A Belden Brand

Eric Byres, Byres VP Engineering, Engineering Tofino Security

• • • • • • •

Founder of the BCIT Critical Infrastructure Security Centre Centre, a leading academic facility for SCADA cyber-security research Canadian representative for IEC TC65/WG10 standards effort for the protection of industrial facilities from cyber attack Chair of ISA99 Security Technologies WG Chair of ISA99 Stuxnet Gap Analysis TG 2006 SANS Institute Security Leadership Award Six ISA and IEEE awards for security research Testified to the US Congress on SCADA Security

Caught on the Horns of a Dilemma • •

Why Patching is Critical Why Patching is Difficult

Zotob Worm Security Incident

• August 18, 2005: 13 US auto plants were shut h td down b by a simple i l worm.

•

•

Despite professionally installed firewalls between the Internet Internet, the company network, network and the control network, the Zotob worm had made its way into the control system (probability via a laptop). laptop) Once in the control system, it was able to travel from p plant to p plant in seconds.

• 50,000 assembly line workers ceased work. • Estimated $14 million loss.

Hiding Behind the Corporate Firewall Doesn't Work

•

•

The Slammer Worm infiltrated: • A nuclear plant via a contractor’s T1 line; • A power utility SCADA system via a VPN; • A petroleum control system via laptop; • A paper machine HMI via dial-up modem. Corporate firewalls existed in at least three of these cases.

SCADA/ICS Incidents Since 2002

• Malware accounts for 2/3 of the external •

cyber b iincidents id t on control t l systems. t Appears to match IT trends. DoS 6%

System Penetration 13% Sabotage 13%

Malware 68%

A Perimeter Defense is Not Enough

• • • •

We can’t just install a firewall in front of the whole SCADA system and forget about security security. The bugs eventually get in. So we must harden the p plant floor. We need Defense in Depth. Crunchy on the Outside - Soft in the Middle

Defense in Depth Strategy Defense-in-Depth

• “By defense-in-depth strategy, we mean the •

protection t ti measures composed d off more than th one security control to protect the property.” “By By use of these kinds of multi multi-layer layer measures, another layer will protect the property even if one layer is destroyed, so the property is protected more firmly.” Yokogawa g Security y Standard of System TI 33Y01B30-01E

But Patching SCADA and ICS is Difficult

• • • • •

Patches can affect the reliability of SCADA/ICS software. software Patches may require staff with special skills to be present. Patches may not be approved by SCADA/ICS vendor. P t h may require Patches i a system t reboot. b t Patches may not be available.

What – No Patch for That?

• •

Many SCADA/ICS vendors have a poor history of releasing security patches. patches According to Sean McBride of Critical Intelligence, less than half the disclosed SCADA/ICS vulnerabilities ever have released patches.

What – No Patch for That?

•

Many systems still used in process control or SCADA are no longer supported with security patches: “Beginning January 1, 2005, Pay-per-incident and Premier supportt will ill no llonger b be available il bl ffor Wi Windows d NT S Server 4.0. This includes security hotfixes. “On July 13, 2010, Extended Support for Windows 2000 Server will end.” “Support of FactoryLink will cease December 31, 2012”

11

The Typical ICS/SCADA Patch Situation

• “No central Patch Management process in • •

place l ffor process or SCADA systems” t ” “Patching is up to each system owner to decide if and when to do” do “IS/IT department is trying to force their overall patching process onto automation systems systems” Joakim Moby, ISA Expo 2006

Solutions for Patching in Process and SCADA Systems • • •

Patching by Criticality Patching in Waves Using Compensating Controls

Patch Management is Not an Option

•

“A procedure for patch management shall be established documented established, documented, and followed followed.” Requirement 4.3.4.3.7 ANSI/ISA-99.02.01

Patch Management is Change Management

•

“Patch management is about managing the risk of change change” Richard Brown, Dow Chemical

Patch Management in Control Systems Inventory All SCADA Devices

Classify SCADA Devices By Risk

Create Patch/ Vulnerability Tracking g System

Feed Results Back on D i Devices

Execute Response Stages

Create Response Plan

Inventory SCADA and Control Devices

•

•

All process control and SCADA devices are inventoried in terms of: • Device type • Criticality to process • Core software/firmware components • Patching requirements Should include both computer and non non-computer computer devices like PLCS, RTUs and SIS.

Categorizing SCADA and Control Devices

•

All devices are categorized into groups that define when and how they are to be patched patched. Example: • “Trial Adopters” receive patches as soon as available and

•

act as Test/Quality Assurance machines. “N T “No Touch” h” machines hi require i manuall iintervention t ti and/or d/ detailed vendor consultation.

Create Patch/Vulnerability Tracking System

•

•

Develop procedure for keeping track of new patches and vulnerabilities: • Level of importance to control operations. • Devices affected • Severity level For each category, the severity level provides a time frame during which a specific patch must be installed to minimize risk.

Create Response Plan

•

Create a system where patch implementation levels are preset and tied to Response Plans for a given device class.

Response Plan

Aggressiveness

Implementation Window

Level of Testing

Al h Alpha

Mi i Minimum

Q Quarterly t l

Hi h High

Bravo

Moderate

By end of following week

Best Effort

Zebra

Maximum

Within 48 hours

Minimal

Patch Deployment

• • •

Each patch deployment is divided into several waves. waves One wave installs the patch/patches onto a number of machines. The purpose of waves is to find issues before the patch is installed on the systems with highest complexity or regulatory impact impact.

1 cyc cle = 34 4 workin ng days s

Typical “Active Active Patching Patching” Cycle 1. Central Test Platforms

4 days • Rapid deployment • Formal responses gathered

2. Critical & Control Group Critical infrastructure

Sample clients & servers (0.5%)

• Devices of low regulatory impact • Allows time for change control and scheduling • Relies on success of previous waves, not on testing • Responses by exception

4 days

3. Early Adopters

• Mainly devices of high regulatory or safety impact • Otherwise as for waves 3 & 4

6 days

4. Mainstream

10 days

5. Late Adopters Validated servers

High risk devices

22

Source: Joakim Moby, Astra-Zeneca, ISA Expo 2006

10 days

Feedback

• •

When cycle has ended, a report is produced that can be used as a guide for the systems still not following the process (such as new systems). Feed patching results back into inventory database and patch/vulnerability database, including issues.

Exceptions to the Patch Management System

• •

Patch Management System catches between 50% and 70% of the critical patches patches. Some critical devices/systems will not fit into this “Microsoft/PC” patch model: • The patch may not be available or approved by the SCADA

• •

vendor in a timely manner. Servers with old operating p g systems y ((like NT4 or Windows 2000) cannot be patched. PLCs and RTUs may not be patchable.

Compensating Controls

•

If patching isn’t possible, a compensating control is needed: • Turn system off. • Improve system isolation. • Reduce system exposure to other systems.

Protection for Unpatchable Systems

•

Example: Industrial firewall for NT Servers restricts possible infection vectors vectors. Internet

Office LAN

NT4 Based Server Plant Network

Control LAN

FCS

Firewall restricts communicating devices and filters bad traffic

Protection for Embedded Systems

•

Example: MODBUS/TCP industrial firewall sanitizes and limits MODBUS commands commands. Internet

Office LAN

Plant Network

Control LAN

MODBUS/TCP FW only allows MODBUS Read commands

Safety System

Conclusions

• Patching is a change management process • • • •

((you mustt a have h a process). ) Patching does NOT end with the Microsoft patches. patches Compensating controls are important to catch vulnerabilities that can’t can t be patched using standard methods. Patching is not a silver bullet bullet. Patching must be done together with other activities to secure SCADA systems systems.

References Tofino Security White Papers and Application Notes • Usingg IEC/ISA-62443 Standards for SCADA Securityy https://www.tofinosecurity.com/blog/using-ansiisa-99-standards-scadasecurity-plus-white-paper

•

Shamoon Malware and SCADA Security – What are the Impacts? https://www.tofinosecurity.com/blog/shamoon-malware-and-scada-security%E2%80%93-what-are-impacts