Securing SCADA Systems: Why Choose Compensating Controls?

In my last blog, I shared some secrets on how to successfully use patching in SCADA and control systems.

This week, I’ll look at the pros and cons of using compensating controls as an alternative to patching, and discuss the requirements for success.

Addressing Vulnerabilities Through Compensating Controls

If you’ve read my previous blog articles on patching, you’ll understand why rapid and continuous patching just won’t work on the plant floor. Fortunately, there are alternatives – otherwise known as 'compensating controls'.

In the telecommunications industry, compensating controls are widely used to safely delay patch deployments so that they can be incorporated into regular maintenance schedules. For example, telecommunications equipment vendors often suggest configuration changes that will block exploits of a known vulnerability without requiring a patch. Microsoft also offers this service to customers – included in most Security Bulletins is a section called “Workarounds”, which they define as follows:

“Workaround refers to a setting or configuration change that does not correct the underlying vulnerability but would help block known attack vectors before you apply the update.”

So far, only a few SCADA/ICS vendors have offered this strategy, but the possibilities are numerous. For example:

  • Product Reconfiguration (e.g. Disable the HTTP port)
  • Suggested Firewall Rules (e.g. Block all HTTP traffic)
  • Suggested IDS Rules/Signatures (e.g. Install signatures for logins using a default password)

Basically, these controls aim to prevent any potentially “dangerous” traffic from getting to the device in the first place. In other words, if you can’t directly fix the flaw, remove the conditions where it could occur.

Advantages of Compensating Controls

The compensating controls strategy offers some clear benefits for both vendor and customer.

Compensating controls are safer.

Patches that turn out to be flawed usually cannot be removed from a system. However, removing an incorrect compensating control is often trivial.

Compensating controls put the asset owner in control of his/her fate.

Patches can affect the entire operating system in a controller or computer. This often results in unintended consequences that are hard to predict. By using a compensating control, such as blocking a vulnerable service,  it is easier to understand the impact on the industrial process.

Compensating controls can be released independently of product development and typically require less QA effort

This translates into a faster response to the customer’s security needs.

Compensating controls are independent of the product firmware.

This means that they often have less impact on product functionality, lowering customer resistance to using them. Finally, this strategy allows support of legacy products that are too old to justify the effort of a full firmware release.

Limitations of Compensating Controls

Of course, there are some limitations to this strategy. For example, vulnerabilities that involve encrypted sessions (such as HTTPS) often cannot be addressed with compensating controls – because it can be difficult to decrypt and inspect the traffic. But for a large number of the PLC and DCS vulnerabilities we have seen, the technique works well.

Requirements for Success

For compensating controls to work on the plant floor, there are a number of key requirements.

First, they must have a low impact on process reliability and safety. Let’s face it – any security “solution” that impacts process reliability or safety will be rejected by the customer.

Second, they must be simple to deploy. In the words of a manager of a chemical company to the ISA-99 Committee on Control System Security: “We have to make this [security] something a plant superintendent, engineer, or senior operator can do in their spare time, or it will flop.” For example, if the compensating control involves hardware (like the Tofino Security Appliance), then the field technician should be required to do no more than:

  1. Attach the hardware to a DIN rail or panel.
  2. Attach instrument power.
  3. Plug in network cables.
  4. Walk away...

Good examples of this “Zero Configuration Deployment” strategy are the “Fixed Configuration Firewalls” offered by a number of Safety Integrated System (SIS) vendors. These firewalls contain factory configured protocol and signature rule sets designed specifically to match product and vulnerability requirements. They require no configuration by the end user – they just work out of the box.

But that isn’t the only benefit...

  • Compensating controls are easy to install.
  • Compensating controls can often be installed in a live system without requiring a shutdown.
  • Compensating controls are easily upgradeable to address new threats as they appear.

Typical fixed configuration firewalls provided by automation vendors for securing mission critical operations such as safety systems and pipeline compressor stations.

Compensating Controls Strategy Secures Exposed PLCs

An excellent example of how compensating controls can quickly defend against vulnerabilities was demonstrated by Schneider Electric in late 2011. In December of that year, security researcher Ruben Santamarta publicly disclosed details of multiple vulnerabilities in Schneider’s Modicon PLC product line.

At the time of the disclosure, Schneider had produced a fix for two of the reported vulnerabilities, but was still working on patches for the others.

To help customers secure their PLCs while the other patches were being developed and tested, Schneider produced a guide entitled “Mitigation of the PLC Vulnerabilities Using a Tofino SA”. This detailed guide explained how to use a Hirschmann Tofino industrial firewall to filter out harmful traffic before it reached the PLC.

The following illustration shows how these firewalls are placed in front of one or more PLCs. The firewalls are then configured with rules designed to address each of the vulnerabilities.

Hirschmann Tofino Firewalls protecting PLCs were recommended by Schneider Electric as a compensating control against publically released vulnerabilities.

Special Rules for Complex Issues

In the Schneider Electric example, some of the mitigations were pretty simple to create. For example, blocking a debug service vulnerability was simple. All the user had to do was install a firewall. As long as they didn’t specifically add rules allowing the debug traffic, the vulnerability was mitigated.

Other mitigations can be more intricate...

For example, the FTP Buffer Overflow vulnerability (ICS-ALERT-12-020-03) could have been addressed by just blocking all FTP traffic. Unfortunately, since FTP traffic was essential for some processes, that approach wasn’t acceptable to Schneider’s customers.

Instead, Tofino Security worked with Schneider to create “Special Rules”. These rules contained algorithms that looked specifically for the behaviors that suggest that the FTP protocol is being exploited. If this behavior is discovered, then FTP messages are immediately blocked by the firewall.

Using Security Profiles with Compensating Controls

All these mitigations were easy to deploy through the use of “Security Profiles”. A security profile is a collection of firewall rules, special rules, and protocol definitions designed to address the vulnerabilities for a specific control product. The profile can also include complex checks that a traditional firewall cannot achieve.

Combining the security profiles with the Tofino Firewall allowed Schneider to provide a defense for their customers. A defense that was immediately effective – and that did not require any changes to automation equipment or network configurations. Schneider clients could download a single, easy-to-deploy package of tailored rules that they could deploy without impacting operations.

Site engineers could confirm that the new rules would not harm their operations by using Tofino Test Mode – a feature that allows rules to be tested without actually blocking traffic. As a result, industrial facilities can defend themselves against new threats without having to rely solely on patches for their PLCs, DCS and network hardware.

Final Thoughts on Strategies for Secure Control Systems

There is no denying that SCADA and industrial control systems are difficult and risky to patch rapidly. Patching can work – but only when it is based on proper change control and aligns with maintenance schedules. Furthermore, a patch strategy depends on the rapid release of well-designed and tested software updates for all control products as vulnerabilities are discovered. In my experience, when looking for a way to secure SCADA and industrial control systems, adopting a compensating controls strategy is a much more flexible and much less risky alternative.

Do you have questions or concerns that are not addressed in this article? Do you have experience using compensating controls? Send me your comments.

Related Content to Download

White Paper - "Solving the SCADA/ICS Security Patch Problem"

 

Download this paper and learn about:

  • The challenges of patching for control systems
  • Vendor data on patching deployment rates on ICS products and what can be achieved in the future
  • Compensating control-based solutions for security vulnerabilities

This document is vendor neutral and is ideal for serious consideration of the topic.

Related Links

 

RSS Feed Subscribe to the "Practical SCADA Security" news feed

Author Eric Byres

Comments

6

I like the idea of these small firewall type device but can't I just use a firewall do do the same thing? Is there a cost savings to do these smaller appliances over a firewall.

Yes, you could do many compensating controls with a regular IT firewall. For example, at the start of my blog I suggest a possible control could be "Suggested Firewall Rules (e.g. Block all HTTP traffic)".

And that is a key point - a compensating control isn't "Buy a Tofino". A compensating control is to "Do what is needed to block a threat" until you can safely patch your system (assuming you can safely patch).

Now I believe that the Tofino can be used as a very effective compensating control. It is a firewall, but it is hardened and tailored to secure industrial control systems. Depending on the IT firewall you compare it to, there may be a cost savings with the Tofino. But the real advantage is that it makes securing control systems something that is easy and reliable.

Take the Schneider Electric example above. Some of the mitigations would be easy to create on a standard firewall. But other mitigations would be really hard. That is where Tofino comes in.

We worked with the Schneider team to develop a "special rule" that would block a complex HTTP Buffer Overflow vulnerability, yet still allow valid HTTP traffic. I doubt any normal firewall administrator could have created a similar rule. But using these “Special Rules”, a typical controls engineer that knew nothing about firewalls could block this attack vector in a matter of minutes.

And that is where we see the real benefit of the Tofino solution. Rather than spend hours learning about firewall configuration, and then hours building rules and testing them, the controls engineer can get his system secure in minutes and get back to his or her real job. And that is making the plant produce product as safely and efficiently as possible.

I like your thoughts about compensating controls. Which could be the only solution inside the operational network.

But if we talk about HTTP treats to PLCs connected to the other networks less secure like Internet or even office then I could suggest the Linux box as a firewall and create SSH tunels to PLCs as local port forwarding before you connect to actual PLC. That could be realized as simple two step process, (1) user create a SSL tunnel specifying credentials; (2) connect to the PLC through the local port forwarding.

Tofino Security even could make a dedicated box just for this purpose :)

As it happens, we do make such a product. The Tofino VPN LSM (Loadable Security Module) is an SSL VPN we offer - check it out at http://www.e-catalog.beldensolutions.com/link/57078-24455-49853-100124/e...

The VPN LSM can be combined with the Firewall LSM and one or more of the Enforcer DPI modules. That way a person authenticates to the Tofino and the firewall and DPI policies are then applied against that session, according to the person's credentials. So you could make a support person log in with the VPN and then once in his session would only allow Modbus read commends to be sent (for viewing) to specific PLCs.

Security appliances for Industrial Control Systems (ICS) should allow fast reconfiguration and whenever possible a semi-automated approach should be used. I do understand the danger of automated configuration but the idea here is to have a way to quickly reconfigure the security. Eric is right in saying that compensation will make a huge difference in ICS. Yes security profile will again help speeding the process of changing security rules. It is however important that there is a process in place that will help identifying the vulnerabilities. Protection mechanisms that rely on known vulnerabilities are only efficient if the vulnerability is known.

Your comment "Protection mechanisms that rely on known vulnerabilities are only efficient if the vulnerability is known." is right on the money. The AV industry has proven your point - computers running AV software didn't have protection against Flame or Stuxnet for over a year because no one knew about the vulnerabilities.

That is why I have been so adamant that Tofino products must support traditional L3/L4 firewall technology, Deep Packet Inspection (DPI) technology and "signature-like" special rule technology. This gives users a belt and suspenders approach to security and provides far better coverage of both known and unknown attacks. And if users can eventually patch their systems (say during an annual shutdown), then they should. The more layers of defence, the better!

Add new comment