Operational Technology in Industrial IoT Can’t Tolerate IT-Style Patching. “Threat Analysis” is a Safe and Powerful Solution

Speaking about the Industrial Internet of Things – (IIoT). Operational technology in industrial internet of things (IoT) can’t tolerate IT style patching. Using “Threat Analysis” is a Safe and Powerful Solution. Within companies and across the entire global the IIoT ecosystem – is an intricately intertwined and negotiated merger of IT and OT. OT systems are not only business-critical, they can be nation-critical, or life-and-death critical.

Every industrial internet of things (IIoT) customer I speak to wants the strongest possible security. Not internet of things – industrial internet of things (IIoT).

Who inside the customer’s organization will execute and own this process? In meeting after meeting with customers building IIoT capabilities, I encounter a natural but sometimes tense uncertainty between IT and OT/LOB professionals when it comes to IIoT security.

This capital uncertainty – is itself – a security vulnerability because it delays essential security deployment.

A recent Forrester survey of IT and OT/LOB leaders showed IT and OT managers evenly divided on whether IT or OT is responsible for security, according to InformationWeek’s DARKReading. As an alarming result of this standoff, reports Forrester, an unacceptably large number of companies – 59 percent – are willing to “tolerate medium-to-high risk in relation to IoT security.”

I believe this is incorrect and wrong for companies to allow this neglect to continue — as well as dangerous for their entire operations.

Consider the differences between enterprise IT and OT:

• Availability:
  IT considers 99 percent uptime acceptable, while OT requires 99.999 percent up-time – the difference between 8.76 hours and 5.25 minutes of annual downtime.
  • System life:
  IT systems are refreshed, on average, every three to five years. OT systems, by contrast, last 10 to 15 years.
  • Patching:
  IT patching/updates can be done whenever updates are available, but OT patching/updates risk interrupting strategic, revenue-generating industrial operations.

There are many other IT/OT differences as well – such as varying approaches to the cloud.

However, all differences are subsumed by the universal need for the most resilient IIoT security available.

An approach I favor is helping industrial companies use the hard-won, long-fought lessons of IT to leapfrog to an advanced state of IIoT security. IIoT is expertly architected and deployed to meet OT’s differentiated requirements. Some believe that the OT systems are another form of data center, the heavily protected core of enterprise IT.

There are some promising ideas one can adapt from decades of IT experience. Using these ideas and then adding to them to provide new levels of IIoT security, while honoring the specific needs of OT. Among these adaptions are separation of end-point networks, micro-segmentation and user behavior analytics (UBA). I will discuss these in future pieces.

With patching, IT and OT speak different languages. Enter “threat analysis.”

We understand the patching process aims to update, fix or improve a software program. Usually a quick fix – and often a haphazard one, at that. When it comes to patching, however a direct port of everyday IT practice to OT is not always feasible.

When it comes to patching, IT and OT speak different languages.

It’s essential that the IIoT industry – IT and OT coming together for the benefit of their enterprises. This will require thinking more deeply and with greater imagination to develop robust cybersecurity techniques. By necessity these operations will need to be more agile and effective than reflexive patching.

Patches can create problems for OT. As we’re seeing with patches for the Meltdown and Spectre CPU vulnerabilities, sometimes a patch can make things worse. Early patches for Meltdown and Spectre impacted entire system performance.

The hard truth is that the soft underbelly of the modern industrial economy is largely old OT machines. In the world of IT, if something is infected, the first instinct is to shut it down fast, and patch it (or replace it). But in OT, often the opposite is true: keep it up and running.

Some crucial OT systems have been on factory floors for 15 to 25 years or more. These babies can’t be easily taken down and patched. Even if an appropriate patch were available — those systems generally don’t have enough memory or CPU bandwidth to accept patches.

Finally, there’s the issue of the relative complexity and fragility of OT systems compared to IT systems.

IT systems can be taken down, patched, and started up again to deliver identical service. IT can run racks loaded with identical servers, and if one goes down or burns out, the next one in line takes over without a hitch. But OT systems are often highly orchestrated combinations of software and hardware that have “personalities.”

Even when companies can take down machines for patching – when they come back up – the results can be unpredictable. It’s not the same system because the patch has introduced wild cards that can proliferate through other elements of the system.

In OT, unpredictability is not acceptable.

Bottom line: there must be a better way to protect IIoT systems than patching reflexively OR ignoring a security threat because patching just isn’t feasible – for all the reasons I just outlined.

THE BETTER WAY: “THREAT ANALYSIS”

The better approach in OT is to examine security challenges in a far more granular manner than currently. I propose that we use the age old threat analysis approach to patching.

First step in threat analysis:

Hold off taking any immediate action. That means hold off the patching, not patching, anything else. Wait a second until we validate if a system vulnerability actually exists – and if it does – how can it be exploited?

There are multiple factors to consider.

Some systems that operate deep inside enterprises may indeed have vulnerabilities. Because the system is so isolated within the enterprise, the actual security risk is less than the risk of shutting the systems down for patching – assuming a patch even exists.

The calculus changes when evaluating systems that are exposed to the Cloud or the Internet – are where the security risk is obviously much greater.

Threat analysis:

Threat analysis: would then quickly identify which systems can probably go on operating without patches, and which systems need to be stopped for patching.

Threat analysis: would also validate a vulnerability. It is important to ask another question: if this vulnerability can be exploited by certain threats, is there a way to stop this short of patching?

For example, security experts could create a set of pre-determined scripts within the network, or on the endpoint device itself. That would help identify the appropriate response to a number of different threats. These scripts would serve as an “if/then” template to formalize, automate and accelerate responses to threats. The point is to think with more sophistication than a binary patch/don’t patch decision.

Software companies must support the development of threat analysis by telling customers more about the patches they release. Key pieces of information we’d like to see are how vulnerabilities can be exploited and possible ways to protect against them.

This extra transparency would give customers more information to make decisions on the right security moves for affected systems. Security experts need to be confident a patch will, at the very least, maintain the same risk level that existed before a vulnerability was discovered.

Threat analysis: must be extremely granular. If an enterprise has 100 devices running, each one requires its own threat analysis, which would include a comparison of vulnerabilities vs. patch benefits, as well as a resulting “menu” of security options.

The primary goal, of course, is to enhance security while at the same time maximizing OT uptime.

Threat analysis: is more nuanced and multi-dimensional than go/no-go patching decisions.

But there’s a challenge the industry must solve to get from where we are to where we should be: right now, following the process described above takes time, costs money, requires highly skilled professionals – and even then, it’s not easy to do.

The vendor community needs to begin acting on an agreed upon a set of standards. We need standard, and possibly legislation about how reports and address vulnerabilities will be handled. Meaning – not covered up. This entire process can be automated.

What worked so well in IT just doesn’t fit OT.

It’s time for industry-wide innovation beyond the choice between patch, patch, patch – or letting un-patched systems run vulnerable.

Our goal must be to build powerful, effective processes, and then automate them to put this new approach. All of this can be within the reach of industrial companies and nations on a global basis.

Just because we can see this better future clearly doesn’t mean it is close.

But let’s start now to get there, together.

Satish Gannu

Satish joined ABB in February 2017 as Chief Security Officer and Group VP Architecture and Analytics, ABB Ability™, responsible for the security of all products, services and cyber security services. Satish brings to this position a background in computer programming and more than 25 years of experience in security and analytics.