ISA-99 has published a draft set of cyber security metrics that is worth a closer look. Metrics are an indispensable part of every cyber security program because without them such program
a) will not be fact-based
b) lacks the capability of systematic improvement (you cannot improve what you can’t measure).
A total of 17 high-priority cybersecurity conformance metrics is defined in the document, and their stated purpose comes with an ambitious claim. Quote:
“Measurements collected for these metrics provide the information needed to assess and manage all IACS risks and provide the basis for senior management to identify the source of the problem and authorize corrective action.”
Let’s look at these metrics. Some statistics:
- Approximately half of the metrics are addressing logical access control
- 70% of the metrics use firewall logs as their data source
- Two out of 17 metrics refer to encryption.
Which installations and facilities did Work Group 4 have in mind when defining these metrics? Let’s do a quick reality check:
- For the majority of IACS, there is either no access control at all (because it’s not supported by the respective product or had been deliberately disabled), no access control in the IT meaning of the term (e.g. user-based instead of role-based credentials), no session management (log off / log on) that would enable access control.
- Firewalls within process networks are a rare asset, and even rarer are organizations that would bother to monitor their logs.
- Encryption of network traffic in process networks is both rarely needed and rarely implemented.
The bottom line is that the fact base which can be covered by ISA-99’s metrics does not represent typical plant floors – it will only cover tiny parts of process networks that are loaded with technical gizmos that infosec people are used to, and what matches their mindset. Nowhere gets this more clear than in metric CM.11 that covers security incidents:
“Number of security incidents reported that have been traced to a delay in deploying patches.”
Funny enough, that’s the only type of incidents that ISA-99 seems to be interested in. A less biased observer with field experience would probably be more interested in a metric that measures the number of software malfunctions due to the deployment of security patches.
As a side note, none of the areas addressed by ISA’s metrics would have any effect on the gold standard of sophisticated ICS attacks, aka Stuxnet.
What is highly puzzling is the fact that ISA-99 actually believes that their metrics would
a) assess all IACS risks, and
b) are useful for senior management.
It leaves to be explained which type of action is expected by management from insight gained by meticulous investigation of firewall logs, reflecting topics like “number of dropped or rejected data packets related to a source address”. Nevertheless it can be stated that if ISA-99 claims that their metrics cover all IACS risks, their understanding of such risks appears to be severely limited.
What IACS security metrics need to look like
Metrics are a fundamental part of the RIPE Framework, and the respective document lists a total of 71 of them. The eight core metrics are
RIPE.SI.Capability: The organization’s capability to completely and accurately catalog hardware and software systems in a system inventory
RIPE.NA.Capability: The organization’s capability to produce complete and accurate network models
RIPE.DF.Capability: The organization’s capability to produce complete and accurate data flow diagrams
RIPE.WF.Capability: The organization’s capability to produce complete and accurate workforce information related to individuals with legitimate control system access (including contractors), identifying their roles and responsibilities, applicable policies and standard operating procedures, and training record and training requirements
RIPE.TP.Capability: The organization’s capability to produce and execute a cyber security and robustness training program
RIPE.OP.Capability: The organization’s capability to produce and enforce Policies and Standard Operating Procedures
RIPE.SP.Capability: The organization’s capability to produce and enforce System Procurement Guidelines
RIPE.PP.Capability: The organization’s capability to produce and enforce plant planning guidelines applicable to configuration change.
All metrics are expressed as a cardinal number between zero and 100 and thus allow for comparisons and benchmarks (unlike ISA’s metrics).
As a matter of fact, all of ISA-99’s metrics are of little value until the items addressed in the RIPE metrics are covered by a cyber security program. Only the full-spectrum RIPE metrics give stakeholders a clear picture of where they stand, and what needs to be improved. Focusing on infosec peculiarities and technical gizmos such as SIEMs has little validity when the biggest security problems on the plant floor are still associated with insecure engineering laptops carried by contractors. The common language translation for little validity paired with hyperbole is “ivory tower”.
Asset owners are encouraged to start out with valid and relevant ICS security metrics right away by enlisting in the RIPE Framework. It’s not a draft, it’s immediately available, and even though it’s not for free, its implementation is arguably much more cost-efficient than figuring out how to put consensus-based standards to good use or throwing scarce resources at remote topics. As a case in point, even ICS security cost and budget are covered by RIPE metrics, and these are certainly numbers that are of interest to senior management.