The International Standards Organization recently published an updated version of their guidance for information security risk management, but they have missed the mark entirely on quantitative methods.
What is ISO 27005?
The ISO/IEC-27005 is one of the key standards published under the ISO/IEC-27000 series on Information Security. The most recent version, as of this article, is ISO/IEC-27005:2022. The standard covers techniques for conducting information security risk management, from establishing a risk management process to performing the actual risk assessments and selecting treatment options. The overall risk management process is not unique to ISO/IEC-27005. It borrows heavily from the ISO/IEC-31000 series on (general) risk management and the European Standard IEC-31010.
Now, a quick disclaimer: This is not a critique of the entire ISO/IEC-27000 series nor the ISO/IEC-27005 standard itself. They contain many useful practices that we also look to in relation to the overall risk management process.
iso 27005 for risk management
I recommend anyone working with information security risk management to familiarize themselves with these standards. But they are not perfect. The ISO/IEC-27005 needs to cater to a wide range of organizations. Governmental organizations, non-profit organizations, private businesses, small businesses and large corporations. They cannot cover everything in the level of detail we may wish.
Their coverage of methods for quantitative risk assessments, however, leaves lots to be desired. And this is a problem, as it fosters a misunderstood view of what quantitative methods are and can do.
Why is this a problem? It’s problematic because this “minor omission” is deeply integrated into IT risk management across organizations today – large and small. These businesses and the standards organizations echo the same myths about quantitative risk assessments, artificially increasing the barrier of entry to the extent that even large organizations with major risk exposure don’t even consider quantitative methods as a viable option.
Let’s take an example:
The Danish Standards Organization (DS) recently published a guide on risk management. While we commend this effort (and any effort to further knowledge about risk management), the presentation of quantitative methods is not flattering:
There are several ways to calculate risk, when you need to understand likelihood and consequences. If you are a small business, it will often be easiest to do so simply with a qualitative approach.
Another approach is the quantitative method, where you calculate likelihood based on e.g., percentages and consequence based on financial loss, if an incident occurs. This can be time-consuming and often hard to obtain the data, that makes it possible to use a quantitative method, which is why most organizations choose the qualitative method.
(Translated from Guide til risikostyring udarbejdet af Dansk Standard og Alexandra Instituttet)
Which method would you go for if you’ve only read the paragraph above?
They start out well
Let’s dive into ISO/IEC-27005, and how it presents quantitative methods for risk management.
The first reference to quantitative methods is in the definitions chapter, where they describe criteria for level of risk:
Criteria for level of risk can be qualitative (e.g., very high, high, medium, low) or quantitative (e.g., expressed in terms of expected value of monetary loss, loss of lives or market share over a given period of time).
This is a decent representation of a quantitative metric for level of risk. Rather than reporting the risk on a scale, it is reported in terms of monetary value. If we wanted to extend this definition slightly, we should understand that quantitative risk assessments don’t provide one value of expected loss. Expected loss is expressed as distributions with different likelihoods, such as in a Loss Exceedance Curve (LEC), illustrating our uncertainty about future events.
So what is wrong with iso 27005?
ISO 27005 present quantitative techniques for risk analysis as:
Techniques for risk analysis based on consequences and likelihood can be:
a) qualitative, using a scale of qualifying attributes (e.g., high, medium, low); or
b) quantitative, using a scale with numerical values (e.g., monetary cost, frequency or probability of occurrence); or
c) semiquantitative, using qualitative scales with assigned values.
Talking about quantitative methods as “scales with numerical values” could be okay if the scale was a continuous one without limits. However, this is not the case in ISO/IEC-27005. In the appendix under A.1.1.3 Quantitative approach, they present only one example of quantitative risk assessment methods: The Finite Scales (A.1.1.3.1). This is basically an ordinal scale with numbers. While yes, you employ numbers rather than qualitative descriptions (high, medium, low), this method is subject to exactly the same fallacies and issues as other qualitative methods, and bears no resemblance to quantitative risk analysis methods such as simulation-based FAIR.
In other words, the Finite Scales example represents all that people believe is wrong with quantitative methods, and none of what actually makes quantitative methods effective – and a more accurate representation of our uncertainty about future outcomes.
What You Can Do Today
We need to discuss risk assessment techniques in a way that acknowledges the advantages and challenges of both (real) quantitative methods and qualitative methods. We need to look to other professions such as actuaries and statisticians who have studied and analyzed probabilities for decades before risk matrixes even existed. We also need to stop inflating the effort required to implement quantitative risk assessment techniques. Quantitative risk assessments are not only for large, multinational organizations. Many of our current clients belong to the SME segment – all of which are getting the advantages of quantitative risk assessments.
Educate yourself on what quantitative risk analysis really is.
- Start with this 10-minute video on OpenFAIR.
- Have a look at the publicly available standards of OpenFAIR risk analysis.
- Familiarize yourself with the IEC-31010 standard which contains a much larger selection of risk assessment techniques than ISO/IEC-27005, including several quantitative methods, such as Monte Carlo simulations which are the basis of FAIR.
- Read the works of Sam Savage, Douglas Hubbard, Jack Jones and others on quantification of information security risk.
- Reach out to us for a talk on how you can move towards more probabilistic, quantitative risk management based on actual science.
