In a previous article, When a robot gives better estimates than a human, Bo Thygesen from ACI describes how we use the LENS model to avoid human bias and have a “robot” estimate probability and loss for hundreds of systems better and faster than a human can do.
How can you get started right now? Start thinking about traits – the distinguishing characteristics of a system. Let’s start with what you might be doing today.
Sec_Questionnaire_2023Q2 – System.xlsm
You sent Excel questionnaires to all system owners with 90 questions about security. The questions are from well-known frameworks or standards, so you know they’re good questions and support best practices. Right?
You asked the system owner to score, e.g., Ensure Adequate Audit Log Storage and Uninstall or Disable Unnecessary Services on Enterprise Assets and Software.
Budgets are tight, so people are busy, to begin with. Some, but not all, of your colleagues, get back to you with the questionnaires filled in. One system owner spent a whole day answering these 90 questions, and another did it in 2,5 hours with the help of two external suppliers.
Did you get good answers? Were you able to reduce uncertainty in your risk assessment with those answers? Maybe, but what if you could reduce the uncertainty just as well (at least), with only 5 or 6 good questions, that are easy to answer?
Design a few good questions
We can ask many questions and get to know details about a system. Many details will have a low impact, making them less relevant. So how do we design good questions? You need to find experts on your systems and let them get to work! Systems have traits (or attributes, if you will) that are distinguishing characteristics that drive risk. To make sure that a trait is suitable, it must fulfil five requirements:
1. Variable
Traits that can only have one state are not variable and, therefore, not suitable.
Example: Is the system accessible via the open internet? In the case where a company only uses Software-as-a-Service, the state would always be yes.
2. Knowable
The trait must be something that can be easily answered with little or no effort – fingertip knowledge or looking up a value.
Example trait and knowable state: Which technology is used for authentication? MFA or tokens.
3. Objective
Traits should not be subjective, or opinion-based.
Example trait and state that is NOT objective: Knowledge level of operations staff? Very good.
4. Significant
The trait must significantly reduce the uncertainty about risk. Insignificant traits impact risk only very little or in certain rare circumstances. They are insignificant to reducing the uncertainty of loss magnitude or probability when aggregating risk across the entire organization.
Example of an insignificant trait and state: Number of external port scans daily? 50,000-100,000.
5. Independent
A trait must not depend on or be included in another trait. This would cause traits to be included in calculations twice down the line.
Three examples of traits and their states
If you want to use the traits with the LENS model and integrate them in calculations, you probably need external assistance to mathematically determine which traits are statistically most impactful. My good colleague Frederik Thygesen explains more about traits and states in this short video.
Traits can still be useful to you even though you don’t use the LENS model (yet). Let’s try to list a few:
Trait #1: Which type of authentication is used?
- Multifactor authentication
- Single sign-on
- Username and password (single factor)
- Unknown
Trait #2: With how many other systems are data exchanged?
- None
- 1-2
- 3-5
- More than 5
- Unknown
Trait #3: Does the system store or process personal data?
- No
- Yes, employee data
- Yes, customer data
- Unknown
The states are not a common ordinal scale across questions (like 1-5). Rather, they are appropriate to the trait and the company. They are distinct and easy to answer.
Notice that states are ordered according to increasing risk and that the last one is called “Unknown”. It’s certainly a red flag if the state of a trait that is supposed to be knowable is unknown.
But…
Yes. There are situations where a detailed security audit or assessment has merit and is a good tool for the job. But to reduce the uncertainty of estimates, they are often less effective. Now you have one more tool in your toolbox you can start to enhance and learn to use. You’re welcome.
This is interesting – I need a tool to…
No. Don’t start with a tool. Start by designing good questions – good traits – and use the tools you have available now and people are familiar with. There’s no shame in using Excel for a lot of things.
When you’re ready to do more, then yes, you can benefit from using tools designed for quantitative risk management like ACI’s Quantitative Assessment Platform (qAp) which is launching in 2023.
Stay tuned!
