To Quantify or not to Quantify, That is the Question

Ahmeed Ahmeed, Cyber and Information Security Director, Inteva Products

The CISO at Acme Corp’s long-sought cybersecurity tool is funded on the spot after a bad cyber-attack that cost her organization millions of dollars. As she accepts the new funds, she recollects the years of failed attempts to convince the board to fund the tool. The board was just not convinced to make the expensive investment at the time. Today, her organization pays the price in folds.

Whose fault is it? Is it the board’s fault for not taking the CISOs advice to spend the money and mitigate the risk, or is it the CISO’s fault for not convincing the board of the necessary additional funding? To answer that, we should talk about risk management first.

Risk management is at the core of every successful Information Security program. Security leaders are very much risk-aware by experience. They can spot the difference between an unlikely catastrophe and an almost-certain operational hiccup because they are aware of the two components that make up risk (impact severity and probability of occurrence).

Risk assessment and risk management are not always an information security function. If you are lucky enough to work for a financial institution or an insurance provider, chances are that an established and mature risk management structure already exists at the organizational level. However, the same cannot be said for the majority of industries where islands of risk management may or may not exist inside pockets of different departments and functional units. As a security leader of such organizations, you will have to build your risk management program and structure from the ground up.

To assess risks easily and effectively, you can use risk qualification. This is where the assessment process uses a wide subjective brush, and the 

assessment results fall into a few buckets like high, medium, or low risk. ‘Quick and easy’ is what makes risk qualification so popular, but it comes with the price of inaccuracy, inconsistency, and, worst of all, its inability to translate into business language. The results of your assessment will yield a local unit that may not be well understood outside of your department. Think of a European friend telling you the temperature outside is 38^o Celsius. You just have no feel for it. The same goes for the board. A qualitative “very high” risk could mean different things to different people.

To overcome these risk qualification problems, security leaders use risk quantification instead. Risk quantification is when you apply a real-world unit to measure risks. Most commonly, that unit is currency (a dollar figure). This solves the problem of using business language off the bat. Additionally, risk quantification solves other inherent issues with risk qualification, like subjectivity and reliability of results. That is because risks, in this case, are calculated using mathematical modeling.

"When you deal with statistical data, you’ll notice that event frequencies are distributed on a spectrum and are not always a flat binomial fifty-fifty coin toss."

But it’s not all warm and fuzzy with risk quantification. Quantification comes with a high price of complexity, difficulty of production, and time consumption. For the majority of organizations, it is not feasible to quantify all risks.

The solution is then to use risk quantification for when it is worth the time and effort. That is usually when (1) Security leaders are legitimizing funding for a security expense, (2) When results are too close for a verdict, (3) When the results are too subjective to defend, and (4) When security leaders are making a major decision with their security architecture.

Now that it is clear when to quantify, the question remaining, is how to quantify. If your organization is new to risk quantification, it is best to ease into it. First, use simple modeling by substituting impact severity with impact in currency (say US dollars). Your basic risk formula becomes

Soon you will find that this quantification equation is too simple to cover complex risk scenarios. At that point, you are ready for the next step, which is to 

model your risk in a decision tree, event/incident tree, or fault tree analysis. The tree model allows you to convert the simple single equation of flat consequence and probability into a map of interconnected events, each with its own consequence and probability.

1 Decision Tree analysis for data center outage risk. A $15M outage cost is diluted to $41 value due to the low probability. This figure was created using Ybian tree plan excel add-on.

Tree modeling takes your risk assessment a long way with much more accurate measurements, better identification of hidden dependencies, and the variety of ways a risk is realized; but it doesn’t address the distribution of probabilities. When you deal with statistical data, you’ll notice that event frequencies are distributed on a spectrum and are not always a flat binomial fifty-fifty coin toss. To employ probability distribution in your model, you can simulate your scenarios using the Monte Carlo method. In a Monte Carlo simulation, you model your risk with the relative probability distributions and use computation to simulate the scenario a high number of times. The results will be an aggregation of the simulation results running thousands of times. The results then can provide a distribution of monetary losses due to the risk being realized.

The CISO of Acme Corp would’ve had a better chance in funding the security tool prior to the attack if the board had access to a dollar-to-dollar comparison between funding and taking the risk, and Acme Corp could’ve avoided the attack.

In conclusion, setting criteria for when to quantify and establishing the foundation for risk quantification are two keys to the success of a risk-based security program.