What is the purpose of an Analysis of Competing Hypotheses?
Recently, I taught cybersecurity experts how to use Structured Analytic Techniques (SATs). 
Included, of course, was the Analysis of Competing Hypotheses (ACH).
And because this technique is so relevant, I want to portay the ACH itself and varios use cases.
This article includes the following content:
Table of Contents
How do you perform an Analysis of Competing Hypotheses?
To fully understand the ACH, I recommend reading Richards Heuer’s explanations in his genius book „Psychology of Intelligence Analysis“. You can download it here for free.
Dick Heuer is the inventor of ACH and to anyone who wants to learn more about ACH, I highly recommend reading the original. Parts of the ACH are described in more detail than for example in the standard work „Structured Analytic Techniques for Intelligence Analysis“ by Richards Heuer and Randolph Pherson.
The corresponding chapter starts on page 95.
So how do you perform an ACH? Here is the step by step guide of the Analysis of Competing Hypotheses:
Step 1) The starting point is as complete a set of hypotheses as possible (i.e. of possibly true explanations for an event, a development or similar). So start by identifying the possible hypotheses to be considered. Use a group of analysts with different perspectives to brainstorm the possibilities.
Step 2) Make a list of significant evidence and arguments for and against each hypothesis.
Step 3) Prepare a matrix with hypotheses across the top and evidence down the side. Analyze the „diagnosticity“ of the evidence and arguments–that is, identify which items are most helpful in judging the relative likelihood of the hypotheses. So here I compare each hypothesis with all the evidence, facts and assumptions that influence my thinking about the case. Thus, I apply each piece of information / data etc. to each hypothesis and ask myself the question, „To what extent would I expect to see this information, assumption, etc. if the corresponding hypothesis were true?“
The evaluation options are:
1) I would expect that: C (for consistent) or CC (very consistent)
2) I wouldn’t expect that: I (for inconsistent) or II (very inconsistent)
3) Not applicable: N/A
Step 4) Refine the matrix. Reconsider the hypotheses and delete evidence and arguments that have no diagnostic value.
Step 5) Draw tentative conclusions about the relative likelihood of each hypothesis. Proceed by trying to disprove the hypotheses rather than prove them. In the end, the most likely hypothesis is the one with the least evidence against it (which has the fewest Is), not the one with the most evidence for it (which has the most Cs).
Step 6) Analyze how sensitive your conclusion is to a few critical items of evidence. Consider the consequences for your analysis if that evidence were wrong, misleading, or subject to a different interpretation.
Step 7) Report conclusions. Discuss the relative likelihood of all the hypotheses, not just the most likely one.
Step 8) Identify milestones for future observation that may indicate events are taking a different course than expected.
The ACH thus corresponds to the procedure of medical diagnostics, in which, in addition to the most obvious explanation for a disease, differential diagnoses are also established that are also consistent with the symptoms and could explain the disease. Here, too, tests, examinations and questioning are used to try to exclude possible diagnoses (Dr. House and team send their regards 
).
So with the ACH, I can develop a sense of the most likely explanation even in complex or confusing situations. It is the Swiss army knife of analytical techniques.
The ACH is not a simple technique, but one that is worth having in your toolbox. To help you gain a deeper understanding, I’ve put together different use cases.
Use Case 1: ACH and Cyber Threat Intelligence (CTI) - Example 1
In 2017, Digital Shadows has taken the effort to perform an ACH on the origin of WannaCry – old but gold, to see a use case of the technique in the cyber domain.
Here you can read the whole case study.
Spoiler alert: the most likely authorship, according to Digital Shadows‘ analysis, is: An unsophisticated financially-motivated cybercriminal actor.
And one more little note from my own experience. If you look at the hypothesis set of Digital Shadows, you will notice that the hypotheses H3 and H4 are not completely mutually exclusive:
– H3: A nation state or state-affiliated actor conducting a disruptive operation
– H4: A nation state or state-affiliated actor aiming to discredit the National Security Agency (NSA)
Why? Because I can conduct a disruptive operation in order to discredit another actor (here the NSA).
Sometimes you can’t avoid having hypotheses that are not mutually exclusive. However, there may be difficulties in the execution of the ACH or in the result in such cases, because unclear results may be obtained. Like in this case: Nearly all assessments of H3 and H4 are the same.
Knowing this in advance can be helpful in order not to get frustrated.
Nevertheless, the ACH remains a suitable tool even for these cases, as it can at least show tendencies.
Use Case 2: ACH and Cyber Threat Intelligence (CTI) - Example 2
The ACH can also be used for Incidence Response or for investigation engagement of a potential risk, as can be seen in this example.
eCrimeLabs published a very short application at the end of 2021 using the ACH for the example of “THE DFIR REPORT” BazarCall to Conti Ransomware via Trickbot and Cobalt Strike.
Quote: “The threat actors used BazarCall to install Trickbot in the environment which downloaded and executed a Cobalt Strike Beacon. From there the threat actor discovered the internal network before moving laterally to a domain controller for additional discovery. A couple days later, the threat actors came back and executed Conti ransomware across the domain.”
Definitely an interesting application area for the ACH. 🤓
In principle, I could also imagine that the ACH could be used to investigate the following questions in the area of CTI:
– Is an attack on our system completed / over or is it on-going?
– Can we safely use our system again after decryption in this case?
Use Case 3: ACH and Foresight Analysis
The question in this use case is: How can I apply ACH in the context of foresight analysis?
From my perspective, ACH is not my favorite technique when it comes to generating scenarios. Nevertheless, it has its strengths, especially when there are already many data points that I can evaluate with an eye to the future.
Kristan Wheaton published a paper on this in 2018 that not only explains how ACH can be used for foresight, but also extends ACH to Structured Analysis of Competing Hypotheses (SACH).
So those in the field of foresight analysis may find further inspiration here.
Use Case 4: ACH and Data Analytics and Customer Behavior
And finally I want to show you a very interesting use case of the ACH. It is very interesting because it is not related to security: It is about data analytics or customer behavior.
Moe Kiss shows a (simplified) example where the question is:
„Why aren’t more mobile site users moving to our iOS app?“
So we’re moving way outside the realm here of what you might think of as the ACH. This is precisely why the article is so exciting and makes one thing very clear:
➡ The ACH can in principle be used in ANY case where I want to answer a question based on incomplete, contradictory or too extensive data with the help of competing hypotheses.
That completes this blog-post on ACH which originated in a series of posts I did on LinkedIn.
I hope there was something exciting for everyone.
If you would like to have yourself or your staff trained in the use of Structured Analytic Techniques or need advice on your intelligence unit, feel free to contact me and we’ll see how I can support you. 👍