Interview question
Walk me through how you would use Analysis of Competing Hypotheses on an intrusion where the evidence is ambiguous. What makes ACH different from just weighing the evidence?
ACH questions separate people who can name the technique from people who understand its central counterintuitive claim: supporting evidence is nearly worthless.
What a strong answer covers
- States the core insight: evidence consistent with a hypothesis is weak because it is usually consistent with several; what discriminates is inconsistency.
- Enumerates hypotheses before examining evidence closely, and includes ones considered unlikely.
- Lists evidence including relevant absences — what should be present if a hypothesis were true.
- Builds a matrix marking each item consistent, inconsistent or not applicable per hypothesis.
- Discards evidence with no diagnostic value rather than counting it as support.
- Ranks by fewest inconsistencies, not by most supporting evidence.
- Runs a sensitivity check — which single item, if wrong, collapses the conclusion.
- Reports alternatives alongside the lead hypothesis and defines indicators that would shift the ranking.
- Attributes the technique to Heuer or the intelligence analysis literature without inventing detail.
Expert answer
The thing that makes ACH different is counterintuitive: evidence that supports your favoured hypothesis is nearly worthless, because it usually supports several of them equally well. What carries information is evidence that is inconsistent with a hypothesis. So ACH works by elimination rather than accumulation, which is the opposite of how people naturally reason.
The process. First, enumerate hypotheses — before looking hard at the evidence, and deliberately including ones I think are unlikely. Getting a second person to add to the list matters here, because the hypotheses I omit are the ones my priors already excluded.
Second, list the evidence, including absences. "No exfiltration observed despite six days of functioning C2" is evidence, and it is often more diagnostic than anything present.
Third, build the matrix: hypotheses across, evidence down, each cell marked consistent, inconsistent, or not applicable.
Fourth, refine. Any row that is consistent with everything has no diagnostic value and should be struck out rather than counted as support for the leading hypothesis. This step is where most of the value is.
Fifth, rank by fewest inconsistencies. Sixth, sensitivity-test: which single item, if it turned out to be wrong or planted, collapses the conclusion? That item deserves the most scrutiny, and in cyber it is very often the one that was easiest to observe.
Finally, report the alternatives alongside the lead, and define the indicators that would change the ranking so collection can go after them.
Concretely, on an ambiguous ransomware intrusion I might carry four hypotheses: an opportunistic financially motivated affiliate; a targeted intrusion by an actor with prior access; destructive activity disguised as ransomware; and an insider. "A known ransomware family was deployed" is consistent with all four, so it gets struck. "No exfiltration despite a working channel" is inconsistent with the usual double-extortion pattern and starts to discriminate. "Initial access used credentials valid before the compromise window" bears on the insider hypothesis. That is the matrix doing work.
In practice nobody builds a full matrix during a live incident. What survives into daily work is the habit: generate alternatives unprompted, look for what would disconfirm, and strike the evidence that does not discriminate.
Mistakes that cost candidates points
- Describing ACH as weighing evidence for and against, which inverts its central idea.
- Not mentioning that non-diagnostic evidence should be discarded.
- Ranking by supporting evidence rather than by fewest inconsistencies.
- Omitting absences from the evidence list.
- No sensitivity check and no indicators for future revision.
- Inventing specific citations or dates for the technique's origin.
You have read the answer, which is the easy part. Answer it in your own words and get graded against this same rubric, with the follow-up probe an interviewer would ask next.
Go deeper
Related questions
- Tell me about a time your analysis turned out to be wrong. What happened, and what did you change?
- The SOC escalates: a finance team member's account authenticated successfully from an IP in a country the company has no presence in, at 03:00 local time. MFA was satisfied. No alerts have fired since. What do you do, and what do you tell the SOC lead?
- How do the Diamond Model, the Cyber Kill Chain and MITRE ATT&CK relate to each other? Do they compete?
- An endpoint alert shows that a scheduled task was created which launches a PowerShell one-liner from a user's AppData directory. Map this to MITRE ATT&CK.
- You have one piece of evidence: a single IP address that a compromised host was beaconing to. Walk me through how you would develop this.