Interview question
What is the difference between saying something is "likely" and saying you have "high confidence" in an assessment? Can you have high confidence in an unlikely outcome?
The single highest-leverage tradecraft question in CTI interviews. A large share of candidates conflate the two axes, and interviewers use it as a fast proxy for real analytic experience.
What a strong answer covers
- Probability is the estimate of how likely something is to be true.
- Confidence is how much you trust that estimate, based on sourcing quality and reasoning strength.
- They are independent axes and vary separately.
- Answers yes to the second question and explains why: strong corroborated evidence can support a firm judgement that something is improbable.
- Gives a concrete worked example of high confidence with low probability.
- May reference an estimative scale such as the ICD 203 bands, and notes not to mix a verbal term with a numeric percentage in the same statement.
- Notes that stacking both without explaining them — "high confidence it is likely" — reads as hedging.
- Mentions that confidence should be justified by its basis, not merely asserted.
Expert answer
They are two different axes, and keeping them separate is most of the skill.
Probability is my estimate of how likely something is to be true — "this activity is likely the work of a financially motivated actor". Confidence is how much I trust that estimate, given how good my sourcing is and how strong the reasoning chain is.
Yes, you can absolutely have high confidence in an unlikely outcome, and that combination is common. If several independent, reliable sources with genuine visibility all indicate a particular group has been dormant for a year, and I have no contrary evidence, then "it is unlikely this intrusion is theirs" is a low-probability judgement I hold at high confidence. The evidence is good; it just points away from the hypothesis.
The reverse happens too. A single unverified forum post claiming a breach might leave me at roughly even chance on whether it happened, held at low confidence, because I have almost nothing to work with.
For probability I would use a defined scale so the words mean the same thing to everyone — the ICD 203 bands are the common reference, running from almost no chance through unlikely, roughly even chance, likely, very likely, to almost certain. The important discipline is to pick one register: state the verbal term or a numeric range, not both in the same sentence, and never mix two different scales in one product.
Confidence I would express in three bands with the basis attached. High means corroborated by multiple reliable independent sources with few surviving alternatives. Moderate means credible sourcing with gaps, or plausible alternatives I cannot exclude. Low means fragmentary or single-sourced, or the reasoning rests on assumptions I cannot test. Saying "moderate confidence" without saying why is only half the job.
The sentence to avoid is "we assess with high confidence that it is likely X" delivered as one lump, because it reads as hedging stacked on hedging. If both apply, separate them: state the judgement with its probability, then justify the confidence on its own terms.
Mistakes that cost candidates points
- Treating probability and confidence as the same thing or as one scale.
- Answering "no" to whether high confidence can attach to an unlikely outcome.
- Giving no worked example.
- Mixing a verbal estimative term with a numeric percentage in one statement.
- Asserting a confidence level without explaining its basis.
You have read the answer, which is the easy part. Answer it in your own words and get graded against this same rubric, with the follow-up probe an interviewer would ask next.
Go deeper
Related questions
- It is 2am during a live incident and the incident commander asks you "is this ransomware?" You genuinely do not know yet. What do you say?
- Critique this line from a draft report: "It is believed that this sophisticated attack may possibly be linked to a nation-state actor." Rewrite it.
- How do the Diamond Model, the Cyber Kill Chain and MITRE ATT&CK relate to each other? Do they compete?
- An endpoint alert shows that a scheduled task was created which launches a PowerShell one-liner from a user's AppData directory. Map this to MITRE ATT&CK.
- You have one piece of evidence: a single IP address that a compromised host was beaconing to. Walk me through how you would develop this.