Analytic confidence and estimative language
These are two different axes. You can be highly confident that something is unlikely. If you cannot say that sentence and explain it, you will lose points in every scenario round.
This is the highest-leverage page on the site for interview performance. It covers a small amount of material that a large share of candidates get wrong, and interviewers use it as a fast proxy for whether someone has done intelligence work or only read about it.
The two axes
Probability is your estimate of how likely something is to be true. Confidence is how much you trust that estimate, given the quality of your sourcing and the strength of your reasoning. They vary independently:
| Statement | Probability | Confidence | Reading |
|---|---|---|---|
| Multiple independent, reliable sources agree this group is not active | Low | High | Confident that it is unlikely |
| One unverified forum post claims a breach | Roughly even | Low | A coin flip we cannot yet improve |
| Strong technical overlap across several corroborated intrusions | High | High | The rare comfortable case |
| A single vendor blog asserts a nation-state link with no evidence shown | High per source | Low | Report the claim, not the conclusion |
Probability bands
US intelligence community directive ICD 203 defines a standard ladder, and most mature CTI teams use it or something close to it. The point is not the exact wording — it is that your team agrees on it and never mixes two scales in one product.
| Term | Range |
|---|---|
| Almost no chance / remote | 01–05% |
| Very unlikely / highly improbable | 05–20% |
| Unlikely / improbable | 20–45% |
| Roughly even chance | 45–55% |
| Likely / probable | 55–80% |
| Very likely / highly probable | 80–95% |
| Almost certain / nearly certain | 95–99% |
Confidence levels
Confidence is usually expressed in three bands, and each should be justified by why, not merely asserted:
- High confidence — corroborated by multiple reliable, independent sources; consistent with a well-understood pattern; few competing explanations survive.
- Moderate confidence — credible sourcing but with gaps, or plausible alternatives that cannot be excluded.
- Low confidence — fragmentary, uncorroborated, single-sourced, or the reasoning depends on assumptions you cannot test. Say so and publish anyway if it is decision-relevant.
"We assess with high confidence that it is likely APT-something was responsible." This reads as hedging stacked on hedging. Either state the probability and separately justify the confidence, or say plainly what you do not know.
Words that hide a judgement
These make an assessment unfalsifiable, which means it cannot inform a decision:
- "May" / "could" / "possibly" used alone. Almost anything could happen. If you cannot put a band on it, say why you cannot.
- "Sophisticated" — usually a stand-in for "we do not understand it yet". Describe the specific capability instead.
- "Advanced persistent threat" as a description of behaviour rather than a named cluster.
- "Suspected to be linked to" with no statement of what the link is or how strong.
- Passive voice around the judgement — "it is believed that" hides who believes it and on what basis.
Writing it properly
A well-formed assessment has four parts, and they should be separable:
- The judgement, with an estimative term. "This activity is likely the work of a financially motivated intrusion set rather than a state-directed one."
- The confidence, with its basis. "We hold this at moderate confidence: the tooling and victimology are consistent across three corroborated intrusions, but all reporting traces to two sources that may share collection."
- The key evidence, briefly.
- What would change it. "Evidence of targeting without a monetisation path would move us toward a state-directed assessment." This last part is what senior analysts do and juniors omit.
The "what would change my mind" clause also demonstrates the habit behind Analysis of Competing Hypotheses, which is the discipline this language exists to serve. Pair it with honest source grading and you have covered most of what a scenario round actually scores.
Run a mock interview on this material. You get the question, a graded response against a vetted rubric, and the probing follow-up an interviewer would ask next.
Related reading
- Attribution and its limits Attribution is the fastest way to fail a scenario interview. The skill is not naming the actor — it is knowing what your evidence can and cannot support, and saying so.
- Source grading for analysts A reliable source can report something false. An unreliable source can be right. Grading them on one axis loses the information that matters.
- Structured analytic techniques ACH is not a spreadsheet ritual. It is a discipline for finding the evidence that would prove you wrong, which is the only kind that carries much information.
- Writing intelligence reports You will be judged on your writing more than your tooling. The bar is a judgement up front, evidence behind it, and an explicit "so what" for someone you have named.