Interview question
Explain the Admiralty source grading scale. How would you grade a well-regarded vendor's report making a claim that no other source has corroborated?
A question with a specific correct shape: the scale grades source and information separately, and this scenario is exactly the case where the two diverge.
What a strong answer covers
- Source reliability graded A to F: A completely reliable, B usually reliable, C fairly reliable, D not usually reliable, E unreliable, F cannot be judged.
- Information credibility graded 1 to 6: 1 confirmed by other sources, 2 probably true, 3 possibly true, 4 doubtful, 5 improbable, 6 cannot be judged.
- Reliability is a property of the source and its track record; credibility is a property of this specific claim.
- Grades the scenario around A6 or B6 — high reliability, credibility that cannot be judged for want of corroboration.
- Explains that this pairing is common and awkward, and that the notation exists to force you to notice it.
- Names the trap: letting source reliability leak into the credibility grade.
- Carries the grade through to the confidence level in the resulting assessment.
- Mentions circular reporting as the thing that fakes corroboration.
Expert answer
The Admiralty scale, sometimes called the NATO system, grades every piece of reporting twice. Source reliability runs A to F: A completely reliable, B usually reliable, C fairly reliable, D not usually reliable, E unreliable, F reliability cannot be judged. Information credibility runs 1 to 6: 1 confirmed by other sources, 2 probably true, 3 possibly true, 4 doubtful, 5 improbable, 6 truth cannot be judged. A rating looks like B2.
The point of the pairing is that reliability is about the source — its track record, methodology, access and incentives — while credibility is about this particular claim, judged mainly on independent corroboration and internal consistency.
For the scenario described, I would grade it around A6 or B6. The source is highly reliable, and the credibility of this specific claim genuinely cannot be judged because nothing corroborates it yet. Not 1, and not 2 either — "probably true" would be importing the source's reputation into the credibility axis, which is exactly the failure the two-axis system is designed to prevent.
That combination is common and uncomfortable, and the notation earns its keep precisely because it forces you to write down that you are trusting rather than corroborating. It is not a reason to discard the report. It is a reason to be explicit that a subsequent assessment rests on a single uncorroborated source, and therefore cannot be held at high confidence however good that source is.
The other thing I would check before believing I had corroboration is circular reporting. If two more outlets pick this up next week and both cite the original, I still have one source, not three. Genuine independence also means independent collection — two vendors drawing on the same telemetry partner are not independent even if they never cite each other.
Mistakes that cost candidates points
- Getting the direction of the scales wrong, or merging them into one rating.
- Grading the uncorroborated claim as 1 or 2 because the source is trusted.
- Not explaining why reliability and credibility are separate.
- Discarding the report entirely rather than grading it honestly.
- No mention of circular reporting or of independence of collection.
You have read the answer, which is the easy part. Answer it in your own words and get graded against this same rubric, with the follow-up probe an interviewer would ask next.
Go deeper
Related questions
- What threat intelligence do you read regularly, and how do you decide whether to believe a vendor report?
- A vendor publishes a report claiming an intrusion set is actively targeting your sector, with 200 indicators appended. Your CISO forwards it and asks "are we affected?" Walk me through your response.
- Four separate publications report the same breach claim. How do you decide whether that counts as corroboration?
- How do the Diamond Model, the Cyber Kill Chain and MITRE ATT&CK relate to each other? Do they compete?
- An endpoint alert shows that a scheduled task was created which launches a PowerShell one-liner from a user's AppData directory. Map this to MITRE ATT&CK.