How Canary verifies an audit

Concrete and reproducible. Confidence is not evidence.

1. Claim extractionWe list the checkable assertions an audit makes about a tool — where it sends data, to whom, in what jurisdiction, and whether that is disclosed.
2. Evidence retrievalWe install and run the tool in an isolated container, feed it traceable decoy data, and intercept every outbound request (TLS terminated against our own CA, iptables transparent redirect). A decoy beacon self-test confirms the capture is live — so 'no egress' is a proven negative, not a blind spot.
3. Evidence matchingEach claim is matched against the tool's full public doc surface (README, docs site, privacy/telemetry files, changelog, source). Before any 'undisclosed' is asserted, an independent adversarial reviewer is tasked with refuting it.
4. Coverage calculationWe compute Evidence Coverage: the share of an audit's claims that are backed by independently captured evidence rather than assertion.
5. Integrity scoringWe compute a 0–100 Integrity Score from real, checkable attributes — capture self-test, captured traffic, adversarial disclosure check, signature, version pin — and sign the verdict so the score itself is auditable.

Reproduce it

The scanner is open-methodology. Each verdict ships the exact command, the pinned version, the raw captured requests (redacted), a content hash and an Ed25519 signature. Re-run the scan and you get the same evidence. See any verdict for its reproduce block.