Evidence package

Evidence you can hand to somebody else

An auditor should not have to trust your screenshots, and you should not have to give them a login. An evidence package is the assessment, in a file, that verifies itself.

Where this is today

The archive is built, and it is in the product. What is not done yet is the part that matters most to a stranger: a package of ours, published, that anybody can fetch and check.

AvailableExports of every report, as HTML, Markdown, CSV or XLSX. Each one records the analysis run and the build that produced it.
AvailableThe evidence behind a finding, and the history of who decided what. Stored per run, and readable. The package is a way of handing it over, not a new place to keep it.
PreviewThe portable archive and its hash index. Built, downloadable from the reports screen, and it verifies with sha256sum on a machine that has never heard of us.
PlannedEVIDENT assessing EVIDENT, published. The first package we publish, produced by the pipeline rather than by hand. Until that exists, this row is the honest reason the one above is not Available.

Available is in the product today. Preview is built and reachable, and still changing shape. Planned is specified and not built — published in advance so it can be judged before it exists.

The evidence package panel in EVIDENT, showing the assessment it is of, the number of files, the digest of the conclusions and the command that verifies the archive — Asking the product what it would send. The digest is of the conclusions with the packaging left out, so the same assessment packaged tomorrow carries the same one. The command underneath is the whole verification procedure: no account, no network, and a tool the recipient already has. The list is the package repeating what it does not carry — read out of the file itself rather than written on the screen, so the two cannot come to disagree.

What is in the archive

evident-assessment-YYYY-MM-DD.zip
  manifest.json        what this is, when, of what, by which ruleset
  assessment.json      every field, and what was concluded about it
  evidence.json        what was read, what was inferred, and by which rule version
  findings.json        the findings, each naming its evidence and rule version
  controls.json        control mappings — absent until a framework maps any
  review-history.json  every human decision, actor, time, reason, and what it replaced
  reports/             the rendered documents, as Markdown
  hashes.sha256        a hash for every file above, in the format sha256sum reads

What makes it useful

It verifies with a tool you already have. sha256sum -c hashes.sha256, offline, on a machine that has never heard of us. Our verifier recomputes every digest rather than trusting what the package says about itself — a check that trusted the manifest would pass any package whose manifest was rewritten alongside its contents.
One number answers "has anything changed". The manifest carries a digest of the conclusions with the packaging left out, so a package built this morning and the same one built tonight agree, and last quarter's package can be compared with this one without diffing two hundred files.
It carries the reasoning, not just the conclusions. Each finding names the evidence behind it and the exact rule version that raised it, so the archive can be read by somebody who was not there.
It carries the human decisions, with what they replaced. Who confirmed what, who rejected what and why, with dates — and a superseded decision travels beside the one that replaced it rather than being flattened into a current value.
It says what it does not carry. No credentials, no rows from the analysed databases, no working copy of your source, no identities beyond the actors on its own decisions. Written inside every package, so nobody reads an absence as a finding.
It says what it cannot prove. The package is not signed. The digests show it is intact; they do not show who wrote it, and the manifest says so rather than leaving you to work it out.

Why this page still says Preview

This page described the format before it was built, so that it could be argued with in advance. It is built now: the archive above is what the product writes, it downloads from the reports screen, and a package taken from a running deployment opens with unzip and passes sha256sum -c with no account and no network.

It is still marked Preview, and the reason is the row above that is not: we have not yet published a package of our own. A format whose whole argument is that you should not have to take our word for it cannot be sold as finished while the only way to see one is to take our word for it. The first published package will be EVIDENT assessing EVIDENT, produced by the pipeline rather than assembled by hand — and that is what moves this to Available.