Star
Search corralai.dev

The record, and the warehouse over it

How corral stores a verdict. Four lanes: THE RUN (corral certify, corral review and a human ruling, all of which die when the job ends); THE STORE (a signed verdict written as one gzipped JSON entry into the hash-linked orphan branch corral/ledger, with corral ledger verify walking the chain, corral ledger checkpoint pruning it, and Sigstore Rekor holding the receipt); THE VIEW (any DuckDB reading the branch in place over HTTPS with read_json_auto and httpfs, or corral ledger push sending the same rows to a local audits.duckdb file or to MotherDuck, creating eight tables and one view); and QUESTIONS (which files keep shipping proven gaps, whether a kill rate moved or resampled, corral models rank, corral brief, and what an audit costs). A dashed arrow carries priors from the branch back to the next run.

The problem: a runner forgets

A GitHub Actions runner is a throwaway machine. It boots, it works, it is destroyed. Whatever it measured about your code — which faults your tests missed, which reviewer’s claims held up, what the audit cost — dies with it. The log output survives for a while as text nobody queries, and then the retention window closes and that goes too.

So the honest state of most CI is: it has no memory. Every run starts from nothing and knows nothing about the hundred runs before it. You cannot ask “which files have been weak all quarter,” because nothing kept the answer.

The usual fix is to stand up a database, which means a server, a schema, a connection string, credentials in CI, and a bill — a lot of apparatus to hold what is, in the end, an append-only list of small facts.

The trick: the record is a git branch

Corral does not stand anything up. Every judgment it makes — an audit, a review, a human ruling — is written as one gzipped JSON file, and those files are committed to an orphan branch in your own repository called corral/ledger.

An orphan branch shares no history with main. It never appears in your diffs, never touches your working tree, and cannot collide with your code. It is storage that happens to be versioned, sitting in a place every CI runner already has credentials for.

That single move solves the impermanence:

  • The runner gets a memory. The Action checks out corral/ledger before the run, so a fresh machine arrives holding everything the repository has ever learned about itself.
  • The record survives the runner. After the run, the new entry is committed back. The machine is destroyed; the finding is not.
  • It travels with the repository. Fork it, mirror it, move to a self-hosted forge, hand it to an auditor — the record goes along, because it is the repository.
  • There is nothing to operate. No server, no schema migration, no secret beyond the token the workflow already has.

Each entry carries the hash of the entry before it and is signed with your certify key when one is configured. Edit an entry and its signature breaks; remove one and the next entry’s link breaks. corral ledger verify walks the chain from a clean checkout and names exactly which entry is wrong — so a stranger checks the record against its own signatures rather than against your word. Nothing is ever edited in place: a wrong entry is retracted by a later entry that says so, and every reader then leaves it out.

Check corral’s own record

Hash and link integrity need no key. Signatures need the key that made them, and corral’s is published: it is the one line in LEDGER_PUBKEY at the root of the repository.

git clone --branch corral/ledger --single-branch \
  https://github.com/pdbethke/corralai.git corral-ledger
corral ledger verify --pub 909429443700441a9f662c07c5f1b7110333fd7fb94fb21e8991e40f2cb6dc09 corral-ledger

Leave out --pub and corral looks for the key in CORRALAI_LEDGER_PUBKEY. With neither, it checks no signatures at all and says so: the entries read as signed, unverified, which is neither a pass nor a failure. corral never reaches for your local certify key to check a ledger, so you and a stranger get the same answer from the same record.

The key does not rest on this page’s say-so. The same key signs the statements corral enters into Sigstore Rekor, and Rekor entry 2759598612, integrated on 2026-09-08, records it. So you can check the key against a public log that corral does not run, and that log has held it since before this page published it.

The payoff: any DuckDB reads it, in place

Here is the part that makes the branch worth more than a log file. Those entries are JSON, and DuckDB reads JSON over HTTPS. So the record is already in an analytical engine — from the very first entry, with nothing installed and nothing running.

No clone. No server. No schema to adopt. This runs right now, against corral’s own public record:

import duckdb, json, urllib.request

repo = "pdbethke/corralai"
names = [f["name"] for f in json.load(urllib.request.urlopen(
    f"https://api.github.com/repos/{repo}/contents/scans?ref=corral/ledger"))
    if f["name"].endswith(".json.gz")]
urls = [f"https://raw.githubusercontent.com/{repo}/corral/ledger/scans/{n}" for n in names]

con = duckdb.connect()
con.execute("INSTALL httpfs; LOAD httpfs")
print(con.sql(f"""
  SELECT coalesce(kind, 'scan') AS kind, count(*) AS entries, min(pushed)::date AS first
  FROM read_json_auto({urls!r}, union_by_name = true)
  GROUP BY 1 ORDER BY 2 DESC"""))
┌──────────────┬─────────┬────────────┐
│     kind     │ entries │   first    │
├──────────────┼─────────┼────────────┤
│ adjudication │      59 │ 2026-09-06 │
│ review       │      25 │ 2026-09-06 │
│ scan         │       2 │ 2026-09-08 │
└──────────────┴─────────┴────────────┘

That is corral’s own ledger as of 2026-09-08 — audits, cold reviews, and the human rulings on them, in one chain. The counts grow as runs land, so treat the numbers as a snapshot and the shape as the claim: run the query yourself and the kinds will be there.

The same read_json_auto([urls]) works from any DuckDB client — the CLI, R, Java, the WebAssembly build in a browser. Locally, corral opens the directory as the same tables for you: corral verify --db, corral models rank --db, corral seal --db and corral brief all read .corral/ledger directly.

The questions a single run cannot answer

This is the reason to keep the entries at all. Each of these is one GROUP BY over the branch:

-- Which files keep shipping DEMONSTRATED gaps. Countable across repos, unlike
-- kill rates: a dense function and a small accessor are not the same
-- measurement, so averaging their rates says nothing.
SELECT repo, path, count(*) AS runs, sum(proven_missed) AS demonstrated_gaps
FROM corral_audits
GROUP BY 1, 2
HAVING demonstrated_gaps > 0
ORDER BY demonstrated_gaps DESC;

-- Drift, or sampling? Faults are generated fresh per run, so one number is a
-- sample. A spread tells you whether a change is real.
SELECT path, count(*) AS runs,
       round(avg(kill_rate), 2) AS mean,
       round(min(kill_rate), 2) AS worst,
       round(max(kill_rate), 2) AS best
FROM corral_audits
GROUP BY 1 ORDER BY mean;

-- The honest denominator: how much of what changed could be audited at all.
SELECT repo, sum(audited) AS audited, sum(candidates) AS candidates
FROM corral_audits GROUP BY 1;

Which model catches bugs in which language, what an audit costs per file, whose review claims held once execution checked them — all the same shape.

The next run reads it

The branch is not only for you. A run that starts empty is handed the earlier entries and uses them:

  • every fault already planted on a file’s exact bytes is shown to the generator as a prior, so it plants somewhere new instead of re-sitting the same exam;
  • a verdict for a file whose bytes have not changed is reused, and marked as reused rather than re-billed;
  • the review planner knows which scopes were reviewed, when, and which have changed since.

Every verdict says whether it was primed this way. A record that feeds the next run is the difference between a history and a memory.

The outside witness: Sigstore’s public log

Signing proves integrity — the verdict you are reading is byte-for-byte the one produced. It does not prove independent authenticity, because the key is still yours. A reader who distrusts you entirely gains nothing from checking a signature against a key you handed them.

That gap is what a public transparency log closes. With --attest corral writes the verdict as an in-toto statement; when a local signing key is configured (CORRALAI_CERTIFY_KEY_FILE), it also signs that statement into a DSSE envelope. Adding --transparency uploads the envelope to Sigstore’s Rekor, a public, append-only log run by people with no stake in your audit, and prints the receipt:

attestation logged: rekor index 2759598612 (uuid 108e9186e8c5677a...)

The index and UUID are recorded on the ledger entry itself, and with --push, in the warehouse. That is what makes the whole thing walkable by someone who trusts none of it — you can do every step of this right now, on the flask audit sitting on corral’s branch:

  1. Read the entry off the branch: scans/20260908T154158Z-36e4a824f340-d65faf854374.json.gz — the run in the certify video.
  2. Find bundle.Scan.RekorLogIndex in it — 2759598612, the same number you watch being issued on screen.
  3. Fetch that entry from Sigstore, a log we do not run and cannot edit.
  4. Confirm it is this file and not another:
corral verify --attest corral-audit-statement.json --rekor-index 2759598612

That last step compares the envelope’s own sha256 against the hash Rekor recorded, so a claimed log entry has to actually be that file. At no point does the check route through us.

Two things to know before you use it:

  • The entry is public and permanent. Once logged it cannot be removed or edited by anyone, including you. It carries the statement — repository URL, audited commit, per-file paths, kill rates, survivor and proven-gap counts, and the models in each role — and never the audited source itself. Decide that this is what you want before the first upload, not after.
  • It fails open. An unreachable log or a rejected entry prints one line and leaves the scan’s verdict, exit code, statement, envelope and ledger entirely untouched. Transparency is a witness, not a gate.

--transparency is a CLI flag today. The Action exposes attest but not yet transparency.

Sharing it: MotherDuck

The branch is one repository’s record. When you want the view across many, corral ledger push appends the same entries to a warehouse you own — a plain DuckDB file, or MotherDuck:

corral ledger push .corral/ledger md:my_database
# or a plain file, if you would rather not involve anyone
corral ledger push .corral/ledger ./audits.duckdb

A scan can also push as it goes, with --push on the run itself (the seats are required as on any run; the model names are an example):

corral certify --repo . --push md:my_database \
  --writer-model gemini-3.8-flash --mutant-model gemini-3.8-flash \
  --derive-model gemini-3.8-flash --critic-model claude-haiku-4-5 \
  -- .venv/bin/python -m pytest -q

From the Action that is push: plus a motherduck-token: secret when the target is md:. MotherDuck reads its token from the motherduck_token environment variable; there is deliberately no --motherduck-token flag, because a token on a command line lands in shell history.

A certify push creates a five-table bundle on first use (a review push adds corral_reviews, corral_findings and corral_adjudications) — corral_scans, corral_audits (one row per audited file), corral_mutants, corral_model_calls and corral_events — plus a corral_seal view that cross-references a pushed statement’s hash against the rows it produced. Rows are append-only; overwriting is how a trend is lost. Scripts and authored tests stay in the ledger unless you ask for them.

Each row carries the run URL and, only when the scan also ran with --attest, the sha256 of the signed statement it came from — so that row traces back to an attestation a third party can verify. Without --attest the statement_sha256 column is left empty rather than fabricated. The qualifier columns (timed_out, test_writer_failed, pool_test_unsound) sit beside the numbers rather than in a separate table, because aggregation is exactly where a proven_missed of 0 that means nothing could be proven gets misread as clean.

There is no hosted tier and nothing is collected: your key, your runner, your warehouse. Any DuckDB works, so MotherDuck is a destination rather than a requirement.

The browser demo, and what it actually shows

The DuckDB integration page on this site runs DuckDB itself — compiled to WebAssembly, in your browser — over a corral dataset, with eight preset queries and a live query box. A ?q=<sql> deep link pre-fills and runs a query, so a claim can be one click from the SQL that grounds it.

Be clear on what that page is: an older, hand-exported extract — parquet files of corral’s historical stores (audit_ledger, bug_catches, scans, scan_files) from the machine that produced them. It predates the ledger branch and has not been backfilled into it. It is a good demonstration that the query surface is real and safe to hand to anyone; it is not the current record. The current record is the branch, and the snippet above reads it live.

Because DuckDB there runs client-side in a WebAssembly sandbox over public, read-only data, there is no server-side SQL — nothing to inject into. The runtime loads from a CDN (the WebAssembly build exceeds a static host’s per-file cap); the data is self-hosted parquet.

Where to go next