The GitHub Action

The Action is how corral becomes a gate rather than something you remember to run. It installs itself, audits only the files a pull request touched, and writes the verdict to the run page verbatim.

- uses: actions/checkout@v4
  with:
    fetch-depth: 0
- uses: pdbethke/corralai@v0.8.1
  with:
    test-command: "go test ./..."
    anthropic-key: ${{ secrets.ANTHROPIC_API_KEY }}

That is the whole workflow. There is no corral for you to install — the action’s first step go installs it into a private GOBIN and puts it on PATH. The one requirement is a go binary already on the runner; GitHub-hosted runners ship one.

Pin a tag, not a branch

v0.3.0 and later carry an action.yml; v0.1.0 and v0.2.0 predate the action and do not. Prefer pdbethke/corralai@v0.8.1 over @main, so a push to main cannot change what runs in your CI, or pin the commit SHA you reviewed if you want a reference a re-tag also cannot move.

fetch-depth: 0 is not optional

The changed-file set is computed with a three-dot range against the merge base, because that is what “what this PR changed” means. GitHub’s default checkout is depth 1 and has no merge base to find. On a shallow checkout the diff computation fails closed — exit 1, never a silent full-repo scan. A missing fetch-depth: 0 is the single most common way a first run breaks.

What it said about this repository

We ran it against the file that starts our own program, cmd/corral/main.go, as a gated check on a real commit:

kill rate 0.25 (30 survivor(s), 1 proven missed)
the pool's authored test PROVED 1 of 30 survivor(s) catchable by execution

Forty faults planted, ten caught, thirty missed, and one of those thirty proven catchable by a test the pool wrote itself. The run took 11m12s on a 2-core hosted runner.

That number is bad and it is ours. A gate whose author can quietly not-publish the result is furniture.

What one run costs

Roughly (mutants × your suite’s whole runtime) per audited file. It scales with how long your tests take and how many files the PR touched — not with the size of the change. The 11m12s above is a measurement from one repo, not a law; your first run is the only timing that really applies to yours.

An earlier version of our own documentation predicted around two hours for that file. It was wrong by roughly ten times, in the direction that would have talked you out of trying it. We are leaving the correction visible because it is the same failure mode this tool exists to catch: a plausible number nobody executed.

Three levers keep a run bounded:

• top — audit at most this many of the highest-ranked candidate files (default 25). The diff narrows the candidates; top bounds what is left.
• paths: on the workflow — so a docs-only PR doesn’t spend time printing NOTHING IN SCOPE.
• Leave diff-base at its default. Passing it empty audits the whole repository, which is a deliberate opt-in, not a default.

A diff that touches no auditable candidate is a legitimate pass: the action prints NOTHING IN SCOPE: and exits 0.

Giving it teeth

By default a graded file exits 0 no matter what kill rate it measured — a file where every mutant survived merges as cleanly as a perfect one. That is deliberate: adding a default would silently change the exit code of every existing caller.

- uses: pdbethke/corralai@v0.8.1
  with:
    test-command: "go test ./..."
    anthropic-key: ${{ secrets.ANTHROPIC_API_KEY }}
    min-kill-rate: "0.7"

The check is per file, not on the aggregate — a well-tested file elsewhere in the PR cannot average out or mask a weak one. 0.7 means at least 70%: a file at exactly 0.70 passes, 0.69 fails the run, and the report names every breaching file on its own line:

  KILL-RATE BREACH: 1 file(s) below --min-kill-rate 0.70:
    0.40  pkg/widget.go (0.30 below threshold)

Reach for this only once you have real timings from your own repo. A required check that can take hours is not a merge gate anyone will keep.

Where the verdict shows up

The report goes to the job summary — the page you land on when you click the check — as the report verbatim, not a rendering of it. The kill rate, the weakest files, and the lines that qualify what those numbers mean (NOT AUDITED, DID NOT FINISH, WRITER FAILED, TEST UNSOUND) are the same bytes corral printed. A second renderer would be free to drift, and drift in a summary always flatters the run, because the lines that get dropped are the qualifying ones.

It uses $GITHUB_STEP_SUMMARY, so it needs no permissions: block and works on fork pull requests where a PR-comment token doesn’t exist. The report reaches the summary even when the run fails — a red X whose reason was discarded is the problem this exists to fix.

Who pays, and how not to pay for a stranger

The bill lands on the repository that runs the workflow: your runner minutes, your API key. On a public repo an outside contributor’s pull request would spend your money, once per push, for as long as they keep pushing.

GitHub withholds secrets from fork pull requests, so an audit on a fork skips on its own — but don’t rely on that alone. The specific way it gets undone is someone reaching for pull_request_target because “fork PRs skip.” That trigger runs with your secrets while checking out the contributor’s code, so it does not merely spend your key, it exposes it. If fork PRs skipping looks like a bug, it is the feature.

Say it in the workflow instead, and add an opt-in so no pull request — yours included — starts a paid job merely by existing:

on:
  pull_request:
    types: [opened, synchronize, reopened, labeled]
    paths: ["**.go"]
  workflow_dispatch:

jobs:
  audit:
    if: >-
      github.event_name == 'workflow_dispatch' ||
      (github.event.pull_request.head.repo.full_name == github.repository &&
       contains(github.event.pull_request.labels.*.name, 'audit'))

.github/workflows/self-audit.yml in this repository is exactly that shape — non-blocking, top: "1", Go-only paths, fork-guarded and label-gated — and is the honest starting point to copy.

Keys are additive

anthropic-key, gemini-key and openai-key are not alternatives; set as many as your role routing needs. An unset key is never exported as an empty variable, and no key value is ever echoed.

A key alone does not move providers. corral routes each role to its own model and the defaults are claude-*, so pointing only a key at another vendor leaves Claude model names aimed at an endpoint where they don’t exist. Set derive-model, writer-model, mutant-model and critic-model too. The critic must differ from the writer — that decorrelation is enforced, not advisory — and critic-model: off disables it entirely, which is reasonable when one vendor gives you only one usable model. The critic never gates the verdict either way.

A receipt a reviewer can verify

A job summary is a screenshot. attest: "true" publishes the verdict as a signed in-toto statement through GitHub’s attestation API, and attaches the statement itself to the run as an artifact:

permissions:
  contents: read
  id-token: write
  attestations: write

# ...
    attest: "true"

Signing is keyless, through the workflow’s own OIDC identity. That matters more than the format. A certify --local run signs with a key generated on the machine, so on a disposable runner every audit is signed by a fresh key that chains to nothing; an attestation chains to the repository, the workflow and the commit — which is what a reviewer actually wants to check. Free on public repositories.

Anyone can then verify it without trusting the log, or you:

gh attestation verify corral-audit-statement.json --repo <owner>/<repo>

gh attestation needs a recent GitHub CLI — it does not exist in 2.45, which some distributions still ship, and an older CLI answers with a help dump rather than an error. The same answer straight from the API, with no CLI version to care about:

DIGEST=$(sha256sum corral-audit-statement.json | cut -d" " -f1)
gh api repos/<owner>/<repo>/attestations/sha256:$DIGEST

What the statement says

Every audited file’s kill rate, survivors and proven gaps — with the flags that say what a zero means (timedOut, testWriterFailed, poolTestUnsound, because provenMissed: 0 means nothing was proven rather than the suite is clean whenever one is set) — plus the thresholds it was judged against, the model in each role, and the audited-of-candidates denominator, so a clean result cannot be flattered by omitting how little was looked at.

It is written before the gate’s exit code is honoured, and the attest steps run under always(). A receipt you only keep when the verdict flatters you is not evidence, and the failing runs are the ones a reviewer most needs.

Sending verdicts to your own warehouse

One audit answers is this change defended. It cannot answer whether your tests are doing anything across your projects over time, and that needs somewhere to put the rows:

    push: "md:my_database"                     # or a DuckDB path
    motherduck-token: ${{ secrets.MOTHERDUCK_TOKEN }}

The warehouse is yours. corral has no hosted tier and collects nothing — your key, your runner, your database — and any DuckDB works, so this is a destination rather than a lock-in.

It is append-only, because a receipt you can UPDATE is not a receipt and overwriting is how a trend is lost. Every row carries the sha256 of the signed statement it came from and the run URL, so a row traces back to something a third party can verify.

What it is for is the question a single run cannot answer. A kill rate is a sample: mutants are generated fresh each time, and the same unchanged file has scored 0.375 and then 0.525 here. Forty rows are a distribution — “this file drifted from 0.9 to 0.6 over two months” is a claim no individual run supports. See the warehouse for the queries worth running.

The gate that does not flap

min-kill-rate is the obvious merge gate and the wrong default. A kill rate is a proportion of freshly generated mutants, so it moves between runs on code nobody touched — this project’s own warehouse records a file whose rate swung 0.375 to 0.525 across a replicate series with nothing changed but the seed. Set the threshold near a healthy value and it goes red on good work; the team turns it off, and you have no gate at all.

max-proven-missed keys on the claim that does not move:

    max-proven-missed: "0"

A proven-missed gap is a survivor the herd then killed with a test it wrote and ran. Not a proportion, not an opinion — a specific demonstrated bug your suite does not catch, established by execution. 0 means any demonstrated gap fails the build.

It fails closed. If the herd had survivors but could author no test that graded them, proven_missed reads 0 because nothing was proven — not because the suite is clean. The run reports that case separately (PROVEN-GAP UNMEASURED) and fails rather than passing on a question nobody answered.

Both gates compose: min-kill-rate as a loose floor, max-proven-missed as the thing that actually blocks a merge.

One key, if that is all you have

Three provider inputs and a decorrelation rule read like corral needs an account with every vendor. It does not. The gate’s verdict is measured by execution in a jail — the critic is advisory and never gates it — so a single key runs the whole thing:

- uses: pdbethke/corralai@v0.8.1
  with:
    test-command:  "pytest -q"
    openai-key:    ${{ secrets.OPENAI_API_KEY }}
    mutant-model:  gpt-5.1-codex
    writer-model:  gpt-5.1-codex
    critic-model:  "off"
    shadow-model:  "off"
    min-kill-rate: "0.8"

That is a real gate. Faults get planted, your suite runs against every one of them in the jail, and the kill-rate either clears the bar or fails the check.

What you give up, and corral says so out loud. With every graded seat on one vendor there is no independent read, and the verdict block prints:

decorrelation: every graded seat is google — if this code was WRITTEN by a
google model, the same lineage planted the faults and graded the tests. Point a
role at another vendor (--critic-model / --mutant-model) for an independent
read.

That warning is the point of the mode, not an apology for it. If an agent from one lineage wrote the change, having the same lineage decide what counts as a fault is a narrower check than it looks — the failures a model cannot imagine in its own output are exactly the ones it will not plant. A second vendor in any one seat costs one more key and buys back the independence.

A disabled critic reports as disabled. The verdict says critic review: not run — no test-critic was assigned, never “no vacuous tests flagged”. An absent reviewer does not get to hand out a clean bill of health.

Exit codes

• 0 — the scan graded at least one file and every audited file met min-kill-rate (if given); or nothing was in scope. With min-kill-rate unset, a weak-but-gradable suite still exits 0 — read the report for the number.
• 1 — a real failure: files were in scope and none could be graded (COULD-NOT-GRADE:, e.g. every candidate’s baseline suite was already broken or flaky), enumeration failed, or at least one file scored below min-kill-rate (KILL-RATE BREACH:).

The full reference

Every input, the quoting rules for test-command, why the action deliberately avoids actions/setup-go, and why files are graded one at a time on this substrate are documented in docs/corral/github-action.md in the repository.