Step 10 · Demo

Try the method

A public demo split you can run without access to private fixtures.

Repo: github.com/camerontjs-dot/verified-done

Most agent benchmarks report a pass rate. This one separates three outcomes:

verified pass — external check confirmed the change and the agent stayed in scope
false completion — agent claimed done; verifier disagreed
scope violation — agent edited a forbidden file

Run it

git clone https://github.com/camerontjs-dot/verified-done
cd verified-done
python3 runner/run.py list
python3 runner/run.py selftest

selftest proves every verifier discriminates (start fails, solution passes). Latest public summary: 8/8 tasks discriminate; scope detection fires correctly.

Two splits

Headline numbers from the private sealed lab are cited, not shipped. The demo tasks here were authored fresh so the method is inspectable without burning held-out fixtures.

Scrubbed live evidence (verify-tool ablation): LIVE_EVIDENCE.md (Findings A–F).

What the live evidence shows

A (Haiku): same model, run_verify off → 3/3 false completion; on → 3/3 clean.
D/E (local coder-14b): n=10 per arm, pooled across two Ollama runtimes: 20/20 false completions without the tool, 1/20 with it. The n=2 row is superseded.
C (local qwen3.5:9b): withdrawn as a control. The old “abstained either way” label was a timeout, then a harness bug that discarded reasoning.
F (local qwen3:14b): the actual model-dependence cell. 10/10 FC without the tool, 9/10 with it, after the adapter fix.

Not a model leaderboard. Eight tasks is a demonstration of the scoring contract. Live local arms are exploratory; n=10 per cell, not n=2.