Apache-2.0 · Python 3.11+ · open source
mini-ork turns a goal into a planned, executed and verified run across a fleet of different models. The verdict on every change is what the code actually did when it ran — tests, type checks, schemas, real execution in an isolated sandbox — not a model's opinion of its own output.
curl -fsSL https://raw.githubusercontent.com/SourceShift/mini-ork/main/install.sh | sh
$ mini-ork certify --base base --head fix-correct \ --issue "median([1, 2, 3, 4]) returns 3, …should return 2.5." PROVEN the fix holds across 100% of 4 perturbed inputs (supermajority) change 1 file · sha256 506d0947 proof .mini-ork/certificates/88f4ff0d-….json cost $0.03 · 129s exit code: 0 $ mini-ork certify --base base --head fix-cheat \ --issue "median([1, 2, 3, 4]) returns 3, …should return 2.5." REFUTED the fix does not generalise — only 0% of 4 invariants hold change 1 file · sha256 aeaffdf3 proof .mini-ork/certificates/c575f369-….json cost $0.03 · 89s exit code: 1
Both patches make the example in the bug report pass. The second only
special-cases [1, 2, 3, 4] — a plain reproduction test passes,
every generated invariant fails.
Wiring agents into a graph is commodity. mini-ork adds the layers that decide whether agent work is actually shippable.
The source of truth for a change is its execution outcome, captured in an isolated runtime. A change is scored on what it did — did the test run, did the assertion pass — not on a reviewer's approval.
You don't pay frontier prices for work a cheaper model can pass. Route each node to a lane by role, across the providers you already have.
Every run leaves a trail of verified outcomes, and the system feeds that signal back — routing, prompts, and memory.
A run is a recipe: a task class, a workflow, and an artifact contract. Every stage leaves evidence behind, and a run can resume from the step or turn where it stopped.
Match the goal to a task class and its recipe.
Decompose into nodes, lanes and gates — with repair on bad JSON.
Dispatch each node to a lane, in an isolated worktree, with a budget.
Run the gates: tests, types, schemas, differential and adequacy checks.
Extract only what the evidence supports into memory.
Score and non-regression-gate proposed improvements before they land.
recipes/code-fix/ task_class.yaml # when this recipe applies workflow.yaml # nodes, lanes, gates, phases artifact_contract.yaml # what a run must produce
Every number below comes with its method, its sample size, and what it does not show. Read RESULTS.md ↗
| Test | Result | What it does not show |
|---|---|---|
| External repositories marshmallow · pyparsing · toolz, each with an upstream fix and a plausible wrong fix | 5 of 6 verdicts correct 1 abstention · 0 wrong | The abstention is a correct fix where generated tests assumed a behaviour the library does not have — the oracle says UNVERIFIED rather than guessing. |
| Adversarial SWE-bench corpus | 21 of 21 PROVEN verdicts correct 0 false completions | An earlier version let one special-cased patch through; the current invariant generator was built to stop that. |
| Held-out tasks from this repository's own history | 22 of 40 solved hidden tests · $0.42 per task | Real API spend with open-weight models; the solver never saw the grading tests. |
| Cost of one certificate | ≈ $0.03 1–3 minutes | List price, not a discounted or negotiated rate. |
119 shipped capabilities across seven pillars. The full, code-anchored list lives in the feature inventory ↗.
Full run lifecycle, task classifier, planner with repair, recovery DAG, multi-epic scheduler.
Provider registry, six routing policies, role-aware fallback chains, throttle guards.
Durable-DAG resume, single-writer leases with fencing, idempotent receipts, circuit breakers.
Extensible gate registry, evidence-cited rejections, promotion gated on measurable evidence.
Anti-Goodhart reward contract, bandit router, GRPO writeback, reflection, closed apply loop.
Live event stream, run detail and DAG overlay in the IDE and board, “why did this fail”, OTel export.
A stable CLI plus worktree-aware, file-surface-leased workflows for concurrent agents.
Architecture, operator docs, the Python SDK, and the schemas.
The oracle is only as strong as what you can actually run.
Correctness is strongest on code with real tests and thinnest on subjective or untestable work. There, mini-ork is built to surface the uncertainty and refuse the promotion — not to manufacture confidence.
On 15 held-out tasks, learned routing spent 44% less than a fixed strong lane but solved 5 instead of 8 — the run's own verdict disagreed with the hidden tests on 40% of runs. Lower cost at equal correctness is not shown yet.
mini-ork contains a loop that can rewrite its own prompts and workflows and promote the change unattended. It ships off by default. If you turn it on, read the warning in the README first.
Clone it, run a dry run, and point mini-ork certify at a change you don't trust.
git clone --recurse-submodules https://github.com/SourceShift/mini-ork.git && cd mini-ork && make install