Methodology

A diagnostic for the operating system around coding agents: task design, control, proof, ownership, coordination, and economics.

The instrument

The audit has 21 questions across seven dimensions, with three questions in each:

  1. Outcomes and task boundariesDefines what done means, what is out of scope, and how much autonomy the task can safely carry.
  2. Context and specificationsGives agents current, scoped sources of truth and explicit rules for uncertainty.
  3. Environments and permissionsContains access, secrets, destructive actions, and blast radius.
  4. Verification and evaluationsTests behavior independently and ties every completion claim to inspectable evidence.
  5. Human review and ownershipKeeps accountable people in control of acceptance, escalation, and recovery.
  6. Orchestration and handoffsUses multiple agents only when justified and coordinates dependencies and shared state.
  7. Cost and observabilityMakes runs, failures, latency, and economics visible enough to improve.

Every question uses evidence-anchored options from 0 to 3. Choose the highest anchor your team can support with current, inspectable practice—not the practice you intend to add.

Dimension scoring

Three answers produce a dimension total from 0 to 9. No hidden weighting is applied inside a dimension.

TotalBandInterpretation
0–2Ad hocDependent on prompting and individual recovery.
3–5EmergingUseful practices exist but are incomplete or inconsistent.
6–7RepeatableThe workflow is consistent and can be reproduced.
8–9ControlledControls are strong, durable, and evidenced.

Overall maturity

Overall maturity is the lower of:

  1. the median band across all seven dimensions; and
  2. the weakest critical band across Environments and permissions, Verification and evaluations, and Human review and ownership.

The critical cap prevents strong documentation or orchestration from masking unsafe access, unverifiable output, or missing human accountability.

Orchestration readiness

The separate orchestration-ready status requires Controlled overall and Controlled Orchestration and handoffs.

Multi-agent use alone never raises maturity. More agents can increase coordination cost and risk when dependency boundaries, handoffs, and shared-state controls are weak.

Risk flags

The engine can surface at most three priority flags:

  • uncontrolled blast radius;
  • unverifiable output;
  • human ownership gaps;
  • shared-state collisions;
  • multi-agent coordination without strong handoffs; and
  • invisible economics.

Critical-control risks receive priority. Ties are resolved deterministically, so identical answers always produce identical results.

Actions and pilot

The engine ranks dimensions by band, total, criticality, and fixed instrument order. Lower totals surface actual deficits before already-perfect dimensions in the same band; criticality resolves an otherwise equal score. It selects the weakest question in each of the top three dimensions and maps it to an evidence-producing action.

The 14-day pilot applies those controls to one bounded, reversible workflow: baseline on days 1–3, implement on days 4–11, then run, exercise failure, review, and retake on days 12–14.

What counts as evidence

Evidence should be current, inspectable, and mapped to the claim: task contracts, policy output, access logs, executable checks, evaluation fixtures, diffs, captures, review records, rollback exercises, run ledgers, and per-task economics.

A model summary, “done” message, configured control, or intended future process is not sufficient by itself.