# Time-to-Solution Dashboard — Spec v0.1

**One bar:** can an agent build a full system in a day? You cannot improve that if you
cannot measure it. This is the quantitative counterpart to the Agentic-Readiness
Scorecard — the scorecard is an opinion; this is the instrument that makes it a number.

The whole design is one idea: **every agent and tool emits a small event as it works;
those events aggregate into a handful of metrics that name the real blocker instead of
guessing at it.**

---

## 1. What we measure (and why each one)

| Metric | Definition | Why it's the one that matters |
|---|---|---|
| **Time-to-solution (TTS)** | Wall-clock from *intent received* to *solution verified*, split by domain/phase | The headline. If you can't see where the day goes, you can't compress it. |
| **Human-touch count (HTC)** | Every human intervention, classified `prompt` / `unblock` / `approve` | The autonomy signal. The goal is *few*, and only `approve` at gates. `prompt` and `unblock` are the work the agentic layer must absorb. |
| **Iterations-to-converge** | Design loops per domain before its deliverable met criteria | If it's 3, the tool is too slow to be *in* the loop. Amesim (headless) did many; the GUIs did ~3. |
| **Rework / defect-escape** | Count + log of events where a downstream step invalidated an upstream artifact | The trust tax. Every escape is a place automated cross-domain verification was missing. |
| **Seam-crossing cost** | Time + failure count at each tool→tool handoff | Usually the biggest hidden tax. Where the "one neutral contract" and identity gaps show up as minutes. |
| **Verification coverage** | Fraction of deliverables with a machine-checkable criterion + full provenance | How much of "done" is *proven* vs *asserted*. Formal SVA on the FPGA = 100%; most everything else = 0. |
| **Cost** | Tokens / compute / license-hours per build | Is a day-scale build economically better than a team-week? |

**Derived:** **Autonomy ratio** = agent-active time / (agent-active + human-touch time).
**Escape rate** = rework events / deliverables. **Seam tax** = Σ seam time / TTS.

---

## 2. The event model (the contract)

Everything is one append-only JSON-lines stream. An agent or a tool wrapper emits one
line per meaningful action. Minimal, tool-agnostic, human-readable.

```json
{
  "ts": "2026-07-11T06:00:00Z",     // ISO-8601 UTC
  "run_id": "qx250-2026-07-11",      // one build
  "event": "work_completed",          // see event types below
  "phase": "analyze",                 // analyze|define|engineer|deliver|comply
  "domain": "performance",            // reliability|performance|rflp|electrical|fpga|software|program|manufacturing|verification|standards
  "tool": "Amesim",                   // MADe|Amesim|Capital|Teamcenter|Questa|NX|bridge|agent
  "actor": "agent:sonnet-1",          // agent:<id> | human:<id>
  "deliverable": "qx250-6dof-suite",  // stable id of the thing produced
  "duration_s": 4200,                 // active seconds for this action (optional)
  "result": "pass",                   // pass|fail|blocked|partial
  "meta": {}                          // event-specific fields (below)
}
```

### Event types + their `meta`
- `work_started` / `work_completed` — a unit of agent work. `work_completed` carries `duration_s`, `result`, and `meta.iterations` (loops taken to converge).
- `human_touch` — a human had to intervene. `meta.kind` = `prompt` (gave direction) | `unblock` (fixed something the agent couldn't) | `approve` (gate sign-off). `duration_s` = how long the human was in the loop.
- `seam_crossing` — a tool→tool handoff. `meta.from`, `meta.to`, `duration_s`, `meta.failures` (retries before it took).
- `rework` — a downstream step invalidated an upstream artifact. `meta.cause`, `meta.upstream` (what had to change).
- `verification` — a criterion was checked. `meta.method` = `formal` | `test` | `analysis` | `manual`; `meta.machine_checkable` (bool); `result` = pass|fail.
- `cost` — periodic. `meta.tokens`, `meta.license_hours`.

That's the entire schema. New tools instrument by emitting these; no central registry, no
per-tool bespoke telemetry.

---

## 3. Where it plugs into *this* pipeline (instrumentation points)

| Stage | Emit on | Who emits |
|---|---|---|
| Intent received | first `work_started` of the run | the orchestrating agent |
| Reliability (MADe) | GUI actions wrapped → `work_*`, `human_touch` on every manual step | computer-use driver |
| Performance (Amesim) | `ame_apy` run start/stop → `work_*` with `iterations`; compile-fix → `rework` | the headless runner |
| RFLP / verification / standards (TC SOA) | each SOA call → `work_*`; naming-rule reject → `seam_crossing` failure | the SOA client |
| RFLP / program (AW web) | each UI action → `work_*`; renderer timeout → `unblock`; hidden-affordance hunt → `human_touch` | the browser driver |
| Electrical (Capital + bridge) | bundle build, generate, import → `work_*`; bundle→tool → `seam_crossing` | the MCP bridge |
| FPGA (Questa) | `vsim` run → `work_*`; SVA proof → `verification{method:formal, machine_checkable:true}` | the sim runner |
| Cross-domain conflict caught | e.g. as-analysed ≠ as-designed → `rework` | the conformance checker (to build) |
| Any human prompt in chat | → `human_touch{kind:prompt}` | the agent, per turn |
| Gate approval | → `human_touch{kind:approve}` | at the WBS milestone |

The key: **the agentic WBS is the natural emitter.** Each work package fires `work_started`
/ `work_completed`; each criterion fires `verification`; each gate fires `human_touch:approve`.
The dashboard is then just a view over the program's own event stream.

---

## 4. v1 (minimal — runnable against the next build)

1. **`events.jsonl`** — the append-only log. Agents/wrappers append lines. (No DB needed.)
2. **`aggregate.py`** — stdlib only; reads `events.jsonl`, writes `metrics.json`.
3. **A static dashboard** — renders `metrics.json` (self-contained HTML, no server).

That's the whole v1. It runs on a laptop, needs no infrastructure, and works the moment a
single wrapper starts emitting events. Start by instrumenting the two extremes — the
headless runners (cheap, high-signal) and the human-touch counter (one line per chat turn) —
and the picture is already actionable.

## 5. Roadmap
- **v2:** real-time (tail the log), per-run comparison (is build N faster than N-1?), the
  seam matrix as a heatmap, cost/economics.
- **v3:** the conformance checker that auto-emits `rework` when domains disagree — closing the
  loop from *measuring* escapes to *preventing* them.
- **v4:** bind targets to the WBS gates (a milestone can't close until HTC-per-deliverable and
  verification-coverage are within bounds) — the dashboard becomes a control, not just a mirror.

---

*The honest note:* the seed data shipped with v1 is a **reconstruction** of the QX-250
weekend build, not instrumented capture — good enough to prove the instrument and to show the
shape of the truth (lots of human touches, cheap iteration only where headless, real escapes).
The point is to capture the **next** build for real.