feat(coordinator): add the LLM-as-coordinator baseline (SPEC R11, Table 8)

[minion1227]

The gap

SPEC §1.3 invariant R11 — "Trained coordinator > LLM-as-coordinator" — is, in the words of the merged verifier analysis/coordinator_vs_llm.py, "the whole thesis of TRINITY": a tiny (<20K-param) SLM + linear head trained with sep-CMA-ES should route better than simply prompting an LLM to act as the coordinator. If a frozen LLM picking the models and roles matched the trained head, the evolved coordinator would not be earning its keep.

The R11 verifier is merged (#360), and so is scripts/coordinator_vs_llm_report.py. But nothing in src/ could ever produce an LLM-as-coordinator run — so the baseline accuracy it grades had to be typed in from the paper. This PR is the producer.

It's a drop-in, and changes nothing

orchestration/session.py already routes through a Policy protocol:

class Policy(Protocol):
    def decide(self, transcript_text: str, *, sample: bool, rng=None) -> tuple[int, Role]: ...

…precisely so alternative coordinators can be swapped in — its own docstring notes "tests pass a mock". LLMCoordinatorPolicy implements that protocol, so measuring R11 needs no change to the session loop, the trained policy, or the submission path. test_it_satisfies_the_session_policy_protocol_end_to_end drives it through the real run_trajectory rather than asserting the shape by hand.

Both coordinators receive the identical session._transcript_text context, or the comparison would not be fair.

Two integrity properties the comparison depends on

1. A parse failure is not a routing decision. On an unparseable reply the policy falls back deterministically, but it counts every fallback. Crucially, an agent outside the pool — or a 0/out-of-range menu index — is a parse failure, never a guess: coercing an unknown name to index 0 would fabricate a routing decision the model never made, and R11 would then grade that fabrication.

A run that mostly fell back is not an LLM-as-coordinator at all, it's a constant policy wearing the name. DecisionStats.is_representative() gates whether the run may be compared, so R11 can't pass for the wrong reason.

2. The baseline is not free. The trained head costs ~0 output tokens per decision; an LLM coordinator spends a full completion per turn, on top of the answering call. DecisionStats accumulates that spend so an R11/R12 comparison can be budget-matched instead of accuracy-only — an accuracy-only read flatters the baseline by ignoring the tokens it burns to make each choice.

Design notes

• The client is synchronous because Policy.decide is — the trained coordinator is a local forward pass, and run_trajectory calls it without awaiting. So this takes a plain chat callable and a live run supplies a sync bridge, rather than forcing an async signature onto the protocol.
• StubDecider backs the offline tests, mirroring how fugu.conductor.StubConductor backs PromptedConductor — the established pattern in this repo for a prompted baseline with a no-network counterpart.
• Agents are chosen by name, not index: an index invites an off-by-one between the model's 1-based reading of the menu and the pool's 0-based indexing. A 1-based index is still accepted, exactly as the prompt advertises, and tested both ways.
• sample/rng are accepted and ignored — the baseline's stochasticity lives in the model's own decoding, not in a policy-side categorical.

Verification

• pytest tests/ -q → 1798 passed, 0 failed (35 new)
• ruff check src/ scripts/repo_governance/ → clean; mypy on the new module → clean
• import trinity.coordinator.llm_policy leaves torch out of sys.modules

Purely additive: no existing caller changes behaviour, and nothing is wired into the submission path.

Note on CI: main currently has unresolved merge-conflict markers in src/trinity/analysis/__init__.py, so the package does not import and every open PR is red on the merge commit. That is fixed by #402 / #405, not by this branch, which touches no file in analysis/. The run above was taken with those markers resolved locally.

🤖 Generated with Claude Code