feat(analysis): reachability levels L0/L1/L2 for the oracle ceiling (ORACLE §2.2)

[minion1227]

The gap

docs/ORACLE_CEILING_DIAGNOSTIC.md §2.2 names the diagnostic's false-negative failure mode: TRINITY is not "pick one model", it is (model, role) decisions over up to 5 turns. So an oracle computed from single-turn Worker correctness alone is a strict lower bound on the reachable ceiling — a thin gap measured that way can read as "routing is hopeless" when multi-turn collaboration would in fact have helped. §9's defense table lists the fix as "L0/L1/L2 reachability; L0 is a lower bound".

That fix was never built, and the shipped code says so in two places:

• scripts/oracle_ceiling.py:858 exposes --level with choices=["L0"] and the help text "L1/L2 are future".
• scripts/oracle_ceiling.py:564 — the verdict's INCONCLUSIVE branch advises "Widen the reachability level (L1/L2) or collect more samples", advice nothing can currently act on.

Meanwhile §6 defines the deciding quantity as H = routing_headroom measured "at the widest reachability level (L2 if run, else L1)". Since neither can be run, every verdict today rests on L0 while the decision rule believes it is reading something wider. This PR supplies the missing layer.

What's here

analysis/reachability.py computes the oracle at each supplied level and applies the §2.2/§6 rule literally: only the widest measured level can rule routing out.

Level	What it measures
L0	single-turn Worker — cheapest; a strict lower bound
L1	best-role-per-model — each model as a single answerer in any role
L2	short multi-turn probe — sampled, so it carries its own CI

A thin L0 (or L1) now yields LOWER_BOUND_ONLY and names the next level to measure, instead of the POOL_BOUND conclusion the data cannot support.

Two correctness points worth review

1. Only the oracle is monotone — the headroom is not. Each level's option set contains the one below it (L1 lets a model answer in any role, Worker included). A max over a superset can only grow, so routing_oracle and best_single are both non-decreasing in the level — and their difference is therefore under no such constraint. Widening reachability can lift the best single answerer as much as it lifts the ceiling.

So the integrity guard flags a drop in the oracle (mathematically impossible → a collection bug) and never assumes the headroom moves in a particular direction. This matters because §2.2's phrasing ("only rules out routing if L1 and L2 are also thin") invites comparing headrooms across levels; that comparison is not well-founded, so each level's own headroom is read instead. test_headroom_may_fall_as_the_level_widens_without_being_a_violation pins this.

A detected collection bug outranks every routing conclusion (INCONSISTENT) — a bad matrix must not be reported as a finding about routing.

2. A sampled level needs a CI. §2.2 requires L2 be "reported with its own CI": a point estimate from a handful of sequences cannot establish absence of headroom. L2 without a CI yields NEEDS_CI, not a verdict. Thinness is gated on the CI upper bound (§6's <= 0.02), not the point estimate — so a headroom of 0.0 with a CI of [-0.01, 0.30] is correctly not thin.

Why it's low-risk

• Purely additive. No existing caller changes behaviour; the existing single-matrix verdict path in oracle_ceiling.py is untouched. The widest-level rule only engages when several level reports are supplied.
• Reuses the canonical decoder. union_oracle.oracle_from_matrix is called rather than reimplemented, so this cannot drift from the on-disk schema. An L1 matrix is just composite "model/role" keys in the same oracle_matrix schema — no new format to keep in lockstep.
• Pure stdlib + the existing numpy path. No network, no GPU, no torch.

Scope, stated honestly

This is the analysis layer and is fully offline. Actually collecting an L1 or L2 matrix needs live role-loop / trajectory calls, exactly as collecting L0 already does — that is out of scope here and unchanged. This PR makes the levels analyzable and verdict-safe; it does not claim to have measured L1 or L2 on real data.

I also deliberately did not add exports to analysis/__init__.py: that file currently holds the unresolved conflict markers #402/#405 are fixing, and touching it would create exactly the three-way conflict that broke it. Happy to add the export in a follow-up once main is repaired.

Verification

• pytest tests/ -q → 1844 passed, 0 failed (28 new)
• ruff check src/ scripts/repo_governance/ → clean; mypy on the new module → clean

Note on CI: main has unresolved merge-conflict markers in src/trinity/analysis/__init__.py (introduced by dc15f97), so the package does not import and this PR is red on the merge commit. This branch adds a new module and touches no existing file in analysis/, and the 1844-passed run above was taken with those markers resolved locally.

The fix is not contested: #402, #405, #407, #408 and #403 all rewrite that file to the byte-identical result (every one is index 9916618..6865bc2), so merging any one of them repairs main and the rest become no-ops on that file. Note that #407 / #408 / #403 are already green — they touch that file for their own feature and resolved it in passing. Only PRs that don't touch it (this one, #415, #416, #412) are red.

🤖 Generated with Claude Code