feat(analysis): verify SPEC R4 (TRINITY > random routing) offline - #407
Merged
James-CUDA merged 2 commits intoJul 23, 2026
Merged
Conversation
This was referenced Jul 22, 2026
fix(analysis): resolve unresolved merge-conflict markers in __init__ (package failed to import)
#405
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #406
What
docs/SPEC.md§1.3 lists R4 — "TRINITY > random routing" (RLPR: 0.41 vs 0.32).eval.pychecks it live, but there was no offline verifier module — R4 is the last §1.3 invariant without one (R1/R2/R3/R5/R7/R8/R9/R10/R11/R12/R13 are covered).This adds
src/trinity/analysis/random_routing.py(+ tests) — the basic sanity floor: does the trained head beat random model+role selection, per benchmark and on the equally-weighted union.API (mirrors the merged R11
coordinator_vs_llmmodule)analyze_benchmark(trinity, random_routing, *, benchmark, tol)→RandomRoutingComparison(margin, comparable, holds)analyze_benchmarks({benchmark: (trinity, random_routing)})→ per-benchmark rows +r4_holds, equal-weight union accuracies/margin,violationsrender(...)→ markdown reportStrict win (a tie is not a pass); tolerant to missing/non-numeric scores; accepts pair tuples or
{trinity, random_routing}mappings (aliasrandom).Verification
Pure numpy/stdlib — no torch, no network, no GPU. 10 offline tests;
ruff+mypyclean onsrc/.