Skip to content

feat(coordinator): add the R9 encoder-side ablations (no_svf / last_token) - #426

Merged
James-CUDA merged 1 commit into
James-CUDA:mainfrom
minion1227:minion_encoder_ablations
Jul 23, 2026
Merged

feat(coordinator): add the R9 encoder-side ablations (no_svf / last_token)#426
James-CUDA merged 1 commit into
James-CUDA:mainfrom
minion1227:minion_encoder_ablations

Conversation

@minion1227

Copy link
Copy Markdown
Contributor

The gap

SPEC §1.3 R9"removing SVF / Thinker / tri-role / penultimate-token all hurt" — has a merged verifier and no producer. analysis/ablations.py (#376) takes {full, no_svf, no_thinker, no_trirole, last_token} and reports each ablation's drop, but those four names appear nowhere in src/ outside docstrings and the example JSON at scripts/ablations_report.py:11.

#425 added the two role ablations. This adds the two that concern the feature pipeline, completing the set the verifier asks for.

no_svf — a pure θ transformation

Removing SVF adaptation is running the encoder at its stock singular values, i.e. all scales 1.0. That is exactly the vector SVFAdapter.identity_scales() already hands out, so the ablation is defined by shipped code rather than by a new literal:

head_W, _ = params.unpack(theta, spec)
return params.pack(head_W, np.ones(spec.n_svf))
  • Reuses the canonical params.pack / unpack, so it cannot drift from the θ layout.
  • θ keeps its full width (spec.n_total), so an ablated run stays directly comparable with the full model's and needs no separate ParamSpec.
  • The head block passes through untouched — an ablated run differs in exactly one component.
  • No code path changes at all. Nothing is monkeypatched and no module is edited; this is arithmetic on a parameter vector.

is_svf_ablated() is included as a run-manifest assertion, and it's deliberately exact: a run whose scales merely sit near 1.0 is not a no_svf run, and there's a test pinning that.

last_token — one default-preserving parameter

slm.py hardcoded out.hidden_states[-1][0, -2, :]. It now reads self.token_index, which defaults to -2.

  • The shipped config is byte-for-byte unchanged. from_config reads an optional hidden_state.token_index; absent, it stays at SPEC §3.2's penultimate token. A test asserts from_config() on the repo's own configs/trinity.yaml still yields -2.
  • The default is also declared at class level, so an instance built via object.__new__ — which tests/test_slm_head_input_suffix.py does — keeps reading the canonical position. That existing test file is untouched.
  • make_ablated_encoder rejects an explicit token_index= alongside a variant rather than silently overriding it: two intentions for one knob should be an error.

Why a parameter rather than a subclass

A LastTokenEncoder subclass would avoid editing slm.py, but encode() is a single method — overriding it means duplicating ~40 lines of tokenize / EOS-append / forward / L2-normalize. A copy that drifts from the real extraction would silently corrupt the ablation, which is the one thing this PR exists to measure. A default-preserving parameter keeps exactly one extraction path.

Scope, stated honestly

This ships the producers, not R9's numbers. Filling in the accuracies needs benchmark runs on the GPU box, as does any call into CoordinatorEncoder. What is offline-verifiable — and tested — is the θ transformation, the position registry, the tensor-level read semantics, and the wiring that carries a token index from config into the encoder.

Why it's low-risk

  • no_svf touches no code path. last_token is a defaulted parameter; every existing caller and the shipped config behave identically.
  • Not a fitness-semantics change: this alters which feature is read and which θ is evaluated, not how reward is computed.
  • Torch-free at import. import trinity.coordinator.encoder_ablations pulls no torch, and the validation branches of make_ablated_encoder raise before the lazy encoder import — pinned by a subprocess test, since the in-process test_no_torch_imported guards are order-fragile (test_union_oracle.py uses the robust subprocess form; both checks here follow it, with PYTHONPATH pointed at the checkout under test).

Verification

Full suite, A/B against the same tree without these files:

base with this PR
passed 1,724 1,760 (+36)
failed 11 11 — byte-identical list

The 11 are pre-existing test_report_script_* subprocess failures, untouched by this branch.

  • ruff check src/ scripts/repo_governance/ → clean; mypy on both touched modules → clean
  • tests/test_slm_head_input_suffix.py and tests/test_torch_coordinator_policy.py pass unchanged

Note on CI: main has unresolved merge-conflict markers in src/trinity/analysis/__init__.py (introduced by dc15f97), so the package does not import and this PR is red on the merge commit. This branch touches no file in analysis/.

The fix is not contested: #402, #405, #407, #408 and #403 all rewrite that file to the byte-identical result (every one is index 9916618..6865bc2), so merging any one of them repairs main and the rest become no-ops on that file. #407 / #408 / #403 are already green — they touch that file for their own feature and resolved it in passing. Only PRs that don't touch it are red.

🤖 Generated with Claude Code

…oken)

SPEC R9's verifier is merged (analysis/ablations.py, James-CUDA#376) and consumes
{full, no_svf, no_thinker, no_trirole, last_token}, but nothing in src/
could produce those numbers. James-CUDA#425 added the two role ablations; this adds
the two that concern the feature pipeline.

no_svf is a pure theta transformation: the SVF block is set to the identity
(all scales 1.0), which is exactly what SVFAdapter.identity_scales() already
calls 'no adaptation'. It reuses the canonical params.pack/unpack, keeps
theta at full width so an ablated run stays comparable with the full one,
and changes no code path.

last_token reads the appended EOS instead of SPEC 3.2's penultimate token.
slm.py gains a token_index parameter defaulting to -2, so the shipped config
and every existing caller are byte-for-byte unchanged; the default is also
declared at class level so instances built via object.__new__ (as the
offline tests do) keep reading the canonical position.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@James-CUDA
James-CUDA force-pushed the minion_encoder_ablations branch from fb2cd4d to b136629 Compare July 23, 2026 11:36
@James-CUDA
James-CUDA merged commit 29aed53 into James-CUDA:main Jul 23, 2026
1 of 2 checks passed
@minion1227
minion1227 deleted the minion_encoder_ablations branch July 23, 2026 11:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants