[minion1227]

The bug

_compute_novelty loads the reigning king's head + SVF into the policy to collect the king's routing decisions, and never puts the submitter's weights back (pr_eval.py:349):

policy.configure(king_theta, spec)
king_decisions = _routing_decisions(policy, eval_items, ref_count=ref_count)
return novelty_score(submitter_decisions, king_decisions)

That policy is the shared object the benchmark loop scores with. It's configured with the submitter's θ exactly once, before the loop (:574), and novelty runs only for benchmarks[0] — at the end of the first iteration (:610).

So with COMPETITION_BENCHMARKS = (math500, mmlu, livecodebench), for every submission evaluated while a king exists:

• hidden_acc, audit_acc, live_acc for mmlu and livecodebench are measured on the king's head + SVF, not the submitter's — that's two thirds of the composite (sum(...) / len(per_benchmark));
• the overfit gate on those two benchmarks compares king-vs-king;
• no miner can displace the king on 2 of 3 boards — they merely reproduce its score, and WIN_MARGIN then requires beating it;
• conversely a weak head silently inherits the king's accuracies on those benchmarks.

Proof

A fake policy recording every configure call, with a king on disk:

BEFORE:  theta after _compute_novelty is the KING's?  True
         → benchmarks 2 and 3 would be scored with: KING's head

AFTER:   normal path    -> ends on SUBMITTER theta: True | saw king mid-flight: True
         exception path -> ends on SUBMITTER theta: True

The fix

Hand the policy back on the submitter's θ. submitter_theta is hoisted in evaluate_pr and passed in explicitly (so a future caller can't silently reintroduce this), and the restore sits in a finally so a failure mid-comparison can't leak the king's weights into the caller either. The king is still loaded mid-flight — novelty measures exactly what it did before.

20 insertions, 6 deletions in one function plus its call site.

Tests

8 regression tests (no torch / encoder / network — the routing plumbing is stubbed):

• the policy is returned on the submitter's θ (the regression);
• the king is still loaded mid-flight, so the fix doesn't defeat the comparison it exists to make;
• θ is restored even when the comparison raises;
• the three-benchmark loop shape: every bench scored with the submitter;
• the three early-return paths (no king / no eval items / unreadable king weights) must not touch θ at all;
• the signature pins submitter_theta as required.

Rigor check: removing only the restore while keeping the new signature fails exactly the 4 behavioural tests and leaves the 4 early-return/signature ones passing — so these guard the behaviour, not just the shape.

ruff on the CI target passes; the 9 pre-existing ruff findings in pr_eval.py are unchanged (identical count before/after, and CI does not lint that path). 93 pr_eval/submission/novelty tests green.

🤖 Generated with Claude Code