Re-run the whole-repo mutation campaign and re-ledger it

[thedavidmeister]

Closes #64

The ledger described one library at a commit 211 behind HEAD. This runs the
campaign over everything the repo now compiles and writes the second entry.

Scope and harness

Whole repo at ed91bfa, 15 units, 264 mutants. Every source file under
src/ and script/ with a body of its own; script/Deploy.sol is an empty
contract and has none.

The suite command is nix develop -c forge test against five local anvil
chains, one per configured network, each with the chain id the real network
uses and the Zoltu factory at its canonical address, and each with one block
mined before that code is set so the factory first has code at block 2 rather
than at genesis. That runs 307 of the repo's 311 tests. The four it cannot
run need Base archive history back to block 1117029 or Base's genesis WETH9
allocation at block 0.

Fuzz seed pinned (FOUNDRY_FUZZ_SEED=0x5eed) so a mutant cannot be killed by
a seed and survive under another.

The lying harness, and what defused it

Any mutation of a file that feeds a deployed contract's creation code moves
that creation code, so testSnapshotMatchesSource fails — on every such
mutant, whatever it is. A filter that includes it reports KILLED for all of
them and measures nothing.

For those four files — both concretes and both interfaces — the suite command
therefore regenerates the rolling snapshots from the MUTATED source first
(forge script ./script/Build.sol --sig "run()"), and restores the generated
files afterwards. The source anchor then has nothing to say, and only a
behavioural test can kill anything.

That it worked is measurable rather than asserted: A01 (the address
registry's root authority) and I01 (the genesis head's preimage) SURVIVED
under that command. Neither could survive if the source anchor were what was
doing the killing.

Proof the suite ran

mutation-probe aborts before the first mutant on a red, silent or zero-test
baseline, and scores KILLED only from the suite's own tally or a non-zero exit
with a tally present. Every probe log opens with baseline: green (307 passed) — (311 passed) for the two run against public RPC endpoints, (312 passed) for the re-probe after the new tests. 241 KILLED verdicts each come
from a run that reported a failing tally out of 307+ executed tests, and every
one of the 15 units produced at least one KILLED, so no unit is a filter that
matched nothing.

Where a run could not be trusted it is reported as such rather than scored:
two mutants time out because the binary search stops narrowing, and are
NO-RUN.

Results

unit	mutants	killed	not killed
src/lib/LibRainDeploySnapshot.sol	70	70	—
src/lib/LibRainDeploy.sol	56	50	4 survived, 2 no-run
src/concrete/MigrationRegistry.sol	34	34	—
src/abstract/RainDeploySuitesBase.sol	15	15	—
script/Build.sol	15	4	11 survived
src/abstract/RainDeployVerifySnapshot.sol	14	13	1 survived
src/concrete/AddressRegistry.sol	12	11	1 survived
src/lib/LibMigrationRegistry.sol	12	12	—
src/abstract/RainDeployBroadcast.sol	7	7	—
src/abstract/RegistryDeploySuites.sol	7	4	3 survived
src/abstract/RainDeployVerifyBase.sol	6	6	—
src/abstract/RainDeployVerifyChain.sol	6	6	—
src/interface/IMigrationRegistryV1.sol	5	4	1 survived
src/lib/LibAddressRegistry.sol	3	3	—
src/interface/IAddressRegistryV1.sol	2	2	—
total	264	241	21 survived, 2 no-run

The seven gaps this closes

Each row survived the first pass, has a test added here, and is re-probed
KILLED at 99b9ca9.

mutant	what walked past the suite	test added
W12	the frozen record matched on the address a release RECORDS rather than the one it DERIVES	testFrozenSnapshotMatchesTheDerivationNotTheRecordedAddress
W13	testSnapshotInternallyConsistent checking only the first declared suite	testSnapshotInternallyConsistentReachesEverySuite
V02	the derivation clearing the derived address's code but not its nonce	testDerivationClearsTheDerivedNonce
R06	run() deriving the pins it is meant to check, making the pre-fork comparison a value against itself	StalePinDeploy leg of testRunSelectsTheSuiteFromTheEnvBeforeTheKeyNeverDefaultsAndBroadcastsIt
R07	run() dropping the suite's dependency list	MissingDependencyDeploy leg of the same test
L32	a read that REVERTS carrying exactly the bytes a successful answer would carry, accepted as an answer	testCheckResolvedAddressesRevertingAddressAnswerReverts
L41	checkResolvedAddressesOnNetworks stopping at the first network	testCheckResolvedAddressesOnNetworksReachesEveryNetwork

The two broadcast legs go inside the existing test rather than beside it, and
answer to the DEPLOYMENT_SUITE it has already set: that variable is
process-wide and forge runs tests concurrently, so a second writer of it is a
second racer, which is what this contract's own docstring records having been.

The twenty-three that are not gaps

Equivalent — no behavioural difference (7)

• L13 deployBlock = high instead of low: low only rises to mid + 1 <= high and high only falls to mid >= low, so they meet exactly.
• G03, G04 a candidate's sourceCreationCode taken from its recorded
  CREATION_CODE rather than type(X).creationCode: identical while the
  committed snapshot is current, which is the tree state every other check
  already enforces.
• A01 the address registry's root authority: CLAUDE.md states that setting
  a real root is an ordinary source change that moves the creation code, the
  address, the code hash and the release together. Nothing should refuse it.
• I01 MIGRATION_HEAD_GENESIS's preimage: the interface constrains it to
  being nonzero, one shared value, and not a migration. All three still hold.
  The zero case (I02) is killed.
• G01, W14 the released-suite concatenation and the record-walk root: this
  repo has frozen no release, so the record is empty, and every root and every
  concatenation of two empty lists is the same one. W14 stays that way after
  the first release too — a walk of the wrong root returns empty and passes —
  which is the residue of The record root is spelled three times, and a walk of the wrong root makes every record-anchored assertion pass with no subject #81 that survives its fix.

Defensive guards nothing else can distinguish (3)

• L19 mstore(0, 0) before the factory call: the EVM masks addresses to 160
  bits everywhere the result is used, so a dirty scratch word changes no
  outcome.
• L25 the !success clause: a failed call leaves deployedAddress zero
  because the assembly's own if success guard never writes it, so the
  zero-address clause catches the same case.
• L55 NoNetworks in deployAndBroadcast: deployToNetworks raises the
  same error for the same input.

Executed by no test (11) — D05–D15, script/Build.sol's run() and
cutRelease() and the regeneration they call. Nothing in the suite runs
either: doing so writes the repo's real generated files, which races the
emitter tests over those paths and would regenerate a stale snapshot out from
under testSnapshotMatchesSource. This is #132's subject and #138 is open for
it; note that #138 makes the BASE's entry points concrete and testable, while
Build.build()'s own wiring — D11–D15 — stays unexecuted.

Non-terminating (2) — L11 low = mid and L12 while (low <= high)
both stop the binary search narrowing. The suite never reports, which is
NO-RUN rather than a verdict; a hang is not something a forge test can be
written to catch.

Adversarial pass

One candidate, confirmed, already on the record: the four LibRainDeploySnapshotTest
emitter tests race in pairs over two committed src/lib/ files and can zero
one. It was hit here as a baseline flake (2 failures in 12 runs of that
contract on ed91bfa) before #135 was found to be filed for it, with #137
open to fix it by giving the writers a directory.

That race is deliberately not fixed here. #137 fixes it properly, and this
branch touches none of the files #137 touches, so the two do not conflict.

The campaign itself was run with the race removed locally, which is the
conservative direction and not a convenient one: a flake can only ever ADD a
failure, so leaving it in would have scored some mutants KILLED on an
interleaving rather than on an assertion. Every KILLED verdict below therefore
comes from a test that actually discriminated, and the 70/70 on
LibRainDeploySnapshot.sol — the unit those four tests belong to — is a real
70, not a flaky one.

No other candidate survived refutation. The reads that looked promising —
tagPrecedes indexing right[i] off left.length, freeze creating the tag
directory before its writes, the already-deployed branch skipping the
dependency check — are each documented preconditions or documented decisions
with the guard that makes them safe in the same function.

Ledger

audit/mutation-test-scans.json gains the second entry. Two keys beyond the
existing shape, because the run produced two outcomes the shape has nowhere to
put: notExecutedBySuite and nonTerminating. filed is empty and
alreadyFiled names #135, because the one confirmed candidate was already on
the record when the campaign reached it.

QA

• Discriminating tests: testFrozenSnapshotMatchesTheDerivationNotTheRecordedAddress, testSnapshotInternallyConsistentReachesEverySuite, testDerivationClearsTheDerivedNonce, testCheckResolvedAddressesRevertingAddressAnswerReverts, testCheckResolvedAddressesOnNetworksReachesEveryNetwork, and the two new legs of testRunSelectsTheSuiteFromTheEnvBeforeTheKeyNeverDefaultsAndBroadcastsIt (StalePinDeploy, MissingDependencyDeploy). Each passes unmutated — 311 tests green on this branch, three consecutive runs — and each fails under exactly the mutation it is for: all seven mutants SURVIVED the pass at ed91bfa before these tests existed, and all seven are re-probed KILLED at 99b9ca9.
• Mutations applied: 264 across 15 units, per-unit table above. The seven this PR closes, as line -> mutation -> killing test: RainDeployVerifySnapshot.sol recorded == LibRainDeploy.zoltuAddress(released[j].creationCode) -> recorded == released[j].storedDeployedAddress -> testFrozenSnapshotMatchesTheDerivationNotTheRecordedAddress; RainDeployVerifySnapshot.sol i < suites.length -> i < suites.length && i < 1 -> testSnapshotInternallyConsistentReachesEverySuite; RainDeployVerifyBase.sol delete vm.resetNonce(formulaAddress); -> testDerivationClearsTheDerivedNonce; RainDeployBroadcast.sol suite.storedDeployedAddress, suite.storedBytecodeHash -> LibRainDeploy.zoltuAddress(suite.creationCode), keccak256(suite.storedRuntimeCode) -> the StalePinDeploy leg; RainDeployBroadcast.sol suite.dependencies -> new address[](0) -> the MissingDependencyDeploy leg; LibRainDeploy.sol if (!success || returnData.length != 0x20) -> if (returnData.length != 0x20) -> testCheckResolvedAddressesRevertingAddressAnswerReverts; LibRainDeploy.sol network loop i < networks.length -> i < networks.length && i < 1 -> testCheckResolvedAddressesOnNetworksReachesEveryNetwork.
• Oracle: the interface and NatSpec contracts, not the implementation. RainDeployVerifySnapshot's docstring states the record is matched "by address: the address a file DECLARES against the address a suite's creation code DERIVES". deriveDeployment's states "the nonce goes too: CREATE2 collides on a non-zero nonce as well as on non-empty code". RainDeployBroadcast's states the recorded pins are passed rather than derived because "a guard that compares a value to itself is not a guard", and that dependencies is what "refuses to broadcast on any network where one of them has no code". ResolvedAddressReadFailed's states a read that "reverts, answers nothing … or answers something that is not one word cannot be compared, and is never a pass". checkResolvedAddressesOnNetworks's states it "runs checkResolvedAddresses on every network". Every expected value is read off those statements; none is copied from the code under test.
• Category check: The mutation ledger predates almost the entire current source tree #64 asks for the campaign re-run at the current commit, at whole-repo scope, and a second entry appended to audit/mutation-test-scans.json in the existing shape, prioritising the units with no ledgered coverage. Covered: whole repo at ed91bfa, 15 units and 264 mutants none of which were ledgered before, and the ledger entry. The issue's prioritisation list is stale in the one respect its own verification section flags: script/Build.sol now has test/script/Build.t.sol, and RainDeployBroadcast.run and LibRainDeploySnapshot.freeze both have tests.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 0f97e3ca-83d6-42d4-beac-6a969d1048e6

📥 Commits

Reviewing files that changed from the base of the PR and between 74260ae and 1d25e2e.

📒 Files selected for processing (8)

• audit/mutation-test-scans.json
• test/concrete/MissingDependencyDeploy.sol
• test/concrete/MockChainDependentOwner.sol
• test/concrete/MockRevertingAnswerOwner.sol
• test/concrete/StalePinDeploy.sol
• test/src/abstract/RainDeployBroadcast.t.sol
• test/src/abstract/RainDeployVerifySnapshot.t.sol
• test/src/lib/LibRainDeploy.t.sol

Included review availability: Your plan includes up to 1 review per rolling hour; 0 remain after this review.

---

Walkthrough

Added deployment fixtures and regression tests for stale address pins, missing dependencies, snapshot derivation, nonce handling, reverted reads, and multi-network address checks. Appended a mutation-test scan record for the release commit.

Changes

Deployment validation

Layer / File(s)	Summary
Broadcast validation fixtures and tests test/concrete/MissingDependencyDeploy.sol, test/concrete/StalePinDeploy.sol, test/src/abstract/RainDeployBroadcast.t.sol	Added deployment fixtures and tests that reject stale recorded addresses and missing dependencies.
Snapshot consistency tests test/src/abstract/RainDeployVerifySnapshot.t.sol	Added tests for derived-address matching, validation across all declared suites, and clearing a derived address nonce.
Resolved-address validation tests test/concrete/MockChainDependentOwner.sol, test/concrete/MockRevertingAnswerOwner.sol, test/src/lib/LibRainDeploy.t.sol	Added mock owners and tests for reverted address reads and checks across multiple networks.
Mutation scan ledger update audit/mutation-test-scans.json	Appended mutation-test results for commit ed91bfa and tag sol-v0.1.5.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk: 🔵 Low · up to 1d25e

The PR adds mutation-test coverage and updates the audit ledger without changing production behavior. The ledger metadata should be aligned with the scan provenance, or the scan should be documented as independent; this is a bounded follow-up rather than a production merge blocker.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	The PR reruns the whole-repository campaign at commit ed91bfa, adds coverage tests, and appends results to the mutation ledger as required by [#64].
Out of Scope Changes check	✅ Passed	The added ledger record, test fixtures, and regression tests directly support the mutation campaign and its identified coverage gaps.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the primary changes: rerunning the whole-repository mutation campaign and recording the results in the ledger.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch 2026-08-18-issue-64-mutation-campaign

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}