Skip to content

Fuzz Tests and more - #360

Open
Primata wants to merge 9 commits into
m1from
fuzz-test
Open

Fuzz Tests and more#360
Primata wants to merge 9 commits into
m1from
fuzz-test

Conversation

@Primata

@Primata Primata commented May 18, 2026

Copy link
Copy Markdown

Description

Extends the Move #[test] attribute with property-based fuzz testing, inspired by Foundry's fuzzer (crates/evm/fuzz/). Parameters of a #[test] function are treated as fuzz inputs and exercised with generated values; failing cases are shrunk to a minimal counterexample and (optionally) persisted to a replayable regression corpus.

#[test]                          fun fuzz_u64(_a: u64)      { } // 16 random draws (implicit fuzz)
#[test(_a in 1..=100)]           fun bounded(_a: u64)       { } // bounded fuzz (inclusive range)
#[test(_a in 1..100)]            fun half_open(_a: u64)     { } // half-open range
#[test(_a != 42)]                fun excludes(_a: u64)      { } // domain minus an exclusion
#[test(_a in 1 | 5..=10 | 20)]   fun union(_a: u64)         { } // union of literals/ranges
#[test(_a = [1, 2, 3])]          fun matrix(_a: u64)        { } // explicit matrix (3 deterministic runs)

const FIXTURE_AMOUNT: u64 = 42;
#[test]                          fun with_fix(_amount: u64) { } // FIXTURE_* values bias the draws

Failing fuzz cases are automatically shrunk to a minimal reproducing input and reported as minimal counterexample: [...]. With --fuzz-corpus-dir, failing inputs are persisted and replayed on subsequent runs so a regression never silently disappears.

Fuzz behavior and strategy (read this first)

1. Implicit fuzz on unassigned parameters

A #[test] parameter that is not explicitly assigned (= v, in ..., or != ...) is treated as an implicit fuzz input over an unrestricted domain. This intentionally replaces the legacy compiler's hard "Missing test parameter assignment in test" error.

  • With a FuzzValueSource registered (the unit-test runner installs DefaultFuzzSource), a bare #[test] fun f(a: u64) expands into --fuzz-runs cases (default 16) and runs.
  • With no source registered, the planner reports a clear "no fuzz value source registered" diagnostic instead of the old missing-assignment error.
  • Zero-parameter functions are unaffected: no fuzzing, no error.

Migration note: a pre-existing test that relied on the missing-assignment error to flag an under-specified signature is now fuzzed instead. Assign the parameter explicitly (e.g. #[test(a = 0)]) to pin it. Documented inline in tests/unit_test/test/fuzz_implicit.move.

2. Multi-parameter semantics: fuzz dims are zipped, not Cartesian

This is the key design point and the reason fuzzing does not blow up combinatorially.

  • Fuzz dimensions are zipped. With fuzz_runs = 16 and three fuzz params a, b, c, the expansion produces 16 total cases, where case i uses (a[i], b[i], c[i]). Every parameter is drawn fresh in every run -- they vary together, exactly like QuickCheck/proptest. It is not "vary a while holding b,c fixed."
  • Explicit matrices (a = [v1, v2, ...]) are Cartesian. Deterministic matrix dimensions form a Cartesian product, and the fuzz dimensions are zipped inside each Cartesian point. So #[test(a = [1,2,3])] with fuzz b, c produces 3 x 16 = 48 cases.
  • Total = det_product x fuzz_runs, capped at MAX_FUZZ_CASES = 1024. Exceeding the cap is a compile error that names the offending expansion.
  • fuzz_runs is the min of the fuzz dimensions' produced value counts (normally --fuzz-runs); a constraint that can only yield k < runs distinct values caps the run count at k.

3. Per-type sampling strategy (DefaultFuzzSource)

Each draw is a weighted mix of three strategies, mirroring Foundry's knobs:

  • Edge (EDGE_WEIGHT) -- boundary values.
  • Dictionary (--fuzz-dictionary-weight, default 40) -- values harvested from the program.
  • Random (remainder) -- full-width random draws.
Type Strategy
u8 ... u256 Edges (0, 1, 2, MAX/2-1, MAX/2, MAX/2+1, MAX-1, MAX), dictionary uints, and full-width random (drawn with enough limbs to cover the whole width, not narrowed through u128). Range-aware: draws are sampled within an active range, with range-bracketed boundary values for edge coverage. A finite literal domain (in [a, b, c], no ranges) is drawn from directly so the run count is not capped.
address / signer / &signer Edge anchors (0x0, 0x1, 0x2), dictionary addresses, and random. Range-aware in-range sampling (a full-width random address essentially never lands in a bounded interval, so the sampler draws within the range). Explicit literal domains and fixtures are drained first so the user always sees the values they listed.
bool The {false, true} universe, biased by fixtures. Range constraints are rejected (b in lo..hi is meaningless for bool).
vectors / structs Explicitly out of scope -- error explicitly.

Dictionary and fixtures. FuzzDictionary::from_env mines named address aliases and module-constant values; per-parameter fixture pools are mined from FIXTURE_<name> constants and fed ahead of random/edge values. RNG is an inline SplitMix64 (no proptest dependency), with the base --fuzz-seed mixed with a per-parameter salt so each parameter draws a distinct but reproducible stream.

4. Domain / exclude constraints and validation

  • in ... builds the domain (allowed set: literals, ranges, unions); != accumulates into the exclude set. Combining in/!= on one parameter is allowed (domain narrows, excludes accumulate). Mixing = with !=/in is rejected.
  • Out-of-range constraints are rejected at plan-build time, matching the policy coerce_numeric_to_width already applies to concrete #[test(a = ...)] values. a != 300 on a u8 is a clear compile error (value 300 is out of range for this integer parameter (max 255)), not a silently-wrapped a != 44. The same check covers integer literals, integer range bounds, and (defensively) address range bounds.

5. Shrinking

On a fuzz failure, the runner walks FuzzValueSource::shrink up to 100 steps to find a minimal counterexample. A shrink candidate is accepted only if it reproduces the same failure -- same status code, sub-status, and abort location (MoveError's identity, which ignores the message). Accepting any error would let the shrinker wander onto an unrelated abort and report a counterexample for the wrong bug.

OUT_OF_GAS failures are persisted but not shrunk: shrinking searches for a smaller input that still exhausts gas, but smaller inputs almost always consume less gas, so each probe is a full gas-bounded re-execution that nearly always fails to reproduce -- up to ~100x(#args) wasted executions for no benefit.

6. Corpus persistence and replay

With --fuzz-corpus-dir, failing inputs (preferring the shrunk-minimal vector) are appended to <corpus-dir>/failures/ and replayed on the next run as <test>#regression[i] cases. Replays are bounded per function (MAX_REGRESSION_REPLAYS_PER_FN = 256); when a corpus exceeds the cap, the newest entries are replayed and the overflow is reported rather than silently truncated (compile-time MAX_FUZZ_CASES does not cover post-planning replays).

How to read this PR

Read in phase order; each phase is self-contained and compiles + tests independently.

Phase What Where
1 Grammar + AST extension: !=, in, .., ..=, pipe-union, [...] in attributes legacy-move-compiler/src/parser/, expansion/, move-model/src/ast.rs
2 Plan-builder rewrite: ParamSpec, matrix x fuzz expansion, implicit fuzz on absence, out-of-range validation move-compiler-v2/src/plan_builder.rs, fuzz.rs
3 Default fuzz source: deterministic RNG, dictionary, fixtures, Foundry-aligned type strategies move-compiler-v2/src/fuzz.rs, tools/move-unit-test/src/lib.rs
4 Trait extension + shrinking + corpus persistence/replay move-compiler-v2/src/fuzz.rs, fuzz_corpus.rs, tools/move-unit-test/src/lib.rs and test_runner.rs

File-by-file

Grammar layer (Phase 1)

File Change
legacy-move-compiler/src/parser/lexer.rs Add Tok::PeriodPeriodEqual (..=) with longest-match in the . handler. No valid existing Move source contains the ..= sequence (= can never start an expression/pattern after ..), so tokenization of existing code is unaffected; the token is consumed only in attribute parsing.
legacy-move-compiler/src/parser/ast.rs Add ConstraintOp{Ne,In}, Attribute_::Constrained, AttributeValue_::{List,Range,Union}. Cover new arms in attribute_name() and ast_debug.
legacy-move-compiler/src/parser/syntax.rs Rewrite parse_attribute_value into _value / _range_value / _primary_value with precedence union > range > primary. Branch parse_attribute on != and contextual in.
legacy-move-compiler/src/expansion/ast.rs Mirror parser AST. Add AttributeName_::Disambiguated(sym, slot) so multiple constrained entries on one parameter coexist in the UniqueMap.
legacy-move-compiler/src/expansion/translate.rs Translate new arms; assign disambiguated slots for Constrained entries during unique_attributes.
move-model/src/ast.rs Final-layer mirror: Attribute::Constrained, AttributeValue::{List,Range,Union}, ConstraintOp.
move-model/src/builder/module_builder.rs Refactor translate_attribute into a recursive helper translate_attribute_value handling all variants.

Plan builder (Phases 2 + 4)

move-compiler-v2/src/plan_builder.rs:

  • ParamSpec::{Concrete, Matrix, Fuzz} per parameter; absence implies Fuzz (implicit fuzz).
  • parse_test_attribute folds Assign / Constrained(In) / Constrained(Ne) into specs; rejects mixing = with !=/in.
  • build_test_info returns Vec<ExpandedCase> (case + per-arg ArgOrigin). Cartesian over deterministic dims, zipped across fuzz dims; caps total at MAX_FUZZ_CASES = 1024. Expanded case names embed an ordinal (fn#3[a=...]) so identical-valued cases do not collide in the per-module BTreeMap.
  • construct_test_plan_with_fuzz_source returns TestPlanBuild { plans, fuzz_metadata } so the runner can look up per-arg shrink info for a failing case.
  • format_move_value is now pub and shared with the runner's counterexample renderer (so the expanded-case name and the shrink output format identically).
  • Backwards-compat: construct_test_plan(env, filter) keeps the original Option<Vec<ModuleTestPlan>> signature (uses NoFuzzSource).

Fuzz value source (Phases 3 + 4)

move-compiler-v2/src/fuzz.rs:

  • Trait FuzzValueSource -- sample (required), shrink / mutate (default no-op, purely additive).
  • NoFuzzSource -- error-on-use stub.
  • DefaultFuzzSource -- the strategy described above (edge/dictionary/random per type, range-aware sampling, fixtures, SplitMix64 RNG).
  • Helpers consolidated: reduce_into_range (single source of truth for [0, modulus) coercion), range_edge_endpoint (shared, clamped boundary picker used by both uint and address samplers), bigint_to_address (guards negatives to 0x0, truncates >32-byte values; defensive given upstream validation).
  • Validation: validate_uint_domain / validate_address_domain reject out-of-range constraint values, surfaced as labeled plan-build diagnostics.
  • FuzzPlanMetadata sidecar: BTreeMap<(ModuleId, expanded_test_name), Vec<ArgOrigin>> with an O(log n) keyed lookup.

Corpus (Phase 4)

move-compiler-v2/src/fuzz_corpus.rs (new):

  • Layout <corpus-dir>/{failures,seeds}/<addr>.<module>.<test>.bcs.
  • WireValue proxy enum for serialization (MoveValue has no free-standing Deserialize).
  • load_failures, load_seeds, append_failure, append_seed -- idempotent (deduped by serialized bytes) via a shared append_entry helper.
  • New dep: serde on move-compiler-v2.

Runner integration (Phase 4)

tools/move-unit-test/src/lib.rs:

  • New UnitTestingConfig CLI flags: --fuzz-runs (16), --fuzz-seed (0), --fuzz-dictionary-weight (40), --fuzz-corpus-dir (None).
  • FuzzRunnerCtx { metadata, source, corpus_dir }, attached to TestPlan::runner_metadata via type-erased Arc<dyn Any>.
  • compile_to_test_plan instantiates DefaultFuzzSource, populates the ctx, and replays the regression corpus into each ModuleTestPlan (bounded by MAX_REGRESSION_REPLAYS_PER_FN, overflow reported).

tools/move-unit-test/src/test_runner.rs:

  • SharedTestingConfig.fuzz_ctx extracted from the type-erased runner metadata.
  • shrink_if_fuzz (same-failure acceptance), shrink_persist_and_note, and move_error_of helpers; persist_to_corpus writes the (preferably shrunk) failing args. OUT_OF_GAS persists without shrinking.

legacy-move-compiler/src/unit_test/mod.rs:

  • One field: TestPlan.runner_metadata: Option<Arc<dyn Any + Send + Sync>> -- a downstream-runner extension point; legacy code never introspects it.

Public API additions

All previously-public functions are kept (backwards-compatible). New surface:

  • move_compiler_v2::fuzz: FuzzConfig, FuzzDictionary, FixturePool, DefaultFuzzSource, NoFuzzSource, FuzzValueSource (trait), Domain, RangeSpec, ParamSpec, ArgOrigin, FuzzPlanMetadata.
  • move_compiler_v2::plan_builder: TestPlanBuild, construct_test_plan_with_fuzz_source (introduced here), ExpandedCase, format_move_value.
  • move_compiler_v2::fuzz_corpus: load_failures, load_seeds, append_failure, append_seed, failures_dir, seeds_dir.
  • move_unit_test::FuzzRunnerCtx.

Design rationale (the why)

Decision Why
Grammar extension instead of pseudo-functions like oneof(...) Cleanest long-term, matches Rust pattern-match conventions, no parser churn later.
in as a contextual keyword Attribute identifiers cannot contain spaces, so name in value is unambiguous inside #[...].
Zip, not Cartesian, for fuzz dims Foundry's runs = N means N total trials, not NNN. With Cartesian, 3 fuzz params x 16 runs = 4096 cases (over cap); with zip it is 16. Matrix dims still product.
Reject out-of-range constraints (do not wrap) Consistency: concrete #[test(a = 300)] on a u8 already errors via coerce_numeric_to_width. Wrapping a != 300 into a != 44 would silently exclude a value the user never wrote and hide their mistake.
Shrink accepts only the same failure A smaller input that triggers a different abort is a different bug; reporting it as "the minimal counterexample" would mislead.
Persist OOG, do not shrink it Shrinking toward smaller inputs reduces gas usage, so OOG shrinks almost never reproduce -- pure wasted re-execution. Persisting the original keeps the regression.
FuzzValueSource with default shrink/mutate Implementers only need sample; NoFuzzSource works unchanged.
param_name: &str not Symbol Keeps the trait SymbolPool-free so third-party impls drop in without move-model knowledge.
TestPlan.runner_metadata: Arc<dyn Any> FuzzRunnerCtx references move-compiler-v2 types; TestPlan lives in legacy-move-compiler, which cannot depend on it. Type erasure bridges the layers.
WireValue proxy in corpus MoveValue needs a MoveTypeLayout to Deserialize; a proxy covering exactly the sampled primitive surface keeps the on-disk format stable.

Test coverage

  • move-compiler-v2 testsuite: all fuzz golden fixtures pass (fuzz_* under tests/unit_test/test/).
  • tools/move-unit-test: full suite passes, including end-to-end fuzz execution (tests/fuzz_runner.rs).
  • aptos-framework: builds clean.

Fixtures under move-compiler-v2/tests/unit_test/test/:

  • fuzz_matrix.move -- explicit matrix expansion (Cartesian).
  • fuzz_implicit.move -- bare #[test] on parameters triggers implicit fuzz (documents the behavior change).
  • fuzz_constraints.move -- every grammar form: !=, in, lists, ranges, unions, combined.
  • fuzz_mix_assign_constraint.move -- rejects mixing = with !=/in.
  • fuzz_primitives.move -- end-to-end fuzz on u64, u8, bool, address, multi-param.
  • fuzz_fixtures.move -- FIXTURE_* constants feed fixture pools.
  • fuzz_empty_matrix.move -- a = [] is rejected at plan-build.
  • fuzz_out_of_range.move -- out-of-range integer literal/range constraints are rejected with a clear diagnostic.

tests/fuzz_runner.rs (move-unit-test) runs the plan and the MoveVM end-to-end: implicit-fuzz cases produce uniquely-named runs that all execute without panicking, and numeric matrices coerce + run.

Known gaps / future work

  1. Coverage-guided corpus mutation (full Foundry parity). FuzzValueSource::mutate and the seeds/ subdirectory are wired, but no inspector captures MoveVM coverage maps, so the corpus stays regression-driven. Hook point: wrap a coverage inspector around the session in test_runner.rs::execute_via_move_vm and feed maps to a CorpusFuzzSource over DefaultFuzzSource.
  2. Vector and struct parameter fuzzing. Out of scope; the trait already accepts arbitrary Type, so it is an additive extension in the source impl.
  3. Shrink-loop verification fixture. The same-failure shrink path is exercised manually; a deliberately-failing should_fail fuzz fixture asserting the shrunk output would lock it in (the golden-suite fuzz cases are stubs that always pass, so they cannot drive shrink).
  4. Corpus round-trip across runs. Persist/replay is covered by unit logic but a multi-run integration test is recommended.

Recommended reviewer reading order

  1. legacy-move-compiler/src/parser/syntax.rs -- grammar.
  2. move-compiler-v2/src/plan_builder.rs -- how attributes become test cases (the heart).
  3. move-compiler-v2/src/fuzz.rs -- value sampler, strategy, validation.
  4. move-compiler-v2/src/fuzz_corpus.rs -- disk format.
  5. tools/move-unit-test/src/lib.rs -- CLI + plan-build wiring + replay.
  6. tools/move-unit-test/src/test_runner.rs -- runner-side shrink/persist hooks.
  7. The unit_test fixtures (.move + .exp) for end-to-end behavior.

Quick verification commands

# All fuzz fixtures
cargo test -p move-compiler-v2 --test testsuite -- fuzz_

# End-to-end fuzz execution (plan + MoveVM)
cargo test -p move-unit-test --test fuzz_runner

# Update baselines after intentional changes
UB=1 cargo test -p move-compiler-v2 --test testsuite -- fuzz_

# Downstream framework still builds
cargo check -p aptos-framework

Type of Change

  • New feature
  • Bug fix
  • Breaking change
  • Performance improvement
  • Refactoring
  • Dependency update
  • Documentation update
  • Tests

Which Components or Systems Does This Change Impact?

  • Validator Node
  • Full Node (API, Indexer, etc.)
  • Move/Aptos Virtual Machine
  • Aptos Framework
  • Aptos CLI/SDK
  • Developer Infrastructure
  • Move Compiler
  • Other (specify)

Checklist

  • I have read and followed the CONTRIBUTING doc
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I identified and added all stakeholders and component owners affected by this change as reviewers
  • I tested both happy and unhappy path of the functionality
  • I have made corresponding changes to the documentation

@Primata Primata changed the title pre fixing errors Fuzz Tests and more May 18, 2026
Comment thread third_party/move/move-compiler-v2/src/plan_builder.rs

@musitdev musitdev left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New files should be copyrighted Movement or MoveIndustries.

Comment on lines +4 to +6
// Copyright (c) The Diem Core Contributors
// Copyright (c) The Move Contributors
// SPDX-License-Identifier: Apache-2.0

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's copyrighted Diem, it's a new file or a file moved from old Diem repo?

@@ -0,0 +1,101 @@
// Copyright (c) Aptos Foundation

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The same this file is new or copied from Aptos?

@pegahcarter pegahcarter left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review pass. 10 findings; the first three are the ones I'd treat as blocking, since they mean the headline features don't fully do what the description says.

Blocking

  1. Address fuzzing barely fuzzes - the random branch is unreachable, so it draws from a handful of known addresses.
  2. A constraint with a mismatched literal type is silently dropped, and the parameter is fully fuzzed instead.
  3. --fuzz-corpus-dir is inert on every user-facing path (move test / aptos move test / framework tests).

Worth fixing
4. Bool fixtures can escape the declared domain.
5. Same-position parameters share an identical value stream, so coverage is narrower than it appears.
6. Fuzz runs use a fresh random seed with no CLI way to pin it - intermittent CI failures won't be reproducible.
7. Implicit fuzz silently swallowed the misspelled-parameter-name check.
8. Repeated in clauses widen the domain rather than narrowing it.
9. Clippy failure blocking rust-lints.
10. Stated defaults in the docs/description don't match the constants, and matrices are pairwise, not Cartesian.

Separately: CI can't currently validate any of this. rust-check-merge-base, rust-targeted-unit-tests, and the three container builds all fail with a missing .github/actions/rust-ci-image-setup, which means the branch needs a rebase onto m1.

Things I specifically checked and found sound: the Arc<dyn Any> layering bridge, move_error_of's message-insensitive comparison, the ..= lexer addition (can't appear in existing Move source), pairwise_index_rows' covering property, the fuzz-batch contiguity assumption, and all six new framework Move tests traced for overflow and divide-by-zero.

AccountAddress::from_hex_literal("0x2").unwrap_or(AccountAddress::ONE),
];
*rng.pick(&edges).unwrap()
} else if !dict.addresses.is_empty() {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Address fuzzing almost never produces a random address.

The pick order here is: edge value -> dictionary value -> random. But FuzzDictionary::from_env grabs every named address in the package, so the dictionary is basically never empty. That means the random_address branch never runs, and random_address is effectively dead code.

In practice #[test] fun f(a: address) with 64 runs tries maybe a dozen distinct addresses instead of exploring the address space.

--fuzz-dictionary-weight can't work around it either: sample_addresses never reads config.dictionary_weight (sample_uints does), and the retry cap is hardcoded to n * 64 instead of the configured multiplier.

Suggestion: weight the three sources like sample_uints does, and read the configured cap.

// plan-build time, so `bigint_to_address`'s truncation is never the thing
// that silently narrows a user's constraint.
validate_address_domain(domain, exclude)?;
let dom_addrs: Vec<AccountAddress> = domain

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A constraint with the wrong literal type is silently ignored, so the parameter gets fully fuzzed.

Literals are collected with extract_address, which only matches Value::Address. So:

#[test(_a in [1, 2, 3])] fun f(_a: address)

produces an empty domain, domain_active stays false, and every address is sampled. No error, no warning - the constraint just disappears.

This is easy to hit by accident because address ranges do accept plain integers, and this PR's own fixture uses _a in 1..=10 on a signer. Someone writing the list version of the same thing silently loses their constraint.

sample_bools has the same issue via extract_bool: #[test(_b in [1, 0])] fuzzes over {false, true}.

Suggestion: make a type-mismatched literal a plan-build error, the same way out-of-range values already are.

if let Some(f) = fixtures {
// Fixtures don't expand the bool universe but bias the picker — Foundry
// achieves this via weighting; we just duplicate them in the pool.
pool.extend(f.bools.iter().copied());

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixtures can escape the declared domain (bool only).

The pool is built from the domain, then pool.extend(f.bools) appends fixtures afterwards, filtered only against the exclude set - never against the domain.

So with const FIXTURE_B: bool = false; in the module, #[test(_b in [true])] fun f(_b: bool) will sample false, which the user explicitly ruled out.

sample_uints and sample_addresses both run fixtures through in_domain. Only this one skips it.

let count = if n == 0 { self.config.runs } else { n };
// Mix the configured base seed with the parameter-specific seed so that
// two parameters of the same type don't generate identical streams.
let mut rng = Rng(self.config.seed.wrapping_mul(0x9E3779B97F4A7C15).wrapping_add(seed));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Every parameter in the same position draws the same values.

The RNG salt is just the parameter index, so fun f(a: u64) and fun g(b: u64) receive an identical sequence of 64 values - across the whole build, not just within one function.

The new framework tests hit this: fuzz_request_respects_capacity(_, cap_raw, interval_raw, req_raw) and fuzz_refill_never_exceeds_capacity(_, cap_raw, interval_raw, elapsed_raw) get the same cap_raw and interval_raw values, so they explore less than it looks like they do.

The doc comment says the salt exists "so that two parameters of the same type don't generate identical streams" - that only holds inside a single function.

Also: when seed == 0 (the library default, and --fuzz-seed 0), seed.wrapping_mul(K) is 0, so the stream depends only on the index.

Suggestion: hash the parameter name and the module/function identity into the salt too.

let ctx: Arc<dyn std::any::Any + Send + Sync> = Arc::new(FuzzRunnerCtx {
metadata: build.fuzz_metadata,
source: fuzz_source.clone(),
corpus_dir: None,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

--fuzz-corpus-dir never does anything for real users.

corpus_dir is hardcoded to None here, and this path doesn't include the regression-replay logic added to UnitTestingConfig::compile_to_test_plan.

move test, aptos move test, and the framework unit tests all go through this function - the other entrypoint is only used by move-unit-test's own tests. So corpus persistence and replay are both dead on every user-facing path.

move-cli's Test clap struct also doesn't expose any of the four new fuzz flags, so fuzz_runs / fuzz_seed / fuzz_dictionary_weight read here are always the defaults.

/// drawn per run (logged at the top of each fuzz batch) so every run searches
/// differently; pass an explicit value to pin it and reproduce a prior run.
#[clap(long = "fuzz-seed")]
pub fuzz_seed: Option<u64>,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fuzz runs aren't reproducible from the command line.

fuzz_seed defaults to None and both entrypoints then call unwrap_or_else(random_seed), so every run picks a new seed.

Combined with the move-cli issue (no flag reaches UnitTestingConfig; crates/aptos/src/move_tool/mod.rs:595 and aptos-move/framework/tests/move_unit_test.rs:36 both use ..Default::default()), the six new framework fuzz tests run different inputs on every CI run and there's no supported way to pin the seed.

A test that fails on part of the input space becomes an intermittently red build you can't reproduce locally - only by reading seed=N out of captured CI output.

Suggestion: either default to DEFAULT_FUZZ_SEED for the unit-test suite, or plumb --fuzz-seed through the CLIs.

// value source" diagnostic when one is not. This is a
// deliberate behavior change — see the runner-facing docs in
// `tests/unit_test/test/fuzz_implicit.move`.
owned_default = ParamSpec::Fuzz {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implicit fuzz removed the only check that caught a typo'd parameter name.

This used to report "Missing test parameter assignment in test" when a name in #[test(...)] matched no parameter. Now the unmatched parameter just becomes ParamSpec::Fuzz.

Names in #[test(...)] that match nothing were already ignored, so:

#[test(acount = @0x1)] fun f(account: &signer)   // typo

now compiles and runs 64 times with random signer addresses - usually a confusing abort, or a vacuous pass.

Suggestion: add a diagnostic for #[test(...)] entries that don't match any parameter. That check is now the only thing that can catch this.

return false;
},
};
let target = match op {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Repeated in clauses widen the domain instead of narrowing it.

merge_test_param_entry folds each Constrained(In) into the same Domain, and membership is dom_lits.contains(v) || any_range_contains(v) - a union.

So #[test(a in 1..10, a in 100..200)] samples from 1..10 or 100..200. The comment here and the fixture comment in fuzz_constraints.move both say the domain narrows.

A duplicate = is rejected by insert_or_reject, but a duplicate in produces no diagnostic. Suggestion: either reject it or document the union semantics.

None => return Vec::new(),
};

let parameters: Vec<_> = function.get_parameters_ref().iter().cloned().collect();

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI blocker (clippy). rust-lints fails on this line:

error: called `iter().cloned().collect()` on a slice to create a `Vec`.
       Calling `to_vec()` is both faster and more readable

Fix: let parameters: Vec<_> = function.get_parameters_ref().to_vec();

@pegahcarter pegahcarter Aug 18, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in ee2e0d2759 on pegahcarter:fuzz-test-lint-doc-fixes — swapped to .to_vec().

Verified on Rust 1.86.0: cargo clippy -p move-compiler-v2 --all-targets with CI's exact xclippy flags is clean, exit 0.

//
// What you observe depends on whether a `FuzzValueSource` is registered:
// * move-unit-test runner: installs `DefaultFuzzSource`, so each parameter is
// sampled and the test expands into `--fuzz-runs` cases (default 16). A

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docs don't match the code. This comment says "default 16" but DEFAULT_FUZZ_RUNS is 64.

The PR description has the same drift, plus two more:

  • says MAX_FUZZ_CASES = 1024, code says 2048
  • says "explicit matrices are Cartesian", but pairwise_index_rows implements 2-way covering. Three [1,2,3] matrices give 10 cases, not 27.

The Cartesian -> pairwise change is a real behavior difference and is currently only recorded in a tests/fuzz_runner.rs assertion. Worth fixing the description so reviewers don't assume matrices cover every combination.

@pegahcarter pegahcarter Aug 18, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in ee2e0d2759 on pegahcarter:fuzz-test-lint-doc-fixes16 -> 64 here, and the stale "Cartesian" comment at plan_builder.rs:242 is corrected in the same commit.

Verified: cargo test -p move-compiler-v2 --test testsuite -- fuzz_ gives 8 passed / 0 failed, so no .exp baseline moves.

Two notes: the MAX_FUZZ_CASES = 1024 figure exists only in the PR description, not in code (the constant is 2048), and the other Cartesian mentions in plan_builder.rs are accurate as written. The PR description still needs updating for all three items — that part I can't fix from a branch.

@pegahcarter

pegahcarter commented Aug 18, 2026

Copy link
Copy Markdown

Update: fixes pushed for 2 of the 10 findings

I've pushed the two mechanical fixes to a branch you can cherry-pick or pull from:

Branch: pegahcarter:fuzz-test-lint-doc-fixes · Commit: ee2e0d275

git remote add pegahcarter https://github.com/pegahcarter/aptos-core.git
git fetch pegahcarter fuzz-test-lint-doc-fixes
git cherry-pick ee2e0d275

What's in it

1. The rust-lints blockerplan_builder.rs:240

-let parameters: Vec<_> = function.get_parameters_ref().iter().cloned().collect();
+let parameters: Vec<_> = function.get_parameters_ref().to_vec();

get_parameters_ref() returns &[Parameter], so this is type-identical (both require Parameter: Clone) and is clippy's own suggested replacement.

2. Two stale doc constants

  • plan_builder.rs:242 — the comment said explicit matrices "Cartesian-multiply". pairwise_index_rows covers matrix dims pairwise, so three [1,2,3] matrices give 10 cases, not 27. Corrected, with a pointer to pairwise_index_rows.
  • fuzz_implicit.move:26 — documented --fuzz-runs default was 16; DEFAULT_FUZZ_RUNS is 64.

I checked the rest of the feature for the same drift. The MAX_FUZZ_CASES = 1024 figure appears only in the PR description, not in any file (the constant is 2048), and the other Cartesian mentions in plan_builder.rs (331, 336, 489, 494, 514) are accurate as written — including "Pairwise == Cartesian for 0/1/2 matrix params" — so I left those alone. The PR description itself still needs updating for the run count, the cap, and the Cartesian-to-pairwise change; I can't fix that from a branch.

Verification

Now compile-verified locally on Rust 1.86.0 (the pinned rust-toolchain.toml version):

  • cargo clippy -p move-compiler-v2 --all-targets with CI's exact xclippy flag set (-Dwarnings -Wclippy::all plus the 22 -A allows from .cargo/config.toml) — clean, exit 0. The clippy::iter_cloned_collect error is gone.
  • cargo test -p move-compiler-v2 --test testsuite -- fuzz_8 passed, 0 failed, including fuzz_implicit.move, so the doc edit doesn't shift any .exp baseline.

Note this doesn't validate the rest of the PR — CI still can't run until the branch is rebased onto m1, since rust-check-merge-base, rust-targeted-unit-tests, and the three container builds are all failing on a missing .github/actions/rust-ci-image-setup.

Still open

The other 8 findings are untouched — in particular the three I'd treat as blocking (unreachable random-address sampling, silently-dropped mismatched-type constraints, and --fuzz-corpus-dir being inert on every user-facing path). Those need design decisions from you rather than mechanical edits.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants