Skip to content

[Backend 8] refactor(training): the backend takes frozen modules and blocks as arguments - #1211

Draft
maocheng23 wants to merge 5 commits into
maocheng/backend-07-microbatchfrom
maocheng/backend-08-declarations
Draft

maocheng23 wants to merge 5 commits into
maocheng/backend-07-microbatchfrom
maocheng/backend-08-declarations

Conversation

@maocheng23

@maocheng23 maocheng23 commented Nov 18, 2026 •

Copy link
Copy Markdown
Collaborator

Motivation

Next step of the training-backend design (#1177), stacked on the microbatch PR. FSDPTrainingBackend.prepare_model decided what to keep out of sharding and what to keep whole by probing the object it was handed: getattr(model, "frozen_target_modules", ()), an attribute build_bundle set on the model after constructing it; the model's own lm_head / embed_tokens attributes; and getattr(optimizer_target, "_no_split_modules"). A model built by any other route silently had its frozen target tables sharded. The design's rule is that values cross and a runner never reaches into another part's object.

Change

Part Before After
ObjectiveModel.frozen runtime.frozen_target_modules = ... in build_bundle, after construction declared at construction: build_objective_model(frozen_modules=...), which build_bundle passes (the draft's bound target tables and the teacher head, each once)
ProgramSpec.partition — PartitionHints(blocks=...): class names of the modules a sharding runner keeps whole, read from the draft's _no_split_modules by program_spec, sorted; a draft without the attribute, or with None, names none
FSDPTrainingBackend.prepare_model probed the model takes frozen= and blocks=; still refuses a frozen module with a trainable parameter
Trainer — passes model.frozen and program_spec.partition.blocks
TrainingProgram protocol — declares frozen

runtime.fsdp_param_dtype, the third attribute build_bundle sets after construction, is left for the PR that resolves the precision once: it is not a declaration of the program.

Behaviour-change ledger

# What Before After Kind
1 Training numerics, and what FSDP ignores and wraps, for every model build_bundle builds — identical: recorded trajectories, bitwise (see Evidence) unchanged
2 A wrapped model that is not an ObjectiveModel and carries frozen lm_head / embed_tokens attributes of its own those two were kept out of sharding nothing is, unless the caller passes frozen= removed probe; no production caller: Trainer is the only caller of prepare_model and always passes a program
3 An ObjectiveModel built without build_bundle (the Python builders, tests), wrapped by FSDP its bound tables and teacher head were sharded unchanged: it declares no frozen modules unless its builder passes them unchanged
4 prepare_model(wrap=False, frozen=(trainable,)) not checked raises earlier check
5 A caller of prepare_model other than Trainer the backend found the blocks and frozen modules itself it gets one root unit and nothing ignored unless it passes blocks= and frozen= callers have to pass the declarations; Trainer is the only one in the package
6 A program handed to Trainer — has to expose frozen, possibly empty part of TrainingProgram

Evidence

  • Recorded trajectories: 90 cases, 0 differences against main.
  • The oracle sees both declarations: with frozen=() every FSDP case fails in its first step (KeyError: 'draft_model.embed_tokens.weight'); with blocks=() 4,155 recorded values change.
  • Full CPU suite: 1,983 passed; the failure set (15 failed, 1 error, all environmental on this machine) is identical to main's.
  • New tests (test_program_spec.py): the declarations of all nine architectures against their registrations and draft classes; an assembled run of every architecture declares its bound tables then its teacher head, and the backend leaves exactly those alone; a model built without the declaration has none; P-EAGLE's two block names come out in one order; the backend refuses a trainable module. test_domain_trainer.py pins what Trainer passes to the backend.
  • Removed test: test_frozen_target_tables_are_ignored_by_fsdp, whose only subject was the attribute probe.

Not verified

  • CUDA: not run. FSDP ran on CPU through a local shim (device_id=cpu) for the trajectories; the CUDA-gated launch tests that read backend.auto_wrap_block_classes were not run.

After Codex review

One read-only Codex round (maocheng/backend-07-microbatch...HEAD). The first version made frozen a property computed for every objective model. Codex showed two regressions of that for models built through the Python builders, which never had frozen_target_modules set and so always sharded their tables:

  • P1 excluding the tables moved FSDP's shard boundaries, so a two-rank checkpoint written before the change no longer fit its rank-local fp32 masters on resume.
  • P1 an fp32 EAGLE3 with a bound embedding kept the embedding uncast next to projections that FSDP cast, and failed on a dtype mismatch in its first forward.

Fixed in a follow-up commit: the frozen modules are a constructor argument that build_bundle passes, and a model built another way shards as it did (ledger row 3). Also fixed: P2 the block names are sorted, because P-EAGLE holds its two in a set whose order differs between processes; P2 two CUDA checks (objective_fsdp_check.py, test_equiv_trainer_split.py) called prepare_model without the declarations and no longer exercised the trainer's wrapping.

Second Codex round

A second read-only round on the fixes above: all five first-round findings confirmed fixed, and one regression found that the first version already had.

  • P1 program_spec read _no_split_modules as an attribute every draft has. A draft that is not a canonical draft class has none, and one may set it to None; the backend used to treat both as no blocks, and does again. The fixtures that had been given the attribute to get past the lookup no longer carry it.
  • P2 / P3 what the backend is told is now checked where it reaches FSDP's constructor (the assembled runs wrap in DDP and could not show a sharded frozen table); the block order is checked with the names held in reverse and in a set; ledger rows 5 and 6 record what callers have to do now.

Not changed: the assembled-run test builds its process group with the end-to-end tests' own helper, which picks the backend from the machine and not from SPECFORGE_DEVICE; that is that helper's limitation and the same for every test that uses it.

🤖 Generated with Claude Code

maocheng23 and others added 5 commits November 18, 2026 14:27
…guments

FSDPTrainingBackend.prepare_model probed the model it was handed:
getattr(model, "frozen_target_modules", ()), which build_bundle set after
construction, plus the model's lm_head / embed_tokens attributes, and
getattr(optimizer_target, "_no_split_modules").

- ObjectiveModel.frozen is the program's own declaration: the draft's
  bound target tables and the teacher head.
- ProgramSpec.partition (PartitionHints.blocks) carries the draft class's
  indivisible blocks.
- prepare_model takes frozen= and blocks=; Trainer passes model.frozen and
  program_spec.partition.blocks. The probing of lm_head / embed_tokens on
  the wrapped model is gone: an ObjectiveModel has neither attribute.

Recorded training trajectories are bit-identical to main (90 cases).

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Review findings on the declarations change:

- ObjectiveModel.frozen was computed from the draft's target bindings and
  the teacher head for every objective model. Models built outside
  build_bundle never had frozen_target_modules set, so their tables used
  to be sharded with the rest: excluding them moved FSDP's shard
  boundaries (a checkpoint's rank-local masters no longer fit on resume)
  and left a bound fp32 embedding uncast next to cast projections. The
  frozen modules are now a constructor argument, frozen_modules=, which
  build_bundle passes; a model built another way shards as it did.
- ProgramSpec.partition.blocks is sorted: P-EAGLE's two names sit in a
  set, whose order differs between the ranks' processes.
- Two CUDA checks called prepare_model without the declarations and so
  no longer exercised the wrapping the trainer uses.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…8-declarations

# Conflicts:
#	specforge/training/contracts.py
Second review round on the declarations change:

- program_spec read `_no_split_modules` as an attribute every draft has.
  A draft that is not a canonical draft class has none, and one may set
  it to None; the backend used to treat both as no blocks. It does again,
  and the fixtures that had been given the attribute to get past the
  lookup no longer carry it.
- Tests: what the backend is told is checked where it reaches FSDP's
  constructor (the assembled runs wrap in DDP and could not show a
  sharded frozen table); the block order is checked with the names held
  in reverse and in a set.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant