Repository navigation
[Backend 8] refactor(training): the backend takes frozen modules and blocks as arguments - #1211
Draft
maocheng23 wants to merge 5 commits into
Draft
maocheng23 wants to merge 5 commits into
maocheng23 wants to merge 5 commits into
Conversation
…guments FSDPTrainingBackend.prepare_model probed the model it was handed: getattr(model, "frozen_target_modules", ()), which build_bundle set after construction, plus the model's lm_head / embed_tokens attributes, and getattr(optimizer_target, "_no_split_modules"). - ObjectiveModel.frozen is the program's own declaration: the draft's bound target tables and the teacher head. - ProgramSpec.partition (PartitionHints.blocks) carries the draft class's indivisible blocks. - prepare_model takes frozen= and blocks=; Trainer passes model.frozen and program_spec.partition.blocks. The probing of lm_head / embed_tokens on the wrapped model is gone: an ObjectiveModel has neither attribute. Recorded training trajectories are bit-identical to main (90 cases). Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Review findings on the declarations change: - ObjectiveModel.frozen was computed from the draft's target bindings and the teacher head for every objective model. Models built outside build_bundle never had frozen_target_modules set, so their tables used to be sharded with the rest: excluding them moved FSDP's shard boundaries (a checkpoint's rank-local masters no longer fit on resume) and left a bound fp32 embedding uncast next to cast projections. The frozen modules are now a constructor argument, frozen_modules=, which build_bundle passes; a model built another way shards as it did. - ProgramSpec.partition.blocks is sorted: P-EAGLE's two names sit in a set, whose order differs between the ranks' processes. - Two CUDA checks called prepare_model without the declarations and so no longer exercised the wrapping the trainer uses. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…8-declarations # Conflicts: # specforge/training/contracts.py
Second review round on the declarations change: - program_spec read `_no_split_modules` as an attribute every draft has. A draft that is not a canonical draft class has none, and one may set it to None; the backend used to treat both as no blocks. It does again, and the fixtures that had been given the attribute to get past the lookup no longer carry it. - Tests: what the backend is told is checked where it reaches FSDP's constructor (the assembled runs wrap in DDP and could not show a sharded frozen table); the block order is checked with the names held in reverse and in a set. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Next step of the training-backend design (#1177), stacked on the microbatch PR.
FSDPTrainingBackend.prepare_modeldecided what to keep out of sharding and what to keep whole by probing the object it was handed:getattr(model, "frozen_target_modules", ()), an attributebuild_bundleset on the model after constructing it; the model's ownlm_head/embed_tokensattributes; andgetattr(optimizer_target, "_no_split_modules"). A model built by any other route silently had its frozen target tables sharded. The design's rule is that values cross and a runner never reaches into another part's object.Change
ObjectiveModel.frozenruntime.frozen_target_modules = ...inbuild_bundle, after constructionbuild_objective_model(frozen_modules=...), whichbuild_bundlepasses (the draft's bound target tables and the teacher head, each once)ProgramSpec.partitionPartitionHints(blocks=...): class names of the modules a sharding runner keeps whole, read from the draft's_no_split_modulesbyprogram_spec, sorted; a draft without the attribute, or withNone, names noneFSDPTrainingBackend.prepare_modelfrozen=andblocks=; still refuses a frozen module with a trainable parameterTrainermodel.frozenandprogram_spec.partition.blocksTrainingProgramprotocolfrozenruntime.fsdp_param_dtype, the third attributebuild_bundlesets after construction, is left for the PR that resolves the precision once: it is not a declaration of the program.Behaviour-change ledger
build_bundlebuildsObjectiveModeland carries frozenlm_head/embed_tokensattributes of its ownfrozen=Traineris the only caller ofprepare_modeland always passes a programObjectiveModelbuilt withoutbuild_bundle(the Python builders, tests), wrapped by FSDPprepare_model(wrap=False, frozen=(trainable,))prepare_modelother thanTrainerblocks=andfrozen=Traineris the only one in the packageTrainerfrozen, possibly emptyTrainingProgramEvidence
main.frozen=()every FSDP case fails in its first step (KeyError: 'draft_model.embed_tokens.weight'); withblocks=()4,155 recorded values change.main's.test_program_spec.py): the declarations of all nine architectures against their registrations and draft classes; an assembled run of every architecture declares its bound tables then its teacher head, and the backend leaves exactly those alone; a model built without the declaration has none; P-EAGLE's two block names come out in one order; the backend refuses a trainable module.test_domain_trainer.pypins whatTrainerpasses to the backend.test_frozen_target_tables_are_ignored_by_fsdp, whose only subject was the attribute probe.Not verified
device_id=cpu) for the trajectories; the CUDA-gated launch tests that readbackend.auto_wrap_block_classeswere not run.After Codex review
One read-only Codex round (
maocheng/backend-07-microbatch...HEAD). The first version madefrozena property computed for every objective model. Codex showed two regressions of that for models built through the Python builders, which never hadfrozen_target_modulesset and so always sharded their tables:Fixed in a follow-up commit: the frozen modules are a constructor argument that
build_bundlepasses, and a model built another way shards as it did (ledger row 3). Also fixed: P2 the block names are sorted, because P-EAGLE holds its two in a set whose order differs between processes; P2 two CUDA checks (objective_fsdp_check.py,test_equiv_trainer_split.py) calledprepare_modelwithout the declarations and no longer exercised the trainer's wrapping.Second Codex round
A second read-only round on the fixes above: all five first-round findings confirmed fixed, and one regression found that the first version already had.
program_specread_no_split_modulesas an attribute every draft has. A draft that is not a canonical draft class has none, and one may set it toNone; the backend used to treat both as no blocks, and does again. The fixtures that had been given the attribute to get past the lookup no longer carry it.Not changed: the assembled-run test builds its process group with the end-to-end tests' own helper, which picks the backend from the machine and not from
SPECFORGE_DEVICE; that is that helper's limitation and the same for every test that uses it.🤖 Generated with Claude Code