Repository navigation
[Backend 7] refactor(training): the runner pools the microbatch reduction's denominators - #1210
Draft
maocheng23 wants to merge 5 commits into
Draft
maocheng23 wants to merge 5 commits into
maocheng23 wants to merge 5 commits into
Conversation
…inators microbatch_mean all-reduced each loss term's denominator from inside the model's forward, on the default process group, and scaled the loss by that group's world size. It was the last collective across samples issued outside the step runner. - The objective registry declares the reduction (TrainingRequirements. reduction: "window" or "microbatch"); DSpark declares "microbatch" and ProgramSpec carries it. - Under global_microbatch a training call returns its terms in ObjectiveResult.terms; microbatch_mean is deleted and no objective issues a collective across samples. - DataParallelRunner pools every term's denominator in one reduction on the data-parallel group, between forward and backward, and forms the loss. A training call whose terms contradict the declaration raises. Evaluation is unchanged: each term is normalized on its own rank. Recorded training trajectories are bit-identical to main (90 cases). Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…icrobatch # Conflicts: # specforge/training/contracts.py
Review findings on the microbatch reduction: - Numerics: stacking the terms' denominators promoted them to one dtype before the reduction. A sum is rounded in the dtype it is taken in (256 + 1 is 256 in bfloat16), so a plan with denominators of two dtypes changed. Denominators of one dtype share a reduction; the bundled DSpark plan has one dtype and still issues one collective. - Denominators are placed on the model's device before they are packed. - Per-term normalization detaches the denominator again, as microbatch_mean did, so the rank-local loss and the runner's agree on a differentiable denominator. - A protocol whose prediction batch contradicts the reduction its registration declares is refused in the forward, which also covers evaluation, where no runner checks. - objective_result(with_terms=True) without normalize raises instead of returning no terms. - The two-GPU check regains a numerical assertion on the pooled denominators, through the runner. docs/draft_training.md described DSpark's result as already pooled. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…icrobatch # Conflicts: # specforge/training/contracts.py
Second review round on the microbatch reduction: - A packed reduction is not the per-term reductions it replaced: it rounds in one dtype, a backend may sum the ranks of each element in another order (Gloo's ring, three ranks, fractional denominators), and moving a host scalar to the device changes a one-rank division. The runner now reduces each denominator separately, in its own dtype, shape and place, which is what microbatch_mean did; only the group differs (the data-parallel one). - The denominator a program returns is detached where it is consumed. - The window counts its micro-batches in float32 whatever the first term's dtype is; a bfloat16 count stops at 256. - objective_result allocates its one only in the branches that return it. - docs/draft_training.md: what DSpark's denominators count under D-PACE, and when evaluation shards may be uneven. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Next step of the training-backend design (#1177), stacked on the program PR. One collective across samples was still issued outside the runner:
microbatch_meaninalgorithms/objectives/base.pyall-reduced each loss term's denominator from inside the model's forward, on the default process group, and multiplied the loss by the default group's world size so that the wrapper's average would cancel. A program that issues its own data-parallel collective cannot run under a runner that reduces differently, and it reads the world size, which the design says a program must not know.Change
The design's
microbatchreduction, executed by the runner.TrainingRequirements.reduction(objective registry)"window"(default) or"microbatch"; DSpark declares"microbatch"ProgramSpec.reductionprogram_specObjectiveResult.termsglobal_microbatch: every term as(coefficient, numerator, denominator), the last two this rank's. The numerator / denominator pair still holds the loss normalized on this rank alonemicrobatch_meanPlanResult.objective_result(normalize=, with_terms=)issues no collectiveDataParallelRunner_loss_terms: onereduce_sumper term's denominator, between forward and backward, on the backend's data-parallel group; formsworld size * sum_t c_t * numerator_t / pooled_twindow, or none undermicrobatch, raisesEvaluation is unchanged: outside training the program normalizes each term on its own rank and the evaluator pools the pass, as the design's Evaluation is reduced differently describes.
Behaviour-change ledger
execute_batch(..., training=True)on aglobal_microbatchbatch, called directlymicrobatch_meanobjectives.baseValueErrorin the forward, in training and in evaluationmicrobatchmainPlanResult.objective_result(global_denominator=...)normalize=,with_terms=)Row 3 changes a configuration I found no recipe for: DSpark's registration does not support sequence parallelism, which the config resolver rejects for it. It is the value the design specifies.
Evidence
main. The DSpark cases cover two and four ranks, accumulation, DDP and FSDP, fp32 and bf16.main's.test_step_frame.py): every term's denominator is pooled between forward and backward, each in its own dtype and as the program gave it; the gradient under a faked second rank; a pooled zero denominator divides by one; one rank normalizes by its own denominators; a window averages its micro-batch means; declaration and result have to agree, both ways; evaluation returns what the program computed and reduces nothing. Each fails under a mutation of the runner (no world-size factor, no zero guard, no agreement check).test_loss_plan.py) now goes through the runner under DDP and still matches the reference gradient.Not verified
tests/test_algorithms/objective_fsdp_check.pyis a manual two-GPU script; itsmicrobatch_meanhalf is removed, its FSDP half is unchanged and was not run.After Codex review
One read-only Codex round (
maocheng/backend-06-program...HEAD). Fixed in a follow-up commit:microbatch_meandid; the declaration is checked against the prediction batch in the forward, which covers evaluation (ledger row 6);objective_result(with_terms=True)withoutnormalizeraises.tests/test_algorithms/objective_fsdp_check.py, the manual two-GPU script, has a numerical assertion on the pooled denominators again, through the runner. It was not run.docs/draft_training.mddescribed DSpark's result as already pooled.Second Codex round
A second read-only round on the fixes above. Its main point changed the approach back:
microbatch_meandid. The design's "one reduction for all terms" is therefore not implemented here: it changes rounding in those cases and is a decision of its own.objective_resultallocates its one only where it is returned; the guide says what DSpark's denominators count under D-PACE and when evaluation shards may be uneven; the evaluation test spies on the runner's reduction instead of on a collective that a group of one never issues.🤖 Generated with Claude Code