Skip to content

[Backend 7] refactor(training): the runner pools the microbatch reduction's denominators - #1210

Draft
maocheng23 wants to merge 5 commits into
maocheng/backend-06-programfrom
maocheng/backend-07-microbatch
Draft

maocheng23 wants to merge 5 commits into
maocheng/backend-06-programfrom
maocheng/backend-07-microbatch

Conversation

@maocheng23

@maocheng23 maocheng23 commented Nov 18, 2026 •

Copy link
Copy Markdown
Collaborator

Motivation

Next step of the training-backend design (#1177), stacked on the program PR. One collective across samples was still issued outside the runner: microbatch_mean in algorithms/objectives/base.py all-reduced each loss term's denominator from inside the model's forward, on the default process group, and multiplied the loss by the default group's world size so that the wrapper's average would cancel. A program that issues its own data-parallel collective cannot run under a runner that reduces differently, and it reads the world size, which the design says a program must not know.

Change

The design's microbatch reduction, executed by the runner.

Part Before After
TrainingRequirements.reduction (objective registry) — "window" (default) or "microbatch"; DSpark declares "microbatch"
ProgramSpec.reduction — read from the registration by program_spec
ObjectiveResult.terms — in a training call under global_microbatch: every term as (coefficient, numerator, denominator), the last two this rank's. The numerator / denominator pair still holds the loss normalized on this rank alone
microbatch_mean all-reduce per term inside the forward, default group deleted; PlanResult.objective_result(normalize=, with_terms=) issues no collective
DataParallelRunner took the already-reduced loss _loss_terms: one reduce_sum per term's denominator, between forward and backward, on the backend's data-parallel group; forms world size * sum_t c_t * numerator_t / pooled_t
Declaration vs result — a training call that returns terms under window, or none under microbatch, raises

Evaluation is unchanged: outside training the program normalizes each term on its own rank and the evaluator pools the pass, as the design's Evaluation is reduced differently describes.

Behaviour-change ledger

# What Before After Kind
1 Training numerics of every objective, DSpark included — identical: recorded trajectories, bitwise (see Evidence) unchanged
2 Collectives of a DSpark micro-batch one all-reduce per term (two) on the default group one all-reduce per term (two) on the backend's data-parallel group same collectives, another group (row 3)
3 World size in the loss, and the group of the reduction the default group's the data-parallel group's same in every run whose tensor- and sequence-parallel sizes are one; above one, the old factor and reduction also counted ranks that share a batch
4 execute_batch(..., training=True) on a global_microbatch batch, called directly returned the globally normalized loss returns the rank-local loss and the terms internal API; only tests called it directly
5 microbatch_mean importable from objectives.base deleted internal API
6 A protocol whose prediction batch contradicts the reduction its registration declares — ValueError in the forward, in training and in evaluation new refusal; no registered protocol does this
7 The count of a window's micro-batches under microbatch accumulated in the first term's denominator dtype accumulated in float32 same for fp32 denominators (DSpark); a bfloat16 first term stopped counting at 256 on main
8 PlanResult.objective_result(global_denominator=...) accepted the keyword is gone (normalize=, with_terms=) internal API

Row 3 changes a configuration I found no recipe for: DSpark's registration does not support sequence parallelism, which the config resolver rejects for it. It is the value the design specifies.

Evidence

  • Recorded trajectories: 90 cases, 0 differences against main. The DSpark cases cover two and four ranks, accumulation, DDP and FSDP, fp32 and bf16.
  • The oracle sees this path: dropping the world-size factor changes 4,826 recorded values, not pooling the denominators changes 5,986.
  • Full CPU suite: 1,979 passed; the failure set (15 failed, 1 error, all environmental on this machine) is identical to main's.
  • New tests (test_step_frame.py): every term's denominator is pooled between forward and backward, each in its own dtype and as the program gave it; the gradient under a faked second rank; a pooled zero denominator divides by one; one rank normalizes by its own denominators; a window averages its micro-batch means; declaration and result have to agree, both ways; evaluation returns what the program computed and reduces nothing. Each fails under a mutation of the runner (no world-size factor, no zero guard, no agreement check).
  • The two-rank gloo test of independent term denominators with accumulation (test_loss_plan.py) now goes through the runner under DDP and still matches the reference gradient.

Not verified

  • CUDA: not run. tests/test_algorithms/objective_fsdp_check.py is a manual two-GPU script; its microbatch_mean half is removed, its FSDP half is unchanged and was not run.

After Codex review

One read-only Codex round (maocheng/backend-06-program...HEAD). Fixed in a follow-up commit:

  • P1, numerics. Stacking the terms' denominators promoted them to one dtype before the reduction. A sum is rounded in the dtype it is taken in (256 + 1 is 256 in bfloat16), so a loss plan with denominators of two dtypes would have changed. Denominators of one dtype now share a reduction. DSpark's plan has one dtype; nothing recorded moved.
  • P2 denominators are placed on the model's device before they are packed; per-term normalization detaches the denominator again, as microbatch_mean did; the declaration is checked against the prediction batch in the forward, which covers evaluation (ledger row 6); objective_result(with_terms=True) without normalize raises.
  • P2 tests/test_algorithms/objective_fsdp_check.py, the manual two-GPU script, has a numerical assertion on the pooled denominators again, through the runner. It was not run. docs/draft_training.md described DSpark's result as already pooled.

Second Codex round

A second read-only round on the fixes above. Its main point changed the approach back:

  • P1, numerics. A packed reduction is not the per-term reductions it replaced, even within one dtype: a backend may sum the ranks of each element of a vector in another order (Gloo's ring, three ranks, a fractional or very large denominator), and moving a host scalar denominator to the device changes a one-rank division in the last bit. The runner now reduces each term's denominator on its own, in its own dtype, shape and place, which is what microbatch_mean did. The design's "one reduction for all terms" is therefore not implemented here: it changes rounding in those cases and is a decision of its own.
  • P1 the window's micro-batch count no longer takes the first term's dtype (ledger row 7).
  • P2 a denominator is detached where the runner consumes it; objective_result allocates its one only where it is returned; the guide says what DSpark's denominators count under D-PACE and when evaluation shards may be uneven; the evaluation test spies on the runner's reduction instead of on a collective that a group of one never issues.

🤖 Generated with Claude Code

maocheng23 and others added 5 commits November 18, 2026 14:27
…inators

microbatch_mean all-reduced each loss term's denominator from inside the
model's forward, on the default process group, and scaled the loss by that
group's world size. It was the last collective across samples issued
outside the step runner.

- The objective registry declares the reduction (TrainingRequirements.
  reduction: "window" or "microbatch"); DSpark declares "microbatch" and
  ProgramSpec carries it.
- Under global_microbatch a training call returns its terms in
  ObjectiveResult.terms; microbatch_mean is deleted and no objective
  issues a collective across samples.
- DataParallelRunner pools every term's denominator in one reduction on
  the data-parallel group, between forward and backward, and forms the
  loss. A training call whose terms contradict the declaration raises.

Evaluation is unchanged: each term is normalized on its own rank.

Recorded training trajectories are bit-identical to main (90 cases).

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…icrobatch

# Conflicts:
#	specforge/training/contracts.py
Review findings on the microbatch reduction:

- Numerics: stacking the terms' denominators promoted them to one dtype
  before the reduction. A sum is rounded in the dtype it is taken in
  (256 + 1 is 256 in bfloat16), so a plan with denominators of two dtypes
  changed. Denominators of one dtype share a reduction; the bundled
  DSpark plan has one dtype and still issues one collective.
- Denominators are placed on the model's device before they are packed.
- Per-term normalization detaches the denominator again, as
  microbatch_mean did, so the rank-local loss and the runner's agree on a
  differentiable denominator.
- A protocol whose prediction batch contradicts the reduction its
  registration declares is refused in the forward, which also covers
  evaluation, where no runner checks.
- objective_result(with_terms=True) without normalize raises instead of
  returning no terms.
- The two-GPU check regains a numerical assertion on the pooled
  denominators, through the runner. docs/draft_training.md described
  DSpark's result as already pooled.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…icrobatch

# Conflicts:
#	specforge/training/contracts.py
Second review round on the microbatch reduction:

- A packed reduction is not the per-term reductions it replaced: it
  rounds in one dtype, a backend may sum the ranks of each element in
  another order (Gloo's ring, three ranks, fractional denominators), and
  moving a host scalar to the device changes a one-rank division. The
  runner now reduces each denominator separately, in its own dtype,
  shape and place, which is what microbatch_mean did; only the group
  differs (the data-parallel one).
- The denominator a program returns is detached where it is consumed.
- The window counts its micro-batches in float32 whatever the first
  term's dtype is; a bfloat16 count stops at 256.
- objective_result allocates its one only in the branches that return it.
- docs/draft_training.md: what DSpark's denominators count under D-PACE,
  and when evaluation shards may be uneven.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant