Skip to content

feat(training): train several drafters from one capture stack - #1216

Draft
aaronlinear wants to merge 5 commits into
sgl-project:mainfrom
aaronlinear:multi-draft
Draft

aaronlinear wants to merge 5 commits into
sgl-project:mainfrom
aaronlinear:multi-draft

Conversation

@aaronlinear

Copy link
Copy Markdown

Motivation

Training two drafters on the same target takes two capture stacks today, so the target forward runs twice for every prompt. With this PR one recipe can train several drafters from one capture stack: the target runs once per prompt and the plugin writes one sample per drafter.

prompt -> target forward (once) -> plugin cuts one sample per drafter
                                     |-> control_dir/dflash/refs.jsonl -> dflash trainer
                                     '-> control_dir/dspark/refs.jsonl -> dspark trainer

Modifications

A recipe gets an optional drafters list. Each entry is a plain single-draft run; model, data, runtime and the capture servers stay shared.

run_id: qwen3
training: {num_epochs: 10, learning_rate: 6.0e-4}
drafters:
  - name: dflash
    draft: specforge/config/draft/presets/qwen3-8b-dflash.yaml
    trainer: {cuda_visible_devices: ["1", "2", "3"]}
  - name: dspark
    draft: specforge/config/draft/presets/qwen3-8b-dspark.yaml
    trainer: {cuda_visible_devices: ["4", "5", "6", "7"]}
Area Change
Config drafters and flatten_drafter, which turns a drafter into a single-draft Config (qwen3-dflash, output_dir/dflash, control_dir/dflash, shared store id). It only merges, derives names or raises, so the trainer, checkpoint and export code keep seeing the old shape.
Plugin A request can carry outputs, one per drafter, each with its own sample id, features and layer_ids. The sink cuts each drafter's layers from the captured ones and writes everything in the same batch put. It advertises spec_capture_outputs in /server_info, and the producer refuses a server without it.
Producer One request per prompt and one ref per drafter, all or nothing, published to each drafter's channel in the same order. Capture pauses on the fullest channel. A drafter whose trainer finished drops out, a failed trainer fails every channel, and cleanup scans every channel.
Launch One trainer per drafter with --drafter <name>, its own GPUs and control_dir/<name>/logs/trainer.log. The capture servers get the sorted union of the drafters' layers, and --plan lists what each drafter resolved to.
Docs A section in disaggregated_training.md, the field reference in examples/configs/README.md, and the recipe managed-local/qwen3-8b-dflash-dspark-1server.yaml.

Without drafters nothing should change. A single-draft run is a list of one in the same code path, sends the same request bytes, and its --plan output matches main for all 80 checked-in recipes in three roles. Single-draft runs also keep working with the plugin that is installed today; only multi-drafter runs need the updated one.

It is five commits, one per row above, and each passes its tests on its own.

Some limits I'm not sure are fine for a first version. EAGLE3 or P-EAGLE can't share a recipe with DFlash or DSpark family drafters yet: model is shared, and model.vocab_mapping_path is required by one side and rejected by the other, so it raises at plan time. I think per-drafter model keys for the draft-side fields would fix it. A drafter that finishes early keeps its unread samples, at most one high watermark, in the store until the run ends. trainer.cuda_visible_devices only works with managed_local; with external servers the CLI warns that the trainers it starts share GPUs. Resume, NVLink and inbox_server_url are rejected with drafters for now.

Related Issues

None.

Accuracy Test

Not run yet, which is why this is a draft. The plan is to train DFlash and DSpark on Qwen3-8B from one capture server and compare each against its solo run (--drafter <name> runs one drafter alone).

On CPU (macOS) there are 27 new tests: a table of config conflicts, the plugin's layer cut, the real adapter against the real plugin sink over a fake Mooncake store, the pause and failure rules of the drive loop, and the launch plan. tests/test_runtime, tests/test_config, tests/test_application and tests/test_algorithms pass, with the SGLang-dependent modules run in an SGLang env. I couldn't run test_sglang_capture_plugin_integration.py because my local SGLang is not the pinned commit, so CI is its first run.

Benchmark & Profiling

Not measured yet. The plugin writes more bytes per request on its single writer thread, so capture speed with SGLANG_SPEC_CAPTURE_TIMING=1 is the first thing I want to look at in the GPU run. Store memory roughly doubles with two drafters at the same watermark.

Checklist

🤖 Generated with Claude Code

aaronlinear and others added 5 commits November 18, 2026 21:48
A recipe can now list several drafters that train from one capture stack. `flatten_drafter` turns each one into a plain single-draft `Config`, so the trainer and checkpoint code keep seeing the old shape. It only merges, derives names from the drafter's name, or raises.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A spec can now carry `outputs`, one per drafter, each with its own sample id, features and `layer_ids`. The sink cuts each output's layers from the captured ones and writes them all in the same batch put, so a prompt is still one target pass. The plugin advertises `spec_capture_outputs` in `/server_info`; a spec without `outputs` works as before.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The producer now takes a list of drafters. Each prompt is one capture request, and the adapter returns one ref per drafter, all or nothing. The drive loop publishes each ref to its drafter's channel, pauses on the fullest channel, and drops a drafter once its trainer has stopped. Any failure fails every channel, and cleanup scans all of them. A single-draft run is a list of one and sends the same request as before.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
`specforge train` on a config with `drafters` now plans one producer and one trainer per drafter. Each trainer gets `--drafter <name>`, its own GPUs and `control_dir/<name>/logs/trainer.log`, and flattens the recipe itself, so it runs as a plain single-draft trainer. The capture servers get the sorted union of the drafters' layers, and `--plan` lists what each drafter resolved to. `--drafter` without a role runs that drafter alone.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The new recipe trains DFlash and DSpark on Qwen3-8B from one capture server. The recipe tests now check each drafter as the single-draft run it flattens to, and the defaults fixture has one record per drafter. Both resolve to the same defaults as their single-draft recipes.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant