Document 1M-token training and add a context-parallelism example by qgallouedec · Pull Request #6846 · huggingface/trl

qgallouedec · 2026-08-21T00:09:03Z

Training on sequences far longer than the usual few thousand tokens:

docs/source/long_context_training.md, a new how-to guide, moved out of distributing_training.md and extended with the configurations I verified and the levers in the order you hit them
examples/sft_qwen3_8b_1m_context/, a runnable example that trains a book-length sequence per step on one 8xH100 node

Measured

One 8xH100 node, bf16, per_device_train_batch_size=1, loss_type="chunked_nll" (the default). Qwen3-8B at 1,048,576 tokens: 380 s/step, 56.2 GB per GPU. Runs end to end, 12 steps, and saves.

Depends on

transformers >= 5.16 for gradient checkpointing offload (huggingface/transformers#48444, merged). Without it Qwen3-8B at 1M does not fit. The example checks the version at startup.

Corrections since the first version

The step times I first posted came from machines where the system CUDA shadowed the cuDNN bundled with torch, so torch fell back to FlashAttention and every number was inflated. huggingface/transformers#48163 looked like a 1.7x win there and is now closed: PyTorch already prefers cuDNN on Hopper by itself.

Earlier revisions required huggingface/accelerate#4175 instead. Measured back to back on one node, the two are the same (379.75 s vs 379.76 s), so the example uses the transformers one and needs no unreleased accelerate.

bot-ci-comment · 2026-08-21T00:11:35Z

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bae6acb881

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

- One folder per example, named method + task (grpo_wordle, sft_gpt_oss, ...); each folder holds everything the example needs (scripts, notebooks, prompts, chat templates, eval code) - Split the openenv, vlm, and ppo buckets into per-example folders - Drop thin single-trainer example scripts, redundant with trl/scripts (CLI) and the runnable snippets in each trainer's doc page - Drop orphaned examples/cli_configs - Rewrite example_overview.md as a single index of examples; rewire all doc links, Colab badges, and the harbor runtime path strings

@sergiopaniego

Suggested by @sergiopaniego in review.

The reorg removed the Scripts / OpenEnv Scripts / OpenEnv Notebooks sections; jobs_training.md and openenv.md still linked to their anchors.

The reorg renamed folders but kept legacy file names (sft_qwen_vl.ipynb under sft_qwen3_vl/, openenv_sudoku_grpo.ipynb, grpo_trl_lora_qlora.ipynb, *_vlm.py, bare ppo.py/rloo.py/gspo.py). Every folder's main entry is now examples/<name>/<name>.py or .ipynb; variant and helper files keep their suffixed names. All Colab badges, doc links, and docstring run commands updated in the same sweep.

Deleting examples/scripts/gold.py removed text-mode GOLD's only end-to-end recipe (there is no trl gold CLI). It comes back as examples/gold_chatbot_arena/, with its doc section and Index row. The online DPO benchmark blocks still invoke the deleted examples/scripts/online_dpo.py; the prose now says to run them from a v1.10.0 checkout.

- openenv.md: enumerate the 8 ready-to-use OpenEnv examples instead of pointing at the generic Index - jobs_training.md: say which Index entries are uv-submittable, and stop calling trl/scripts/sft.py an example script - add the missing run-command docstrings (grpo_2048, ppo_sentiment, ppo_tldr) and drop rloo_math's pip line that disagreed with its script header - examples/README.md: defer the layout description to the docs page instead of duplicating it - distillation_trainer.md: the section documents the CLI, not an example script - drop the stale examples/notebooks/wandb/ gitignore entry - add a test asserting the example_overview Index stays in sync with the examples/ folders

…ckend

…indow models

… config beside it

…t checkpoint

`accelerate launch` does not read the script's dependency header, so the `transformers>=5.0` it declares is not enforced. On v4 the `rope_parameters` override is dropped silently and the run starts around loss 10.6 instead of 4.4.

Also quote the 1M step time measured on a correctly configured machine.

cursor

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.

^{Reviewed by Cursor Bugbot for commit 78dde06. Configure here.}

…rate one huggingface/transformers#48444 landed the same offload behind `gradient_checkpointing_kwargs={"offload": True}`, so the example no longer needs an unreleased accelerate. Measured back to back on one 8xH100 node: 379.76 s/step with transformers, 379.75 s/step with huggingface/accelerate#4175, same loss.

`gradient_checkpointing` is already `True` in `SFTConfig`. Also correct the `logging_steps` default in the comment (10, not 500) and attribute the two version requirements separately.

The sequence parallelism and million-token material was two thirds of `distributing_training`, which is otherwise about multi-GPU and multi-node launching. It gets its own page next to the other how-to guides, and `distributing_training` points at it.

The opening sentence called them the same thing, which the note right below it contradicts: here CP is ring attention on FSDP2 and SP is ALST/Ulysses on DeepSpeed.

qgallouedec · 2026-09-01T14:39:27Z

most of this is moved content. I want to re-write the whole section in a follow-up pr

chatgpt-codex-connector Bot reviewed Aug 21, 2026

View reviewed changes

qgallouedec force-pushed the reorg-examples branch from 2a20c0d to b4be6ba Compare August 22, 2026 19:02

qgallouedec force-pushed the long-context-example branch from 5905aef to 6004d03 Compare August 22, 2026 19:02

qgallouedec mentioned this pull request Aug 22, 2026

Add expert-parallelism example: SFT of 100B-753B MoE models #6869

Draft

qgallouedec force-pushed the reorg-examples branch from b4be6ba to c15eb3a Compare August 24, 2026 12:29

qgallouedec force-pushed the long-context-example branch from 6004d03 to 97e00f8 Compare August 24, 2026 12:29

qgallouedec force-pushed the reorg-examples branch from c15eb3a to f0f2368 Compare August 24, 2026 14:38

qgallouedec force-pushed the long-context-example branch from 97e00f8 to 34a50e7 Compare August 24, 2026 14:38

qgallouedec force-pushed the reorg-examples branch from 7e70831 to 0042a0f Compare August 24, 2026 16:25

qgallouedec force-pushed the long-context-example branch from 34a50e7 to 581e0b8 Compare August 24, 2026 16:25

qgallouedec added 19 commits August 24, 2026 15:18

Move gold_vlm.py out of the package tree into examples/gold_qwen3_vl

ac10935

Add the contributor guidance for new examples to the README

c9dbd06

Suggested by @sergiopaniego in review.

Point stale example_overview anchors at the Index

dcab11b

The reorg removed the Scripts / OpenEnv Scripts / OpenEnv Notebooks sections; jobs_training.md and openenv.md still linked to their anchors.

Document the 1M-token SFT recipe in the context parallelism guide

d34d270

Document multi-node context length scaling and the cuDNN attention ba…

bbc66b0

…ckend

Document the attention-mask constraints of context parallelism

b0af7cc

Document MLP sequence tiling, the last lever for 30B+ dense models at 1M

a7b25e9

Refresh the constraints section: Hopper-only cuDNN, current sliding-w…

c1f5311

…indow models

Add a 1M-token context parallelism example

16dff22

Drop the cuDNN lever: Transformers selects that backend on Hopper

fb367b3

Rename the example to sft_qwen_3_8_1M_context and keep its accelerate…

7ead702

… config beside it

Let SFTTrainer tokenize the 1M-token example dataset

169a4ca

Document selective activation checkpointing for long-context training

90e4bbe

Add the million-token measurement to the selective checkpointing section

059e076

Quote the context-parallel numbers in the context parallelism section

0206ec1

qgallouedec mentioned this pull request Aug 26, 2026

[Umbrella] Long-context SFT fixes (do not merge) huggingface/accelerate#4184

Draft

qgallouedec and others added 7 commits August 26, 2026 03:20

Rename the example to sft_qwen3_8b_1m_context

b74a283

Keep the 1M guide to the levers the example uses

a1d5575

Document the accelerate branch the 1M example needs, and log every step

d331e43

Quote what the shipped config measures for Qwen3-8B at 1M

d2e4280

Correct the 1M example's measured numbers and stop writing a redundan…

381c45a

…t checkpoint

List the 1M-context example in the example index

05f2450

Merge branch 'main' into long-context-example

a67521f

cursor Bot reviewed Aug 26, 2026

View reviewed changes

Comment thread examples/sft_qwen3_8b_1m_context/sft_qwen3_8b_1m_context.py

qgallouedec added 2 commits August 26, 2026 13:24

Merge branch 'main' into long-context-example

430a417

Merge branch 'main' into long-context-example

c9b84cc

cursor Bot reviewed Aug 27, 2026

View reviewed changes

Comment thread examples/sft_qwen3_8b_1m_context/sft_qwen3_8b_1m_context.py

qgallouedec and others added 5 commits August 27, 2026 22:32

Merge branch 'main' into long-context-example

d1710fc

Merge branch 'main' into long-context-example

1e12b42

Require accelerate 1.15 for the activation checkpointing offload

365a871

Also quote the 1M step time measured on a correctly configured machine.

Quote the re-measured 1M step time in the docs table

78dde06

cursor Bot reviewed Sep 1, 2026

View reviewed changes

Comment thread examples/sft_qwen3_8b_1m_context/sft_qwen3_8b_1m_context.py

qgallouedec added 6 commits September 1, 2026 14:02

Drop the accelerate config keys that already hold their default

cc87ab2

Drop the SFTConfig value that already holds its default

2caef2b

`gradient_checkpointing` is already `True` in `SFTConfig`. Also correct the `logging_steps` default in the comment (10, not 500) and attribute the two version requirements separately.

Drop the RoPE schema from the version requirement

b3b4948

Title the guide by what it gets you

b594334

qgallouedec marked this pull request as draft September 1, 2026 14:21

Stop equating sequence parallelism with context parallelism

8418fa4

The opening sentence called them the same thing, which the note right below it contradicts: here CP is ring attention on FSDP2 and SP is ALST/Ulysses on DeepSpeed.

qgallouedec commented Sep 1, 2026

View reviewed changes

qgallouedec and others added 2 commits September 1, 2026 14:41

Drop the comment above the version check

acbcb56

Merge branch 'main' into long-context-example

ededcfe

qgallouedec changed the title ~~[WIP] Document 1M-token training and add a context-parallelism example~~ Document 1M-token training and add a context-parallelism example Sep 1, 2026

Navigation Menu

Uh oh!

Document 1M-token training and add a context-parallelism example - #6846

Document 1M-token training and add a context-parallelism example#6846
qgallouedec wants to merge 45 commits into
mainfrom
long-context-example

qgallouedec commented Aug 21, 2026 •

edited

Loading

Uh oh!

bot-ci-comment Bot commented Aug 21, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cursor Bot left a comment

Uh oh!

Uh oh!

qgallouedec Sep 1, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

qgallouedec commented Aug 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Measured

Depends on

Corrections since the first version

Uh oh!

bot-ci-comment Bot commented Aug 21, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

qgallouedec Sep 1, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

qgallouedec commented Aug 21, 2026 •

edited

Loading