From NVIDIA Megatron-LM for visibility by RaymondLi0 · Pull Request #18 · bigcode-project/Megatron-LM

RaymondLi0 · 2023-01-24T20:01:13Z

No description provided.

…footprint (#2572) Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: Jieming Zhang <jiemingz@nvidia.com> Co-authored-by: Robin Zhang <robinz@nvidia.com>

… memory footprint (#2572)" This reverts commit 8baf014.

Co-authored-by: root <root@gpu-h100-0348.cm.cluster> Co-authored-by: root <root@gpu-h100-0193.cm.cluster> Co-authored-by: root <root@gpu-h100-0082.cm.cluster> Co-authored-by: root <root@gpu-h100-0495.cm.cluster> Co-authored-by: William Dykas <wdykas@cw-pdx-cs-001-vscode-02.cm.cluster> Co-authored-by: root <root@gpu-h100-0213.cm.cluster> Co-authored-by: root <root@gpu-h100-0435.cm.cluster> Co-authored-by: root <root@gpu-h100-0188.cm.cluster> Co-authored-by: root <root@gpu-h100-0032.cm.cluster> Co-authored-by: root <root@gpu-h100-0023.cm.cluster> Co-authored-by: root <root@gpu-h100-0368.cm.cluster> Co-authored-by: root <root@gpu-h100-0203.cm.cluster> Co-authored-by: root <root@gpu-h100-0229.cm.cluster> Co-authored-by: root <root@gpu-h100-0123.cm.cluster> Co-authored-by: root <root@gpu-h100-0217.cm.cluster> Co-authored-by: root <root@gpu-h100-0496.cm.cluster> Co-authored-by: root <root@gpu-h100-0022.cm.cluster> Co-authored-by: root <root@gpu-h100-0176.cm.cluster> Co-authored-by: root <root@gpu-h100-0261.cm.cluster> Co-authored-by: root <root@gpu-h100-0029.cm.cluster> Co-authored-by: root <root@gpu-h100-0215.cm.cluster> Co-authored-by: Jon Barker <jbarker@nvidia.com>

Signed-off-by: oliver könig <okoenig@nvidia.com>

Co-authored-by: Yashaswi Karnati <144376261+yashaswikarnati@users.noreply.github.com>

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

Signed-off-by: oliver könig <okoenig@nvidia.com>

Co-authored-by: Xin Yao <yaox12@outlook.com> Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com> Co-authored-by: Xin Yao <xiny@nvidia.com>

Co-authored-by: Philip Petrakian <ppetrakian@nvidia.com>

Co-authored-by: Jon Barker <jbarker@oci-hsg-cs-001-vscode-01.cm.cluster>

…#3057) Signed-off-by: Dong Hyuk Chang <donghyukc@nvidia.com>

Signed-off-by: Keshav Santhanam <ksanthanam@nvidia.com>

Signed-off-by: Dong Hyuk Chang <donghyukc@nvidia.com>

Co-authored-by: shifangx <shifangx@nvidia.com> Co-authored-by: Mcore Bot <mcore-bot@nvidia.com>

…" (#3069)

Signed-off-by: Deepak Narayanan <dnarayanan@nvidia.com>

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

Signed-off-by: Deepak Narayanan <dnarayanan@nvidia.com>

Co-authored-by: Jon Barker <jbarker@oci-hsg-cs-001-vscode-01.cm.cluster> Co-authored-by: Charlie Truong <chtruong@nvidia.com>

Signed-off-by: Keshav Santhanam <ksanthanam@nvidia.com>

Co-authored-by: oliver könig <okoenig@nvidia.com>

…zation. (#3253) Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

Signed-off-by: Keshav Santhanam <ksanthanam@nvidia.com>

Co-authored-by: litianjian <litianjian@bytedance.com> Co-authored-by: Yan Bai <baiyan1996@icloud.com> Co-authored-by: Siddharth Singh <sidsingh@nvidia.com> Co-authored-by: Robert Kirby <rkirby@cw-dfw-cs-001-vscode-01.cm.cluster> Co-authored-by: Siddharth Singh <136645615+sidsingh-nvidia@users.noreply.github.com> Co-authored-by: oliver könig <okoenig@nvidia.com> Co-authored-by: Teodor-Dumitru Ene <teodord.ene@gmail.com> Co-authored-by: Teodor-Dumitru Ene <34819528+tdene@users.noreply.github.com>

This reverts commit 1123fc0.

Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Philip Petrakian <ppetrakian@nvidia.com>

This reverts commit 9119fae.

Co-authored-by: oliver könig <okoenig@nvidia.com> Co-authored-by: Yashaswi Karnati <144376261+yashaswikarnati@users.noreply.github.com>

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

Signed-off-by: Keval Morabia <28916987+kevalmorabia97@users.noreply.github.com>

…ger (#3362)

Signed-off-by: meg miranda <mmiranda@nvidia.com> Co-authored-by: Philip Petrakian <pgpetrak@gmail.com> Co-authored-by: Philip Petrakian <ppetrakian@nvidia.com>

Signed-off-by: sajadn <snorouzi@nvidia.com>

Co-authored-by: xielaixin <xielx@shanghaitech.edu.cn>

Co-authored-by: litianjian <litianjian@bytedance.com> Co-authored-by: Yan Bai <baiyan1996@icloud.com> Co-authored-by: Siddharth Singh <sidsingh@nvidia.com> Co-authored-by: Robert Kirby <rkirby@cw-dfw-cs-001-vscode-01.cm.cluster> Co-authored-by: Robert Kirby <rkirby@nvidia.com> Co-authored-by: Robert Kirby <ArEsKay3@users.noreply.github.com> Co-authored-by: Siddharth Singh <136645615+sidsingh-nvidia@users.noreply.github.com> Co-authored-by: oliver könig <okoenig@nvidia.com>

Signed-off-by: Keshav Santhanam <ksanthanam@nvidia.com>

Co-authored-by: yobi byte <yobibyte@users.noreply.github.com>

Co-authored-by: liming127 <liming127@meituan.com>

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

Co-authored-by: Jon Barker <jbarker@oci-hsg-cs-001-vscode-01.cm.cluster> Co-authored-by: Teodor-Dumitru Ene <34819528+tdene@users.noreply.github.com>

Signed-off-by: lit <lit@nvidia.com>

RaymondLi0 changed the base branch from multi-query-attention to before-merge June 20, 2023 20:12

RaymondLi0 changed the base branch from before-merge to multi-query-attention June 20, 2023 20:12

Phlip79 and others added 28 commits January 21, 2026 22:47

Update oncall schedule (#3017)

dd72aee

Various CUDA graph improvements on capture time, replay time, memory …

8baf014

…footprint (#2572) Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: Jieming Zhang <jiemingz@nvidia.com> Co-authored-by: Robin Zhang <robinz@nvidia.com>

chore: rotate oncall schedule

bbbedbb

Add script for batch running CI tests across distinct nodes (#3047)

8c768ea

Revert "Various CUDA graph improvements on capture time, replay time,…

c8caeb2

… memory footprint (#2572)" This reverts commit 8baf014.

Catch case of negative tokens to generate (#2985)

91ebe29

ci: Disable broken GRPO tests

27168e2

Signed-off-by: oliver könig <okoenig@nvidia.com>

Sync GitHub and Slack teams (#3037)

f85ad03

Support custom Router implementations in MoELayer (#2891)

7857383

Co-authored-by: Yashaswi Karnati <144376261+yashaswikarnati@users.noreply.github.com>

ci: Remove Github transition comment from CI (#2881)

cfa240a

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

ci: Override N_REPEAT (#3051)

8db8d08

Signed-off-by: oliver könig <okoenig@nvidia.com>

Supporting inference when called within an asyncio loop (#2816)

0136876

Update type hints and doc strings for moe_utils.py (#2821)

03e0915

Co-authored-by: Xin Yao <yaox12@outlook.com> Co-authored-by: Aaron Gokaslan <aaronGokaslan@gmail.com> Co-authored-by: Xin Yao <xiny@nvidia.com>

Remove calculation of padding token in moe routing loss (#2142)

10c6f01

Co-authored-by: Philip Petrakian <ppetrakian@nvidia.com>

Bug fix with --no-use-tokenizer-from-checkpoint-args (#3049)

029f48f

Co-authored-by: Jon Barker <jbarker@oci-hsg-cs-001-vscode-01.cm.cluster>

Revert "Bug fix with --no-use-tokenizer-from-checkpoint-args (#3049)" (…

0683679

…#3057) Signed-off-by: Dong Hyuk Chang <donghyukc@nvidia.com>

Add health endpoint to dynamic text gen server (#3009)

93567e8

Signed-off-by: Keshav Santhanam <ksanthanam@nvidia.com>

Update copy-pr-bot.yaml [skip ci]

3593301

ci: Skip test_precision_aware_optimizer (#3062)

30dea5d

Signed-off-by: Dong Hyuk Chang <donghyukc@nvidia.com>

Support multimodule communication (#2031)

485ed18

Co-authored-by: shifangx <shifangx@nvidia.com> Co-authored-by: Mcore Bot <mcore-bot@nvidia.com>

Revert "Support multimodule communication (#2031)" (#3068)

0972f02

Revert "Remove calculation of padding token in moe routing loss (#2142)…

4cfaa7d

…" (#3069)

Add ability to save wgrads and dgrads (#3032)

dbde759

Signed-off-by: Deepak Narayanan <dnarayanan@nvidia.com>

ci: Mark test_mode_partial_cudagraph unit tests as flaky (#3064)

55fe705

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

Keep FSDP's and DDP's finish_grad_sync API identical (#3070)

3a7d74d

Signed-off-by: Deepak Narayanan <dnarayanan@nvidia.com>

(REPLAY) Bug fix with --no-use-tokenizer-from-checkpoint-args (#3059)

389436b

Co-authored-by: Jon Barker <jbarker@oci-hsg-cs-001-vscode-01.cm.cluster> Co-authored-by: Charlie Truong <chtruong@nvidia.com>

Optimizing post-processing of requests (#2920)

53a2b19

xuwchen and others added 30 commits February 12, 2026 08:58

Add Qwen3-VL support with Megatron-FSDP (#2841)

7df15bd

Refactor Mamba chunked prefill (#3265)

c65fb25

Signed-off-by: Keshav Santhanam <ksanthanam@nvidia.com>

Improved parallel logging of learning rate (#3319)

47938af

Co-authored-by: oliver könig <okoenig@nvidia.com>

Add enhanced event tracking with TTFT measurement and compact seriali…

a51c1c8

…zation. (#3253) Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

Add assertion that max_requests is divisible by tp_size (#3304)

dbc444f

Signed-off-by: Keshav Santhanam <ksanthanam@nvidia.com>

Revert "Move to using the Inference OpenAI API server (#3107)"

4184bfa

This reverts commit 1123fc0.

Update moe github test cases. (#3077)

9119fae

Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Philip Petrakian <ppetrakian@nvidia.com>

Revert "Update moe github test cases. (#3077)"

e0aa16b

This reverts commit 9119fae.

Split layer_specs to return Submodules instead of ModuleSpecs (#3255)

28ccdaa

Co-authored-by: oliver könig <okoenig@nvidia.com> Co-authored-by: Yashaswi Karnati <144376261+yashaswikarnati@users.noreply.github.com>

ci: Remove gpu sanity check (#3420)

76a9f47

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

[Critical-Bug] Fix Uneven PP for Mamba models (Nemotron3-nano) (#3399)

d10eb6f

Signed-off-by: Keval Morabia <28916987+kevalmorabia97@users.noreply.github.com>

Fix for rl (#3390)

2611830

Add check for full_iteration scope before instantiating CudaGraphMana…

4578ed8

…ger (#3362)

Fix broken links throughout (#3230)

698feec

Signed-off-by: meg miranda <mmiranda@nvidia.com> Co-authored-by: Philip Petrakian <pgpetrak@gmail.com> Co-authored-by: Philip Petrakian <ppetrakian@nvidia.com>

Extract intermediate embeddings of transformer block (#3060)

d401490

Signed-off-by: sajadn <snorouzi@nvidia.com>

Update copy-pr-bot.yaml [skip ci]

545bff9

Decouple topk and loss from DSA Indexer (#3248)

b890099

Co-authored-by: xielaixin <xielx@shanghaitech.edu.cn>

Make Mamba inference state memory ratio configurable (#3322)

4beb8ca

Signed-off-by: Keshav Santhanam <ksanthanam@nvidia.com>

Fix configs for RL model environments (#3441)

cbb47c8

Replace pickle with json in rl_utils (#3351)

8f1c2f8

Co-authored-by: yobi byte <yobibyte@users.noreply.github.com>

fix: correct typo in demo training example (#3428)

057c804

Co-authored-by: liming127 <liming127@meituan.com>

Clean up logging inside inference flask server (#3437)

b218e64

ci: Update release-docs workflow to use FW-CI-templates v0.72.0 (#3438)

3c69780

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

Fix --tokenizer-hf-include-special-tokens (#3422)

74ef64e

Co-authored-by: Jon Barker <jbarker@oci-hsg-cs-001-vscode-01.cm.cluster> Co-authored-by: Teodor-Dumitru Ene <34819528+tdene@users.noreply.github.com>

Update num_tokens_to_generate default for Gym (#3453)

267cf1f

Fix slowdown in inference flask server (#3445)

0627623

Add a normalized scale for MTP per token loss (#3159)

a22c40e

Signed-off-by: lit <lit@nvidia.com>

[Bugfix] Fix nan loss caused by zero token in MTP (#3396)

d7500d4

Signed-off-by: lit <lit@nvidia.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

From NVIDIA Megatron-LM for visibility#18

From NVIDIA Megatron-LM for visibility#18
RaymondLi0 wants to merge 6304 commits intobigcode-project:multi-query-attentionfrom
NVIDIA:main

RaymondLi0 commented Jan 24, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

Conversation

RaymondLi0 commented Jan 24, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants