Fetch from nvidia Megatron-LM by RaymondLi0 · Pull Request #5 · ElementAI/Megatron-LM

RaymondLi0 · 2022-08-03T20:32:25Z

No description provided.

Co-authored-by: Xin Yao <xiny@nvidia.com> Co-authored-by: Antoni-Joan Solergibert <asolergibert@nvidia.com>

Co-authored-by: Antoni-Joan Solergibert <asolergibert@nvidia.com>

Signed-off-by: oliver könig <okoenig@nvidia.com>

Signed-off-by: Dong Hyuk Chang <donghyukc@nvidia.com>

Co-authored-by: Siddharth Singh <sidsingh@nvidia.com>

Signed-off-by: qiyuw <qiyuw@nvidia.com> Co-authored-by: qiyuw <qiyuw@nvidia.com> Co-authored-by: oliver könig <okoenig@nvidia.com> Co-authored-by: Kunlun Li <94586211+kunlunl@users.noreply.github.com>

Signed-off-by: meg miranda <mmiranda@nvidia.com>

…unit tests. (#3524) Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: Deepak Narayanan <dnarayanan@nvidia.com>

Signed-off-by: oliver könig <okoenig@nvidia.com>

Signed-off-by: Keshav Santhanam <ksanthanam@nvidia.com>

Signed-off-by: oliver könig <okoenig@nvidia.com>

Signed-off-by: Keshav Santhanam <ksanthanam@nvidia.com>

Signed-off-by: oliver könig <okoenig@nvidia.com>

#4024) Signed-off-by: Cory Ye <cye@nvidia.com> Co-authored-by: Cory Ye <cye@nvidia.com> Co-authored-by: Cory Ye <44509866+cspades@users.noreply.github.com>

Signed-off-by: Cory Ye <cye@nvidia.com> Co-authored-by: Cory Ye <cye@nvidia.com> Co-authored-by: Cory Ye <44509866+cspades@users.noreply.github.com>

#3822) Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Jianbin Chang <shjwudp@gmail.com> Co-authored-by: Cory Ye <44509866+cspades@users.noreply.github.com>

…kward_hook formal (#4029) Co-authored-by: Cory Ye <44509866+cspades@users.noreply.github.com>

…#4054) Co-authored-by: Cory Ye <44509866+cspades@users.noreply.github.com>

Co-authored-by: Cory Ye <44509866+cspades@users.noreply.github.com>

) Signed-off-by: yexiaochuan <yexiaochuan@baidu.com> Co-authored-by: yexiaochuan <yexiaochuan@baidu.com> Co-authored-by: Cory Ye <44509866+cspades@users.noreply.github.com>

… tests (#3287) Co-authored-by: Cory Ye <44509866+cspades@users.noreply.github.com>

Signed-off-by: Keshav Santhanam <ksanthanam@nvidia.com>

Co-authored-by: Cory Ye <44509866+cspades@users.noreply.github.com>

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Signed-off-by: Hao Wu <skyw@nvidia.com>

…4114) Signed-off-by: Keshav Santhanam <ksanthanam@nvidia.com>

Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: dimapihtar <dpykhtar@nvidia.com>

Signed-off-by: Hao Wu <skyw@nvidia.com>

Co-authored-by: Cory Ye <44509866+cspades@users.noreply.github.com>

Signed-off-by: Akshat Kumar <akshat230405@gmail.com>

…#4084) Signed-off-by: Deyu Fu <deyuf@nvidia.com> Co-authored-by: Tom Long <tolong@oci-hsg-cs-001-vscode-02.cm.cluster> Co-authored-by: yaoyu-33 <yaoyu.094@gmail.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Deyu Fu <deyuf@nvidia.com>

…dance (#4035) Signed-off-by: oliver könig <okoenig@nvidia.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>

Signed-off-by: oliver könig <okoenig@nvidia.com>

Signed-off-by: Maanu Grover <maanug@nvidia.com>

Co-authored-by: Antoni-Joan Solergibert <asolergibert@nvidia.com> Co-authored-by: Philip Petrakian <ppetrakian@nvidia.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

…4140) Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

…dlock (#4139) Signed-off-by: oliver könig <okoenig@nvidia.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>

ko3n1g force-pushed the main branch from ad5a627 to cd71d4c Compare February 18, 2026 09:18

github-actions bot and others added 29 commits March 4, 2026 09:14

chore: rotate oncall schedule

470c6ea

Fix illegal memory access with mamba inference (#3631)

2a931a3

Fix illegal memory access with mamba inference (bis) (#3696)

bb31e93

remove duplicate rerun_state_machine.set_mode(rerun_mode) (#3279)

4fa9b5a

Co-authored-by: Xin Yao <xiny@nvidia.com> Co-authored-by: Antoni-Joan Solergibert <asolergibert@nvidia.com>

Correct indexing when cp_comms_type is a list (#3389)

d3a8584

Co-authored-by: Antoni-Joan Solergibert <asolergibert@nvidia.com>

Fix optional chat_completions returnables (#3519)

bfd160b

ci: Claude code review (#3704)

f84e84e

Signed-off-by: oliver könig <okoenig@nvidia.com>

ci: Fix event payload (#3705)

bd1406f

Signed-off-by: oliver könig <okoenig@nvidia.com>

ci: Use issue number (#3706)

33476ff

Signed-off-by: oliver könig <okoenig@nvidia.com>

ci: Finalize Claude review (#3707)

fd21af4

Signed-off-by: oliver könig <okoenig@nvidia.com>

ci: Add codecov yml (#3455)

b1b4df6

Signed-off-by: Dong Hyuk Chang <donghyukc@nvidia.com>

Robust signaling for coordinator inference (#3563)

7ea354b

Co-authored-by: Siddharth Singh <sidsingh@nvidia.com>

Fix memory issue in mxfp8 model init (#3461)

f91c4bb

Signed-off-by: qiyuw <qiyuw@nvidia.com> Co-authored-by: qiyuw <qiyuw@nvidia.com> Co-authored-by: oliver könig <okoenig@nvidia.com> Co-authored-by: Kunlun Li <94586211+kunlunl@users.noreply.github.com>

Update copy-pr-bot.yaml [skip ci]

c5d8f6b

adding public_docs_features: True to get proper legal footer… (#3681)

fa6063d

Signed-off-by: meg miranda <mmiranda@nvidia.com>

add --overlap-param-gather support for layer-wise optimizer. lots of …

a2381d8

…unit tests. (#3524) Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: Deepak Narayanan <dnarayanan@nvidia.com>

ci: Mount and enforce HF_HOME (#3700)

657d33b

Signed-off-by: oliver könig <okoenig@nvidia.com>

Add flags for changing Mamba inference state tensor dtype (#3660)

da47e64

Signed-off-by: Keshav Santhanam <ksanthanam@nvidia.com>

chore: CLI launch internal CI (#3695)

94a903b

Signed-off-by: oliver könig <okoenig@nvidia.com>

Change Review Process (#3659)

485428f

ci: Separate queues for internal/external contributors (#3718)

d1cce0c

Signed-off-by: oliver könig <okoenig@nvidia.com>

Update to correct token (#3724)

6bd5c12

build: Bump to NGC PyTorch 26.02 (#3474)

f5b2ec0

Signed-off-by: oliver könig <okoenig@nvidia.com>

Claude: use Opus 4.6 and auto-review on ready (#3727)

bb84493

Claude to add complexity label (#3709)

41daf81

Offload Flask frontend to separate process (#3648)

0d42bc6

Signed-off-by: Keshav Santhanam <ksanthanam@nvidia.com>

fix(moe): fix TE general_gemm API change (#3582)

d3528a2

Review process fixes (#3728)

43df309

ci: Update golden values after PyT bump (#3733)

bde8264

Signed-off-by: oliver könig <okoenig@nvidia.com>

rapatel and others added 30 commits April 1, 2026 20:24

m-fsdp: wire use_precision_aware_optimizer from ddp_config to ParamAn… (

fe5291f

#4024) Signed-off-by: Cory Ye <cye@nvidia.com> Co-authored-by: Cory Ye <cye@nvidia.com> Co-authored-by: Cory Ye <44509866+cspades@users.noreply.github.com>

Megatron-FSDP: Add MXFP8 transpose helper buffer for Hybrid FSDP (#3918)

606ac26

Signed-off-by: Cory Ye <cye@nvidia.com> Co-authored-by: Cory Ye <cye@nvidia.com> Co-authored-by: Cory Ye <44509866+cspades@users.noreply.github.com>

feat(fsdp): use TE general_gemm for mixed-precision wgrad in FSDP path (

8d7a3f8

#3822) Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Jianbin Chang <shjwudp@gmail.com> Co-authored-by: Cory Ye <44509866+cspades@users.noreply.github.com>

Megatron-FSDP: Make _pre_forward_param_unshard and _register_post_bac…

748ac49

…kward_hook formal (#4029) Co-authored-by: Cory Ye <44509866+cspades@users.noreply.github.com>

Megatron-FSDP: Fix insufficient double buffers during gradient reduce (…

3dc2251

…#4054) Co-authored-by: Cory Ye <44509866+cspades@users.noreply.github.com>

Fix M-FSDP MXFP8 related BUGs (#3991)

41f3b6f

Co-authored-by: Cory Ye <44509866+cspades@users.noreply.github.com>

FIX: Use decoupled gradients for precision-aware M-FSDP grad norm (#3746

a52ceeb

) Signed-off-by: yexiaochuan <yexiaochuan@baidu.com> Co-authored-by: yexiaochuan <yexiaochuan@baidu.com> Co-authored-by: Cory Ye <44509866+cspades@users.noreply.github.com>

[Megatron-FSDP] Fix compatibility with frozen parameters and add unit…

150e37a

… tests (#3287) Co-authored-by: Cory Ye <44509866+cspades@users.noreply.github.com>

Align chat completions endpoint with vLLM (#4063)

cb3bb41

Signed-off-by: Keshav Santhanam <ksanthanam@nvidia.com>

[M-FSDP] Refactor uneven dtensor to full tensor and add UT (#3190)

159e347

Co-authored-by: Cory Ye <44509866+cspades@users.noreply.github.com>

Add agent instruction files (#4102)

5a7f520

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Bump eopt version (#4100)

17a67b9

Signed-off-by: Hao Wu <skyw@nvidia.com>

Update copy-pr-bot.yaml [skip ci]

8b8ceb5

Refactor emerging optimizer integration (#4113)

5b512b4

Signed-off-by: Hao Wu <skyw@nvidia.com>

Fix over provisioning of Mamba state memory when max_requests is set (#…

dcc6d62

…4114) Signed-off-by: Keshav Santhanam <ksanthanam@nvidia.com>

base strategy simplification (#4001)

2697b82

Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: dimapihtar <dpykhtar@nvidia.com>

add support for DCP and FSDP async save (#4027)

69f3b34

Signed-off-by: dimapihtar <dpihtar@gmail.com> Signed-off-by: dimapihtar <dpykhtar@nvidia.com>

Add more emerging optimizers (#3907) (#4119)

c9797ad

Signed-off-by: Hao Wu <skyw@nvidia.com>

Fix FSDP checkpoint conversion and loading for Qwen3.5-VL (#3936)

76e4daa

Co-authored-by: Cory Ye <44509866+cspades@users.noreply.github.com>

docs: update mcore optimizer docstrings to google style (#2799)

a025a69

Signed-off-by: Akshat Kumar <akshat230405@gmail.com>

Update oncall schedule (#4117)

07db9f7

docs: add developer-guide skill with CI/CD and failure navigation gui…

c0e3134

…dance (#4035) Signed-off-by: oliver könig <okoenig@nvidia.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>

chore: Move skills (#4136)

8758d16

Signed-off-by: oliver könig <okoenig@nvidia.com>

ci: Let Claude react to comment (#4135)

fd76254

Signed-off-by: oliver könig <okoenig@nvidia.com>

Nemotron3 Super GB200 release config (#4118)

d865bba

Signed-off-by: Maanu Grover <maanug@nvidia.com>

Enable CUDA graph for ADAM optimizer (#3429)

3d87bfc

Co-authored-by: Antoni-Joan Solergibert <asolergibert@nvidia.com> Co-authored-by: Philip Petrakian <ppetrakian@nvidia.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Claude review should recommend testing (#4137)

499266a

cleanup: remove unused scatter_gather_tensors_in_pipeline argument (#…

10e7b74

…4140) Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

fix: Remove fail-fast (-x) and guard distributed teardown against dea…

0b8306b

…dlock (#4139) Signed-off-by: oliver könig <okoenig@nvidia.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fetch from nvidia Megatron-LM#5

Fetch from nvidia Megatron-LM#5
RaymondLi0 wants to merge 6998 commits intoElementAI:load-iterfrom
NVIDIA:main

RaymondLi0 commented Aug 3, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

Conversation

RaymondLi0 commented Aug 3, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants