Skip to content

[JAX] Fix FSDP when FSDP+EP is active #2649

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
jberchtold-nvidia wants to merge 4 commits into
base: main
Choose a base branch
from
+9 −1
Open

[JAX] Fix FSDP when FSDP+EP is active #2649

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
08fda9c
Fix FSDP when FSDP+EP
jberchtold-nvidia Feb 3, 2026
bbfbc2b
Merge branch 'main' into jberchtold/fix-fsdp-when-ep-is-active
jberchtold-nvidia Feb 4, 2026
fb1a6a9
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Feb 4, 2026
9244cb3
Merge branch 'main' into jberchtold/fix-fsdp-when-ep-is-active
jberchtold-nvidia Feb 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion transformer_engine/jax/cpp_extensions/gemm.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -936,7 +936,15 @@ def _parse_operand_output_specs(

# Non-contracting dims of RHS always needs to be gathered along the FSDP axis
rhs_non_cspecs = tuple(
None if spec is not None and spec == gsr.fsdp_resource else spec
(
None
if spec is not None
and (
spec == gsr.fsdp_resource
or (isinstance(spec, tuple) and gsr.fsdp_resource in spec)
)
else spec
)
for spec in rhs_non_cspecs
)

Expand Down
Loading