Transfer support for Cosmos3 ModularPipeline

[yzhautouskay]

What does this PR do?

Adds the transfer (structure-controlled) path to the Cosmos3 Modular pipeline (Cosmos3OmniModularPipeline / Cosmos3OmniBlocks), letting users condition video generation on one or more control videos (segmentation, depth, edge/canny, blur, world-scenario map) in addition to the text prompt. This mirrors the transfer path in the task-based Cosmos3OmniPipeline and is element-wise consistent with it

What this adds

• A transfer workflow for the modular pipeline, triggered automatically when control_videos is passed -- no new pipeline class, it coexists with the existing modular modes (text2image, text2video, image2video, video2video, action, sound).
• Transfer is implemented as first-class modular blocks (matching how the existing modalities are built), not a monolithic step:
  • text: Cosmos3TransferTextStep
  • before-denoise: Cosmos3TransferSetupStep, Cosmos3TransferPrepareLatentsStep, Cosmos3TransferPackSequenceStep, Cosmos3TransferSetTimestepsStep
  • denoise: per-chunk loop blocks (Cosmos3TransferChunkVaeEncoderStep -> prepare -> denoise -> scheduler), with the denoiser consuming the packed static contexts through the shared denoiser_input_fields mechanism
  • decode: Cosmos3TransferDecodeChunkStep, Cosmos3TransferStitchStep
  • composed/wired in modular_blocks_cosmos3.py via the Auto* blocks (selected on control_videos), with "transfer" added to available_workflows.

Before submitting

• Did you use an AI agent (Claude Code, Codex, Cursor, etc.) to help with this PR? If so:
  • Did you read the Coding with AI agents guide?
  • Did you self-review the diff against .ai/review-rules.md?
• Did you read the contributor guideline?
• Did you read our philosophy doc? (important for complex PRs)
• Was this discussed/approved via a GitHub issue or the forum? Please add a link to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?
• Are you the author (or part of the team) of the model/pipeline (only applicable for model/pipeline related PRs)?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

[yiyixuxu]

thanks a lot, i left some feedbacks

[yiyixuxu]

Can you consider using the same pattern as Helio for its chunk loop? https://github.com/huggingface/diffusers/blob/main/src/diffusers/modular_pipelines/helios/denoise.py

Basically, we make the outer chunk loop a proper LoopSequentialPipelineBlocks (see https://github.com/huggingface/diffusers/blob/main/src/diffusers/modular_pipelines/helios/denoise.py#L706) and write the inner denoiser loop as a plain block (see https://github.com/huggingface/diffusers/blob/main/src/diffusers/modular_pipelines/helios/denoise.py#L355)

[yzhautouskay]

Yeah, I have looked at Helios pipeline, and the constraint was that LoopSequentialPipelineBlocks rejects non-leaf sub-blocks, and our inner timestep loop (Cosmos3TransferDenoiseStep -> Cosmos3DenoiseLoopWrapper) is itself a LoopSequentialPipelineBlocks, so the two loops can't both be loop-wrappers

And I went with keeping the inner loop as the decomposed LoopSequentialPipelineBlocks because that's the structure every other cosmos modality uses (vision/sound/action all subclass Cosmos3DenoiseLoopWrapper), leaving the chunk loop as the only transfer-specific piece, doing it with __call__ override of SequentialPipelineBlocks

Your proposal also works, but the transfer's inner denoise becomes a monolithic leaf block, diverging from the other modalities (cross-pipeline consistency with Helios vs. intra-cosmos consistency)

Let me know if I am missing something, and you would prefer the refactor

[yiyixuxu]

i started a PR on a better loop block design here https://github.com/huggingface/diffusers/pull/14159/changes

so that we can provide a unified loop abstraction that can be freely composed - you can put a loop as sub-block of another loop and to create the nested chunk loop here; so that user don't have to implement custom looop block with a __call__ override

I think the custom loop block here does not work with get_workflow(), it just flattens all the subblocks into a SequentialPipelineblocks . i.e.. i think a workflow-specific pipeline created this way won't work

transfer_pipe= Cosmos3OmniBlocks().get_workflow("transfer").init_pipeline(...)
out = transfer_pipe(...)

but my PR was still in WIP and I don't want to block this PR. I think we can merge this first and we can refactor under my PR? i will tag you for review once it's ready

[github-actions]

Hi @yzhautouskay, thanks for the PR! It does not appear to link an issue it fixes. If this PR addresses an existing issue, please add a closing keyword (e.g. Fixes #1234) to the PR description so the issue is linked. See the contribution guide for more details. If this PR intentionally does not fix a tracked issue, a maintainer can add the no-issue-needed label to silence this reminder.