Add real Ideogram 4 unconditional transformer support - #3076
Merged
Conversation
added 3 commits
August 13, 2026 19:59
--ideogram_load_unconditional_transformer loads the frozen image-only unconditional transformer via the same indexed state-dict machinery as the conditional path, respecting ideogram_fp8_base_upcast, and wires it into both pipeline constructions so validation runs the real asymmetric-CFG branch. --ideogram_uncond_ramtorch builds the unconditional transformer on CPU and reuses the shared _apply_ramtorch_layers helper (new force flag bypasses the global --ramtorch gate) to stream its layers to the accelerator per forward pass. AnyFlow's _unconditional_batch now marks batches with is_unconditional_pass, and Ideogram4.model_predict dispatches marked batches to the unconditional transformer with zeroed llm conditioning and image-only geometry, matching the pipeline's uncond branch. Without the flag, the proxy path through the conditional transformer is unchanged.
Covers flag-off/flag-on loading (single vs dual _build_transformer calls, frozen eval-mode result, fp8 upcast handling), the ramtorch offload path, the model_predict dispatch geometry, and AnyFlow's is_unconditional_pass batch marker. The fp8 upcast test helper now pins the new config flags so Mock truthiness cannot trigger the unconditional load.
Extends the Ideogram 4 quickstart Validation section (and its es, ja, zh, hi, pt-BR translations) with ideogram_load_unconditional_transformer and ideogram_uncond_ramtorch, the ~10 GiB fp8 / ~18 GiB upcast VRAM cost, and the AnyFlow fused-guidance behaviour when the real unconditional branch is loaded.
bghira
force-pushed
the
feature/ideogram-unconditional-transformer
branch
from
August 14, 2026 02:00
889f505 to
c24adf9
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Ideogram 4 performs CFG through a second, image-only unconditional transformer with zeroed text conditioning. Until now SimpleTuner approximated the unconditional branch by running negative prompts through the conditional transformer, which produces structurally different guidance — particularly visible as noisy targets when AnyFlow distillation fuses guidance at higher scales.
--ideogram_load_unconditional_transformerloads the frozen unconditional transformer through the same indexed state-dict machinery as the conditional path, respecting--ideogram_fp8_base_upcast, and wires it into both pipeline constructions so validation runs the real asymmetric-CFG branch.--ideogram_uncond_ramtorchbuilds the unconditional transformer on CPU and streams its layers to the accelerator per forward pass via the shared_apply_ramtorch_layershelper (newforceflag bypasses the global--ramtorchgate), for fitting both transformers on smaller cards. VRAM cost of the resident option: ~10 GiB fp8, ~18 GiB upcast._unconditional_batchmarks batches withis_unconditional_pass, andIdeogram4.model_predictdispatches marked batches to the unconditional transformer with zeroed llm conditioning, so fused-guidance distillation trains against the model's true unconditional output.Testing
.venv/bin/python -m unittest tests.test_ideogram_unconditional tests.test_ideogram_fp8_linear tests.helpers.distillation.test_anyflow_distiller -v -f— 75 tests pass (1 environment-dependent skip). Coverage includes flag-off/flag-on loading, fp8 upcast handling, the ramtorch offload path, model_predict dispatch geometry, and the unconditional batch marker.