New Model: Ideogram 4 by dxqb · Pull Request #1522 · Nerogar/OneTrainer

dxqb · 2026-06-14T14:22:04Z

Test in preview branch: https://github.com/Nerogar/OneTrainer/tree/preview

Summary

Summary:

Adds support for Ideogram 4, including model loading/saving, data loading, sampling, and training setups for both LoRA and Fine Tune methods.
Unconditional transformer can be optionally loaded for sampling
Dynamic timestep shifting is according to the ideogram shifting schedule (differs from Flux)

Note:

this model requires JSON prompting: https://github.com/ideogram-oss/ideogram4/blob/main/docs/prompting.md#plain-text-vs-json-prompts
OneTrainer captions are still just 1 line. The entire JSON has to be in 1 line, see https://github.com/ideogram-oss/ideogram4/blob/main/docs/prompting.md#plain-text-vs-json-prompts
Better JSON prompting and captioning is beyond the scope of this PR: [Feat]: Advanced captioning / JSON #1508
For testing purposes, only:
No compatibility with inference tools (not even attempted)
loads a bf16 checkpoint and then quantizes using the regular OneTrainer pipeline. Loading the official nf4/fp8 checkpoints requires [Feat]: support loading quantized transformer files (non-GGUF) #1351

Test plan

pre-commit run --all-files passes
Launched the affected UI or script and exercised the change
Tested with at least one real preset / config when relevant (note which: Ideogram both)

AI assistance

AI-assisted — I have read every line in this diff and can defend each change

…model composition in ModelType - Gradient checkpointing and layer offloading are now configured per component (text encoder, transformer, VAE) rather than globally - ModelType centralizes model composition and training method associations Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…16 (Nerogar#147…" This reverts commit 574ec55.

Several model savers (Ernie, Flux2, Z-Image, ...) duplicate the same deepcopy + tokenizer __deepcopy__ workaround to produce a dtype-converted copy of a diffusers pipeline for saving. Extract it into a shared helper so new savers can reuse it. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Adds a tokenizer_attrs parameter (default ("tokenizer",)) so savers with extra/different tokenizer attributes (Flux's tokenizer_2, SD3's tokenizer_3, HiDream's tokenizer_3/tokenizer_4) can use the same helper. Replaces the duplicated deepcopy + tokenizer __deepcopy__ workaround in Chroma, Ernie, Flux, Flux2, HiDream, HunyuanVideo, PixArtAlpha, Qwen, Sana, StableDiffusion3 and Z-Image with calls to the shared helper. No behavior change. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Move the per-callsite checkpointing_or_offloading_enabled() guard into enable_checkpointing() itself, so every Base*Setup can call enable_checkpointing_for_* unconditionally. Also extend the central gate to allow a compile-only path (no checkpointing/offloading, but still per-layer torch.compile wrapping) when config.compile is set. Three direct diffusers enable_gradient_checkpointing() calls (SD/SDXL unet, Wuerstchen v2 prior) keep their explicit guard since they bypass this central mechanism.

Ports the Ideogram 4 image generation model into OneTrainer, including model loading/saving, data loading, sampling, training setups for LoRA and Fine Tune, and corresponding UI and preset additions. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

BobJohnson24 · 2026-06-14T18:29:18Z

I think that enabling cfg sampling for the cond with uncond model not loaded is a reasonable choice and does not need to be gated behind cfg 1.0. Not sure if posting it here or in the discord is better, just something I encountered and had an opinion on.

dxqb · 2026-06-14T18:33:19Z

I think that enabling cfg sampling for the cond with uncond model not loaded is a reasonable choice and does not need to be gated behind cfg 1.0. Not sure if posting it here or in the discord is better, just something I encountered and had an opinion on.

the idea isn't to arbitrarily gate it - an unconditional is needed for cfg > 1.0.
Are you proposing that using the conditional transformer to get the unconditional works well? With an empty prompt?

BobJohnson24 · 2026-06-14T18:38:50Z

I think that enabling cfg sampling for the cond with uncond model not loaded is a reasonable choice and does not need to be gated behind cfg 1.0. Not sure if posting it here or in the discord is better, just something I encountered and had an opinion on.

the idea isn't to arbitrarily gate it - an unconditional is needed for cfg > 1.0. Are you proposing that using the conditional transformer to get the unconditional works well? With an empty prompt?

Yeah, sorry I should have been more clear about this. Using the cond as the uncond with an empty prompt is indeed an approach that works "reasonably" well, i.e. good enough for fast sample previews, and what I would like to propose.

dxqb · 2026-06-14T18:42:39Z

I think that enabling cfg sampling for the cond with uncond model not loaded is a reasonable choice and does not need to be gated behind cfg 1.0. Not sure if posting it here or in the discord is better, just something I encountered and had an opinion on.

the idea isn't to arbitrarily gate it - an unconditional is needed for cfg > 1.0. Are you proposing that using the conditional transformer to get the unconditional works well? With an empty prompt?

Yeah, sorry I should have been more clear about this. Using the cond as the uncond with an empty prompt is indeed an approach that works "reasonably" well, i.e. good enough for fast sample previews, and what I would like to propose.

ok, will add that. but also consider to just load the unconditional if you have the RAM.
with the new PR to set different offloading fractions for different components, you can train the conditional with no offloading efficiently, and then use maximum offloading on the unconditional transformer that's only used during sampling.

BobJohnson24 · 2026-06-14T18:48:50Z

I think that enabling cfg sampling for the cond with uncond model not loaded is a reasonable choice and does not need to be gated behind cfg 1.0. Not sure if posting it here or in the discord is better, just something I encountered and had an opinion on.

the idea isn't to arbitrarily gate it - an unconditional is needed for cfg > 1.0. Are you proposing that using the conditional transformer to get the unconditional works well? With an empty prompt?

Yeah, sorry I should have been more clear about this. Using the cond as the uncond with an empty prompt is indeed an approach that works "reasonably" well, i.e. good enough for fast sample previews, and what I would like to propose.

ok, will add that. but also consider to just load the unconditional if you have the RAM. with the new PR to set different offloading fractions for different components, you can train the conditional with no offloading efficiently, and then use maximum offloading on the unconditional transformer that's only used during sampling.

Good point, I am not that short on ram. Personally I just didn't know if the lora would be applied on the uncond or not, and was trying to play it safe with the sampling.

dxqb · 2026-06-14T18:52:42Z

Good point, I am not that short on ram. Personally I just didn't know if the lora would be applied on the uncond or not, and was trying to play it safe with the sampling.

the LoRA is currently not applied to the unconditional for sampling. maybe it should be, on Discord there was a good example that this is helpful (almost necessary according to that sample)

# Conflicts: # modules/modelSetup/BaseErnieSetup.py # modules/modelSetup/BaseWuerstchenSetup.py # modules/util/checkpointing_util.py # training_presets/#flux2 LoRA 8GB.json

# Conflicts: # modules/ui/TimestepDistributionWindow.py # modules/ui/TrainingTab.py

Mirrors upstream commit 75a44d2, which converted the rest of the codebase from the trailing factory.register() call to the @factory.register decorator form.

# Conflicts: # modules/modelLoader/mixin/HFModelLoaderMixin.py # requirements-global.txt

…xt caching, autocast arg count) Audit per WORKFLOW_PREVIEW.md checklist after merging #1522: IdeogramModel still used the old to() API instead of release(), three files still referenced config.latent_caching instead of image_caching/text_caching, and BaseIdeogramSetup still called create_autocast_context/disable_fp16_autocast_context with the old weight-list argument. Also adds the Ideogram LoRA preset to run_lora_presets.sh, which was missing.

dxqb and others added 12 commits May 25, 2026 18:11

Merge branch 'master' into split-offload

5a41835

Revert "Revert "Upgrade transformers to 5.9 and huggingface-hub to 1.…

2f0620b

…16 (Nerogar#147…" This reverts commit 574ec55.

Merge branch 'upstream' into split-offload

0b4ddc4

Merge commit '2f0620be0' into ideogram-base

d12ddd8

Merge branch 'upstream' into split-offload

15f40a8

Merge branch 'split-offload' into ideogram-base

74ba5e3

Merge branch 'save-pipeline-dtype-util' into ideogram-base

0c23f3a

dxqb added the preview merged in the preview branch label Jun 14, 2026

Pin mgds to dxqb/mgds PR Nerogar#57 (ideogram branch)

67e5a42

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

dxqb mentioned this pull request Jun 14, 2026

on-demand loading of text encoders #1509

Draft

3 tasks

dxqb mentioned this pull request Jun 14, 2026

split "Latent Caching" into "Image Caching" and "Text Caching" #1462

Open

2 tasks

dxqb added a commit that referenced this pull request Jun 14, 2026

Merge PR #1522 (New Model: Ideogram 4) into preview

347a43e

This was referenced Jun 14, 2026

Upgrade transformers to 5.9 #1506

Closed

[Feat]: Support for Ideogram 4 #1513

Open

dxqb linked an issue Jun 15, 2026 that may be closed by this pull request

[Feat]: Support for Ideogram 4 #1513

Open

dxqb added 4 commits June 18, 2026 01:24

Merge remote-tracking branch 'Nerogar/master' into ideogram-base

618cc68

# Conflicts: # modules/modelSetup/BaseErnieSetup.py # modules/modelSetup/BaseWuerstchenSetup.py # modules/util/checkpointing_util.py # training_presets/#flux2 LoRA 8GB.json

Merge branch 'ideogram-base' into ideogram

aa7225e

# Conflicts: # modules/ui/TimestepDistributionWindow.py # modules/ui/TrainingTab.py

Use decorator form for factory.register() in Ideogram model files

71b8067

Mirrors upstream commit 75a44d2, which converted the rest of the codebase from the trailing factory.register() call to the @factory.register decorator form.

Merge branch 'master' into ideogram

4f35755

dxqb added 3 commits June 19, 2026 08:11

Merge remote-tracking branch 'Nerogar/master' into ideogram-base

b67dd10

# Conflicts: # modules/modelLoader/mixin/HFModelLoaderMixin.py # requirements-global.txt

Merge branch 'ideogram-base' into ideogram

96f2f4f

Merge remote-tracking branch 'origin/ideogram' into ideogram

c0e1d73

dxqb added a commit that referenced this pull request Jun 19, 2026

Merge PR #1522 (New Model: Ideogram 4) into preview

9f37333

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

New Model: Ideogram 4#1522

New Model: Ideogram 4#1522
dxqb wants to merge 20 commits into
Nerogar:masterfrom
dxqb:ideogram

dxqb commented Jun 14, 2026 •

edited

Loading

Uh oh!

BobJohnson24 commented Jun 14, 2026

Uh oh!

dxqb commented Jun 14, 2026

Uh oh!

BobJohnson24 commented Jun 14, 2026 •

edited

Loading

Uh oh!

dxqb commented Jun 14, 2026

Uh oh!

BobJohnson24 commented Jun 14, 2026

Uh oh!

dxqb commented Jun 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

dxqb commented Jun 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

AI assistance

Uh oh!

BobJohnson24 commented Jun 14, 2026

Uh oh!

dxqb commented Jun 14, 2026

Uh oh!

BobJohnson24 commented Jun 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dxqb commented Jun 14, 2026

Uh oh!

BobJohnson24 commented Jun 14, 2026

Uh oh!

dxqb commented Jun 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

dxqb commented Jun 14, 2026 •

edited

Loading

BobJohnson24 commented Jun 14, 2026 •

edited

Loading