merge chroma radiance (tests, diff) #5

Ednaordinary · 2025-11-30T07:44:03Z

No description provided.

Ednaordinary · 2025-12-13T13:53:48Z

@lodestone-rock hey, I helped implement the original version of chroma into diffusers and I'm working on getting radiance in! what would you recommend for default params (both for 0.4 and x0)? thanks for making chroma!

sync

Ednaordinary · 2025-12-17T00:37:03Z

contd. huggingface#12850

lodestone-rock · 2025-12-17T01:48:40Z

the default params should be no different than chroma
atleast that's how i trained it
it's just now it's in pixel space

lodestone-rock · 2025-12-17T01:49:58Z

i'm not quite sure how to translate comfyui default params to huggingface, but in comfy there's this default workflow for chroma radiance and that's pretty good

Ednaordinary · 2025-12-17T02:03:51Z

hmm.. I've adapted best I've could (example shown in diffusers pr), but something in what I believe is likely the scheduler or the nerf model is wrong

lodestone-rock · 2025-12-17T02:14:30Z

try using simple euler instead for sampling instead of heun?

Ednaordinary · 2025-12-17T05:39:31Z

looks like at some point while debugging I removed the attention mask and didn't add it back.. getting closer, but still not there, just yet. It also seems both Euler and Heun are converging on the same image, which is interesting

* fix torchao quantizer for new torchao versions Summary: `torchao==0.16.0` (not yet released) has some bc-breaking changes, this PR fixes the diffusers repo with those changes. Specifics on the changes: 1. `UInt4Tensor` is removed: pytorch/ao#3536 2. old float8 tensors v1 are removed: pytorch/ao#3510 In this PR: 1. move the logger variable up (not sure why it was in the middle of the file before) to get better error messages 2. gate the old torchao objects by torchao version Test Plan: import diffusers objects with new versions of torchao works: ```bash > python -c "import torchao; print(torchao.__version__); from diffusers import StableDiffusionPipeline" 0.16.0.dev20251229+cu129 ``` Reviewers: Subscribers: Tasks: Tags: * Apply style fixes --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

add output to auto blocks + core denoising block for better doc string

* polish caching docs. * Update docs/source/en/optimization/cache.md Co-authored-by: Steven Liu <[email protected]> * Update docs/source/en/optimization/cache.md Co-authored-by: Steven Liu <[email protected]> * up --------- Co-authored-by: Steven Liu <[email protected]>

* Fix QwenImage txt_seq_lens handling * formatting * formatting * remove txt_seq_lens and use bool mask * use compute_text_seq_len_from_mask * add seq_lens to dispatch_attention_fn * use joint_seq_lens * remove unused index_block * WIP: Remove seq_lens parameter and use mask-based approach - Remove seq_lens parameter from dispatch_attention_fn - Update varlen backends to extract seqlens from masks - Update QwenImage to pass 2D joint_attention_mask - Fix native backend to handle 2D boolean masks - Fix sage_varlen seqlens_q to match seqlens_k for self-attention Note: sage_varlen still producing black images, needs further investigation * fix formatting * undo sage changes * xformers support * hub fix * fix torch compile issues * fix tests * use _prepare_attn_mask_native * proper deprecation notice * add deprecate to txt_seq_lens * Update src/diffusers/models/transformers/transformer_qwenimage.py Co-authored-by: YiYi Xu <[email protected]> * Update src/diffusers/models/transformers/transformer_qwenimage.py Co-authored-by: YiYi Xu <[email protected]> * Only create the mask if there's actual padding * fix order of docstrings * Adds performance benchmarks and optimization details for QwenImage Enhances documentation with comprehensive performance insights for QwenImage pipeline: * rope_text_seq_len = text_seq_len * rename to max_txt_seq_len * removed deprecated args * undo unrelated change * Updates QwenImage performance documentation Removes detailed attention backend benchmarks and simplifies torch.compile performance description Focuses on key performance improvement with torch.compile, highlighting the specific speedup from 4.70s to 1.93s on an A100 GPU Streamlines the documentation to provide more concise and actionable performance insights * Updates deprecation warnings for txt_seq_lens parameter Extends deprecation timeline for txt_seq_lens from version 0.37.0 to 0.39.0 across multiple Qwen image-related models Adds a new unit test to verify the deprecation warning behavior for the txt_seq_lens parameter * fix compile * formatting * fix compile tests * rename helper * remove duplicate * smaller values * removed * use torch.cond for torch compile * Construct joint attention mask once * test different backends * construct joint attention mask once to avoid reconstructing in every block * Update src/diffusers/models/attention_dispatch.py Co-authored-by: YiYi Xu <[email protected]> * formatting * raising an error from the EditPlus pipeline when batch_size > 1 --------- Co-authored-by: Sayak Paul <[email protected]> Co-authored-by: YiYi Xu <[email protected]> Co-authored-by: cdutr <[email protected]>

* Bugfix for dreambooth flux2 img2img2 * Bugfix for dreambooth flux2 img2img2 * Bugfix for dreambooth flux2 img2img2 * Bugfix for dreambooth flux2 img2img2 * Bugfix for dreambooth flux2 img2img2 * Bugfix for dreambooth flux2 img2img2 Co-authored-by: tcaimm <[email protected]> --------- Co-authored-by: tcaimm <[email protected]>

… LoRA Tests (huggingface#12962) * Improve incorrect LoRA format error message * Add flag in PeftLoraLoaderMixinTests to disable text encoder LoRA tests * Apply changes to LTX2LoraTests * Further improve incorrect LoRA format error msg following review --------- Co-authored-by: Sayak Paul <[email protected]>

* initial scheme of unified-sp * initial all_to_all_double * bug fixes, added cmnts * unified attention prototype done * remove raising value error in contextParallelConfig to enable unified attention * bug fix * feat: Adds Test for Unified SP Attention and Fixes a bug in Template Ring Attention * bug fix, lse calculation, testing bug fixes, lse calculation - switched to _all_to_all_single helper in _all_to_all_dim_exchange due contiguity issues bug fix bug fix bug fix * addressing comments * sequence parallelsim bug fixes * code format fixes * Apply style fixes * code formatting fix * added unified attention docs and removed test file * Apply style fixes * tip for unified attention in docs at distributed_inference.md Co-authored-by: Sayak Paul <[email protected]> * Update distributed_inference.md, adding benchmarks Co-authored-by: Sayak Paul <[email protected]> * Update docs/source/en/training/distributed_inference.md Co-authored-by: Sayak Paul <[email protected]> * function name fix * fixed benchmark in docs --------- Co-authored-by: KarthikSundar2002 <[email protected]> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Sayak Paul <[email protected]>

* initial * add kayers

* init * add * add 1 * Update __init__.py * rename * 2 * update * init with encoder * merge2pipeline * Update pipeline_glm_image.py * remove sop * remove useless func * Update pipeline_glm_image.py * up (cherry picked from commit cfe19a3) * review for work only * change place * Update pipeline_glm_image.py * update * Update transformer_glm_image.py * 1 * no negative_prompt for GLM-Image * remove CogView4LoraLoaderMixin * refactor attention processor. * update * fix * use staticmethod * update * up * up * update * Update glm_image.md * 1 * Update pipeline_glm_image.py * Update transformer_glm_image.py * using new transformers impl * support * resolution change * fix-copies * Update src/diffusers/pipelines/glm_image/pipeline_glm_image.py Co-authored-by: YiYi Xu <[email protected]> * Update pipeline_glm_image.py * use cogview4 * Update pipeline_glm_image.py * Update pipeline_glm_image.py * revert * update * batch support * update * version guard glm image pipeline * validate prompt_embeds and prior_token_ids * try docs. * 4 * up * up * skip properly * fix tests * up * up --------- Co-authored-by: zRzRzRzRzRzRzR <[email protected]> Co-authored-by: yiyixuxu <[email protected]>

…2971)

…ingface#12974) * make transformers version check stricter for glm image. * public checkpoint.

* allow to * update version * fix version again * again * Update src/diffusers/pipelines/pipeline_utils.py Co-authored-by: Copilot <[email protected]> * style * xfail * add pr --------- Co-authored-by: Copilot <[email protected]> Co-authored-by: Sayak Paul <[email protected]>

* update * `disable_mmap` in `from_pretrained` --------- Co-authored-by: DN6 <[email protected]>

* up * style --------- Co-authored-by: [email protected] <[email protected]>

… component (huggingface#12963) * Don't attempt to move the text_encoder. Just move the generated_ids. * The inputs to the text_encoder should be on its device

* Add `ChromaInpaintPipeline` * Set `attention_mask` to `dtype=torch.bool` for `ChromaInpaintPipeline`. * Revert `.gitignore`.

* fix qwen-image cp * relax attn_mask limit for cp * CP plan compatible with zero_cond_t * move modulate_index plan to top level

* flux2-klein * Apply suggestions from code review Co-authored-by: Sayak Paul <[email protected]> * Klein tests (#2) * tests * up * tests * up * support step-distilled * Apply suggestions from code review Co-authored-by: dg845 <[email protected]> * Apply suggestions from code review Co-authored-by: dg845 <[email protected]> * doc string etc * style * more * copies * klein lora training scripts (#3) * initial commit * initial commit * remove remote text encoder * initial commit * initial commit * initial commit * revert * img2img fix * text encoder + tokenizer * text encoder + tokenizer * update readme * guidance * guidance * guidance * test * test * revert changes not needed for the non klein model * Update examples/dreambooth/train_dreambooth_lora_flux2_klein.py Co-authored-by: Sayak Paul <[email protected]> * fix guidance * fix validation * fix validation * fix validation * fix path * space --------- Co-authored-by: Sayak Paul <[email protected]> * style * Update src/diffusers/pipelines/flux2/pipeline_flux2_klein.py * Apply style fixes * auto pipeline --------- Co-authored-by: Sayak Paul <[email protected]> Co-authored-by: dg845 <[email protected]> Co-authored-by: Linoy Tsaban <[email protected]> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…#12980) * update mellonparams docstring to incude the acutal param definition render in mellon * style --------- Co-authored-by: [email protected] <[email protected]>

* LTX 2 transformer single file support * LTX 2 video VAE single file support * LTX 2 audio VAE single file support * Make it easier to distinguish LTX 1 and 2 models

…ted. (huggingface#12832) * gracefully error out when attn-backend x cp combo isn't supported. * Revert "gracefully error out when attn-backend x cp combo isn't supported." This reverts commit c8abb5d. * gracefully error out when attn-backend x cp combo isn't supported. * up * address PR feedback. * up * Update src/diffusers/models/modeling_utils.py Co-authored-by: Dhruv Nair <[email protected]> * dot. --------- Co-authored-by: Dhruv Nair <[email protected]>

…istep.py (huggingface#12936) * docs: improve docstring scheduling_cosine_dpmsolver_multistep.py * Update src/diffusers/schedulers/scheduling_cosine_dpmsolver_multistep.py Co-authored-by: Steven Liu <[email protected]> * Update src/diffusers/schedulers/scheduling_cosine_dpmsolver_multistep.py Co-authored-by: Steven Liu <[email protected]> * fix --------- Co-authored-by: Steven Liu <[email protected]>

…ingface#12986) Chore: Replace CONTRIBUTING.md with a symlink to documentation

This reverts commit 76f51a5.

make style to push new changes.

* Feature: Add BriaFiboEditPipeline to diffusers * Introduced BriaFiboEditPipeline class with necessary backend requirements. * Updated import structures in relevant modules to include BriaFiboEditPipeline. * Ensured compatibility with existing pipelines and type checking. * Feature: Introduce Bria Fibo Edit Pipeline * Added BriaFiboEditPipeline class for structured JSON-native image editing. * Created documentation for the new pipeline in bria_fibo_edit.md. * Updated import structures to include the new pipeline and its components. * Added unit tests for the BriaFiboEditPipeline to ensure functionality and correctness. * Enhancement: Update Bria Fibo Edit Pipeline and Documentation * Refined the Bria Fibo Edit model description for clarity and detail. * Added usage instructions for model authentication and login. * Implemented mask handling functions in the BriaFiboEditPipeline for improved image editing capabilities. * Updated unit tests to cover new mask functionalities and ensure input validation. * Adjusted example code in documentation to reflect changes in the pipeline's usage. * Update Bria Fibo Edit documentation with corrected Hugging Face page link * add dreambooth training script * style and quality * Delete temp.py * Enhancement: Improve JSON caption validation in DreamBoothDataset * Updated the clean_json_caption function to handle both string and dictionary inputs for captions. * Added error handling to raise a ValueError for invalid caption types, ensuring better input validation. * Add datasets dependency to requirements_fibo_edit.txt * Add bria_fibo_edit to docs table of contents * Fix dummy objects ordering * Fix BriaFiboEditPipeline to use passed generator parameter The pipeline was ignoring the generator parameter and only using the seed parameter. This caused non-deterministic outputs in tests that pass a seeded generator. * Remove fibo_edit training script and related files --------- Co-authored-by: kfirbria <[email protected]>

Ednaordinary added 12 commits November 30, 2025 00:24

init (pipeline, transformer, nerf, etc)

ee2740a

undo regressions

e0707bd

fixes

07b368e

random tweaks

d3bf917

make saveable

1a20f53

update

ed746c5

fix img_ids

0a19b7e

remove some prints

df3ab44

updates

1a04172

get closer to fixing attn mask

a829909

fix attention

54f3bbf

make inference run

655b8e6

Ednaordinary and others added 5 commits December 13, 2025 07:01

cleanup

0191cfd

add x0, change final layer some

72748ec

Merge pull request #6 from Ednaordinary/main

ae023a3

sync

make style make quality

5a4312a

removed copied from (modified)

414c63e

fix attention mask

e3ee921

Ednaordinary and others added 5 commits December 16, 2025 23:23

updates

3814a6a

Merge branch 'main' into chroma-radiance

5e47460

add lru cache

17388e8

sync autocast logic

0770561

yiyixuxu and others added 30 commits January 9, 2026 23:53

[Modular] better docstring (huggingface#12932)

418313b

add output to auto blocks + core denoising block for better doc string

Fix typos (huggingface#12705)

5b20211

Fix link to diffedit implementation reference (huggingface#12708)

b86bd99

[Modular] Changes for using WAN I2V (huggingface#12959)

3114f6a

* initial * add kayers

Update distributed_inference.md to reposition sections (huggingface#1…

3c70440

…2971)

[chore] make transformers version check stricter for glm image. (hugg…

7feb4fc

…ingface#12974) * make transformers version check stricter for glm image. * public checkpoint.

disable_mmap in pipeline from_pretrained (huggingface#12854)

1ecfbfe

* update * `disable_mmap` in `from_pretrained` --------- Co-authored-by: DN6 <[email protected]>

[Modular] mellon utils (huggingface#12978)

d8f4dd2

* up * style --------- Co-authored-by: [email protected] <[email protected]>

LongCat Image pipeline: Allow offloading/quantization of text_encoder…

b351be2

… component (huggingface#12963) * Don't attempt to move the text_encoder. Just move the generated_ids. * The inputs to the text_encoder should be on its device

Add ChromaInpaintPipeline (huggingface#12848)

5efb81f

* Add `ChromaInpaintPipeline` * Set `attention_mask` to `dtype=torch.bool` for `ChromaInpaintPipeline`. * Revert `.gitignore`.

fix Qwen-Image series context parallel (huggingface#12970)

7f43cb1

* fix qwen-image cp * relax attn_mask limit for cp * CP plan compatible with zero_cond_t * move modulate_index plan to top level

[modular] fix a bug in mellon param & improve docstrings (huggingface…

f112eab

…#12980) * update mellonparams docstring to incude the acutal param definition render in mellon * style --------- Co-authored-by: [email protected] <[email protected]>

add klein docs. (huggingface#12984)

74654df

LTX 2 Single File Support (huggingface#12983)

8af8e86

* LTX 2 transformer single file support * LTX 2 video VAE single file support * LTX 2 audio VAE single file support * Make it easier to distinguish LTX 1 and 2 models

[Docs] Replace root CONTRIBUTING.md with symlink to source docs (hugg…

3996788

…ingface#12986) Chore: Replace CONTRIBUTING.md with a symlink to documentation

make style && make quality

76f51a5

Revert "make style && make quality"

75edff9

This reverts commit 76f51a5.

[chore] make style to push new changes. (huggingface#12998)

29b15f4

make style to push new changes.

Merge branch 'main' into chroma-radiance

1d3e2ea

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

merge chroma radiance (tests, diff) #5

merge chroma radiance (tests, diff) #5

Uh oh!

Ednaordinary commented Nov 30, 2025

Uh oh!

Ednaordinary commented Dec 13, 2025

Uh oh!

Ednaordinary commented Dec 17, 2025 •

edited

Loading

Uh oh!

lodestone-rock commented Dec 17, 2025

Uh oh!

lodestone-rock commented Dec 17, 2025

Uh oh!

Ednaordinary commented Dec 17, 2025 •

edited

Loading

Uh oh!

lodestone-rock commented Dec 17, 2025

Uh oh!

Ednaordinary commented Dec 17, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

merge chroma radiance (tests, diff) #5

Are you sure you want to change the base?

merge chroma radiance (tests, diff) #5

Uh oh!

Conversation

Ednaordinary commented Nov 30, 2025

Uh oh!

Ednaordinary commented Dec 13, 2025

Uh oh!

Ednaordinary commented Dec 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lodestone-rock commented Dec 17, 2025

Uh oh!

lodestone-rock commented Dec 17, 2025

Uh oh!

Ednaordinary commented Dec 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lodestone-rock commented Dec 17, 2025

Uh oh!

Ednaordinary commented Dec 17, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

Ednaordinary commented Dec 17, 2025 •

edited

Loading

Ednaordinary commented Dec 17, 2025 •

edited

Loading