Repository navigation
[LoRA] apply .alpha to Qwen-Image lora_A/lora_B LoRAs - #15016
Merged
Merged
Conversation
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
Author
|
Hi @sayakpaul @stevhliu, could you please take a look at my PR when you get a chance? Once everything looks good, I’d appreciate your review and approval of the changes. Thanks for your time and help! |
sayakpaul
reviewed
Oct 20, 2026
| """LoRA tests for the QwenImage pipeline.""" | ||
|
|
||
| @pytest.mark.parametrize("alpha", [None, 2, 4, 8]) | ||
| def test_lora_state_dict_applies_alpha(self, alpha): |
Member
There was a problem hiding this comment.
No need to add this test. These are community-controlled checkpoints not something we can fully control.
Yes, confirmed. With the Qwen-Image-2.1-Fix-v2.0 LoRA from the issue, all 30 modules load with alpha/rank applied and no .alpha keys are left over. Thanks @dcaayushd! |
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
sayakpaul
approved these changes
Oct 22, 2026
mingyi456
added a commit
to mingyi456/diffusers
that referenced
this pull request
Oct 24, 2026
…/mingyi456/diffusers into animatextconditioner-single-file * 'animatextconditioner-single-file' of https://github.com/mingyi456/diffusers: (50 commits) Remove expired non_ema_revision deprecation blocks from training examples (huggingface#15118) Remove duplicated converter and fix import order Update src/diffusers/loaders/single_file_utils.py add fibo-2 LoRA training and loading (huggingface#15084) [ci] fix compilation CI. (huggingface#15092) [LoRA] apply .alpha to Qwen-Image lora_A/lora_B LoRAs (huggingface#15016) Use HF Trusted Publisher token in update_metadata workflow (huggingface#15096) [ci] skip bf16 precision test on T4. (huggingface#15091) [LTX-2] Add `motion_speed` to condition the positional time base apart from the playback frame rate (huggingface#14967) [ci] fix misnamed test steps in push_tests.yml (huggingface#15094) [ci] use python 3.11 when using metadata. (huggingface#15093) Fix model imports when torchao is older than 0.15 (huggingface#15095) [tests] xfail some kandinsky6 tests (huggingface#14977) Add RFD4-Proteina (huggingface#15088) Add Bria fibo-2 (huggingface#14991) Fix PyArrow compatibility in CPU CI (huggingface#15082) [feat] add KVAE-Audio autoencoder (huggingface#14384) move unpack out of the flux, krea2 and ltx decoders into the core denoise groups (huggingface#14737) post-release 0.41.0 (huggingface#15062) FLUX.3: match the released BFL reference image sizing (huggingface#15073) ...
38 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Fixes #14938
Qwen-Image LoRAs using
lora_A/lora_Bweights could lose their per-module.alphavalues during conversion. Whenalphadiffered from the LoRA rank, the adapter was loaded at the wrong strength—for example, a rank-32 LoRA with alpha 83 loaded at approximately 1/2.6 of its intended strength.This change:
.alpha / rankduringlora_A/lora_Bconversion.ValueErrorwhen a matchinglora_Aorlora_Bweight is missing..alphametadata.lora_down/lora_upconversion behavior unchanged.Added tests cover alpha values of
None, 2, 4, and 8, unmatched A/B weights, stray alpha keys, and bfloat16 dtype preservation.Real-checkpoint verification with
e-n-v-y/Qwen-Image-2.1-Fix-v2.0showed:.alphakeys in the raw checkpoint..alphakeys after conversion.(alpha / rank) * (lora_B @ lora_A)with maximum absolute differences between5.6e-09and3.1e-05across three sampled modules.The existing
lora_down/lora_upconversion output was identical to the upstream output for a synthetic Qwen-Image-Lightning checkpoint.Before submitting
self-reviewskill? The skill was not installed in the environment; its checked-in rubric was applied manually instead.Validation
ruff check: passed.ruff format --check: passed.git diff --check: passed.Self-review notes
The checked-in self-review rubric was applied manually because the
self-reviewCLI skill was unavailable. No blocking correctness, dead-code, copied-code, or documentation issues were found.AI assistance was used for implementation, testing, verification, and drafting this description. Please review and edit the wording before submitting.