FIX Bug when merging negatively weighted adapters (#2918)

BenjaminBossan · Conzel · commit 59c8dc344a87 · 2025-11-25T14:59:35.000Z
See #2796 (comment) There was a bug with handling negative weights when merging multiple LoRA adapters into a single one using add_weighted_adapter (#2811). It should now be handled. The accompanying tests did not reveal the error because they compared A and B separately, not after multiplying (the "delta weight").
diff --git a/tests/test_custom_models.py b/tests/test_custom_models.py
@@ -62,7 +62,6 @@
 )
 from peft.tuners.lora.config import BdLoraConfig
 from peft.tuners import lora
-from peft.tuners.lora.config import BdLoraConfig
 from peft.tuners.tuners_utils import BaseTunerLayer
 from peft.utils import AuxiliaryTrainingWrapper, infer_device
 

Original file line number	Diff line number	Diff line change
`@@ -62,7 +62,6 @@`
`62`	`62`	`)`
`63`	`63`	`from peft.tuners.lora.config import BdLoraConfig`
`64`	`64`	`from peft.tuners import lora`
`65`		`-from peft.tuners.lora.config import BdLoraConfig`
`66`	`65`	`from peft.tuners.tuners_utils import BaseTunerLayer`
`67`	`66`	`from peft.utils import AuxiliaryTrainingWrapper, infer_device`
`68`	`67`