Extracting LoRA model from dreambooth model.

[cerega66]

I recently discovered that from a db-trained model, without LoRA training, LoRA can be extracted. At the same time, the quality of the LoRA model is quite good and is almost as good as the original db model. I don't think I've seen this fact anywhere. Or I didn't search well. I'm wondering if there are any instructions for this, or maybe someone else has done this? I've settled on dim 300 for the time being, although it may make sense to take more, but at 300 the LoRA model already weighs ~ 700mb.

[morphinapg]

Are you doing this from the Kohya webui? I tried it today and while it extracted a Lora (about 600mb at the default 128 rank), using that Lora in the A1111 webui under Extra Networks resulted in pretty much zero difference compared to the base model. I don't really understand why.

[cerega66]

Sorry for not replying for a long time.
I am using this repo for extraction. It is based on scripts from Kohya.
https://github.com/sashaok123/LoRA-EXTRACTOR
I first indicate the model that I received after training, then the model on which I trained. I have done over 100+ lora models this way and have never had the same problem as you.
I also came to the conclusion that when extracting, the lore dimension cannot be more than 320, but I did not find information why.

[morphinapg]

Sorry for not replying for a long time. I am using this repo for extraction. It is based on scripts from Kohya. https://github.com/sashaok123/LoRA-EXTRACTOR I first indicate the model that I received after training, then the model on which I trained. I have done over 100+ lora models this way and have never had the same problem as you. I also came to the conclusion that when extracting, the lore dimension cannot be more than 320, but I did not find information why.

Yeah in Kohya's webui I selected my trained model, and then I selected the model I started training from, and kept ranks at default 128, and it appeared to extract Lora just fine, but when I loaded it into my prompt in A1111's txt2img, while having the base model loaded, it wasn't appearing to have any effect at all on the outcome. Whether I included the Lora or not, it gave pretty much the same image. It's so weird.

It's frustrating because I like the results I get training with regular Dreambooth more than training using Lora, so if I could do that quality of training, and then simplify it down to a Lora that would be perfect.

It would be nice if this extension could add similar functionality, so we wouldn't have to rely on external tools.