Dreambooth XL Lora not working with Fooocus 2.3.1

[apoorvmish13]

I have created a LoRA using the instructions at on a GCP VM with NVIDIA L4 GPU. But when I tried to use it on Fooocus with sd_xl_base_1.0 as the base model, the LoRA does not get loaded. Other LoRAs downloaded from civitai get loaded and work perfectly.

On debugging, I found that fooocus is expecting LoRA keys in the following format:
'lora_unet_time_embed_0', 'lora_unet_time_embed_2', 'lora_unet_label_emb_0_0', 'lora_unet_label_emb_0_2', 'lora_unet_input_blocks_0_0', 'lora_unet_input_blocks_1_0_in_layers_0', 'lora_unet_input_blocks_1_0_in_layers_2', 'lora_unet_input_blocks_1_0_emb_layers_1', 'lora_unet_input_blocks_1_0_out_layers_0', 'lora_unet_input_blocks_1_0_out_layers_3', 'lora_unet_input_blocks_2_0_in_layers_0', 'lora_unet_input_blocks_2_0_in_layers_2', 'lora_unet_input_blocks_2_0_emb_layers_1', 'lora_unet_input_blocks_2_0_out_layers_0', 'lora_unet_input_blocks_2_0_out_layers_3', 'lora_unet_input_blocks_3_0_op', 'lora_unet_input_blocks_4_0_in_layers_0', 'lora_unet_input_blocks_4_0_in_layers_2', 'lora_unet_input_blocks_4_0_emb_layers_1', 'lora_unet_input_blocks_4_0_out_layers_0', 'lora_unet_input_blocks_4_0_out_layers_3', 'lora_unet_input_blocks_4_0_skip_connection', 'lora_unet_input_blocks_4_1_norm', 'lora_unet_input_blocks_4_1_proj_in', 'lora_unet_input_blocks_4_1_transformer_blocks_0_attn1_to_q', 'lora_unet_input_blocks_4_1_transformer_blocks_0_attn1_to_k', 'lora_unet_input_blocks_4_1_transformer_blocks_0_attn1_to_v'

Whereas the actual keys in the LoRA are in a slightly different format:
'unet.down_blocks.1.attentions.0.transformer_blocks.0.attn1.to_k.lora.down.weight', 'unet.down_blocks.1.attentions.0.transformer_blocks.0.attn1.to_k.lora.up.weight', 'unet.down_blocks.1.attentions.0.transformer_blocks.0.attn1.to_out.0.lora.down.weight', 'unet.down_blocks.1.attentions.0.transformer_blocks.0.attn1.to_out.0.lora.up.weight', 'unet.down_blocks.1.attentions.0.transformer_blocks.0.attn1.to_q.lora.down.weight', 'unet.down_blocks.1.attentions.0.transformer_blocks.0.attn1.to_q.lora.up.weight', 'unet.down_blocks.1.attentions.0.transformer_blocks.0.attn1.to_v.lora.down.weight', 'unet.down_blocks.1.attentions.0.transformer_blocks.0.attn1.to_v.lora.up.weight', 'unet.down_blocks.1.attentions.0.transformer_blocks.0.attn2.to_k.lora.down.weight', 'unet.down_blocks.1.attentions.0.transformer_blocks.0.attn2.to_k.lora.up.weight', 'unet.down_blocks.1.attentions.0.transformer_blocks.0.attn2.to_out.0.lora.down.weight', 'unet.down_blocks.1.attentions.0.transformer_blocks.0.attn2.to_out.0.lora.up.weight'

Am I looking in the right direction? I have noticed that in fooocus, there is code to generate expected LoRA keys by modifying unet and clip keys of base model. Is it possible that LoRA keys generated by dreambooth for sdxl are not created in the expected format by fooocus?

[asomoza]

Hi, I think none of the UIs use the diffusers format, to use the LoRA with them you need to use the --output_kohya_format arg.

[apoorvmish13]

Thanks a lot @asomoza, this worked :)