Support MuLan, a plug-and-play language adapter to adapt existing diffusion model for up to 110+ languages without additional training

[Zeqiang-Lai]

Model/Pipeline/Scheduler description

# pip install mulankit
from diffusers import StableDiffusionPipeline
+ import mulankit

pipe = StableDiffusionPipeline.from_pretrained('Lykon/dreamshaper-8')
+ pipe = mulankit.transform(pipe, 'mulanai/mulan-lang-adapter::sd15_aesthetic.pth')
image = pipe('一只蓝色的🐶 in the 바다').images[0]

一只蓝色的 🐶 in the 바다 (Dreamshaper-8)	レゴシュワルツェネッガー (SDXL-lightning)	一只可爱的猫头鹰 (MVDream)	海浪风景 (AnimateDiff)

MuLan supports

• Base models: Stable Diffusion 1.5, 2.1, XL, Pixart-Alpha/Sigma.
• Downstream models: ControlNet, LCM, LoRA, finetuned models and etc.
• Video models: AnimateDiff.
• 3D models: MVDream.

Open source status

• The model implementation is available.
• The model weights are available (Only relevant if addition is not a scheduler).

Provide useful links for the implementation

https://github.com/mulanai/MuLan
https://huggingface.co/mulanai/mulan-lang-adapter

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.