Skip to content

StableDiffusionControlNetImg2ImgPipeline cannot load CKPT or Safetensor file directly #3907

@thenol

Description

@thenol

How to load a ckpt or safetensor file directly using StableDiffusionControlNetImg2ImgPipeline ?

Activity

patrickvonplaten

patrickvonplaten commented on Jul 4, 2023

@patrickvonplaten
Contributor

Think for now we have to manually convert and load with from_pretrained . More than happy to accept a pull request working on this though!

0sparsh2

0sparsh2 commented on Jul 5, 2023

@0sparsh2

How to load a ckpt or safetensor file directly using StableDiffusionControlNetImg2ImgPipeline ?

I was trying out a similar pipeline, ControlNetInpaint
Over there the following steps worked for me, you can try the same ->

  1. Locally store a safetensors model
  2. Link the folder in which it's located
  3. Use from_pretrained and use_safetensors = True
    It worked for me
StableDiffusionControlNetInpaintPipeline.from_pretrained(
                "PATH_TO_THE_FOLDER_WITH_SAFETENSORS", controlnet=controlnet_model,
                torch_dtype=torch.float16, use_safetensors=True)
WhiteWolf47

WhiteWolf47 commented on Jul 6, 2023

@WhiteWolf47

Think for now we have to manually convert and load with from_pretrained . More than happy to accept a pull request working on this though!

@patrickvonplaten wdym by "manually convert and load with from_pretrained"? Can you please give an example if i have .ckpt file of the model!

patrickvonplaten

patrickvonplaten commented on Jul 12, 2023

@patrickvonplaten
Contributor

@yiyixuxu maybe you can check this as well (should be the same PR / mechanism): #2584

patrickvonplaten

patrickvonplaten commented on Jul 13, 2023

@patrickvonplaten
Contributor

And another one: #4052 - wow this is a wanted feature!

patrickvonplaten

patrickvonplaten commented on Jul 13, 2023

@patrickvonplaten
Contributor

@yiyixuxu let me take this as you're already on a bunch of other issues :-)

patrickvonplaten

patrickvonplaten commented on Jul 18, 2023

@patrickvonplaten
Contributor

Hey @thenol,

Can you check if this works for you: #4084

EmilioNicolas

EmilioNicolas commented on Jul 19, 2023

@EmilioNicolas

Think for now we have to manually convert and load with from_pretrained . More than happy to accept a pull request working on this though!

Thanks @patrickvonplaten for the PR that's looking great, but meanwhile as I am working with REPLICATE-COG and sticking to new releases of the library, how could I load a CKPT file for a stable_diffusion_controlnet_img2img? Should I follow this? https://huggingface.co/docs/diffusers/v0.18.2/en/using-diffusers/other-formats#pytorch-ckpt

patrickvonplaten

patrickvonplaten commented on Jul 19, 2023

@patrickvonplaten
Contributor

Hey @EmilioNicolas,

You can load .ckpt files also with the from_single_file(...) function

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @EmilioNicolas@thenol@patrickvonplaten@0sparsh2@WhiteWolf47

        Issue actions

          StableDiffusionControlNetImg2ImgPipeline cannot load CKPT or Safetensor file directly · Issue #3907 · huggingface/diffusers