adding trust_remote_code argument for loading dataset in controlnet traininig example

[YanivDorGalron]

changes mode to both the readme and the train_controlnet.py example

[asomoza]

Hi, thanks for your contribution, can you help me understand why is this needed? do you have a use case for enabling remote code inside a dataset?

Also it should be False by default not True, so only people that needs it and know what they're doing can use it and enable it.

[YanivDorGalron]

Hi,

Thank you for reviewing my pull request.

I followed the instructions in the README for training ControlNet with the fill50k circles dataset on my multi-GPU setup. During this process, I encountered a prompt asking if I trust_remote_code (actually multiple identical prompt for each gpu). Due to this multiprocessing, I couldn't respond 'yes' to the prompt which was the result of a remote code on the dataset side. Considering this example is specific to the fill50k dataset, I assumed setting trust_remote_code=True by default was appropriate.

[asomoza]

oh I see, I still haven't had the opportunity to train a ControlNet, so I didn't know that the demo and that dataset need the remote code, its weird that people didn't open this issue before, ccing @sayakpaul since I don't have enough experience with this yet.

[sayakpaul]

@YanivDorGalron thanks for your PR. Could you show us for which ControlNet checkpoint this is useful?

[YanivDorGalron]

Hi @sayakpaul , I run accelerate config default and right after that this command:

export MODEL_DIR="stable-diffusion-v1-5/stable-diffusion-v1-5"
export OUTPUT_DIR="."

accelerate launch train_controller.py \
    --pretrained_model_name_or_path=$MODEL_DIR \
    --output_dir=$OUTPUT_DIR \
    --dataset_name=fusing/fill56k \
    --resolution=512 \
    --learning_rate=1e-5 \
    --validation_image "../conditioning_image_1.png" "./conditioning_image_2.png" \
    --validation_prompt "red circle with blue background" "cyan circle with brown floral background" \
    --train_batch_size=4

[sayakpaul]

That checkpoint should be well supported. I don't understand why do we need trust_remote_code.

[YanivDorGalron]

The trusted_remote_code is needed for the fill50k dataset. The one that the example uses

[sayakpaul]

I guess we could condition this by checking if we're using the fill-50k dataset otherwise it looks like a bit security-harming.

[YanivDorGalron]

We could set the default to be false but write the correct command in the readme. (which is speciific for fill50k). does that sound appropriate?

[sayakpaul]

Sounds perfect to me! Thanks so much!

[YanivDorGalron]

I have made the suggested modification.
now at every command where fill50k is used a new flag argument was added:

 --dataset_name=fusing/fill50k \ 
 --trust_remote_code \

in addition i made the needed modification for the relevant py files to include the argument
its a store_true argument therefore by default is False.

[sayakpaul]

Thank you so much!

[sayakpaul]

@YanivDorGalron can you run make style && make quality?

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[YanivDorGalron]

@YanivDorGalron can you run make style && make quality?

These commands modify files unrelated to this PR. Should I proceed with pushing those changes as well?

        modified:   examples/controlnet/train_controlnet_flux.py
        modified:   examples/research_projects/geodiff/geodiff_molecule_conformation.ipynb
        modified:   examples/research_projects/gligen/demo.ipynb

[sayakpaul]

You can only add git add examples/controlnet/train_controlnet_flux.py and commit the results

[YanivDorGalron]

done!

[YanivDorGalron]

@sayakpaul I noticed that the PR failed some tests. I'm not sure if the failures are related to my changes— is there anything I can do on my end?

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.