[core] respect local_files_only=True when using sharded checkpoints

[sayakpaul]

What does this PR do?

See: #11948

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[DN6]

So the purpose of this check is to verify if the necessary sharded files are present in the model repo before attempting a download, presumably to avoid a large download if all files aren't present. If we cannot connect to the hub, we just have to assume the necessary shard files are already present locally.

I think we can just skip this check if local_files_only=True and then check if all the shard filenames are present in the cached_folder

[sayakpaul]

How about now?

[DN6]

I think just this is sufficient

if not local_files_only:
    # run model_info check

Run snapshot download

Then after the cached_filenames is created, iterate over the files to verify they exist

for filename in cached_filename:
      if not if not os.path.exists(filename):
           raise EnvironmentError("expected file not present in {cached_folder}")

[sayakpaul]

1. We don't have to run snapshot_download() when local_files_only=False, that might be unnecessary.
2. Why run snapshot_download() after also running model_info()?
3. Even if we run snapshot_download() regardless of local_files_only var, I think we should have it inside try-except in case the endpoint cannot be pinged for some reason and raise the ConnectionError as before.

[sayakpaul]

See if b7af511 resolves this.

[sayakpaul]

Ah I see what you mean. Let me update. Sorry about the back and forth.

[sayakpaul]

@DN6 see if the latest changes work for you.