Image resizes sometimes miss and cannot be generated

[indykoning]

Bug description

We've seen this happen to a couple of customers.
Seemingly random image resizes stop working altogether and will no longer get generated by Statamic.
"Seemingly random" coincides with the image cache looking quite empty, so we expect someone has pressed the clear cache button.
These are image caches with 20+ GB of images.

(See Additional details for the probable cause)

How to reproduce

Reproducing is difficult on a clean installation.
The key is slowing the deletion of files down to the point you can request an image generation BEFORE that image is deleted from disk, but AFTER the Glide::cacheStore()->flush(); command has run.

Logs

Environment

Environment
Laravel Version: 11.47.0
PHP Version: 8.4.14
Composer Version: 2.9.5
Environment: production
Debug Mode: OFF
Maintenance Mode: OFF
Timezone: Europe/Amsterdam
Locale: nl

Cache
Config: CACHED
Events: CACHED
Routes: CACHED
Views: CACHED

Drivers
Broadcasting: log
Cache: redis
Database: mysql
Logs: daily
Mail: smtp
Queue: sync
Session: file

Statamic
Addons: 3
Sites: 1
Stache Watcher: Enabled
Static Caching: full
Version: 5.70.0 PRO

Statamic Addons
duncanmcclean/static-cache-manager: 4.0.1
statamic-rad-pack/runway: 7.14.0
statamic/eloquent-driver: 4.35.2

Statamic Eloquent Driver
Asset Containers: file
Assets: eloquent
Blueprints: file
Collection Trees: file
Collections: file
Entries: eloquent
Fieldsets: file
Form Submissions: eloquent
Forms: file
Global Sets: file
Global Variables: file
Navigation Trees: file
Navigations: file
Revisions: eloquent
Sites: file
Taxonomies: file
Terms: eloquent
Tokens: file

Installation

Existing Laravel app

Additional details

Debugging the key generated for the cache store that gets empty upon clearing the cache one of the images missing from disk is:

cms/src/Console/Commands/GlideClear.php

Line 36 in 95a6684

public function handle()

Looking at how the Glide clear command is written i suspect high traffic sites are impacted since the cache is cleared BEFORE the images are deleted.

Clearing the cache before deleting is a good thing since the images would get generated immediately if missing.
However with a large amount of images, this may take long enough that some images will get "generated" before they get deleted.

So cache claims the image already exists, even though the image has been deleted in the meantime.