Getting "Delta protocol violation: Generic action error: filed to get tombstones" when vacuuming table

[rtyler]

Converting this to an issue because I actually have a use-case in production where we have seen this.

Discussed in #3200

^{Originally posted by AnskeVan February 10, 2025}
Hi, using deltalake 0.22.3 (and 0.21.0 before that), I'm running a daily job that merges some data to a table, after which I z-order (table.optimize.z_order(attributes)) the table and then vacuum (table.vacuum(dry_run=False) it.
This mostly works fine, but a few times a month, it throws the exception "Delta protocol violation: Generic action error: filed to get tombstones" (the typo not made by me, I assume they mean to throw with message 'failed to get tombstones').
delta.logRetentionDuration in the table configuration is 7 days, and delta.autoOptimize.autoCompact is true. No other table configuration is returned in metadata.
Error seems to be thrown here:

delta-rs/crates/core/src/protocol/checkpoints.rs

Line 182 in 4ef9fb3

.map_err(|_| ProtocolError::Generic("filed to get tombstones".into()))?

, seems it's trying to create a new checkpoint during the vacuuming, not sure why....
I have no clue why this happens or how to prevent it, does anyone have any suggestions?