[fix](cloud) Skip empty rowsets before accessor lookup in batch delete_rowset_data#60919
Open
liaoxin01 wants to merge 1 commit intoapache:masterfrom
Open
[fix](cloud) Skip empty rowsets before accessor lookup in batch delete_rowset_data#60919liaoxin01 wants to merge 1 commit intoapache:masterfrom
liaoxin01 wants to merge 1 commit intoapache:masterfrom
Conversation
…e_rowset_data
Empty rowsets produced by base compaction of empty rowsets have no
resource_id set (rowset_meta_size=181). When batch delete_rowset_data
encounters these rowsets, accessor_map_.find("") fails and sets ret=-1,
which causes the caller lambda to skip txn_remove for the entire batch.
This prevents recycle KV keys from being cleaned up, creating a
perpetual loop where the same rowsets are scanned every recycle round.
Move the num_segments <= 0 check before the accessor_map_ lookup so
these empty rowsets are safely skipped without poisoning the batch
return value.
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
There was a problem hiding this comment.
Pull request overview
Fixes batch delete_rowset_data behavior in the cloud recycler by skipping empty rowsets (e.g., base compaction outputs with num_segments=0 and unset resource_id) before doing an accessor_map_ lookup, preventing a spurious batch failure that can block recycle KV cleanup and cause repeated rescans.
Changes:
- Move the
num_segments <= 0early-continue check ahead ofaccessor_map_.find(rs.resource_id())in batch rowset deletion. - Add explanatory comments describing the failure mode and why the ordering matters.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Contributor
Author
|
run buildall |
Cloud UT Coverage ReportIncrement line coverage Increment coverage report
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposed changes
In batch
delete_rowset_data, empty rowsets (e.g. base compaction output of empty rowsets) havenum_segments=0and noresource_idset. Theaccessor_map_.find("")fails and setsret=-1, which causes the caller to skiptxn_removefor the entire batch. This prevents recycle KV keys from being cleaned up, creating a perpetual loop where the same rowsets are scanned every recycle round.The fix moves the
num_segments <= 0check before theaccessor_map_lookup so these empty rowsets are safely skipped without poisoning the batch return value.Problem summary
resource_id="",rowset_meta_size=181,num_segments=0accessor_map_.find("")fails, setsret = -1,txn_removeskipped for entire batch