Skip to content

[fix](cloud) Skip empty rowsets before accessor lookup in batch delete_rowset_data#60919

Open
liaoxin01 wants to merge 1 commit intoapache:masterfrom
liaoxin01:fix/recycler-empty-resource-id-rowset
Open

[fix](cloud) Skip empty rowsets before accessor lookup in batch delete_rowset_data#60919
liaoxin01 wants to merge 1 commit intoapache:masterfrom
liaoxin01:fix/recycler-empty-resource-id-rowset

Conversation

@liaoxin01
Copy link
Contributor

Proposed changes

In batch delete_rowset_data, empty rowsets (e.g. base compaction output of empty rowsets) have num_segments=0 and no resource_id set. The accessor_map_.find("") fails and sets ret=-1, which causes the caller to skip txn_remove for the entire batch. This prevents recycle KV keys from being cleaned up, creating a perpetual loop where the same rowsets are scanned every recycle round.

The fix moves the num_segments <= 0 check before the accessor_map_ lookup so these empty rowsets are safely skipped without poisoning the batch return value.

Problem summary

  • Empty rowsets from base compaction have resource_id="", rowset_meta_size=181, num_segments=0
  • accessor_map_.find("") fails, sets ret = -1, txn_remove skipped for entire batch
  • Normal rowsets in the same batch: object storage data already deleted, but recycle KV not cleaned up
  • Next recycle round re-scans the same rowsets, forming a dead loop

…e_rowset_data

Empty rowsets produced by base compaction of empty rowsets have no
resource_id set (rowset_meta_size=181). When batch delete_rowset_data
encounters these rowsets, accessor_map_.find("") fails and sets ret=-1,
which causes the caller lambda to skip txn_remove for the entire batch.
This prevents recycle KV keys from being cleaned up, creating a
perpetual loop where the same rowsets are scanned every recycle round.

Move the num_segments <= 0 check before the accessor_map_ lookup so
these empty rowsets are safely skipped without poisoning the batch
return value.
Copilot AI review requested due to automatic review settings February 28, 2026 14:27
@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes batch delete_rowset_data behavior in the cloud recycler by skipping empty rowsets (e.g., base compaction outputs with num_segments=0 and unset resource_id) before doing an accessor_map_ lookup, preventing a spurious batch failure that can block recycle KV cleanup and cause repeated rescans.

Changes:

  • Move the num_segments <= 0 early-continue check ahead of accessor_map_.find(rs.resource_id()) in batch rowset deletion.
  • Add explanatory comments describing the failure mode and why the ordering matters.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@liaoxin01
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

Cloud UT Coverage Report

Increment line coverage 33.33% (2/6) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 79.35% (1798/2266)
Line Coverage 64.75% (32177/49697)
Region Coverage 65.60% (16096/24537)
Branch Coverage 56.14% (8579/15282)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants