Skip to content

Introduce fair parallelism for trace part merge to prevent long-merge starvation #13776

@hanahmily

Description

@hanahmily

Problem

Long-running trace merges can delay shorter merge tasks and degrade compaction responsiveness.

Current context

  • Trace merge loop currently uses global concurrency gating via mergeMaxConcurrencyCh in banyand/trace/merger.go.
  • Merge and sidx merge latency are tracked but scheduling is not fairness-aware.

Proposal

  • Add fairness-oriented merge scheduling (e.g., small/large queue classes or age-based prioritization).
  • Ensure short merges are not indefinitely blocked by large merge jobs.
  • Add observability for queue wait time and merge scheduling decisions.

Acceptance criteria

  • Merge scheduling policy is deterministic and test-covered.
  • Small merges complete within bounded delay under mixed workloads.
  • Metrics/logs expose queue time, execution time, and scheduling class.

Metadata

Metadata

Assignees

No one assigned

    Labels

    databaseBanyanDB - SkyWalking native database

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions