Skip to content

[Bug] AppendTableITCase.testCompactionInStreamingMode is flaky and can time out under CI load #10183

Description

@LuciferYang

Search before asking

  • I searched in the issues and found nothing similar.

Paimon version

master (645ce6f)

Compute Engine

Flink

Minimal reproduce step

AppendTableITCase.testCompactionInStreamingMode, and its sibling testCompactionInStreamingModeWithMaxWatermark, run a streaming INSERT from a datagen source at rows-per-second = 1 into an append table configured with compaction.min.file-num = 4, continuous.discovery-interval = 1 s, and a 500 ms checkpoint interval. They then poll once a second, for up to 60 s, for a snapshot whose commitKind is COMPACT.

In CI the poll sometimes reaches the 60 s limit and the test fails:

java.lang.RuntimeException: Time up for streaming execute, don't get expected result.

One example: https://github.com/apache/paimon/actions/runs/36149147465/job/108117843839

What doesn't meet your expectations?

The test should not time out. Nothing is broken in the append-compaction pipeline; the failure is a timing race inside the test.

On a healthy run the first COMPACT snapshot arrives in under 20 s: the streaming job starts, four small files accumulate at about one per second, the coordinator picks them up on its next discovery cycle, compaction runs, and the compacted snapshot commits. The 60 s budget is usually comfortable. Under CI load, streaming job deployment and checkpoint/commit progress can stall, and the first COMPACT snapshot occasionally shows up after 60 s.

The same test was reported in #1634 and closed without a fix.

Anything else?

#1634 suggested lowering compaction.min.file-num as a workaround. That only shortens the file-accumulation phase (roughly 4 s at one file per second), which is not where the time goes when the test times out. The stall is in job deployment and checkpoint progress, so giving the wait a larger timeout is the direct fix.

Are you willing to submit a PR?

  • I'm willing to submit a PR!

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions