test(cubestore): failing test for the result corruption in #11545

[connor-wayne-mcelroy]

Failing test for #11545, as requested.

multi_stage_gated_join_with_sort_and_limit in rust/cubestore/cubestore/src/sql/mod.rs is a self-contained Cube Store test — one table, plain INSERTs, no Cube, no pre-aggregation matching, no source database. It currently fails.

What it asserts

A single rollup-shaped table holds 1100 orders in each of two months. The query is the SQL Cube's Tesseract planner emits verbatim for a multi_stage measure that gates one base measure on another (CASE WHEN {sale} > 0 THEN {ticket_fraction} END with add_group_by), with only the table name substituted: two per-key leaf aggregations over the same table, a DISTINCT key set built from their UNION ALL, a LEFT JOIN back to each leaf, and a top-level ORDER BY ... LIMIT.

Each order contributes exactly 1 to ticket_fraction, so every month must return 1100.

query	2026-02	2026-03
February alone	1100 ✅	—
March alone	—	1100 ✅
February–March	1100 ✅	1401 ❌

Same table, same query text, same physical rows — only the two date literals differ. 1401 is larger than the number of distinct keys that feed the sum, which this query's algebra cannot produce: the key set is SELECT DISTINCT, each leaf is pre-grouped to one row per (order, month), and the joins are on the full composite key.

What narrows it down

While reducing this from a production model, three things turned out to be load-bearing:

• The top-level ORDER BY and LIMIT together. Removing either one — the ORDER BY, or the LIMIT — makes the same query return correct values. That points at the sort/limit pushdown into ClusterSend (pull_up_cluster_send's LogicalPlan::Sort branch and the worker_sort_and_limit descriptor) rather than at the join. LIMIT 10000 is far above the 2-row result, so it is the pushdown, not the limit value.
• Crossing a record batch. At ≤ 2048 distinct (order, month) keys the query is always correct; at 2049+ it corrupts. The test sits just over that line at 2200 keys. This is why the bug is invisible on short date ranges and why which months are wrong shifts with the width of the range asked for.
• Scan order disagreeing with the leaves' ORDER BY. Order ids interleave across the two months, so index order (by id) and the leaves' ORDER BY month disagree, as they do for real order ids. With ids assigned month-by-month (so the two orders agree) the same query returns correct values.

Row counts are right throughout — SELECT count(*) over the key set and over each join stage all return 2200. Only the summed values are wrong, so nothing duplicates rows; the join pairs keys with values that do not belong to them.

One note on the test harness

The test runs its runtime on an explicitly sized thread. Planning this query shape recurses deeply enough to overflow libtest's default 2 MiB stack in a debug build — it aborts the whole test binary with fatal runtime error: stack overflow before reaching any assertion, which is why the explicit stack is there. That looks unrelated to the corruption (release frames are much smaller, and production gives select workers 4 MiB via CUBESTORE_SELECT_WORKER_STACK_SIZE), but it may be the same deep-recursion path as the "recursion limit reached" crash, so flagging it rather than burying it.

Current output:

assertion failed: `(left == right)`

Diff < left / right > :
 [
     1100,
<    1401,
>    1100,
 ]

Versions

Reproduces identically on cubejs/cubestore:v1.7.4, v1.7.19 and latest, single-node and router+workers, on freshly built tables. Decimal measures behave the same as the ints used here; the test uses ints to keep the assertions readable.

Signed-off-by: Connor McElroy connor.mcelroy@xyzz.dev