Fix COUNT bug when pulling up scalar subqueries (fallback planner)#397
Draft
Alena0704 wants to merge 1 commit into
Draft
Fix COUNT bug when pulling up scalar subqueries (fallback planner)#397Alena0704 wants to merge 1 commit into
Alena0704 wants to merge 1 commit into
Conversation
Alena0704
force-pushed
the
count-aggr-first-fallback
branch
from
e640835 to
68c09fb
Compare
Alena0704
force-pushed
the
count-aggr-first-fallback
branch
2 times, most recently
from
c00def6 to
9ffda49
Compare
A correlated subquery like
select ... from t1 where t1.a > (select count(*) from t2 where t2.a = t1.d)
must return COUNT = 0 (not NULL) for outer rows that have no match. The
fallback planner pulled such subqueries into an INNER join, which dropped
every no-match row -- the classic COUNT bug.
PR #376 (df87bf0) turned the join into a LEFT join and wrapped the aggregate
in COALESCE(agg, default) with COUNT -> 0. Two problems with that alone:
* The comparison stayed on the join itself (a "Join Filter"), so a matched
row that failed the comparison was treated as unmatched, null-extended,
and let back in by the COALESCE default -- e.g. count(*) returned 100
instead of 99. Fixed here in pull_up_sublinks_qual_recurse(): when the
pulled-up join is LEFT, evaluate the comparison ABOVE the join (wrap it in
a FromExpr) rather than as the join condition.
* It switched to a LEFT join for every COUNT-containing subquery, even where
an INNER join was already correct (e.g. "1 = (select count(*) ...)", whose
no-match value 0 can never satisfy the predicate), churning many unrelated
plans. Narrowed here: only use the LEFT join + COALESCE when a no-match row
could actually survive, i.e. when "outerExpr OP default" does not fold to a
constant FALSE/NULL. Otherwise keep the INNER join unchanged.
The INNER path is otherwise untouched.
Tests in subselect_gp cover COUNT and mixed COUNT+other aggregate expressions
on both the optimizer=off and ORCA->planner fallback paths.
Co-Authored-By: excaliiibur <excaliiibur@foxmail.com>
Alena0704
force-pushed
the
count-aggr-first-fallback
branch
from
9ffda49 to
0f6cfed
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
It is the same apache/cloudberry#1836 but I couldn't continue fixing there because of lack of access to the initial branch.
A correlated subquery like
must return COUNT = 0 (not NULL) for outer rows that have no match. The fallback planner pulled such subqueries into an INNER join, which dropped every no-match row -- the classic COUNT bug.
PR #376 (df87bf0) turned the join into a LEFT join and wrapped the aggregate in COALESCE(agg, default) with COUNT -> 0. But the comparison stayed on the join itself (a "Join Filter"), so a matched row that failed the comparison was treated as unmatched, null-extended, and let back in by the COALESCE default -- e.g. count(*) returned 100 instead of 99.
Fix: when the pulled-up join is LEFT, evaluate the comparison ABOVE the join (wrap it in a FromExpr) instead of as the join condition. The INNER path is unchanged.
Tests in subselect_gp cover COUNT and mixed COUNT+other aggregate expressions on both the optimizer=off and ORCA->planner fallback paths.