Skip to content

[SPARK-56030] [SQL] Normalize inner Project when below Project -> Filter#54859

Open
mihailoale-db wants to merge 1 commit intoapache:masterfrom
mihailoale-db:normalizeproject1
Open

[SPARK-56030] [SQL] Normalize inner Project when below Project -> Filter#54859
mihailoale-db wants to merge 1 commit intoapache:masterfrom
mihailoale-db:normalizeproject1

Conversation

@mihailoale-db
Copy link
Contributor

What changes were proposed in this pull request?

In this PR I propose to normalize inner Project when below Project -> Filter.

Why are the changes needed?

Right now

SELECT a.key, b.key AS key_b
FROM t1 b
FULL JOIN t2 a USING (key)
WHERE b.key IS NULL OR a.key IS NULL

throws LOGICAL_PLAN_COMPARISON_MISMATCH for analyzer dual-runs and normalizing would fix it.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Added + existing tests.

Was this patch authored or co-authored using generative AI tooling?

Yes.

@mihailoale-db
Copy link
Contributor Author

@dtenedor PTAL when you find time. Thanks!

@mihailoale-db
Copy link
Contributor Author

@cloud-fan PTAL when you find time. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant