push down filter: extend a projection if some pushed filters are unsupported #14
+129
−24
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
askalt
force-pushed
the
askalt/fix-projection-after-filter-pushdown
branch
from
3e58c46 to
0136e6d
Compare
novartole
reviewed
Aug 14, 2025
novartole
reviewed
Aug 14, 2025
askalt
force-pushed
the
askalt/fix-projection-after-filter-pushdown
branch
from
0136e6d to
d8867f9
Compare
novartole
reviewed
Aug 14, 2025
…nsupported Consider the next scenario: 1. `supports_filters_pushdown` returns `Exact` on some filter, e.g. "a = 1", where column "a" is not required by the query projection. 2. "a" is removed from the table provider projection by "optimize projection" rule. 3. `supports_filters_pushdown` changes a decision and returns `Inexact` on this filter the next time. For example, input filters were changed and it prefers to use a new one. 4. "a" is not returned to the table provider projection which leads to filter that references a column which is not a part of the schema. This patch fixes this issue introducing the next logic within a filter push-down rule: 1. Collect columns that are not used in the current table provider projection, but required for filter expressions. Call it `additional_projection`. 2. If `additional_projection` is empty -- leave all as it was before. 3. Otherwise extend a table provider projection and wrap a plan with an additional projection node to preserve schema used prior to this rule.
askalt
force-pushed
the
askalt/fix-projection-after-filter-pushdown
branch
from
d8867f9 to
b7a992b
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Consider the next scenario:
supports_filters_pushdownreturnsExacton some filter, e.g. "a = 1", where column "a" is not required by the query projection."a" is removed from the table provider projection by "optimize projection" rule.
supports_filters_pushdownchanges a decision and returnsInexacton this filter the next time. For example, input filters were changed and it prefers to use a new one."a" is not returned to the table provider projection which leads to filter that references a column which is not a part of the schema.
This patch fixes this issue introducing the next logic within a filter push-down rule:
Collect columns that are not used in the current table provider projection, but required for filter expressions. Call it
additional_projection.If
additional_projectionis empty -- leave all as it was before.Otherwise extend a table provider projection and wrap a plan with an additional projection node to preserve schema used prior to this rule.