Skip to content

Main is broken #15896

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
xudong963 opened this issue Apr 30, 2025 · 1 comment · Fixed by #15898
Closed

Main is broken #15896

xudong963 opened this issue Apr 30, 2025 · 1 comment · Fixed by #15898
Labels
bug Something isn't working

Comments

@xudong963
Copy link
Member

xudong963 commented Apr 30, 2025

Image

Example failure
https://github.com/apache/datafusion/actions/runs/14740728702/job/41378119017

     Running bin/sqllogictests.rs (target/ci/deps/sqllogictests-f0f60dad4353b70d)
External error: query failed: DataFusion error: Optimizer rule 'optimize_projections' failed
caused by
Check optimizer-specific invariants after optimizer rule: optimize_projections
caused by
@gstvg
Copy link
Contributor

gstvg commented Apr 30, 2025

After #15149, flatten stopped working for List(FixedSizeList) because it expected the inner fixed size list to be casted to list, which wasn't happening anymore.

Looking back, #15160 correctly stopped using ArrayFunctionSignature::RecursiveArray, which I expected would make only the top level list be coerced, and it would break, but it didn't. It appears that recursive lists continued to be coerced

And then #15149 refactored that code path with the correct behavior, exposing the issue in flatten

I opened #15898 with a potential fix. Other options include:

  1. Reverting to ArrayFunctionSignature::RecursiveArray again, but for deeply recursive fixed size lists, it would unnecessarily coerce some of them(which includes building offsets buffers).
  2. Creating a new ArrayFunctionSignature variant for this case
  3. Manually handling fixed size list in flatten

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants