Skip to content

Fix Spark slice function Null type to GenericListArray casting issue #20466

@erenavsarogullari

Description

@erenavsarogullari

Describe the bug

Currently, Spark slice function accepts Null Arrays and return Null for this particular queries. DataFusion-Spark slice function also needs to return NULL when Null Array is set.
Spark Behavior (tested with latest Spark master):

> SELECT slice(NULL, 1, 2);
+-----------------+
|slice(NULL, 1, 2)|
+-----------------+
|             null|
+-----------------+

DF Behaviour:
Current:

query error
SELECT slice(NULL, 1, 2);
----
DataFusion error: Internal error: could not cast array of type Null to arrow_array::array::list_array::GenericListArray<i32>.
This issue was likely caused by a bug in DataFusion's code. Please help us to resolve this by filing a bug report in our issue tracker: https://github.com/apache/datafusion/issues

New:

query ?
SELECT slice(NULL, 1, 2);
----
NULL

To Reproduce

Explained under description section.

Expected behavior

Explained under description section.

Additional context

No response

Metadata

Metadata

Labels

bugSomething isn't working

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions