|
19 | 19 |
|
20 | 20 | # Upgrade Guides
|
21 | 21 |
|
| 22 | +## DataFusion `47.0.0` |
| 23 | + |
| 24 | +### Specifying nullable arguments for coercible signatures in UDFs |
| 25 | + |
| 26 | +DataFusion 47 changes how UDFs handle nullable arguments in [`TypeSignatureClass`] (primarily used in coercible signatures). |
| 27 | + |
| 28 | +Previously, DataFusion treated [`NativeType::Null`] as a special case when matching native types to a signature class. |
| 29 | +However, this led to inconsistent behavior and internal errors — when the signature class was not `TypeSignatureClass::Native`, passing `NULL` to a UDF would result in an error such as: |
| 30 | + |
| 31 | +```text |
| 32 | +May miss the matching logic in `matches_native_type` |
| 33 | +``` |
| 34 | + |
| 35 | +If your function uses a coercible signature ([`Signature::Coercible`]) and should accept `NULL` as a valid input, you need to convert it to use [`Coercion::Implicit`] (if it doesn't already) and explicitly include `TypeSignatureClass::Native(logical_null())` in the `allowed_source_types`. |
| 36 | + |
| 37 | +--- |
| 38 | + |
| 39 | +#### Example: Updating an exact coercible signature |
| 40 | + |
| 41 | +Before: |
| 42 | + |
| 43 | +```rust |
| 44 | +pub fn new() -> Self { |
| 45 | + Self { |
| 46 | + signature: Signature::coercible( |
| 47 | + vec![Coercion::new_exact(TypeSignatureClass::Native( |
| 48 | + logical_string(), |
| 49 | + ))], |
| 50 | + Volatility::Immutable, |
| 51 | + ), |
| 52 | + } |
| 53 | +} |
| 54 | +``` |
| 55 | + |
| 56 | +After: |
| 57 | + |
| 58 | +```rust |
| 59 | +use datafusion_common::types::{logical_null, NativeType}; |
| 60 | + |
| 61 | +pub fn new() -> Self { |
| 62 | + Self { |
| 63 | + signature: Signature::coercible( |
| 64 | + vec![Coercion::new_implicit( |
| 65 | + TypeSignatureClass::Native(logical_string()), |
| 66 | + vec![TypeSignatureClass::Native(logical_null())], |
| 67 | + NativeType::String, |
| 68 | + )], |
| 69 | + Volatility::Immutable, |
| 70 | + ), |
| 71 | + } |
| 72 | +} |
| 73 | +``` |
| 74 | + |
| 75 | +--- |
| 76 | + |
| 77 | +#### Example: Updating an existing implicit coercion |
| 78 | + |
| 79 | +Even if you're already using `Coercion::Implicit`, you still need to explicitly allow `NULL` values. |
| 80 | +Without this, passing `NULL` will fail to match the signature and result in an internal error. |
| 81 | + |
| 82 | +```diff |
| 83 | +TypeSignature::Coercible(vec![Coercion::new_implicit( |
| 84 | + TypeSignatureClass::Native(logical_binary()), |
| 85 | +- vec![TypeSignatureClass::Native(logical_string())], |
| 86 | ++ vec![ |
| 87 | ++ TypeSignatureClass::Native(logical_string()), |
| 88 | ++ TypeSignatureClass::Native(logical_null()), |
| 89 | ++ ], |
| 90 | + NativeType::String, |
| 91 | +)]) |
| 92 | +``` |
| 93 | + |
| 94 | +You can view [PR #15404] for more examples and implementation details. |
| 95 | + |
| 96 | +[`TypeSignatureClass`]: https://docs.rs/datafusion-expr/latest/datafusion_expr/enum.TypeSignatureClass.html |
| 97 | +[`NativeType::Null`]: https://docs.rs/datafusion-common/latest/datafusion_common/types/enum.NativeType.html#variant.Null |
| 98 | +[`Signature::Coercible`]: https://docs.rs/datafusion-expr/latest/datafusion_expr/enum.TypeSignature.html#variant.Coercible |
| 99 | +[`Coercion::Implicit`]: https://docs.rs/datafusion-expr/latest/datafusion_expr/enum.Coercion.html#variant.Implicit |
| 100 | +[`PR #15404`]: https://github.com/apache/datafusion/pull/15404 |
| 101 | + |
22 | 102 | ## DataFusion `46.0.0`
|
23 | 103 |
|
24 | 104 | ### Use `invoke_with_args` instead of `invoke()` and `invoke_batch()`
|
|
0 commit comments