-
Notifications
You must be signed in to change notification settings - Fork 13.7k
[FLINK-37914][table] Add built-in OBJECT_UPDATE function #26806
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
- sql: OBJECT_UPDATE(object, key, value [, key, value , ...]) | ||
table: OBJECT.objectUpdate(key, value [, key, value , ...]) | ||
description: | | ||
Updates existing fields in a structured object by providing key-value pairs. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am curious:
- I assume that arrays are not included in this
- are there restrictions on the values, i.e. can they be maps, lists, objects or arrays. If there are restrictions , we should document them. If there are no restrictions we should include examples of more complex objects.
- in the example I assume "com.example.User" is a "path".
- could dots in the field name clash with the path?
- could you give an example or pointer to documentaiton around how to construct these paths in the documentation - examples around nested objects would be useful - including an array of objects where you want to update the value of the 3rd object.
- can we set null values.
- if the field is defined as not nullable - do we error if an attempt to put a null there occurs.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- Arrays should be supported. In the background, the
OBJECT_UPDATE
uses a structured type, and this test evaluates its functionality. - There should be no restrictions on the values. We only check if the type of the updated value matches the one in the structured type.
- For now, we decided to skip any validation on the field name format. The user is allowed to write
OBJECT_UPDATE(obj, "my.field.name", 14)
. This should be up to the user to pass valid field names. For the first iteration, this validation was not necessary. - If I understand you correctly, you cannot update an
ARRAY
type withOBJECT_UPDATE
. TheOBJECT_UPDATE
applies toSTRUCTURED_TYPE
. - Yes, values can be null
- Yes, validation errors will be thrown if the field name is null:
- Here is the validation check, where we see if the field is a non-null String literal.
- Tests backing this logic.
56a3f1e
to
e437f51
Compare
* | ||
* <pre>{@code | ||
* // Create a structured object representing a user | ||
* User userObject = objectOf("com.example.User", "name", "Bob", "age", 25); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These examples are a bit misleading. They read as if User userObject = objectOf
is Java code and the function returns a class instance User
. The explanation in docs is better.
.../flink-table-api-java/src/main/java/org/apache/flink/table/api/internal/BaseExpressions.java
Outdated
Show resolved
Hide resolved
...in/java/org/apache/flink/table/types/inference/strategies/ObjectUpdateInputTypeStrategy.java
Outdated
Show resolved
Hide resolved
...in/java/org/apache/flink/table/types/inference/strategies/ObjectUpdateInputTypeStrategy.java
Outdated
Show resolved
Hide resolved
...rc/main/java/org/apache/flink/table/types/inference/strategies/ObjectUpdateTypeStrategy.java
Outdated
Show resolved
Hide resolved
...rc/main/java/org/apache/flink/table/types/inference/strategies/ObjectUpdateTypeStrategy.java
Outdated
Show resolved
Hide resolved
...lanner/src/test/java/org/apache/flink/table/planner/functions/StructuredFunctionsITCase.java
Show resolved
Hide resolved
a40c147
to
5a1cb17
Compare
What is the purpose of the change
This pull request implements the
OBJECT_UPDATE
built-in function as part of FLIP-520: Simplify StructuredType handling. The function allows users to update existing fields in structured objects by providing key-value pairs, enabling mutation operations on structured types in both SQL and Table API without requiring custom UDFs.Brief change log
ObjectUpdateInputTypeStrategy
for validating input arguments (structured object + key-value pairs)ObjectUpdateTypeStrategy
for inferring return types (same as input structured type)ObjectUpdateFunction
runtime function for performing field updatesOBJECT_UPDATE
toBuiltInFunctionDefinitions
with proper type inference strategiesobjectUpdate()
method on expressionsVerifying this change
This change added tests and can be verified as follows:
ObjectUpdateInputTypeStrategyTest
for input validation scenariosStructuredFunctionsITCase
for end-to-end SQL functionalityDoes this pull request potentially affect one of the following parts:
@Public(Evolving)
: yes (new built-in function)Documentation