Repository navigation
Conversation
Timm0
marked this pull request as ready for review
November 9, 2026 16:41
Collaborator
AHeise
reviewed
Nov 11, 2026
AHeise
left a comment
Contributor
There was a problem hiding this comment.
Nice to see SQL and the Table API converge on one factory. I diffed the moved create/alter/convert logic against master and it matches, apart from the validation order (inline). Clean split into refactor + API commits.
Main ask: there's no test for redeploying over an existing materialized table. Every materializeInto(...).execute() in MaterializedTableTableApiITCase is a first create, and every ALTER is SQL. I'd add at least:
- SQL-created MT (with
WITH/DISTRIBUTED BY/PARTITIONED BY) →materializeIntoredeploy, asserting options, distribution, partition keys and freshness afterwards - a redeploy with a changed but compatible query (exercises
hasSchemaDefinition=falseagainst a stored schema) - a rejected column change leaving the table SUSPENDED, as documented
Also, a couple of the swapped ITCases were the only plain-TableEnvironment SQL CREATE coverage (e.g. without FRESHNESS); I'd keep one SQL variant next to the new Table API ones.
Timm0
force-pushed
the
FLINK-41214
branch
2 times, most recently
from
November 12, 2026 11:19
fbd043a to
dfa6629
Compare
- Move the create/alter/convert logic of `AbstractCreateMaterializedTableConverter` and `SqlCreateOrAlterMaterializedTableConverter` into `MaterializedTableOperationFactory` - Add `MaterializedTableMergeContext` as the factory input, implemented for SQL by `SqlMaterializedTableMergeContext` - Move the schema helpers of the planner `MaterializedTableUtils` to `MaterializedTableSchemaUtils`, and `ValidateAndExtractColumnChangesTest` with them
- Add `Table#materializeInto(String)` and `materializeInto(String, Duration)` returning a `MaterializedPipeline` - `MaterializedPipeline#execute()` returns a `MaterializedTable` handle - Build the operation through `MaterializedTableOperationFactory` with `QueryMaterializedTableMergeContext` - Reject a regular or temporary table at the path, an invalid freshness, and queries not expressible in SQL - Add `MaterializedPipelineImplTest` - Switch ITCase CREATEs to `materializeInto` - Add the Table API page of the materialized table docs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What is the purpose of the change
Table API programs can't declare a materialized table, they have to fall back to SQL
CREATE MATERIALIZED TABLE. This addsTable#materializeInto, whoseMaterializedPipeline#execute()has the semantics ofCREATE OR ALTER MATERIALIZED TABLE ... AS <query>. The declaration logic moves from the planner converters into a shared api-java factory, so SQL and the Table API produce the same operations.Brief change log
Table#materializeInto(String)andmaterializeInto(String, Duration),MaterializedPipelineandMaterializedTable(@PublicEvolving).MaterializedTableOperationFactorywith the create/alter/convert logic ofAbstractCreateMaterializedTableConverterandSqlCreateOrAlterMaterializedTableConverter. The SQL converter now feeds it throughSqlMaterializedTableMergeContext, the Table API throughQueryMaterializedTableMergeContext.MaterializedTableUtilstoMaterializedTableSchemaUtils.MaterializedPipelineImplTestfor validation rules, ITCase CREATEs switched tomaterializeInto,ValidateAndExtractColumnChangesTestmoved toapi-java.docs/content/docs/sql/materialized-table/table-api.md, with an zh copy.Verifying this change
This change added tests and can be verified as follows:
MaterializedPipelineImplTest, which rejects a regular or temporary table at the path, an invalid freshness , and queries that can't be stored as SQLMaterializedTableTableApiITCaseCREATE statements tomaterializeIntowherever the API can express themValidateAndExtractColumnChangesTestfrom the planner to api-javaSqlNodeToOperationConvertTableToMaterializedTableTest, and the gatewayMaterializedTableStatementITCase,MaterializedTableConversionITCase,SqlGatewayRestEndpointMaterializedTableITCaseandDeployScriptITCaseDoes this pull request potentially affect one of the following parts:
@Public(Evolving): yesDocumentation
Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code Opus 5.5