s3cluster joins, part 1 #972

ianton-ru · 2025-08-22T16:27:35Z

Changelog category (leave one):

New Feature

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

JOIN with *Cluster table functions

Documentation entry for user-facing changes

Solved #819

Queries like

SELECT a.c1, b.c2
  FROM
    s3Cluster(...) AS a
  JOIN
    local_table AS b
  ON
    a.key=b.key

execute JOIN's on remote nodes.
This behavior is with default value for new setting object_storage_cluster_join_mode='allow'.

With this PR with setting object_storage_cluster_join_mode='local' query rewrited on

SELECT a.c1, a.key
  FROM
    s3Cluster(...) AS a

and JOIN's executed on initiator.

'global' mode (select from right table and send result to swarm nodes to make join on swarm nodes) somewhere later in separate PR.

Exclude tests:

ianton-ru · 2025-08-22T16:29:14Z

Don't forget to change branch to merge on antalya-25.6.5 after #949 merged there.

…/antalya-25.6.5/s3cluster_global_join

filimonov · 2025-08-27T08:33:12Z

src/Storages/IStorageCluster.cpp

+    auto table_function_node = table_function_searcher.getNode();
+
+    if (!table_function_node)
+        throw Exception(ErrorCodes::LOGICAL_ERROR, "Can't fiund table function node");


Suggested change

throw Exception(ErrorCodes::LOGICAL_ERROR, "Can't fiund table function node");

throw Exception(ErrorCodes::LOGICAL_ERROR, "Can't find table function node");

filimonov · 2025-09-08T09:02:53Z

src/Core/Settings.cpp

+
+- `local` — Replaces the database and table in the subquery with local ones for the destination server (shard), leaving the normal `IN`/`JOIN.`
+- `global` — Unsupported for now. Replaces the `IN`/`JOIN` query with `GLOBAL IN`/`GLOBAL JOIN.`
+- `allow` — Default value. Allows the use of these types of subqueries.


Suggested change

- `allow` — Default value. Allows the use of these types of subqueries.

- `allow` — Default value. Allows the use of these types of subqueries. The join will be executed on the initiator.

should we also support 'forbid' ?

BTW - what will happen when joining xxxCluster with non-distributed table? Maybe a test?

No, it's a default old behavour - query executed on remote nodes "as is" and join executed there.
I guess this is optimal for casual, non-swarm mode, when right table is present on all cluster nodes.

filimonov · 2025-09-08T12:36:19Z

src/Storages/IStorageCluster.cpp

+    using Base = InDepthQueryTreeVisitorWithContext<SearcherVisitor>;
+    using Base::Base;
+
+    explicit SearcherVisitor(QueryTreeNodeType type_, ContextPtr context) : Base(context), type(type_) {}


it looks like the code is only for analyzer (i.e. build on the top of QueryTree, not on the top of AST). It's ok for new features, but maybe it's better to document it clearly. I was also thinking about adding throw when analizer is disabled. WDYT?

Full support for non-anylyzer code can be more challenging (see for example JoinToSubqueryTransformVisitor)

Added exception

filimonov · 2025-09-08T12:37:57Z

src/Storages/IStorageCluster.cpp

+            passed_node = node;
+    }
+
+    QueryTreeNodePtr getNode() const { return passed_node; }


Actually i never looked close on the anazyzer code, thought it should be simpler (i.e. avoiding tree traversal, but i can be wrong).

filimonov · 2025-09-08T12:59:21Z

src/Storages/IStorageCluster.cpp

+            query_tree_distributed->setAlias(table_function_ast.alias);
+
+            // Find add used columns from table function to make proper projection list
+            CollectUsedColumnsForSourceVisitor collector(table_function_node, context);


Maybe things like tryResolveIdentifierFromJoinTreeNode / tryResolveIdentifierFromJoinTree / tryResolveIdentifierFromTableExpression can be reused.

Enmk · 2025-09-09T13:21:18Z

src/Core/Settings.cpp

+Possible values:
+
+- `local` — Replaces the database and table in the subquery with local ones for the destination server (shard), leaving the normal `IN`/`JOIN.`
+- `global` — Unsupported for now. Replaces the `IN`/`JOIN` query with `GLOBAL IN`/`GLOBAL JOIN.`


if it is unsupported, then why was it added? I suggest ditching that value from enum

It's for part 2, when right table is calculated on initiator and is sent as part of the query to the swarm nodes.

Enmk · 2025-09-09T13:21:54Z

src/Core/Settings.cpp

@@ -1713,6 +1713,22 @@ Possible values:
 - `global` — Replaces the `IN`/`JOIN` query with `GLOBAL IN`/`GLOBAL JOIN.`
 - `allow` — Allows the use of these types of subqueries.
 )", IMPORTANT) \
+    DECLARE(ObjectStorageClusterJoinMode, object_storage_cluster_join_mode, ObjectStorageClusterJoinMode::ALLOW, R"(
+Changes the behaviour of object storage cluster function ot table.


Suggested change

Changes the behaviour of object storage cluster function ot table.

Changes the behaviour of object storage cluster function of table.

?

…uster_join_mode

github-actions · 2025-09-09T18:05:55Z

Workflow [PR], commit [c0468e7]

…lobal_join

Respect prefer_global_in_and_join setting with *Cluster functions

a8041af

ianton-ru added antalya antalya-25.6 antalya-25.6.5 labels Aug 22, 2025

ianton-ru and others added 3 commits August 22, 2025 20:13

Merge branch 'feature/antalya-25.6.5/alternative_syntax' into feature…

dc71790

…/antalya-25.6.5/s3cluster_global_join

Fix WHERE

628b035

Setting

77b4f1f

ianton-ru changed the title ~~WIP s3cluster global join~~ s3cluster joins, part 1 Sep 1, 2025

filimonov approved these changes Sep 8, 2025

View reviewed changes

ianton-ru changed the base branch from feature/antalya-25.6.5/alternative_syntax to antalya-25.6.5 September 8, 2025 22:21

Enmk reviewed Sep 9, 2025

View reviewed changes

Throw exception on allow_experimental_analyzer=0 && object_storage_cl…

70f47ed

…uster_join_mode

ianton-ru and others added 2 commits September 11, 2025 12:25

Merge antalya-25.6.5

6cc62ff

Merge branch 'antalya-25.6.5' into feature/antalya-25.6.5/s3cluster_g…

c0468e7

…lobal_join

Enmk merged commit 53d1f5e into antalya-25.6.5 Sep 15, 2025
308 of 324 checks passed

	throw Exception(ErrorCodes::LOGICAL_ERROR, "Can't fiund table function node");
	throw Exception(ErrorCodes::LOGICAL_ERROR, "Can't find table function node");

	- `allow` — Default value. Allows the use of these types of subqueries.
	- `allow` — Default value. Allows the use of these types of subqueries. The join will be executed on the initiator.

	Changes the behaviour of object storage cluster function ot table.
	Changes the behaviour of object storage cluster function of table.

s3cluster joins, part 1 #972

s3cluster joins, part 1 #972

Uh oh!

Conversation

ianton-ru commented Aug 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changelog category (leave one):

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Documentation entry for user-facing changes

Exclude tests:

Uh oh!

ianton-ru commented Aug 22, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Sep 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ianton-ru commented Aug 22, 2025 •

edited

Loading

github-actions bot commented Sep 9, 2025 •

edited

Loading