What would you like to happen?
Motivation
The Java ClickHouseIO connector at sdks/java/io/clickhouse/ currently supports only writes. Reading from ClickHouse in a Beam pipeline requires falling back to JdbcIO with the ClickHouse JDBC driver, which loses ClickHouse-specific schema fidelity (LowCardinality, Array(T), Nullable(T), Nested, tuples) and native columnar transport.
Proposal
Add ClickHouseIO.read() and ClickHouseIO.readAll() PTransforms symmetric to the existing write path:
- Native
Row output using the ClickHouse client's native format
- Query/projection via user-supplied
SELECT template
- Splittable reader for parallel reads across partitions / shards,
modeled on JdbcIO's splitting pattern
- Reuse the existing connection configuration builder from the writer
Benefits
- Closes an obvious read/write asymmetry — every other major DB IO in
Beam (JDBC, Cassandra, Mongo, HBase, Spanner, BigQuery, Snowflake)
supports both directions
- ClickHouse is increasingly common as an analytics source; users
currently work around this with JdbcIO at the cost of type fidelity
Testing plan
- Unit tests mirroring the existing writer test file
- Integration test using the module's existing testcontainers-based
ClickHouse setup (ClickHouseContainer is already wired in build.gradle)
- No new dependencies required
Issue Priority
Priority: 2 (default / most feature requests should be filed as P2)
Issue Components
What would you like to happen?
Motivation
The Java
ClickHouseIOconnector atsdks/java/io/clickhouse/currently supports only writes. Reading from ClickHouse in a Beam pipeline requires falling back toJdbcIOwith the ClickHouse JDBC driver, which loses ClickHouse-specific schema fidelity (LowCardinality,Array(T),Nullable(T),Nested, tuples) and native columnar transport.Proposal
Add
ClickHouseIO.read()andClickHouseIO.readAll()PTransforms symmetric to the existing write path:Rowoutput using the ClickHouse client's native formatSELECTtemplatemodeled on
JdbcIO's splitting patternBenefits
Beam (JDBC, Cassandra, Mongo, HBase, Spanner, BigQuery, Snowflake)
supports both directions
currently work around this with
JdbcIOat the cost of type fidelityTesting plan
ClickHouse setup (
ClickHouseContaineris already wired inbuild.gradle)Issue Priority
Priority: 2 (default / most feature requests should be filed as P2)
Issue Components