Use INT4 type alias for CockroachDB JDBC compatibility #2485
Conversation
Explicitly define integer columns as INT4 so the CockroachDB JDBC driver correctly maps them to Java's Integer type, despite them being stored internally as INT8. This enables CockroachDB support for the metastore. Fixes: apache#2464
There was a problem hiding this comment.
Thank you for the change @sathesuraj !
Just to double sure so that we can call Polaris CockroachDB compatible, and protect Cocroach DB integration works in future relases (any regressions is caught), my suggestion would be to add eq class for PostgresRelationalJdbcLifeCycleManagement , mostly we will have to change the container to use cocroach container instead of pg
can be done via
return new CockroachContainer(
dockerImage("cockroach").asCompatibleSubstituteFor("cockroachdb/cockroach"));
may be we can refactor the base class a lit bit too, and the wiring of it to a profile would work mostly same.
Really appreciate you taking time to testing it cocroach and adding this capability to Polaris !
| @@ -21,7 +21,7 @@ SET search_path TO POLARIS_SCHEMA; | |||
|
|
|||
| CREATE TABLE IF NOT EXISTS version ( | |||
| version_key TEXT PRIMARY KEY, | |||
| version_value INTEGER NOT NULL | |||
| version_value INT4 NOT NULL | |||
There was a problem hiding this comment.
seems like cocroach-db uses pg driver as we have checks for the display type here
https://github.com/apache/polaris/blob/main/persistence/relational-jdbc/src/main/java/org/apache/polaris/persistence/relational/jdbc/DatabaseType.java#L46
would be nice to have a comment.
There was a problem hiding this comment.
Yes, cockroach DB uses pg driver. I'll add a comment for it.
There was a problem hiding this comment.
Thanks for your contribution @sathesuraj !
@singhpk234 made a good point about testing. As far as I can see current JDBC persistence tests use only H2. It is great for light-weight unit tests, but does not provide assurance that the code + schema actually work with PostgreSQL or CockroachDB.
Now that we're starting to make non-trivial schema changes, it would be nice to make tests similar to JdbcRestCatalogIT and RelationalJdbcBootstrapCommandTest but using actual PG and CockroachDB docker images. See RelationalJdbcAdminProfile for inspiration.
Ideally, it would be nice to add those tests in this PR, but if you do not have capacity ATM, I think it should be ok to merge these schema changes "as is" and add tests later.
|
If we do not add tests in this PR, let's keep #2464 open until we have end-to-end tests. |
Explicitly defined integer columns as INT4 in SQL script so that a CockroachDB JDBC driver correctly maps them to Java's Integer type, despite them being stored internally as INT8 in CockroachDB. This enables CockroachDB support for the metastore and will continue to work for PostgreSQL.
Fixes: #2464