Skip to content

feat: [PSQL-367] add Apache Iceberg FDW #462

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 18 commits into from
May 20, 2025
Merged

feat: [PSQL-367] add Apache Iceberg FDW #462

merged 18 commits into from
May 20, 2025

Conversation

burmecia
Copy link
Member

@burmecia burmecia commented May 18, 2025

What kind of change does this PR introduce?

This PR is to add Apache Iceberg FDW.

What is the current behavior?

There is no Apache Iceberg FDW.

What is the new behavior?

Apache Iceberg FDW is added, for now it supports:

  • AWS S3 tables
  • REST Catalog + S3 storage

Additional context

  • There are some defects as below needs to be fixed in upstream iceberg-rust, so for now we just use a patched version of iceberg-rust
  • iceberg-catalog-s3tables crate is not officially released yet in upstream, so we use it directly from upstream source code
  • Because of the new dependency added, some code refactoring is also added:
    • Rust version upgraded to 1.85.1
    • S3 FDW removed code for mocking
    • rustls upgraded to 0.23.27 which requires to specify cryptographic primitives provider at start, so we added a function setup_rustls_default_crypto_provider to do that and for now only BigQuery FDW needs it
    • created a sub-workspace for Wasm wrappers and unified Rust version and edition spec
    • replaced local-stack with minio container, and shared it in S3 and Iceberg FDW testing
    • other minor CI adjustments to adopt all the above changes

@burmecia burmecia marked this pull request as ready for review May 20, 2025 08:29
@burmecia burmecia changed the title feat: add Apache Iceberg FDW feat: [PSQL-367] add Apache Iceberg FDW May 20, 2025
@burmecia burmecia added enhancement New feature or request iceberg labels May 20, 2025
@burmecia burmecia merged commit 4043a7a into main May 20, 2025
6 checks passed
@burmecia burmecia deleted the bo/feat/iceberg-fdw branch May 20, 2025 23:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request iceberg
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants