feat: add tentative data & delete writer api#463
Conversation
|
I would expect separate PRs for |
078acb1 to
e05193c
Compare
Sure. This PR provides the factory pattern and stub implementations for all three writer types. The actual implementations are planned for follow-up PRs as outlined in issue #441 . |
Add FileWriterFactory to create data and delete writers, with comprehensive improvements per review feedback. File Organization: - Separate DataWriter to data_writer.h/.cc - Separate PositionDeleteWriter to position_delete_writer.h/.cc - Separate EqualityDeleteWriter to equality_delete_writer.h/.cc - Separate FileWriterFactory to file_writer_factory.h/.cc - Keep only FileWriter base interface in writer.h/.cc Key Features: - Input validation for all factory methods (path, schema, spec) - Thread safety documentation (NOT thread-safe) - State management in stub implementations (is_closed tracking) - Support for Parquet and Avro formats - Pass-by-value + std::move for sink parameters Implementation: - FileWriterFactory directly creates writers (true factory pattern) - Writers use friend pattern - only factory can construct them - Internal MakeXxxInternal functions handle cross-file construction - Stub implementations validate inputs before returning NotImplemented Related to apache#441
e05193c to
bd1a6e4
Compare
wgtmac
left a comment
There was a problem hiding this comment.
Thanks for adding this! I've removed unnecessary implementations and undecided factory to avoid distraction. Since this is just a tentative api design. Let's move on and fix any issue during the followup implementation.
Add complete API structure for Iceberg data file writers including:
Add FileWriterFactory to create data and delete writers, with
comprehensive improvements per review feedback.
File Organization:
Key Features:
Implementation: