-
Notifications
You must be signed in to change notification settings - Fork 284
Open
1 / 11 of 1 issue completedLabels
epicEpic issueEpic issue
Description
What's the feature are you trying to implement?
This issue is used to track implementing appending data to iceberg table. In summary, after we complete all subtasks in this issue, we should be able to execution following insert sql statment in datafusion to insert table to iceberg table:
insert into t value (1, 'a');
For more details about insert statement, please refer to datafusion doc.
Commit path
- Add a method in TableCommit to apply changes to TableMetadata to create a new TableMetadata #1386
- Add retry in
Transaction::commit
method #1387 - Implement saving
TableMetadata
to new location. #1388 - Swith metadata pointer to new location for different catalogs(sql, hive, glue, etc) #1389
Writer path
- Implement fanout partitioned data writer.
- Implement non fanout partitioned data writer.
- Implement data writer for non partitioned table.
Datafusion integration
- Implement
insert_into
method forIcebergTableProvider
Willingness to contribute
I would be willing to contribute to this feature with guidance from the Iceberg Rust community
kevinjqliu and warriors-makerjonathanc-n, timoguin, ZENOTME, mike-luabase, CTTY and 8 more
Sub-issues
Metadata
Metadata
Assignees
Labels
epicEpic issueEpic issue