-
Notifications
You must be signed in to change notification settings - Fork 371
Add tests for Schema Evolution on Catalogs #2478
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Add tests for Schema Evolution on Catalogs #2478
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the PR! the tests are looking good! Left some small comments 👍🏼
table.update_schema().add_column("col1", StringType()).add_column("col2", StringType()).add_column( | ||
"col3", StringType() | ||
).commit() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
small nit to format for better readability (if the linter allows it)
table.update_schema().add_column("col1", StringType()).add_column("col2", StringType()).add_column( | |
"col3", StringType() | |
).commit() | |
table.update_schema() | |
.add_column("col1", StringType()) | |
.add_column("col2", StringType()) | |
.add_column( "col3", StringType()) | |
.commit() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The linter deeply does not like this (even with the backslashes to make it syntactically correct)
deb1b89
to
839e774
Compare
Co-authored-by: Fokko Driesprong <[email protected]>
Co-authored-by: Fokko Driesprong <[email protected]>
Please add a test that adds data to a schema after evolution, you will notice it doesn't pick up the latest schema: #2467 |
Rationale for this change
PyIceberg is missing a variety of Catalog Tests compared to the Java implementation. This adds 3 different tests for schema evolution to help match the Java library coverage.
Are these changes tested?
Tests should pass.
Are there any user-facing changes?
Just tests.