-
Notifications
You must be signed in to change notification settings - Fork 325
Partition statistics metadata reading #2033
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Partition statistics metadata reading #2033
Conversation
class PartitionStatisticsFile(IcebergBaseModel): | ||
pass |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This does look quite odd 😄
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's add these fields: https://iceberg.apache.org/spec/#partition-statistics
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Silly me - meant to subclass StatisticsCommonFields
but looks like you figured that out #2146 😄
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hey @smaheshwar-pltr thanks for working on this, it would be good to get this in for the remove orphan files operation 👍
class PartitionStatisticsFile(IcebergBaseModel): | ||
pass |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's add these fields: https://iceberg.apache.org/spec/#partition-statistics
# Rationale for this change Took @smaheshwar-pltr's draft PR and added a test: #2033 Cherry-picked his original work to ensure it gets attributed to the original author. # Are these changes tested? # Are there any user-facing changes? <!-- In the case of user-facing changes, please add the changelog label. --> --------- Co-authored-by: Sreesh Maheshwar <[email protected]>
Closes #2034
(Will test)
Rationale for this change
Are these changes tested?
Are there any user-facing changes?