Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor for BigQueryPartition class #54

Merged

Conversation

prodriguezdefino
Copy link
Collaborator

In preparation for the unbounded source implementation. Test coverage for partition related methods was improved as well.

…ed table is expired and no longer accessible.
…rom time to time, the pipeline should be able to recover reading from the beginning of the split or the last checkpointed offset
@jayehwhyehentee
Copy link
Collaborator

/gcbrun

@jayehwhyehentee
Copy link
Collaborator

/gcbrun

BigQueryPartition.formatPartitionRestrictionBasedOnInfo(
Optional.of(info), "event", "2010-10-10"))
.isEqualTo("event BETWEEN '2010-10' AND '2010-11'");
public void testCheckPartitionCompletedMonth() {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add edge case tests for February as leap month, non-leap month, a 30 day month, a 31 day month.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add not completed 31 day month case. LGTM otherwise.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@jayehwhyehentee
Copy link
Collaborator

/gcbrun

@jayehwhyehentee
Copy link
Collaborator

/gcbrun

@vishalkarve15 vishalkarve15 merged commit 9748984 into GoogleCloudDataproc:main Dec 8, 2023
4 checks passed
@jayehwhyehentee jayehwhyehentee deleted the partition_refactor branch December 12, 2023 05:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants