Skip to content

Conversation

antoineeripret
Copy link

Full context: googleapis/google-cloud-python#14488

The objective of this change is to ensure that we can leverage partitioning and clustering when using the pands_gbq library. New arguments added to the function (and documented in the code).

clustering_columns
time_partitioning_column
time_partitioning_type
time_partitioning_expiration_ms
range_partitioning_column
range_partitioning_range

This allows us to execute the following code (for example):

pandas_gbq.to_gbq(
    df, 
    "xxxxx.yyyy", 
    project_id="zzzzz",
    if_exists="replace", 
    credentials=credentials, 
    time_partitioning_column="date", 
    clustering_columns=['country', 'page']
)

Which would end up in the following configuration being applied in BigQuery:

image

Copy link

google-cla bot commented Sep 22, 2025

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@product-auto-label product-auto-label bot added size: m Pull request size is medium. api: bigquery Issues related to the googleapis/python-bigquery-pandas API. labels Sep 22, 2025
@antoineeripret antoineeripret marked this pull request as ready for review October 2, 2025 12:54
@antoineeripret antoineeripret requested review from a team as code owners October 2, 2025 12:54
@sycai sycai changed the title Add partitioning and clustering to the to_gbq function feat: Add partitioning and clustering to the to_gbq function Oct 2, 2025
@sycai sycai requested a review from tswast October 2, 2025 17:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the googleapis/python-bigquery-pandas API. size: m Pull request size is medium.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants