Skip to content

Conversation

sycai
Copy link
Contributor

@sycai sycai commented Apr 28, 2025

The basic logic is:

  • If the input is a query, dry run it
  • If the input is a table reference, convert it to a SQL ("SELECT * FROM ...") and dry run it.

The dry run stats are returned in a Pandas Series, similar with https://github.com/googleapis/python-bigquery-dataframes/blob/30a62372b2fd72deaf7dbc1dbce8b48f03b3041c/bigframes/core/blocks.py#L818-L885

I didn't perform the type conversion from BQ schema to Pandas dtypes due to non-trivial efforts. We could either:

  • Re-use BQ Python client's conversion function, which is buried somewhere deep in the codebase, and is possibly not supposed to be invoked by external packages.
  • Define our own conversion function, which needs to be kept in sync with the BQ client library.

@product-auto-label product-auto-label bot added size: m Pull request size is medium. api: bigquery Issues related to the googleapis/python-bigquery-pandas API. labels Apr 28, 2025
sycai added 2 commits April 28, 2025 22:55
@sycai sycai marked this pull request as ready for review April 28, 2025 23:35
@sycai sycai requested review from a team as code owners April 28, 2025 23:35
@sycai sycai requested review from GarrettWu and shollyman April 28, 2025 23:35
@sycai sycai requested review from chalmerlowe and tswast and removed request for GarrettWu and shollyman April 28, 2025 23:36
@sycai sycai added the do not merge Indicates a pull request not ready for merge, due to either quality or timing. label Apr 29, 2025
@sycai sycai changed the title feat: add dry_run option for read_gbq feat: [WIP] add dry_run option for read_gbq Apr 29, 2025
@sycai sycai removed request for chalmerlowe and tswast April 29, 2025 21:20
@sycai
Copy link
Contributor Author

sycai commented Apr 29, 2025

The next step is add logics of type conversions from BQ schema to Pandas

@Linchin
Copy link
Contributor

Linchin commented Sep 17, 2025

I will close this PR as we are migrating the repo into googleapis/google-cloud-python. Please open a PR there if you want to continue working on it.

@Linchin Linchin closed this Sep 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: bigquery Issues related to the googleapis/python-bigquery-pandas API. do not merge Indicates a pull request not ready for merge, due to either quality or timing. size: m Pull request size is medium.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

None yet

3 participants