improve(bq_driver): SQLTables perf improvement - #1627
Open
Anshu6250 wants to merge 14 commits into
Open
Conversation
shivamd-gpartner
force-pushed
the
impr_sql_tables2
branch
from
July 31, 2026 13:01
7226eec to
66293a0
Compare
shivamd-gpartner
temporarily deployed
to
internal
July 31, 2026 13:02 — with
GitHub Actions
Inactive
shivamd-gpartner
force-pushed
the
impr_sql_tables2
branch
2 times, most recently
from
July 31, 2026 13:37
8c27c41 to
7fd9cbc
Compare
shivamd-gpartner
temporarily deployed
to
internal
July 31, 2026 13:38 — with
GitHub Actions
Inactive
shivamd-gpartner
temporarily deployed
to
internal
July 31, 2026 14:18 — with
GitHub Actions
Inactive
This reverts commit c0c8f31.
shivamd-gpartner
force-pushed
the
impr_sql_tables2
branch
from
July 31, 2026 14:42
7fd9cbc to
3d4e170
Compare
shivamd-gpartner
temporarily deployed
to
internal
July 31, 2026 14:42 — with
GitHub Actions
Inactive
shivamd-gpartner
temporarily deployed
to
internal
July 31, 2026 15:29 — with
GitHub Actions
Inactive
shivamd-gpartner
temporarily deployed
to
internal
July 31, 2026 15:49 — with
GitHub Actions
Inactive
Anshu6250
force-pushed
the
impr_sql_tables2
branch
from
August 5, 2026 08:43
9396410 to
1b46d98
Compare
Anshu6250
force-pushed
the
impr_sql_tables2
branch
from
August 5, 2026 08:46
1b46d98 to
92e52be
Compare
shivamd-gpartner
marked this pull request as ready for review
August 5, 2026 15:21
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
perf result- link
Added google-cloud-cpp logs and can be verified with regex search "page_token: "[^"]" in file without max results there are 34 extra responses or can also search "next_page_token" which proves the hypothesis that is increasing the latency.
test_output_withoit_max_rsult.log
test_output_without_max_result.log
LiteralFromOdbcPattern method improves :- WildcardTableSearch majorly
With
SQL_ATTR_METADATA_ID = SQL_FALSE, ODBC defines the catalog, schema andtable arguments as search patterns, and
SQLTablestreated all three that wayunconditionally. For a call that names an exact project and dataset, that meant
two wasted round trips before the useful one:
projects.list -> enumerate every visible project, regex-match to find the one named
datasets.list -> enumerate every dataset in it, regex-match to find the one named
tables.list -> the call we actually wanted
projects.listis the worst of the two: the API has no server-side name filter,so it always returns everything just to select a name the caller already gave us.
LiteralFromOdbcPattern()checks whether a filter contains any unescaped%or_. If it doesn't, the pattern can only ever match itself, so enumerate-then-matchis equivalent to using the name directly — the driver now does that and skips the
listing. This generalises the existing
SQL_ATTR_METADATA_ID = SQL_TRUEfast pathto literal patterns.
No API call changed. No new endpoint, no changed request shape or parameters;
the surviving
tables.listis identical, and the table-name filter is still matchedclient-side as before. Two calls are simply no longer made. Escaped wildcards
(
\_,\%) are unescaped and still treated as literals, so pattern semantics arepreserved.
EscapeOdbcPattern method improves :- FilterTablesOnDefaultDataset
When the application passes
schema = NULLand the DSN setsFilterTablesOnDefaultDataset=1,SQLTablessubstituted the configured datasetname straight into
dataset_filter— a variable everything downstream interpretsas an ODBC LIKE pattern. Since
_is a single-character wildcard, a configuredODBC_TEST_DATASETcompiles toODBC.TEST.DATASETand also matchesODBCxTESTyDATASET, so users could be shown tables from datasets they neverconfigured. BigQuery dataset names use
_as the conventional separator, so mostDSNs using this option are exposed.
EscapeOdbcPattern()escapes%,_and\at the substitution site so the namematches only itself. It also lets the existing literal fast path skip
datasets.listfor that call.No API call changed — no new endpoint, no changed request shape; only how the
schema argument is interpreted before the existing calls are made.