fix(mcp): reduce LLM-facing tool friction, token cost, and a scratch-table leak - #332
Open
StefanSteiner wants to merge 2 commits into
Open
StefanSteiner wants to merge 2 commits into
StefanSteiner wants to merge 2 commits into
Conversation
…table leak Improves how reliably an LLM can drive the MCP and cuts the always-in-context token cost of the tool catalog, plus fixes two real scratch-table bugs. Bugs: - query_data / query_file dropped their `_tmp_*` scratch table only after the query, so a failed query orphaned it (and it then surfaced in `describe`). The DROP now runs whether the query succeeds or fails. - The SQL table-name substitution was a naive substring replace that corrupted columns like `metadata` when the alias was `data`; it is now whole-word via `replace_identifier`. - `describe` now hides transient scratch tables (`_tmp_*`, `__hyperdb_merge_*`), not just `_hyperdb_*` internals. Ergonomics: - query_file exposes rows as `data` by default (matching query_data and the documented example, which previously failed). - `kind` is optional on attach_database and copy_query temp_attach, defaulting to `local_file`. - Unknown-table (42P01) errors now suggest `describe` and name the `data` alias instead of the generic "check SQL syntax". Tokens: - Trimmed the essay-length tool descriptions (~15.8k -> ~11.7k chars, ~26%), moving format/edge-case depth into get_readme while keeping actionable rules inline. get_readme stays under its 24k budget.
`is_internal_table`'s doc linked to the private `INTERNAL_SCRATCH_PREFIXES` const, which `cargo doc` rejects under `RUSTDOCFLAGS=-D warnings` (rustdoc::private_intra_doc_links). Demote it to a plain code span.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Makes the HyperDB MCP more reliable and cheaper for an LLM to drive. Came out of a systematic pass exercising all 33 tools live and tracing the friction points back to source. Two axes: fewer LLM mistakes, and lower always-in-context token cost.
Bugs fixed
query_data/query_filedropped their_tmp_*staging table only after the query ran, so a failed query orphaned it — and it then showed up indescribeas a mystery table. TheDROPnow runs whether the query succeeds or fails.sql.replace(alias, temp)that corrupted a column likemetadatawhen the alias wasdata. Replaced with a whole-wordreplace_identifierhelper (unit-tested). Known limitation documented: it is identifier-boundary aware but not SQL-aware, so a string literal exactly equal to the alias is still rewritten — rare, since the alias is a table name.describeclutter: table listings now hide transient scratch tables (_tmp_*,__hyperdb_merge_*), not just_hyperdb_*internals.Ergonomics
query_filedefault table is nowdata(matchingquery_dataand the documentedFROM dataexample, which previously failed because the default was the unpredictable file stem).kindis now optional onattach_databaseandcopy_query'stemp_attach, defaulting tolocal_file— no more required ceremony field with only one legal value.42P01now suggests runningdescribeand names thedataalias, instead of the generic "check SQL syntax."Token cost
load_file,export,chart,load_files,set_table_metadata,kv_set+ 3 updated): ~15.8k → ~11.7k chars (~26%) of always-in-context tool-catalog text. The format-selection and edge-case depth moved intoget_readme(consolidated format-preference block), which stays under its 24k budget.Type-name casing normalization (
TEXTvstext) was deliberately deferred — it originates from two engine code paths asserted on by many tests, and type names are case-insensitive so it causes no actual LLM mistakes (high-risk / low-value).Test plan
cargo test -p hyperdb-mcp --all-targetsagainst pinned.hyperd/current— 669 tests, 0 failures (incl. newreplace_identifierandis_internal_tableunit tests)cargo clippy --workspace --all-targets --all-features -- -D warnings— cleancargo fmt --all— no diffnpx markdownlint-cli2on the touched CHANGELOG — 0 issues