feat(common): canonical single-query serialization for async chart data - #43410
Conversation
Add a JSON-safe, self-contained serialization for one chart-data query, the unit a later PR runs as its own async task. serialize_query() emits the *raw* query dict (from QueryContext.cache_values) plus datasource/form_data/result_type/ result_format/force/custom_cache_timeout; load_serialized_query() rebuilds a single-query QueryContext via QueryContextFactory — the same path that produced the original — so the reconstructed query hashes to an identical query_cache_key and reads/writes the same DATA-cache entry as the sync path. Serializing the raw query (not the processed QueryObject.to_dict(), which emits raw datetimes, renames filters->filter, and drops time_range/datasource) is what makes the round trip cache-key-stable, and force/custom_cache_timeout (which live on the context, not the query) are carried explicitly so they survive per query. Unit tests (payload extraction, JSON-safety, factory reconstruction) plus an integration round-trip test asserting identical query_cache_key + force/ custom_cache_timeout survival. The async-chain rename/dedup of the misleading form_data param lands with PR 3, which rewrites those files.
|
Bito Automatic Review Skipped - Branch Excluded |
✅ Deploy Preview for superset-docs-preview ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
dosubot
Bot
added
api
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## gaq-to-gtf #43410 +/- ##
==============================================
- Coverage 78.88% 78.88% -0.01%
==============================================
Files 2882 2883 +1
Lines 164715 164715
Branches 38028 38027 -1
==============================================
- Hits 129938 129937 -1
- Misses 32331 32332 +1
Partials 2446 2446
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
/review |
Code Review Agent Run #59350eActionable Suggestions - 0Filtered by Review RulesBito filtered these suggestions based on rules created automatically for your feedback. Manage rules.
Review Details
Bito Usage GuideCommands Type the following command in the pull request comment and save the comment.
Refer to the documentation for additional commands. Configuration This repository uses Documentation & Help AI Code Review powered by |
1 participant
SUMMARY
Adds a canonical, JSON-safe, self-contained serialization for a single chart-data query — the atomic unit a later PR (PR 3) will run as its own async GTF task. Today the async path serializes the entire query context (the request dict misleadingly named
form_data); PR 3 will instead fan out one task perQueryObject, and each task needs to reconstruct exactly its query so it caches under the samequery_cache_keyas the synchronous path. This PR provides that primitive (with tests); PR 3 consumes it.API (
superset/common/query_serialization.py):serialize_query(query_context, query_index) -> SerializedQuery— a JSON-safe dict of the raw query (fromQueryContext.cache_values["queries"]) plus the context-level inputs needed to rebuild it:datasource,form_data,result_type,result_format,force,custom_cache_timeout.load_serialized_query(payload) -> QueryContext— rebuilds a single-queryQueryContextviaQueryContextFactory(the same path that produced the original), soquery_cache_keymatches.Why the raw query, not
QueryObject.to_dict(). Reconstructing fromto_dict()would change the cache key:to_dict()emitsfrom_dttm/to_dttmas raw datetimes, uses the keyfilterwhere the factory expectsfilters, and omitstime_range/datasource/result_type. Serializing the raw schema-shaped query and rebuilding through the factory reuses the exact key-producing path (the same patternSlice.query_contextalready relies on), giving a provably identicalquery_cache_key.force/custom_cache_timeoutsurvival. These live onQueryContext, notQueryObject, so a per-query payload must carry them explicitly — they're serialized and restored, and the integration test asserts it.BEFORE / AFTER
No behavior change to the live async flow in this PR — it adds a foundation primitive. The misleading
form_datarename and the duplicate_create_query_context_from_formdedup are intentionally deferred to PR 3, which rewritestasks/async_queries.pyand replacescreate_async_job_command.py(doing the rename here would be throwaway).TESTING INSTRUCTIONS
tests/unit_tests/common/test_query_serialization.py, green locally): payload extraction reads the raw query + context params; payload is JSON round-trippable;load_serialized_queryreconstructs viaQueryContextFactorywithresult_type/result_formatback as enums.tests/integration_tests/query_context_tests.py::test_serialize_query_round_trip_preserves_cache_key): builds a realbirth_namesquery context, serializes + reloads query 0, and asserts the reconstructedquery_cache_keyequals the original and thatforce/custom_cache_timeout/result_type/result_formatsurvive.ADDITIONAL INFORMATION
serialize_query/load_serialized_query)Note: the whole-project frontend type-check has pre-existing failures unrelated to this change.