Skip to content

Serve sets, report the uncompressed result size, version 2.17.0-rc.1 - #82

Merged
mo4islona merged 1 commit into
masterfrom
feat/sets-uncompressed-size
Oct 26, 2026
Merged

mo4islona merged 1 commit into
masterfrom
feat/sets-uncompressed-size

Conversation

@mo4islona

Copy link
Copy Markdown
Contributor

Bump sqd-query-engine to fee829e (compare), sqd-network to f41e6b1 (compare), and the worker version to 2.17.0-rc.1.

Upstream changes that matter here:

  • Sets (feat: sets, filters on value sets too long for a list sqd-query-engine#85): a filter on a value set too long to send as a list. A request declares a set once under sets and filters refer to it by name. The worker passes the request to the engine unchanged, so it needs no code for this. The portal selects the engine per query, as before.
  • The engine's request bound becomes a 16 MiB backstop, above the network's 4 MiB query message.
  • Parallel execution no longer makes a query wait behind other callers' queries.
  • sqd-network main vendors a patched libp2p-stream instead of a libp2p fork, and fixes hung open_stream calls. The libp2p-identity patch in this repo goes away with the fork.

Worker changes:

  • QueryOk carries uncompressed_data_size, the result size before compression (feat(messages): add the uncompressed data size to QueryOk sqd-network#228). The field is outside the signature, so portals that predate it still verify results.
  • Test code passes chunk ids to the assignment builders as DataChunk. One storage test used a chunk id with a -fork suffix, which DataChunk refuses; it now tells the two same-range chunks apart by hash.

All tests pass locally (116). Clippy shows one warning, the same as on master.

🤖 Generated with Claude Code

sqd-query-engine fee829e adds sets, filters on value sets too long to send
as a list. The portal now gets the result size before compression in
QueryOk, which needs sqd-network main; main builds libp2p from crates.io, so
the libp2p-identity patch goes away.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@mo4islona
mo4islona force-pushed the feat/sets-uncompressed-size branch from 1d571e9 to e320dac Compare October 23, 2026 19:58
@mo4islona
mo4islona merged commit 3fd9337 into master Oct 26, 2026
5 checks passed
@mo4islona
mo4islona deleted the feat/sets-uncompressed-size branch October 26, 2026 12:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant