query optimiser that pushes where clauses down to subqueries closer to the source

[samwillis]

This leads on to pushing those where clauses down to the collection subscription which can use the index... see #257

There is a detailed comment explaining the optimiser at the top of packages/db/src/query/optimizer.ts, but in essence it rewrites the query IR moving things around.

[changeset-bot]

🦋 Changeset detected

Latest commit: 71eb5ab

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 6 packages

Name	Type
@tanstack/db	Patch
@tanstack/electric-db-collection	Patch
@tanstack/query-db-collection	Patch
@tanstack/react-db	Patch
@tanstack/vue-db	Patch
@tanstack/db-example-react-todo	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[pkg-pr-new]

@tanstack/db-example-react-todo

@tanstack/db

npm i https://pkg.pr.new/@tanstack/db@256

@tanstack/electric-db-collection

npm i https://pkg.pr.new/@tanstack/electric-db-collection@256

@tanstack/query-db-collection

npm i https://pkg.pr.new/@tanstack/query-db-collection@256

@tanstack/react-db

npm i https://pkg.pr.new/@tanstack/react-db@256

@tanstack/vue-db

npm i https://pkg.pr.new/@tanstack/vue-db@256

commit: 71eb5ab

[github-actions]

Size Change: +2.98 kB (+8.58%) 🔍

Total Size: 37.7 kB

Filename	Size	Change
./packages/db/dist/esm/query/compiler/index.js	1.75 kB	+271 B (+18.32%)	⚠️
./packages/db/dist/esm/query/compiler/joins.js	1.2 kB	+52 B (+4.54%)	🔍
./packages/db/dist/esm/query/optimizer.js	2.24 kB	+2.24 kB (new file)	🆕
./packages/db/dist/esm/utils.js	419 B	+419 B (new file)	🆕

ℹ️ View Unchanged

Filename	Size
./packages/db/dist/esm/collection.js	8.45 kB
./packages/db/dist/esm/deferred.js	230 B
./packages/db/dist/esm/errors.js	150 B
./packages/db/dist/esm/index.js	568 B
./packages/db/dist/esm/local-only.js	815 B
./packages/db/dist/esm/local-storage.js	2.07 kB
./packages/db/dist/esm/optimistic-action.js	294 B
./packages/db/dist/esm/proxy.js	3.75 kB
./packages/db/dist/esm/query/builder/functions.js	531 B
./packages/db/dist/esm/query/builder/index.js	3.49 kB
./packages/db/dist/esm/query/builder/ref-proxy.js	842 B
./packages/db/dist/esm/query/compiler/evaluators.js	1.34 kB
./packages/db/dist/esm/query/compiler/group-by.js	2.09 kB
./packages/db/dist/esm/query/compiler/order-by.js	933 B
./packages/db/dist/esm/query/compiler/select.js	657 B
./packages/db/dist/esm/query/ir.js	318 B
./packages/db/dist/esm/query/live-query-collection.js	2.06 kB
./packages/db/dist/esm/SortedMap.js	1.24 kB
./packages/db/dist/esm/transactions.js	2.29 kB

_{compressed-size-action::db-package-size}

[github-actions]

Size Change: 0 B

Total Size: 1.05 kB

ℹ️ View Unchanged

Filename	Size
./packages/react-db/dist/esm/index.js	152 B
./packages/react-db/dist/esm/useLiveQuery.js	902 B

_{compressed-size-action::react-db-package-size}

[kevin-dp]

LGTM, i left a few comments.

[kevin-dp]

The docstring on this function says:

Step 1: Split all AND clauses at the root level into separate WHERE clauses.

But since we call splitAndClauses recursively here this is not only at the root level but also nested. So it turns any nested conjunctions (i.e. AND clauses) into a flatened array of all conjuncts.

[kevin-dp]

I don't really like that we're passing around mutable state and relying on mapNestedQueries to mutate that state. It feels like this should be a class since it keeps some state and provides methods over that state.

[samwillis]

Agreed, it's getting quite complex. I would like to refactor the compiler into a class, but let's do it in a follow-up PR or this will blow up.