Repository navigation
docs(clients): document QWP in the Node.js client and the new browser client - #569
Merged
Merged
Conversation
|
🚀 Build success! Latest successful preview: https://preview-569--questdb-documentation.netlify.app/docs/ Commit SHA: e706384
|
glasstiger
changed the title
glasstiger
changed the title
… client Rewrite the Node.js client page for @questdb/nodejs-client 5.0.0, which ingests and queries over QWP, and add a page for @questdb/browser-client. Document the qwp.browser.allowed.origins and qwp.browser.tls.termination.enabled server settings, and add the Node.js defaults and differences to the shared connect-string and store-and-forward references.
glasstiger
force-pushed
the
ia_js_client
branch
from
October 26, 2026 13:37
e4e0464 to
1816a00
Compare
glasstiger
changed the title
- Node.js production example: read the feed in upstream batches, flush the end of each batch, and commit it upstream once it is in the store-and-forward journal (sf_durability=append). Create trades with DEDUP UPSERT KEYS(timestamp, symbol, trade_id) so replays cannot duplicate or merge trades. - Document that QuestDB closes idle connections after http.net.connection.timeout (5 minutes) and nginx after 60 seconds. Add proxy_read_timeout to the nginx example, and drive the connection examples from all-endpoints-unreachable instead of disconnected. - Show where onSenderError, onError, and reconnect.onEvent go on the pooled clients (ingress and egress, not cluster), and list the connection event fields on the browser page. - Browser: explain that sessionBootstrap cannot refresh an expiring credential, recommend an auth function for OIDC access tokens, and use one in the pooled example. - Store-and-forward concepts: the Node.js close() rejects with QwpSenderCloseTimeoutError instead of logging a warning. - Connect overview: note the browser client's exceptions to store-and-forward and connect strings. - Name JavaScript and TypeScript in the client page titles, descriptions, intros, sidebar label, and client grid tile.
… clients The pages merged from #575 said that only a connected Java sender retries rejected credentials. A connected Node.js sender that uses store-and-forward or initial_connect_retry=async, and a browser sender started with initialConnectMode "async", also keep retrying: they buffer rows, report auth-failed on each attempt, and resume once QuestDB accepts the credential again. Say so on the QWP client behavior, QWP ingress, client failover, and connect string pages. Narrow the browser session-cookie caution accordingly: a rejected sessionBootstrap ends a query session and a sender in the default mode, while an "async" sender keeps retrying without delivering its rows.
…he JavaScript client pages - Wait for sender.publishedSequence after flush() in the waitForAck example: flushAndGetSequence() returns -1n when auto-flush already sent the rows, and waitForAck(-1n) resolves at once. - Document recovering a store-and-forward journal after a crash in a container, where the lock names a live PID or another host name. - Correct when the Node.js and browser clients retry rejected credentials: a browser sender with `auth` retries a rejection like an outage and reports endpoint-attempt-failed. Fix the neighbour pages that said otherwise. - Read the quick-start row back by its timestamp, so a rerun cannot match an older row. - Add a browser options table with defaults, the initialConnectMode default, the writer field table, and examples for designated timestamps, transactions, binds, timeouts, and zero-copy views. Document auth_timeout_ms for Node.js.
…locked journals - QuestDB 10.1 does not see a cancellation while it streams a result faster than the client reads it. Explain that cancel(), timeouts, and leaving a for-await loop early need a positive initialCredit, and that without it the session fails after cancelDrainTimeoutMs (query_close_timeout_ms). Set the credit in the Node.js timeout example, and add QwpEgressQueryCancelTimeoutError and QwpEgressSessionClosedError to the error tables. - Warn that a query session runs an INSERT or UPDATE again after a reconnect, so a statement whose reply was lost takes effect twice, and list the ways to avoid it. - Explain that a batch QuestDB rejects stays in the store-and-forward journal and blocks it, and how to recover: fix the cause and restart, or move the journal aside.
Resolve conflicts with the Ruby client (#592): - connect/overview: list Ruby alongside web browsers, and add Ruby (Beta) to the QWP support table next to Node.js and Browser (JavaScript). - wire-protocols/qwp-ingress-websocket: drop Ruby from the "bring QWP to" examples, since it now has a client. - wire-protocols/qwp-egress-websocket: same fix. This sentence merged without a textual conflict but went stale, because the Ruby client also runs queries over QWP.
…nt in the JavaScript client pages - Query status table: QuestDB 10.1 never sends status 3 for queries. Bind value errors arrive as 5 or 6, and 11 also covers the server's query timeout and memory limit. - Terminal rejections: later calls reject with QwpReplayRejectedError, and without store-and-forward every unacknowledged batch is lost, including batches flushed after the rejected one and other tables' rows in it. - Error tables: plain Error from setters, the error close() rejects with when publishing fails first, QwpFailoverError on browser connects, and a browser error-class table. - Idle query sessions use up their failover limits in a short outage, and the pool lends the failed connection once. - Browser page: open source and Enterprise credentials and TLS, read-only dashboards, OIDC table creation, and a local development tip. - qwp.browser.allowed.origins: QuestDB's own origin must be listed for auth. - Mark the Node.js and browser clients stable in the overview and grid. - Align the store-and-forward, connect string, client behaviour, client failover, and PGWire pages with the new client pages.
…client, and harden browser auth - Row errors: at() also rejects when the auto-flush that the row triggered fails, and the row then stays queued. The Node.js and browser examples catch setter errors only and let at() propagate, so a flush failure is no longer reported as a skipped row that a caller would write twice. - atNow(): the server stamps the batch, not the row. Rows flushed together share one timestamp and can collapse into one row on a deduplicated table. - Node.js: recommend connectQwpNodeClient() for new services, including ingest-only and query-only ones with query_pool_min=0 or sender_pool_min=0, move The pooled client after Connecting, and run the query examples on pooled leases, so they take the same connect string. - Browser: recommend one pooled client per application and move its section after Authentication. Choose sessionBootstrap or auth by credential lifetime as well as page origin. Mark the session cookie Secure at a TLS-terminating proxy with proxy_cookie_flags. Add a UI component example that shares one client, refreshes a view, and cleans up on unmount. - Connect string and store-and-forward pages: the Node.js .lock.owner directory survives a crash, unlike the kernel-released flock. - QWP egress and client behaviour pages: a browser query session that uses auth fails over on a rejected credential, and a Node.js store-and-forward or async sender retries rejections once it has connected.
… browser exposure risks - Pooled client: connectQwpNodeClient() and a borrow that opens a connection reject with QwpPoolResourceError, whose cause holds the QwpUpgradeError or QwpReplayStoreLockedError. The pooled client, authentication, store-and-forward, and error sections now say so. - lazy_connect sets initial_connect_retry=async and query_pool_min=0, so it rejects a positive query_pool_min, another initial_connect_retry, and reconnect: false under ingress. - idle_timeout_ms and max_lifetime_ms close only idle connections above the pool minimum and open no replacements. The connect string reference notes the Node.js difference. - Callbacks and reconnect placed under cluster apply to both sides in plain JavaScript instead of being ignored. - Transactions: QuestDB acknowledges a transaction's batches only at commit, so a transaction must fit in the replay queue, or at() and flush() reject and none of its rows are committed. - Browser: warn that the origin check is not access control, so QuestDB exposed without authentication opens every table. An auth function that throws raises endpoint-attempt-failed only, so an identity provider outage does not show in an all-endpoints-unreachable banner.
…origin check - Node.js: rename "Failover and high availability" to "Reconnection and failover" and limit the Enterprise note to failing over between hosts. Reconnects, replay, re-executed queries, and the retry limits apply to a single open source server too. - Both client pages: add a "Delivery is at least once" caution that points to DEDUP UPSERT KEYS. - Node.js: choose the entry point by what the service does. The pooled client suits services that query or write from several tasks, and a standalone sender suits one producer, because its close() waits for acknowledgements and its journal is <sf_dir>/<sender_id>. The production example now says why it uses a standalone Sender. - Browser: the origin check covers only /write/v4 and /read/v1 and compares Origin with Host. It does not protect /exec or the other REST endpoints, and a DNS-rebinding page gets through.
javier
requested changes
Oct 29, 2026
…tup without a primary - PGWire Node.js page: recommend the Node.js client for queries as well as ingestion, and keep PGWire for compatibility: tools and libraries that already speak the PostgreSQL protocol, existing pg or postgres code, and QuestDB versions before 10.0. pg is now recommended among the PGWire clients only. - Node.js: a process that starts while no primary is reachable, for example during a failover when only replicas answer, fails its first connection. lazy_connect=on on the pooled client, or initial_connect_retry=async on a Sender, starts anyway and queues rows until a primary appears.
A replica answers a Node.js sender's upgrade with HTTP 421 Misdirected Request, which the client reports as a QwpUpgradeError of kind role-rejected.
With store-and-forward, a new Node.js sender continues the sequence of the unacknowledged frames it recovers from its journal, so publishedSequence and ackedSequence also count frames that an earlier process journalled. When there is nothing to recover, the sequence starts again from 0.
…metrics The memory queue fields in metrics.ingress do not apply to store-and-forward: waitingMemoryReplayAppends stays 0 and memoryReplayUsedBytes is undefined, even during an outage. pendingReplayFrames and pendingReplayBytes report the unacknowledged backlog in either mode.
…drains The Node.js client emits primary-unavailable only from orphan journal drains, when every endpoint rejects the drain with HTTP 421. A sender's own connection never reports it: when no primary is reachable, it reports all-endpoints-unreachable.
javier
approved these changes
Oct 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
@questdb/nodejs-client5.0.0, which ingests and queries over QWP. It covers the pooledconnectQwpNodeClient()handle,Senderwithws::andwss::connect strings,connectQwpNodeSender()andconnectQwpNodeEgress(), every column setter, compiled object-row writers, null handling, acknowledgements and transactions, durable ACK, backpressure, store-and-forward, streaming queries with the complete bind-parameter and result-value mappings, DDL and DML, timeouts and cancellation, zero-copy result views, failover with theonFailoverResetduplicate-row warning, connection events, error payloads, UDP, a production example, and a migration guide from ILP. ILP keeps a short legacy section.@questdb/browser-client: same-origin deployment behind a reverse proxy with an nginx example, applications on their own origin listed inqwp.browser.allowed.origins, session-cookie bootstrap and theauthsubprotocol credential with refreshed OIDC tokens, which credentials a browser application may hold, in-memory replay, queries, the pooled client, failover, error handling, and the differences from the Node.js client.qwp.browser.allowed.originsandqwp.browser.tls.termination.enabledserver settings on the QWP configuration page, with defaults and reloadability taken fromSHOW PARAMETERSon QuestDB 10.1.0.tls_roots, single-letter size suffixes,0rather thanofffor the row and interval auto-flush triggers,sf_durability=append, theconnect_timeoutdefault, pooled slot names, and the close-drain default.Every TypeScript example on both pages type-checks under
strictand was run against QuestDB 10.1.0: the Node.js examples on Node.js 24, and the browser examples in headless Chromium, including session-cookie andauthauthentication, an allowed cross-origin application, and a TLS-terminating nginx proxy.Dependencies
@questdb/nodejs-client5.0.0 and@questdb/browser-client5.0.0 are on npm. npm currently serves@questdb/nodejs-client4.2.0 and has no@questdb/browser-client.