feat: let pool.query pipeline on a connection that is already working

docs/pages/apis/pool.mdx

@@ -31,6 +31,8 @@ type Config = {
   // Maximum number of clients the pool should contain.
   // By default this is set to 10. There is some nuance to setting the maximum size of your pool.
   // See https://node-postgres.com/guides/pool-sizing for more information.
+  // With `maxPipeline` this is still the number of connections, but the number of queries
+  // in flight can reach max * maxPipeline.
   max?: number
 
   // Minimum number of clients the pool should hold on to and _not_ destroy with the idleTimeoutMillis.
@@ -74,6 +76,13 @@ type Config = {
   // Pipelined clients send queries to the server without waiting for previous responses.
   // Default is false. See /features/pipelining for details.
   pipeline?: boolean
+
+  // Maximum number of queries pool.query() sends on the same connection before waiting.
+  // The default is 1: a connection serves one query at a time. A higher value lets pool.query()
+  // use a connection that is already working instead of waiting for a free one, and implies
+  // pipeline: true. pool.connect() is not affected, it still checks out a connection nobody else
+  // is using. Queries can complete in a different order, so this is off by default.
+  maxPipeline?: number
 }
 ```
 
@@ -229,6 +238,10 @@ The number of clients which are not checked out but are currently idle in the po
 
 The number of queued requests waiting on a client when all clients are checked out. It can be helpful to monitor this number to see if you need to adjust the size of the pool.
 
+With `maxPipeline` a connection is counted as idle as soon as its queries are written, not when
+the results arrive, so `idleCount` can be positive while every connection is working and
+`waitingCount` grows once they are all at `maxPipeline`.
+
 ## events
 
 `Pool` instances are also instances of [`EventEmitter`](https://nodejs.org/api/events.html).

docs/pages/features/pipelining.mdx

@@ -46,7 +46,8 @@ All query types work with pipelining: plain text, parameterized, and named prepa
 
 ## Pipelining with a pool
 
-Pass `pipeline: true` in the pool config to enable it on every client the pool creates:
+Pass `pipeline: true` in the pool config to enable it on every client the pool creates. Each client
+pipelines the queries you send on it after `pool.connect()`, `pool.query()` works as usual:
 
 ```js
 import { Pool } from 'pg'
@@ -55,20 +56,57 @@ const pool = new Pool({ pipeline: true })
 
 const client = await pool.connect()
 // client.pipeline is already true
-
 const [users, orders] = await Promise.all([
   client.query('SELECT * FROM users WHERE id = $1', [1]),
   client.query('SELECT * FROM orders WHERE user_id = $1', [1]),
 ])
-
 client.release()
 ```
 
-<Alert>
-  <div>
-    <code>pool.query()</code> checks out a client for a single query and releases it immediately, so pipelining has no effect there. Use <code>pool.connect()</code> to check out a client and send multiple queries on it.
-  </div>
-</Alert>
+## Pipelining pool.query()
+
+Set `maxPipeline` to more than 1 to let `pool.query()` pipeline too:
+
+```js
+const pool = new Pool({ max: 10, maxPipeline: 10 })
+
+const [users, orders] = await Promise.all([
+  pool.query('SELECT * FROM users WHERE id = $1', [1]),
+  pool.query('SELECT * FROM orders WHERE user_id = $1', [1]),
+])
+```
+
+By default (`maxPipeline: 1`) `pool.query()` holds a connection for one query, so a query has to
+wait for a free connection. With a higher `maxPipeline` the pool sends it on a connection that is
+already working, once every connection is busy. It opens connections up to `max` first, then picks
+the one with the fewest queries in flight, up to `maxPipeline` per connection. The clients it
+creates are pipelining clients, so `pipeline: true` is implied.
+
+`max` is still the number of connections. It is not the number of queries in flight anymore, those
+can reach `max * maxPipeline`. This does not add load on the server, PostgreSQL runs a pipeline
+serially on the same backend, but a query can wait behind the ones already sent on its connection.
+Queries can also complete in a different order than without pipelining, which is why this is off by
+default.
+
+`pool.connect()` is not affected: it still checks out a connection nobody else is using, so
+transactions and `SET` keep working as before.
+
+```js
+const client = await pool.connect()
+try {
+  await client.query('BEGIN')
+  await client.query('INSERT INTO users(name) VALUES($1)', ['brianc'])
+  await client.query('COMMIT')
+} finally {
+  client.release()
+}
+```
+
+`pool.query()` refuses a submittable (`pg-cursor`, `pg-query-stream`) while pipelining, because
+those read their rows over several round trips and cannot share a connection. Check out a client
+with `pool.connect()` for them. `connectionTimeoutMillis` counts the wait for a pipeline slot too,
+so a query can time out with a connect error while the pool has a healthy connection sitting at
+`maxPipeline`.
 
 ## Error isolation
 
@@ -102,6 +140,12 @@ const queries = Array.from({ length: 100 }, (_, i) => ({
 const results = await Promise.all(queries.map(q => client.query(q)))
 ```
 
+## Query timeouts
+
+`query_timeout` cannot cancel a single pipelined query, because the queries behind it are already on the wire. 
+When it fires node-postgres closes the connection, so the other queries sharing it fail too. 
+With a pool the connection is then replaced, and `pool.query()` keeps working, but you should have a `pool.on('error')` listener as usual.
+
 ## Graceful shutdown
 
 Calling `client.end()` while pipelined queries are in flight will wait for all of them to complete before closing the connection:

packages/pg-pool/README.md

@@ -36,6 +36,7 @@ const pool2 = new Pool({
   idleTimeoutMillis: 1000, // close idle clients after 1 second
   connectionTimeoutMillis: 1000, // return an error after 1 second if connection could not be established
   maxUses: 7500, // close (and replace) a connection after it has been used 7500 times (see below for discussion)
+  maxPipeline: 1, // queries pool.query() may send on the same connection before waiting, more than 1 lets it use a connection that is still working
 })
 
 // you can supply a custom client constructor