Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: ability to use unlogged table on Postgres #1290

Merged
merged 3 commits into from
Jan 25, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 28 additions & 3 deletions packages/postgres/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,25 +32,50 @@ You can specify the `table` option.
e.g:

```js
const keyv = new Keyv(new KeyvPostgres('postgresql://user:pass@localhost:5432/dbname'), { table: 'cache' });
const keyvPostgres = new KeyvPostgres({ uri: 'postgresql://user:pass@localhost:5432/dbname', table: 'cache' });
const keyv = new Keyv(keyvPostgres);
```

You can specify the `schema` option (default is `public`).

e.g:

```js
const keyv = new Keyv(new KeyvPostgres('postgresql://user:pass@localhost:5432/dbname'), { schema: 'keyv' });
const keyvPostgres = new KeyvPostgres({ uri: 'postgresql://user:pass@localhost:5432/dbname', schema: 'keyv' });
const keyv = new Keyv(keyvPostgres);
```

You can also use a helper function to create `Keyv` with `KeyvPostgres` store.
You can also use the `createKeyv` helper function to create `Keyv` with `KeyvPostgres` store.

```js
import {createKeyv} from '@keyv/postgres';

const keyv = createKeyv({ uri: 'postgresql://user:pass@localhost:5432/dbname', table: 'cache', schema: 'keyv' });
```

## Using an Unlogged Table for Performance

By default, the adapter creates a logged table. If you want to use an unlogged table for performance, you can pass the `useUnloggedTable` option to the constructor.

```js
const keyvPostgres = new KeyvPostgres({ uri: 'postgresql://user:pass@localhost:5432/dbname', useUnloggedTable: true });
const keyv = new Keyv(keyvPostgres);
```

From the [PostgreSQL documentation](https://www.postgresql.org/docs/current/sql-createtable.html#SQL-CREATETABLE-UNLOGGED):

If specified, the table is created as an unlogged table. Data written to unlogged tables is not written to the write-ahead log (see Chapter 28), which makes them considerably faster than ordinary tables. However, they are not crash-safe: an unlogged table is automatically truncated after a crash or unclean shutdown. The contents of an unlogged table are also not replicated to standby servers. Any indexes created on an unlogged table are automatically unlogged as well.

If this is specified, any sequences created together with the unlogged table (for identity or serial columns) are also created as unlogged.

## Connection pooling

The adapter automatically uses the default settings on the `pg` package for connection pooling. You can override these settings by passing the options to the constructor such as setting the `max` pool size.

```js
const keyv = new Keyv(new KeyvPostgres({ uri: 'postgresql://user:pass@localhost:5432/dbname', max: 20 }));
```

## Testing

When testing you can use our `docker compose` postgresql instance by having docker installed and running. This will start a postgres server, run the tests, and stop the server:
Expand Down
23 changes: 17 additions & 6 deletions packages/postgres/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,13 +12,24 @@ export class KeyvPostgres extends EventEmitter implements KeyvStoreAdapter {
opts: KeyvPostgresOptions;
query: Query;
namespace?: string;
constructor(options?: KeyvPostgresOptions) {
constructor(options?: KeyvPostgresOptions | string) {
super();
this.ttlSupport = false;
options = {
dialect: 'postgres',
uri: 'postgresql://localhost:5432', ...options,
};

if (typeof options === 'string') {
const uri = options;
options = {
dialect: 'postgres',
uri,
};
console.log('options', options);
} else {
options = {
dialect: 'postgres',
uri: 'postgresql://localhost:5432',
...options,
};
}

const connect = async () => {
const conn = pool(options!.uri!, options);
Expand All @@ -35,7 +46,7 @@ export class KeyvPostgres extends EventEmitter implements KeyvStoreAdapter {
...options,
};

let createTable = `CREATE TABLE IF NOT EXISTS ${this.opts.schema!}.${this.opts.table!}(key VARCHAR(${Number(this.opts.keySize!)}) PRIMARY KEY, value TEXT )`;
let createTable = `CREATE${this.opts.useUnloggedTable ? ' UNLOGGED ' : ' '}TABLE IF NOT EXISTS ${this.opts.schema!}.${this.opts.table!}(key VARCHAR(${Number(this.opts.keySize!)}) PRIMARY KEY, value TEXT )`;

if (this.opts.schema !== 'public') {
createTable = `CREATE SCHEMA IF NOT EXISTS ${this.opts.schema!}; ${createTable}`;
Expand Down
1 change: 1 addition & 0 deletions packages/postgres/src/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ export type KeyvPostgresOptions = {
ssl?: any;
dialect?: string;
iterationLimit?: number;
useUnloggedTable?: boolean;
} & PoolConfig;

export type Query = (sqlString: string, values?: any) => Promise<any[]>;
12 changes: 12 additions & 0 deletions packages/postgres/test/test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,12 @@ test.beforeEach(async () => {
await keyv.clear();
});

test.it('should be able to pass in just uri as string', async t => {
const keyv = new KeyvPostgres(postgresUri);
await keyv.set('foo', 'bar');
t.expect(await keyv.get('foo')).toBe('bar');
});

test.it('test schema as non public', async t => {
const keyv1 = new KeyvPostgres({uri: 'postgresql://postgres:postgres@localhost:5432/keyv_test', schema: 'keyvtest1'});
const keyv2 = new KeyvPostgres({uri: 'postgresql://postgres:postgres@localhost:5432/keyv_test', schema: 'keyvtest2'});
Expand Down Expand Up @@ -83,3 +89,9 @@ test.it('helper to create Keyv instance with postgres', async t => {
t.expect(await keyv.set('foo', 'bar')).toBe(true);
t.expect(await keyv.get('foo')).toBe('bar');
});

test.it('test unlogged table', async t => {
const keyv = createKeyv({uri: postgresUri, useUnloggedTable: true});
t.expect(await keyv.set('foo', 'bar')).toBe(true);
t.expect(await keyv.get('foo')).toBe('bar');
});
Loading