Hello!
Thank you for your work on this library — it’s really helpful.
I’ve encountered a limitation when using it for exporting a table from PostgreSQL to an external storage.
My use case is the following:
- Read the initial consistent snapshot of a table.
- Then continue consuming WAL using the library.
- The process may terminate at any arbitrary moment, after which replication is no longer needed.
- Therefore, the replication slot should be temporary.
What I would like to do on the PostgreSQL side is something like:
CREATE_REPLICATION_SLOT {replication_slot_name} TEMPORARY LOGICAL pgoutput EXPORT_SNAPSHOT;
This allows me to:
- Create a temporary logical replication slot.
- Obtain the exported snapshot.
- Read the initial state of the table.
- Then continue streaming changes from WAL once the snapshot is consumed.
However, I ran into the following limitation:
- I cannot create a temporary replication slot using
LogicalReplicationStream, which I need to use for streaming.
- The slot created internally by
LogicalReplicationStream is not temporary, so it remains after the application terminates.
- I could create a slot manually using
PgReplicationConnection, but in the case of a temporary slot, it is dropped when the connection is closed.
- Creating it on one connection and using it from
LogicalReplicationStream is not possible either, because a replication slot can only be used by a single backend.
Proposal
Please consider adding support for configuring replication slot parameters in LogicalReplicationStream, so that the created slot can be customized in the same way as it's done in create_replication_slot_with_options in PgReplicationConnection.
For example, allowing something like:
- Passing slot creation options into the constructor/builder.
- Or exposing slot creation configuration explicitly.
This would make it possible to use temporary logical replication slots in a clean and safe way.
Additionally, I would appreciate any suggested workaround that does not require significant refactoring, at least until such a feature is available.
Thank you!
Hello!
Thank you for your work on this library — it’s really helpful.
I’ve encountered a limitation when using it for exporting a table from PostgreSQL to an external storage.
My use case is the following:
What I would like to do on the PostgreSQL side is something like:
This allows me to:
However, I ran into the following limitation:
LogicalReplicationStream, which I need to use for streaming.LogicalReplicationStreamis not temporary, so it remains after the application terminates.PgReplicationConnection, but in the case of a temporary slot, it is dropped when the connection is closed.LogicalReplicationStreamis not possible either, because a replication slot can only be used by a single backend.Proposal
Please consider adding support for configuring replication slot parameters in
LogicalReplicationStream, so that the created slot can be customized in the same way as it's done increate_replication_slot_with_optionsinPgReplicationConnection.For example, allowing something like:
This would make it possible to use temporary logical replication slots in a clean and safe way.
Additionally, I would appreciate any suggested workaround that does not require significant refactoring, at least until such a feature is available.
Thank you!