Skip to content

refactor(replication): use replication origin to track durable progress - #1768

Open
meskill wants to merge 1 commit into
mainfrom
meskill-2026-10-05-refactor-replication---vsr
Open

meskill wants to merge 1 commit into
mainfrom
meskill-2026-10-05-refactor-replication---vsr

Conversation

@meskill

@meskill meskill commented Oct 9, 2026

Copy link
Copy Markdown
Contributor

fixes #1698

Use replications origins on destination shards to track durable lsn on it instead of inferring it from flushed wal and manual lsn tracking.

The new flow:

  1. on replication start the origins are created from every source to every destination shards with the name calculated from replication slot
  2. the destination transactions are marked with source lsn before commit
  3. separate thread runs the origin progress check and the last flushed source lsn
  4. the math calculates minimum of all destination shards and it's flushed source lsn and marks it as confirmed

@codecov

codecov Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

@levkk levkk left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀 🚀 super cool

@meskill
meskill requested a review from jkaczman October 9, 2026 21:27
@meskill

meskill commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor Author

Bench compare to make sure it's not worse, against latest main

Screenshot 2026-10-10 005336 image

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Resharding] use PG's replication progress tracking during replication

2 participants