Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve TLog Streaming Memory Usage #7069

Open
sfc-gh-jslocum opened this issue May 5, 2022 · 0 comments
Open

Improve TLog Streaming Memory Usage #7069

sfc-gh-jslocum opened this issue May 5, 2022 · 0 comments

Comments

@sfc-gh-jslocum
Copy link
Collaborator

In release-7.1 testing an unrelated bug in simulation caused an OOM due to a large amount of TLog peek streams being opened by storage servers.
There are several memory optimizations implemented for change feed streams we could also implement for TLog streams:

  • Reply immediately with an empty version message when the stream is established, instead of waiting for the first set of mutations to reply. This finishes connection establishment and lets the client+server clean up the stream more quickly
  • In the merge cursor case of a single SS connecting to many TLogs, limit the memory used across the logical cursor instead of each of the individual TLog streams
  • Evaluate performance vs memory tradeoffs of alternative values of SERVER_KNOBS->MAXIMUM_PEEK_BYTES
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant