Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use PROVIDED BUFFERS for reading from socket #3523

Closed
romange opened this issue Aug 17, 2024 · 1 comment
Closed

Use PROVIDED BUFFERS for reading from socket #3523

romange opened this issue Aug 17, 2024 · 1 comment
Assignees
Labels
enhancement New feature or request minor nice to have enhancement

Comments

@romange
Copy link
Collaborator

romange commented Aug 17, 2024

Currently, when a dragonfly connection reads from a socket, it must provide a buffer. The buffer can be as large as max_client_iobuf_len flag (64KB). So if we have 1000 connections blocking on socket read we must reserve upto 64MB of memory in advance. That's pretty wasteful because only a fraction of those connections are actually reading from the socket each time.

epoll has a differrent approach where the application is first notified on socket read event only only then performs recv call where it must provide a buffer for reading. This is how valkey, for example, manages qbuf globally instead of preallocating it per each connection.

To benefit from both CPU efficiency and memory efficiency, iouring came up with io_uring_register_buf_ring (provided buffers feature) starting from 5.19. It's a newer, more efficient alternative of the older IORING_OP_PROVIDE_BUFFERS call. We can register buffer rings in kernel and then have shared buffers being selected and used by kernel only when a socket has data for reading.
So we can register, for example 1000 buffers of 1K each, and ensure concurrent reading of 1000 connection with only 1MB of memory.

  1. We should design an interface in FiberSockerBase that works for both epoll and iouring and supports provided buffers feature
  2. We should start using it in Dragonfly.
  3. Bonus: start using bundles (reading into sequences of several provided buffers) for kernels >= 6.10
@romange romange added enhancement New feature or request minor nice to have enhancement labels Aug 17, 2024
@romange romange self-assigned this Aug 17, 2024
@romange
Copy link
Collaborator Author

romange commented Mar 15, 2025

Was mostly implemented by #4736. Needs additional testing but enough to loadtest it

@romange romange closed this as completed Mar 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request minor nice to have enhancement
Projects
None yet
Development

No branches or pull requests

1 participant