You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, when a dragonfly connection reads from a socket, it must provide a buffer. The buffer can be as large as max_client_iobuf_len flag (64KB). So if we have 1000 connections blocking on socket read we must reserve upto 64MB of memory in advance. That's pretty wasteful because only a fraction of those connections are actually reading from the socket each time.
epoll has a differrent approach where the application is first notified on socket read event only only then performs recv call where it must provide a buffer for reading. This is how valkey, for example, manages qbuf globally instead of preallocating it per each connection.
To benefit from both CPU efficiency and memory efficiency, iouring came up with io_uring_register_buf_ring (provided buffers feature) starting from 5.19. It's a newer, more efficient alternative of the older IORING_OP_PROVIDE_BUFFERS call. We can register buffer rings in kernel and then have shared buffers being selected and used by kernel only when a socket has data for reading.
So we can register, for example 1000 buffers of 1K each, and ensure concurrent reading of 1000 connection with only 1MB of memory.
We should design an interface in FiberSockerBase that works for both epoll and iouring and supports provided buffers feature
We should start using it in Dragonfly.
Bonus: start using bundles (reading into sequences of several provided buffers) for kernels >= 6.10
The text was updated successfully, but these errors were encountered:
Currently, when a dragonfly connection reads from a socket, it must provide a buffer. The buffer can be as large as
max_client_iobuf_len
flag (64KB). So if we have 1000 connections blocking on socket read we must reserve upto 64MB of memory in advance. That's pretty wasteful because only a fraction of those connections are actually reading from the socket each time.epoll has a differrent approach where the application is first notified on socket read event only only then performs
recv
call where it must provide a buffer for reading. This is how valkey, for example, manages qbuf globally instead of preallocating it per each connection.To benefit from both CPU efficiency and memory efficiency, iouring came up with
io_uring_register_buf_ring
(provided buffers feature) starting from 5.19. It's a newer, more efficient alternative of the older IORING_OP_PROVIDE_BUFFERS call. We can register buffer rings in kernel and then have shared buffers being selected and used by kernel only when a socket has data for reading.So we can register, for example 1000 buffers of 1K each, and ensure concurrent reading of 1000 connection with only 1MB of memory.
The text was updated successfully, but these errors were encountered: