System.Stream, System.Socket: Partial reads & writes?

[logiclrd]

A common pattern with sockets is to set them to be Non-Blocking (fcntl with the O_NONBLOCK flag). In this mode, if a read or write is attempted and the full buffer size requested cannot be satisfied (a read/recv when fewer bytes are buffer, ready to be received, or a write/send when the available output buffer space cannot accommodate the full buffer provided), then the operation returns a successful response, indicating how many bytes were actually processed, and errno is set to EWOULDBLOCK.

This allows code to transfer as many bytes as it can without blocking the thread and without it being treated as an error state.

Currently, as far as I can tell, there is no way to express this in .NET with the BCL. The Socket implementations assume and require that the sockets be blocking. In the Socket code, if you supply a non-blocking socket, then a partial operation translates to a SocketException with SocketErrorCode == SocketError.WouldBlock, and the number of bytes that were actually processed is lost. In the Stream code, there is no Write family method that has a return type other than void; the API is fundamentally built around writes always fully completing.

I note that there was previously some discussion of partial write support specifically in #48826. The discussion there provided a specific rationale for partial write support: The client is expecting there to be more data to write very soon, and wants to avoid a separate send call for the remaining partial buffer right now, which could result in an extra TCP frame that is inefficiently conveying little data.

I feel like, more generally, this mode of operation is a core part of the way sockets and, on UNIX systems, file descriptors in general work, and should be reflected in the API.

[MihuBot]

I'm a bot. Here is a possible related and/or duplicate issue (I may be wrong):

• Make NetworkStream compatible with non-blocking I/O #22506

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

[logiclrd]

I'm a bot. Here is a possible related and/or duplicate issue (I may be wrong):

• Make NetworkStream compatible with non-blocking I/O #22506

This issue talks about related concepts, but spends a lot of time arguing over whether the interface of Stream could be modified to allow Write to have a non-void return (obviously it can't), and contains inaccurate assertions, such as that at the OS level, Windows does not support the socket non-blocking I/O model where send could do a partial send (it does), or that System.Socket already supports non-blocking I/O for send and receive (it doesn't, because if a partial operation occurs, it gets translated to a System.SocketException and you have no way to recover how many bytes were actually processed).

[wfurt]

The Socket.Send already returns int. So clearly it knows it can write less than asked for. Perhaps we should not throw in that case...??? AFAIK we internally already tracking the blocking status.

Also you can ask for available buffer on socket e.g. get info how much you can write without blocking. While that is still kernel call it is probably pretty cheap and it does not impact the TCP segmentation if you are worried about that. In practice I doubt that has any real impact. When you are getting to the point the socket buffer is full I doubt extra send would have impact on the TCP.

The Stream interface is different story. As high level API it would try to complete given task even if that means multiple IO operations. For WriteAsync, it always puts the socket in non-blocking mode and it deals with the outcome internally.

It would be curious to perhaps write some benchmark or provide C implementation that can demonstrate the benefits. That would help to prioritize this accordingly.

[logiclrd]

The Stream interface is different story. As high level API it would try to complete given task even if that means multiple IO operations.

It does not do this for Read, though. The ReadAtLeast and ReadExactly methods were added for the purpose of making Read have the same high-level semantics as Write. So, for symmetry, really, Write ought to return an int and support partial write operations, and there should be WriteAtLeast and WriteExactly methods. But, the ship has sailed on that one 🙂

Even if we can't have actual symmetry in the methods, though, we could at least have symmetry in the functionality that is offered. The counterpart to the relationship Read <-> ReadExactly could be WriteSome <-> Write.

For WriteAsync, it always puts the socket in non-blocking mode and it deals with the outcome internally.

Is this true?

I just went and took a peek at the code, and, well, the code is very convoluted, and I don't have a local copy to analyze with an IDE that understands the type space, but it uses Socket.SendAsync (and actually has its own specialized internal SendAsyncForNetworkStream method to massage some semantics). Task-based async operations are not intrinsically the same thing as O_NONBLOCK, and I haven't managed to find any place that sets the socket to be non-blocking in order to process SendAsync. Maybe it's there, but I couldn't find it. (I did find a UNIX-specific implementation of SendAsync that explicitly checks whether the socket is non-blocking and alters its behaviour accordingly.)

Even if it is, though, that internal implementation detail doesn't really address the fact that the user has no access to non-blocking partial writes.

[wfurt]

There is no partial Write, but what I was trying to say is that the Send does it and you can access it from the NetworkStream.
And while combining data together may be appealing it would be good to demonstrate the benefits.

[logiclrd]

I understand what you're saying, but I pulled the source code for Socket and NetworkStream and related classes and tried to find some link between NetworkStream and proper non-blocking I/O and couldn't find one. The Write method actually contains this comment (in one of the overloads):

                // Since the socket is in blocking mode this will always complete
                // after ALL the requested number of bytes was transferred.

And if you call WriteAsync, that funnels to SendAsyncForNetworkStream, which allocates an AwaitableSocketAsyncEventArgs and calls SendAsyncForNetworkStream, which calls Socket.SendAsync's SocketAsyncEventArgs overload, which calls SocketAsyncEventArgs.DoOperationSend. Here things diverge: on Windows it uses overlapped I/O, whereas on Linux, it calls SocketAsyncContext.SendAsync (UNIX-specific implementation), which calls SocketAsyncContext.SendToAsync, which calls SocketPal.TryCompleteSendTo.

Now, at this point, the UNIX-specific implementation does handle non-blocking I/O and partial writes -- if the socket is already set to be non-blocking. But, it's only exposed via the NetworkStream WriteAsync interface at the top end, which doesn't provide any representation for this.

So:

1. It never sets the socket to be non-blocking.
2. On Windows, it uses overlapped I/O.
3. On Linux, if you set the socket to be non-blocking, then it handles it under the hood, but
4. From the consuming code's point of view, that detail is completely hidden.

Unless I'm missing something??