Retry on WANT_WRITE & WANT_READ in sendall()

[webknjaz]

This change introduces retries in OpenSSL.SSL.Connection.sendall()
when WANT_WRITE_ERROR or WANT_READ_ERROR happen.

It relies on SSL_MODE_ENABLE_PARTIAL_WRITE being set on the context,
that changes the mode of SSL_write() to return errors only if zero
bytes has been sent making it safe to retry in these cases.

Ideally, the calling code is supposed to poll()/select() the
socket to know when it's okay to attempt the next retry (hence it is
readable or writable) but it's not available in the sendall() method
and just retrying the operation is good enough.

Fixes #176

Refs:

• http://openssl.6102.n7.nabble.com/SSL-MODE-ACCEPT-MOVING-WRITE-BUFFER-td6421.html
• https://stackoverflow.com/a/28992313/595220
• https://www.openssl.org/docs/manmaster/man3/SSL_write.html
• https://stackoverflow.com/a/20817394/595220

[reaperhulk]

Is there a way we can cover this with a test? And is SSL_MODE_ENABLE_PARTIAL_WRITE being set anywhere?

[webknjaz]

And is SSL_MODE_ENABLE_PARTIAL_WRITE being set anywhere?

It's set in the context initializer: https://github.com/pyca/pyopenssl/blob/124a013/src/OpenSSL/SSL.py#L660. Also, the docstring of send() seems to imply the same: https://github.com/pyca/pyopenssl/blob/124a013/src/OpenSSL/SSL.py#L1623-L1625 — I did not understand why is that at first but now it makes sense.

Is there a way we can cover this with a test?

I'm not sure how to properly trigger this in a stable manner at the moment. Although, the suspicion is that maybe if we could limit the buffer size of the test socket and feed a sufficiently big chunk of data into it, it should work.

pyopenssl/tests/test_ssl.py

Lines 2659 to 2690 in 124a013

	def test_wantWriteError(self):
	"""
	`Connection` methods which generate output raise
	`OpenSSL.SSL.WantWriteError` if writing to the connection's BIO
	fail indicating a should-write state.
	"""
	client_socket, server_socket = socket_pair()
	# Fill up the client's send buffer so Connection won't be able to write
	# anything. Only write a single byte at a time so we can be sure we
	# completely fill the buffer. Even though the socket API is allowed to
	# signal a short write via its return value it seems this doesn't
	# always happen on all platforms (FreeBSD and OS X particular) for the
	# very last bit of available buffer space.
	msg = b"x"
	for i in range(1024 * 1024 * 64):
	try:
	client_socket.send(msg)
	except error as e:
	if e.errno == EWOULDBLOCK:
	break
	raise
	else:
	pytest.fail(
	"Failed to fill socket buffer, cannot test BIO want write"
	)
	ctx = Context(SSLv23_METHOD)
	conn = Connection(ctx, client_socket)
	# Client's speak first, so make it an SSL client
	conn.set_connect_state()
	with pytest.raises(WantWriteError):
	conn.do_handshake()

seems to be doing this exactly.

[webknjaz]

I'm trying to come up with with a reproducer @ #955 but it still needs polishing.

[webknjaz]

Suggested change

	keep with sending data over the wire after ``SSL_ERROR_WANT_READ``
	keep sending data over the wire after ``SSL_ERROR_WANT_READ``

[julianz-]

The logic here seems a bit confusing. If result or bytes_written is > 0, then this was a successful partial or full write. No need to call _raise_ssl_error(). Just advance the buffers and continue. If bytes_written <= 0, then this was a failure and we need to see what kind of failure occurred by calling _raise_ssl_error(). The comment that the flag SSL_MODE_ENABLE_PARTIAL_WRITE guarantees no bytes have been written when there is a failure I think is misleading. Even without that flag after a WantReadError or WantWriteError, we can assume no bytes made it and we need to retry the same buffer again. All SSL_MODE_ENABLE_PARTIAL_WRITE does I think is allow for the possibility of partial successful writes.

So I would suggest something like:

bytes_written = _lib.SSL_write(
    self._ssl, data + total_sent, min(left_to_send, 2147483647)
)
if bytes_written > 0:
    # we had a successful partial or full write
    total_sent += bytes_written
    left_to_send -= bytes_written
else:
  # bytes_written <= 0 means there was a failure
  try:
      # Check for retryable errors (WantRead/WantWrite) or fatal errors.
      self._raise_ssl_error(self._ssl, bytes_written)
  except (WantReadError, WantWriteError):
      # Retryable error: Do nothing (total_sent remains unchanged).
      continue
  # Other errors propagate up.

[webknjaz]

@reaperhulk @alex do you have opinions on this suggestion?