TLS handshake fails with OpenSSL 3.0 clients

[NiQ1]

When connecting using the latest alpha version of OpenSSL 3.0 the connection fails. From a Wireshark capture I see that RabbitMQ resets the connection as soon as the client hello packet is received. This is not an OpenSSL issue since I'm having no difficulty connecting to any other TLS service using the same command.

OpenSSL logs -

C:\Projects\amd64\openssl-3.0.0-alpha13\apps>openssl` s_client -connect 127.0.0.1:5671 -debug
CONNECTED(0000012C)
write to 0x28cce125020 [0x28cce0ac990] (297 bytes => 297 (0x129))
0000 - 16 03 01 01 24 01 00 01-20 03 03 76 8d bc 0a f6   ....$... ..v....
0010 - 46 7d 64 0b 47 f3 72 cd-3b 09 03 11 1e 48 68 04   F}d.G.r.;....Hh.
0020 - 3f 6c e5 80 3b b0 b3 62-e4 97 a5 20 63 17 ae d4   ?l..;..b... c...
0030 - eb b0 70 22 7a 67 4d da-60 4f c1 bf b0 85 89 15   ..p"zgM.`O......
0040 - 53 7b a4 97 42 6a aa 61-da 68 b3 3d 00 3e 13 02   S{..Bj.a.h.=.>..
0050 - 13 03 13 01 c0 2c c0 30-00 9f cc a9 cc a8 cc aa   .....,.0........
0060 - c0 2b c0 2f 00 9e c0 24-c0 28 00 6b c0 23 c0 27   .+./...$.(.k.#.'
0070 - 00 67 c0 0a c0 14 00 39-c0 09 c0 13 00 33 00 9d   .g.....9.....3..
0080 - 00 9c 00 3d 00 3c 00 35-00 2f 00 ff 01 00 00 99   ...=.<.5./......
0090 - 00 0b 00 04 03 00 01 02-00 0a 00 16 00 14 00 1d   ................
00a0 - 00 17 00 1e 00 19 00 18-01 00 01 01 01 02 01 03   ................
00b0 - 01 04 00 23 00 00 00 16-00 00 00 17 00 00 00 0d   ...#............
00c0 - 00 2a 00 28 04 03 05 03-06 03 08 07 08 08 08 09   .*.(............
00d0 - 08 0a 08 0b 08 04 08 05-08 06 04 01 05 01 06 01   ................
00e0 - 03 03 03 01 03 02 04 02-05 02 06 02 00 2b 00 09   .............+..
00f0 - 08 03 04 03 03 03 02 03-01 00 2d 00 02 01 01 00   ..........-.....
0100 - 33 00 26 00 24 00 1d 00-20 6d 36 ca 98 a7 69 c5   3.&.$... m6...i.
0110 - 15 38 89 c1 ec 51 02 f7-a8 25 b9 d9 cf 13 03 a1   .8...Q...%......
0120 - 75 0c 8f 52 9b d6 98 e8-35                        u..R....5
read from 0x28cce125020 [0x28cce270083] (5 bytes => -1 (0xFFFFFFFF))
write:errno=10054
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 297 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
---
read from 0x28cce125020 [0x28cce064500] (8192 bytes => -1 (0xFFFFFFFF))

C:\Projects\amd64\openssl-3.0.0-alpha13\apps>

[michaelklishin]

I will convert this issue to a GitHub discussion. Currently GitHub will automatically close and lock the issue even though your question will be transferred and responded to elsewhere. This is to let you know that we do not intend to ignore this but this is how the current GitHub conversion mechanism makes it seem for the users :(

[michaelklishin]

errno=10054 is OpenSSL speak for "connection refused".

RabbitMQ does not implement TLS, Erlang/OTP does. If this can be reproduced against Erlang 23.3 or 24 (current master), please let the Erlang/OTP team know. Unfortunately Erlang 24 support in RabbitMQ 3.8 will have to wait until next month.