Codex crashes when stream is closed during download

[benbierens]

Seen in SwarmTest[20,20] (https://github.com/codex-storage/cs-codex-dist-tests/blob/master/Tests/CodexTests/DownloadConnectivityTests/SwarmTests.cs) on v0.1.8

Modification: Logging for nodes was turned up:

            var nodes = StartCodex(numNodes, s => s.WithLogLevel(CodexPlugin.CodexLogLevel.Trace,
                new CodexPlugin.CodexLogCustomTopics(CodexPlugin.CodexLogLevel.Warn, CodexPlugin.CodexLogLevel.Trace)));

Test runs 20 nodes with a 20MB file.
15 nodes successfully download the file.
4 nodes crash with the following error:

WRN 2024-11-04 08:52:59.762+00:00 Excepting streaming blocks                 topics="codex restapi" tid=1 exc="Unable to send response data, reason: Stream finished or remote side dropped connection" count=646927
INF 2024-11-04 08:52:59.762+00:00 Sent bytes                                 topics="codex restapi" tid=1 cid=zDv*Q223mw bytes=7667712 count=646928
ERR 2024-11-04 08:52:59.762+00:00 Unhandled exception in async proc, aborting topics="codex" tid=1 msg="/src/vendor/nim-chronos/chronos/apps/http/httpserver.nim(1422, 15) `currentState`gensym835 == HttpResponseState.Empty` Response body was already sent [Failed]" count=646929

It's possible but not likely this is caused by the download stream being interrupted. In either case the node should not crash.

[gmega]

I have the same problem. I was indeed closing the download stream due to a misconfigured socket read timeout, and this indeed causes Codex to crash pretty consistently.

[2-towns]

That could be related with the error Connection was closed before full request has been made in CI: https://github.com/codex-storage/nim-codex/actions/runs/13388233002/job/37389790340.

[gmega]

OK I think I know why this is happening, will get PR soon.

[gmega]

OK, looks like this might be a bug in presto, or in the way we're using the API. Will need to dig deeper.