FATAL no writing but write callback called - TcpConnectionImpl.cc:328

[prm-james-hill]

I’m using Drogon’s HttpClient but figured it would make more sense to put this here:

• Using v1.9.1 of Drogon and v1.5.15 of Trantor.
• It’s in an AWS environment that I can’t easily modify or put more verbose logging on. I’ve been unsuccessful in getting any kind of repro in a simpler environment.
• Not sure how to quantify it but I guess pretty infrequent. This env has ~50 TCP connections (as far as HttpClient is concerned) and this gets logged once every ~5,000,000 requests.
• Load on the server - and the client in particular - doesn’t seem to matter. It seems to happen specifically when the network is unstable.
  • If a sendRequest() fails with “Timeout” status then there’s no log.
  • If a sendRequest() fails with “Network failure” status then this log sometimes accompanies it.

I realize it’s a high ask to fix this under these conditions. I’m more curious if it’s safe to ignore this and if we can downgrade the log severity. I haven’t noticed any kind of memory corruption (though haven't actually had it happen under ASan) or resource exhaustion accompanying it.

(I also noticed this change from a couple hours ago. If that’s likely relevant I’m happy to upgrade and report back when it’s released.)

[an-tao]

@prm-james-hill Thanks so much for your feedback.
You can try the latest modifications, I'm currently simplifying the internal logic of this part and maybe it can fix this problem (#308). But don't check the #308 PR, it's still under development

[prm-james-hill]

Thanks for the context, An. If you’re in the process of doing more changes then I’ll just wait for them all to get released. And once I upgrade to that I can report back

[an-tao]

@prm-james-hill Would u please check out the PR #312 ?

[prm-james-hill]

Sorry for the delayed response. I have to hold off until it’s released. I realize that doesn’t really promise there aren’t bugs, but the only environment I can repro in is risky/difficult to change (especially this time of year)

[prm-james-hill]

Finally had a chance to try this out. I’m no longer seeing these on Trantor 1.5.17 (via Drogon 1.9.3). Thanks, An!