[fix](logstash) Add HTTP timeouts and SO_KEEPALIVE to prevent pipeline hang (1.2.1 -> 1.2.2)#65765
Closed
smallhibiscus wants to merge 1 commit into
Closed
Conversation
…e hang (1.2.1 -> 1.2.2) ### What problem does this PR solve? Issue Number: close #xxx Related PR: apache#63181 Problem Summary: logstash-output-doris 1.2.0 hung forever on TCP half-open connections because Future.get() and HttpAsyncClient had no timeouts. apache#63181 migrated to HttpClient4 sync with NoConnectionReuseStrategy, which removes the Future.get() hang and avoids keep-alive connection reuse, but RequestConfig still lacked connect/socket timeouts and SO_KEEPALIVE was not enabled. Without socketTimeout, @client.execute() can still block forever when the peer dies mid-request. This patch (1.2.1 -> 1.2.2): - set connectTimeout / connectionRequestTimeout / socketTimeout on RequestConfig - enable SO_KEEPALIVE via SocketConfig - expose connect_timeout_ms, connection_request_timeout_ms, socket_timeout_ms as configurable options (defaults: 60s / 60s / 600s) ### Release note logstash-output-doris 1.2.2: add HTTP connect/socket timeouts and SO_KEEPALIVE to prevent worker threads hanging forever on TCP half-open connections. New optional configs: connect_timeout_ms, connection_request_timeout_ms, socket_timeout_ms. ### Check List (For Author) - Test: - Manual test / No need to test (config-only HttpClient setup; timeout behavior matches Apache HttpClient4 RequestConfig / SocketConfig) - Behavior changed: Yes (HTTP requests now fail with timeout instead of hanging) - Does this need documentation: No (README updated in this PR) Co-authored-by: Cursor <cursoragent@cursor.com>
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Issue Number: close #xxx
Related PR: #63181
Problem Summary:
logstash-output-doris1.2.0 could hang forever on TCP half-open connections because:Future.get()had no timeoutHttpAsyncClienthad no connect/response timeoutsSO_KEEPALIVEwas not enabled#63181 migrated the plugin to HttpClient4 sync with
NoConnectionReuseStrategy, which removes theFuture.get()hang and avoids keep-alive connection reuse. However,RequestConfigstill lacked connect/socket timeouts andSO_KEEPALIVEwas still off. WithoutsocketTimeout,@client.execute()can still block forever when the peer dies mid-request (same production symptom: worker threads stuck, Kafka input blocked on a full in-memory queue, pipeline throughput drops to 0).This patch (1.2.1 -> 1.2.2):
connectTimeout/connectionRequestTimeout/socketTimeoutonRequestConfigSO_KEEPALIVEviaSocketConfigconnect_timeout_ms,connection_request_timeout_ms,socket_timeout_msas configurable options (defaults: 60s / 60s / 600s)Release note
logstash-output-doris 1.2.2: add HTTP connect/socket timeouts and SO_KEEPALIVE to prevent worker threads hanging forever on TCP half-open connections. New optional configs:
connect_timeout_ms,connection_request_timeout_ms,socket_timeout_ms.Check List (For Author)