Skip to content

CurlHttpClient: tellp() failure in CurlHttpClient::WriteData with Boost filtering_stream #3364

@finch-lab

Description

@finch-lab

Describe the bug

Root Cause:
When using CurlHttpClient::WriteData with a boost::iostreams::filtering_stream (e.g., compression/decompression filters), the call to response->GetResponseBody().tellp() fails and logs:

Regression Issue

  • Select this option if this issue appears to be a regression.

Expected Behavior

This is expected behavior since boost::iostreams::filtering_stream with compression/decompression filters doesn't fully support the tellp() operation due to the non-linear nature of the data transformation.

Current Behavior

Detail:
I've encountered an issue in the AWS SDK(1.1.496) C++ code related to the use of tellp() in the CurlHttpClient::WriteData function.

When the response body is a boost::iostreams::filtering_stream (particularly with compression/decompression filters), the following code fails with an error:

auto cur = response->GetResponseBody().tellp();
if (response->GetResponseBody().fail()) {
    const auto& ref = response->GetResponseBody();
    AWS_LOGSTREAM_ERROR(CURL_HTTP_CLIENT_TAG, "Unable to query response output position (eof: "
            << ref.eof() << ", bad: " << ref.bad() << ")");
    return 0;
}

The error log shows: "Unable to query response output position (eof: 0, bad: 1)"

### Reproduction Steps

 **Reproduction Steps**

1. Create a response body stream using `boost::iostreams::filtering_stream` with compression filters:
```cpp
#include <aws/core/http/standard/StandardHttpResponse.h>
#include <boost/iostreams/filtering_stream.hpp>
#include <boost/iostreams/filter/gzip.hpp>

void ReproduceTellpFailure() {
    // Create a response with boost filtering_stream
    auto response = Aws::MakeShared<Aws::Http::Standard::StandardHttpResponse>(
        "TestResponse", Aws::Http::HttpRequest{}
    );
    
    // Setup compressed response stream
    boost::iostreams::filtering_ostream decompression_stream;
    decompression_stream.push(boost::iostreams::gzip_decompressor());  // Problematic filter
    decompression_stream.push(response->GetResponseBody());  // Connect to AWS response stream
    
    response->SetResponseBody(decompression_stream);

    // Simulate data write (typical usage pattern)
    decompression_stream.write("TestData", 8);
    decompression_stream.flush();

    // Trigger the tellp() check
    auto position = response->GetResponseBody().tellp();  // << Fails here
}

### Possible Solution

 It  will be possible to modify this code to check if the stream supports tellp() before attempting to use it, or use an alternative approach for tracking position? For example:

Use try/catch to handle unsupported tellp() operation
Skip position tracking for certain stream types
Use the existing m_numBytesResponseReceived counter instead of tellp()
This would improve compatibility with clients that use filtering_stream for compressed content handling.

Thank you for your consideration.

### Additional Information/Context

_No response_

### AWS CPP SDK version used

v1.11.496

### Compiler and Version used

gcc: 9.4.0

### Operating System and version

ubuntu20.04

Activity

added
bugThis issue is a bug.
needs-triageThis issue or PR still needs to be triaged.
on Apr 1, 2025
sbera87

sbera87 commented on Apr 1, 2025

@sbera87
Contributor

Have you tried chaining filter stream with std::stringstream buffer that is set originally in the request through
request.SetResponseStreamFactory(..)

std::stringstream buffer; // hook this with the response stream factory
boost::iostreams::filtering_ostream compression_stream;
compression_stream.push(boost::iostreams::gzip_compressor());
compression_stream.push(buffer)
added
response-requestedWaiting on additional info and feedback. Will move to "closing-soon" in 10 days.
on Apr 7, 2025
github-actions

github-actions commented on Apr 12, 2025

@github-actions

Greetings! It looks like this issue hasn’t been active in longer than a week. We encourage you to check if this is still an issue in the latest release. Because it has been longer than a week since the last update on this, and in the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please feel free to provide a comment or add an upvote to prevent automatic closure, or if the issue is already closed, please feel free to open a new one.

added
closing-soonThis issue will automatically close in 4 days unless further comments are made.
on Apr 12, 2025
added
investigatingThis issue is being investigated and/or work is in progress to resolve the issue.
feature-requestA feature should be added or improved.
and removed
closing-soonThis issue will automatically close in 4 days unless further comments are made.
needs-triageThis issue or PR still needs to be triaged.
response-requestedWaiting on additional info and feedback. Will move to "closing-soon" in 10 days.
on Apr 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

bugThis issue is a bug.feature-requestA feature should be added or improved.investigatingThis issue is being investigated and/or work is in progress to resolve the issue.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

    Development

    No branches or pull requests

      Participants

      @sbiscigl@SergeyRyabinin@finch-lab@sbera87

      Issue actions

        CurlHttpClient: tellp() failure in CurlHttpClient::WriteData with Boost filtering_stream · Issue #3364 · aws/aws-sdk-cpp