(short issue description)Support Virtual Host Style Naming for custom S3 Endpoints

[vardhanv]

Describe the feature

Currently, in the S3 AWS SDK, there are only two ways to provide an S3 endpoint.

1. Default (amazonaws.com)
2. Custom endpoint

One of the problems with a (2), is that it does not support regions when the endpoint URL is generated.

For example, when the following configuration is used with Virtual Host Style naming

Endpoint: https://foo.com
Bucket: b1
Region: r1

It will generate S3 requests to b1.foo.com. Is there a way, or can a feature be supported to generate requests to b1.r1.foo.com, just as the SDK does when the endpoint is not specified.

Use Case

Be able to use regions with custom endpoints

Proposed Solution

When the following configuration is used with Virtual Host Style naming

Endpoint: https://foo.com
Bucket: b1
Region: r1

Generate requests to b1.r1.foo.com, just as the SDK does when the default endpoint is used.

Other Information

No response

Acknowledgements

• I may be able to implement this feature request
• This feature might incur a breaking change

AWS Java SDK version used

sdk-java-v2

JDK version used

Unknown

Operating System and version

Unknown

[debora-ito]

Adding the region to the custom endpointOverride should work, it doesn't?

    S3Client s3Client = S3Client.builder()
        .endpointOverride(URI.create("https://s3.us-west-2.amazonaws.com"))
        .region(Region.US_WEST_2)
        .build();

    ListObjectsResponse response = s3Client.listObjects(r -> r.bucket("debora-bucket"));

Request logs (see host):

- Sending Request: DefaultSdkHttpFullRequest(httpMethod=GET, protocol=https, host=debora-bucket.s3.us-west-2.amazonaws.com, encodedPath=, headers=[amz-sdk-invocation-id, User-Agent], queryParameters=[])
- Received successful response: 200, Request ID: xxx, Extended Request ID: xxx

[vardhanv]

Hello,
Thank you for your response.

This (addition of us-west-2 to the URI) host=debora-bucket.s3.us-west-2.amazonaws.com only works when the URI is amazonaws.com. When a custom endpoint is specified only the bucket is prefixed to the URI, not the region.

The question is - How can we make it work for a different endpoint.

I.e if the URI was example.com, the generated host would be debora-bucket.example.com. I am looking to create debora-bucket.my-region.example.com

[debora-ito]

Are you saying that the URI is generated as example.com and it can't be modified to my-region.example.com? Because the following should work fine:

S3Client s3Client = S3Client.builder()
        .endpointOverride(URI.create("https://my-region.example.com"))
        .build();

ListObjectsResponse response = s3Client.listObjects(r -> r.bucket("debora-bucket"));

Sending Request: DefaultSdkHttpFullRequest(httpMethod=GET, protocol=https, host=debora-bucket.my-region.example.com, encodedPath=, headers=[amz-sdk-invocation-id, User-Agent], queryParameters=[])

Now, if you can't modify the URI, can you share a code snippet showing how are you creating the client, so we can have a little more context of your current use case?

Note that virtual host style is not supported when the bucket name contains dots, the SDK will automatically use path style, if this is the issue you're running into.

[github-actions]

It looks like this issue has not been active for more than five days. In the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please add a comment to prevent automatic closure, or if the issue is already closed please feel free to reopen it.