Can S3TransferManager check checksum for large file

[siukingkeung]

I tried on adding checksumSHA256 to PutObjectRequest to upload a file through S3TransferManager:

        val putObjectRequest = PutObjectRequest.builder()
            .bucket(s3BucketName)
            .key(objectName)
            .checksumSHA256(checksum)
            .build()

        val uploadFileRequest = UploadFileRequest.builder()
            .source(File(fileName))
            .putObjectRequest(putObjectRequest)
            .build()

        transferManager.uploadFile(uploadFileRequest)

It works fine for small file ( I even try input wrong checksum and S3TransferManager return error correctly). But when I try to test on a 5GB file, the following error is shown:

software.amazon.awssdk.core.exception.SdkClientException: Failed to send the request: Retry cannot be attempted because the maximum number of retries has been exceeded.
java.util.concurrent.CompletionException: software.amazon.awssdk.core.exception.SdkClientException: Failed to send the request: Retry cannot be attempted because the maximum number of retries has been exceeded.

Is STransferManager support checksum verification on large file?

[zoewangg]

Hi @siukingkeung, currently only MD5 checksum validation is enabled. Flexible checksum (CRC32, CRC32C, SHA-1, and SHA-256) support was added in #3376, but the change hasn't been released yet, and we intend to release it before GA.

[siukingkeung]

Thanks @zoewangg. Is there any plan for release on this feature?

[yasminetalby]

Hello @siukingkeung ,

Apologies for the delay of answer.
Unfortunately, we do not have any public information regarding our release schedules. We are currently active working on the release path of TransferManager v2 as a priority. When a feature is released the open git-hub issue is closed. Tracking these issue will be the most efficient way to be notified as soon as the feature is released.

Sincerely,

Yasmine

[github-actions[bot]]

Hello! Reopening this discussion to make it searchable.