feat: put object with instruction file configured #466

akareddy04 · 2025-05-30T18:47:35Z

Issue #, if available: #355

Description of changes:
*Adds ability to add instruction file configuration for putObject

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Check any applicable:

Were any files moved? Moving files changes their URL, which breaks all hyperlinks to the files.

…a into inst-file-put

…r MPU + instructionFile

…tructionFileConfig

…equest to PutObjectRequest

…nabled

…uctionFile

… variable

kessplas · 2025-06-02T17:43:51Z

src/main/java/software/amazon/encryption/s3/internal/ContentMetadataEncodingStrategy.java

-// Copyright Amazon.com Inc. or its affiliates. All Rights Reserved.
-// SPDX-License-Identifier: Apache-2.0


We need to include these lines in every file, they should not be deleted here.

kessplas · 2025-06-02T17:54:07Z

src/main/java/software/amazon/encryption/s3/internal/ConvertSDKRequests.java

+                                    "To work around this issue you can disable multi part upload," +
+                                    "use the Async client, or not set this value on PutObject." +
+                                    "You may be able to update this value after the PutObject request completes."


Suggested change

"To work around this issue you can disable multi part upload," +

"use the Async client, or not set this value on PutObject." +

"You may be able to update this value after the PutObject request completes."

"To work around this issue you can disable Instruction File on PutObject or disable " +

"multi part upload, or use the Async client, or not set this value on PutObject." +

"You may be able to update this value after the request completes."

The text should be slightly different as it only applies to cases where InstructionFile puts are enabled.

kessplas · 2025-06-02T17:54:44Z

src/main/java/software/amazon/encryption/s3/internal/ConvertSDKRequests.java

+                    // Rather than silently dropping the value,
+                    // we loudly signal that we don't know how to handle this field.
+                    throw new IllegalArgumentException(
+                            f.locationName() + " is an unknown field. " +


Suggested change

f.locationName() + " is an unknown field. " +

f.memberName() + " is an unknown field. " +

Please fix this in the other convertRequest method, we missed that originally.

kessplas · 2025-06-02T17:55:34Z

src/main/java/software/amazon/encryption/s3/internal/ConvertSDKRequests.java

@@ -12,6 +12,131 @@

 public class ConvertSDKRequests {

+  public static PutObjectRequest convertRequest(CreateMultipartUploadRequest request) {


nit: We should add some comments here, i.e. that this is used to set the optional fields for Instruction File puts when the request is a multipart upload, because the instruction file is Put with an ordinary PutObject request.

And likewise for the other convertRequest method which is used to set the optional fields on high-level Multipart PutObject requests.

kessplas · 2025-06-02T18:04:25Z

src/main/java/software/amazon/encryption/s3/internal/MultipartUploadObjectPipeline.java

@@ -255,6 +262,7 @@ public MultipartUploadObjectPipeline build() {
                        .secureRandom(_secureRandom)
                        .build();
            }
+            _contentMetadataEncodingStrategy = new ContentMetadataEncodingStrategy(_instructionFileConfig);


Let's add a null check for instructionFileConfig, it should never be null but for robustness, we should check if null and create a default instruction file config.

kessplas · 2025-06-02T18:24:27Z

src/test/java/software/amazon/encryption/s3/S3EncryptionClientInstructionFileTest.java

+        final InputStream objectStreamForResult = new BoundedInputStream(fileSizeLimit);
+
+        S3Client wrappedClient = S3Client.create();
+        S3Client s3Client = S3EncryptionClient.builder()


We need to set .enableMultipartPutObject(true) to enable multipart put.

kessplas · 2025-06-02T18:26:11Z

src/test/java/software/amazon/encryption/s3/S3EncryptionClientInstructionFileTest.java

+                .bucket(BUCKET)
+                .key(object_key + ".instruction")
+                .build());
+        assertTrue(directInstGetResponse.response().metadata().containsKey("x-amz-crypto-instr-file"));


Let's also set StorageClass in the original put request and check for it here.

kessplas · 2025-06-02T18:26:59Z

src/test/java/software/amazon/encryption/s3/S3EncryptionClientInstructionFileTest.java

+
+
+        CreateMultipartUploadResponse initiateResult = v3Client.createMultipartUpload(builder ->
+                builder.bucket(BUCKET).key(object_key));


Let's also set StorageClass here and check for it in the directInstGetResponse later on.

kessplas · 2025-06-02T18:27:54Z

src/test/java/software/amazon/encryption/s3/internal/ConvertSDKRequestsTest.java

@@ -3,6 +3,7 @@
 import software.amazon.awssdk.awscore.AwsRequestOverrideConfiguration;
 import software.amazon.awssdk.services.s3.model.*;

+import java.awt.event.ComponentListener;


This looks unused

kessplas · 2025-06-02T18:29:00Z

src/test/java/software/amazon/encryption/s3/internal/ConvertSDKRequestsTest.java

+  }
+
+  @Test
+  public void testConversionAllFields() {


nit: Specify that this is for CreateMPUToPutObject

kessplas

Just a few minor things to fix

kessplas · 2025-06-04T23:19:42Z

src/main/java/software/amazon/encryption/s3/internal/ConvertSDKRequests.java

+    /*Converts a CreateMultipartUploadRequest into a PutObjectRequest by setting optional fields needed for
+    putInstructionFile operation.
+     */


This should be a Javadoc comment. Meaning, it comes before the method and it has this syntax (most importantly, the two *s in the first block):

/** * Comment here */ public whatever methodName() {

There are many examples throughout the codebase. Your IDE should also render them as green rather than grey.

In terms of content, it is not really accurate. We should specify that the conversion is used when Instruction File PutObject is enabled and the customer performs a multipart upload. putInstructionFile doesn't exist in v3.

kessplas · 2025-06-04T23:20:20Z

src/main/java/software/amazon/encryption/s3/internal/ConvertSDKRequests.java

@@ -138,7 +140,9 @@ public static PutObjectRequest convertRequest(CreateMultipartUploadRequest reque
  }

  public static CreateMultipartUploadRequest convertRequest(PutObjectRequest request) {
-
+     /*Converts a PutObjectRequest into a CreateMultipartUploadRequest by setting optional fields needed for high-level


kessplas · 2025-06-04T23:21:53Z

src/test/java/software/amazon/encryption/s3/S3AsyncEncryptionClientTest.java

@@ -45,6 +45,7 @@
 import software.amazon.awssdk.services.s3.model.PutObjectResponse;
 import software.amazon.awssdk.services.s3.model.S3Exception;
 import software.amazon.awssdk.services.s3.model.StorageClass;
+import software.amazon.awssdk.services.s3.model.StorageClassAnalysisDataExport;


I don't think this is used

kessplas

LGTM, lets get another review for this though.

kessplas and others added 20 commits November 25, 2024 14:05

step 1 - refactoring

8e51328

wip - instruction file puts

898c55e

cleanup

fd64f1c

Merge branch 'main' of github.com:aws/amazon-s3-encryption-client-jav…

95b3b5a

…a into inst-file-put

Merge branch 'main' of github.com:aws/amazon-s3-encryption-client-jav…

3546538

…a into inst-file-put

more tests in new class

ed761d9

Merge branch 'main' of github.com:aws/amazon-s3-encryption-client-jav…

a34b347

…a into inst-file-put

v3 to v2 tests for default client

d6da2cf

Merge branch 'main' of github.com:aws/amazon-s3-encryption-client-jav…

60feafd

…a into inst-file-put

Fixed nullptr exception in code

288ee65

testing whether v2 does or doesn't translate additional parameters fo…

9c1c50d

…r MPU + instructionFile

Fixed nullptr errors for MultipartUploadObjectPipeline regarding _ins…

ae6b6b4

…tructionFileConfig

Added another convertRequest method to convert CreateMultipartUploadR…

c9378cf

…equest to PutObjectRequest

Added test cases for converting MultipartUploadRequests

ef353d7

Added tests for S3EncryptonClient with Instruction File + Multipart e…

fbdbe92

…nabled

added .instructionFileConfig for PutEncryptedObjectPipeline builder

9e25d8f

cleaned up ContentMetadataEncodingStrategy class

7bf259c

Added test cases to S3AsyncEncryptionClientTest regarging MPU + instr…

ff5e99c

…uctionFile

fixed import issues

a396a7c

removed debugging lines

b1c0898

akareddy04 requested a review from a team as a code owner May 30, 2025 18:47

Anirav Kareddy added 2 commits May 30, 2025 12:47

Replaced passing in ARN of my personal testing bucket with the BUCKET…

b4b2047

… variable

removed unused import

b9ae158

kessplas requested changes Jun 2, 2025

View reviewed changes

Anirav Kareddy added 2 commits June 3, 2025 16:25

fixed all changes requested from PR

4f51c42

Removed HelloWorldProgramExample.java

bf2be6e

kessplas requested changes Jun 4, 2025

View reviewed changes

Anirav Kareddy added 2 commits June 6, 2025 09:21

Fixed the second PR round of reviews

02f6f48

testMultipartPutWithInstructionFile() should be working now

c319d44

kessplas approved these changes Jun 6, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: put object with instruction file configured #466

feat: put object with instruction file configured #466

Uh oh!

akareddy04 commented May 30, 2025 •

edited

Loading

Uh oh!

kessplas Jun 2, 2025

Uh oh!

kessplas Jun 2, 2025

Uh oh!

kessplas Jun 2, 2025

Uh oh!

kessplas Jun 2, 2025

Uh oh!

kessplas Jun 2, 2025

Uh oh!

kessplas Jun 2, 2025

Uh oh!

kessplas Jun 2, 2025

Uh oh!

kessplas Jun 2, 2025

Uh oh!

kessplas Jun 2, 2025

Uh oh!

kessplas Jun 2, 2025

Uh oh!

kessplas left a comment

Uh oh!

kessplas Jun 4, 2025

Uh oh!

kessplas Jun 4, 2025

Uh oh!

kessplas Jun 4, 2025

Uh oh!

kessplas left a comment

Uh oh!

Uh oh!

		// Copyright Amazon.com Inc. or its affiliates. All Rights Reserved.
		// SPDX-License-Identifier: Apache-2.0

	f.locationName() + " is an unknown field. " +
	f.memberName() + " is an unknown field. " +

		@@ -12,6 +12,131 @@

		public class ConvertSDKRequests {

		public static PutObjectRequest convertRequest(CreateMultipartUploadRequest request) {



		CreateMultipartUploadResponse initiateResult = v3Client.createMultipartUpload(builder ->
		builder.bucket(BUCKET).key(object_key));

feat: put object with instruction file configured #466

Are you sure you want to change the base?

feat: put object with instruction file configured #466

Uh oh!

Conversation

akareddy04 commented May 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Check any applicable:

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kessplas left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kessplas left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

akareddy04 commented May 30, 2025 •

edited

Loading