Skip to content

Move S3PostUploadSignedPolicy outside of _bcl folder #3900

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 6 commits into
base: development
Choose a base branch
from

Conversation

GarrettBeatty
Copy link
Contributor

@GarrettBeatty GarrettBeatty commented Jul 1, 2025

Description

  1. Move logic related to generating presigned post urls outside of _bcl folder so the signing logic can be used by netstandard and net core 3.1.
  2. Update json parsing logic to be rely only on System.Text.Json and to be native AOT compatible.
  3. For XML related functionality i did not think it was worth making it native aot compatible. i was having difficulty ( i tried following github.com/NativeAOT and XmlSerializer dotnet/runtime#106580)

Motivation and Context

In order to implement #1901 , we need to utilize the existing GetSignedPolicy function. It is currently in the _bcl folder because it relies on system.text.json.nodes which isnt available in .net core3.1 and the current code wasn't native aot compatible. So in order to fix this, we needed to remove the system.text.json.nodes dependency and move the code out of the _bcl folder.

Testing

In order to ensure it works on net standard and is native AOT compatible i created a console app

Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net8.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
    <PublishAot>true</PublishAot>
    <InvariantGlobalization>true</InvariantGlobalization>
  </PropertyGroup>
  
  <ItemGroup>
    <!-- Reference to the SDK S3 project -->
    <ProjectReference Include="..\sdk\src\Services\S3\AWSSDK.S3.NetStandard.csproj" />
  </ItemGroup>
</Project>

using Amazon;
using Amazon.Runtime;
using Amazon.S3.Util;
using System;
using System.Text.Json;

namespace S3PostAotDemo
{
    class Program
    {
        static void Main(string[] args)
        {
            // Create a sample S3 POST policy document
            // This policy allows uploads:
            // - To a specific bucket
            // - With a public-read ACL
            // - Keys that start with user/uploads/
            // - With content size between 1KB and 10MB
            string policyJson = @"{
  ""expiration"": ""2030-01-01T00:00:00Z"",
  ""conditions"": [
    {""bucket"": ""example-bucket""},
    {""acl"": ""public-read""},
    [""starts-with"", ""$key"", ""user/uploads/""],
    [""starts-with"", ""$Content-Type"", ""image/""],
    [""content-length-range"", 1024, 10485760]
  ]
}";

            Console.WriteLine("S3 Presigned POST Policy Demo (Native AOT)");
            Console.WriteLine("=========================================");
            Console.WriteLine();
            
            try
            {
                // Set up credentials and region
                // NOTE: In a real application, use environment variables, AWS SDK credential providers,
                // or other secure credential sources instead of hardcoded values
                var credentials = new BasicAWSCredentials("AKIAIOSFODNN7EXAMPLE", "test-secret-key");
                var region = RegionEndpoint.USWest1;

                // Get the signed policy
                Console.WriteLine("Generating presigned policy...");
                var signedPolicy = S3PostUploadSignedPolicy.GetSignedPolicy(policyJson, credentials, region);
                
                // Display results
                Console.WriteLine("\nSigned Policy Details:");
                Console.WriteLine($"Policy (Base64): {signedPolicy.Policy}");
                Console.WriteLine($"Signature: {signedPolicy.Signature}");
                Console.WriteLine($"AccessKeyId: {signedPolicy.AccessKeyId}");
                Console.WriteLine($"Algorithm: {signedPolicy.Algorithm}");
                Console.WriteLine($"Date: {signedPolicy.Date}");
                Console.WriteLine($"Credential: {signedPolicy.Credential}");
                
                // Display human-readable policy
                Console.WriteLine("\nHuman-readable policy:");
                Console.WriteLine(signedPolicy.GetReadablePolicy());
                
                // Show how to use this in an HTML form
                Console.WriteLine("\nSample HTML form fields:");
                Console.WriteLine($"<input type=\"hidden\" name=\"key\" value=\"user/uploads/${{filename}}\">");
                Console.WriteLine($"<input type=\"hidden\" name=\"acl\" value=\"public-read\">");
                Console.WriteLine($"<input type=\"hidden\" name=\"Content-Type\" value=\"image/jpeg\">");
                Console.WriteLine($"<input type=\"hidden\" name=\"bucket\" value=\"example-bucket\">");
                Console.WriteLine($"<input type=\"hidden\" name=\"X-Amz-Algorithm\" value=\"{signedPolicy.Algorithm}\">");
                Console.WriteLine($"<input type=\"hidden\" name=\"X-Amz-Credential\" value=\"{signedPolicy.Credential}\">");
                Console.WriteLine($"<input type=\"hidden\" name=\"X-Amz-Date\" value=\"{signedPolicy.Date}\">");
                Console.WriteLine($"<input type=\"hidden\" name=\"Policy\" value=\"{signedPolicy.Policy}\">");
                Console.WriteLine($"<input type=\"hidden\" name=\"X-Amz-Signature\" value=\"{signedPolicy.Signature}\">");
                Console.WriteLine($"<input type=\"file\" name=\"file\">");
                Console.WriteLine($"<input type=\"submit\" value=\"Upload\">");
                
                // Also demonstrate converting to JSON for use in JavaScript applications
                Console.WriteLine("\nJSON representation (for JavaScript apps):");
                string json = signedPolicy.ToJson();
                Console.WriteLine(json);
            }
            catch (Exception ex)
            {
                Console.WriteLine($"\nERROR: {ex.Message}");
                Console.WriteLine(ex.ToString());
            }
            
            Console.WriteLine("\nPress any key to exit...");
            Console.ReadKey();
        }
    }
}

PS C:\dev\repos\aws-sdk-net\S3PostAotDemo> dotnet publish -r win-x64 -c Release
Restore complete (1.9s)
You are using a preview version of .NET. See: https://aka.ms/dotnet-support-policy
  AWSSDK.Core.NetStandard net8.0 succeeded (16.7s) → C:\dev\repos\aws-sdk-net\sdk\src\Core\bin\Release\net8.0\AWSSDK.Core.dll
  AWSSDK.S3.NetStandard net8.0 succeeded (9.9s) → C:\dev\repos\aws-sdk-net\sdk\src\Services\S3\bin\Release\net8.0\AWSSDK.S3.dll
  S3PostAotDemo succeeded (11.6s) → bin\Release\net8.0\win-x64\publish\

Build succeeded in 43.4s
S3 Presigned POST Policy Demo (Native AOT)
=========================================

Generating presigned policy...

Signed Policy Details:
Policy (Base64): eyJleHBpcmF0aW9uIjoiMjAzMC0wMS0wMVQwMDowMDowMFoiLCJjb25kaXRpb25zIjpbeyJidWNrZXQiOiJleGFtcGxlLWJ1Y2tldCJ9LHsiYWNsIjoicHVibGljLXJlYWQifSxbInN0YXJ0cy13aXRoIiwiJGtleSIsInVzZXIvdXBsb2Fkcy8iXSxbInN0YXJ0cy13aXRoIiwiJENvbnRlbnQtVHlwZSIsImltYWdlLyJdLFsiY29udGVudC1sZW5ndGgtcmFuZ2UiLDEwMjQsMTA0ODU3NjBdLHsieC1hbXotY3JlZGVudGlhbCI6IkFLSUFJT1NGT0ROTjdFWEFNUExFLzIwMjUwNzAxL3VzLXdlc3QtMS9zMy9hd3M0X3JlcXVlc3QvIn0seyJ4LWFtei1hbGdvcml0aG0iOiJBV1M0LUhNQUMtU0hBMjU2In0seyJ4LWFtei1kYXRlIjoiMjAyNTA3MDFUMTcwNzAzWiJ9XX0=
Signature: 8838058f9f8a328616b3fa8bf42ddf6517429f155f1083d5a6f43eef0ac80049
AccessKeyId: AKIAIOSFODNN7EXAMPLE
Algorithm: AWS4-HMAC-SHA256
Date: 20250701T170703Z
Credential: AKIAIOSFODNN7EXAMPLE/20250701/us-west-1/s3/aws4_request/

Human-readable policy:
{"expiration":"2030-01-01T00:00:00Z","conditions":[{"bucket":"example-bucket"},{"acl":"public-read"},["starts-with","$key","user/uploads/"],["starts-with","$Content-Type","image/"],["content-length-range",1024,10485760],{"x-amz-credential":"AKIAIOSFODNN7EXAMPLE/20250701/us-west-1/s3/aws4_request/"},{"x-amz-algorithm":"AWS4-HMAC-SHA256"},{"x-amz-date":"20250701T170703Z"}]}

Sample HTML form fields:
<input type="hidden" name="key" value="user/uploads/${filename}">
<input type="hidden" name="acl" value="public-read">
<input type="hidden" name="Content-Type" value="image/jpeg">
<input type="hidden" name="bucket" value="example-bucket">
<input type="hidden" name="X-Amz-Algorithm" value="AWS4-HMAC-SHA256">
<input type="hidden" name="X-Amz-Credential" value="AKIAIOSFODNN7EXAMPLE/20250701/us-west-1/s3/aws4_request/">
<input type="hidden" name="X-Amz-Date" value="20250701T170703Z">
<input type="hidden" name="Policy" value="eyJleHBpcmF0aW9uIjoiMjAzMC0wMS0wMVQwMDowMDowMFoiLCJjb25kaXRpb25zIjpbeyJidWNrZXQiOiJleGFtcGxlLWJ1Y2tldCJ9LHsiYWNsIjoicHVibGljLXJlYWQifSxbInN0YXJ0cy13aXRoIiwiJGtleSIsInVzZXIvdXBsb2Fkcy8iXSxbInN0YXJ0cy13aXRoIiwiJENvbnRlbnQtVHlwZSIsImltYWdlLyJdLFsiY29udGVudC1sZW5ndGgtcmFuZ2UiLDEwMjQsMTA0ODU3NjBdLHsieC1hbXotY3JlZGVudGlhbCI6IkFLSUFJT1NGT0ROTjdFWEFNUExFLzIwMjUwNzAxL3VzLXdlc3QtMS9zMy9hd3M0X3JlcXVlc3QvIn0seyJ4LWFtei1hbGdvcml0aG0iOiJBV1M0LUhNQUMtU0hBMjU2In0seyJ4LWFtei1kYXRlIjoiMjAyNTA3MDFUMTcwNzAzWiJ9XX0=">
<input type="hidden" name="X-Amz-Signature" value="8838058f9f8a328616b3fa8bf42ddf6517429f155f1083d5a6f43eef0ac80049">
<input type="file" name="file">
<input type="submit" value="Upload">

JSON representation (for JavaScript apps):
{"policy":"eyJleHBpcmF0aW9uIjoiMjAzMC0wMS0wMVQwMDowMDowMFoiLCJjb25kaXRpb25zIjpbeyJidWNrZXQiOiJleGFtcGxlLWJ1Y2tldCJ9LHsiYWNsIjoicHVibGljLXJlYWQifSxbInN0YXJ0cy13aXRoIiwiJGtleSIsInVzZXIvdXBsb2Fkcy8iXSxbInN0YXJ0cy13aXRoIiwiJENvbnRlbnQtVHlwZSIsImltYWdlLyJdLFsiY29udGVudC1sZW5ndGgtcmFuZ2UiLDEwMjQsMTA0ODU3NjBdLHsieC1hbXotY3JlZGVudGlhbCI6IkFLSUFJT1NGT0ROTjdFWEFNUExFLzIwMjUwNzAxL3VzLXdlc3QtMS9zMy9hd3M0X3JlcXVlc3QvIn0seyJ4LWFtei1hbGdvcml0aG0iOiJBV1M0LUhNQUMtU0hBMjU2In0seyJ4LWFtei1kYXRlIjoiMjAyNTA3MDFUMTcwNzAzWiJ9XX0=","signature":"8838058f9f8a328616b3fa8bf42ddf6517429f155f1083d5a6f43eef0ac80049","access_key":"AKIAIOSFODNN7EXAMPLE"}

Dry run: fb816228-3d51-4f02-bff4-3711227d8874 -

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist

  • My code follows the code style of this project
  • My change requires a change to the documentation
  • I have updated the documentation accordingly
  • I have read the README document
  • [] I have added tests to cover my changes
  • All new and existing tests passed

License

  • I confirm that this pull request can be released under the Apache 2 license

@GarrettBeatty GarrettBeatty changed the title Move core logic of S3PostUploadSignedPolicy outside of _bcl folder Move S3PostUploadSignedPolicy outside of _bcl folder Jul 1, 2025
@GarrettBeatty GarrettBeatty force-pushed the gcbeatty/presignedpostbcl branch from 6c58f61 to dd78028 Compare July 8, 2025 02:34
@GarrettBeatty GarrettBeatty changed the base branch from gcbeatty/presignedpostpolicyunit to development July 8, 2025 02:34
Comment on lines +51 to +94
private static string
KEY_POLICY = "policy",
KEY_SIGNATURE = "signature",
KEY_ACCESSKEY = "access_key";

/// <summary>
/// The policy document which governs what uploads can be done.
/// </summary>
public string Policy { get; set; }

/// <summary>
/// The signature for the policy.
/// </summary>
public string Signature { get; set; }

/// <summary>
/// The AWS Access Key Id for the credential pair that produced the signature.
/// </summary>
public string AccessKeyId { get; set; }

/// <summary>
/// The security token from session or instance credentials.
/// </summary>
public string SecurityToken { get; set; }

/// <summary>
/// The signing algorithm used. Required as a field in the post Amazon
/// S3 can re-calculate the signature.
/// </summary>
public string Algorithm { get; set; }

/// <summary>
/// The date value in ISO8601 format. It is the same date used in
/// creating the signing key.
/// </summary>
public string Date { get; set; }

/// <summary>
/// In addition to the access key ID, this provides scope information
/// used in calculating the signing key for signature calculation.
/// </summary>
public string Credential { get; set; }


Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is all existing code, i just moved it to the top of file

/// Framework-agnostic implementation to add conditions to a policy document.
/// Uses JsonDocument/JsonElement instead of JsonNode.
/// </summary>
private static byte[] AddConditionsToPolicy(string policy, Dictionary<string, string> newConditions)
Copy link
Contributor Author

@GarrettBeatty GarrettBeatty Jul 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the only function that i really modified in this PR. the functionality is the same but it uses different logic to parse the json

Justification = "This suppression is here to ignore the warnings since we have not made this function AOT compatible yet.")]

#endif
public static S3PostUploadSignedPolicy GetSignedPolicyFromXml(string policyXml)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this function is the same, i just added the AssemblyLoadTrimming warning

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant