Skip to content

Commit 2e7f295

Browse files
imabhichowimabhichow
committed
chore(python): Plaintext migration examples
1 parent 99a32aa commit 2e7f295

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

63 files changed

+3032
-2
lines changed

‎.github/workflows/ci_examples_python.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -96,3 +96,5 @@ jobs:
9696
tox -e dynamodbencryption
9797
# Run legacy migration examples
9898
tox -e legacymigration
99+
# Run plaintext migration examples
100+
tox -e plaintextmigration

‎Examples/runtimes/python/Migration/plaintext_to_awsdbe/__init__.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
# Copyright Amazon.com Inc. or its affiliates. All Rights Reserved.
2+
# SPDX-License-Identifier: Apache-2.0
3+
"""Stub to allow relative imports of examples from tests."""

‎Examples/runtimes/python/Migration/plaintext_to_awsdbe/src/README.md

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
# Plaintext to AWS Database Encryption SDK for DynamoDB Migration
2+
3+
This project demonstrates the steps necessary
4+
to migrate to the AWS Database Encryption SDK for DynamoDb
5+
from a plaintext database.
6+
7+
[Step 0](./plaintext/README.md) demonstrates the starting state for your system.
8+
9+
## Step 1
10+
11+
In Step 1, you update your system to do the following:
12+
13+
- continue to read plaintext items
14+
- continue to write plaintext items
15+
- prepare to read encrypted items
16+
17+
When you deploy changes in Step 1,
18+
you should not expect any behavior change in your system,
19+
and your dataset still consists of plaintext data.
20+
21+
You must ensure that the changes in Step 1 make it to all your readers before you proceed to Step 2.
22+
23+
## Step 2
24+
25+
In Step 2, you update your system to do the following:
26+
27+
- continue to read plaintext items
28+
- start writing encrypted items
29+
- continue to read encrypted items
30+
31+
Before you deploy changes in Step 2,
32+
you are introducing encrypted items to your system,
33+
and must make sure that all your readers are updated with the changes from Step 1.
34+
35+
Before you move onto the next step, you will need to encrypt all plaintext items in your dataset.
36+
Once you have completed this step,
37+
while new items are encrypted and will be authenticated on read,
38+
your system will still accept reading plaintext, unauthenticated items.
39+
In order to complete migration to a system where you always authenticate your items,
40+
you should prioritize moving on to Step 3.
41+
42+
## Step 3
43+
44+
Once all plaintext items are encrypted,
45+
update your system to do the following:
46+
47+
- continue to write encrypted items
48+
- continue to read encrypted items
49+
- do not accept reading plaintext items
50+
51+
Once you have deployed these changes to your system, you have completed migration.

‎Examples/runtimes/python/Migration/plaintext_to_awsdbe/src/__init__.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
# Copyright Amazon.com Inc. or its affiliates. All Rights Reserved.
2+
# SPDX-License-Identifier: Apache-2.0
3+
"""Stub to allow relative imports of examples from tests."""

‎Examples/runtimes/python/Migration/plaintext_to_awsdbe/src/awsdbe/__init__.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
# Copyright Amazon.com Inc. or its affiliates. All Rights Reserved.
2+
# SPDX-License-Identifier: Apache-2.0
3+
"""Stub to allow relative imports of examples from tests."""

‎Examples/runtimes/python/Migration/plaintext_to_awsdbe/src/awsdbe/client/__init__.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
# Copyright Amazon.com Inc. or its affiliates. All Rights Reserved.
2+
# SPDX-License-Identifier: Apache-2.0
3+
"""Stub to allow relative imports of examples from tests."""

‎Examples/runtimes/python/Migration/plaintext_to_awsdbe/src/awsdbe/client/common.py

Lines changed: 206 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,206 @@
1+
# Copyright Amazon.com Inc. or its affiliates. All Rights Reserved.
2+
# SPDX-License-Identifier: Apache-2.0
3+
"""Common Utilities for Migration Examples."""
4+
import boto3
5+
from aws_cryptographic_material_providers.mpl import AwsCryptographicMaterialProviders
6+
from aws_cryptographic_material_providers.mpl.config import MaterialProvidersConfig
7+
from aws_cryptographic_material_providers.mpl.models import (
8+
CreateAwsKmsMrkMultiKeyringInput,
9+
DBEAlgorithmSuiteId,
10+
)
11+
from aws_cryptographic_material_providers.mpl.references import IKeyring
12+
from aws_dbesdk_dynamodb.encrypted.client import EncryptedClient
13+
from aws_dbesdk_dynamodb.structures.dynamodb import (
14+
DynamoDbTableEncryptionConfig,
15+
DynamoDbTablesEncryptionConfig,
16+
)
17+
from aws_dbesdk_dynamodb.structures.structured_encryption import (
18+
CryptoAction,
19+
)
20+
21+
22+
def setup_awsdbe_client_without_plaintext_override(kms_key_id: str, ddb_table_name: str):
23+
"""
24+
Set up a pure AWS Database Encryption SDK EncryptedClient without plaintext override.
25+
26+
:param kms_key_id: The ARN of the KMS key to use for encryption
27+
:param ddb_table_name: The name of the DynamoDB table
28+
:returns EncryptedClient for DynamoDB
29+
"""
30+
# 1. Create a Keyring. This Keyring will be responsible for protecting the data keys that protect your data.
31+
# For this example, we will create a AWS KMS Keyring with the AWS KMS Key we want to use.
32+
# We will use the `CreateMrkMultiKeyring` method to create this keyring,
33+
# as it will correctly handle both single region and Multi-Region KMS Keys.
34+
mat_prov: AwsCryptographicMaterialProviders = AwsCryptographicMaterialProviders(config=MaterialProvidersConfig())
35+
kms_mrk_multi_keyring_input: CreateAwsKmsMrkMultiKeyringInput = CreateAwsKmsMrkMultiKeyringInput(
36+
generator=kms_key_id,
37+
)
38+
kms_mrk_multi_keyring: IKeyring = mat_prov.create_aws_kms_mrk_multi_keyring(input=kms_mrk_multi_keyring_input)
39+
40+
# 2. Configure which attributes are encrypted and/or signed when writing new items.
41+
# For each attribute that may exist on the items we plan to write to our DynamoDbTable,
42+
# we must explicitly configure how they should be treated during item encryption:
43+
# - ENCRYPT_AND_SIGN: The attribute is encrypted and included in the signature
44+
# - SIGN_ONLY: The attribute not encrypted, but is still included in the signature
45+
# - DO_NOTHING: The attribute is not encrypted and not included in the signature
46+
attribute_actions_on_encrypt = {
47+
"partition_key": CryptoAction.SIGN_ONLY,
48+
"sort_key": CryptoAction.SIGN_ONLY,
49+
"attribute1": CryptoAction.ENCRYPT_AND_SIGN,
50+
"attribute2": CryptoAction.SIGN_ONLY,
51+
":attribute3": CryptoAction.DO_NOTHING,
52+
}
53+
54+
# 3. Configure which attributes we expect to be included in the signature
55+
# when reading items. There are two options for configuring this:
56+
#
57+
# - (Recommended) Configure `allowedUnsignedAttributesPrefix`:
58+
# When defining your DynamoDb schema and deciding on attribute names,
59+
# choose a distinguishing prefix (such as ":") for all attributes that
60+
# you do not want to include in the signature.
61+
# This has two main benefits:
62+
# - It is easier to reason about the security and authenticity of data within your item
63+
# when all unauthenticated data is easily distinguishable by their attribute name.
64+
# - If you need to add new unauthenticated attributes in the future,
65+
# you can easily make the corresponding update to your `attributeActionsOnEncrypt`
66+
# and immediately start writing to that new attribute, without
67+
# any other configuration update needed.
68+
# Once you configure this field, it is not safe to update it.
69+
#
70+
# - Configure `allowedUnsignedAttributes`: You may also explicitly list
71+
# a set of attributes that should be considered unauthenticated when encountered
72+
# on read. Be careful if you use this configuration. Do not remove an attribute
73+
# name from this configuration, even if you are no longer writing with that attribute,
74+
# as old items may still include this attribute, and our configuration needs to know
75+
# to continue to exclude this attribute from the signature scope.
76+
# If you add new attribute names to this field, you must first deploy the update to this
77+
# field to all readers in your host fleet before deploying the update to start writing
78+
# with that new attribute.
79+
#
80+
# For this example, we have designed our DynamoDb table such that any attribute name with
81+
# the ":" prefix should be considered unauthenticated.
82+
unsignAttrPrefix: str = ":"
83+
84+
# 4. Create the DynamoDb Encryption configuration for the table we will be writing to.
85+
table_configs = {}
86+
table_config = DynamoDbTableEncryptionConfig(
87+
logical_table_name=ddb_table_name,
88+
partition_key_name="partition_key",
89+
sort_key_name="sort_key",
90+
attribute_actions_on_encrypt=attribute_actions_on_encrypt,
91+
keyring=kms_mrk_multi_keyring,
92+
allowed_unsigned_attribute_prefix=unsignAttrPrefix,
93+
# Specifying an algorithm suite is not required,
94+
# but is done here to demonstrate how to do so.
95+
# We suggest using the
96+
# `ALG_AES_256_GCM_HKDF_SHA512_COMMIT_KEY_ECDSA_P384_SYMSIG_HMAC_SHA384` suite,
97+
# which includes AES-GCM with key derivation, signing, and key commitment.
98+
# This is also the default algorithm suite if one is not specified in this config.
99+
# For more information on supported algorithm suites, see:
100+
# https://docs.aws.amazon.com/database-encryption-sdk/latest/devguide/supported-algorithms.html
101+
algorithm_suite_id=DBEAlgorithmSuiteId.ALG_AES_256_GCM_HKDF_SHA512_COMMIT_KEY_ECDSA_P384_SYMSIG_HMAC_SHA384,
102+
)
103+
table_configs[ddb_table_name] = table_config
104+
tables_config = DynamoDbTablesEncryptionConfig(table_encryption_configs=table_configs)
105+
106+
# 5. Create the EncryptedClient
107+
return EncryptedClient(
108+
client=boto3.client("dynamodb"),
109+
encryption_config=tables_config,
110+
)
111+
112+
113+
def setup_awsdbe_client_with_plaintext_override(kms_key_id: str, ddb_table_name: str, policy: str):
114+
"""
115+
Set up an AWS Database Encryption SDK EncryptedClient with plaintext override.
116+
117+
:param kms_key_id: The ARN of the KMS key to use for encryption
118+
:param ddb_table_name: The name of the DynamoDB table
119+
:param policy: The policy required for the Plaintext Override configuration
120+
:returns EncryptedClient for DynamoDB
121+
122+
"""
123+
# 1. Create a Keyring. This Keyring will be responsible for protecting the data keys that protect your data.
124+
# For this example, we will create a AWS KMS Keyring with the AWS KMS Key we want to use.
125+
# We will use the `CreateMrkMultiKeyring` method to create this keyring,
126+
# as it will correctly handle both single region and Multi-Region KMS Keys.
127+
mat_prov: AwsCryptographicMaterialProviders = AwsCryptographicMaterialProviders(config=MaterialProvidersConfig())
128+
kms_mrk_multi_keyring_input: CreateAwsKmsMrkMultiKeyringInput = CreateAwsKmsMrkMultiKeyringInput(
129+
generator=kms_key_id,
130+
)
131+
kms_mrk_multi_keyring: IKeyring = mat_prov.create_aws_kms_mrk_multi_keyring(input=kms_mrk_multi_keyring_input)
132+
133+
# 2. Configure which attributes are encrypted and/or signed when writing new items.
134+
# For each attribute that may exist on the items we plan to write to our DynamoDbTable,
135+
# we must explicitly configure how they should be treated during item encryption:
136+
# - ENCRYPT_AND_SIGN: The attribute is encrypted and included in the signature
137+
# - SIGN_ONLY: The attribute not encrypted, but is still included in the signature
138+
# - DO_NOTHING: The attribute is not encrypted and not included in the signature
139+
attribute_actions_on_encrypt = {
140+
"partition_key": CryptoAction.SIGN_ONLY,
141+
"sort_key": CryptoAction.SIGN_ONLY,
142+
"attribute1": CryptoAction.ENCRYPT_AND_SIGN,
143+
"attribute2": CryptoAction.SIGN_ONLY,
144+
":attribute3": CryptoAction.DO_NOTHING,
145+
}
146+
147+
# 3. Configure which attributes we expect to be included in the signature
148+
# when reading items. There are two options for configuring this:
149+
#
150+
# - (Recommended) Configure `allowedUnsignedAttributesPrefix`:
151+
# When defining your DynamoDb schema and deciding on attribute names,
152+
# choose a distinguishing prefix (such as ":") for all attributes that
153+
# you do not want to include in the signature.
154+
# This has two main benefits:
155+
# - It is easier to reason about the security and authenticity of data within your item
156+
# when all unauthenticated data is easily distinguishable by their attribute name.
157+
# - If you need to add new unauthenticated attributes in the future,
158+
# you can easily make the corresponding update to your `attributeActionsOnEncrypt`
159+
# and immediately start writing to that new attribute, without
160+
# any other configuration update needed.
161+
# Once you configure this field, it is not safe to update it.
162+
#
163+
# - Configure `allowedUnsignedAttributes`: You may also explicitly list
164+
# a set of attributes that should be considered unauthenticated when encountered
165+
# on read. Be careful if you use this configuration. Do not remove an attribute
166+
# name from this configuration, even if you are no longer writing with that attribute,
167+
# as old items may still include this attribute, and our configuration needs to know
168+
# to continue to exclude this attribute from the signature scope.
169+
# If you add new attribute names to this field, you must first deploy the update to this
170+
# field to all readers in your host fleet before deploying the update to start writing
171+
# with that new attribute.
172+
#
173+
# For this example, we have designed our DynamoDb table such that any attribute name with
174+
# the ":" prefix should be considered unauthenticated.
175+
unsignAttrPrefix: str = ":"
176+
177+
# 4. Create the DynamoDb Encryption configuration for the table we will be writing to.
178+
# Include the plaintext override with the provided policy
179+
table_configs = {}
180+
table_config = DynamoDbTableEncryptionConfig(
181+
logical_table_name=ddb_table_name,
182+
partition_key_name="partition_key",
183+
sort_key_name="sort_key",
184+
attribute_actions_on_encrypt=attribute_actions_on_encrypt,
185+
keyring=kms_mrk_multi_keyring,
186+
# Provide plaintext_override policy to the config here
187+
plaintext_override=policy,
188+
allowed_unsigned_attribute_prefix=unsignAttrPrefix,
189+
# Specifying an algorithm suite is not required,
190+
# but is done here to demonstrate how to do so.
191+
# We suggest using the
192+
# `ALG_AES_256_GCM_HKDF_SHA512_COMMIT_KEY_ECDSA_P384_SYMSIG_HMAC_SHA384` suite,
193+
# which includes AES-GCM with key derivation, signing, and key commitment.
194+
# This is also the default algorithm suite if one is not specified in this config.
195+
# For more information on supported algorithm suites, see:
196+
# https://docs.aws.amazon.com/database-encryption-sdk/latest/devguide/supported-algorithms.html
197+
algorithm_suite_id=DBEAlgorithmSuiteId.ALG_AES_256_GCM_HKDF_SHA512_COMMIT_KEY_ECDSA_P384_SYMSIG_HMAC_SHA384,
198+
)
199+
table_configs[ddb_table_name] = table_config
200+
tables_config = DynamoDbTablesEncryptionConfig(table_encryption_configs=table_configs)
201+
202+
# 5. Create the EncryptedClient
203+
return EncryptedClient(
204+
client=boto3.client("dynamodb"),
205+
encryption_config=tables_config,
206+
)

‎Examples/runtimes/python/Migration/plaintext_to_awsdbe/src/awsdbe/client/migration_step_1.py

Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
# Copyright Amazon.com Inc. or its affiliates. All Rights Reserved.
2+
# SPDX-License-Identifier: Apache-2.0
3+
4+
"""
5+
Migration Step 1.
6+
7+
This is an example demonstrating how to start using the
8+
AWS Database Encryption SDK with a pre-existing table with plaintext items.
9+
In this example, we configure an EncryptedClient to do the following:
10+
- Write items only in plaintext
11+
- Read items in plaintext or, if the item is encrypted, decrypt with our encryption configuration
12+
13+
While this step configures your client to be ready to start reading encrypted items,
14+
we do not yet expect to be reading any encrypted items,
15+
as our client still writes plaintext items.
16+
Before you move on to step 2, ensure that these changes have successfully been deployed
17+
to all of your readers.
18+
19+
Running this example requires access to the DDB Table whose name
20+
is provided in CLI arguments.
21+
This table must be configured with the following
22+
primary key configuration:
23+
- Partition key is named "partition_key" with type (S)
24+
- Sort key is named "sort_key" with type (S)
25+
"""
26+
from aws_dbesdk_dynamodb.structures.dynamodb import PlaintextOverride
27+
28+
from .common import setup_awsdbe_client_with_plaintext_override
29+
30+
31+
def migration_step_1_with_client(kms_key_id: str, ddb_table_name: str, sort_read_value: int = 1):
32+
"""
33+
Migration Step 1: Using the AWS Database Encryption SDK with Plaintext Override.
34+
35+
:param kms_key_id: The ARN of the KMS key to use for encryption
36+
:param ddb_table_name: The name of the DynamoDB table
37+
:param sort_read_value: The sort key value to read
38+
"""
39+
# 1. Create an EncryptedClient with plaintext override.
40+
# For Plaintext Override, use `FORCE_PLAINTEXT_WRITE_ALLOW_PLAINTEXT_READ`.
41+
# This plaintext override means:
42+
# - Write: Items are forced to be written as plaintext.
43+
# Items may not be written as encrypted items.
44+
# - Read: Items are allowed to be read as plaintext.
45+
# Items are allowed to be read as encrypted items.
46+
policy = PlaintextOverride.FORCE_PLAINTEXT_WRITE_ALLOW_PLAINTEXT_READ
47+
encrypted_client = setup_awsdbe_client_with_plaintext_override(
48+
kms_key_id=kms_key_id, ddb_table_name=ddb_table_name, policy=policy
49+
)
50+
51+
# 2. Put an item into your table using the Encrypted Client.
52+
# This item will be stored in plaintext since we are using a plaintext override
53+
# with FORCE_PLAINTEXT_WRITE_ALLOW_PLAINTEXT_READ policy
54+
item_to_encrypt = {
55+
"partition_key": {"S": "PlaintextMigrationExample"},
56+
"sort_key": {"N": str(1)},
57+
"attribute1": {"S": "encrypt and sign me!"},
58+
"attribute2": {"S": "sign me!"},
59+
":attribute3": {"S": "ignore me!"},
60+
}
61+
62+
put_item_request = {
63+
"TableName": ddb_table_name,
64+
"Item": item_to_encrypt,
65+
}
66+
67+
put_item_response = encrypted_client.put_item(**put_item_request)
68+
# Demonstrate that PutItem succeeded
69+
assert put_item_response["ResponseMetadata"]["HTTPStatusCode"] == 200
70+
71+
# 3. Get an item back from the table using the Encrypted Client.
72+
# If this is an item written in plaintext (e.g. any item written
73+
# during Step 0 or 1), then it will be returned as plaintext.
74+
# If this is an encrypted item (e.g. any item written during Step 2 or after),
75+
# then it will be decrypted before being returned.
76+
key_to_get = {"partition_key": {"S": "PlaintextMigrationExample"}, "sort_key": {"N": str(sort_read_value)}}
77+
78+
get_item_request = {"TableName": ddb_table_name, "Key": key_to_get}
79+
get_item_response = encrypted_client.get_item(**get_item_request)
80+
81+
# Demonstrate that GetItem succeeded and returned the decrypted item
82+
assert get_item_response["ResponseMetadata"]["HTTPStatusCode"] == 200
83+
returned_item = get_item_response["Item"]
84+
85+
# Demonstrate we get the expected item back
86+
assert returned_item["partition_key"]["S"] == "PlaintextMigrationExample"
87+
assert returned_item["sort_key"]["N"] == str(sort_read_value)
88+
assert returned_item["attribute1"]["S"] == "encrypt and sign me!"
89+
assert returned_item["attribute2"]["S"] == "sign me!"
90+
assert returned_item[":attribute3"]["S"] == "ignore me!"

0 commit comments

Comments
 (0)