|
1 | | -## My Project |
| 1 | +## Athena Parameterized Queries Blog |
2 | 2 |
|
3 | | -TODO: Fill this README out! |
| 3 | +This is a repository for [Use Amazon Athena parameterized queries to provide data as a service](https://aws.amazon.com/blogs/big-data/use-amazon-athena-parameterized-queries-to-provide-data-as-a-service/) blog post. It contains a CloudFormation template for provisioning accompanying resources and assets for deploying a sample web application to demonstrate new ability to pass executions parameters to Amazon Athena [StartQueryExecution API](https://docs.aws.amazon.com/athena/latest/APIReference/API_StartQueryExecution.html). |
4 | 4 |
|
5 | | -Be sure to: |
| 5 | +### Prerequisites: |
6 | 6 |
|
7 | | -* Change the title in this README |
8 | | -* Edit your repository description on GitHub |
| 7 | +This sample application uses the following AWS services to demonstrate a Data-as-a-Service (DaaS) architecture pattern that uses Amazon Athena to query the Amazon.com customer reviews dataset. |
| 8 | + |
| 9 | +- [AWS Amplify](https://docs.aws.amazon.com/amplify/latest/userguide/welcome.html) |
| 10 | +- [Amazon API Gateway](https://docs.aws.amazon.com/apigateway/latest/developerguide/welcome.html) |
| 11 | +- [Amazon Athena](https://docs.aws.amazon.com/athena/latest/ug/what-is.html) |
| 12 | +- [AWS CloudFormation](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/Welcome.html) |
| 13 | +- [Amazon DynamoDB](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Introduction.html) |
| 14 | +- [AWS Glue](https://aws.amazon.com/glue) and AWS Glue Data Catalog |
| 15 | +- [AWS Identity and Access Management](https://docs.aws.amazon.com/IAM/latest/UserGuide/introduction.html) (IAM) |
| 16 | +- [AWS Lambda](https://docs.aws.amazon.com/lambda/latest/dg/welcome.html) |
| 17 | +- [Amazon S3](https://docs.aws.amazon.com/AmazonS3/latest/userguide/Welcome.html) |
| 18 | + |
| 19 | +The instructions assume you have: |
| 20 | + |
| 21 | +- An AWS Account. For instructions, refer to [Creating an AWS account](https://docs.aws.amazon.com/accounts/latest/reference/manage-acct-creating.html). |
| 22 | +- A CloudTrail trail created. For instructions, refer to [Creating a trail](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-create-a-trail-using-the-console-first-time.html). |
| 23 | +- Amazon S3 Bucket created. For instructions, refer to [Creating a bucket](https://docs.aws.amazon.com/AmazonS3/latest/userguide/create-bucket-overview.html). |
| 24 | +- [Node.js](https://nodejs.org/) version 16+ installed on your device. |
| 25 | + |
| 26 | +### Deploy the CloudFormation Stack |
| 27 | + |
| 28 | +In this section, you deploy a CloudFormation template ([athena-parameterized-queries.yaml](athena-parameterized-queries.yaml)) that creates the following resources: |
| 29 | + |
| 30 | +- AWS Glue Data Catalog database |
| 31 | +- AWS Glue Data Catalog table |
| 32 | +- Amazon Athena workgroup |
| 33 | +- Three Athena prepared statements |
| 34 | +- Three Athena named queries |
| 35 | +- The API Gateway HTTP API |
| 36 | +- The Lambda execution role for Athena queries |
| 37 | +- The Lambda execution role for API Gateway HTTP API authorization |
| 38 | +- Five Lambda functions: |
| 39 | + - Update the AWS Glue Data Catalog |
| 40 | + - Authorize API Gateway requests |
| 41 | + - Submit Athena queries |
| 42 | + - List Athena prepared statements |
| 43 | + - List Athena named queries |
| 44 | + |
| 45 | + |
| 46 | + |
| 47 | + |
| 48 | + |
| 49 | +To deploy the CloudFormation stack template, follow these steps: |
| 50 | + |
| 51 | +1. Navigate to this [GitHub repository](https://github.com/aws-samples/amazon-athena-execution-parameters-blog). |
| 52 | +2. Clone the repository or copy the CloudFormation template `athena-parameterized-queries.yaml`. |
| 53 | +3. On the [AWS CloudFormation](https://console.aws.amazon.com/cloudformation) console, choose **Create stack**. |
| 54 | +4. Select **Upload a template file** and choose **Choose file**. |
| 55 | +5. Upload `athena-parameterized-queries.yaml`, then choose **Next**. |
| 56 | +6. On the **Specify stack details** page, enter the stack name `athena-parameterized-queries`. |
| 57 | +7. On the same page, there are two parameters: |
| 58 | + 1. For **S3QueryResultsBucketName**, enter the S3 bucket name in your AWS Account and in the same AWS Region as where you're running your CloudFormation stack. (For this post, we use the bucket name value, like `my-bucket`). |
| 59 | + 2. For **APIPassphrase**, enter a passphrase to authenticate API requests. You use this later. |
| 60 | +8. Choose **Next**. |
| 61 | +9. On the **Configure stack options** page, choose **Next**. |
| 62 | +10. On the **Review** page, choose **I acknowledge that AWS CloudFormation might create IAM resources with custom names**, and choose **Create stack**. |
| 63 | + |
| 64 | +The script takes less than two minutes to run and change to a `CREATE_COMPLETE` state. If you deploy the stack twice in the same AWS account and Region, some resources may already exist, and the process fails with a message indicating the resource already exists in another template. |
| 65 | + |
| 66 | +11. On the **Outputs** tab, copy the `APIEndpoint` value to use later. |
| 67 | + |
| 68 | +Refer to [Use Amazon Athena parameterized queries to provide data as a service](https://aws.amazon.com/blogs/big-data/use-amazon-athena-parameterized-queries-to-provide-data-as-a-service/) blog for guidance on least-privilege authorization for deployment of the CloudFormation template. |
| 69 | + |
| 70 | +### Deploy the Amplify application |
| 71 | + |
| 72 | +In this section, you deploy your Amplify application. |
| 73 | + |
| 74 | +1. In the cloned repository, open `web-application/.env` in a text editor. |
| 75 | +2. Set `AWS_API_ENDPOINT` as the `APIEndpoint` value from the CloudFormation stack **Outputs**. For example: `AWS_API_ENDPOINT="https://123456abcd.execute-api.your-region.amazonaws.com"`. |
| 76 | +3. Set `API_AUTH_CODE` as the value you input as the CloudFormation stack's `APIPassphrase` parameter argument. For example: `API_AUTH_CODE="YOUR_PASSPHRASE"`. |
| 77 | +4. Navigate to the `web-application/` directory and run `npm install`. |
| 78 | +5. Run `npm run build` to compile distribution assets. |
| 79 | +6. On the [Amplify](https://console.aws.amazon.com/amplify) console, choose **All apps**. |
| 80 | +7. Choose **New app**. |
| 81 | +8. Select **Host web app**, select **Deploy without Git provider**, then choose **Continue**. |
| 82 | +9. For **App name**, enter `Athena Parameterized Queries App`. |
| 83 | +10. For **Environment name**, you don't need to enter a value. |
| 84 | +11. Select **Drag and Drop**. |
| 85 | +12. Locate `dist/` directory inside `web-application/`, drag it into the window and drop it. Ensure you drag the entire directory, not the files within it. |
| 86 | + |
| 87 | + |
| 88 | + |
| 89 | +13. Choose **Save and deploy** to deploy the web application on Amplify. |
| 90 | + |
| 91 | +This step will take less than a minute to complete. |
| 92 | + |
| 93 | +14. Under **App settings**, choose **Access control**, then choose **Manage access** |
| 94 | +15. Select **Apply a global password**, then enter values for **Username** and **Password**. |
| 95 | + |
| 96 | +You will use these credentials to access your Amplify application. |
| 97 | + |
| 98 | +##### Troubleshooting: |
| 99 | + |
| 100 | +- If the Amplify app page shows as blank, refer back to Step 10 and make sure to drag and drop the entire **dist/** directory instead of individual files inside of it |
| 101 | +- If you see **Network Error**, verify that **web-application/.env** file has the correct API Endpoint URL (refer back to Step 12 in **Instruction to deploy the API**) |
| 102 | +- If you see error **status code 403**, verify that **web-application/.env** file has the correct Passphrase (refer back to Step 1) |
| 103 | + |
| 104 | +### Access your Amplify application and run queries |
| 105 | + |
| 106 | +In this section, you use the Amplify application to run Athena parameterized queries against the Amazon.com customer reviews dataset. The left side of the application shows how you can run parameterized queries using Athena prepared statements. The right side of the application shows how you can run parameterized queries without prepared statements, such as if the queries are written in your code; this sample uses named queries within the Athena workgroup. For more information about named queries, refer to [NamedQuery](https://docs.aws.amazon.com/athena/latest/APIReference/API_NamedQuery.html) in the [Amazon Athena API Reference](https://docs.aws.amazon.com/athena/latest/APIReference/Welcome.html). |
| 107 | + |
| 108 | +1. Open the Amplify web application link located under **Domain**. For example: `https://dev123.abcd12345xyz.amplifyapp.com/`. |
| 109 | +2. In the **Sign in** prompt, enter the user name and password your provided as the Amplify application global password. |
| 110 | +3. For **Workgroup Name**, choose `ParameterizedStatementsWG` workgroup. |
| 111 | +4. Choose a statement example on the **Prepared Statement** or **SQL Statement** drop-down menu. |
| 112 | + |
| 113 | +Selecting a statement displays a description about the query, including examples of parameters you can try with this statement, and the original SQL query string. SQL parameters of type string must be surrounded by single quotes, for example: `'your_string_value'`. |
| 114 | + |
| 115 | +5. Enter your query parameters. |
| 116 | + |
| 117 | +The following figure depicts an example of the parameters to input for the `product_helpful_reviews` prepared statement. |
| 118 | + |
| 119 | + |
| 120 | + |
| 121 | +6. Choose **Run Query** to send the query request to the API endpoint. |
| 122 | + |
| 123 | +After the query runs, the results will be returned as a table. The complete query workflow is depicted in the previous architecture diagram. |
| 124 | + |
| 125 | +#### Using execution parameters with the AWS SDK for Python (Boto3) |
| 126 | + |
| 127 | +In this section, you inspect the Lambda function code for using the `StartQueryExecution` API with and without prepared statements. |
| 128 | + |
| 129 | +1. On the [Lambda](https://console.aws.amazon.com/lambda) console, choose **Functions**. |
| 130 | +2. Navigate to `LambdaAthenaFunction-athena-parameterized-queries` Lambda function. |
| 131 | +3. Choose the **Code Source** window. |
| 132 | + |
| 133 | +Examples of passing parameters to the Athena `StartQueryExecution` API using AWS SDK for Python (Boto3) begin on lines 39 and 49. Note the new `ExecutionParameters` option on lines 45 and 55. |
| 134 | + |
| 135 | +The following code uses execution parameters with Athena prepared statements: |
| 136 | + |
| 137 | +``` |
| 138 | +response = athena.start_query_execution( |
| 139 | + QueryString=f'EXECUTE {statement}', # Example: "EXECUTE prepared_statement_name" |
| 140 | + WorkGroup=workgroup, |
| 141 | + QueryExecutionContext={ |
| 142 | + 'Database': 'athena_prepared_statements' |
| 143 | + }, |
| 144 | + ExecutionParameters=input_parameters |
| 145 | +) |
| 146 | +``` |
| 147 | + |
| 148 | +The following code uses execution parameters without Athena prepared statements: |
| 149 | + |
| 150 | +``` |
| 151 | +response = athena.start_query_execution( |
| 152 | + QueryString=statement, # Example: "SELECT \* FROM TABLE WHERE parameter_name = ?" |
| 153 | + WorkGroup=workgroup, |
| 154 | + QueryExecutionContext={ |
| 155 | + 'Database': 'athena_prepared_statements' |
| 156 | + }, |
| 157 | + ExecutionParameters=input_parameters |
| 158 | +) |
| 159 | +``` |
| 160 | + |
| 161 | +### Clean up |
| 162 | + |
| 163 | +In this post, you created several components, which generate cost. To avoid incurring future charges, remove the resources with the following steps: |
| 164 | + |
| 165 | +1. [Delete the S3 bucket’s results prefix](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-folders.html#delete-folders) created after you ran a query on your workgroup. |
| 166 | + |
| 167 | +With the default template, the prefix is named `<S3QueryResultsBucketName>/athena-results`. Use caution in this step. Unless you are [using versioning on your S3 bucket](https://docs.aws.amazon.com/AmazonS3/latest/userguide/Versioning.html), deleting S3 objects cannot be undone. |
| 168 | + |
| 169 | +2. On the Amplify console, select the app to delete and on the **Actions** menu, choose **Delete app**, then confirm. |
| 170 | +3. On the AWS CloudFormation console, select the stack to delete, choose **Delete**, and confirm. |
9 | 171 |
|
10 | 172 | ## Security |
11 | 173 |
|
12 | 174 | See [CONTRIBUTING](CONTRIBUTING.md#security-issue-notifications) for more information. |
13 | 175 |
|
14 | 176 | ## License |
15 | 177 |
|
16 | | -This library is licensed under the MIT-0 License. See the LICENSE file. |
17 | | - |
| 178 | +This project is licensed under the MIT-0 License. See the LICENSE file. |
0 commit comments