-
Notifications
You must be signed in to change notification settings - Fork 18
Add docs for SQL Alchemy ORM support #1237
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
72457dc
6cd61c7
bc607f6
5daa970
5ee014a
4871ac7
068e2e9
f81b1db
a1ebf4c
f081a63
1b8d401
4429afd
4f6500c
9bd824f
ee53f8c
9f484bb
c7ab03f
6e16513
af9e19f
800b3a7
0033f90
65a7b38
6e055b3
49bbeea
92659f3
80fb9d0
9047907
0e08938
d4eb5bd
b344ef5
919284a
0ab9c66
a64d748
fa1f875
c981d2d
5c21749
d5b2b82
79f990f
f955a9c
c6958bc
1480c48
0f9859d
c75e9f7
771b4b0
7941fe0
159b6c3
7e2bb1c
edd4d02
9f80c4e
f0d7d91
cfebe5b
b45cd27
88dfb8e
c999602
9d2f642
560e67c
15abdd8
bc38b78
b47aa72
a3f9c3e
f186bb3
6becc21
865cd48
0e96a54
3e6e164
8fc88f4
830eb63
8e33fd1
7efffd2
e6c24e6
694a38d
3bf7e31
192aa13
5398a2b
e97777c
dd2f122
a23d953
2e50221
f36f719
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,222 @@ | ||
| # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"). | ||
| # You may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
|
|
||
| from sqlalchemy import Column, Integer, String, create_engine | ||
| from sqlalchemy.exc import DBAPIError | ||
| from sqlalchemy.orm import DeclarativeBase, sessionmaker | ||
|
|
||
| from aws_advanced_python_wrapper import release_resources | ||
| from aws_advanced_python_wrapper.errors import ( | ||
| FailoverFailedError, FailoverSuccessError, | ||
| TransactionResolutionUnknownError) | ||
|
|
||
| """ | ||
| SQLAlchemy ORM Failover Example with AWS Advanced Python Wrapper | ||
|
|
||
| This example demonstrates how to handle failover events when using SQLAlchemy ORM | ||
| with the AWS Advanced Python Wrapper. | ||
|
|
||
| """ | ||
|
|
||
|
|
||
| class Base(DeclarativeBase): | ||
| pass | ||
|
|
||
|
|
||
| class BankAccount(Base): | ||
| """Example model for demonstrating failover handling.""" | ||
| __tablename__ = 'bank_test' | ||
|
|
||
| id = Column(Integer, primary_key=True) | ||
| name = Column(String(50)) | ||
| account_balance = Column(Integer) | ||
|
|
||
| def __str__(self) -> str: | ||
| return f"{self.name}: ${self.account_balance}" | ||
|
|
||
|
|
||
| def execute_query_with_failover_handling(query_func): | ||
| """ | ||
| Execute a SQLAlchemy ORM query with failover error handling. | ||
|
|
||
| Args: | ||
| query_func: A callable that executes the desired query | ||
|
|
||
| Returns: | ||
| The result of the query function | ||
| """ | ||
| try: | ||
| return query_func() | ||
|
|
||
| except DBAPIError as dbapi_err: | ||
| e = dbapi_err.orig | ||
| if isinstance(e, FailoverSuccessError): | ||
| # Query execution failed and AWS Advanced Python Wrapper successfully failed over to an available instance. | ||
| # https://github.com/aws/aws-advanced-python-wrapper/blob/main/docs/using-the-python-driver/using-plugins/UsingTheFailoverPlugin.md#failoversuccesserror | ||
|
|
||
| # The connection has been re-established. Retry the query. | ||
| print("Failover successful! Retrying query...") | ||
|
|
||
| # Retry the query | ||
| return query_func() | ||
|
|
||
| elif isinstance(e, FailoverFailedError): | ||
| # Failover failed. The application should open a new connection, | ||
| # check the results of the failed transaction and re-run it if needed. | ||
| # https://github.com/aws/aws-advanced-python-wrapper/blob/main/docs/using-the-python-driver/using-plugins/UsingTheFailoverPlugin.md#failoverfailederror | ||
| print(f"Failover failed: {e}") | ||
| print("Application should open a new connection and retry the transaction.") | ||
| raise e | ||
|
|
||
| elif isinstance(e, TransactionResolutionUnknownError): | ||
| # The transaction state is unknown. The application should check the status | ||
| # of the failed transaction and restart it if needed. | ||
| # https://github.com/aws/aws-advanced-python-wrapper/blob/main/docs/using-the-python-driver/using-plugins/UsingTheFailoverPlugin.md#transactionresolutionunknownerror | ||
| print(f"Transaction resolution unknown: {e}") | ||
| print("Application should check transaction status and retry if needed.") | ||
| raise e | ||
|
|
||
|
|
||
| def create_table(engine): | ||
| """Create the database table with failover handling.""" | ||
| def _create(): | ||
| Base.metadata.create_all(engine) | ||
| print("Table created successfully") | ||
|
|
||
| execute_query_with_failover_handling(_create) | ||
|
|
||
|
|
||
| def drop_table(engine): | ||
| """Drop the database table with failover handling.""" | ||
| def _drop(): | ||
| Base.metadata.drop_all(engine) | ||
| print("Table dropped successfully") | ||
|
|
||
| execute_query_with_failover_handling(_drop) | ||
|
|
||
|
|
||
| def insert_records(session): | ||
| """Insert records with failover handling.""" | ||
| print("\n--- Inserting Records ---") | ||
|
|
||
| def _insert1(): | ||
| account = BankAccount(name='Jane Doe', account_balance=200) | ||
| session.add(account) | ||
| session.commit() # Explicit commit required | ||
| print(f"Inserted: {account}") | ||
| return account | ||
|
|
||
| def _insert2(): | ||
| account = BankAccount(name='John Smith', account_balance=200) | ||
| session.add(account) | ||
| session.commit() | ||
| print(f"Inserted: {account}") | ||
| return account | ||
|
|
||
| execute_query_with_failover_handling(_insert1) | ||
| execute_query_with_failover_handling(_insert2) | ||
|
|
||
|
|
||
| def query_records(session): | ||
| """Query records with failover handling.""" | ||
| print("\n--- Querying Records ---") | ||
|
|
||
| def _query(): | ||
| accounts = session.query(BankAccount).all() | ||
| for account in accounts: | ||
| print(f" {account}") | ||
| return accounts | ||
|
|
||
| return execute_query_with_failover_handling(_query) | ||
|
|
||
|
|
||
| def update_record(session): | ||
| """Update a record with failover handling.""" | ||
| print("\n--- Updating Record ---") | ||
|
|
||
| def _update(): | ||
| account = session.query(BankAccount).filter(BankAccount.name == "Jane Doe").first() | ||
| if account: | ||
| account.account_balance = 300 | ||
| session.commit() | ||
| print(f"Updated: {account}") | ||
| return account | ||
|
|
||
| return execute_query_with_failover_handling(_update) | ||
|
|
||
|
|
||
| def filter_records(session): | ||
| """Filter records with failover handling.""" | ||
| print("\n--- Filtering Records ---") | ||
|
|
||
| def _filter(): | ||
| accounts = session.query(BankAccount).filter(BankAccount.account_balance >= 250).all() | ||
| print(f"Found {len(accounts)} accounts with balance >= $250:") | ||
| for account in accounts: | ||
| print(f" {account}") | ||
| return accounts | ||
|
|
||
| return execute_query_with_failover_handling(_filter) | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| try: | ||
| print("SQLAlchemy ORM Failover Example with AWS Advanced Python Wrapper") | ||
| print("=" * 60) | ||
|
|
||
| engine = create_engine( | ||
| 'mysql+aws_wrapper_mysqlconnector://admin:pwd@' | ||
| 'database.cluster-xyz.us-east-1.rds.amazonaws.com:3306/mysql?' | ||
| 'wrapper_plugins=failover_v2' | ||
| ) | ||
|
|
||
| # Create table | ||
| create_table(engine) | ||
|
|
||
| Session = sessionmaker(bind=engine) | ||
| with Session() as session: | ||
| # Insert records | ||
| insert_records(session) | ||
|
|
||
| # Query records | ||
| query_records(session) | ||
|
|
||
| # Update a record | ||
| update_record(session) | ||
|
|
||
| # Query again to see the update | ||
| query_records(session) | ||
|
|
||
| # Filter records | ||
| filter_records(session) | ||
|
|
||
| session.close() | ||
|
|
||
| # Cleanup | ||
| print("\n--- Cleanup ---") | ||
| drop_table(engine) | ||
|
|
||
| print("\n" + "=" * 60) | ||
| print("Example completed successfully!") | ||
|
|
||
| engine.dispose() | ||
|
|
||
| except Exception as e: | ||
| print(f"Error: {e}") | ||
| import traceback | ||
| traceback.print_exc() | ||
|
|
||
| finally: | ||
| # Clean up AWS Advanced Python Wrapper resources | ||
| release_resources() |
| Original file line number | Diff line number | Diff line change | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,66 @@ | ||||||||||||||
| # SQLAlchemy ORM Support | ||||||||||||||
|
|
||||||||||||||
| > [!IMPORTANT] | ||||||||||||||
| > SQLAlchemy ORM support is currently only available for **MySQL databases**. | ||||||||||||||
|
|
||||||||||||||
| The AWS Advanced Python Wrapper provides a custom SQLAlchemy database backend that enables SQLAlchemy applications to leverage AWS and Aurora functionalities such as failover handling and IAM authentication. | ||||||||||||||
|
|
||||||||||||||
| ## Prerequisites | ||||||||||||||
|
|
||||||||||||||
| - SQLAlchemy 2.0.0+ | ||||||||||||||
|
|
||||||||||||||
| ## Basic Configuration | ||||||||||||||
|
|
||||||||||||||
| To use the AWS Advanced Python Wrapper with SQLAlchemy, call the `create_engine` function with your database URL with the configuration settings appended: | ||||||||||||||
|
|
||||||||||||||
| ```python | ||||||||||||||
| from sqlalchemy import create_engine | ||||||||||||||
|
|
||||||||||||||
| create_engine("mysql+aws_wrapper_mysqlconnector://your_username:your_password@your-cluster-endpoint.cluster-xyz.us-east-1.rds.amazonaws.com:your_port/your_database_name?connect_timeout=10&wrapper_plugins=aurora_connection_tracker%2Cfailover_v2") | ||||||||||||||
| ``` | ||||||||||||||
|
|
||||||||||||||
| See [the SQLALchemy official documentation](https://docs.sqlalchemy.org/en/20/core/engines.html) for more information on engine configuration. | ||||||||||||||
|
|
||||||||||||||
| ### Supported Engines | ||||||||||||||
|
|
||||||||||||||
| | Driver | Database Dialect | | ||||||||||||||
| |-------------------|------------------| | ||||||||||||||
| | `aws_wrapper_mysqlconnector` | `mysql` | | ||||||||||||||
|
|
||||||||||||||
| ## Using Plugins with SQLAlchemy | ||||||||||||||
|
|
||||||||||||||
| The AWS Advanced Python Wrapper supports a variety of plugins that enhance your SQLAlchemy application with features like failover handling, IAM authentication, and more. Most plugins can be enabled simply by adding them to the `wrapper_plugins` parameter in your database URL. | ||||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||||||
|
|
||||||||||||||
| For a complete list of available plugins, see the [List of Available Plugins](./UsingThePythonWrapper.md#list-of-available-plugins) in the main driver documentation. | ||||||||||||||
|
|
||||||||||||||
|
|
||||||||||||||
| ### Failover Plugin | ||||||||||||||
|
|
||||||||||||||
| The Failover Plugin provides automatic failover handling for Aurora clusters. When a database instance becomes unavailable, the plugin automatically connects to a healthy instance in the cluster. | ||||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can we add a link to the failover plugin docs here |
||||||||||||||
|
|
||||||||||||||
| #### Handling Failover Events | ||||||||||||||
|
|
||||||||||||||
| During a failover event, the driver will throw a `DBAPIError` exception after successfully connecting to a new instance. Your application should catch this exception, check which type of failover error occurred, and retry the failed query: | ||||||||||||||
|
|
||||||||||||||
| ```python | ||||||||||||||
| from aws_advanced_python_wrapper.errors import FailoverSuccessError | ||||||||||||||
|
|
||||||||||||||
| def execute_query_with_failover_handling(query_func): | ||||||||||||||
| try: | ||||||||||||||
| return query_func() | ||||||||||||||
| except DBAPIError as dbapi_err: | ||||||||||||||
| err = dbapi_err.orig | ||||||||||||||
| if isinstance(err, FailoverSuccessError): | ||||||||||||||
| # Failover successful, retry the query | ||||||||||||||
| return query_func() | ||||||||||||||
| ``` | ||||||||||||||
|
|
||||||||||||||
| For a complete example, see [MySQLSQLAlchemyFailover.py](../examples/MySQLSQLAlchemyFailover.py). | ||||||||||||||
|
|
||||||||||||||
| For more information about the Failover Plugin, see the [Failover Plugin documentation](./using-plugins/UsingTheFailoverPlugin.md). | ||||||||||||||
|
|
||||||||||||||
| ### Read/Write Splitting | ||||||||||||||
|
|
||||||||||||||
| The Read/Write Splitting Plugin is not supported for SQLAlchemy, since session binds already implement this functionality. | ||||||||||||||
|
|
||||||||||||||
| See the [official SQLAlchemy documentation on the Session API](https://docs.sqlalchemy.org/en/20/orm/session_api.html) for more information on session binds. | ||||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I would love to see this maybe structured a different way or with some additional plugins that ARE supported. You say that there are many other plugins and then only list one so I would suggest either
I would also like rw splitting to be under a header of something like "not supported" It's confusing that it is positioned the same way as failover which is supported. Adding a not supported section would also give you space to mention that lack of pg support again
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We could consider reworking this into a plugin compatibility section, and create a table indicating which plugins are supported and which are not (rw splitting and srw) |
||||||||||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Curious on this "database dialect" - is that from our wrapper or sql alchemy? if it's our wrapper should we not support aurora-mysql and the other mysql database dialects too?