Skip to content

Commit bd98716

Browse files
JuanLeeeJuanLeee
committed
docs: limitless docs
1 parent 3dd5f0f commit bd98716

File tree

3 files changed

+82
-0
lines changed

3 files changed

+82
-0
lines changed

‎docs/examples/PGLimitless.py

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License").
4+
# You may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
import psycopg
16+
17+
from aws_advanced_python_wrapper import AwsWrapperConnection
18+
19+
if __name__ == "__main__":
20+
with AwsWrapperConnection.connect(
21+
psycopg.Connection.connect,
22+
host="limitless-cluster.cluster-xyz.us-east-1.rds.amazonaws.com",
23+
dbname="postgres_limitless",
24+
user="user",
25+
password="password",
26+
plugins="limitless",
27+
autocommit=True
28+
) as awsconn, awsconn.cursor() as awscursor:
29+
awscursor.execute("SELECT * FROM aurora_db_instance_identifier()")
30+
31+
res = awscursor.fetchone()
32+
print(res)

‎docs/using-the-python-driver/UsingThePythonDriver.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -72,6 +72,7 @@ The AWS Advanced Python Driver has several built-in plugins that are available t
7272
| [Aurora Connection Tracker Plugin](./using-plugins/UsingTheAuroraConnectionTrackerPlugin.md) | `aurora_connection_tracker` | Aurora | Tracks all the opened connections. In the event of a cluster failover, the plugin will close all the impacted connections to the host. This plugin is enabled by default. | None |
7373
| [Read Write Splitting Plugin](./using-plugins/UsingTheReadWriteSplittingPlugin.md) | `read_write_splitting` | Aurora | Enables read write splitting functionality where users can switch between database reader and writer instances. | None |
7474
| [Fastest Response Strategy Plugin](./using-plugins/UsingTheFastestResponseStrategyPlugin.md) | `fastest_response_strategy` | Aurora | A host selection strategy plugin that uses a host monitoring service to monitor each reader host's response time and choose the host with the fastest response. | None |
75+
| [Limitless Connection Plugin](./using-plugins/UsingTheLimitlessConnectionPlugin.md) | `limitless` | Aurora | Enables client-side load-balancing of Transaction Routers on Amazon Aurora Limitless Databases . | None |
7576

7677
In addition to the built-in plugins, you can also create custom plugins more suitable for your needs.
7778
For more information, see [Custom Plugins](../development-guide/LoadablePlugins.md#using-custom-plugins).

‎docs/using-the-python-driver/using-plugins/UsingTheLimitlessConnectionPlugin.md

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
# Using the Limitless Connection Plugin
2+
3+
## What is Amazon Aurora Limitless Database?
4+
5+
Amazon Aurora Limitless Database is a new type of database that can horizontally scale to handle millions of write transactions per second and manage petabytes of data.
6+
Users will be able to use the AWS Python Driver with Aurora Limitless Databases and optimize their experience using the Limitless Connection Plugin.
7+
To learn more about Aurora Limitless Database, see the [Amazon Aurora Limitless documentation](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/limitless.html).
8+
9+
## Why use the Limitless Connection Plugin?
10+
11+
Aurora Limitless Database introduces a new endpoint for the databases - the DB shard group (limitless) endpoint that's managed by Route 53.
12+
When connecting to Aurora Limitless Database, clients will connect using this endpoint, and be routed to a transaction router via Route 53.
13+
Unfortunately, Route 53 is limited in its ability to load balance, and can allow uneven work loads on transaction routers.
14+
The Limitless Connection Plugin addresses this by performing client-side load balancing with load awareness.
15+
16+
The Limitless Connection Plugin achieves this with a monitoring thread that periodically polls for available transaction routers and their load metrics, and then caches them.
17+
When a new connection is made, the plugin directs the connection to a transaction router selected from the cache using a weighted round-robin strategy.
18+
Routers with a higher load are assigned a lower weight, and routers with a lower load are assigned a higher weight.
19+
20+
## How to use the Limitless Connection Plugin with the AWS Python Driver
21+
To enable the Limitless Connection Plugin, add the plugin code `limitless` to the [`plugins`](../UsingThePythonDriver.md#connection-plugin-manager-parameters) value.
22+
23+
The URL used to connect to a limitless database is the DB shard group URL.
24+
25+
### Limitless Connection Plugin Parameters
26+
| Parameter | Value | Required | Description | Default Value | Example Value |
27+
|---------------------------------------------------------|:-------:|:--------:|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------|---------------|
28+
| `limitless_transaction_router_monitor_interval_ms` | Integer | No | This property is the interval in milliseconds, that the plugin polls the database for available transaction routers and their load metrics. A lower value will increase the frequency of polling, and a higher value will decrease the frequency of polling. <br><br>Note that there will always be a delay between when the database updates its load metric info and when the Limitless Connection Plugin polls for it. If your Limitless database experiences fluctuating load between transaction routers, you may want to consider lowering `limitlessTransactionRouterMonitorIntervalMs` to reduce this delay and ensure the Limitless Connection Plugin load balancing has fresher info to work with. <br><br>The default value of this property is 7.5 seconds. This is half the interval that the database updates its load metric metadata. This value was chosen as a compromise between having fresher load metric info, but also being conscious of the associated overhead. | `7500` | `30000` |
29+
| `limitless_transaction_router_monitor_disposal_time_ms` | Integer | No | This property is the time in milliseconds that a Limitless transaction router monitor can remain unused before it is disposed. This ensures that in periods of long inactivity, the database isn't being needlessly polled and the resources associated with the monitor can be cleaned up. Note that when a new connection is created, a new Limitless transaction router monitor will also be created to resume polling the database. | `600000` | `300000` |
30+
| `limitless_connection_max_retries_ms` | Integer | No | This property is the max number of retries the Limitless Connection Plugin will attempt when failing to connect to the database. During these retries, the plugin will attempt to connect to the least loaded transaction router that is available. If the max number of connection retries is exceeded, then the plugin will throw a `SQLException`. In this scenario, it is likely that the database is in an unhealthy state, and the `SQLException` should be caught and handled by your application. | `5` | `13` |
31+
| `limitless_wait_for_transaction_router_info` | Boolean | No | In scenarios such as application start-up, the cache of available transaction routers and their load metric info will be empty. If `limitlessWaitForTransactionRouterInfo` is set to `true`, the plugin will wait until the cache is populated before selecting a transaction router and connecting to it. This may be beneficial for applications that create a large number of connections on start-up, since these connections will be load-balanced. <br><br>Alternatively, if this property set to `false` and the cache is empty, the plugin will not wait for the cache to be populated and default to using the DB Shard Group endpoint to connect to until the cache is populated. This will result in connections being routed to a transaction router via Route 53 until the cache is populated. This may be beneficial for applications that prioritize quicker start-up times at the expense of some early connections not being load-balanced by the Limitless Connection Plugin. | `true` | `false` |
32+
| `limitless_get_transaction_router_max_retries` | Integer | No | This property is the max number of times the Limitless Connection Plugin will retry fetching available transaction routers and their load metrics. These retries will occur if the fetched transaction router information is null or empty. If this max is reached, a `SQLException` will be thrown. In this scenario, it is likely that the database is in an unhealthy state, and the thrown `SQLException` should be caught and handled by your application. <br><br>If your application prioritizes failing fast, then consider a lower value for this property. However, if your application prioritizes durability, then consider a higher value. | `5` | `10` |
33+
| `limitless_get_transaction_router_retry_interval_ms` | Integer | No | This property is the interval in milliseconds between retries of fetching available transaction routers and their load metrics. <br><br>If your application prioritizes failing fast, then consider a lower value for this property. However, if your application prioritizes durability, then consider a higher value. | `300` | `1000` |
34+
35+
### Use with other plugins
36+
The Limitless Connection Plugin is compatible with authentication type plugins such as the IAM and AWS Secrets Manager Plugins.
37+
38+
> [!IMPORTANT]\
39+
> The Failover, Host Monitoring, and Read Write Splitting Plugins are also compatible with the Limitless Connection Plugin.
40+
However, we don't recommend using them with the Limitless Connection Plugin because they're not designed to be used with Aurora Limitless Database.
41+
They don't provide any extra value, and add unnecessary computation and memory overhead.
42+
43+
### Use with Connection Pools
44+
Connection pools keep connections open for reuse, but this can work against the client-side load-balancing of the Limitless Connection Plugin and cause an imbalanced load on transaction routers.
45+
To mitigate this, consider setting connection properties that can reduce the number of idle connections or increase the lifetime of connections.
46+
If you're using the internal connection pool, some of these properties are `pool_expiration_check_ns`, `pool_cleanup_interval_ns`.
47+
48+
## Sample Code
49+
[PGLimitless.py](../../examples/PGLimitless.py)

0 commit comments

Comments
 (0)