Skip to content

Default to standard retry #3236

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: version-3
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions build_tools/services.rb
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,10 @@ class ServiceEnumerator
MANIFEST_PATH = File.expand_path('../../services.json', __FILE__)

# Minimum `aws-sdk-core` version for new gem builds
MINIMUM_CORE_VERSION = "3.216.0"
MINIMUM_CORE_VERSION = "3.223.0"

# Minimum `aws-sdk-core` version for new S3 gem builds
MINIMUM_CORE_VERSION_S3 = "3.216.0"
MINIMUM_CORE_VERSION_S3 = "3.223.0"

EVENTSTREAM_PLUGIN = "Aws::Plugins::EventStreamConfiguration"

Expand Down
2 changes: 2 additions & 0 deletions gems/aws-sdk-core/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
Unreleased Changes
------------------

* Feature - Set the default `retry_mode` to `standard` for all clients.

3.222.3 (2025-04-28)
------------------

Expand Down
161 changes: 76 additions & 85 deletions gems/aws-sdk-core/lib/aws-sdk-core/plugins/retry_errors.rb
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,7 @@ class RetryErrors < Seahorse::Client::Plugin

DEFAULT_BACKOFF = lambda do |c|
delay = 2**c.retries * c.config.retry_base_delay
if (c.config.retry_max_delay || 0) > 0
delay = [delay, c.config.retry_max_delay].min
end
delay = [delay, c.config.retry_max_delay].min if (c.config.retry_max_delay || 0).positive?
jitter = c.config.retry_jitter
jitter = JITTERS[jitter] if jitter.is_a?(Symbol)
delay = jitter.call(delay) if jitter
Expand All @@ -41,78 +39,77 @@ class RetryErrors < Seahorse::Client::Plugin
:retry_limit,
default: 3,
doc_type: Integer,
docstring: <<-DOCS)
The maximum number of times to retry failed requests. Only
~ 500 level server errors and certain ~ 400 level client errors
are retried. Generally, these are throttling errors, data
checksum errors, networking errors, timeout errors, auth errors,
endpoint discovery, and errors from expired credentials.
This option is only used in the `legacy` retry mode.
docstring: <<~DOCS)
The maximum number of times to retry failed requests. Only
~ 500 level server errors and certain ~ 400 level client errors
are retried. Generally, these are throttling errors, data
checksum errors, networking errors, timeout errors, auth errors,
endpoint discovery, and errors from expired credentials.
This option is only used in the `legacy` retry mode.
DOCS

option(
:retry_max_delay,
default: 0,
doc_type: Integer,
docstring: <<-DOCS)
The maximum number of seconds to delay between retries (0 for no limit)
used by the default backoff function. This option is only used in the
`legacy` retry mode.
docstring: <<~DOCS)
The maximum number of seconds to delay between retries (0 for no limit)
used by the default backoff function. This option is only used in the
`legacy` retry mode.
DOCS

option(
:retry_base_delay,
default: 0.3,
doc_type: Float,
docstring: <<-DOCS)
The base delay in seconds used by the default backoff function. This option
is only used in the `legacy` retry mode.
docstring: <<~DOCS)
The base delay in seconds used by the default backoff function. This option
is only used in the `legacy` retry mode.
DOCS

option(
:retry_jitter,
default: :none,
doc_type: Symbol,
rbs_type: '(:none | :equal | :full | ^(Integer) -> Integer)',
docstring: <<-DOCS)
A delay randomiser function used by the default backoff function.
Some predefined functions can be referenced by name - :none, :equal, :full,
otherwise a Proc that takes and returns a number. This option is only used
in the `legacy` retry mode.
docstring: <<~DOCS)
A delay randomiser function used by the default backoff function.
Some predefined functions can be referenced by name - :none, :equal, :full,
otherwise a Proc that takes and returns a number. This option is only used
in the `legacy` retry mode.

@see https://www.awsarchitectureblog.com/2015/03/backoff.html
@see https://www.awsarchitectureblog.com/2015/03/backoff.html
DOCS

option(
:retry_backoff,
default: DEFAULT_BACKOFF,
doc_type: Proc,
docstring: <<-DOCS)
A proc or lambda used for backoff. Defaults to 2**retries * retry_base_delay.
This option is only used in the `legacy` retry mode.
docstring: <<~DOCS)
A proc or lambda used for backoff. Defaults to 2**retries * retry_base_delay.
This option is only used in the `legacy` retry mode.
DOCS

# END LEGACY OPTIONS

option(
:retry_mode,
default: 'legacy',
default: 'standard',
doc_type: String,
rbs_type: '("legacy" | "standard" | "adaptive")',
docstring: <<-DOCS) do |cfg|
Specifies which retry algorithm to use. Values are:
docstring: <<~DOCS) do |cfg|
Specifies which retry algorithm to use. Values are:

* `legacy` - The pre-existing retry behavior. This is default value if
no retry mode is provided.
* `legacy` - The pre-existing retry behavior.

* `standard` - A standardized set of retry rules across the AWS SDKs.
This includes support for retry quotas, which limit the number of
unsuccessful retries a client can make.
* `standard` - A standardized set of retry rules across the AWS SDKs.
This includes support for retry quotas, which limit the number of
unsuccessful retries a client can make.

* `adaptive` - An experimental retry mode that includes all the
functionality of `standard` mode along with automatic client side
throttling. This is a provisional mode that may change behavior
in the future.
* `adaptive` - An experimental retry mode that includes all the
functionality of `standard` mode along with automatic client side
throttling. This is a provisional mode that may change behavior
in the future.
DOCS
resolve_retry_mode(cfg)
end
Expand All @@ -121,11 +118,11 @@ class RetryErrors < Seahorse::Client::Plugin
:max_attempts,
default: 3,
doc_type: Integer,
docstring: <<-DOCS) do |cfg|
An integer representing the maximum number attempts that will be made for
a single request, including the initial attempt. For example,
setting this value to 5 will result in a request being retried up to
4 times. Used in `standard` and `adaptive` retry modes.
docstring: <<~DOCS) do |cfg|
An integer representing the maximum number attempts that will be made for
a single request, including the initial attempt. For example,
setting this value to 5 will result in a request being retried up to
4 times. Used in `standard` and `adaptive` retry modes.
DOCS
resolve_max_attempts(cfg)
end
Expand All @@ -134,11 +131,11 @@ class RetryErrors < Seahorse::Client::Plugin
:adaptive_retry_wait_to_fill,
default: true,
doc_type: 'Boolean',
docstring: <<-DOCS) do |cfg|
Used only in `adaptive` retry mode. When true, the request will sleep
until there is sufficent client side capacity to retry the request.
When false, the request will raise a `RetryCapacityNotAvailableError` and will
not retry instead of sleeping.
docstring: <<~DOCS) do |cfg|
Used only in `adaptive` retry mode. When true, the request will sleep
until there is sufficent client side capacity to retry the request.
When false, the request will raise a `RetryCapacityNotAvailableError` and will
not retry instead of sleeping.
DOCS
resolve_adaptive_retry_wait_to_fill(cfg)
end
Expand All @@ -147,10 +144,10 @@ class RetryErrors < Seahorse::Client::Plugin
:correct_clock_skew,
default: true,
doc_type: 'Boolean',
docstring: <<-DOCS) do |cfg|
Used only in `standard` and adaptive retry modes. Specifies whether to apply
a clock skew correction and retry requests with skewed client clocks.
DOCS
docstring: <<~DOCS) do |cfg|
Used only in `standard` and `adaptive` retry modes. Specifies whether to apply
a clock skew correction and retry requests with skewed client clocks.
DOCS
resolve_correct_clock_skew(cfg)
end

Expand All @@ -164,20 +161,19 @@ class RetryErrors < Seahorse::Client::Plugin
option(:clock_skew) { Retries::ClockSkew.new }

def self.resolve_retry_mode(cfg)
default_mode_value =
if cfg.respond_to?(:defaults_mode_config_resolver)
cfg.defaults_mode_config_resolver.resolve(:retry_mode)
end
if cfg.respond_to?(:defaults_mode_config_resolver)
default_mode_value = cfg.defaults_mode_config_resolver.resolve(:retry_mode)
end

value = ENV['AWS_RETRY_MODE'] ||
Aws.shared_config.retry_mode(profile: cfg.profile) ||
default_mode_value ||
'legacy'
value = ENV['AWS_RETRY_MODE'] ||
Aws.shared_config.retry_mode(profile: cfg.profile) ||
default_mode_value ||
'standard'
# Raise if provided value is not one of the retry modes
if value != 'legacy' && value != 'standard' && value != 'adaptive'
raise ArgumentError,
'Must provide either `legacy`, `standard`, or `adaptive` for '\
'retry_mode profile option or for ENV[\'AWS_RETRY_MODE\']'
'Must provide either `legacy`, `standard`, or `adaptive` for '\
'retry_mode profile option or for ENV[\'AWS_RETRY_MODE\']'
end
value
end
Expand All @@ -190,40 +186,41 @@ def self.resolve_max_attempts(cfg)
# Raise if provided value is not a positive integer
if value <= 0
raise ArgumentError,
'Must provide a positive integer for max_attempts profile '\
'option or for ENV[\'AWS_MAX_ATTEMPTS\']'
'Must provide a positive integer for max_attempts profile '\
'option or for ENV[\'AWS_MAX_ATTEMPTS\']'
end
value
end

def self.resolve_adaptive_retry_wait_to_fill(cfg)
value = ENV['AWS_ADAPTIVE_RETRY_WAIT_TO_FILL'] ||
Aws.shared_config.adaptive_retry_wait_to_fill(profile: cfg.profile) ||
'true'
Aws.shared_config.adaptive_retry_wait_to_fill(profile: cfg.profile) ||
'true'
# Raise if provided value is not true or false
if value != 'true' && value != 'false'
raise ArgumentError,
'Must provide either `true` or `false` for '\
'adaptive_retry_wait_to_fill profile option or for '\
'ENV[\'AWS_ADAPTIVE_RETRY_WAIT_TO_FILL\']'
'Must provide either `true` or `false` for '\
'adaptive_retry_wait_to_fill profile option or for '\
'ENV[\'AWS_ADAPTIVE_RETRY_WAIT_TO_FILL\']'
end
value == 'true'
end

def self.resolve_correct_clock_skew(cfg)
value = ENV['AWS_CORRECT_CLOCK_SKEW'] ||
Aws.shared_config.correct_clock_skew(profile: cfg.profile) ||
'true'
Aws.shared_config.correct_clock_skew(profile: cfg.profile) ||
'true'
# Raise if provided value is not true or false
if value != 'true' && value != 'false'
raise ArgumentError,
'Must provide either `true` or `false` for '\
'correct_clock_skew profile option or for '\
'ENV[\'AWS_CORRECT_CLOCK_SKEW\']'
'Must provide either `true` or `false` for '\
'correct_clock_skew profile option or for '\
'ENV[\'AWS_CORRECT_CLOCK_SKEW\']'
end
value == 'true'
end

# @api private
class Handler < Seahorse::Client::Handler
# Max backoff (in seconds)
MAX_BACKOFF = 20
Expand All @@ -249,9 +246,7 @@ def call(context)
# Clock correction needs to be updated from the response even when
# the request is not retryable but should only be updated
# in the case of clock skew errors
if error_inspector.clock_skew?(context)
config.clock_skew.update_clock_correction(context)
end
config.clock_skew.update_clock_correction(context) if error_inspector.clock_skew?(context)

# Estimated skew needs to be updated on every request
config.clock_skew.update_estimated_skew(context)
Expand All @@ -262,7 +257,7 @@ def call(context)

context.metadata[:retries][:capacity_amount] =
config.retry_quota.checkout_capacity(error_inspector)
return response unless context.metadata[:retries][:capacity_amount] > 0
return response unless (context.metadata[:retries][:capacity_amount]).positive?

delay = exponential_backoff(context.retries)
Kernel.sleep(delay)
Expand Down Expand Up @@ -348,9 +343,7 @@ def compute_request_ttl(context)

endpoint = context.http_request.endpoint
estimated_skew = context.config.clock_skew.estimated_skew(endpoint)
if context.config.respond_to?(:http_read_timeout)
read_timeout = context.config.http_read_timeout
end
read_timeout = context.config.http_read_timeout if context.config.respond_to?(:http_read_timeout)

if estimated_skew && read_timeout
(Time.now.utc + read_timeout + estimated_skew)
Expand All @@ -359,8 +352,8 @@ def compute_request_ttl(context)
end
end

# @api private
class LegacyHandler < Seahorse::Client::Handler

def call(context)
response = with_metric { @handler.call(context) }
if response.error
Expand Down Expand Up @@ -429,9 +422,7 @@ def response_truncatable?(context)

def add_handlers(handlers, config)
if config.retry_mode == 'legacy'
if config.retry_limit > 0
handlers.add(LegacyHandler, step: :sign, priority: 99)
end
handlers.add(LegacyHandler, step: :sign, priority: 99) if config.retry_limit.positive?
else
handlers.add(Handler, step: :sign, priority: 99)
end
Expand Down
2 changes: 1 addition & 1 deletion gems/aws-sdk-core/spec/aws/client_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ module Aws
end

expect(e).to be_kind_of(Errors::NoSuchEndpointError)
expect(e.context.retries).to be(3) # updated to retry based on customer request
expect(e.context.retries).to be(2) # updated to retry based on customer request
expect(e.message).to include('us-east-1')
expect(e.message).to include('us-west-1')
expect(e.message).to include('cn-north-1')
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,6 @@ module Plugins
expect(client.config.retry_jitter).to eq(:none)
end

it 'defaults config.retry_mode to legacy' do
expect(client.config.retry_mode).to eq('legacy')
end

it 'uses the legacy handler when retry_mode is legacy' do
client = RetryErrorsSvc::Client.new(retry_mode: 'legacy', region: 'us-west-2')
expect(client.handlers.entries.map(&:handler_class)).to include(RetryErrors::LegacyHandler)
Expand Down
4 changes: 4 additions & 0 deletions gems/aws-sdk-core/spec/aws/plugins/retry_errors_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,10 @@ module Plugins
describe RetryErrors do
let(:client) { RetryErrorsSvc::Client.new(stub_responses: true) }

it 'defaults config.retry_mode to standard' do
expect(client.config.retry_mode).to eq('standard')
end

it 'can configure retry_mode with shared config' do
allow_any_instance_of(Aws::SharedConfig)
.to receive(:retry_mode).and_return('standard')
Expand Down
2 changes: 2 additions & 0 deletions gems/aws-sdk-dynamodb/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
Unreleased Changes
------------------

* Feature - Increase the retry limit for the `standard` retry mode to 10 retries.

1.142.0 (2025-04-28)
------------------

Expand Down
Loading