Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add rolling_deploy_on_docker_failure option #180

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Add rolling_deploy_on_docker_failure option #180

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
dcbf8a0
Add rolling_deploy_on_docker_failure option
Oct 6, 2016
77655b6
Rename to rolling_deploy_on_failure, reraise last error
Feb 28, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -411,6 +411,7 @@ are the same everywhere. Settings are per-project.
ports are not HTTP services, this allows you to only health check the ports
that are. The default is an empty array. If you have non-HTTP services that you
want to check, see Custom Health Checks in the previous section.
* `rolling_deploy_on_failure` => What to do when Centurion encounters an error stopping or starting a container during a rolling deploy. By default, when an error is encountered the deploy will stop and immediately raise that error. If this option is set to `:continue` Centurion will continue deploying to the remaining hosts and raise at the end.

###Deploy a project to a fleet of Docker servers

Expand Down
17 changes: 14 additions & 3 deletions lib/tasks/deploy.rake
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -134,12 +134,19 @@ namespace :deploy do
end

task :rolling_deploy do
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be good to validate that the contents of rolling_deploy_on_failure are one of the expected options.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you rather we have a validate_rolling_deploy_options dependent task? I don't believe we perform any validation on the values of other rolling deploy options, could take a swing at that. Or just put the one validation at the top of this task if you think it's better to keep this changeset smaller.

stop_start_errors = []
on_each_docker_host do |server|
service = defined_service

stop_containers(server, service, fetch(:stop_timeout, 30))

container = start_new_container(server, service, defined_restart_policy)
begin
stop_containers(server, service, fetch(:stop_timeout, 30))
container = start_new_container(server, service, defined_restart_policy)
rescue e
on_fail = fetch(:rolling_deploy_on_failure, :exit)
raise unless on_fail == :continue
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regarding the validation discussion, I'd greenlight this if a log line here made it obvious what happened. Something like

if on_fail == :continue
  info "Caught error #{e.message}, but continuing deploy because rolling_deploy_on_failure is #{on_fail}"
else
  error "Raising exception, as rolling_deploy_on_failure was #{on_fail} and not :continue"
  raise
end

stop_start_errors << e.message
next
end

public_ports = service.public_ports - fetch(:rolling_deploy_skip_ports, [])
public_ports.each do |port|
Expand All @@ -158,6 +165,10 @@ namespace :deploy do

wait_for_load_balancer_check_interval
end

unless stop_start_errors.empty?
raise stop_start_errors.join("\n")
end
end

task :repair do
Expand Down