Skip to content

Conversation

@pburkholder
Copy link

Trying to address a few problems with this PR:

  1. Running all the tasks in sequence without waiting for them resulting in a big spike in resource consumption, hitting quota, and the failing the last run-tasks
  2. Using --wait apparently uncovers some failures that were masked when we just kicked off the task and hoped for the best
  3. Using set -e would cause the later tasks not to run if there was a failure earlier.

I non-prod I noticed that some of the tasks with --wait are failing.

We don't want failing tasks to kill the entire job. We also don't want to bury failures. So this will keep track of the failues and mark the tasks as failing in CircleCI -- while still running as many as possible.
@pburkholder
Copy link
Author

Meanwhile, I've bumped the quota another 5Gb to allow the current cronjob to complete.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant