Skip to content

feat: Add a rds-instance-stop chaos fault #710

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 5 commits into from
Apr 24, 2025
Merged

feat: Add a rds-instance-stop chaos fault #710

merged 5 commits into from
Apr 24, 2025

Conversation

jongwooo
Copy link
Contributor

@jongwooo jongwooo commented Sep 2, 2024 •
edited
Loading

What this PR does / why we need it:

I have implemented the rds-instance-stop chaos fault in this PR based on the proposal.

screenshot

Please see also litmuschaos/chaos-charts#635

Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #

Special notes for your reviewer:

cc. @namkyu1999

Checklist:

  • Fixes #
  • PR messages has document related information
  • Labelled this PR & related issue with breaking-changes tag
  • PR messages has breaking changes related information
  • Labelled this PR & related issue with requires-upgrade tag
  • PR messages has upgrade related information
  • Commit has unit tests
  • Commit has integration tests
  • E2E run Required for the changes

@jongwooo
Copy link
Contributor Author

jongwooo commented Sep 7, 2024 •
edited
Loading

Logs: 
INFO[2024-09-07T12:43:00Z] Experiment Name: rds-instance-stop
INFO[2024-09-07T12:43:00Z] [PreReq]: Getting the ENV for the rds-instance-stop experiment
INFO[2024-09-07T12:43:00Z] [PreReq]: Updating the chaos result of rds-instance-stop experiment (SOT)
INFO[2024-09-07T12:43:02Z] The instance information is as follows        Chaos Duration=30 Chaos Namespace=default Instance Identifier=database-1 Instance Affected Percentage=0 Sequence=parallel
INFO[2024-09-07T12:43:02Z] [Status]: Verify that the aws rds instances are in available state (pre-chaos)
INFO[2024-09-07T12:43:02Z] [Info]: The instances under chaos(IUC) are: [database-1]
INFO[2024-09-07T12:43:03Z] [Status]: RDS instance is in available state
INFO[2024-09-07T12:43:03Z] [Chaos]:Number of Instance targeted: 1
INFO[2024-09-07T12:43:03Z] [Info]: Target instance identifier list, [database-1]
INFO[2024-09-07T12:43:03Z] [Chaos]: Stopping the desired RDS instance
INFO[2024-09-07T12:43:03Z] Stopping RDS instance:                        DBInstanceStatus=stopping DBInstanceIdentifier=database-1
INFO[2024-09-07T12:43:03Z] [Wait]: Wait for RDS instance 'database-1' to get in stopped state
INFO[2024-09-07T12:43:03Z] [Status]: Checking RDS instance status
INFO[2024-09-07T12:43:03Z] The instance state is stopping
INFO[2024-09-07T12:43:05Z] The instance state is stopping
INFO[2024-09-07T12:43:07Z] The instance state is stopping
INFO[2024-09-07T12:43:09Z] The instance state is stopping
INFO[2024-09-07T12:43:12Z] The instance state is stopping
INFO[2024-09-07T12:43:14Z] The instance state is stopping
INFO[2024-09-07T12:43:16Z] The instance state is stopping
INFO[2024-09-07T12:43:18Z] The instance state is stopping
INFO[2024-09-07T12:43:20Z] The instance state is stopping
INFO[2024-09-07T12:43:22Z] The instance state is stopping
INFO[2024-09-07T12:43:24Z] The instance state is stopping
INFO[2024-09-07T12:43:26Z] The instance state is stopping
INFO[2024-09-07T12:43:29Z] The instance state is stopping
INFO[2024-09-07T12:43:31Z] The instance state is stopping
INFO[2024-09-07T12:43:33Z] The instance state is stopping
INFO[2024-09-07T12:43:35Z] The instance state is stopping
INFO[2024-09-07T12:43:37Z] The instance state is stopping
INFO[2024-09-07T12:43:39Z] The instance state is stopping
INFO[2024-09-07T12:43:41Z] The instance state is stopping
INFO[2024-09-07T12:43:44Z] The instance state is stopping
INFO[2024-09-07T12:43:46Z] The instance state is stopping
INFO[2024-09-07T12:43:48Z] The instance state is stopping
INFO[2024-09-07T12:43:50Z] The instance state is stopping
INFO[2024-09-07T12:43:52Z] The instance state is stopping
INFO[2024-09-07T12:43:54Z] The instance state is stopping
INFO[2024-09-07T12:43:56Z] The instance state is stopping
INFO[2024-09-07T12:43:59Z] The instance state is stopping
INFO[2024-09-07T12:44:01Z] The instance state is stopping
INFO[2024-09-07T12:44:03Z] The instance state is stopping
INFO[2024-09-07T12:44:05Z] The instance state is stopping
INFO[2024-09-07T12:44:07Z] The instance state is stopping
INFO[2024-09-07T12:44:10Z] The instance state is stopping
INFO[2024-09-07T12:44:12Z] The instance state is stopping
INFO[2024-09-07T12:44:14Z] The instance state is stopping
INFO[2024-09-07T12:44:16Z] The instance state is stopping
INFO[2024-09-07T12:44:18Z] The instance state is stopping
INFO[2024-09-07T12:44:20Z] The instance state is stopping
INFO[2024-09-07T12:44:22Z] The instance state is stopping
INFO[2024-09-07T12:44:25Z] The instance state is stopping
INFO[2024-09-07T12:44:27Z] The instance state is stopping
INFO[2024-09-07T12:44:29Z] The instance state is stopping
INFO[2024-09-07T12:44:31Z] The instance state is stopping
INFO[2024-09-07T12:44:33Z] The instance state is stopping
INFO[2024-09-07T12:44:35Z] The instance state is stopping
INFO[2024-09-07T12:44:37Z] The instance state is stopping
INFO[2024-09-07T12:44:40Z] The instance state is stopping
INFO[2024-09-07T12:44:42Z] The instance state is stopping
INFO[2024-09-07T12:44:44Z] The instance state is stopping
INFO[2024-09-07T12:44:46Z] The instance state is stopping
INFO[2024-09-07T12:44:48Z] The instance state is stopping
INFO[2024-09-07T12:44:50Z] The instance state is stopping
INFO[2024-09-07T12:44:52Z] The instance state is stopping
INFO[2024-09-07T12:44:55Z] The instance state is stopping
INFO[2024-09-07T12:44:57Z] The instance state is stopping
INFO[2024-09-07T12:44:59Z] The instance state is stopping
INFO[2024-09-07T12:45:01Z] The instance state is stopping
INFO[2024-09-07T12:45:03Z] The instance state is stopping
INFO[2024-09-07T12:45:05Z] The instance state is stopping
INFO[2024-09-07T12:45:07Z] The instance state is stopping
INFO[2024-09-07T12:45:10Z] The instance state is stopping
INFO[2024-09-07T12:45:12Z] The instance state is stopping
INFO[2024-09-07T12:45:14Z] The instance state is stopping
INFO[2024-09-07T12:45:16Z] The instance state is stopping
INFO[2024-09-07T12:45:18Z] The instance state is stopping
INFO[2024-09-07T12:45:20Z] The instance state is stopping
INFO[2024-09-07T12:45:22Z] The instance state is stopping
INFO[2024-09-07T12:45:24Z] The instance state is stopping
INFO[2024-09-07T12:45:27Z] The instance state is stopping
INFO[2024-09-07T12:45:29Z] The instance state is stopping
INFO[2024-09-07T12:45:31Z] The instance state is stopping
INFO[2024-09-07T12:45:33Z] The instance state is stopping
INFO[2024-09-07T12:45:35Z] The instance state is stopping
INFO[2024-09-07T12:45:37Z] The instance state is stopping
INFO[2024-09-07T12:45:39Z] The instance state is stopping
INFO[2024-09-07T12:45:42Z] The instance state is stopping
INFO[2024-09-07T12:45:44Z] The instance state is stopping
INFO[2024-09-07T12:45:46Z] The instance state is stopping
INFO[2024-09-07T12:45:48Z] The instance state is stopping
INFO[2024-09-07T12:45:50Z] The instance state is stopping
INFO[2024-09-07T12:45:52Z] The instance state is stopping
INFO[2024-09-07T12:45:54Z] The instance state is stopping
INFO[2024-09-07T12:45:56Z] The instance state is stopping
INFO[2024-09-07T12:45:59Z] The instance state is stopping
INFO[2024-09-07T12:46:01Z] The instance state is stopping
INFO[2024-09-07T12:46:03Z] The instance state is stopping
INFO[2024-09-07T12:46:05Z] The instance state is stopping
INFO[2024-09-07T12:46:08Z] The instance state is stopping
INFO[2024-09-07T12:46:10Z] The instance state is stopping
INFO[2024-09-07T12:46:12Z] The instance state is stopping
INFO[2024-09-07T12:46:14Z] The instance state is stopping
INFO[2024-09-07T12:46:16Z] The instance state is stopping
INFO[2024-09-07T12:46:18Z] The instance state is stopping
INFO[2024-09-07T12:46:20Z] The instance state is stopping
INFO[2024-09-07T12:46:23Z] The instance state is stopping
INFO[2024-09-07T12:46:25Z] The instance state is stopping
INFO[2024-09-07T12:46:27Z] The instance state is stopping
INFO[2024-09-07T12:46:29Z] The instance state is stopping
INFO[2024-09-07T12:46:31Z] The instance state is stopping
INFO[2024-09-07T12:46:33Z] The instance state is stopping
INFO[2024-09-07T12:46:35Z] The instance state is stopping
INFO[2024-09-07T12:46:38Z] The instance state is stopping
INFO[2024-09-07T12:46:40Z] The instance state is stopping
INFO[2024-09-07T12:46:42Z] The instance state is stopping
INFO[2024-09-07T12:46:44Z] The instance state is stopping
INFO[2024-09-07T12:46:46Z] The instance state is stopping
INFO[2024-09-07T12:46:48Z] The instance state is stopping
INFO[2024-09-07T12:46:51Z] The instance state is stopping
INFO[2024-09-07T12:46:53Z] The instance state is stopping
INFO[2024-09-07T12:46:55Z] The instance state is stopping
INFO[2024-09-07T12:46:57Z] The instance state is stopping
INFO[2024-09-07T12:46:59Z] The instance state is stopping
INFO[2024-09-07T12:47:01Z] The instance state is stopping
INFO[2024-09-07T12:47:04Z] The instance state is stopping
INFO[2024-09-07T12:47:06Z] The instance state is stopping
INFO[2024-09-07T12:47:08Z] The instance state is stopping
INFO[2024-09-07T12:47:10Z] The instance state is stopping
INFO[2024-09-07T12:47:12Z] The instance state is stopping
INFO[2024-09-07T12:47:14Z] The instance state is stopping
INFO[2024-09-07T12:47:16Z] The instance state is stopping
INFO[2024-09-07T12:47:19Z] The instance state is stopping
INFO[2024-09-07T12:47:21Z] The instance state is stopping
INFO[2024-09-07T12:47:23Z] The instance state is stopping
INFO[2024-09-07T12:47:25Z] The instance state is stopping
INFO[2024-09-07T12:47:27Z] The instance state is stopping
INFO[2024-09-07T12:47:29Z] The instance state is stopping
INFO[2024-09-07T12:47:31Z] The instance state is stopping
INFO[2024-09-07T12:47:33Z] The instance state is stopping
INFO[2024-09-07T12:47:36Z] The instance state is stopping
INFO[2024-09-07T12:47:38Z] The instance state is stopping
INFO[2024-09-07T12:47:40Z] The instance state is stopping
INFO[2024-09-07T12:47:42Z] The instance state is stopping
INFO[2024-09-07T12:47:44Z] The instance state is stopping
INFO[2024-09-07T12:47:46Z] The instance state is stopping
INFO[2024-09-07T12:47:48Z] The instance state is stopping
INFO[2024-09-07T12:47:51Z] The instance state is stopping
INFO[2024-09-07T12:47:53Z] The instance state is stopping
INFO[2024-09-07T12:47:55Z] The instance state is stopping
INFO[2024-09-07T12:47:57Z] The instance state is stopping
INFO[2024-09-07T12:47:59Z] The instance state is stopping
INFO[2024-09-07T12:48:01Z] The instance state is stopping
INFO[2024-09-07T12:48:03Z] The instance state is stopping
INFO[2024-09-07T12:48:05Z] The instance state is stopping
INFO[2024-09-07T12:48:08Z] The instance state is stopping
INFO[2024-09-07T12:48:10Z] The instance state is stopping
INFO[2024-09-07T12:48:12Z] The instance state is stopping
INFO[2024-09-07T12:48:14Z] The instance state is stopping
INFO[2024-09-07T12:48:16Z] The instance state is stopping
INFO[2024-09-07T12:48:18Z] The instance state is stopping
INFO[2024-09-07T12:48:20Z] The instance state is stopping
INFO[2024-09-07T12:48:23Z] The instance state is stopping
INFO[2024-09-07T12:48:25Z] The instance state is stopping
INFO[2024-09-07T12:48:27Z] The instance state is stopping
INFO[2024-09-07T12:48:29Z] The instance state is stopping
INFO[2024-09-07T12:48:31Z] The instance state is stopping
INFO[2024-09-07T12:48:33Z] The instance state is stopping
INFO[2024-09-07T12:48:35Z] The instance state is stopping
INFO[2024-09-07T12:48:38Z] The instance state is stopping
INFO[2024-09-07T12:48:40Z] The instance state is stopping
INFO[2024-09-07T12:48:42Z] The instance state is stopping
INFO[2024-09-07T12:48:44Z] The instance state is stopping
INFO[2024-09-07T12:48:46Z] The instance state is stopping
INFO[2024-09-07T12:48:48Z] The instance state is stopping
INFO[2024-09-07T12:48:50Z] The instance state is stopping
INFO[2024-09-07T12:48:52Z] The instance state is stopping
INFO[2024-09-07T12:48:55Z] The instance state is stopping
INFO[2024-09-07T12:48:57Z] The instance state is stopping
INFO[2024-09-07T12:48:59Z] The instance state is stopping
INFO[2024-09-07T12:49:01Z] The instance state is stopping
INFO[2024-09-07T12:49:03Z] The instance state is stopping
INFO[2024-09-07T12:49:05Z] The instance state is stopping
INFO[2024-09-07T12:49:07Z] The instance state is stopping
INFO[2024-09-07T12:49:09Z] The instance state is stopping
INFO[2024-09-07T12:49:12Z] The instance state is stopping
INFO[2024-09-07T12:49:14Z] The instance state is stopping
INFO[2024-09-07T12:49:16Z] The instance state is stopping
INFO[2024-09-07T12:49:18Z] The instance state is stopping
INFO[2024-09-07T12:49:20Z] The instance state is stopping
INFO[2024-09-07T12:49:22Z] The instance state is stopping
INFO[2024-09-07T12:49:24Z] The instance state is stopping
INFO[2024-09-07T12:49:27Z] The instance state is stopping
INFO[2024-09-07T12:49:29Z] The instance state is stopping
INFO[2024-09-07T12:49:31Z] The instance state is stopping
INFO[2024-09-07T12:49:33Z] The instance state is stopping
INFO[2024-09-07T12:49:35Z] The instance state is stopping
INFO[2024-09-07T12:49:37Z] The instance state is stopping
INFO[2024-09-07T12:49:39Z] The instance state is stopping
INFO[2024-09-07T12:49:42Z] The instance state is stopping
INFO[2024-09-07T12:49:44Z] The instance state is stopping
INFO[2024-09-07T12:49:46Z] The instance state is stopping
INFO[2024-09-07T12:49:48Z] The instance state is stopping
INFO[2024-09-07T12:49:50Z] The instance state is stopping
INFO[2024-09-07T12:49:52Z] The instance state is stopping
INFO[2024-09-07T12:49:55Z] The instance state is stopping
INFO[2024-09-07T12:49:57Z] The instance state is stopping
INFO[2024-09-07T12:49:59Z] The instance state is stopping
INFO[2024-09-07T12:50:01Z] The instance state is stopping
INFO[2024-09-07T12:50:03Z] The instance state is stopping
INFO[2024-09-07T12:50:05Z] The instance state is stopping
INFO[2024-09-07T12:50:09Z] The instance state is stopping
INFO[2024-09-07T12:50:11Z] The instance state is stopping
INFO[2024-09-07T12:50:13Z] The instance state is stopping
INFO[2024-09-07T12:50:15Z] The instance state is stopping
INFO[2024-09-07T12:50:17Z] The instance state is stopping
INFO[2024-09-07T12:50:19Z] The instance state is stopping
INFO[2024-09-07T12:50:21Z] The instance state is stopping
INFO[2024-09-07T12:50:24Z] The instance state is stopping
INFO[2024-09-07T12:50:26Z] The instance state is stopping
INFO[2024-09-07T12:50:28Z] The instance state is stopping
INFO[2024-09-07T12:50:30Z] The instance state is stopping
INFO[2024-09-07T12:50:32Z] The instance state is stopping
INFO[2024-09-07T12:50:34Z] The instance state is stopping
INFO[2024-09-07T12:50:36Z] The instance state is stopping
INFO[2024-09-07T12:50:39Z] The instance state is stopping
INFO[2024-09-07T12:50:41Z] The instance state is stopping
INFO[2024-09-07T12:50:43Z] The instance state is stopping
INFO[2024-09-07T12:50:45Z] The instance state is stopping
INFO[2024-09-07T12:50:47Z] The instance state is stopping
INFO[2024-09-07T12:50:49Z] The instance state is stopping
INFO[2024-09-07T12:50:51Z] The instance state is stopping
INFO[2024-09-07T12:50:53Z] The instance state is stopping
INFO[2024-09-07T12:50:56Z] The instance state is stopping
INFO[2024-09-07T12:50:58Z] The instance state is stopping
INFO[2024-09-07T12:51:00Z] The instance state is stopping
INFO[2024-09-07T12:51:02Z] The instance state is stopping
INFO[2024-09-07T12:51:04Z] The instance state is stopping
INFO[2024-09-07T12:51:06Z] The instance state is stopping
INFO[2024-09-07T12:51:08Z] The instance state is stopping
INFO[2024-09-07T12:51:11Z] The instance state is stopping
INFO[2024-09-07T12:51:13Z] The instance state is stopping
INFO[2024-09-07T12:51:15Z] The instance state is stopping
INFO[2024-09-07T12:51:17Z] The instance state is stopping
INFO[2024-09-07T12:51:19Z] The instance state is stopping
INFO[2024-09-07T12:51:21Z] The instance state is stopping
INFO[2024-09-07T12:51:23Z] The instance state is stopping
INFO[2024-09-07T12:51:26Z] The instance state is stopping
INFO[2024-09-07T12:51:28Z] The instance state is stopping
INFO[2024-09-07T12:51:30Z] The instance state is stopping
INFO[2024-09-07T12:51:32Z] The instance state is stopping
INFO[2024-09-07T12:51:34Z] The instance state is stopping
INFO[2024-09-07T12:51:36Z] The instance state is stopping
INFO[2024-09-07T12:51:38Z] The instance state is stopping
INFO[2024-09-07T12:51:41Z] The instance state is stopping
INFO[2024-09-07T12:51:43Z] The instance state is stopping
INFO[2024-09-07T12:51:45Z] The instance state is stopping
INFO[2024-09-07T12:51:47Z] The instance state is stopping
INFO[2024-09-07T12:51:49Z] The instance state is stopping
INFO[2024-09-07T12:51:52Z] The instance state is stopping
INFO[2024-09-07T12:51:54Z] The instance state is stopping
INFO[2024-09-07T12:51:56Z] The instance state is stopping
INFO[2024-09-07T12:51:58Z] The instance state is stopping
INFO[2024-09-07T12:52:00Z] The instance state is stopped
INFO[2024-09-07T12:52:02Z] [Wait]: Waiting for chaos interval of 30s
INFO[2024-09-07T12:52:32Z] [Chaos]: Starting back the RDS instance
INFO[2024-09-07T12:52:32Z] Starting RDS instance:                        DBInstanceStatus=starting DBInstanceIdentifier=database-1
INFO[2024-09-07T12:52:32Z] [Wait]: Wait for RDS instance 'database-1' to get in available state
INFO[2024-09-07T12:52:32Z] [Status]: Checking RDS instance status
INFO[2024-09-07T12:52:33Z] The instance state is starting
INFO[2024-09-07T12:52:35Z] The instance state is starting
INFO[2024-09-07T12:52:37Z] The instance state is starting
INFO[2024-09-07T12:52:39Z] The instance state is starting
INFO[2024-09-07T12:52:41Z] The instance state is starting
INFO[2024-09-07T12:52:43Z] The instance state is starting
INFO[2024-09-07T12:52:45Z] The instance state is starting
INFO[2024-09-07T12:52:47Z] The instance state is starting
INFO[2024-09-07T12:52:50Z] The instance state is starting
INFO[2024-09-07T12:52:52Z] The instance state is starting
INFO[2024-09-07T12:52:54Z] The instance state is starting
INFO[2024-09-07T12:52:56Z] The instance state is starting
INFO[2024-09-07T12:52:58Z] The instance state is starting
INFO[2024-09-07T12:53:00Z] The instance state is starting
INFO[2024-09-07T12:53:02Z] The instance state is starting
INFO[2024-09-07T12:53:05Z] The instance state is starting
INFO[2024-09-07T12:53:07Z] The instance state is starting
INFO[2024-09-07T12:53:09Z] The instance state is starting
INFO[2024-09-07T12:53:11Z] The instance state is starting
INFO[2024-09-07T12:53:13Z] The instance state is starting
INFO[2024-09-07T12:53:15Z] The instance state is starting
INFO[2024-09-07T12:53:18Z] The instance state is starting
INFO[2024-09-07T12:53:20Z] The instance state is starting
INFO[2024-09-07T12:53:22Z] The instance state is starting
INFO[2024-09-07T12:53:24Z] The instance state is starting
INFO[2024-09-07T12:53:26Z] The instance state is starting
INFO[2024-09-07T12:53:28Z] The instance state is starting
INFO[2024-09-07T12:53:31Z] The instance state is starting
INFO[2024-09-07T12:53:33Z] The instance state is starting
INFO[2024-09-07T12:53:35Z] The instance state is starting
INFO[2024-09-07T12:53:37Z] The instance state is starting
INFO[2024-09-07T12:53:39Z] The instance state is starting
INFO[2024-09-07T12:53:41Z] The instance state is starting
INFO[2024-09-07T12:53:43Z] The instance state is starting
INFO[2024-09-07T12:53:46Z] The instance state is starting
INFO[2024-09-07T12:53:48Z] The instance state is starting
INFO[2024-09-07T12:53:50Z] The instance state is starting
INFO[2024-09-07T12:53:52Z] The instance state is starting
INFO[2024-09-07T12:53:54Z] The instance state is starting
INFO[2024-09-07T12:53:56Z] The instance state is starting
INFO[2024-09-07T12:53:58Z] The instance state is starting
INFO[2024-09-07T12:54:01Z] The instance state is starting
INFO[2024-09-07T12:54:03Z] The instance state is starting
INFO[2024-09-07T12:54:05Z] The instance state is starting
INFO[2024-09-07T12:54:07Z] The instance state is starting
INFO[2024-09-07T12:54:09Z] The instance state is starting
INFO[2024-09-07T12:54:11Z] The instance state is starting
INFO[2024-09-07T12:54:13Z] The instance state is starting
INFO[2024-09-07T12:54:16Z] The instance state is starting
INFO[2024-09-07T12:54:18Z] The instance state is starting
INFO[2024-09-07T12:54:20Z] The instance state is starting
INFO[2024-09-07T12:54:22Z] The instance state is starting
INFO[2024-09-07T12:54:24Z] The instance state is starting
INFO[2024-09-07T12:54:26Z] The instance state is starting
INFO[2024-09-07T12:54:29Z] The instance state is starting
INFO[2024-09-07T12:54:31Z] The instance state is starting
INFO[2024-09-07T12:54:33Z] The instance state is starting
INFO[2024-09-07T12:54:35Z] The instance state is starting
INFO[2024-09-07T12:54:37Z] The instance state is starting
INFO[2024-09-07T12:54:39Z] The instance state is starting
INFO[2024-09-07T12:54:41Z] The instance state is starting
INFO[2024-09-07T12:54:43Z] The instance state is starting
INFO[2024-09-07T12:54:46Z] The instance state is starting
INFO[2024-09-07T12:54:48Z] The instance state is starting
INFO[2024-09-07T12:54:50Z] The instance state is starting
INFO[2024-09-07T12:54:52Z] The instance state is starting
INFO[2024-09-07T12:54:54Z] The instance state is starting
INFO[2024-09-07T12:54:56Z] The instance state is starting
INFO[2024-09-07T12:54:58Z] The instance state is starting
INFO[2024-09-07T12:55:00Z] The instance state is starting
INFO[2024-09-07T12:55:03Z] The instance state is starting
INFO[2024-09-07T12:55:05Z] The instance state is starting
INFO[2024-09-07T12:55:07Z] The instance state is starting
INFO[2024-09-07T12:55:09Z] The instance state is starting
INFO[2024-09-07T12:55:11Z] The instance state is starting
INFO[2024-09-07T12:55:13Z] The instance state is starting
INFO[2024-09-07T12:55:15Z] The instance state is starting
INFO[2024-09-07T12:55:18Z] The instance state is starting
INFO[2024-09-07T12:55:20Z] The instance state is starting
INFO[2024-09-07T12:55:22Z] The instance state is starting
INFO[2024-09-07T12:55:24Z] The instance state is starting
INFO[2024-09-07T12:55:26Z] The instance state is starting
INFO[2024-09-07T12:55:28Z] The instance state is starting
INFO[2024-09-07T12:55:30Z] The instance state is starting
INFO[2024-09-07T12:55:32Z] The instance state is starting
INFO[2024-09-07T12:55:35Z] The instance state is starting
INFO[2024-09-07T12:55:37Z] The instance state is starting
INFO[2024-09-07T12:55:39Z] The instance state is starting
INFO[2024-09-07T12:55:41Z] The instance state is starting
INFO[2024-09-07T12:55:43Z] The instance state is starting
INFO[2024-09-07T12:55:46Z] The instance state is starting
INFO[2024-09-07T12:55:48Z] The instance state is starting
INFO[2024-09-07T12:55:50Z] The instance state is starting
INFO[2024-09-07T12:55:52Z] The instance state is starting
INFO[2024-09-07T12:55:54Z] The instance state is starting
INFO[2024-09-07T12:55:56Z] The instance state is starting
INFO[2024-09-07T12:55:58Z] The instance state is starting
INFO[2024-09-07T12:56:01Z] The instance state is starting
INFO[2024-09-07T12:56:03Z] The instance state is starting
INFO[2024-09-07T12:56:05Z] The instance state is starting
INFO[2024-09-07T12:56:07Z] The instance state is starting
INFO[2024-09-07T12:56:09Z] The instance state is starting
INFO[2024-09-07T12:56:11Z] The instance state is starting
INFO[2024-09-07T12:56:13Z] The instance state is starting
INFO[2024-09-07T12:56:16Z] The instance state is starting
INFO[2024-09-07T12:56:18Z] The instance state is starting
INFO[2024-09-07T12:56:20Z] The instance state is starting
INFO[2024-09-07T12:56:22Z] The instance state is starting
INFO[2024-09-07T12:56:24Z] The instance state is starting
INFO[2024-09-07T12:56:26Z] The instance state is starting
INFO[2024-09-07T12:56:29Z] The instance state is starting
INFO[2024-09-07T12:56:31Z] The instance state is starting
INFO[2024-09-07T12:56:33Z] The instance state is starting
INFO[2024-09-07T12:56:35Z] The instance state is starting
INFO[2024-09-07T12:56:37Z] The instance state is starting
INFO[2024-09-07T12:56:39Z] The instance state is starting
INFO[2024-09-07T12:56:41Z] The instance state is starting
INFO[2024-09-07T12:56:44Z] The instance state is starting
INFO[2024-09-07T12:56:46Z] The instance state is starting
INFO[2024-09-07T12:56:48Z] The instance state is starting
INFO[2024-09-07T12:56:50Z] The instance state is starting
INFO[2024-09-07T12:56:52Z] The instance state is starting
INFO[2024-09-07T12:56:54Z] The instance state is starting
INFO[2024-09-07T12:56:56Z] The instance state is starting
INFO[2024-09-07T12:56:59Z] The instance state is starting
INFO[2024-09-07T12:57:01Z] The instance state is starting
INFO[2024-09-07T12:57:03Z] The instance state is starting
INFO[2024-09-07T12:57:05Z] The instance state is starting
INFO[2024-09-07T12:57:07Z] The instance state is starting
INFO[2024-09-07T12:57:09Z] The instance state is starting
INFO[2024-09-07T12:57:11Z] The instance state is starting
INFO[2024-09-07T12:57:14Z] The instance state is starting
INFO[2024-09-07T12:57:16Z] The instance state is starting
INFO[2024-09-07T12:57:18Z] The instance state is starting
INFO[2024-09-07T12:57:20Z] The instance state is starting
INFO[2024-09-07T12:57:22Z] The instance state is starting
INFO[2024-09-07T12:57:24Z] The instance state is available
INFO[2024-09-07T12:57:26Z] [Confirmation]: rds-instance-stop chaos has been injected successfully
INFO[2024-09-07T12:57:26Z] [Status]: Verify that the aws rds instances are in available state (post-chaos)
INFO[2024-09-07T12:57:26Z] [Info]: The instances under chaos(IUC) are: [database-1]
INFO[2024-09-07T12:57:27Z] [Status]: RDS instance is in available state (post chaos)
INFO[2024-09-07T12:57:27Z] [The End]: Updating the chaos result of rds-instance-stop experiment (EOT)

@jongwooo jongwooo marked this pull request as ready for review September 7, 2024 13:17
jongwooo
jongwooo commented Sep 8, 2024
View reviewed changes
pkg/kube-aws/rds-instance-stop/environment/environment.go
experimentDetails.InstanceID = types.Getenv("INSTANCE_ID", "")
experimentDetails.ChaosPodName = types.Getenv("POD_NAME", "")
experimentDetails.Delay, _ = strconv.Atoi(types.Getenv("STATUS_CHECK_DELAY", "2"))
experimentDetails.Timeout, _ = strconv.Atoi(types.Getenv("STATUS_CHECK_TIMEOUT", "600"))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stopping an RDS instance takes about 10 minutes. Therefore, I set STATUS_CHECK_TIMEOUT to 600.

@namkyu1999 namkyu1999 added the enhancement New feature or request label Feb 16, 2025
uditgaurav
uditgaurav reviewed Apr 24, 2025
View reviewed changes
chaoslib/litmus/rds-instance-stop/lib/rds-instance-stop.go
Comment on lines +142 to +145
log.Infof("[Wait]: Wait for RDS instance '%v' to get in available state", identifier)
if err := awslib.WaitForRDSInstanceUp(experimentsDetails.Timeout, experimentsDetails.Delay, experimentsDetails.Region, identifier); err != nil {
return stacktrace.Propagate(err, "rds instance failed to start")
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instances can remain stopped for up to 168 hours. This duration is generally sufficient, so it should not pose any issues.

@uditgaurav
Copy link
Member

Thank you so much for this contribution and your patience! 🙌

uditgaurav
uditgaurav reviewed Apr 24, 2025
View reviewed changes
uditgaurav
uditgaurav reviewed Apr 24, 2025
View reviewed changes
uditgaurav
uditgaurav reviewed Apr 24, 2025
View reviewed changes
uditgaurav
uditgaurav reviewed Apr 24, 2025
View reviewed changes
experiments/kube-aws/rds-instance-stop/README.md
<tr>
<td> RDS Instance Stop </td>
<td> This experiment causes termination of an RDS instance before bringing it back to available state using the instance identifier after the specified chaos duration. We can also control the number of target instance using instance affected percentage</td>
<td> <a href="https://litmuschaos.github.io/litmus/experiments/categories/aws/rds-instance-stop/"> Here </a> </td>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to add this document then provide the URL

@uditgaurav
Copy link
Member

The requested changes can be taken in the subsequent PRs. Thanks!

ispeakc0de
ispeakc0de reviewed Apr 24, 2025
View reviewed changes
experiments/kube-aws/rds-instance-stop/experiment/rds-instance-stop.go

if experimentsDetails.EngineName != "" {
// Marking AUT as running, as we already checked the status of application under test
msg := "AUT: Running"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
msg := "AUT: Running"
msg := "RDS Instance: Running"

ispeakc0de
ispeakc0de reviewed Apr 24, 2025
View reviewed changes
experiments/kube-aws/rds-instance-stop/experiment/rds-instance-stop.go

// Verify the aws rds instance is available (pre-chaos)
if chaosDetails.DefaultHealthCheck {
log.Info("[Status]: Verify that the aws rds instances are in available state (pre-chaos)")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should be added before the the probe - line#85

ispeakc0de
ispeakc0de reviewed Apr 24, 2025
View reviewed changes
experiments/kube-aws/rds-instance-stop/experiment/rds-instance-stop.go

if experimentsDetails.EngineName != "" {
// Marking AUT as running, as we already checked the status of application under test
msg := "AUT: Running"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same as above

@uditgaurav uditgaurav merged commit 5c22472 into litmuschaos:master Apr 24, 2025
8 of 9 checks passed
@jongwooo jongwooo mentioned this pull request Apr 24, 2025
Merged
ThaSami pushed a commit to ThaSami/litmus-go that referenced this pull request Apr 24, 2025
@jongwooo @ThaSami
feat: Add a rds-instance-stop chaos fault (litmuschaos#710)
841bb72 
* feat: Add a rds-instance-stop chaos fault

Signed-off-by: Jongwoo Han <[email protected]>

---------

Signed-off-by: Jongwoo Han <[email protected]>
Signed-off-by: Sami Shabaneh <[email protected]>
@jongwooo jongwooo deleted the add-a-rds-instance-stop-chaos-fault branch April 28, 2025 23:34
uditgaurav added a commit that referenced this pull request Apr 30, 2025
@ThaSami @neelanjan00
@uditgaurav @bjoky @ispeakc0de @jongwooo
Fix: handle pagination in ssm describeInstanceInformation & API Rate …
07de11e 
…Limit (#738)

* Fix: handle pagination in ssm describe

Signed-off-by: Sami Shabaneh <[email protected]>

* implement exponential backoff with jitter for API rate limiting

Signed-off-by: Sami Shabaneh <[email protected]>

* Refactor

Signed-off-by: Sami Shabaneh <[email protected]>

* Update pkg/cloud/aws/ssm/ssm-operations.go

Co-authored-by: Neelanjan Manna <[email protected]>
Signed-off-by: Sami Shabaneh <[email protected]>

* fixup

Signed-off-by: Sami Shabaneh <[email protected]>

* Update pkg/cloud/aws/ssm/ssm-operations.go

Co-authored-by: Udit Gaurav <[email protected]>
Signed-off-by: Sami Shabaneh <[email protected]>

* Fix: include error message from stderr if container-kill fails (#740) (#741)

Signed-off-by: Björn Kylberg <[email protected]>
Signed-off-by: Sami Shabaneh <[email protected]>

* fix(logs): Fix the error logs for container-kill fault (#745)

Signed-off-by: Shubham Chaudhary <[email protected]>
Signed-off-by: Sami Shabaneh <[email protected]>

* fix(container-kill): Fixed the container stop command timeout issue (#747)

Signed-off-by: Shubham Chaudhary <[email protected]>
Signed-off-by: Sami Shabaneh <[email protected]>

* feat: Add a rds-instance-stop chaos fault (#710)

* feat: Add a rds-instance-stop chaos fault

Signed-off-by: Jongwoo Han <[email protected]>

---------

Signed-off-by: Jongwoo Han <[email protected]>
Signed-off-by: Sami Shabaneh <[email protected]>

* Update pkg/cloud/aws/ssm/ssm-operations.go

Signed-off-by: Sami Shabaneh <[email protected]>

* fix go fmt ./...

Signed-off-by: Udit Gaurav <[email protected]>
Signed-off-by: Sami Shabaneh <[email protected]>

* Filter instances on api call

Signed-off-by: Sami Shabaneh <[email protected]>

* fixes lint

Signed-off-by: Udit Gaurav <[email protected]>

---------

Signed-off-by: Sami Shabaneh <[email protected]>
Signed-off-by: Björn Kylberg <[email protected]>
Signed-off-by: Shubham Chaudhary <[email protected]>
Signed-off-by: Jongwoo Han <[email protected]>
Signed-off-by: Udit Gaurav <[email protected]>
Co-authored-by: Neelanjan Manna <[email protected]>
Co-authored-by: Udit Gaurav <[email protected]>
Co-authored-by: Björn Kylberg <[email protected]>
Co-authored-by: Shubham Chaudhary <[email protected]>
Co-authored-by: Jongwoo Han <[email protected]>
Co-authored-by: Udit Gaurav <[email protected]>
SSanjeevi pushed a commit to SSanjeevi/litmus-go that referenced this pull request Jul 31, 2025
@jongwooo @SSanjeevi
feat: Add a rds-instance-stop chaos fault (litmuschaos#710)
1b4b0d6 
* feat: Add a rds-instance-stop chaos fault

Signed-off-by: Jongwoo Han <[email protected]>

---------

Signed-off-by: Jongwoo Han <[email protected]>
Signed-off-by: SSanjeevi <[email protected]>
SSanjeevi pushed a commit to SSanjeevi/litmus-go that referenced this pull request Jul 31, 2025
@ThaSami @neelanjan00
@uditgaurav @bjoky @ispeakc0de @jongwooo @SSanjeevi
Fix: handle pagination in ssm describeInstanceInformation & API Rate …
8b16d03 
…Limit (litmuschaos#738)

* Fix: handle pagination in ssm describe

Signed-off-by: Sami Shabaneh <[email protected]>

* implement exponential backoff with jitter for API rate limiting

Signed-off-by: Sami Shabaneh <[email protected]>

* Refactor

Signed-off-by: Sami Shabaneh <[email protected]>

* Update pkg/cloud/aws/ssm/ssm-operations.go

Co-authored-by: Neelanjan Manna <[email protected]>
Signed-off-by: Sami Shabaneh <[email protected]>

* fixup

Signed-off-by: Sami Shabaneh <[email protected]>

* Update pkg/cloud/aws/ssm/ssm-operations.go

Co-authored-by: Udit Gaurav <[email protected]>
Signed-off-by: Sami Shabaneh <[email protected]>

* Fix: include error message from stderr if container-kill fails (litmuschaos#740) (litmuschaos#741)

Signed-off-by: Björn Kylberg <[email protected]>
Signed-off-by: Sami Shabaneh <[email protected]>

* fix(logs): Fix the error logs for container-kill fault (litmuschaos#745)

Signed-off-by: Shubham Chaudhary <[email protected]>
Signed-off-by: Sami Shabaneh <[email protected]>

* fix(container-kill): Fixed the container stop command timeout issue (litmuschaos#747)

Signed-off-by: Shubham Chaudhary <[email protected]>
Signed-off-by: Sami Shabaneh <[email protected]>

* feat: Add a rds-instance-stop chaos fault (litmuschaos#710)

* feat: Add a rds-instance-stop chaos fault

Signed-off-by: Jongwoo Han <[email protected]>

---------

Signed-off-by: Jongwoo Han <[email protected]>
Signed-off-by: Sami Shabaneh <[email protected]>

* Update pkg/cloud/aws/ssm/ssm-operations.go

Signed-off-by: Sami Shabaneh <[email protected]>

* fix go fmt ./...

Signed-off-by: Udit Gaurav <[email protected]>
Signed-off-by: Sami Shabaneh <[email protected]>

* Filter instances on api call

Signed-off-by: Sami Shabaneh <[email protected]>

* fixes lint

Signed-off-by: Udit Gaurav <[email protected]>

---------

Signed-off-by: Sami Shabaneh <[email protected]>
Signed-off-by: Björn Kylberg <[email protected]>
Signed-off-by: Shubham Chaudhary <[email protected]>
Signed-off-by: Jongwoo Han <[email protected]>
Signed-off-by: Udit Gaurav <[email protected]>
Co-authored-by: Neelanjan Manna <[email protected]>
Co-authored-by: Udit Gaurav <[email protected]>
Co-authored-by: Björn Kylberg <[email protected]>
Co-authored-by: Shubham Chaudhary <[email protected]>
Co-authored-by: Jongwoo Han <[email protected]>
Co-authored-by: Udit Gaurav <[email protected]>
Signed-off-by: SSanjeevi <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ksatchit ksatchit ksatchit approved these changes

@ispeakc0de ispeakc0de ispeakc0de approved these changes

@uditgaurav uditgaurav uditgaurav approved these changes

Assignees

@jongwooo jongwooo

Labels
enhancement New feature or request
Projects
Open Source Contribution Academy 2024
Status: Done
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@jongwooo @uditgaurav @ksatchit @ispeakc0de @namkyu1999