Skip to content

Commit 67b8380

Browse files
committed
Update test doc
1 parent 7682500 commit 67b8380

File tree

4 files changed

+37
-37
lines changed

4 files changed

+37
-37
lines changed

Testing.md

+35-35
Original file line numberDiff line numberDiff line change
@@ -1,20 +1,20 @@
11
## Testing Backup and Restore
2-
To test the project we need to simulate running a live system , taking it down and restoring the system. For this test we have two Docker Compose files for the system `docker-compose-live.yaml` and `docker-compose-restore-test.yaml` in a real restore scenario we will only have on Docker Compose file as we expect the restore is being done on a separate instance. We also have separate files for Backup and Restore (`docker-compose-backup-service.yaml` and `docker-compose-restore-service.yaml`)
2+
To test the project we need to simulate running a live system , taking it down and restoring the system. For this test we have two Docker Compose files for the system `docker-compose-live.yaml` and `docker-compose-live-two.yaml` in a real restore scenario we will only have on Docker Compose file as we expect the restore is being done on a separate instance. We also have separate files for Backup and Restore (`docker-compose-backup-service.yaml` and `docker-compose-restore-service.yaml`)
33
Run:
44

5-
```./start-live-system.sh```
5+
```./start-live.sh```
66
This will start the example live system;
77
Wait for > 5 minutes for the first backup to happen. You can confirm by running
88

9-
`docker logs -f live-system-backup-1`
9+
`docker logs -f live-backup-1`
1010

1111
You should see logs similar to:
1212

1313
```
1414
crond 4.5 dillon's cron daemon, started with loglevel notice
1515
2024-10-21 03:50:01,338 - INFO: Starting backup container
1616
2024-10-21 03:50:01,647 - INFO: Backup process container: compassionate_ganguly
17-
2024-10-21 03:50:01,848 - INFO: 2024-10-21 03:50:01,848 - INFO: Status for compose project 'live-system'
17+
2024-10-21 03:50:01,848 - INFO: 2024-10-21 03:50:01,848 - INFO: Status for compose project 'live'
1818
2024-10-21 03:50:01,848 - INFO: 2024-10-21 03:50:01,848 - INFO: Repository: '/restic_data'
1919
2024-10-21 03:50:01,849 - INFO: 2024-10-21 03:50:01,848 - INFO: Backup currently running?: True
2020
2024-10-21 03:50:01,849 - INFO: 2024-10-21 03:50:01,848 - INFO: Include project name in backup path?: False
@@ -28,28 +28,28 @@ crond 4.5 dillon's cron daemon, started with loglevel notice
2828
2024-10-21 03:50:03,933 - INFO: 2024-10-21 03:50:03,933 - INFO: Successfully initialized repository: /restic_data
2929
2024-10-21 03:50:03,933 - INFO: 2024-10-21 03:50:03,933 - INFO: ------------------------- Detected Config -------------------------
3030
2024-10-21 03:50:03,934 - INFO: 2024-10-21 03:50:03,933 - INFO: service: postgres
31-
2024-10-21 03:50:03,934 - INFO: 2024-10-21 03:50:03,934 - INFO: Container NAME live-system-postgres-1
32-
2024-10-21 03:50:03,940 - INFO: 2024-10-21 03:50:03,939 - INFO: Container NAME live-system-postgres-1
31+
2024-10-21 03:50:03,934 - INFO: 2024-10-21 03:50:03,934 - INFO: Container NAME live-postgres-1
32+
2024-10-21 03:50:03,940 - INFO: 2024-10-21 03:50:03,939 - INFO: Container NAME live-postgres-1
3333
2024-10-21 03:50:03,940 - INFO: 2024-10-21 03:50:03,940 - INFO: - postgres (is_ready=True) -> /backup/databases/postgres/test-postgres.sql
3434
2024-10-21 03:50:03,940 - INFO: 2024-10-21 03:50:03,940 - INFO: service: mariadb
35-
2024-10-21 03:50:03,941 - INFO: 2024-10-21 03:50:03,940 - INFO: Container NAME live-system-mariadb-1
35+
2024-10-21 03:50:03,941 - INFO: 2024-10-21 03:50:03,940 - INFO: Container NAME live-mariadb-1
3636
2024-10-21 03:50:03,948 - INFO: 2024-10-21 03:50:03,948 - INFO: - mariadb (is_ready=True) -> /backup/databases/mariadb/all_databases.sql
3737
2024-10-21 03:50:03,948 - INFO: 2024-10-21 03:50:03,948 - INFO: service: web
3838
2024-10-21 03:50:03,949 - INFO: 2024-10-21 03:50:03,948 - INFO: - volume: /workspace/restic-compose-backup/src/tests -> /backup/volumes/web/srv/tests
3939
2024-10-21 03:50:03,949 - INFO: 2024-10-21 03:50:03,948 - INFO: -------------------------------------------------------------------
4040
2024-10-21 03:50:03,949 - INFO: 2024-10-21 03:50:03,948 - INFO: Backing up databases
4141
2024-10-21 03:50:03,949 - INFO: 2024-10-21 03:50:03,948 - INFO: Backing up postgres in service postgres
42-
2024-10-21 03:50:03,950 - INFO: 2024-10-21 03:50:03,948 - INFO: Container NAME live-system-postgres-1
43-
2024-10-21 03:50:03,950 - INFO: 2024-10-21 03:50:03,948 - INFO: Container NAME live-system-postgres-1
42+
2024-10-21 03:50:03,950 - INFO: 2024-10-21 03:50:03,948 - INFO: Container NAME live-postgres-1
43+
2024-10-21 03:50:03,950 - INFO: 2024-10-21 03:50:03,948 - INFO: Container NAME live-postgres-1
4444
2024-10-21 03:50:03,950 - INFO: 2024-10-21 03:50:03,948 - INFO: Backing PostgresContainer container postgres /backup/databases/postgres/test-postgres.sql
45-
2024-10-21 03:50:03,950 - INFO: 2024-10-21 03:50:03,948 - INFO: Container NAME live-system-postgres-1
46-
2024-10-21 03:50:03,951 - INFO: 2024-10-21 03:50:03,949 - INFO: Container NAME live-system-postgres-1
47-
2024-10-21 03:50:03,951 - INFO: 2024-10-21 03:50:03,949 - INFO: Container NAME live-system-postgres-1
45+
2024-10-21 03:50:03,950 - INFO: 2024-10-21 03:50:03,948 - INFO: Container NAME live-postgres-1
46+
2024-10-21 03:50:03,951 - INFO: 2024-10-21 03:50:03,949 - INFO: Container NAME live-postgres-1
47+
2024-10-21 03:50:03,951 - INFO: 2024-10-21 03:50:03,949 - INFO: Container NAME live-postgres-1
4848
2024-10-21 03:50:04,145 - INFO: 2024-10-21 03:50:04,144 - INFO: Database backup exit code: =======> 0
4949
2024-10-21 03:50:04,145 - INFO: 2024-10-21 03:50:04,145 - INFO: Exit code: 0
5050
2024-10-21 03:50:04,146 - INFO: 2024-10-21 03:50:04,145 - INFO: Backing up mariadb in service mariadb
51-
2024-10-21 03:50:04,146 - INFO: 2024-10-21 03:50:04,145 - INFO: Container NAME live-system-mariadb-1
52-
2024-10-21 03:50:04,146 - INFO: 2024-10-21 03:50:04,145 - INFO: Container NAME live-system-mariadb-1
51+
2024-10-21 03:50:04,146 - INFO: 2024-10-21 03:50:04,145 - INFO: Container NAME live-mariadb-1
52+
2024-10-21 03:50:04,146 - INFO: 2024-10-21 03:50:04,145 - INFO: Container NAME live-mariadb-1
5353
2024-10-21 03:50:04,348 - INFO: 2024-10-21 03:50:04,347 - INFO: Database backup exit code: =======> 0
5454
2024-10-21 03:50:04,348 - INFO: 2024-10-21 03:50:04,347 - INFO: Exit code: 0
5555
2024-10-21 03:50:04,348 - INFO: 2024-10-21 03:50:04,347 - INFO: Backup files from ===> /backup/
@@ -62,7 +62,7 @@ crond 4.5 dillon's cron daemon, started with loglevel notice
6262

6363
### Validate database restore
6464

65-
```docker exec -it live-system-mariadb-1 bash```
65+
```docker exec -it live-mariadb-1 bash```
6666

6767
```mysql -uroot -pmy-secret-pw```
6868

@@ -85,15 +85,15 @@ crond 4.5 dillon's cron daemon, started with loglevel notice
8585

8686
Stop the live system to avoid conflicts as the backup service may try to backup the restored system (The backup and restore services are not intended to run on the same system)
8787

88-
```./stop-live-system.sh```
88+
```./stop-live.sh```
8989

9090
Run the restore
9191

92-
```./restore-backup.sh```
92+
```./start-live-two-with-restore.sh```
9393

9494
Check the restore logs
9595

96-
```docker logs -f restore-test-restore-1```
96+
```docker logs -f live-two-restore-1```
9797

9898
You should see logs similar to:
9999

@@ -102,28 +102,28 @@ You should see logs similar to:
102102
103103
2024-10-21 03:58:19,579 - INFO: Starting Restore container
104104
2024-10-21 03:58:19,864 - INFO: Restore process container: charming_cohen
105-
2024-10-21 03:58:20,074 - INFO: 2024-10-21 03:58:20,073 - INFO: Status for compose project 'restore-test'
105+
2024-10-21 03:58:20,074 - INFO: 2024-10-21 03:58:20,073 - INFO: Status for compose project 'live-two'
106106
2024-10-21 03:58:20,074 - INFO: 2024-10-21 03:58:20,073 - INFO: Repository: '/restic_data'
107107
2024-10-21 03:58:20,074 - INFO: 2024-10-21 03:58:20,073 - INFO: Restore currently running?: True
108108
2024-10-21 03:58:20,074 - INFO: 2024-10-21 03:58:20,074 - INFO: Include project name in restore path?: False
109109
2024-10-21 03:58:20,075 - INFO: 2024-10-21 03:58:20,074 - INFO: Checking docker availability
110110
2024-10-21 03:58:20,698 - INFO: 2024-10-21 03:58:20,697 - INFO: ------------------------- Detected Config -------------------------
111111
2024-10-21 03:58:20,698 - INFO: 2024-10-21 03:58:20,697 - INFO: service: web
112112
2024-10-21 03:58:20,698 - INFO: 2024-10-21 03:58:20,697 - INFO: service: postgres
113-
2024-10-21 03:58:20,698 - INFO: 2024-10-21 03:58:20,698 - INFO: Container NAME restore-test-postgres-1
114-
2024-10-21 03:58:20,704 - INFO: 2024-10-21 03:58:20,703 - INFO: Container NAME restore-test-postgres-1
113+
2024-10-21 03:58:20,698 - INFO: 2024-10-21 03:58:20,698 - INFO: Container NAME live-two-postgres-1
114+
2024-10-21 03:58:20,704 - INFO: 2024-10-21 03:58:20,703 - INFO: Container NAME live-two-postgres-1
115115
2024-10-21 03:58:20,704 - INFO: 2024-10-21 03:58:20,703 - INFO: - postgres (is_ready=True) -> /backup/databases/postgres/test-postgres.sql
116116
2024-10-21 03:58:20,704 - INFO: 2024-10-21 03:58:20,703 - INFO: service: mariadb
117-
2024-10-21 03:58:20,704 - INFO: 2024-10-21 03:58:20,703 - INFO: Container NAME restore-test-mariadb-1
117+
2024-10-21 03:58:20,704 - INFO: 2024-10-21 03:58:20,703 - INFO: Container NAME live-two-mariadb-1
118118
2024-10-21 03:58:20,709 - INFO: 2024-10-21 03:58:20,708 - INFO: - mariadb (is_ready=True) -> /backup/databases/mariadb/all_databases.sql
119119
2024-10-21 03:58:20,709 - INFO: 2024-10-21 03:58:20,708 - INFO: -------------------------------------------------------------------
120120
2024-10-21 03:58:20,709 - INFO: 2024-10-21 03:58:20,708 - INFO: Restoring databases
121121
2024-10-21 03:58:20,710 - INFO: 2024-10-21 03:58:20,709 - INFO: Restoring up postgres in service postgres
122-
2024-10-21 03:58:20,710 - INFO: 2024-10-21 03:58:20,709 - INFO: Container NAME restore-test-postgres-1
123-
2024-10-21 03:58:20,710 - INFO: 2024-10-21 03:58:20,709 - INFO: Container NAME restore-test-postgres-1
124-
2024-10-21 03:58:20,710 - INFO: 2024-10-21 03:58:20,709 - INFO: Container NAME restore-test-postgres-1
125-
2024-10-21 03:58:20,710 - INFO: 2024-10-21 03:58:20,709 - INFO: Container NAME restore-test-postgres-1
126-
2024-10-21 03:58:20,711 - INFO: 2024-10-21 03:58:20,709 - INFO: Database restore_command: =======> restic -r /restic_data dump latest /backup/databases/postgres/test-postgres.sql | psql --host=restore-test-postgres-1 --port=5432 --username=pguser --dbname=postgres
122+
2024-10-21 03:58:20,710 - INFO: 2024-10-21 03:58:20,709 - INFO: Container NAME live-two-postgres-1
123+
2024-10-21 03:58:20,710 - INFO: 2024-10-21 03:58:20,709 - INFO: Container NAME live-two-postgres-1
124+
2024-10-21 03:58:20,710 - INFO: 2024-10-21 03:58:20,709 - INFO: Container NAME live-two-postgres-1
125+
2024-10-21 03:58:20,710 - INFO: 2024-10-21 03:58:20,709 - INFO: Container NAME live-two-postgres-1
126+
2024-10-21 03:58:20,711 - INFO: 2024-10-21 03:58:20,709 - INFO: Database restore_command: =======> restic -r /restic_data dump latest /backup/databases/postgres/test-postgres.sql | psql --host=live-two-postgres-1 --port=5432 --username=pguser --dbname=postgres
127127
2024-10-21 03:58:21,321 - INFO: SET
128128
2024-10-21 03:58:21,321 - INFO: SET
129129
2024-10-21 03:58:21,321 - INFO: SET
@@ -252,9 +252,9 @@ You should see logs similar to:
252252
2024-10-21 03:58:21,388 - INFO: 2024-10-21 03:58:21,387 - INFO: Database Restore exit code: =======> 0
253253
2024-10-21 03:58:21,388 - INFO: 2024-10-21 03:58:21,387 - INFO: Exit code: 0
254254
2024-10-21 03:58:21,388 - INFO: 2024-10-21 03:58:21,388 - INFO: Restoring up mariadb in service mariadb
255-
2024-10-21 03:58:21,388 - INFO: 2024-10-21 03:58:21,388 - INFO: Container NAME restore-test-mariadb-1
256-
2024-10-21 03:58:21,389 - INFO: 2024-10-21 03:58:21,388 - INFO: Container NAME restore-test-mariadb-1
257-
2024-10-21 03:58:21,389 - INFO: 2024-10-21 03:58:21,388 - INFO: Database restore_command: =======> restic -r /restic_data dump latest /backup/databases/mariadb/all_databases.sql | mysql --host=restore-test-mariadb-1 --port=3306 --user=root
255+
2024-10-21 03:58:21,388 - INFO: 2024-10-21 03:58:21,388 - INFO: Container NAME live-two-mariadb-1
256+
2024-10-21 03:58:21,389 - INFO: 2024-10-21 03:58:21,388 - INFO: Container NAME live-two-mariadb-1
257+
2024-10-21 03:58:21,389 - INFO: 2024-10-21 03:58:21,388 - INFO: Database restore_command: =======> restic -r /restic_data dump latest /backup/databases/mariadb/all_databases.sql | mysql --host=live-two-mariadb-1 --port=3306 --user=root
258258
2024-10-21 03:58:22,817 - INFO: 2024-10-21 03:58:22,816 - INFO: Database Restore exit code: =======> 0
259259
2024-10-21 03:58:22,817 - INFO: 2024-10-21 03:58:22,817 - INFO: Exit code: 0
260260
2024-10-21 03:58:22,817 - INFO: 2024-10-21 03:58:22,817 - INFO: Restoring files to ===> /restored_data/
@@ -263,7 +263,7 @@ You should see logs similar to:
263263
```
264264
You can now check the databases for restored data, for example
265265

266-
```docker exec -it restore-test-mariadb-1 bash``
266+
```docker exec -it live-two-mariadb-1 bash``
267267

268268
```mysql -uroot -pmy-secret-pw```
269269

@@ -285,9 +285,9 @@ You should see logs similar to:
285285
```
286286
### Validate volume restore
287287

288-
We have two services to validate that volumes are restored correctly. see `web` and `web-2` in the `docker-compose-restore-test.yaml` file. When the `docker-compose-restore-service.yaml` file is used the `web` and `web-2` will wait for restore to complete before starting. The service then run ls on the restored mounts to confirm that the restore happened run;
288+
We have two services to validate that volumes are restored correctly. see `web` and `web-2` in the `docker-compose-live-two.yaml` file. When the `docker-compose-restore-service.yaml` file is used the `web` and `web-2` will wait for restore to complete before starting. The service then run ls on the restored mounts to confirm that the restore happened run;
289289

290-
`docker logs -f restore-test-web-1`
290+
`docker logs -f live-two-web-1`
291291

292292
You will see the result
293293

@@ -301,13 +301,13 @@ tests.py
301301

302302
Stop backup system
303303

304-
```./stop-backup-system.sh```
304+
```./stop-live-two.sh```
305305

306306
## Clean up
307307

308308
Remove Restored data:
309309

310-
```sudo rm -rf ./restored_data```
310+
```sudo rm -rf ./live_two_filestores```
311311

312312
Remove Backup Repo
313313

start-live-system.sh start-live.sh

File renamed without changes.

stop-live-two.sh

+1-1
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
docker compose -p restore-test -f docker-compose-live-two.yaml -f docker-compose-restore-service.yaml down
1+
docker compose -p live-two -f docker-compose-live-two.yaml -f docker-compose-restore-service.yaml down

stop-live.sh

+1-1
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
docker compose -p live-system -f docker-compose-backup-service.yaml -f docker-compose-live.yaml down
1+
docker compose -p live -f docker-compose-backup-service.yaml -f docker-compose-live.yaml down

0 commit comments

Comments
 (0)