You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: tests/integration/README.md
+11-11
Original file line number
Diff line number
Diff line change
@@ -6,7 +6,7 @@
6
6
We have an integration test utility script, [`./tests/integration/run.py`](./run.py), for running integration tests. The script should be called within the dev virtual environment and from the root directory of the repository.
At this point, some expected result files do not exist yet, since you normally want to run `macaron` once, inspect the result files, then turn them into expected result files if they look good enough. To do this, you can run in **interactive** mode. In this mode, the utility stops at each step and asks if you want to run or skip a step. For `compare` steps, the utility also asks if you want to "update" the expected result file instead of compare.
88
88
89
89
```bash
90
90
# Run a test case in interactive mode.
91
-
$ python3 ./tests/integration/run.py run -i ./test/case/directory
91
+
$ python ./tests/integration/run.py run -i ./test/case/directory
92
92
```
93
93
94
94
After you have finished running the test case, you can rerun the test case to make sure everything works as expected.
95
95
96
96
```bash
97
97
# Run a test case end-to-end.
98
-
$ python3 ./tests/integration/run.py run ./test/case/directory
98
+
$ python ./tests/integration/run.py run ./test/case/directory
99
99
```
100
100
101
101
### Inspect test cases
@@ -104,7 +104,7 @@ Besides the interactive mode, the `run` command also has another special mode ca
104
104
105
105
```bash
106
106
# Run a test case in dry-run mode.
107
-
$ python3 ./tests/integration/run.py run -d ./test/case/directory
107
+
$ python ./tests/integration/run.py run -d ./test/case/directory
108
108
```
109
109
110
110
### Validate test cases before pushing commits to remote or running in CI
@@ -119,14 +119,14 @@ All commands (`check`, `vet`, `run`, and `update`) can process multiple test cas
119
119
120
120
```bash
121
121
# Run two test cases one after another.
122
-
$ python3 ./tests/integration/run.py run ./test_case_a/directory ./test_case_b/directory
122
+
$ python ./tests/integration/run.py run ./test_case_a/directory ./test_case_b/directory
123
123
```
124
124
125
125
You can also use the `...` path wildcard to allow for discovering testcase directories recursively under a root directory.
126
126
127
127
```bash
128
128
# Run all test cases discovered recursively under a directory.
129
-
$ python3 ./tests/integration/run.py run ./all/cases/...
129
+
$ python ./tests/integration/run.py run ./all/cases/...
130
130
```
131
131
132
132
### Select a subset of test cases to run
@@ -147,21 +147,21 @@ We typically have the test cases for the container image being a subset of the t
147
147
148
148
```bash
149
149
# Test the container image with test cases having the `docker` tag.
150
-
$ python3 ./tests/integration/run.py run --include-tag docker ./all/cases/...
150
+
$ python ./tests/integration/run.py run --include-tag docker ./all/cases/...
151
151
```
152
152
153
153
The `--include-tag` flag can be specified multiple times. A selected test case must contain all tags specified with the `--include-tag` flag.
154
154
155
155
```bash
156
156
# Test the container image with test cases having the `docker` tag.
157
-
$ python3 ./tests/integration/run.py run --include-tag tag-a --include-tag tag-b ./all/cases/...
157
+
$ python ./tests/integration/run.py run --include-tag tag-a --include-tag tag-b ./all/cases/...
158
158
```
159
159
160
160
There is also the `--exclude-tag` flag. A selected test case must also not contain any tag specified with the `--exclude-tag` flag.
161
161
162
162
```bash
163
163
# Only run test cases not tagged with `npm`.
164
-
$ python3 ./tests/integration/run.py run --exclude-tag npm ./all/cases/...
164
+
$ python ./tests/integration/run.py run --exclude-tag npm ./all/cases/...
165
165
```
166
166
167
167
You can simply think of each `--include-tag`/`--exclude-tag` argument as adding an additional constraint that a selected test case must satisfy".
0 commit comments