You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Nextest supports running _scripts_ when certain events occur during a test run. Scripts can be scoped to particular tests via [filtersets](../filtersets/index.md).
10
+
11
+
Nextest currently recognizes two types of scripts:
12
+
13
+
*[_Setup scripts_](setup.md), which execute at the start of a test run.
14
+
*[_Pre-timeout scripts_](pre-timeout.md), which execute before nextest terminates a test that has exceeded its timeout.
15
+
16
+
Scripts are configured in two parts: _defining scripts_, and _setting up rules_ for when they should be executed.
17
+
18
+
## Defining scripts
19
+
20
+
Scripts are defined using the top-level `script.<type>` configuration.
21
+
22
+
For example, to define a setup script named "my-script", which runs `my-script.sh`:
23
+
24
+
```toml title="Setup script definition in <code>.config/nextest.toml</code>"
25
+
[script.setup.my-script]
26
+
command = 'my-script.sh'
27
+
# Additional options...
28
+
```
29
+
30
+
See [_Defining setup scripts_](setup.md#defining-setup-scripts) for the additional options available for configuring setup scripts.
31
+
32
+
To instead define a pre-timeout script named "my-script", which runs `my-script.sh`:
33
+
34
+
```toml title="Pre-timeout script definition in <code>.config/nextest.toml</code>"
35
+
[script.pre-timeout.my-script]
36
+
command = 'my-script.sh'
37
+
# Additional options...
38
+
```
39
+
40
+
See [_Defining pre-timeout scripts_](pre-timeout.md#defining-pre-timeout-scripts) for the additional options available for configuring pre-timeout scripts.
41
+
42
+
### Command specification
43
+
44
+
All script types support the `command` option, which specifies how to invoke the script. Commands can either be specified using Unix shell rules, or as a list of arguments. In the following example, `script1` and `script2` are equivalent.
45
+
46
+
```toml
47
+
[script.<type>.script1]
48
+
command = 'script.sh -c "Hello, world!"'
49
+
50
+
[script.<type>.script2]
51
+
command = ['script.sh', '-c', 'Hello, world!']
52
+
```
53
+
54
+
### Namespacing
55
+
56
+
Script names must be unique across all script types.
57
+
58
+
This means that you cannot use the same name for a setup script and a pre-timeout script:
59
+
60
+
```toml title="Pre-timeout script definition in <code>.config/nextest.toml</code>"
61
+
[script.setup.my-script]
62
+
command = 'setup.sh'
63
+
64
+
# Reusing the `my-script` name for a pre-timeout script is NOT permitted.
65
+
[script.pre-timeout.my-script]
66
+
command = 'pre-timeout.sh'
67
+
```
68
+
69
+
## Setting up rules
70
+
71
+
In configuration, you can create rules for when to use scripts on a per-profile basis. This is done via the `profile.<profile-name>.scripts` array. For example, you can configure a setup script that generates a database if tests from the `db-tests` package, or any packages that depend on it, are run.
72
+
73
+
```toml title="Basic rules"
74
+
[[profile.default.scripts]]
75
+
filter = 'rdeps(db-tests)'
76
+
setup = 'db-generate'
77
+
```
78
+
79
+
(This example uses the `rdeps`[filterset](../filtersets/index.md) predicate.)
80
+
81
+
Scripts can also filter based on platform, using the rules listed in [_Specifying platforms_](../configuration/specifying-platforms.md):
82
+
83
+
```toml title="Platform-specific rules"
84
+
[[profile.default.scripts]]
85
+
platform = { host = "cfg(unix)" }
86
+
setup = 'script1'
87
+
```
88
+
89
+
A set of scripts can also be specified. All scripts in the set will be executed.
90
+
91
+
```toml title="Multiple setup scripts"
92
+
[[profile.default.scripts]]
93
+
filter = 'test(/^script_tests::/)'
94
+
setup = ['script1', 'script2']
95
+
```
96
+
97
+
Executing pre-timeout scripts follows the same pattern. For example, you can configure a pre-timeout script for every test that contains `slow` in its name.
98
+
99
+
```toml title="Basic pre-timeout rules"
100
+
[[profile.default.scripts]]
101
+
filter = 'test(slow)'
102
+
pre-timeout = 'capture-backtrace'
103
+
```
104
+
105
+
A single rule can specify any number of setup scripts and any number of pre-timeout scripts.
Nextest runs *pre-timeout scripts* before terminating a test that has exceeded
17
+
its timeout.
18
+
19
+
## Defining pre-timeout scripts
20
+
21
+
Pre-timeout scripts are defined using the top-level `script.pre-timeout` configuration. For example, to define a script named "my-script", which runs `my-script.sh`:
22
+
23
+
```toml title="Script definition in <code>.config/nextest.toml</code>"
24
+
[script.pre-timeout.my-script]
25
+
command = 'my-script.sh'
26
+
```
27
+
28
+
Pre-timeout scripts can have the following configuration options attached to them:
29
+
30
+
- TODO
31
+
32
+
### Example
33
+
34
+
```toml title="Advanced script definition"
35
+
[script.pre-timeout.db-generate]
36
+
command = 'cargo run -p db-generate'
37
+
# TODO options
38
+
```
39
+
40
+
## Script execution
41
+
42
+
A given pre-timeout script _S_ is executed when the current profile has at least one rule where the `platform` predicates match the current execution environment, the script _S_ is listed in `pre-timeout`, and a test matching the `filter` has reached its configured timeout.
43
+
44
+
Pre-timeout scripts are executed serially, in the order they are defined (_not_ the order they're specified in the rules). If any pre-timeout script exits with a non-zero exit code, an error is logged but the test run continues.
45
+
46
+
Nextest sets the following environment variables when executing a pre-timeout script:
47
+
48
+
***`NEXTEST_PRE_TIMEOUT_TEST_PID`**: the ID of the process running the test.
49
+
***`NEXTEST_PRE_TIMEOUT_TEST_NAME`**: the name of the running test.
50
+
***`NEXTEST_PRE_TIMEOUT_TEST_PACKAGE_NAME`**: the name of the package in which the test is located.
51
+
***`NEXTEST_PRE_TIMEOUT_TEST_BINARY_NAME`**: the name of the test binary, if known.
52
+
***`NEXTEST_PRE_TIMEOUT_TEST_BINARY_KIND`**: the kind of the test binary, if known.
Nextest supports running _scripts_ when certain events occur during a test run. Scripts can be scoped to particular tests via [filtersets](../filtersets/index.md).
15
+
Nextest runs *setup scripts* before tests are run.
21
16
22
-
Nextest currently recognizes two types of scripts:
17
+
## Defining setup scripts
23
18
24
-
*_Setup scripts_, which are executed at the start of a test run.
25
-
*_Pre-timeout scripts_, which are executed before nextest begins terminating a test that has exceeded its timeout.
26
-
27
-
Scripts are configured in two parts: _defining scripts_, and _setting up rules_ for when they should be executed.
28
-
29
-
## Defining scripts
30
-
31
-
Scripts are defined using the top-level `script` configuration. For example, to define a script named "my-script", which runs `my-script.sh`:
19
+
Setup scripts are defined using the top-level `script.setup` configuration. For example, to define a script named "my-script", which runs `my-script.sh`:
32
20
33
21
```toml title="Script definition in <code>.config/nextest.toml</code>"
34
-
[script.my-script]
22
+
[script.setup.my-script]
35
23
command = 'my-script.sh'
36
24
```
37
25
38
26
Commands can either be specified using Unix shell rules, or as a list of arguments. In the following example, `script1` and `script2` are equivalent.
39
27
40
28
```toml
41
-
[script.script1]
29
+
[script.setup.script1]
42
30
command = 'script.sh -c "Hello, world!"'
43
31
44
-
[script.script2]
32
+
[script.setup.script2]
45
33
command = ['script.sh', '-c', 'Hello, world!']
46
34
```
47
35
48
-
Scripts can have the following configuration options attached to them:
36
+
Setup scripts can have the following configuration options attached to them:
49
37
50
38
-**`slow-timeout`**: Mark a script [as slow](../features/slow-tests.md) or [terminate it](../features/slow-tests.md#terminating-tests-after-a-timeout), using the same configuration as for tests. By default, scripts are not marked as slow or terminated (this is different from the slow timeout for tests).
51
39
-**`leak-timeout`**: Mark scripts [leaky](../features/leaky-tests.md) after a timeout, using the same configuration as for tests. By default, the leak timeout is 100ms.
@@ -55,68 +43,21 @@ Scripts can have the following configuration options attached to them:
55
43
### Example
56
44
57
45
```toml title="Advanced script definition"
58
-
[script.db-generate]
46
+
[script.setup.db-generate]
59
47
command = 'cargo run -p db-generate'
60
48
slow-timeout = { period = "60s", terminate-after = 2 }
61
49
leak-timeout = "1s"
62
50
capture-stdout = true
63
51
capture-stderr = false
64
52
```
65
53
66
-
## Setting up rules
67
-
68
-
In configuration, you can create rules for when to use scripts on a per-profile basis. This is done via the `profile.<profile-name>.scripts` array. For example, you can configure a setup script that generates a database if tests from the `db-tests` package, or any packages that depend on it, are run.
69
-
70
-
```toml title="Basic rules"
71
-
[[profile.default.scripts]]
72
-
filter = 'rdeps(db-tests)'
73
-
setup = 'db-generate'
74
-
```
75
-
76
-
(This example uses the `rdeps`[filterset](../filtersets/index.md) predicate.)
77
-
78
-
Scripts can also filter based on platform, using the rules listed in [_Specifying platforms_](../configuration/specifying-platforms.md):
79
-
80
-
```toml title="Platform-specific rules"
81
-
[[profile.default.scripts]]
82
-
platform = { host = "cfg(unix)" }
83
-
setup = 'script1'
84
-
```
85
-
86
-
A set of scripts can also be specified. All scripts in the set will be executed.
87
-
88
-
```toml title="Multiple setup scripts"
89
-
[[profile.default.scripts]]
90
-
filter = 'test(/^script_tests::/)'
91
-
setup = ['script1', 'script2']
92
-
```
93
-
94
-
Executing pre-timeout scripts follows the same pattern. For example, you can configure a pre-timeout script for every test that contains `slow` in its name.
95
-
96
-
```toml title="Basic pre-timeout rules"
97
-
[[profile.default.scripts]]
98
-
filter = 'test(slow)'
99
-
pre-timeout = 'capture-backtrace'
100
-
```
101
-
102
-
A single rule can specify any number of setup scripts and any number of pre-timeout scripts.
103
-
104
-
```toml title="Combination rules"
105
-
[[profile.default.scripts]]
106
-
filter = 'test(slow)'
107
-
setup = ['setup-1', 'setup-2']
108
-
pre-timeout = ['pre-timeout-1', 'pre-timeout-2']
109
-
```
110
-
111
54
## Script execution
112
55
113
-
### Setup scripts
114
-
115
56
A given setup script _S_ is only executed if the current profile has at least one rule where the `filter` and `platform` predicates match the current execution environment, and the setup script _S_ is listed in `setup`.
116
57
117
58
Setup scripts are executed serially, in the order they are defined (_not_ the order they're specified in the rules). If any setup script exits with a non-zero exit code, the entire test run is terminated.
118
59
119
-
####Environment variables
60
+
### Environment variables
120
61
121
62
Setup scripts can define environment variables that will be exposed to tests that match the script. This is done by writing to the `$NEXTEST_ENV` environment variable from within the script.
122
63
@@ -154,18 +95,6 @@ fn my_env_test() {
154
95
}
155
96
```
156
97
157
-
### Pre-timeout scripts
158
-
159
-
A given pre-timeout script _S_ is executed when the current profile has at least one rule where the `platform` predicates match the current execution environment, the script _S_ is listed in `pre-timeout`, and a test matching the `filter` has reached its configured timeout.
160
-
161
-
Pre-timeout scripts are executed serially, in the order they are defined (_not_ the order they're specified in the rules). If any pre-timeout script exits with a non-zero exit code, an error is logged but the test run continues.
162
-
163
-
Nextest sets the following environment variables when executing a pre-timeout script:
164
-
165
-
***`NEXTEST_PRE_TIMEOUT_TEST_PID`**: the ID of the process running the test.
166
-
***`NEXTEST_PRE_TIMEOUT_TEST_NAME`**: the name of the running test.
167
-
***`NEXTEST_PRE_TIMEOUT_TEST_BINARY`**: the name of the binary running the test.
0 commit comments