You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Nextest supports running _scripts_ when certain events occur during a test run. Scripts can be scoped to particular tests via [filtersets](../filtersets/index.md).
10
+
11
+
Nextest currently recognizes two types of scripts:
12
+
13
+
*[_Setup scripts_](setup.md), which execute at the start of a test run.
14
+
*[_Pre-timeout scripts_](pre-timeout.md), which execute before nextest terminates a test that has exceeded its timeout.
15
+
16
+
Scripts are configured in two parts: _defining scripts_, and _specifying rules_ for when they should be executed.
17
+
18
+
## Defining scripts
19
+
20
+
Scripts are defined using the top-level `script.<type>` configuration.
21
+
22
+
For example, to define a setup script named "my-script", which runs `my-script.sh`:
23
+
24
+
```toml title="Setup script definition in <code>.config/nextest.toml</code>"
25
+
[script.setup.my-script]
26
+
command = 'my-script.sh'
27
+
# Additional options...
28
+
```
29
+
30
+
See [_Defining setup scripts_](setup.md#defining-setup-scripts) for the additional options available for configuring setup scripts.
31
+
32
+
To instead define a pre-timeout script named "my-script", which runs `my-script.sh`:
33
+
34
+
```toml title="Pre-timeout script definition in <code>.config/nextest.toml</code>"
35
+
[script.pre-timeout.my-script]
36
+
command = 'my-script.sh'
37
+
# Additional options...
38
+
```
39
+
40
+
See [_Defining pre-timeout scripts_](pre-timeout.md#defining-pre-timeout-scripts) for the additional options available for configuring pre-timeout scripts.
41
+
42
+
### Command specification
43
+
44
+
All script types support the `command` option, which specifies how to invoke the script. Commands can either be specified using Unix shell rules, or as a list of arguments. In the following example, `script1` and `script2` are equivalent.
45
+
46
+
```toml
47
+
[script.<type>.script1]
48
+
command = 'script.sh -c "Hello, world!"'
49
+
50
+
[script.<type>.script2]
51
+
command = ['script.sh', '-c', 'Hello, world!']
52
+
```
53
+
54
+
### Namespacing
55
+
56
+
Script names must be unique across all script types.
57
+
58
+
This means that you cannot use the same name for a setup script and a pre-timeout script:
59
+
60
+
```toml title="Pre-timeout script definition in <code>.config/nextest.toml</code>"
61
+
[script.setup.my-script]
62
+
command = 'setup.sh'
63
+
64
+
# Reusing the `my-script` name for a pre-timeout script is NOT permitted.
65
+
[script.pre-timeout.my-script]
66
+
command = 'pre-timeout.sh'
67
+
```
68
+
69
+
## Specifying rules
70
+
71
+
In configuration, you can create rules for when to use scripts on a per-profile basis. This is done via the `profile.<profile-name>.scripts` array. For example, you can configure a setup script that generates a database if tests from the `db-tests` package, or any packages that depend on it, are run.
72
+
73
+
```toml title="Basic rules"
74
+
[[profile.default.scripts]]
75
+
filter = 'rdeps(db-tests)'
76
+
setup = 'db-generate'
77
+
```
78
+
79
+
(This example uses the `rdeps`[filterset](../filtersets/index.md) predicate.)
80
+
81
+
Scripts can also filter based on platform, using the rules listed in [_Specifying platforms_](../configuration/specifying-platforms.md):
82
+
83
+
```toml title="Platform-specific rules"
84
+
[[profile.default.scripts]]
85
+
platform = { host = "cfg(unix)" }
86
+
setup = 'script1'
87
+
```
88
+
89
+
A set of scripts can also be specified. All scripts in the set will be executed.
90
+
91
+
```toml title="Multiple setup scripts"
92
+
[[profile.default.scripts]]
93
+
filter = 'test(/^script_tests::/)'
94
+
setup = ['script1', 'script2']
95
+
```
96
+
97
+
Executing pre-timeout scripts follows the same pattern. For example, you can configure a pre-timeout script for every test that contains `slow` in its name.
98
+
99
+
```toml title="Basic pre-timeout rules"
100
+
[[profile.default.scripts]]
101
+
filter = 'test(slow)'
102
+
pre-timeout = 'capture-backtrace'
103
+
```
104
+
105
+
A single rule can specify any number of setup scripts and any number of pre-timeout scripts.
Nextest runs *pre-timeout scripts* before terminating a test that has exceeded
17
+
its timeout.
18
+
19
+
## Defining pre-timeout scripts
20
+
21
+
Pre-timeout scripts are defined using the top-level `script.pre-timeout` configuration. For example, to define a script named "my-script", which runs `my-script.sh`:
22
+
23
+
```toml title="Script definition in <code>.config/nextest.toml</code>"
24
+
[script.pre-timeout.my-script]
25
+
command = 'my-script.sh'
26
+
```
27
+
28
+
Pre-timeout scripts can have the following configuration options attached to them:
See [_Specifying rules_](index.md#specifying-rules).
43
+
44
+
## Pre-timeout script execution
45
+
46
+
A given pre-timeout script _S_ is executed when the current profile has at least one rule where the `platform` predicates match the current execution environment, the script _S_ is listed in `pre-timeout`, and a test matching the `filter` has reached its configured timeout.
47
+
48
+
Pre-timeout scripts are executed serially, in the order they are defined (_not_ the order they're specified in the rules). If any pre-timeout script exits with a non-zero exit code, an error is logged but the test run continues.
49
+
50
+
Nextest sets the following environment variables when executing a pre-timeout script:
51
+
52
+
***`NEXTEST_PRE_TIMEOUT_TEST_PID`**: the ID of the process running the test.
53
+
***`NEXTEST_PRE_TIMEOUT_TEST_NAME`**: the name of the running test.
54
+
***`NEXTEST_PRE_TIMEOUT_TEST_PACKAGE_NAME`**: the name of the package in which the test is located.
55
+
***`NEXTEST_PRE_TIMEOUT_TEST_BINARY_NAME`**: the name of the test binary, if known.
56
+
***`NEXTEST_PRE_TIMEOUT_TEST_BINARY_KIND`**: the kind of the test binary, if known.
Nextest supports running _scripts_ when certain events occur during a test run. Scripts can be scoped to particular tests via [filtersets](../filtersets/index.md).
21
-
22
-
Nextest currently recognizes two types of scripts:
23
-
24
-
*_Setup scripts_, which are executed at the start of a test run.
25
-
*_Pre-timeout scripts_, which are executed before nextest begins terminating a test that has exceeded its timeout.
26
-
27
-
Scripts are configured in two parts: _defining scripts_, and _setting up rules_ for when they should be executed.
17
+
## Defining setup scripts
28
18
29
-
## Defining scripts
30
-
31
-
Scripts are defined using the top-level `script` configuration. For example, to define a script named "my-script", which runs `my-script.sh`:
19
+
Setup scripts are defined using the top-level `script.setup` configuration. For example, to define a script named "my-script", which runs `my-script.sh`:
32
20
33
21
```toml title="Script definition in <code>.config/nextest.toml</code>"
34
-
[script.my-script]
22
+
[script.setup.my-script]
35
23
command = 'my-script.sh'
36
24
```
37
25
38
26
Commands can either be specified using Unix shell rules, or as a list of arguments. In the following example, `script1` and `script2` are equivalent.
39
27
40
28
```toml
41
-
[script.script1]
29
+
[script.setup.script1]
42
30
command = 'script.sh -c "Hello, world!"'
43
31
44
-
[script.script2]
32
+
[script.setup.script2]
45
33
command = ['script.sh', '-c', 'Hello, world!']
46
34
```
47
35
48
-
Scripts can have the following configuration options attached to them:
36
+
Setup scripts can have the following configuration options attached to them:
49
37
50
-
-**`slow-timeout`**: Mark a script [as slow](../features/slow-tests.md) or [terminate it](../features/slow-tests.md#terminating-tests-after-a-timeout), using the same configuration as for tests. By default, scripts are not marked as slow or terminated (this is different from the slow timeout for tests).
51
-
-**`leak-timeout`**: Mark scripts [leaky](../features/leaky-tests.md) after a timeout, using the same configuration as for tests. By default, the leak timeout is 100ms.
38
+
-**`slow-timeout`**: Mark a setup script [as slow](../features/slow-tests.md) or [terminate it](../features/slow-tests.md#terminating-tests-after-a-timeout), using the same configuration as for tests. By default, scripts are not marked as slow or terminated (this is different from the slow timeout for tests).
39
+
-**`leak-timeout`**: Mark setup scripts [leaky](../features/leaky-tests.md) after a timeout, using the same configuration as for tests. By default, the leak timeout is 100ms.
52
40
-**`capture-stdout`**: `true` if the script's standard output should be captured, `false` if not. By default, this is `false`.
53
41
-**`capture-stderr`**: `true` if the script's standard error should be captured, `false` if not. By default, this is `false`.
54
42
55
43
### Example
56
44
57
-
```toml title="Advanced script definition"
58
-
[script.db-generate]
45
+
```toml title="Advanced setup script definition"
46
+
[script.setup.db-generate]
59
47
command = 'cargo run -p db-generate'
60
48
slow-timeout = { period = "60s", terminate-after = 2 }
61
49
leak-timeout = "1s"
62
50
capture-stdout = true
63
51
capture-stderr = false
64
52
```
65
53
66
-
## Setting up rules
54
+
## Specifying setup script rules
67
55
68
-
In configuration, you can create rules for when to use scripts on a per-profile basis. This is done via the `profile.<profile-name>.scripts` array. For example, you can configure a setup script that generates a database if tests from the `db-tests` package, or any packages that depend on it, are run.
56
+
See [_Specifying rules_](index.md#specifying-rules).
69
57
70
-
```toml title="Basic rules"
71
-
[[profile.default.scripts]]
72
-
filter = 'rdeps(db-tests)'
73
-
setup = 'db-generate'
74
-
```
75
-
76
-
(This example uses the `rdeps`[filterset](../filtersets/index.md) predicate.)
77
-
78
-
Scripts can also filter based on platform, using the rules listed in [_Specifying platforms_](../configuration/specifying-platforms.md):
79
-
80
-
```toml title="Platform-specific rules"
81
-
[[profile.default.scripts]]
82
-
platform = { host = "cfg(unix)" }
83
-
setup = 'script1'
84
-
```
85
-
86
-
A set of scripts can also be specified. All scripts in the set will be executed.
87
-
88
-
```toml title="Multiple setup scripts"
89
-
[[profile.default.scripts]]
90
-
filter = 'test(/^script_tests::/)'
91
-
setup = ['script1', 'script2']
92
-
```
93
-
94
-
Executing pre-timeout scripts follows the same pattern. For example, you can configure a pre-timeout script for every test that contains `slow` in its name.
95
-
96
-
```toml title="Basic pre-timeout rules"
97
-
[[profile.default.scripts]]
98
-
filter = 'test(slow)'
99
-
pre-timeout = 'capture-backtrace'
100
-
```
101
-
102
-
A single rule can specify any number of setup scripts and any number of pre-timeout scripts.
103
-
104
-
```toml title="Combination rules"
105
-
[[profile.default.scripts]]
106
-
filter = 'test(slow)'
107
-
setup = ['setup-1', 'setup-2']
108
-
pre-timeout = ['pre-timeout-1', 'pre-timeout-2']
109
-
```
110
-
111
-
## Script execution
112
-
113
-
### Setup scripts
58
+
## Setup script execution
114
59
115
60
A given setup script _S_ is only executed if the current profile has at least one rule where the `filter` and `platform` predicates match the current execution environment, and the setup script _S_ is listed in `setup`.
116
61
117
62
Setup scripts are executed serially, in the order they are defined (_not_ the order they're specified in the rules). If any setup script exits with a non-zero exit code, the entire test run is terminated.
118
63
119
-
####Environment variables
64
+
### Environment variables
120
65
121
66
Setup scripts can define environment variables that will be exposed to tests that match the script. This is done by writing to the `$NEXTEST_ENV` environment variable from within the script.
122
67
@@ -154,18 +99,6 @@ fn my_env_test() {
154
99
}
155
100
```
156
101
157
-
### Pre-timeout scripts
158
-
159
-
A given pre-timeout script _S_ is executed when the current profile has at least one rule where the `platform` predicates match the current execution environment, the script _S_ is listed in `pre-timeout`, and a test matching the `filter` has reached its configured timeout.
160
-
161
-
Pre-timeout scripts are executed serially, in the order they are defined (_not_ the order they're specified in the rules). If any pre-timeout script exits with a non-zero exit code, an error is logged but the test run continues.
162
-
163
-
Nextest sets the following environment variables when executing a pre-timeout script:
164
-
165
-
***`NEXTEST_PRE_TIMEOUT_TEST_PID`**: the ID of the process running the test.
166
-
***`NEXTEST_PRE_TIMEOUT_TEST_NAME`**: the name of the running test.
167
-
***`NEXTEST_PRE_TIMEOUT_TEST_BINARY`**: the name of the binary running the test.
0 commit comments