Skip to content
This repository was archived by the owner on Feb 8, 2024. It is now read-only.

Commit 9a22da7

Browse files
committed
Add preview docs for setup workflows
1 parent 49d7ca6 commit 9a22da7

File tree

2 files changed

+137
-0
lines changed

2 files changed

+137
-0
lines changed

docs/path-filtering.md

+1
Original file line numberDiff line numberDiff line change
@@ -91,6 +91,7 @@ The regular expressions support the full [Python re syntax](https://docs.python.
9191

9292
You can find more information about the elements used here:
9393

94+
- [Setup Workflows](./setup-workflows.md)
9495
- [Pipeline Variables](https://circleci.com/docs/2.0/pipeline-variables/)
9596
- [Logic Statements](https://circleci.com/docs/2.0/configuration-reference/#logic-statements)
9697
- [Conditional Steps](https://circleci.com/docs/2.0/reusing-config/#defining-conditional-steps)

docs/setup-workflows.md

+136
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,136 @@
1+
# Setup Workflows
2+
3+
Setup workflows are a new CircleCI feature which allow users to
4+
dynamically generate configuration within a job, and by extension
5+
dynamically decide which work to execute. The resulting work will be
6+
executed within the same pipeline.
7+
8+
Prior to this feature, configuration of a pipeline was relatively
9+
static: while we had [conditional steps](https://circleci.com/docs/2.0/reusing-config/#defining-conditional-steps) and [pipeline parameters](https://circleci.com/docs/2.0/pipeline-variables/#pipeline-parameters-in-configuration), it was
10+
not possible to manipulate a pipeline's configuration in an arbitrary
11+
way.
12+
13+
Setup workflows allow us to dynamically generate or manipulate our
14+
configuration. This is a powerful means of extension and abstraction,
15+
and enables features that were difficult to accomplish with static
16+
configuration - for example [path filtering](./path-filtering.md) for monorepo support, or
17+
[merging several configuration files](https://github.com/circle-makotom/circle-advanced-setup-workflow).
18+
19+
## Concepts
20+
21+
Setup workflows introduce new, optional pipeline lifecycle phases,
22+
which allow work to be executed. Where previously a pipeline has
23+
consisted of two phases:
24+
25+
1. Configuration Processing
26+
1. Work Execution
27+
28+
it can now have two additional phases:
29+
30+
1. *Setup* Configuration Processing
31+
1. *Setup* Work Execution
32+
1. Configuration Processing
33+
1. Work Execution
34+
35+
![Schematic](./setup-workflows.png)
36+
37+
During the first phase, the _setup configuration_ is processed, which
38+
describes the work to be executed during the _setup work execution_
39+
phase. The workflow executed as part of the _setup work execution_ phase
40+
is called a _setup workflow_. _Setup workflows_ are tagged as such in the
41+
UI and API. The _setup workflow_ then continues the pipeline to enter
42+
the next phase.
43+
44+
Behind the scenes this continuation is implemented as a call to a
45+
[public pipeline continuation API](https://circleci.com/docs/api/v2/#operation/continuePipeline). This API accepts a _continuation key_,
46+
which is a secret, unique-per-pipeline key, that is automatically
47+
injected into the environment of jobs executed as part of a setup
48+
workflow. We advise against extracting this key. It also accepts a
49+
configuration string, as well as a set of pipeline parameters.
50+
51+
We provide several orbs to cover common use cases for setup workflows,
52+
such as [path filtering](https://circleci.com/developer/orbs/orb/circleci/path-filtering).
53+
54+
This means each pipeline that has entered the setup phase has two
55+
configurations, the _setup configuration_, and the regular
56+
configuration.
57+
58+
## Limitations
59+
60+
Some limitations apply to setup workflows:
61+
62+
- the setup phase requires configuration version 2.1 or higher
63+
- a pipeline can only be continued once
64+
- a pipeline can only be continued within six hours of its creation
65+
- a pipeline cannot be continued with another setup configuration
66+
- there can only be one workflow in the setup configuration
67+
- pipeline parameters submitted at continuation time cannot overlap
68+
with pipeline parameters submitted at trigger time
69+
70+
## Enabling Setup Workflows
71+
72+
To enable setup workflows, simply enable "Setup Workflows" in the
73+
project settings (under "Advanced"). Now you can push a setup
74+
configuration to a branch. To designate a configuration as a setup
75+
configuration, and thus trigger the setup phase, use the top-level
76+
`setup: true` stanza (see below for a full example). Regardless of the
77+
project setting, only setup configurations will trigger the setup
78+
phase.
79+
80+
## Full Example
81+
82+
In this example we presume that a `generate-config` script or executable
83+
exists, which outputs a YAML string, based on some work it performs.
84+
It could potentially inspect git history, or pipeline values that get
85+
passed to it, or do anything else you can do inside a job.
86+
87+
```yaml
88+
version: 2.1
89+
90+
setup: true
91+
92+
orbs:
93+
continuation: circleci/continuation:0.1.2
94+
95+
jobs:
96+
setup:
97+
executor: continuation/default
98+
steps:
99+
- checkout
100+
- run:
101+
name: Generate config
102+
command: |
103+
./generate-config > generated_config.yml
104+
- continuation/continue:
105+
configuration_path: generated_config.yml
106+
107+
workflows:
108+
setup:
109+
jobs:
110+
- setup
111+
```
112+
113+
## Advanced Topics
114+
115+
### Using Custom Executors
116+
117+
Alternative executors can be used, but require certain dependencies to
118+
be installed for the continuation step to work (currently: `curl`,
119+
`jq`).
120+
121+
### Choosing not to Continue a Pipeline
122+
123+
If the setup workflow decides that no further work shall be executed,
124+
it is good practice to finish the pipeline, avoiding accidental
125+
continuation. The continuation orb has a command for this:
126+
127+
```yaml
128+
steps:
129+
- continuation/finish
130+
```
131+
132+
### Not Using the Continuation Orb
133+
134+
If you have special requirements not covered by the continuation orb,
135+
you can implement the same functionality in different ways. Refer to
136+
the [orb source code](https://app.circleci.com/pipelines/github/CircleCI-Public/continuation-orb) for reference.

0 commit comments

Comments
 (0)