Skip to content

Commit 2112302

Browse files
MathewBiddleMathewBiddle
authored
big move to workbench (#142)
* big move to workbench * porting README * adding citation removing dup license
1 parent 9d950de commit 2112302

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

62 files changed

+3811
-26
lines changed

‎.github/workflows/README.md

+198
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,198 @@
1+
# Carpentries Workflows
2+
3+
This directory contains workflows to be used for Lessons using the {sandpaper}
4+
lesson infrastructure. Two of these workflows require R (`sandpaper-main.yaml`
5+
and `pr-receive.yaml`) and the rest are bots to handle pull request management.
6+
7+
These workflows will likely change as {sandpaper} evolves, so it is important to
8+
keep them up-to-date. To do this in your lesson you can do the following in your
9+
R console:
10+
11+
```r
12+
# Install/Update sandpaper
13+
options(repos = c(carpentries = "https://carpentries.r-universe.dev/",
14+
CRAN = "https://cloud.r-project.org"))
15+
install.packages("sandpaper")
16+
17+
# update the workflows in your lesson
18+
library("sandpaper")
19+
update_github_workflows()
20+
```
21+
22+
Inside this folder, you will find a file called `sandpaper-version.txt`, which
23+
will contain a version number for sandpaper. This will be used in the future to
24+
alert you if a workflow update is needed.
25+
26+
What follows are the descriptions of the workflow files:
27+
28+
## Deployment
29+
30+
### 01 Build and Deploy (sandpaper-main.yaml)
31+
32+
This is the main driver that will only act on the main branch of the repository.
33+
This workflow does the following:
34+
35+
1. checks out the lesson
36+
2. provisions the following resources
37+
- R
38+
- pandoc
39+
- lesson infrastructure (stored in a cache)
40+
- lesson dependencies if needed (stored in a cache)
41+
3. builds the lesson via `sandpaper:::ci_deploy()`
42+
43+
#### Caching
44+
45+
This workflow has two caches; one cache is for the lesson infrastructure and
46+
the other is for the the lesson dependencies if the lesson contains rendered
47+
content. These caches are invalidated by new versions of the infrastructure and
48+
the `renv.lock` file, respectively. If there is a problem with the cache,
49+
manual invaliation is necessary. You will need maintain access to the repository
50+
and you can either go to the actions tab and [click on the caches button to find
51+
and invalidate the failing cache](https://github.blog/changelog/2022-10-20-manage-caches-in-your-actions-workflows-from-web-interface/)
52+
or by setting the `CACHE_VERSION` secret to the current date (which will
53+
invalidate all of the caches).
54+
55+
## Updates
56+
57+
### Setup Information
58+
59+
These workflows run on a schedule and at the maintainer's request. Because they
60+
create pull requests that update workflows/require the downstream actions to run,
61+
they need a special repository/organization secret token called
62+
`SANDPAPER_WORKFLOW` and it must have the `public_repo` and `workflow` scope.
63+
64+
This can be an individual user token, OR it can be a trusted bot account. If you
65+
have a repository in one of the official Carpentries accounts, then you do not
66+
need to worry about this token being present because the Carpentries Core Team
67+
will take care of supplying this token.
68+
69+
If you want to use your personal account: you can go to
70+
<https://github.com/settings/tokens/new?scopes=public_repo,workflow&description=Sandpaper%20Token>
71+
to create a token. Once you have created your token, you should copy it to your
72+
clipboard and then go to your repository's settings > secrets > actions and
73+
create or edit the `SANDPAPER_WORKFLOW` secret, pasting in the generated token.
74+
75+
If you do not specify your token correctly, the runs will not fail and they will
76+
give you instructions to provide the token for your repository.
77+
78+
### 02 Maintain: Update Workflow Files (update-workflow.yaml)
79+
80+
The {sandpaper} repository was designed to do as much as possible to separate
81+
the tools from the content. For local builds, this is absolutely true, but
82+
there is a minor issue when it comes to workflow files: they must live inside
83+
the repository.
84+
85+
This workflow ensures that the workflow files are up-to-date. The way it work is
86+
to download the update-workflows.sh script from GitHub and run it. The script
87+
will do the following:
88+
89+
1. check the recorded version of sandpaper against the current version on github
90+
2. update the files if there is a difference in versions
91+
92+
After the files are updated, if there are any changes, they are pushed to a
93+
branch called `update/workflows` and a pull request is created. Maintainers are
94+
encouraged to review the changes and accept the pull request if the outputs
95+
are okay.
96+
97+
This update is run weekly or on demand.
98+
99+
### 03 Maintain: Update Package Cache (update-cache.yaml)
100+
101+
For lessons that have generated content, we use {renv} to ensure that the output
102+
is stable. This is controlled by a single lockfile which documents the packages
103+
needed for the lesson and the version numbers. This workflow is skipped in
104+
lessons that do not have generated content.
105+
106+
Because the lessons need to remain current with the package ecosystem, it's a
107+
good idea to make sure these packages can be updated periodically. The
108+
update cache workflow will do this by checking for updates, applying them in a
109+
branch called `updates/packages` and creating a pull request with _only the
110+
lockfile changed_.
111+
112+
From here, the markdown documents will be rebuilt and you can inspect what has
113+
changed based on how the packages have updated.
114+
115+
## Pull Request and Review Management
116+
117+
Because our lessons execute code, pull requests are a secruity risk for any
118+
lesson and thus have security measures associted with them. **Do not merge any
119+
pull requests that do not pass checks and do not have bots commented on them.**
120+
121+
This series of workflows all go together and are described in the following
122+
diagram and the below sections:
123+
124+
![Graph representation of a pull request](https://carpentries.github.io/sandpaper/articles/img/pr-flow.dot.svg)
125+
126+
### Pre Flight Pull Request Validation (pr-preflight.yaml)
127+
128+
This workflow runs every time a pull request is created and its purpose is to
129+
validate that the pull request is okay to run. This means the following things:
130+
131+
1. The pull request does not contain modified workflow files
132+
2. If the pull request contains modified workflow files, it does not contain
133+
modified content files (such as a situation where @carpentries-bot will
134+
make an automated pull request)
135+
3. The pull request does not contain an invalid commit hash (e.g. from a fork
136+
that was made before a lesson was transitioned from styles to use the
137+
workbench).
138+
139+
Once the checks are finished, a comment is issued to the pull request, which
140+
will allow maintainers to determine if it is safe to run the
141+
"Receive Pull Request" workflow from new contributors.
142+
143+
### Receive Pull Request (pr-receive.yaml)
144+
145+
**Note of caution:** This workflow runs arbitrary code by anyone who creates a
146+
pull request. GitHub has safeguarded the token used in this workflow to have no
147+
priviledges in the repository, but we have taken precautions to protect against
148+
spoofing.
149+
150+
This workflow is triggered with every push to a pull request. If this workflow
151+
is already running and a new push is sent to the pull request, the workflow
152+
running from the previous push will be cancelled and a new workflow run will be
153+
started.
154+
155+
The first step of this workflow is to check if it is valid (e.g. that no
156+
workflow files have been modified). If there are workflow files that have been
157+
modified, a comment is made that indicates that the workflow is not run. If
158+
both a workflow file and lesson content is modified, an error will occurr.
159+
160+
The second step (if valid) is to build the generated content from the pull
161+
request. This builds the content and uploads three artifacts:
162+
163+
1. The pull request number (pr)
164+
2. A summary of changes after the rendering process (diff)
165+
3. The rendered files (build)
166+
167+
Because this workflow builds generated content, it follows the same general
168+
process as the `sandpaper-main` workflow with the same caching mechanisms.
169+
170+
The artifacts produced are used by the next workflow.
171+
172+
### Comment on Pull Request (pr-comment.yaml)
173+
174+
This workflow is triggered if the `pr-receive.yaml` workflow is successful.
175+
The steps in this workflow are:
176+
177+
1. Test if the workflow is valid and comment the validity of the workflow to the
178+
pull request.
179+
2. If it is valid: create an orphan branch with two commits: the current state
180+
of the repository and the proposed changes.
181+
3. If it is valid: update the pull request comment with the summary of changes
182+
183+
Importantly: if the pull request is invalid, the branch is not created so any
184+
malicious code is not published.
185+
186+
From here, the maintainer can request changes from the author and eventually
187+
either merge or reject the PR. When this happens, if the PR was valid, the
188+
preview branch needs to be deleted.
189+
190+
### Send Close PR Signal (pr-close-signal.yaml)
191+
192+
Triggered any time a pull request is closed. This emits an artifact that is the
193+
pull request number for the next action
194+
195+
### Remove Pull Request Branch (pr-post-remove-branch.yaml)
196+
197+
Tiggered by `pr-close-signal.yaml`. This removes the temporary branch associated with
198+
the pull request (if it was created).

‎.github/workflows/pr-close-signal.yaml

+22
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
name: "Bot: Send Close Pull Request Signal"
2+
3+
on:
4+
pull_request:
5+
types:
6+
[closed]
7+
8+
jobs:
9+
send-close-signal:
10+
name: "Send closing signal"
11+
runs-on: ubuntu-22.04
12+
if: ${{ github.event.action == 'closed' }}
13+
steps:
14+
- name: "Create PRtifact"
15+
run: |
16+
mkdir -p ./pr
17+
printf ${{ github.event.number }} > ./pr/NUM
18+
- name: Upload Diff
19+
uses: actions/upload-artifact@v4
20+
with:
21+
name: pr
22+
path: ./pr

0 commit comments

Comments
 (0)