Skip to content

Commit 1540810

Browse files
decyjphrdecyjphr
authored
Changes for handling org level rulesets (#519)
* initial changes for handling org level rulesets * repo level rulesets and better PR validation message * add key_prefix to name fields for autolinks * add code to handle ruleset events * add code to report errors as checks * Update README.md * clean up PR handling and error handling messages
1 parent 95db107 commit 1540810

26 files changed

+2838
-875
lines changed

‎README.md

Lines changed: 48 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -5,21 +5,29 @@
55
`Safe-settings`– an app to manage policy-as-code and apply repository settings to repositories across an organization.
66

77
1. In `safe-settings` all the settings are stored centrally in an `admin` repo within the organization. This is important. Unlike [Settings Probot](https://github.com/probot/settings), the settings files cannot be in individual repositories.
8-
> **Note** It is possible to override this behavior and specify a custom repo instead of the `admin` repo.<br>
8+
> **Note**
9+
> It is possible to override this behavior and specify a custom repo instead of the `admin` repo.<br>
910
> This could be done by setting an `env` variable called `ADMIN_REPO`.
1011
11-
1. There are 3 levels at which the settings could be managed:
12+
1. In `safe-settings` the settings can have 2 types of targets:
13+
1. `org` - These settings are applied to the `org`. `Org`-targeted settings are defined in `.github/settings.yml` . Currently, only `rulesets` are supported as `org`-targeted settings.
14+
2. `repo` - These settings are applied to `repos`
15+
16+
2. For The `repo`-targeted settings there can be at 3 levels at which the settings could be managed:
1217
1. Org-level settings are defined in `.github/settings.yml`
13-
> **Note** It is possible to override this behavior and specify a different filename for the `settings` yml repo.<br>
18+
> **Note**
19+
> It is possible to override this behavior and specify a different filename for the `settings` yml repo.<br>
1420
> This could be done by setting an `env` variable called `SETTINGS_FILE_PATH`.
1521
16-
1. `Suborg` level settings. A `suborg` is an arbitrary collection of repos belonging to projects, business units, or teams. The `suborg` settings reside in a yaml file for each `suborg` in the `.github/suborgs` folder.
17-
1. `Repo` level settings. They reside in a repo specific yaml in `.github/repos` folder
18-
1. It is recommended to break the settings into org-level, suborg-level, and repo-level units. This will allow different teams to define and manage policies for their specific projects or business units. With `CODEOWNERS`, this will allow different people to be responsible for approving changes in different projects.
22+
2. `Suborg` level settings. A `suborg` is an arbitrary collection of repos belonging to projects, business units, or teams. The `suborg` settings reside in a yaml file for each `suborg` in the `.github/suborgs` folder.
23+
3. `Repo` level settings. They reside in a repo specific yaml in `.github/repos` folder
24+
3. It is recommended to break the settings into org-level, suborg-level, and repo-level units. This will allow different teams to define and manage policies for their specific projects or business units. With `CODEOWNERS`, this will allow different people to be responsible for approving changes in different projects.
1925

20-
> **Note** `Suborg` and `Repo` level settings directory structure cannot be customized.
26+
> **Note**
27+
> `Suborg` and `Repo` level settings directory structure cannot be customized.
2128
22-
> **Note** The settings file must have a `.yml` extension only. `.yaml` extension is ignored, for now.
29+
> **Note**
30+
> The settings file must have a `.yml` extension only. `.yaml` extension is ignored, for now.
2331
2432
## How it works
2533

@@ -36,6 +44,10 @@ The App listens to the following webhook events:
3644

3745
- **pull_request.opened**, **pull_request.reopened**, **check_suite.requested**: If the settings are changed, but it is not in the `default` branch, and there is an existing PR, the code will validate the settings changes by running safe-settings in `nop` mode and update the PR with the `dry-run` status.
3846

47+
- **repository_ruleset**: If the `ruleset` settings are modified in the UI manually, `safe-settings` will `sync` the settings to prevent any unauthorized changes.
48+
49+
- **member_change_events**: If a member is added or removed from a repository, `safe-settings` will `sync` the settings to prevent any unauthorized changes.
50+
3951
### Restricting `safe-settings` to specific repos
4052
`safe-settings` can be turned on only to a subset of repos by specifying them in the runtime settings file, `deployment-settings.yml`.
4153
If no file is specified, then the following repositories - `'admin', '.github', 'safe-settings'` are exempted by default.
@@ -45,7 +57,8 @@ To apply `safe-settings` __only__ to a specific list of repos, add them to the `
4557

4658
To ignore `safe-settings` for a specific list of repos, add them to the `restrictedRepos` section as `exclude` array.
4759

48-
> **Note** The `include` and `exclude` attributes support as well regular expressions.
60+
> **Note**
61+
> The `include` and `exclude` attributes support as well regular expressions.
4962
5063
### Custom rules
5164

@@ -210,11 +223,36 @@ The App can be configured to apply the settings on a schedule. This could be a w
210223
To periodically converge the settings to the configuration, set the `CRON` environment variable. This is based on [node-cron](https://www.npmjs.com/package/node-cron) and details on the possible values can be found [here](#env-variables).
211224

212225
### Pull Request Workflow
226+
It is
213227
`Safe-settings` explicitly looks in the `admin` repo in the organization for the settings files. The `admin` repo could be a restricted repository with `branch protections` and `codeowners`
214228

215229
In that set up, when changes happen to the settings files and there is a PR for merging the changes back to the `default` branch in the `admin` repo, `safe-settings` will run `checks` – which will run in **nop** mode and produce a report of the changes that would happen, including the API calls and the payload.
216230

217-
The checks will fail if `org-level` branch protections are overridden at the repo or suborg level with a lesser number of required approvers.
231+
For e.g. If we have `override` validators that will fail if `org-level` branch protections are overridden at the repo or suborg level with a lesser number of required approvers, here is an screenshot of what users will see in the PR.
232+
<p>
233+
<img width="467" alt="image" src="https://github.com/github/safe-settings/assets/57544838/cc5d59fb-3d7c-477b-99e9-94bcafd07c0b">
234+
</p>
235+
236+
> **NOTE**
237+
> If you don't want the PR message to have these details, it can be turned off by `env` setting `CREATE_PR_COMMENT`=`false`
238+
239+
Here is a screenshot of what the users will see in the `checkrun` page:
240+
<p>
241+
<img width="462" alt="image" src="https://github.com/github/safe-settings/assets/57544838/c875224f-894b-45da-a9cc-4bfc75c47670">
242+
</p>
243+
244+
### Error handling
245+
The app creates a `Check` at the end of its processing to indicate if there were any errors. The `Check` is called `safe-settings` and corrosponds to the latest commit on the `default` branch of the `admin` repo.
246+
247+
Here is an example of a `checkrun` result:
248+
<p>
249+
<img width="944" alt="image" src="https://github.com/github/safe-settings/assets/57544838/7ccedcea-628e-4055-a5a5-b8e45123777e">
250+
</p>
251+
252+
And the `checkrun` page will look like this:
253+
<p>
254+
<img width="860" alt="image" src="https://github.com/github/safe-settings/assets/57544838/893ff4e6-904c-4a07-924a-7c23dc068983">
255+
</p>
218256

219257
### The Settings file
220258

‎docs/sample-settings/org-ruleset.json

Lines changed: 115 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,115 @@
1+
{
2+
"name": "demo repo ruleset",
3+
"target": "branch",
4+
"enforcement": "disabled",
5+
"bypass_actors": [
6+
{
7+
"actor_id": 3974045,
8+
"actor_type": "Team",
9+
"bypass_mode": "pull_request"
10+
},
11+
{
12+
"actor_id": 7898,
13+
"actor_type": "RepositoryRole",
14+
"bypass_mode": "always"
15+
},
16+
{
17+
"actor_id": 210920,
18+
"actor_type": "Integration",
19+
"bypass_mode": "always"
20+
},
21+
{
22+
"actor_id": 1,
23+
"actor_type": "OrganizationAdmin",
24+
"bypass_mode": "always"
25+
},
26+
{
27+
"actor_id": 5,
28+
"actor_type": "RepositoryRole",
29+
"bypass_mode": "pull_request"
30+
}
31+
],
32+
"conditions": {
33+
"ref_name": {
34+
"include": [
35+
"~DEFAULT_BRANCH"
36+
],
37+
"exclude": [
38+
"refs/heads/oldmaster"
39+
]
40+
},
41+
"repository_name": {
42+
"include": [
43+
"test*"
44+
],
45+
"exclude": [
46+
"test",
47+
"test1"
48+
],
49+
"protected": true
50+
}
51+
},
52+
"rules": [
53+
{
54+
"type": "creation"
55+
},
56+
{
57+
"type": "update"
58+
},
59+
{
60+
"type": "deletion"
61+
},
62+
{
63+
"type": "required_linear_history"
64+
},
65+
{
66+
"type": "required_signatures"
67+
},
68+
{
69+
"type": "pull_request",
70+
"parameters": {
71+
"dismiss_stale_reviews_on_push": true,
72+
"require_code_owner_review": true,
73+
"require_last_push_approval": true,
74+
"required_approving_review_count": 10,
75+
"required_review_thread_resolution": true
76+
}
77+
},
78+
{
79+
"type": "commit_message_pattern",
80+
"parameters": {
81+
"name": "test commit_message_pattern",
82+
"negate": true,
83+
"operator": "starts_with",
84+
"pattern": "skip*"
85+
}
86+
},
87+
{
88+
"type": "commit_author_email_pattern",
89+
"parameters": {
90+
"name": "test commit_author_email_pattern",
91+
"negate": false,
92+
"operator": "regex",
93+
"pattern": "^.*@example.com$"
94+
}
95+
},
96+
{
97+
"type": "committer_email_pattern",
98+
"parameters": {
99+
"name": "test committer_email_pattern",
100+
"negate": false,
101+
"operator": "regex",
102+
"pattern": "^.*@example.com$"
103+
}
104+
},
105+
{
106+
"type": "branch_name_pattern",
107+
"parameters": {
108+
"name": "test branch_name_pattern",
109+
"negate": false,
110+
"operator": "regex",
111+
"pattern": "^(feature|bugfix|improvement|library|prerelease|release|hotfix)\/[a-z0-9._-]+$"
112+
}
113+
}
114+
]
115+
}

‎docs/sample-settings/org-ruleset.yml

Lines changed: 153 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,153 @@
1+
rulesets:
2+
- name: demo
3+
# Name of the rule
4+
target: branch
5+
# The target of the ruleset. Can be one of:
6+
# - branch
7+
# - tag
8+
enforcement: disabled
9+
# The enforcement level of the ruleset. `evaluate` allows admins to test
10+
# rules before enforcing them.
11+
# - disabled
12+
# - active
13+
# - evaluate
14+
bypass_actors:
15+
# The actors that can bypass the rules in this ruleset
16+
- actor_id: 3974045
17+
actor_type: Team
18+
# type: The type of actor that can bypass a ruleset
19+
# - RepositoryRole
20+
# - Team
21+
# - Integration
22+
# - OrganizationAdmin
23+
bypass_mode: pull_request
24+
# When the specified actor can bypass the ruleset. `pull_request`
25+
# means that an actor can only bypass rules on pull requests.
26+
# - always
27+
# - pull_request
28+
29+
- actor_id: 1
30+
actor_type: OrganizationAdmin
31+
bypass_mode: always
32+
33+
- actor_id: 7898
34+
actor_type: RepositoryRole
35+
bypass_mode: always
36+
37+
- actor_id: 210920
38+
actor_type: Integration
39+
bypass_mode: always
40+
41+
conditions:
42+
ref_name:
43+
# Parameters for a repository ruleset ref name condition
44+
include: ["~DEFAULT_BRANCH"]
45+
# Array of ref names or patterns to include. One of these
46+
# patterns must match for the condition to pass. Also accepts
47+
# `~DEFAULT_BRANCH` to include the default branch or `~ALL` to
48+
# include all branches.
49+
exclude: ["refs/heads/oldmaster"]
50+
# Array of ref names or patterns to exclude. The condition
51+
# will not pass if any of these patterns match.
52+
repository_name:
53+
include: ["test*"]
54+
# Array of repository names or patterns to include.
55+
# One of these patterns must match for the condition
56+
# to pass. Also accepts `~ALL` to include all
57+
# repositories.
58+
exclude: ["test","test1"]
59+
# Array of repository names or patterns to exclude. The
60+
# condition will not pass if any of these patterns
61+
# match.
62+
protected: true
63+
# Whether renaming of target repositories is
64+
# prevented.
65+
66+
rules:
67+
- type: creation
68+
- type: update
69+
- type: update_allows_fetch_and_merge
70+
- type: deletion
71+
- type: required_linear_history
72+
- type: required_signatures
73+
- type: required_deployments
74+
parameters:
75+
required_deployment_environments: ["test"]
76+
- type: pull_request
77+
parameters:
78+
dismiss_stale_reviews_on_push: true
79+
# New, reviewable commits pushed will dismiss previous pull
80+
# request review approvals.
81+
require_code_owner_review: true
82+
# Require an approving review in pull requests that modify
83+
# files that have a designated code owner
84+
require_last_push_approval: true
85+
# Whether the most recent reviewable push must be approved
86+
# by someone other than the person who pushed it.
87+
required_approving_review_count: 10
88+
# The number of approving reviews that are required before a
89+
# pull request can be merged.
90+
required_review_thread_resolution: true
91+
# All conversations on code must be resolved before a pull
92+
# request can be merged.
93+
94+
- type: required_status_checks
95+
# Choose which status checks must pass before branches can be merged
96+
# into a branch that matches this rule. When enabled, commits must
97+
# first be pushed to another branch, then merged or pushed directly
98+
# to a branch that matches this rule after status checks have
99+
# passed.
100+
parameters:
101+
strict_required_status_checks_policy: false
102+
# Whether pull requests targeting a matching branch must be
103+
# tested with the latest code. This setting will not take
104+
# effect unless at least one status check is enabled.
105+
required_status_checks:
106+
- context: CodeQL
107+
integration_id: 1234
108+
- context: GHAS Compliance
109+
integration_id: 1234
110+
111+
- type: commit_message_pattern
112+
parameters:
113+
name: test commit_message_pattern
114+
# required:
115+
# - operator
116+
# - pattern
117+
negate: true
118+
operator: starts_with
119+
# The operator to use for matching.
120+
# - starts_with
121+
# - ends_with
122+
# - contains
123+
# - regex
124+
pattern: skip*
125+
# The pattern to match with.
126+
127+
- type: commit_author_email_pattern
128+
parameters:
129+
name: test commit_author_email_pattern
130+
negate: false
131+
operator: regex
132+
pattern: "^.*@example.com$"
133+
134+
- type: committer_email_pattern
135+
parameters:
136+
name: test committer_email_pattern
137+
negate: false
138+
operator: regex
139+
pattern: "^.*@example.com$"
140+
141+
- type: branch_name_pattern
142+
parameters:
143+
name: test branch_name_pattern
144+
negate: false
145+
operator: regex
146+
pattern: ".*\/.*"
147+
148+
- type: "tag_name_pattern"
149+
parameters:
150+
name: test tag_name_pattern
151+
negate: false
152+
operator: regex
153+
pattern: ".*\/.*"

0 commit comments

Comments
 (0)