Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update release process #465

Merged
merged 18 commits into from
Jun 4, 2020
Merged

Update release process #465

Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
4b83244
initial proposal
k15r May 13, 2020
b161631
update job generation documentation
k15r May 23, 2020
6e32721
add before cutoff check
k15r May 25, 2020
015daf2
introduce pattern definitions for version formats
k15r May 26, 2020
a3fe1c3
cleanup branch
k15r May 26, 2020
1c67cb3
fix formattingfix formattingfix formattingfix formattingfix
k15r May 26, 2020
d0a5c16
reformat page
k15r May 26, 2020
dac25a9
add documenatation for the release testing sheet
k15r May 27, 2020
9858ec4
Merge remote-tracking branch 'upstream/master' into update-release-pr…
k15r May 27, 2020
643154f
add instructions how to rerun the release
k15r May 27, 2020
b4888f0
refactor the docu a bit
k15r May 27, 2020
a949780
remove unnecessary ordered lists
k15r May 27, 2020
80640e2
fix indentation
k15r May 27, 2020
a61b1b0
fix ordered lists
k15r May 27, 2020
6a88902
Main sheet
k15r May 29, 2020
862d18b
Apply suggestions from code review
k15r May 29, 2020
45eda66
introduce pattern/placeholders table
k15r May 29, 2020
06b477f
Update guidelines/releases-guidelines/02-kyma-release-process.md
k15r May 29, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
210 changes: 128 additions & 82 deletions guidelines/releases-guidelines/02-kyma-release-process.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,168 +12,214 @@ A Kyma release includes the following items:
* A Git tag
* A release branch

## Definitions

The table below includes placeholders used throughout this document. When executing the commands, replace each of them with a suitable release number or version.

| Placeholders | Description | Pattern | Example|
|-------|------------|---------|--------|
| `RELEASE` | Release number| `{major}.{minor}` | `1.13`|
| `RELEASE_VERSION` | Release version | `{major}.{minor}.{patch}` or `{major}.{minor}.{patch}-rc{candidate}` | `1.13.0` or `1.13.0-rc1` |
| `RELEASE_VERSION_SHORT` | Release version without additional characters | `{major}{minor}` | `113`|
| `RELEASE_VERSION_DASH` | Release version with dashes |`{major}-{minor}-{patch}` or `{major}-{minor}-{patch}-rc{candidate}`| `1-13-0` or `1-13-0-rc1`|

## Preparation

> **NOTE:** This section applies only to new major and minor versions. If you release a patch, skip the preparation and go to the [**Steps**](#kyma-release-process-kyma-release-process-steps) section.

To prepare a release:
### kyma-project/kyma

1. Create a release branch in the `kyma` repository. The name of this branch should follow the `release-x.y` pattern, such as `release-1.4`.
#### Perform initial checks

Check if the master branch contains any PR-images:

```bash
git grep -e 'version:\s*[Pp][Rr]-.*' -e 'image:.*:[Pp][Rr]-.*' --before-context=2 -r resources
```

Ask the teams for fixes if this command returns any output.

#### Create a release branch

>**NOTE:** a release branch needs to be created per new major / minor version. Patch releases and release candidates do not have a dedicated release branch.

Create a release branch in the `kyma` repository. The name of this branch should follow the `release-{major}.{minor}` pattern, such as `release-1.4`.

```bash
git fetch upstream
git checkout -b release-{release_version} upstream/master
git push upstream release-{release_version}
git checkout --no-track -b release-{RELEASE} upstream/master
git push -u upstream release-{RELEASE}
```

> **NOTE:** If you don't create the Kyma release branch at this point and add a `post-rel{release_version}-kyma-release-candidate` post-submit job to the `test-infra` master, then pushing anything to the Kyma release branch, creating or rebasing the branch, triggers a new GitHub release.
>**CAUTION:** If you don't create the Kyma release branch at this point and add a `post-rel{RELEASE_VERSION_SHORT}-kyma-release-candidate` post-submit job to the `test-infra` master, then pushing anything to the Kyma release branch, creating or rebasing the branch, triggers a new GitHub release.

### kyma-project/test-infra

2. [Define new release jobs](#kyma-release-process-kyma-release-process-preparation-define-new-release-jobs) in the `test-infra` repository.
#### Update the jobs on master branch

### Define new release jobs
1. Create a PR to `master` containing the following changes to create the new job definitions:

1. Go to the `test-infra` repository.
2. Open `templates/config.yaml`
3. Add the new release to `global.releases`. Remove the oldest release on the list.
4. Set `global.nextRelease` to the future release version.
5. Run `dep ensure -v --vendor-only` in the `development/tools` directory to install dependencies.
5. Run `go run development/tools/cmd/rendertemplates/main.go --config templates/config.yaml` in the root of the repository to generate jobs.
6. Run `go test ./development/tools/jobs/...` in the root of the repository. If any of the tests is marked red, fix it using these guidelines:
* For release tests using `GetKymaReleasesSince` or `jobsuite.Since` with a release that is no longer supported, change the method to `GetAllKymaReleases` or `jobsuite.AllReleases` respectively.
* For release tests using `GetKymaReleasesUntil` or `jobsuite.Until` with a release that is no longer supported, remove the part of the test which includes the method.
7. If tests are green, commit all jobs. The new release jobs are ready.
1. Open `templates/config.yaml`
2. Add the new release to `global.releases`. Remove the oldest release on the list.
3. Set `global.nextRelease` to the future release version.
4. Run `make` in the root of the repository to generate jobs and run tests. If any of the tests is marked red, fix it using these guidelines:
* For release tests using `GetKymaReleasesSince` or `jobsuite.Since` with a release that is no longer supported, change the method to `GetAllKymaReleases` or `jobsuite.AllReleases` respectively.
* For release tests using `GetKymaReleasesUntil` or `jobsuite.Until` with a release that is no longer supported, remove the part of the test which includes the method.
5. If tests are green, commit all jobs.

## Steps
Follow these steps to release another Kyma version.
2. Once the PR is merged to master you can proceed.

### kyma-project/test-infra
#### Create a release branch

1. Create a release branch in the `test-infra` repository. The name of this branch should follow the `release-x.y` pattern, such as `release-1.4`.
>**NOTE:** a release branch needs to be created per new major / minor version. Patch releases and release candidates do not have a dedicated release branch. If this branch already exists this step will be skipped.

> **NOTE:** This step applies only to new major and minor versions.
Create a release branch in the `test-infra` repository

```bash
git fetch upstream
git checkout -b {RELEASE_NAME} upstream/master
```
```bash
git fetch upstream
git checkout --no-track -b release-{RELEASE} upstream/master
git push -u upstream release-{RELEASE}
```

2. Ensure that the `prow/RELEASE_VERSION` file from the `test-infra` repository on a release branch contains the correct version to be created. The file should contain a release version following the `{A}.{B}.{C}` or `{A}.{B}.{C}-rc{D}` format, where `A`,`B`, `C`, and `D` are numbers. If you define a release candidate version, a pre-release is created.

Make sure the `RELEASE_VERSION` file includes just a single line, **without the newline character at the end**:
## Steps

```bash
echo -n {RELEASE_VERSION} > prow/RELEASE_VERSION
```
Follow these steps to release another Kyma version. Execute these steps for every patch release or release candidate.

### kyma-project/test-infra


Ensure that the `prow/RELEASE_VERSION` file from the `test-infra` repository on a release branch contains the correct version to be created. If you define a release candidate version, a pre-release is created.

1. Make sure the `RELEASE_VERSION` file includes just a single line, **without the newline character at the end**:

```bash
echo -n {RELEASE_VERSION} > prow/RELEASE_VERSION
```

2. If you had to change the RELEASE_VERSION, create a PR to update it on the release branch.
3. Once this PR is merged you can proceed.

3. Push the branch to the `test-infra` repository.

4. Update the `RELEASE_VERSION` file with the name of the next minor release candidate and merge the pull request to `master`. For example, if the `RELEASE_VERSION` on the `master` branch is set to `1.4.2`, then change the version to `1.5.0-rc1`.

### kyma-project/kyma

#### Create a PR to the release branch

1. Inside the release branch do the following changes.

i. In `installation/resources/installer.yaml` replace `eu.gcr.io/kyma-project/develop/installer:{image_tag}` with `eu.gcr.io/kyma-project/kyma-installer:{release_version}`
1. In `installation/resources/installer.yaml` replace `eu.gcr.io/kyma-project/develop/installer:{image_tag}` with `eu.gcr.io/kyma-project/kyma-installer:{RELEASE_VERSION}`

ii. Find these lines in `tools/kyma-installer/kyma.Dockerfile`:
2. Find these lines in `tools/kyma-installer/kyma.Dockerfile`:

```
ARG INSTALLER_VERSION="{kyma_operator_version}"
ARG INSTALLER_DIR={kyma_operator_path}
FROM $INSTALLER_DIR/kyma-operator:$INSTALLER_VERSION
```
```
ARG INSTALLER_VERSION="{kyma_operator_version}"
ARG INSTALLER_DIR={kyma_operator_path}
FROM $INSTALLER_DIR/kyma-operator:$INSTALLER_VERSION
```

Replace them with:
Replace them with:

```
FROM {kyma_operator_path}/kyma-operator:{kyma_operator_version}
```
```
FROM {kyma_operator_path}/kyma-operator:master-{kyma_operator_version}
```

iii. In the `resources/core/values.yaml` file, replace the `clusterAssetGroupsVersion` value with your release branch name. For example, for the 1.4 release, find the following section:
3. In the `resources/core/values.yaml` file, find the `clusterAssetGroupsVersion`

```yaml
docs:
# (...) - truncated
clusterAssetGroupsVersion: master
```
```yaml
docs:
# (...) - truncated
clusterAssetGroupsVersion: master
```

And replace the `clusterAssetGroupsVersion` value with the following:
And replace the `clusterAssetGroupsVersion` value with the following:

```yaml
docs:
# (...)
clusterAssetGroupsVersion: release-1.4
```
```yaml
docs:
# (...)
clusterAssetGroupsVersion: release-{RELEASE}
```

iv. Create a pull request with your changes to the release branch.
2. Create a pull request with your changes to the release branch.

![PullRequest](./assets/release-PR.png)

2. If `pre-release-pr-image-guard` fails, ask the owners to change PR-XXX images of the components to the master version.

> **CAUTION:** Never use `/test all` as it might run tests that you do not want to execute.

3. Execute remaining tests. There are dependencies between jobs, so follow the provided order of steps.
3. If `pre-release-pr-image-guard` fails, ask the owners to change PR-XXX images of the components to the master version.

i. Run `kyma-integration` by adding the `/test pre-rel{release_number}-kyma-integration` comment to the PR.

#### Execute the tests for the release PR

> **NOTE:** You don't have to wait until the `pre-rel{release_number}-kyma-integration` job finishes to proceed with further jobs.
> **CAUTION:** Never use `/test all` as it might run tests that you do not want to execute.

1. Execute remaining tests. There are dependencies between jobs, so follow the provided order of steps.

ii. Run `/test pre-rel{release_number}-kyma-installer` and wait until it finishes.
1. Run `kyma-integration` by adding the `/test pre-rel{RELEASE_VERSION_SHORT}-kyma-integration` comment to the PR.

iii. Run `/test pre-rel{release_number}-kyma-artifacts` and wait until it finishes.
> **NOTE:** You don't have to wait until the `pre-rel{RELEASE_VERSION_SHORT}-kyma-integration` job finishes to proceed with further jobs.

iv. Run the following tests in parallel and wait for them to finish:
2. Execute the next steps in the following order
1. Run `/test pre-rel{RELEASE_VERSION_SHORT}-kyma-installer` and wait until it finishes.
2. Run `/test pre-rel{RELEASE_VERSION_SHORT}-kyma-artifacts` and wait until it finishes.
3. Run the following tests in parallel and wait for them to finish:

```bash
/test pre-rel{release_number}-kyma-gke-integration
/test pre-rel{release_number}-kyma-gke-central-connector
/test pre-rel{release_number}-kyma-gke-upgrade
```
```
/test pre-rel{RELEASE_VERSION_SHORT}-kyma-gke-integration
/test pre-rel{RELEASE_VERSION_SHORT}-kyma-gke-central-connector
/test pre-rel{RELEASE_VERSION_SHORT}-kyma-gke-upgrade
```

4. If you detect any problems with the release, such as failing tests, wait for the fix that can be either delivered on a PR or cherry-picked to the PR from the `master` branch. Prow triggers the jobs again. Rerun manual jobs as described in **step 3**.
2. If you detect any problems with the release, such as failing tests, wait for the fix that can be either delivered on a PR or cherry-picked to the PR from the `master` branch. Prow triggers the jobs again. Rerun manual jobs as described in .

5. After all checks pass, merge the PR, using the `rebase and merge` option.
3. After all checks pass, merge the PR, using the `rebase and merge` option.

> **CAUTION:** By default, the `rebase and merge` option is disabled. Contact one of the `kyma-project/kyma` repository admins to enable it.

6. Merging the PR to the release branch runs the postsubmit jobs, which:
4. Merging the PR to the release branch runs the postsubmit jobs, which:

* create a GitHub release and trigger documentation update on the official Kyma website
* trigger provisioning of the cluster from the created release. The cluster name contains the release version with a period `.` replaced by a dash `-`. For example: `gke-release-1-4-0-rc1`. Use the cluster to test the release candidate.
* trigger provisioning of the cluster from the created release. Use the cluster to test the release candidate.

> **CAUTION**: The cluster is automatically generated for you, but removal is a manual action. The release cluster, the IP Addresses, and the DNS records must be deleted manually after tests on the given cluster are done.

If you don't have access to the GCP project, post a request in the Slack team channel.

```bash
gcloud container clusters get-credentials gke-release-1-4-0-rc2 --zone europe-west4-c --project sap-kyma-prow-workloads
gcloud container clusters get-credentials gke-{RELEASE_VERSION_DASH} --zone europe-west4-c --project sap-kyma-prow-workloads
```

Follow [these](https://kyma-project.io/docs/#installation-install-kyma-with-your-own-domain-access-the-cluster) instructions to give Kyma teams access to start testing the release candidate.

7. The Github release postsumbit job creates a tag in the `kyma-project/kyma` repository, which triggers the [`post-kyma-release-upgrade`](https://github.com/kyma-project/test-infra/blob/master/prow/jobs/kyma/kyma-release-upgrade.yaml) pipeline. The purpose of this job is to test upgradability between the previous Kyma release, i.e. the latest release that is not a release candidate, and the brand new release published by the release postsubmit job.
5. The Github release postsumbit job creates a tag in the `kyma-project/kyma` repository, which triggers the [`post-kyma-release-upgrade`](https://github.com/kyma-project/test-infra/blob/master/prow/jobs/kyma/kyma-release-upgrade.yaml) pipeline. The purpose of this job is to test upgradability between the previous Kyma release, i.e. the latest release that is not a release candidate, and the brand new release published by the release postsubmit job.

For example, if `1.7.0-rc2` is released, the pipeline will try to upgrade `1.6.0` to `1.7.0-rc2`.

If you detect any problems with the upgrade, contact the teams responsible for failing components.

> **CAUTION:** The job assumes no manual migration is involved. If the upgrade process requires any additional actions, the pipeline is likely to fail. In such case, the owners of the components concerned are responsible for running manual tests or modifying the pipeline.

8. Update the `RELEASE_VERSION` file to contain the next patch RC1 version on the release branch. Do it immediately after the release, otherwise, any PR to a release branch overrides the previously published Docker images.
6. Update the `RELEASE_VERSION` file to contain the next patch RC1 version on the release branch. Do it immediately after the release, otherwise, any PR to a release branch overrides the previously published Docker images.

For example, if the `RELEASE_VERSION` file on the release branch contains `1.4.1`, change it to `1.4.2-rc1`.

9. Validate the `yaml` and changelog files generated under [releases](https://github.com/kyma-project/kyma/releases).
7. Validate the `yaml` and changelog files generated under [releases](https://github.com/kyma-project/kyma/releases).

10. Update the release content manually with links to:
8. Update the release content manually with links to:

- Instructions on local Kyma installation
- Instructions on cluster Kyma installation
- Release notes

For installation instructions, use the links from the previous release and update the version number in URLs. If contributors want you to change something in the instructions, they would address you directly. Contact technical writers for the link to release notes.

11. Create a spreadsheet with all open issues labeled as `test-missing`. Every team assigned to an issue must cover the outstanding test with manual verification on every release candidate. After the test is finished successfully, the responsible team must mark it as completed in the spreadsheet. Every issue identified during testing must be reported. To make the testing easier, provision a publicly available cluster with the release candidate version after you complete all steps listed in this document.
9. Create a new sheet in the [Release Testing](https://docs.google.com/spreadsheets/d/1ty3OciQzgzv0GagTG2Dku9os2AfMupbGNf8QxjHaO88)
>**NOTE:** Make sure that you are not signed in with your SAP Google Account
1. Open the **Main** sheet
2. Click the **generate new sheet** button.
3. You will be asked for a GitHub personal access token. This token does not need any additional scopes!


> **NOTE:** After the Kyma release is complete, proceed with [releasing Kyma CLI](/guidelines/releases-guidelines/03-kyma-cli-release-process.md).

## Post-release tasks

Update `prow/RELEAS_VERSION` in the `master` branch of the `test-infra` repository with the name of the next minor release candidate and merge the pull request to `master`. For example, if the `RELEASE_VERSION` on the `master` branch is set to `1.4.2`, then change the version to `1.5.0-rc1`.
18 changes: 15 additions & 3 deletions guidelines/releases-guidelines/03-kyma-cli-release-process.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ A Kyma CLI release consists of:

```bash
git fetch upstream
git checkout -b {RELEASE_NAME} upstream/master
git checkout --no-track -b {RELEASE_NAME} upstream/master
git push upstream {RELEASE_NAME}
```

Expand All @@ -33,9 +33,21 @@ A Kyma CLI release consists of:

```bash
git tag -a {RELEASE_VERSION} -m "Release {RELEASE_VERSION}"
git push upstream {RELEASE_VERSION}
```

where {RELEASE_VERSION} could be `1.4.0-rc1`, `1.4.0-rc2` or `1.4.0`.

5. Push the tag to trigger a postsubmit job that creates the GitHub release. Validate if the release is available under [releases](https://github.com/kyma-project/cli/releases).
5. Create the new version
1. Push the tag to trigger a postsubmit job that creates the GitHub release. Validate if the release is available under [releases](https://github.com/kyma-project/cli/releases).

```bash
git push upstream {RELEASE_VERSION}
```

2. Verify the [Prow Status](https://status.build.kyma-project.io/?repo=kyma-project%2Fcli&type=postsubmit) of the matching revision ({RELEASE_VERSION}).
3. If the post submit job failed, you can re-trigger it by removing the tag from upstream and pushing it again.

```bash
git push --delete upstream {RELEASE_VERSION}
git push upstream {RELEASE_VERSION}
```