Automate releases using GitHub Actions

[GeorgesStavracas]

Currently there still is a fair amount of manual work to do in order to get a release out. It's documented in the RELEASE_HOWTO.md file. Ideally, the most of the release process would be automated, necessitating only a tag to kick off a full release.

To do this, we need to satisfy some preconditions:

• The main branch must always correctly run meson dist (ci: Exercise creating dist tarball #1089)
• Translations should be generated automatically (when? how?)
• Force-pushing to main must be entirely blocked - even for maintainers

[whot]

fwiw, input-wacom recently switched to doing that, in case you need an example to get ideas from.

[flexagoon]

@GeorgesStavracas regarding the automatic generation of translations, if every single change will be going through a PR, one option would probably be to create another action that checks if any of the files mentioned in POTFILES.in have been modified, that way the translations will be up to date on any commit.

Now there are two options:

1. Make this a blocking check and prevent the PR from being merged if the translations are out of date

2. Make the action automatically commit updated translations into the PR branch

The second option is definitely easier for contributors, but it would add a lot of extra unnecessary "update the translation" commits. I'm not sure if that's really desired.

---

But there's also another option - the release action can automatically update the translations, commit them and force push them to the release tag.

This seems like a great approach, though I'm not sure if force pushing a commit to an existing tag on GitHub is possible. I'll need to play around with GitHub and branch protection rules and try it out.

[flexagoon]

I did some experimentation and yes, it is absolutely possible to run an action when a tag is pushed and then add a commit to that existing tag. The only issue is that all developers will have to run git fetch --tags --force locally after every release, otherwise git will refuse to push.

So the choice is probably between three workflows:

1. Generating translations after every pr, as outlined in my previous comment
2. Generating and force-pushing translations when a new tag is added
3. Instead of adding any tags, the maintainers manually trigger the "release" github action, which commits updated translations and then that action creates the tag

Which of those would be preferred?

[GeorgesStavracas]

I think what you outlined at (3) is what I had in mind when opening this issue. What I wanted to do is some sort of manual action where I'd input:

• Release version (e.g. 1.19.0)
• Pre-release or not
• Changelog

And the action would do essentially everything outlined in RELEASE_HOWTO.md.

[flexagoon]

Hmm, I don't think there is any way to interactively provide inputs to a GitHub action.

Perhaps there could be a releases.yaml/toml/json/whatever file that triggers a release when someone commits to it?

[flexagoon]

Oh, wait, you can actually provide inputs to an action:

https://github.blog/changelog/2020-07-06-github-actions-manual-triggers-with-workflow_dispatch/

However, you can't provide a multiline string to it

https://github.com/orgs/community/discussions/12882

So I'm not sure if it'll be a good fit for changelogs, unless you're fine with using this workaround

[GeorgesStavracas]

Nah, it's fine to update the NEWS file before running the action :)

[flexagoon]

Oops, I didn't see that the repo already has a NEWS file.

I'll try to work on this issue then!

[flexagoon]

The RELEASE_HOWTO file mentions building the documentation (although it lists the old docbook meson flag instead of the new one). Is this step still necessary or does the pages.yml action already handle this?

[GeorgesStavracas]

I think that's outdated, pages.yml takes care of that already.

[flexagoon]

@GeorgesStavracas another question:

The version and the changelog can be taken from the NEWS file automatically with some regex. Should I also use it to automatically determine if the release is a pre-release? (Since this project uses odd numbers in minor versions to denote pre-releases)

[GeorgesStavracas]

I think I'd prefer if the version and pre-release checks were manual. I've made mistakes in the NEWS file in the past, it would be disappointing if that ends up messing up releases if it ever happens again.

Not sure if that's possible, but it would be great if the action could check if the release version is strictly bigger than the previous release of the branch. So e.g. on the main branch, it'd check for the previous stable or unstable tag, but on a stable branch, it would look for the last stable tag of that version. Not a blocker though!

(I'm very hype about this! Looking forward to see what you're cooking up there.)

[flexagoon]

Alright, I'll probably make it so that those values are automatically determined by default but you can also specify them when running the action to override the default values.

At this point I managed to automate most of the release process, but I'm not sure what to do with the SECURITY file and the bug report template. It seems like updating them requires the knowledge of what versions are no longer supported. Is this something that can be automatically determined or should I leave those files to be updated manually?

[GeorgesStavracas]

Let's limit the scope for just generating the release for now. Updating the SECURITY file is such a minuscule effort that it can be automated later, if necessary.

I also just thought of something: do you think it would be possible to add a "Dry run" option to the action that runs the the action but doesn't actually do anything, and just prints what it was going to do? Dry runs are super useful to make sure things will work ahead of time.

[flexagoon]

Sure, that would be possible to add, though I'm not really sure what step of the release process this would be useful for - the compilation of the code is already checked by CI on each commit and the rest is basically just updating some files and creating the release.

The only thing that this can probably be useful for is the NEWS file parsing - I can add a flag that skips all of the other steps and just prints the extracted release information, if you want that

[GeorgesStavracas]

Sounds good!

[flexagoon]

What about updating the version in the meson.build file? I assumed that it has to be updated to the current version, but it seems like it's actually supposed to be set to the next version? How is that usually done? And if meson.build contains the number of the next version, perhaps it can be used to extract that information more reliably instead of the NEWS file?

[GeorgesStavracas]

The version in meson.build is updated manually after the release, I don't think the action will have to do anything with it. At most, if you want to go fancy, the action could check if the action release version is equal to meson release version?

[flexagoon]

At most, if you want to go fancy, the action could check if the action release version is equal to meson release version?

That'll mean that there will be three separate sources to extract the version, which just adds unnecessary complexity, so I probably just won't touch meson.build then.

[GeorgesStavracas]

Fixed by #1454