Are bottles ever suitable?

[letmaik]

@zbeekman You mentioned in WRF-CMake/wrf#22 that it would be awesome if WRF-CMake makes it into the Homebrew package manager, which would mean upstreaming the formula to homebrew-core and then letting the homebrew build bots create bottles (=binary distributions). Is that correct?

I thought about this a bit more and identified a potential blocker. First, some context.

Currently when users build WRF-CMake from source then they are more or less on their own, meaning that if they have an incompatible compiler or dependency version on their system then they have to fix it and we can't do much about it. We could mention CI-tested compiler and library versions somewhere, but that's a bonus. In general it all works out and it's nearly never an issue, but it may be.

When users download our pre-built binaries then they can be sure that this exact binary including all bundled dependencies has been tested and validated by our CI setup. There is nearly nothing that can go wrong there, apart from having the wrong mpi runner installed, but that's it.

With Homebrew bottles, we don't have this gating process and also don't have control over library versions. We can influence the compiler to some degree but it's not an ultimate solution.

Given all this, does it make sense at all to try to move the WRF-CMake formula to homebrew-core? How could we prevent that new bottles are built that accidentally lead to breaks that are only visible at runtime? Keep in mind that our CI-based tests take many hours to run and they couldn't be integrated into the test part of the formula. We could have a very simple "doesn't crash when running a super-short simulation" test but this is tricky as well, since some failure scenarios become only visible in debug mode and otherwise just happily corrupt memory in release mode which may not become obvious for very small tests.

In summary, we want to be careful when releasing binary distributions and I don't see a good way to have sufficient quality control with brew bottles in this case.

[zbeekman]

You mentioned in WRF-CMake/WRF#22 that it would be awesome if WRF-CMake makes it into the Homebrew package manager, which would mean upstreaming the formula to homebrew-core and then letting the homebrew build bots create bottles (=binary distributions). Is that correct?

yes

In summary, we want to be careful when releasing binary distributions and I don't see a good way to have sufficient quality control with brew bottles in this case.

Sure, that makes sense. Homebrew is a rolling release package manager, so you may not want to ship WRF there because of that.

FYI, it may be quite some time before WRF-CMake (or assuming it gets pulled in upstream, then just WRF) would even be eligible for homebrew-core. This is because the software source repository must be "popular" enough. (Stars, forks, watchers.)

So I think this is a topic that we could table for quite a while.

I have some ideas and suggestions for ensuring there are no surprises (like setting up nightly testing with brew install --head wrf) but let's sit tight for the time being.

[letmaik]

OK, makes sense. Just wanted to have it discussed. Let's come back to it when the time's right.