Ensure reproducible builds on Linux#1731
Conversation
|
Thank you for working on reproducible Linux builds. This is a useful goal, but I can't merge this as-is. The PR description says that "each flag/tool feature is probed before use, so older toolchains and the FreeBSD/macOS Some compiler/linker/archive features are probed, but several new packaging requirements are used unconditionally: FreeBSD/macOS are also not completely unaffected: the new reproducibility flags in There is also a scope issue. The current Linux Before this can be considered for merge, please:
|
|
here is the log: Note that the ci failure seems unrelated to the code. |
|
@curious-rabbit A few items still need another revision before I can merge. Below is a list (sorry for the verbosity, I wanted to give as much details as possible because this feature is important) 1. The new block is declared at the top of the file, but the actual There is also a more serious concern: under CPack, this project sets
2. The Both 3. Makeself stores the POSIX
Please pick one of these rather than a third option. 4. The The recipe relies on 5. Minor:
6. Reproducibility log only covers the legacy Makefile path The log you provided exercises only: make NOGUI=1 -j1 && make NOGUI=1 -j1 packagethe console tar.gz + makeself path. The PR now also modifies CMakeLists.txt, build_cmake_deb.sh, build_cmake_rpm.sh, build_cmake_opensuse.sh, and the appimage target. Please either narrow the PR back to the legacy tar.gz/Makeself artifacts, or provide reproducibility logs for the DEB, RPM, openSUSE and AppImage paths as well, built from two distinct source paths and compared by SHA-256, like your existing log. All in all, the direction is good and I'd like to see this land. Once the items above (especially #1, #2 and #3, which are correctness issues, not style) are addressed I'll take another pass. |
Only the DEB log is here. I cannot produce the other three easily without setting up a bunch of VMs. RPM and openSUSE need an RPM-based machine to build on, since that part of the CMake config only runs on Fedora or openSUSE, not on the Ubuntu system I use. The underlying timestamp fix should work though. So the legacy and DEB paths are verified reproducible. I wont find the time to verify RPM but we can narrow this PR down to just the legacy and DEB paths. I also noticed that the veracrypt package in Arch is already in their reproducible list somehow. |
|
@curious-rabbit I reviewed the latest revision. The main correctness issues from my previous review look addressed. The remaining issue is scope. Your latest comment says you can narrow this PR to the verified legacy Makefile + DEB paths, but the branch still touches RPM/openSUSE and AppImage code paths that aren't verified here. To avoid asking you for another round of mechanical cleanup, would you be okay if I push a small maintainer edit to this PR branch? One important detail: this PR is opened from the master branch of your fork ( If you agree, I will limit the edit to narrowing the scope:
|
|
@idrassi Feel free to add any changes you want. the entire for exists only for this features anyway. |
Make the Linux build and packaging pipeline deterministic so identical
sources yield byte-identical binaries and installer archives regardless
of build path, host, user or wall-clock time. Every change degrades
safely: each flag/tool feature is probed before use, so older toolchains
and the FreeBSD/macOS build paths are unaffected.
src/Makefile
fixed constant for git-less tarball builds).
-fno-record-gcc-switches, each gated behind a cc-option probe so
compilers that lack them (GCC < 8) still build.
src/Build/Include/Makefile.inc
selected via functional probes; old binutils falls back to a normal
archive. Remove the stale archive first so rebuilds cannot merge
leftover members.
src/Main/Main.make
SOURCE_DATE_EPOCH so checkout-time stamps do not leak into archives.
mtime, numeric 0/0 ownership and 'gzip -n', and drive makeself with
--packaging-date / --tar-extra.
Note that other builds like Windows are not touched and probably impossible to build reproducible. Not like there is any point to this on closed source platforms anyway