build: compile against LLVM 17 as well as 16 - #264
Open
o2alexanderfedin wants to merge 1 commit into
Open
Conversation
…at is actually used
Groundwork for building elflift as WebAssembly. Measured, not assumed: configuring elfconv
against a prebuilt wasm LLVM 17.0.6 with emcmake now SUCCEEDS and 19 of 25 TUs compile. This
commit carries the changes that are safe on BOTH LLVM versions; the two genuine ports it
uncovered are deliberately NOT in it.
FOUR LLVM 16-to-17 BREAKS, THREE FIXED HERE. Every one is compatible in both directions,
because the native build still runs on LLVM 16 and that is this repository's working path.
1. llvm/ADT/Triple.h becomes llvm/TargetParser/Triple.h, 9 files. Verified the new path
EXISTS in LLVM 16 (/usr/lib/llvm-16/include/llvm/TargetParser/Triple.h), so unconditional.
2. llvm::AttributeMask moved to its own header in 17. VERSION-GUARDED on LLVM_VERSION_MAJOR
at least 17 - the header is ABSENT in 16 and an unconditional include breaks the native
build with "llvm/IR/AttributeMask.h file not found". This one was caught late; see below.
3. The <set> header no longer arrives transitively through LLVM headers. TraceLifter.h now
includes it directly. Without it, DecoderWorkList raises nine "too few template arguments
for class template set" errors and cascades into a misleading "no matching constructor
for BBBag" in VroTraceLifter.cpp.
4. PassManagerBuilder and ConstantExpr::getSelect are REMOVED in 17. NOT fixed here - they
are real ports, not includes, and get their own commit. lifter/Lift.cpp's
PassManagerBuilder include was dead and is dropped.
THE LINK LIST IS TRIMMED TO WHAT SYMBOLS ACTUALLY DEMAND. remill's CMake asked for
aarch64/arm/x86/sparc/nvptx/interpreter/mcjit components. Measured with llvm-nm over every
undefined symbol of the built archives: 467 undefined in, 23 unresolved out, and all 23 are
internal - 5 wasm-linker-provided, 18 from the two TUs that do not yet compile. ZERO LLVM
symbols missing. Re-measured against an archive set with NO codegen libraries at all - no
WebAssembly, AsmPrinter, CodeGen, SelectionDAG or Target - and the unresolved set was
BYTE-IDENTICAL. Nothing reaches for a target library for any ISA, including WebAssembly
itself. The list is now support/core/irreader/bitreader/bitwriter/passes. remill's
tests/X86 and tests/AArch64 are wrapped in if(REMILL_ENABLE_TESTING): they need GTest and
native asm and are not needed to build elflift.
NOT A REGRESSION ON THE NATIVE AArch64 PATH - AND THE FIRST ATTEMPT TO PROVE THAT WAS VACUOUS.
That run reported bitcode-identical while ninja had FAILED on the AttributeMask include: the
script piped ninja through grep, so the copy of the STALE image binary succeeded and was read
as build success. The lift then ran the unpatched binary, which guarantees an identical digest
by construction. A test that cannot fail is not a test, and this one reported green over a
broken build. It also nearly buried a real defect - the unguarded LLVM-17 include.
The harness now deletes the old binary first, reads ninja's exit code with NO pipe, aborts on
failure, and prints the fresh binary's digest. Re-run against LLVM 16:
NINJA_EXIT=0 - FRESH_BINARY_SHA=075374a7033e2a54 - detected funcs num 900 on both runs
bitcode sha256 ec2dc6f94af1ec7a0df56869ceff64f04577bbff6f01a9c6fadd6003cac5f8c4, IDENTICAL
STILL NOT BUILDING FOR WASM, and this commit does not claim otherwise. Remaining: port
Optimizer.cpp off PassManagerBuilder, rewrite the ConstantExpr::getSelect arm, and provide
wasm bfd/libdwarf/libelf - the last confined to lifter/Binary/Loader.{h,cpp} plus one
bfd_section_vma call at TraceManager.cpp:158. gflags, glog and XED were expected to be missing
and are NOT: all three cross-compile to wasm cleanly. XED needs --host-cpu=ia32, because the
64-bit path passes -m64 which emcc reads as wasm64 and the link then demands -mwasm64.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Lets the tree compile against LLVM 17 as well as 16, without pinning either.
backend/remill/CMakeLists.txtalready usesfind_package(LLVM CONFIG REQUIRED)and derivesthe version, so nothing was actually pinned — only the source had drifted. Two breaks:
llvm/ADT/Triple.h→llvm/TargetParser/Triple.h.Triplemoved out of ADT. The newpath exists in LLVM 16 too, so this is compatible in both directions rather than a bump.
llvm::AttributeMaskgot its own header in 17; before that it arrived with<llvm/IR/Attributes.h>.The second is version-guarded on purpose:
Adding that include unconditionally breaks LLVM 16 with
fatal error: 'llvm/IR/AttributeMask.h' file not found— which is exactly what happened herefirst, and is why the guard is in rather than a plain include.
Evidence
ninja test_dependencies && ctestagainst LLVM 16.0.6 on the:arm64image: 2/2 pass.LLVM 17 was exercised separately against a prebuilt wasm LLVM 17.0.6 while getting
elflifttoconfigure for WebAssembly; 19 of 25 translation units compile there, and the remaining breaks
are
PassManagerBuilder(removed in 17,lib/BC/Optimizer.cpp) andConstantExpr::getSelect(
lib/BC/Util.cpp:1362) — not addressed here, since both are real ports rather thaninclude moves and deserve their own change.
Scope note
Deliberately source-only. I carry two build-system changes downstream — trimming
llvm_map_components_to_libnamesto the components actually referenced, and gating the testsubdirectories behind
REMILL_ENABLE_TESTING— but both exist to serve a WebAssembly build ofthe lifter, not this repository, so they are left out. Glad to send either if useful.