Skip to content

lifter: read the ELF through LLVM instead of bfd, libdwarf and libelf - #265

Open
o2alexanderfedin wants to merge 2 commits into
yomaytk:mainfrom
o2alexanderfedin:pr/llvm-elf-loader
Open

lifter: read the ELF through LLVM instead of bfd, libdwarf and libelf#265
o2alexanderfedin wants to merge 2 commits into
yomaytk:mainfrom
o2alexanderfedin:pr/llvm-elf-loader

Conversation

@o2alexanderfedin

Copy link
Copy Markdown

What

Reads the ELF through llvm::object::ELFObjectFile and llvm::DWARFContext instead of
binutils, and fixes a function-sizing bug that bfd's API had forced.

elflift linked three binutils libraries to do what LLVM already does and the lifter
already links: bfd to open the binary and walk sections and symbols, libelf for the ehdr,
program headers and symbol sizes, and libdwarf to recover function entries from .eh_frame
when the binary is stripped. All three go, and libiberty goes with them — it was only ever
bfd's transitive requirement. No LLVM library is added: LLVMObject and
LLVMDebugInfoDWARF arrive with the LLVM that remill already links.

Two commits, deliberately separate:

1. The port — byte-identical, which is its whole warrant.
Verified against the stock lifter in the :arm64 image, both loader paths, bitcode and wasm:

subject path funcs bitcode wasm
static, not stripped .symtab 900 ec2dc6f9… b15e9a6f… (5 654 545 B)
static, stripped .eh_frame 899 883bbf48… 735ce583… (5 624 783 B)

The stripped arm matters: GetSblFromEhFrame — the entire libdwarf half — had never been
exercised by any check I could find, and LLVM's .eh_frame reader reproduces libdwarf exactly.

asection * becomes a small SectionRange (name/vma/size) owned by the loader. Containment
queries run over every section, unfiltered, because bfd's did — ELFSection is the wrong
type for that job since it drops sections based at 0x0. The LLVM objects sit behind a pimpl,
so no LLVM header reaches Loader.h, and TraceManager.h loses its <bfd.h>.

2. The sizing fix — this one changes output on purpose.
bfd's asymbol carries no size, so the loader re-read the ELF through libelf, built a
NAME → size map over .symtab and .dynsym, and joined on the name. That join is lossy.
On one static-glibc hello: 12 names collide and 15 function symbols get a length that is not
their own
. The worst is free_mem — eight distinct static functions, of which seven receive
100 bytes in place of their real 36, 40, 48, 56, 104, 184 or 276, because the map keeps
whichever came last. TraceManager prefers the symbol size over section arithmetic, so the
lifter disassembled the wrong number of bytes for those seven. LLVM exposes st_size on the
symbol already in hand, so the workaround dies with the abstraction that forced it.

It is a second commit precisely because the first one's claim is "nothing changed", and the two
cannot both be true in one patch.

Evidence

  • ninja test_dependencies && ctest: 2/2 pass, including the integration test that runs the
    lifted wasm under WasmEdge and matches Hello, World!.
  • ldd on the built elflift: no bfd, dwarf, elf or iberty.
  • Both artifacts also executed through an independent WASI runtime; each instantiates and runs
    _start to completion, and the pair reaches the same terminal state — different bitcode, same
    behaviour.

Notes

While writing this the first build was green, linked, emitted the right function count — and
produced wrong bitcode, because I admitted STT_GNU_IFUNC symbols on the strength of bfd's
source. The bfd in the build image does not flag them, so the original filter admitted none of
the binary's 10 ifuncs. Admitting them shifted every later symbol index by 10 and, through an
unstable sort over tied addresses, changed which alias named each lifted function — while the
count stayed identical. Worth knowing if anyone touches that filter: a count check does not see it.

Touches lifter/TraceManager.cpp, as does the amd64 front-end PR — if both are wanted, one
needs a trivial rebase on the other.

o2alexanderfedin and others added 2 commits August 7, 2026 18:49
The lifter linked three binutils libraries to do what LLVM already does: bfd to open the
binary and walk its sections and symbols, libelf to read the ehdr, the program headers and
symbol sizes, and libdwarf to recover function entries from `.eh_frame` when the binary is
stripped. All three are replaced by llvm::object::ELFObjectFile and llvm::DWARFContext,
which the lifter already links through remill. Three dependencies are deleted, not ported --
and libiberty goes with them, since it was only ever bfd's transitive requirement.

Verified byte-identical on both loader paths against the stock lifter shipped in the
:arm64 image, on a static-glibc aarch64 hello:

  not stripped (.symtab path)   900 funcs  bc ec2dc6f9...  wasm b15e9a6f... 5 654 545 B
  stripped     (.eh_frame path) 899 funcs  bc 883bbf48...  wasm 735ce583... 5 624 783 B

The stripped arm is new. The recorded harness only ever lifted a non-stripped binary, so
`GetSblFromEhFrame` -- the entire libdwarf half -- was never executed by it.

THE HARNESS EARNED ITS KEEP AND THE DEFECT IS WORTH RECORDING. The first build of this port
was green, linked, and emitted 900 functions, and its bitcode was WRONG: ec2dc6f9 became
1b89e054. Cause: STT_GNU_IFUNC. bfd in this binutils does not raise BSF_FUNCTION for it, so
the original filter admitted none of the binary's 10 ifunc symbols; I admitted them from
memory of bfd's source. That shifted every later symbol index by 10 and, because the sort is
by address and std::sort is NOT stable, re-scrambled which alias won at the 96 addresses
where glibc puts several names on one function. Dropping STT_GNU_IFUNC restored the digest
exactly. The count was never wrong -- 900 both times -- so a function-count check would have
passed the defect through.

`asection *` is replaced by `SectionRange *`, a name/vma/size triple owned by the loader.
Containment queries run over EVERY section, unfiltered, because bfd's did; `ELFSection` is
the wrong type for that job since it drops sections based at 0x0. TraceManager loses its
<bfd.h> include, and with the LLVM objects held behind a pimpl no LLVM header reaches
Loader.h either, so the header's third-party surface is now zero.

No LLVM component is named in CMakeLists on purpose: llvm_map_components_to_libnames resolves
to the STATIC archives, whose interface demands zstd::libzstd_shared, which is absent in the
build image and fails configuration outright.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Nu4maFxHPncwvdP8XcSEbD
Now that the loader reads the ELF through LLVM, a function's length comes off the symbol the
loader is already holding. bfd could not do that: its `asymbol` is format-neutral and carries
no size, so the loader re-read the binary through libelf, built a NAME -> size map over
`.symtab` and `.dynsym`, and joined on the name -- the only key bfd left it.

That join is lossy, and it is wrong on a hello-world. Measured on the static-glibc aarch64
subject the harness uses: 12 names collide, and 15 function symbols are handed a length that
is not their own. The worst is `free_mem` -- eight distinct static functions in eight
translation units, seven of which receive 100 bytes in place of their real 36, 40, 48, 56,
104, 184 or 276, because the map keeps whichever entry came last. TraceManager prefers the
symbol size over section arithmetic, so the lifter disassembled the wrong number of bytes for
those seven: some truncated, some over-read.

THIS CHANGES THE EMITTED BITCODE ON PURPOSE, which is why it is not folded into the
dependency removal it sits on top of. That commit's entire warrant is that it changed nothing
-- bitcode and wasm byte-identical on both loader paths -- and proving "three libraries
removed, behaviour unchanged" requires the change to be separable from any behaviour fix.

  not stripped (.symtab path)   ec2dc6f9... -> fba665d1...   900 funcs, unchanged
  stripped     (.eh_frame path) 883bbf48... unchanged        899 funcs, unchanged

The stripped arm does not move and cannot: `.eh_frame` carries its own address ranges and
never consulted the map.

STATED LIMIT: the new sizes are argued correct from the ELF, not demonstrated by execution.
The build image has no wasmedge -- `elfconv.sh` line 275 exits 127 on every lift, before and
after -- so no lifted artifact was run here. What is measured is that the sizes now come from
the symbol that owns them, and that nothing else in the pipeline moved.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Nu4maFxHPncwvdP8XcSEbD
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant