[LLD] Linker Crash Report

[aayush76541]

Facing a linker crash when cross compiling a C/C++ based internal kernel module.

Linker Command being used: ld.lld -r --whole-archive {input_files} -o {output_file}

Crash backtrace:

Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var `LLVM_SYMBOLIZER_PATH` to point to it):
0  ld.lld    0x0000000000951fed
1  libc.so.6 0x00007f64fa23e730
2  ld.lld    0x0000000000b378cb
3  ld.lld    0x0000000000b37e53 lld::elf::LinkerScript::addOrphanSections() + 771
4  ld.lld    0x0000000000a993e5 void lld::elf::LinkerDriver::link<llvm::object::ELFType<(llvm::endianness)1, true>>(llvm::opt::InputArgList&) + 3125
5  ld.lld    0x0000000000aa0807 lld::elf::LinkerDriver::linkerMain(llvm::ArrayRef<char const*>) + 6743
6  ld.lld    0x0000000000aa2108 lld::elf::link(llvm::ArrayRef<char const*>, llvm::raw_ostream&, llvm::raw_ostream&, bool, bool) + 5784
7  ld.lld    0x000000000098536d lld::unsafeLldMain(llvm::ArrayRef<char const*>, llvm::raw_ostream&, llvm::raw_ostream&, llvm::ArrayRef<lld::DriverDef>, bool) + 1053
8  ld.lld    0x00000000008cf96f lld_main(int, char**, llvm::ToolContext const&) + 623
9  ld.lld    0x0000000000801612 main + 114
10 libc.so.6 0x00007f64fa2295d0
11 libc.so.6 0x00007f64fa229680 __libc_start_main + 128
12 ld.lld    0x00000000008cf237

/bin/bash: line 3:     3 Segmentation fault      (core dumped) ld.lld -r --whole-archive {input_files} -o {output}
LLD 19.1.0 (compatible with GNU linkers)

Environment Information:
Clang: 19.1.0 (for linking)
GCC: 12.1.0 (for compilation)

I found an open bug related to the handling of relocated sections when executing addOrphanSections, but the issue does not provide any logs or crash info: #156417. Hence, not sure if it's the same issue.

[llvmbot]

@llvm/issue-subscribers-lld-elf

Author: None (aayush76541)

Facing a linker crash when cross compiling a C/C++ based internal kernel module.

Linker Command being used: ld.lld -r --whole-archive {input_files} -o {output_file}

Crash backtrace:

Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var `LLVM_SYMBOLIZER_PATH` to point to it):
0  ld.lld    0x0000000000951fed
1  libc.so.6 0x00007f64fa23e730
2  ld.lld    0x0000000000b378cb
3  ld.lld    0x0000000000b37e53 lld::elf::LinkerScript::addOrphanSections() + 771
4  ld.lld    0x0000000000a993e5 void lld::elf::LinkerDriver::link<llvm::object::ELFType<(llvm::endianness)1, true>>(llvm::opt::InputArgList&) + 3125
5  ld.lld    0x0000000000aa0807 lld::elf::LinkerDriver::linkerMain(llvm::ArrayRef<char const*>) + 6743
6  ld.lld    0x0000000000aa2108 lld::elf::link(llvm::ArrayRef<char const*>, llvm::raw_ostream&, llvm::raw_ostream&, bool, bool) + 5784
7  ld.lld    0x000000000098536d lld::unsafeLldMain(llvm::ArrayRef<char const*>, llvm::raw_ostream&, llvm::raw_ostream&, llvm::ArrayRef<lld::DriverDef>, bool) + 1053
8  ld.lld    0x00000000008cf96f lld_main(int, char**, llvm::ToolContext const&) + 623
9  ld.lld    0x0000000000801612 main + 114
10 libc.so.6 0x00007f64fa2295d0
11 libc.so.6 0x00007f64fa229680 __libc_start_main + 128
12 ld.lld    0x00000000008cf237

/bin/bash: line 3:     3 Segmentation fault      (core dumped) ld.lld -r --whole-archive {input_files} -o {output}
LLD 19.1.0 (compatible with GNU linkers)

Environment Information:
Clang: 19.1.0 (for linking)
GCC: 12.1.0 (for compilation)

I found an open bug related to the handling of relocated sections when executing addOrphanSections, but the issue does not provide any logs or crash info: #156417. Hence, not sure if it's the same issue.

[EugeneZelenko]

Could you please try 21 or main branch?

[smithp35]

I found an open bug related to the handling of relocated sections when executing addOrphanSections, but the issue does not provide any logs or crash info: #156417. Hence, not sure if it's the same issue.

If you want to check that, you should be able to do llvm-readelf --sections on your input files #156417 required a section with a relocation section that appears before its target section (info link). If all of your sections are of the form:

  [Nr] Name              Type            Address          Off    Size   ES Flg Lk Inf Al
  [ 3] .text.realloc     PROGBITS        0000000000000000 000040 000538 00  AX  0   0  4
  [ 4] .rela.text.realloc RELA           0000000000000000 0006b0 0002a0 18   I  7   3  8

Where the Inf of the relocation section is lower than the [Nr] then #156417 doesn't apply.

If you are able to rebuild lld from source then there is a PR for #156417 #156354

If it is different then a reproducer will be useful, if you can add --reproduce=<filename.tar> and upload it then that would help.

[aayush76541]

We've finished our own debugging (reproducer.tar.zip attached). The fix here does stop ld.lld from crashing, though it doesn't make linking work.

g++ -c -fpatchable-function-entry=5 foo.cpp -o foo.o
g++ -c -fpatchable-function-entry=5 bar.cpp -o bar.o
ld.lld -r --whole-archive -static foo.o bar.o -o combined.a
… where foo.cpp and bar.cpp each use a class with an inline member function.

ld.lld crashed because gcc <=12 did NOT place the sections generated by -fpatchable-function-entry in the group they described. When such a COMDAT group is discarded (because lld already has one in the appropriate COMDAT set), the "{rela.}__patchable_function_entries" sections were left unexpectedly "dangling".

More precisely:
1. We start with a class that has an inline member function
2. Two nearly identical c++ files are compiled with -fpatchable-function-entry=5 into .o's using gcc12. The relevant sections for both .o's look like this:

  [ 1] .group
       GROUP            0000000000000000  0000000000000040  14
       0000000000000008 0000000000000004  4                 4
       [0000000000000000]:
  [ 6] .text._ZN5Class6memberEj
       PROGBITS         0000000000000000  000000000000006c  0
       0000000000000015 0000000000000000  0                 2
       [0000000000000206]: ALLOC, EXEC, GROUP
  [ 7] __patchable_function_entries
       PROGBITS         0000000000000000  0000000000000088  6
       0000000000000010 0000000000000000  0                 8
       [0000000000000083]: WRITE, ALLOC, LINK ORDER
  [ 8] .rela__patchable_function_entries
       RELA             0000000000000000  0000000000000208  14
       0000000000000030 0000000000000018  7                 8
       [0000000000000040]: INFO LINK
COMDAT group section [    1] `.group' [_ZN5Class6memberEj] contains 1 sections:
   [Index]    Name
   [    6]   .text._ZN5Class6memberEj

Section 1 is a COMDAT group containing _just_ section 6, that section describing the inline function. Section 8 references Section 7 which references Section 6.

3. When lld/ELF/InputFiles.cpp's ObjFile::parse() ran

• The first .o adds its section 1's group to the COMDAT set
• The second .o finds _ZN5Class6memberEj is already present in the COMDAT set, so it marks its section 1 as discarded, along with all dependent sections listed in section 1's data (here just section 6)

4. When lld/ELF/LinkerScript.cpp's addOrphanSections() runs

• The sections for the first .o are processed just fine
• The sections for the second .o run into problems:
  § Section 7 is skipped because SHF_LINK_ORDER is set
  § Section 8 gets added because -r is passed
  § In addInputSec, getRelocatedSection returns section 7 but out is nullptr. We deref that and die. out was nullptr because Section 1&6 were discarded.
• With the latest fix, ld.lld fails more gracefully though somewhat opaquely: ld.lld: error: bar.o:(__patchable_function_entries): sh_link points to discarded section :()

5. GCC >= 13 and Clang place the {.rela}__patchable_function_entries sections in the COMDAT group. This ensures they are all discarded together:

    COMDAT group section [    1] `.group' [_ZN5Class6memberEj] contains 3 sections:
       [Index]    Name
       [    6]   .text._ZN5Class6memberEj
       [    7]   __patchable_function_entries
       [    8]   .rela__patchable_function_entries
   

The relevant GCC bug is here:
105169 – Compiling C++ code with -fpatchable-function-entry=16,14 results in references to discarded sections

So the open question now is whether lld should handle -fpatchable-function-entry's and COMDATs for GCC <= 12 instead of erroring out. In other words, should ld.lld gracefully handle sections named "{.rela}__patchable_function_entries" that happen to point to discarded sections, treating them as discarded as well? It probably depends on how many people are hitting this. Our corp has the option of backporting the gcc fix from the future back to our gcc 12 branch.