Multiple definitions of `__sync_fetch_and_add_4` et al.

[emgre]

Similar to #412, on armv7-unknown-linux-gnueabihf we get symbols __sync_fetch_and_add_4 and __sync_fetch_and_add_8 (and probably others) clashing when a C/C++ tries to link with our library. We've tried on arm-unknown-linux-gnueabihf, aarch64-unknown-linux-gnu and x86_64-unknown-linux-gnu and it doesn't seem to generate these symbols.

On Zulip, Alex Crichton suggested to flag the symbol as weak. This issue is off my level of competence, so I'll probably leave it to someone who knows better than me.

Some links that I've found while investigating this (for cross-reference):

• https://www.mail-archive.com/debian-arm@lists.debian.org/msg21551.html
• Keep codegen units unmerged when building compiler builtins rust#70846 (comment)
• Multiple definitions of atomic builtins on armv5te-unknown-linux-gnu rust#48625
• Add atomic support for pre-ARMv6 on Linux #115

[Amanieu]

The solution is the same as #412: each intrinsic needs to be defined in its own mod, so that rustc can put each function in a separate object file. This will fix the linker errors.

[Amanieu]

Hopefully this should be fixed by #452 which will be included in rust-lang/rust#93696. Please try it in the next nightly once that is merged.

[joelriendeau]

Hi @Amanieu, I'm encountering the same issue on the nightly (today's version) for armv5te-unknown-linux-musleabi.

/opt/arm-buildroot-linux-musleabi_sdk-buildroot/bin/../lib/gcc/arm-buildroot-linux-musleabi/11.2.0/../../../../arm-buildroot-linux-musleabi/bin/ld: /opt/arm-buildroot-linux-musleabi_sdk-buildroot/bin/../lib/gcc/arm-buildroot-linux-musleabi/11.2.0/libgcc.a(linux-atomic.o): in function __sync_fetch_and_add_4': /root/wsl/sfl_buildroot/buildroot/output/build/host-gcc-final-11.2.0/build/arm-buildroot-linux-musleabi/libgcc/../../../libgcc/config/arm/linux-atomic.c:116: multiple definition of __sync_fetch_and_add_4'; ../../../modules/leodream/leodream/armv5te-unknown-linux-musleabi/release/libleodream.a(compiler_builtins-ed49ad36634beb21.compiler_builtins.22da61b5-cgu.66.rcgu.o):/cargo/registry/src/github.com-1ecc6299db9ec823/compiler_builtins-0.1.70/src/macros.rs:360: first defined here
/opt/arm-buildroot-linux-musleabi_sdk-buildroot/bin/../lib/gcc/arm-buildroot-linux-musleabi/11.2.0/../../../../arm-buildroot-linux-musleabi/bin/ld: /opt/arm-buildroot-linux-musleabi_sdk-buildroot/bin/../lib/gcc/arm-buildroot-linux-musleabi/11.2.0/libgcc.a(linux-atomic.o): in function __sync_synchronize': /root/wsl/sfl_buildroot/buildroot/output/build/host-gcc-final-11.2.0/build/arm-buildroot-linux-musleabi/libgcc/../../../libgcc/config/arm/linux-atomic.c:277: multiple definition of __sync_synchronize'; ../../../modules/leodream/leodream/armv5te-unknown-linux-musleabi/release/libleodream.a(compiler_builtins-ed49ad36634beb21.compiler_builtins.22da61b5-cgu.201.rcgu.o):/cargo/registry/src/github.com-1ecc6299db9ec823/compiler_builtins-0.1.70/src/macros.rs:360: first defined her

Is there anything I can do?

[snowp]

@Amanieu Hitting this issue on x86_64-apple-ios and moving to Rust 1.60.0 (which I believe includes the proposed fix) does not seem to fix it.

  | duplicate symbol '___extendhfsf2' in:
  | external/Envoy/Envoy.framework/Envoy(extendhfsf2.o)
  | /Applications/Xcode-13.2.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/13.0.0/lib/darwin/libclang_rt.iossim.a(extendhfsf2.c.o)
  | duplicate symbol '___mulodi4' in:
  | external/Envoy/Envoy.framework/Envoy(compiler_builtins-da0162959ba64ab6.compiler_builtins.f13b4771-cgu.77.rcgu.o)
  | /Applications/Xcode-13.2.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/13.0.0/lib/darwin/libclang_rt.iossim.a(mulodi4.c.o)
  | duplicate symbol '___mulosi4' in:

What is the general advice here? Should we weaken the compiler_builtins symbols in favor of the libclang_rt ones? Are these symbols considered compatible enough that Rust code depending on the compiler builtin ones can safely use the clang ones instead?

[bjorn3]

Should we weaken the compiler_builtins symbols in favor of the libclang_rt ones?

We should already put all compiler_builtins symbols in separate object files and the linker should only pull in object files from archives if at least one symbol in them is referenced. Together this should mean that only either the object file providing this from compiler_builtins or libclang_rt should be pulled in and not both. I'm not sure where it goes wrong though that would cause this issue. If I locally try to list which object files in libcompiler_builtins.rlib provide which symbols it looks like ___mulodi4 and ___mulosi4 are provided by separate object files that don't provide any other symbols which is indeed how it should be.

Are these symbols considered compatible enough that Rust code depending on the compiler builtin ones can safely use the clang ones instead?

Yes, in fact they should be identical between clang and gcc even.

[keith]

I'm not sure where it goes wrong though that would cause this issue.

The issue in this case is the link is using -all_load to force all objects from all archives to be loaded. In our project this saves roughly 10% of link time, so it's a build performance optimization for debug builds. I imagine if the symbols could be weak it would actually remove the need for the separate object files, but I'm not sure?

[bjorn3]

The issue in this case is the link is using -all_load to force all objects from all archives to be loaded.

I see. compiler_builtins indeed needs to be linked without -force_load (which -all_load implies for all archives) to prevent this kind of symbol conflicts. Would it be possible to explicitly use -force_load just for the non-rust libraries? Or is the rust part of your project large enough that that would negate all benefits of -all_load?

I imagine if the symbols could be weak it would actually remove the need for the separate object files, but I'm not sure?

Support for weak symbols is rather spotty across platforms I believe and the exact behavior differs between platforms.

[keith]

Ideally that would be possible but practically in the build system it would be difficult to add that for every single one of the transitive libraries, which is why we take the -all_load approach instead. I do wish I could do -all_load -no_force_load compiler_builtins.rlib as well.

Support for weak symbols is rather spotty across platforms I believe and the exact behavior differs between platforms.

Ah yea fair enough. Do you see any other downsides to doing weak symbols as well as the current solution? That would satisfy this use case for macho at least (although maybe it only apply to the one format is enough to not do it to not cause confusion)

[bjorn3]

Do you see any other downsides to doing weak symbols as well as the current solution?

Not sure.

[hgy59]

facing this issue too, when building fd-find (https://github.com/sharkdp/fd/) version 8.4.0 for armv5te-unknown-linux-gnueabi.

error: linking with `/spksrc/toolchain/syno-88f6281-6.1/work/arm-marvell-linux-gnueabi/bin/arm-marvell-linux-gnueabi-gcc` failed: exit status: 1
  |
  = note: "/spksrc/toolchain/syno-88f6281-6.1/work/arm-marvell-linux-gnueabi/bin/arm-marvell-linux-gnueabi-gcc" "/tmp/rustcRs09iU/symbols.o" "/spksrc/cross/fd/work-88f6281-6.1/fd-8.4.0/target/armv5te-unknown-linux-gnueabi/release/deps/fd-6304219af5693be7.fd.159f5a60-cgu.0.rcgu.o" "-Wl,--as-needed" "-L" "/spksrc/cross/fd/work-88f6281-6.1/fd-8.4.0/target/armv5te-unknown-linux-gnueabi/release/deps" "-L" "/spksrc/cross/fd/work-88f6281-6.1/fd-8.4.0/target/release/deps" "-L" "/spksrc/cross/fd/work-88f6281-6.1/fd-8.4.0/target/armv5te-unknown-linux-gnueabi/release/build/jemalloc-sys-f50f4f34961a302a/out/build/lib" "-L" "/home/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/armv5te-unknown-linux-gnueabi/lib" "-Wl,-Bstatic" "/tmp/rustcRs09iU/libjemalloc_sys-cbdb1a14ad051ce5.rlib" "-Wl,--start-group" "-Wl,--end-group" "/home/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/armv5te-unknown-linux-gnueabi/lib/libcompiler_builtins-74981b485bdced9d.rlib" "-Wl,-Bdynamic" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-znoexecstack" "-L" "/home/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/armv5te-unknown-linux-gnueabi/lib" "-o" "/spksrc/cross/fd/work-88f6281-6.1/fd-8.4.0/target/armv5te-unknown-linux-gnueabi/release/deps/fd-6304219af5693be7" "-Wl,--gc-sections" "-pie" "-Wl,-zrelro,-znow" "-Wl,-O1" "-nodefaultlibs"
  = note: /spksrc/toolchain/syno-88f6281-6.1/work/arm-marvell-linux-gnueabi/bin/../lib/gcc/arm-marvell-linux-gnueabi/4.6.4/libgcc.a(linux-atomic.o): In function `__sync_fetch_and_add_4':
          linux-atomic.c:(.text+0x0): multiple definition of `__sync_fetch_and_add_4'
          /home/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/armv5te-unknown-linux-gnueabi/lib/libcompiler_builtins-74981b485bdced9d.rlib(compiler_builtins-74981b485bdced9d.compiler_builtins.28be1565-cgu.211.rcgu.o):/cargo/registry/src/github.com-1ecc6299db9ec823/compiler_builtins-0.1.73/src/macros.rs:418: first defined here
          /spksrc/toolchain/syno-88f6281-6.1/work/arm-marvell-linux-gnueabi/bin/../lib/gcc/arm-marvell-linux-gnueabi/4.6.4/libgcc.a(linux-atomic.o): In function `__sync_fetch_and_sub_4':
          linux-atomic.c:(.text+0x3c): multiple definition of `__sync_fetch_and_sub_4'
...

[AdrianBunk]

Support for weak symbols is rather spotty across platforms I believe and the exact behavior differs between platforms.

The file is arm_linux.rs (and the weak attribute is already used in src/arm.rs).

Linux on ARM is a pretty limited scope, and there are other widespread usages like glibc used to provide weak versions of pthread symbols in libc.so for decades (that changed recently for unrelated reasons when libpthread was merged into libc).