Compile fails with LLVM_ENABLE_RUNTIMES="libcxx;libcxxabi;libc;libunwind" on Mac M1

[billhoffman]

c++ -v
Apple clang version 16.0.0 (clang-1600.0.26.3)
Target: arm64-apple-darwin23.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

In file included from /Users/hoffman/Work/llvm/llvm-project/libunwind/src/AddressSpace.hpp:35:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.0.sdk/usr/include/dlfcn.h:36:10: fatal error: 'stdbool.h' file not found

cmake -DLLVM_ENABLE_PROJECTS=clang -DCMAKE_INSTALL_PREFIX=/Users/hoffman/Work/llvm-main-inst-mine -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_RUNTIMES="libcxx;libcxxabi;libc;libunwind" -G Ninja ../llvm

[asl]

Looks like the issue with Apple-installed SDKs as the error is:

/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.0.sdk/usr/include/dlfcn.h:36:10: fatal error: 'stdbool.h' file not found

Would be better to report to Apple

[billhoffman]

Bad example... :)

/Users/hoffman/Work/llvm/llvm-project/libunwind/src/Unwind-sjlj.c:16:10: fatal error: 'stdbool.h' file not found

Also, the apple sdk compiler can find stdbool.h. However, the bootstrapped clang is causing the trouble: 'llvm-project/build2/bin/clang'

It is some interaction with the system and the bootstrapped compiler. Because the same build tree that was working stopped working after a mac system update. Basically when building libunwind there are multiple instances of stdbool.h not being found. If I just have a hello world that includes stdbool.h the bootstrapped compiler can't find it. The system one can find it.

[billhoffman]

I think the problem might be that something moved on the mac. If I run the clang with -v you can see these warnings:

ignoring nonexistent directory "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.0.sdk/usr/local/include"
ignoring nonexistent directory "/Users/hoffman/Work/llvm/llvm-project/build2/lib/include"
ignoring nonexistent directory "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.0.sdk/Library/Frameworks"
#include "..." search starts here:
#include <...> search starts here:
 /Users/hoffman/Work/llvm/llvm-project/libunwind/include
 /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.0.sdk/usr/include
 /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.0.sdk/System/Library/Frameworks (framework directory)
End of search list.

If you look for stdbool.h in the SDK directory you see this:

find /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs -name stdbool.h
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/v1/stdbool.h
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks/Kernel.framework/Versions/A/Headers/stdbool.h

[billhoffman]

So, I am not seeing the problem in the 19.x branch only in origin/main. So, it is likely an issue with llvm and not the system I am building on.

[billhoffman]

In the 19.x clang compiler that is in the build tree I can compile a foo.c that has a single line #include <stdbool.h> This fails with the main branch clang.

Here is the broken one:

 "/Users/hoffman/Work/llvm/myllvm-project/build/bin/clang-20" -cc1 -triple arm64-apple-macosx14.0.0 -Wundef-prefix=TARGET_OS_ -Werror=undef-prefix -Wdeprecated-objc-isa-usage -Werror=deprecated-objc-isa-usage -emit-obj -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name foo.c -mrelocation-model pic -pic-level 2 -mframe-pointer=non-leaf -ffp-contract=on -fno-rounding-math -funwind-tables=1 -fcompatibility-qualified-id-block-type-checking -fvisibility-inlines-hidden-static-local-var -fbuiltin-headers-in-system-modules -fdefine-target-os-macros -target-cpu apple-m1 -target-feature +zcm -target-feature +zcz -target-feature +v8.4a -target-feature +aes -target-feature +altnzcv -target-feature +ccdp -target-feature +complxnum -target-feature +crc -target-feature +dotprod -target-feature +fp-armv8 -target-feature +fp16fml -target-feature +fptoint -target-feature +fullfp16 -target-feature +jsconv -target-feature +lse -target-feature +neon -target-feature +pauth -target-feature +perfmon -target-feature +predres -target-feature +ras -target-feature +rcpc -target-feature +rdm -target-feature +sb -target-feature +sha2 -target-feature +sha3 -target-feature +specrestrict -target-feature +ssbs -target-abi darwinpcs -debugger-tuning=lldb -fdebug-compilation-dir=/Users/hoffman/Work/llvm/t -target-linker-version 1115.7.3 -v -fcoverage-compilation-dir=/Users/hoffman/Work/llvm/t -resource-dir /Users/hoffman/Work/llvm/myllvm-project/build/lib -internal-isystem /usr/local/include -internal-isystem /Users/hoffman/Work/llvm/myllvm-project/build/lib/include -internal-externc-isystem /usr/include -ferror-limit 19 -stack-protector 1 -fblocks -fencode-extended-block-signature -fregister-global-dtors-with-atexit -fgnuc-version=4.2.1 -fskip-odr-check-in-gmf -fmax-type-align=16 -fcolor-diagnostics -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o foo.o -x c foo.c
clang -cc1 version 20.0.0git based upon LLVM 20.0.0git default target arm64-apple-darwin23.6.0
ignoring nonexistent directory "/usr/local/include"
ignoring nonexistent directory "/Users/hoffman/Work/llvm/myllvm-project/build/lib/include"
ignoring nonexistent directory "/usr/include"
#include "..." search starts here:
#include <...> search starts here:
 /System/Library/Frameworks (framework directory)
 /Library/Frameworks (framework directory)
End of search list.
foo.c:1:10: fatal error: 'stdbool.h' file not found
    1 | #include <stdbool.h>
      |          ^~~~~~~~~~~

Here is the same thing with the build tree clang from 19.x:

 /Users/hoffman/Work/llvm/llvm-19.x/build/./bin/clang -v -c foo.c
clang version 19.1.2 ([email protected]:billhoffman/llvm-project.git 6c1fd539e43ea3c3bf052495704403a76d4e7979)
Target: arm64-apple-darwin23.6.0
Thread model: posix
InstalledDir: /Users/hoffman/Work/llvm/llvm-19.x/build/bin
 (in-process)
 "/Users/hoffman/Work/llvm/llvm-19.x/build/bin/clang-19" -cc1 -triple arm64-apple-macosx14.0.0 -Wundef-prefix=TARGET_OS_ -Werror=undef-prefix -Wdeprecated-objc-isa-usage -Werror=deprecated-objc-isa-usage -emit-obj -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name foo.c -mrelocation-model pic -pic-level 2 -mframe-pointer=non-leaf -ffp-contract=on -fno-rounding-math -funwind-tables=1 -fcompatibility-qualified-id-block-type-checking -fvisibility-inlines-hidden-static-local-var -fbuiltin-headers-in-system-modules -fdefine-target-os-macros -target-cpu apple-m1 -target-feature +zcm -target-feature +zcz -target-feature +v8.4a -target-feature +aes -target-feature +altnzcv -target-feature +ccdp -target-feature +complxnum -target-feature +crc -target-feature +dotprod -target-feature +fp-armv8 -target-feature +fp16fml -target-feature +fptoint -target-feature +fullfp16 -target-feature +jsconv -target-feature +lse -target-feature +neon -target-feature +pauth -target-feature +perfmon -target-feature +predres -target-feature +ras -target-feature +rcpc -target-feature +rdm -target-feature +sb -target-feature +sha2 -target-feature +sha3 -target-feature +specrestrict -target-feature +ssbs -target-abi darwinpcs -debugger-tuning=lldb -fdebug-compilation-dir=/Users/hoffman/Work/llvm/t -target-linker-version 1115.7.3 -v -fcoverage-compilation-dir=/Users/hoffman/Work/llvm/t -resource-dir /Users/hoffman/Work/llvm/llvm-19.x/build/lib/clang/19 -internal-isystem /usr/local/include -internal-isystem /Users/hoffman/Work/llvm/llvm-19.x/build/lib/clang/19/include -internal-externc-isystem /usr/include -ferror-limit 19 -stack-protector 1 -fblocks -fencode-extended-block-signature -fregister-global-dtors-with-atexit -fgnuc-version=4.2.1 -fskip-odr-check-in-gmf -fmax-type-align=16 -fcolor-diagnostics -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o foo.o -x c foo.c
clang -cc1 version 19.1.2 based upon LLVM 19.1.2 default target arm64-apple-darwin23.6.0
ignoring nonexistent directory "/usr/local/include"
ignoring nonexistent directory "/usr/include"
#include "..." search starts here:
#include <...> search starts here:
 /Users/hoffman/Work/llvm/llvm-19.x/build/lib/clang/19/include
 /System/Library/Frameworks (framework directory)
 /Library/Frameworks (framework directory)
End of search list.

In addition, if I run ninja -t deps on the 19.x where things work you can see where it is getting stdbool.h

ninja -t deps | grep stdbool
    /Users/hoffman/Work/llvm/llvm-19.x/build/lib/clang/19/include/stdbool.h

In 19.x you can see there is a -internal-isystem /Users/hoffman/Work/llvm/llvm-19.x/build/lib/clang/19/include which is where stdbool.h is. However, in the broken one you can see -internal-isystem /Users/hoffman/Work/llvm/myllvm-project/build/lib/include Along with the warning ignoring nonexistent directory "/Users/hoffman/Work/llvm/myllvm-project/build/lib/include" So, it looks like the internal include is wrong. FWIW, the stdbool.h exists in the right place in the main one lib/clang/20/include/stdbool.h, it is just not being used as an include.

[ldionne]

So it seems like this has to do with Clang's search paths only, not libunwind.

Just from a quick search, this could potentially be 2b5cd8b, 938ddbf or 9616399.

CC @ian-twilightcoder @MaskRay

[ian-twilightcoder]

Which SDK are you using? I would expect to see this on macOS 15 SDKs prior to Beta 5, but not in Beta 5 or the released version.

[billhoffman]

I think this means I am running 15 SDK:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.sdk -> MacOSX.sdk
However, the issue seems to be that this /Users/hoffman/Work/llvm/llvm-19.x/build/lib/clang/20/include is showing up as /Users/hoffman/Work/llvm/myllvm-project/build/lib/include in the isystem-include for the clang built in the tree.

AFAIK I am not using anything Beta, just released Xcode.

clang --version
Apple clang version 16.0.0 (clang-1600.0.26.3)
Target: arm64-apple-darwin23.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

[billhoffman]

OK, this is all my fault... Sorry for the trouble here. In trying to fix this issue #109895 I made this patch

+++ b/clang/lib/Driver/Driver.cpp
@@ -193,8 +193,13 @@ std::string Driver::GetResourcesPath(StringRef BinaryPath) {
     P = llvm::sys::path::parent_path(Dir);
     // This search path is also created in the COFF driver of lld, so any
     // changes here also needs to happen in lld/COFF/Driver.cpp
-    llvm::sys::path::append(P, CLANG_INSTALL_LIBDIR_BASENAME, "clang",
-                            CLANG_VERSION_MAJOR_STRING);
+    llvm::sys::path::append(P, CLANG_INSTALL_LIBDIR_BASENAME);
+    // Darwin and AIX does not use per-target runtime directory.
+    if (!llvm::Triple(llvm::sys::getProcessTriple()).isOSDarwin() &&
+        !llvm::Triple(llvm::sys::getProcessTriple()).isOSAIX() ) {
+      llvm::sys::path::append(P, "clang",
+                              CLANG_VERSION_MAJOR_STRING);
+    }
   }

This issue can be closed, maybe the location of libc++.modules.json should be changed on the mac to match linux as changing this breaks other things... Again, sorry for the trouble, but if someone on here has a better way to fix the finding of libc++.modules.json on Mac I would appreciate it.