macOS debuginfo: add support for using atos to get file:line info in backtraces.

[JohanEngelen]

Resolves issue #3864

I thought this requires running dsymutil (see #4006), but apparently it does not.

Tested on macOS Monterey 12.6, Apple M1, with the following:

import std.exception;

void a() { b(); }

void b() { c(); }

void c() { d(); }

void d() { 
    enforce(false);
}

void main()
{
    a();
}

> ~/ldc/build14/bin/ldc2 -g crashtest.d && ./crashtest
[email protected](10): Enforcement failed
----------------
/Users/johan/ldc/ldc/runtime/phobos/std/exception.d:519 pure @safe noreturn std.exception.bailOut!(Exception).bailOut(immutable(char)[], ulong, scope const(char)[]) [0x10042dcdb]
/Users/johan/ldc/ldc/runtime/phobos/std/exception.d:439 pure @safe bool std.exception.enforce!().enforce!(bool).enforce(bool, lazy const(char)[], immutable(char)[], ulong) [0x10042db8b]
/Users/johan/ldcweka/llvmsymbolizer/crashtest.d:10 void crashtest.d() [0x10042dafb]
/Users/johan/ldcweka/llvmsymbolizer/crashtest.d:7 void crashtest.c() [0x10042daab]
/Users/johan/ldcweka/llvmsymbolizer/crashtest.d:5 void crashtest.b() [0x10042da97]
/Users/johan/ldcweka/llvmsymbolizer/crashtest.d:3 void crashtest.a() [0x10042da83]
/Users/johan/ldcweka/llvmsymbolizer/crashtest.d:15 _Dmain [0x10042dbc7]

[JohanEngelen]

@schveiguy @Geod24 @ljmf00

[JohanEngelen]

@p0nce

[JohanEngelen]

Note that this fixes debuginfo for vanilla LLVM on macOS. In other words, we no longer need the debuginfo hack for macOS, so -g linking will no longer give warnings/errors while still having file:line information in backtraces.

[schveiguy]

Where can I grab a binary for this on ARM so I can test?

[JohanEngelen]

Where can I grab a binary for this on ARM so I can test?

@schveiguy Got any ideas of where I can upload the tarball?

@schveiguy using Wetransfer.com: https://we.tl/t-kibu0jpfkI
Thanks for testing.

[schveiguy]

I see line numbers now! But I'm trying to reproduce the link thing with the release 1.30.0 compiler, and it isn't failing... so I'm not sure how to really test that...

[JohanEngelen]

I see line numbers now! But I'm trying to reproduce the link thing with the release 1.30.0 compiler, and it isn't failing... so I'm not sure how to really test that...

maybe you have the MACOSX_DEPLOYMENT_TARGET setting permanently in your environment.

[schveiguy]

I made sure to unset that variable before compiling. And I did test with 1.28.0 and it failed.

[ljmf00]

@ljmf00

I don't have a mac nor use macOS so I can't test this, but perhaps, if this patch fixes the issue, it can be upstreamed?

[JohanEngelen]

I see line numbers now! But I'm trying to reproduce the link thing with the release 1.30.0 compiler, and it isn't failing... so I'm not sure how to really test that...

Hm, for me, the 1.30.0 release from Github does error when linking:

❯ ./install.sh install ldc-1.30.0 -p ~/dcompilers
❯ ~/dcompilers/ldc-1.30.0/bin/ldc2 -g crashtest.d && ./crashtest
ld: warning: alignment (1) of atom 'anon' is too small and may result in unaligned pointers
ld: warning: pointer not aligned at address 0x1002880FF ('anon' + 255 from crashtest.o)
ld: unaligned pointer(s) for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Error: /usr/bin/cc failed with status: 1

[JohanEngelen]

@jacob-carlborg I think I did what you suggested :)

[JohanEngelen]

@kinke With this merged, we can remove the hack from our LLVM branch, right?

[JohanEngelen]

@kinke With this merged, we can remove the hack from our LLVM branch, right?

I've added a commit that sets the LDC-specific LLVM option to vanilla LLVM's default behavior.

[jacob-carlborg]

@JohanEngelen I would prefer to not be dependent on an external tool, atos.

Note, with this change a .dSYM file or object file is required. For development this should be fine. For this functionality to work in production, it will require distributing both a .dSYM file and the atos binary. I don't know what license atos has, if it's allowed to be distributed.

[JohanEngelen]

@JohanEngelen I would prefer to not be dependent on an external tool, atos.

Note, with this change a .dSYM file or object file is required. For development this should be fine. For this functionality to work in production, it will require distributing both a .dSYM file and the atos binary. I don't know what license atos has, if it's allowed to be distributed.

Preferences aside, this is apparently the standard workflow on macOS for C/C++ and other languages. We should at least enable that workflow before considering to extend it. I don't know of a way to make it work other than dSYM or object files (including debug info gives link errors), without changing macOS's system linker or defining our own debuginfo format. It fortunately does not require running dsymutil on executables, although clang does run it for every -g link.

I use atos (like other C tooling) because we do not have the resources to reimplement that functionality and to maintain it (note the very long time this bug has persisted). I don't see strong arguments to warrant that effort for druntime.

Note: You can still get the old behavior (using compile flag with LDC built with the hacked LLVM version), and then the user can deal with the linker errors himself.

[p0nce]

For this functionality to work in production, it will require distributing both a .dSYM file and the atos binary.

Will a shared library made with LDC load if there is none of that? (and no debug info either). druntime is never initialized.

[schveiguy]

Hm, for me, the 1.30.0 release from Github does error when linking

I haven't updated to MacOS 13, have you?

[JohanEngelen]

For this functionality to work in production, it will require distributing both a .dSYM file and the atos binary.

Will a shared library made with LDC load if there is none of that? (and no debug info either). druntime is never initialized.

You don't need any of that to run a program. Jacob means that you need dSYM or object files if you want file:line information. If dSYM files or object files are not available, then you get the same output as now: no file:line information upon exception backtrace.

[JohanEngelen]

Hm, for me, the 1.30.0 release from Github does error when linking

I haven't updated to MacOS 13, have you?

I'm still on Monterey 12.6

[JohanEngelen]

Will be merging in the next days if no objections.