Enable BPF Type Format (BTF)CONFIG_DEBUG_INFO_BTF

[javierhonduco]

Describe the bug

BTF is necessary for running modern BPF applications, enabling it would allow for greater observability and debuggability of software running on Raspberry Pis. Without it, many BPF applications won't run.

Steps to reproduce the behaviour

Run any modern BPF application, such as lightswitch on any Raspberry Pi running Raspberry Pi OS. I only tested in on a Pi 4 though.

Device (s)

Raspberry Pi 4 Mod. B

System

(does not apply)

Logs

(does not apply)

Additional context

libbpf fails to load programs and logs:

kernel BTF is missing at /sys/kernel/btf/vmlinux, was CONFIG_DEBUG_INFO_BTF enabled?

[pelwell]

If you'd tried to build the kernel yourself you would have discovered that you can't just enable DEBUG_INFO_BTF - it depends on DEBUG_INFO being enabled for the whole kernel build, which it isn't. The kconfig file says:

	prompt "Debug information"
	depends on DEBUG_KERNEL
	help
	  Selecting something other than "None" results in a kernel image
	  that will include debugging info resulting in a larger kernel image.
	  This adds debug symbols to the kernel and modules (gcc -g), and
	  is needed if you intend to use kernel crashdump or binary object
	  tools like crash, kgdb, LKCD, gdb, etc on the kernel.

	  Choose which version of DWARF debug info to emit. If unsure,
	  select "Toolchain default".

config DEBUG_INFO_NONE
	bool "Disable debug information"
	help
	  Do not build the kernel with debugging information, which will
	  result in a faster and smaller build.

1. Just selecting DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y increased the 64-bit kernel size from 32,449,104 bytes to 279,913,152 bytes (yes, that's an order of magnitude larger).
2. Selecting DEBUG_INFO_REDUCED=y dropped that to 102,210,280 bytes, so only 3 times larger, but that's still enormous. Unfortunately, DEBUG_INFO_BTF is explicitly incompatible with DEBUG_INFO_REDUCED.
3. I tried building with DEBUG_INFO_BTF enabled, and it failed because pahole is not available, which seems to be a tool for generating the BTF type info.

I'll leave this issue open for comments, but I'm pretty sure the answer is going to be a no.

[javierhonduco]

Hi, yes, this was a rough proposal without going into the exact config changes to see if it would be something you would consider and to understand any historical context.

The specific flags I have experimented with:

CONFIG_DEBUG_INFO=y
CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y
CONFIG_DEBUG_INFO_COMPRESSED_ZLIB=y
CONFIG_DEBUG_INFO_BTF=y

Which in my box resulted in a ~5MB increase. This seems very low, even with compression, so perhaps I did something wrong. Another possibility would be to extract the debug info from the final Image. I think it would be very positive to do this and upload it to a debuginfod server to make Rasberry Pi OS debuggable and more amenable to profiling using DWARF info (rather than plain kallsyms which don't have line information). I guess that would be another issue. A quick Google search surfaced this question on the forums: https://forums.raspberrypi.com/viewtopic.php?t=359436.

To make these changes I've added the new flags to ./arch/arm64/configs/bcm2711_defconfig, and then ran:

$ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- bcm2711_defconfig
$ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- Image modules dtbs -j$(nproc --all)