Using a release build of a GDExtension (godot-cpp) with a debug build of Godot will lead to heap corruption

[dsnopek]

Godot version

4.4.1-stable

godot-cpp version

4.4.1-stable

System information

Godot v4.4.1.stable - Pop!_OS 22.04 LTS on X11 - X11 display driver, Single-window, 3 monitors - OpenGL 3 (Compatibility) - NVIDIA GeForce RTX 4070 Ti - AMD Ryzen 9 7950X 16-Core Processor (32 threads)

Issue description

Using debug build of a GDExtension (specifically created with godot-cpp) with a release build of Godot will lead to heap corruption and may or may not lead to a crash or unexpected behavior

This is something that (as far as I can tell) has always been the case for all of Godot 4, and we generally tell folks to always pair debug GDExtensions with debug Godot, and release GDExtensions with release Godot.

However, this is a tricky support issue because it may cause a crash, or... it might not! The issues that folks find are intermittent and frequently different from run to run.

So, I think it would be good to either detect the situation and error out early, OR fix it so that this heap corruption doesn't happen (and I have an idea for how to do that :-))

The root cause of this is in Memory::alloc_static():

godot-cpp/src/core/memory.cpp

Lines 37 to 42 in 8bb931c

	void *Memory::alloc_static(size_t p_bytes, bool p_pad_align) {
	#ifdef DEBUG_ENABLED
	bool prepad = false; // Already pre paded in the engine.
	#else
	bool prepad = p_pad_align;
	#endif

It's making the assumption that a debug build of the GDExtension is used with a debug build of Godot, and a debug build of Godot will add the necessary padding automatically, and so it doesn't need to.

But since a release build of Godot won't add the padding, other functions that depend on it (like memnew_arr()) will stomp over unrelated memory, corrupting it.

Here are some related issues and PRs:

• Possible heap corruption issue #842
• Fix incorrect memory allocation in release builds. #1053
• Valgrind: Invalid read of size 4 (pthread_mutex_lock.c:80) Zylann/godot_voxel#733

Steps to reproduce

1. Install any GDExtension that has both a release and debug build
2. Copy the debug build over the release build
3. Make a release export
4. Run the exported project: it may crash or it may not, you might need to run it multiple times to see a crash. As an alternative, you can run it through valgrind and it'll point out the heap corruption to you :-)

Minimal reproduction project

n/a

[dsnopek]

My proposed solution is to add new memory allocation functions to the GDExtension interface which allow us to explicitly request padding or not - see PR godotengine/godot#108725

Then debug builds of GDExtensions can always request the necessary padding - see PR #1821

And then there shouldn't be any more heap corruption :-)

But that will all be for Godot 4.6+

Do we want to try to "fix" this for earlier versions? We could do that by always adding the padding, but that will mean that we allocate double padding when using a debug build of the extension with a debug build of Godot. Would that be worth it?