CommandQueueMT: Reduce contention + Fix race conditions

[RandomShaper]

See #112452 for an explanation of why this would be beneficial.

TL;DR Keeping the lock held over the command queue of the rendering server in separate thread mode is not needed because there's a single thread dealing with such server, Therefore, the lock is only needed for thread safety of the command queue itself, which means the lock can be released while commands are run, allowing other threads to add commands without waiting.

Not tested at all due to lack of time...

UPDATE: I've marked this PR as cherry-pickable into some releases, but probably the only commit that should be cherry-picked is the first one (CommandQueueMT: Fix race conditions), which fixes a clear bug. The rest of the PR is a performance improvement, and as such it's better scheduled only for the current dev branch.

[brycehutchings]

I did some basic testing of this fix using the Sponza scene (https://github.com/Calinou/godot-sponza). I confirmed it fixes the performance problem so that the performance loading a very complex glTF in a background thread takes the same time regardless of if using the Separate or Safe thread model. I also compared it to the performance without the fix for loading with Safe thread model and it was comparable, so I didn't see a degradation due to the memcpy.

[RandomShaper]

@brycehutchings Thanks a lot for testing and reviewing this.

[RandomShaper]

Rebased.

[dsnopek]

I noticed the issue with the lock being held while the RenderingServer is flushing the queue (and stalling everything that's queuing stuff for the next frame) while looking at Perfetto traces on Samsung Galaxy XR.

This PR seems to solve that problem entirely!

Here's a trace from master running my fork of the GDQuest TPS demo:

The set_render_display_info is the first thing on this frame that is attempting to queue a Callable for the rendering server, and it ends up stalled there until the render thread finishes rendering the previous frame. You can see all the process stuff waiting until the end.

(NOTE: this won't show up in traces of this project on Meta Quest 3, because Meta's runtime uses xrWaitFrame() to delay running process until close to the end of the frame anyway to improve input latency. However, it's still possible to trigger the issue there too, just not in this project and not as drastically.)

And here is a trace from this PR:

Notice that the process stuff happens super early now and overlaps the rendering, and it's only the RenderingServer::sync() that stalls until the rendering server is done, which is exactly what we would expect.

(NOTE: this trace is actually worse for input latency, but I think that's really a problem with xrWaitFrame() on Samsung Galaxy XR. Inadvertent lock contention on Godot's RenderingServer shouldn't be used as the solution for that :-))

[dsnopek]

The code looks good to me - after I noticed the issue in Perfetto, but before I found this PR, I was thinking about trying to implement this same change (ie copying the queue in _flush()). However, I'm not all that familiar with this code, so I'm glad that @RandomShaper did it :-)

I can say that the result seems correct from my testing!

[akien-mga]

Based on Bryce and David's review and testing, and Pedro's familiarity with this code, I think we can safely¹ merge this.

Footnotes

1. TIWAGOS, this code is touchy and regression prone. Keep it in mind when reviewing potential regression reports during the 4.6 dev/beta phase. ↩

[Repiteo]

Thanks!