Support overriding the number of channels in the AudioDriverPulseAudio buffer

[goatchurchprime]

Replaced by: #105682

This PR adds the option audio/driver/override_channels to force Godot to create up to 8 channels internally even when your output hardware has only the stereo two.

Without this feature it is impossible to recreate and debug issues such as goatchurchprime/two-voip-godot-4#44 that I have had reports of (though lacking the hardware, I have never encountered it).

Not many developers know about this hidden feature where Godot generates up to 4 AudioEffectInstances for each AudioEffect (2 channels per instance) if you happen to plug a particular type of headphones into your USB slot. This is why it is not surprising there are bugs in some of the AudioEffects such as PR #92532 that aren't getting merged since no one can test them.

With this feature it will be possible to force Godot to create the extra channels that you do not have in order to recreate this situation.

The reason this PR is a small change is that Godot already handles the case of a mismatches between the number of internal channels and real channels when the number of channels is odd since each AudioEffectInstance can only take 2 channels.

With a bit of encouragement (and access to the hardware) I might be able to extend this feature to override the number of channels from 8 down to 2, which could fix a lot of issues for people who have no time to develop for surround sound systems. I honestly have no idea what this 8 channel feature is supposed to be for, since we can't even import surround sound WAV files anyway (pick an example here: https://drive.google.com/drive/folders/1JxmeedtAtgmoafXv9rroiDOS2vEX7N4b ). It is most unhelpful to find your game is doing something completely unexpected just because someone plugged in a weird set of gaming headphones, so it could be pretty vital to be able to turn this channel multiplicity feature off if you're not doing any work to make use of the surround sound, since it is more than likely to result in bugs than an improvement in quality.

[Calinou]

That reminds me, we should make the VU meter bars thinner when there are many of them so that each column doesn't become too wide. Additionally, we should label what each bar is for (e.g. left/right channel), since there are much more channels to keep track of in a 5.1 or 7.1 setup.

I honestly have no idea what this 8 channel feature is supposed to be for, since we can't even import surround sound WAV files anyway (pick an example here: drive.google.com/drive/folders/1JxmeedtAtgmoafXv9rroiDOS2vEX7N4b ).

Surround sound in games is mostly used for better spatialization (AudioStreamPlayer3D). It can give you a better impression of a sound being located in front of/behind you than stereo with a low-pass filter applied when the sound is behind you (HRTF). Surround audio data is typically not used for sound effects themselves (positional or otherwise) or background music – these remain stereo, or even mono.

[Calinou]

Should mono configurations also be allowed? I'm not sure if you can still get a mono output on a PC intentionally in 2025. (For audio input, you can, e.g. if using a Bluetooth headset + microphone.)

Also, do configurations with 5 or 7 channels make sense? To my knowledge, this is what exists:

• Mono, 1 channel
• Stereo (2.0), 2 channels
• 2.1, 3 channel. Seems to be abstracted away from the software, e.g. a soundbar with a subwoofer besides it. In practice, the software sees a 2.0 setup.
• 5.1, 6 channels
• 7.1, 8 channels

If 5 and 7 channels don't actually exist in real audio setups, then the property could have an enum hint to specify which values are allowed (and give them a description):

PROPERTY_HINT_ENUM, "Stereo:2,Surround 5.1:6,Surround 7.1:8"

[goatchurchprime]

I like this, but it's so far outside of my knowledge and could be misleading about what we have implemented. What I propose now is simple and close to the C++ implementation. If I do more of this I will have to some surround sound hardware to plug my PC into so it's not done completely blind.

[goatchurchprime]

Surround sound in games is mostly used for better spatialization (AudioStreamPlayer3D). It can give you a better impression of a sound being located in front of/behind you than stereo with a low-pass filter applied when the sound is behind you (HRTF). Surround audio data is typically not used for sound effects themselves (positional or otherwise) or background music – these remain stereo, or even mono.

This refers another thing I've noticed: the dire shortage of audio demo projects to exhibit almost any feature. I'll try and put together a spatial audio example that creates some output on these other channels and use the audio/driver/override_channel_out setting to hear if any noise comes from them.

I see the main purpose of this PR is to unblock that AudioEffectCapture bugfix that is waiting for people to obtain the right hardware to test it (which is not happening in a hurry). Once that's fixed, we can make a sort of visual simulator of the surround sound effects like I've done in godotengine/godot-demo-projects#1172 with the different speakers lighting up according to the sound going out to them on the master bus.

[goatchurchprime]

I've hit my first question I need an expert to answer.

Look how the panning responds to the different channels as I move the the source from one side of the listener to the other while playing the sound in the editor.

Screencast.from.2025-03-07.15-14-25.webm

Channels 0 and 1 are very weak, while channels 4 and 5 are responding strongly.

Turns out they roughly implemented the 7.1 inverse speaker orientations as shown here

And then according to the code when in Stereo mode we take just the first two speakers, Front-Left and Front-Right from this configuration instead of the expected Side-Left and Side-Right speakers -- which would be more correct for headphone wearers and VR players.

I always wondered why the panning never worked for me and I had to jack up the panning_strength to max just to hear any difference!

There's an old comment in the code that says we ought to do something about this:

//TODO: hardcoded main speaker directions for 2, 3.1, 5.1 and 7.1 setups - 
these are simplified and could also be made configurable

The paper they based this code on A Novel Multichannel Panning Method for Standard and Arbitrary Loudspeaker Configurations used a minimum of 6 speakers, so it's not a lot of help.

It's obvious to me that the stereo case ought to be a very special case that is defined on its own rather than as a slice of some experimental multi-speaker solution.

[goatchurchprime]

This is a polar coordinates plot of the captured intensity in the left and right channels (yellow and blue) of a single tone from an AudioStreamPlayer3D.

I've checked and the sound in the left ear is exactly zero for source that is 45 degrees behind and on the right (eg at position (3,0,-3)). I believe this is an artifact of having chosen those first two speaker_directions in the stereo case at front-left and front-right positions.

code.zip

Increasing the panning strength doesn't do much, but if you reduce the panning strength to 0.3 those dead spots become quite extreme.

I cannot see how this is remotely correct. I haven't found any online references that tell me what I should expect with regards to the panning strength of a point source of sound, since everything goes straight to talking about highfalutin ambisonic spectral graphs, and this question is way too elementary for people to consider.

I've noticed that when the sound is close to zero in one ear you get the same effect as color banding where you can hear the individual steps. If it goes quickly it just sounds like bad quality audio, so I don't think you ever want the sound to go to zero in one ear-- even if this was the right answer, which it isn't.

[berarma]

I like the idea of having tools to better test audio without having to replicate the final setup. I'm just not sure if this would be enough, I need to try it.

And then according to the code when in Stereo mode we take just the first two speakers, Front-Left and Front-Right from this configuration instead of the expected Side-Left and Side-Right speakers -- which would be more correct for headphone wearers and VR players.

I think Godot outputs audio suitable for speaker setups only, not headphones. Headphones need to apply a different spatial model. An easier alternative would be to interpolate the front channels to simulate in the headphones what the user would hear when using stereo speakers.

When only one channel has sound in a speaker setup, both ears still receive the sound, this doesn't happen with headphones. To simulate the same effect, the headphones left channel should be a balanced mix of the left and right channels with a bit less volume and slight low-frequency phase shift in the sound coming from the right channel. That way, no ear is muted ever, as it would happen with a speaker setup.

I don't think using the side channels would be more correct, unless the mentioned interpolation is applied.

There's an old comment in the code that says we ought to do something about this:

//TODO: hardcoded main speaker directions for 2, 3.1, 5.1 and 7.1 setups - 
these are simplified and could also be made configurable

I think this comment is about the positions of the speakers. As they are, it isn't equivalent to the 7.1 setup in the diagram posted and I don't think it follows any recommended setup concerning distances and angles between speakers and the user. This is what should be configurable or at least follow some predefined setup.

The paper they based this code on A Novel Multichannel Panning Method for Standard and Arbitrary Loudspeaker Configurations used a minimum of 6 speakers, so it's not a lot of help.

It's obvious to me that the stereo case ought to be a very special case that is defined on its own rather than as a slice of some experimental multi-speaker solution.

I think it should work well for the stereo case, just not for headphones. Another way to get the stereo and 3.1 audio would be to downmix from 5.1 but I think the result would be pretty similar.

[goatchurchprime]

The reason this is just a bug, and not something we should make any further attempts to rationalize, is the following line:

real_t initial_gain = 0.5 * powf(1.0 + r[speaker_num].direction.dot(source_direction), tightness) / r[speaker_num].effective_number_of_speakers;

What this means is the gain is exactly zero if source_direction = -r[speaker_num].direction for any positive value of tightness, no matter how small. Now, tightness = global_panning_strength * 2.0f * panning_strength.

The docs for this parameter say:

A value of 0.0 disables stereo panning entirely, leaving only volume attenuation in place. A value of 1.0 completely mutes one of the channels if the sound is located exactly to the left (or right) of the listener.

The default value of 0.5 is tuned for headphones. When using speakers, you may find lower values to sound better as speakers have a lower stereo separation compared to headphones.

So, no matter how low a value you put for panning_strength the "stereo separation" will be 100% at the positions directly opposite the speaker vectors. There will be a lot of overlap around the rest of the area, but there will be these awful regions of distortion as one channel briefly mutes out and goes back again.

This is, quite simply, not the correct equation to use for stereo sound as it cannot even implement panning_strength as intended. There's going to be a better answer, and it will just be a one-line fix when we find it. It could be something as simple as: gain = lerp(panning_strength, 1.0, initial_gain)

[berarma]

So, no matter how low a value you put for panning_strength the "stereo separation" will be 100% at the positions directly opposite the speaker vectors. There will be a lot of overlap around the rest of the area, but there will be these awful regions of distortion as one channel briefly mutes out and goes back again.

Yes, I had overlooked that variable. Godot should be configured by default for headphones but it doesn't seem to work as documented. I'll look into this.

[berarma]

I've tried to address all issues I've found with #103926.

[berarma]

I see the main purpose of this PR is to unblock that AudioEffectCapture bugfix that is waiting for people to obtain the right hardware to test it (which is not happening in a hurry). Once that's fixed, we can make a sort of visual simulator of the surround sound effects like I've done in godotengine/godot-demo-projects#1172 with the different speakers lighting up according to the sound going out to them on the master bus.

I think showing the gains for each channel would be enough, there's no need to sample the output. At least for the debugging I've been doing, that's more important than the samples being played.

It would be useful to be able to change the selected channels (override_channel_out) from the code, at runtime. And also with more granularity, using an array for the speaker indexes with max size equal to the actual speaker count.

For example, being able to use my stereo speakers to play the front or back speakers, but also the left or right ones (front and back), or any combination of speakers. This way I could test turning my head in the four directions respect to my speakers.

Another interesting feature would be the ability to downmix the output. I'm already doing it in #103926 for some setups, but it would be useful for further testing of other setups and the global output.

I hope this goes places, we need to get the audio team back to life. :)

[goatchurchprime]

Merged this with the latest and tested. By listening on headphones I can hear that the output have gone to the correct ears (ie the wrong way round when you set override_channel_out=3.

It's satisfying that you when you graph the capture of the first two channels that the predicted asymetry is visible:

[Mickeon]

Something went wrong here during the rebasing step