audio: crossover: Improve robustness for invalid configuration #23514

Job	Run time
build-windows (mnfst, ptl)	10m 57s
build-windows (mnfst, imx8 imx8x)	10m 15s
LP64-WIP	4m 7s
build-windows (mnfst, imx8m imx8ulp)	13m 14s
build-linux (mnfst, imx8 imx8x)	5m 3s
build-windows (-d, mnfst, lnl)	9m 9s
build-windows (mnfst, lnl)	8m 17s
build-windows (mnfst, mtl)	9m 29s
build-windows (mnfst, tgl)	9m 57s
manifest-check	2m 42s
build-windows (mnfst, tgl-h)	7m 48s
build-linux (mnfst, mtl)	4m 11s
build-linux (-d, mnfst, lnl)	4m 45s
build-linux (mnfst, tgl)	4m 19s
build-linux (mnfst, ptl)	5m 46s
build-linux (mnfst, imx8m imx8ulp)	5m 7s
build-linux (mnfst, lnl)	4m 39s
build-linux (mnfst, wcl imx95)	7m 3s
build-linux (mnfst, tgl-h)	4m 20s
compare-linux-win	31s
	2h 11m 39s

Provide feedback