Discrepancy in actual and expected output for subgraph test : batchNormalization options.axis=0 + gelu

[hthadicherla]

Issue

I was running this conformance test for NVIDIA's TensorRT-RTX EP with ORT backend and encountered a failure.

Input:

bnInput     shape [3,1,2] float32 = [-1, 0, 1, 2, 3, 4]
bnMean      shape [3]     float32 = [0, 3, 6]
bnVariance  shape [3]     float32 = [1.0, 1.5, 2.0]

Graph:

batchNormalization(input=bnInput, mean=bnMean, variance=bnVariance, {axis: 0}) -> bnOutput
gelu(bnOutput) -> output

Expected output: shape [3,1,2] float32

[-0.15865567326545715, 0, -0.08366739749908447,
 -0.16910307109355927, -0.03595117852091789, -0.1112278625369072]

Error happens at index 4

actual -0.035951320081949234 should be close enough to expected -0.03595117852091789 by ULP distance: expected <= 24 but got 38

Observations

I tested this with other execution providers in ORT and other backends and encountered similar or same error.

Backend	Actual	ULP Difference
ORT CPU EP	−0.035951320081949234	38
TFLite	−0.035951387137174606	56
LiteRT (GPU)	−0.035951387137174606	56

Question for the Working Group

• What is the expected behavior for this test ?
• Should the discrepancy between the actual and expected outputs be investigated, or is the current error tolerance too strict and in need of adjustment?

[adrastogi]

cc: @fdwr / @RafaelCintron

[fdwr]

or is the current error tolerance too strict

It's definitely too strict. @BruceDai - Is this particular combo that useful? Gelu has an erf, which is highly approximated, and erf has a much looser tolerance (hundreds of times looser), and so there's no way the combo of normalize-batch and gelu would have less error than of them both together. Additionally those two ops mix ULP and ATOL (where erf would squash any input from normalize-batch into -1.0 and 1.0). Can we just test that it executes successfully, but not worry about checking the compounded error (since each is validated individually anyway), or even just delete it 😅 (it doesn't really add much value to the test suite AFAICS)?