debug api scale_inv_min is always zero when using padding or divisibility requirements

[jomitchellnv]

The debug_api provides the ability to acquire log information specific to FP8 through FP8TensorStats. However, we noticed that when we add padding (32 for MXFP8 or 16 for NVFP4) to the end of a sequence to make it divisible, the scale_inv_min is always zero.

This is because, there exists a consecutive block of 32 0s, which means that one of your scale_inv values will be 0, and therefore your scale_inv_min will always be zero.

See this slide

Thus, we are always going to have some zeros in that batch, because its a requirement, which makes scale_inv_min a non useful metric for us.

If NVIDIA docker image is used you don't need to specify these.
Otherwise, please provide:

• OS version
• PyTorch version
• Python version
• Transformer Engine version
• CUDA version
• CUDNN version

Device details

• GPU model

Additional context

Add any other context about the problem here.

[ptrendx]

Hmmm, I guess it depends exactly on what you are doing, but I don't think this explanation is correct - you should never have to add full 32/16 elements as padding in the data (just add enough to have the shape divisible by 32/16).
On the other hand, we do pad the scale inverse ourselves, since it has even stronger requirements for the shape (corresponding to 128x128 blocks of data), so it would not be surprising to have a lot of 0s there. @pggPL could you check if we are correctly omitting those 0s in MXFP8/NVFP4 when we are calculating the scale inverse minimum?