Heuristic Fallback in cuDNN: Why THD Format Reverts to SM80 on Hopper GPUs

[TGS123]

"Why does the forward pass call the cuDNN SM90 interface when the learnable_sink option is enabled with qkv_format="bshd", but reverts to the cuDNN SM80 interface when qkv_format="thd" is selected?"

[ptrendx]

@cyanguwa Could you take a look?

[cyanguwa]

I'm assuming you're using cuDNN 9.18. It was a known bug and is fixed in 9.19. Could you please give it a try and let us know if there's still an issue? Thanks!

https://developer.nvidia.com/cudnn-downloads