Disabling cudnn regresses ImageUpscaleWithModel performance on ROCM 6.4

[alexheretic]

Custom Node Testing

• I have tried disabling custom nodes and the issue persists (see how to disable custom nodes if you need help)

Expected Behavior

See #10302 (comment)

rocm 6.4 + flash-attn performs significantly better for image upscaling with cudnn left on. This regressed after #10302

cudnn default: ImageUpscaleWithModel(4x-UltraSharp.pth) 1.84s/it

Actual Behavior

cudnn = False: ImageUpscaleWithModel(4x-UltraSharp.pth) 11.95s/it

Steps to Reproduce

Run an image upscale workflow with rocm 6.4 + flash-attn.

Debug Logs

Total VRAM 16368 MB, total RAM 64217 MB                                                                                                                                     
pytorch version: 2.9.0+rocm6.4                                                                                                                                              
AMD arch: gfx1100                                                                                                                                                           
ROCm version: (6, 4)                                                                                                                                                        
Set vram state to: NORMAL_VRAM                                                                                                                                              
Device: cuda:0 AMD Radeon RX 7900 GRE : native                                                                                                                              
Using Flash Attention                                                                                                                                                       
Python version: 3.13.7 (main, Aug 15 2025, 12:34:02) [GCC 15.2.1 20250813]                                                                                                  
ComfyUI version: 0.3.66                                                                                                                                                     
ComfyUI frontend version: 1.28.7

Other

No response

[Reber01Good]

It is not just upscaling speed, but everything: #10460. MIOPEN_FIND_MODE=2 (fast) is needed in combination with cudnn for optimal performance on RDNA3. comfyanonymous just forced a curcial setting to the optimal value for RDNA4, for all amd gpus, which is wrong. This is 3 people that claim this is a bad change. cudnn needs a launch option and it should default to True on RDNA3, maybe even 2. MIOPEN_FIND_MODE is also a very obscure setting which is key to optimal performance.

If more of us speak up this issue can't be ignored for much longer.

[alexheretic]

Unfortunately there isn't consensus on the affect of these settings in different environments. See discussion in #10302.

Some people mention better vae perf with cudnn off. I haven't seen that, it's generally similar on or off for me except for this regression with upscale perf.

Even this is only true, for me, on rocm 6.4. I tested on rocm7-nightly and the upscale perf had regressed again even worse, but this time with cudnn on 🫠 (results).

I think everyone wants the best perf by default we just haven't nailed down what that is for all environments.