why dm_nfnet_f2 inference is very very slow?

[mobassir94]

dear @rwightman
i made inference on a dataset using following two models :

1. model 1 -> dm_nfnet_f2, batch size = 64, image size = 768
2. model 2 -> eca_nfnet_l0, batch size = 64, image size = 512

model 1 takes 1h 4min 6s
model 2 takes 10min