[FEATURE] Training Scripts of MobileNetv4

[wenhui-ml]

Hello, thanks you very much for adding MobileNetv4 into this repo, excellent job!
But the problem is that I cannot train mobilenetv4 with a good result on cifar10 as the below script，so I would get some advice from you.

"CUDA_VISIBLE_DEVICES=0 python train.py data/imagenet/ --model mobilenetv4_conv_medium.e500_r224_in1k --pretrained --num-classes 10 --img-size 224 -b 256 --sched step --epochs 500 --decay-epochs 2.4 --decay-rate .969 --opt rmsproptf --opt-eps .001 -j 32 --warmup-lr 1e-6 --weight-decay 1e-5 --drop 0.3 --drop-path 0.2 --model-ema --model-ema-decay 0.9999 --aa rand-m9-mstd0.5 --remode pixel --reprob 0.3 --amp --lr .0016 --dist-bn reduce"

Is some else excellent training scripts to train mobilenetv4? Many thanks.

[rwightman]

@wenhui-ml for cifar you're probably going to want to dial the augmentations down a lot, keep the adaptation length (epochs) and LR quite a bit lower. There are probably reasonable adaptation hparams for a mobilenet-like model floating around out there as a better starting point. What you've got up there is something that'd be better for pretraining on imagenet....