Training setting of MobileNetV2

[deJQK]

I am trying to train the MobileNetV2. The original setting for training efficientnet seems to only use 2 GPU with batch size of 128 for each, which is too slow. I tried to modify the setting for 8 GPU with batch size of 256 on each, and scale the learning rate correspondingly. The setting for the training is as following:

#!/usr/bin/env bash

MODEL=mobilenetv2_100
nGPUs=8

python3 -W ignore -m torch.distributed.launch --nproc_per_node=$nGPUs --use_env main.py --model $MODEL \
-b 256 --lr .128 \
--sched step --epochs 450 --decay-epochs 2.4 --decay-rate .97 \
--opt rmsproptf --opt-eps .001 -j 96 --warmup-lr 1e-6 \
--weight-decay 1e-5 --drop 0.3 --drop-path 0.2 \
--model-ema --model-ema-decay 0.9999 \
--aa rand-m9-mstd0.5 --remode pixel --reprob 0.2 \
--data-path ./data/imagenet \
--output_dir /path/to/mbv2_w1

However, the accuracy is only 72.36%, much lower than the reported accuracy of 72.956%. Is the batch size setting necessary to achieve the reported results? Thanks.

[rwightman]

@deJQK I have to see if I have the 100/110/120 v2 hparams on a machine somewhere, you can find a more recent issue (#1021) in github here where I posted some hparams I used for mobilenetv2 0.5 and mnasnet with lamb optimizer ... that would be adapatable by upping the augmentation

For the ones above, but 0.3 drop is likely too high, maybe 0.2, and drop path 0.1-0.15. I typicaly use decay epoch of 1 now, with decay rate around 0.987-0.988 w/ rmsprop, and often throw in lr noise form 0.5 - 0.9 or 1.0 of training. Unfortunately batch size does impact result, it's definitely easier to get better results for rmsprop with smaller global batch sizes, lamb tends to hold up a bit better for larger global bach...