Train resnet50 on A100*8

[wchen61]

How can I change the parameters to train resnet50 on A100 * 8?
(https://github.com/rwightman/pytorch-image-models/blob/main/docs/training_hparam_examples.md)

./distributed_train.sh 2 /imagenet -b 64 --model resnet50 --sched cosine --epochs 200 --lr 0.05 --amp --remode pixel --reprob 0.6 --aug-splits 3 --aa rand-m9-mstd0.5-inc1 --resplit --split-bn --jsd --dist-bn reduce

Is it reasonable to set batch size 256 and lr 0.8 to speed up the training?

./distributed_train.sh 8 /imagenet -b 256 --model resnet50 --sched cosine --epochs 200 --lr 0.8 --amp --remode pixel --reprob 0.6 --aug-splits 3 --aa rand-m9-mstd0.5-inc1 --resplit --split-bn --jsd --dist-bn reduce

[rwightman]

@wchen61 those hparams are old, there are a number of recipe 'templates" in https://arxiv.org/abs/2110.00476 ... they can be easily adapted to the timm args, and there are already discussions on here that can be searched for specific command lines for some of the main resnet50 recipes...