Specifics of ImageNet Training Set

[michaelschleiss]

I am having trouble reproducing the accuracy numbers from this repo with many models. They are usually 1-2%-points below the reported accuracy numbers. My feeling is that it has to do with the training data. When people refer to Imagenet 1k I always assume that they mean the ImageNet split from the ILSVRC 2012 classification challenge. When you go to image-net.org to download the data from the 2012 classification challenge there is the big training dataset ~138GB with 1,281,167 images (task 1 + 2) and there is some additional data (task3). After that, in the following years, again 5500 additional images were added for the object detection challenge.

@rwightman Now my question is: Exactly what data was used to train the models in this repo?

[hankyul2]

Hi. I am not @rwightman, but want to help you .😊

ImageNet-1k Train/Val:

• Train = training images(Task 1&2).
• Validation = validation images(all tasks).

Accuray numbers reported in results-imagenet.csv are not only trained with imagenet-1k, but some models are trained with external dataset(e.g. imagenet-21k) or other training techniques(e.g. distilation).

If you want to reproduce similar (within 1.0%) accuracy shown in results-imagenet.csv, you should consider to check original paper or repository. Fortunately, many recent works are based on this repository, so you can easily use their hyperparameters.

Examples)

deit_small_patch16_224, 79.9 shown in facebookresearch/deit

python3 -m torch.distributed.launch --nproc_per_node=4 --master_port=12345 train.py --local_rank=0 imageNet --model vit_small_patch16_224 --aa rand-m9-mstd0.5-inc1 --mixup .8 --cutmix 1.0 --aug-repeats 3 --remode pixel --reprob 0.25 --drop-path .1 --opt adamw --weight-decay .05 --sched cosine --epochs 300 --lr 1e-3 --warmup-lr 1e-6 -b 256 -j 16 --amp --channels-last --log-wandb

I also sugget to look recent paper called resnet strikes back. This paper summarize and compare optimal imagenet-1k train setup.