What makes timm vit_base_patch_16 achieve 86.006 vs 84.15 in original vit paper on Imagenet

[jojo23333]

Hi, I just came across the VIT checkpoint used in this repo, as it is indicated in the results here:
https://github.com/rwightman/pytorch-image-models/blob/master/results/results-imagenet.csv

vit_base_patch16_384 achieves an 86.006 vs 84.15 reported in the paper Table 5:

There is a large gap between these two reported scores, I understand some of the improvements can be achieved by training with a longer scheme and better hyper-parameter choice. But since the +2 point boost is somewhat huge, so I wonder what could cause this huge performance boost. Is there some special technique applied in this? Where can I found the corresponding config file, checkpoint, and training details for the scores reported in the results-imagenet.csv?

Many thanks!

[rwightman]

@jojo23333 the checkpoints were updated with the best options from this paper that I was a part of, How to train your ViT? Data, Augmentation, and Regularization in Vision Transformers -- https://arxiv.org/abs/2106.10270

More augmenation and regularization was used w/ the 21k pretraining, and a search over both those and the transfer hparams was performed