Why the acc of pretrained models are different from the acc in imageNet-result.csv

[Vigelos]

Thanks Ross and the community. I am a novice in computer vision. I am now quite confused that the acc I got from the pretrained models are different from the acc in imageNet-result.csv. I have no idea whether I have made some mistakes or misunderstanding:( .The top1-acc I got from pretrained ResNet34 on ImageNet1k is just about 64% while the value is about 75% in imageNet-result.csv.

ps: my code

import timm
from copy import deepcopy
import torch
import torch.nn as nn
import os
import torch.nn.functional as F
import torchvision.models as models;
from torch.utils.data import DataLoader;
from torchvision import datasets;
from torchvision import transforms;
import torchvision;
device = "cuda";

vit = timm.create_model('resnet34',pretrained=True);

transform = transforms.Compose(
    [transforms.ToTensor(),
     transforms.Resize([224,224])
    ])
root = 'ImageNet'
def get_imagenet(root, train = True, transform = None, target_transform = None):
    if train:
        root = os.path.join(root, 'train')
    else:
        root = os.path.join(root, 'val')
    return datasets.ImageFolder(root = root,
                               transform = transform,
                               target_transform = target_transform)
train_set = get_imagenet(root,train=True,transform = transform)
valid_set = get_imagenet(root,train=False,transform = transform)
trainloader = torch.utils.data.DataLoader(train_set, batch_size=100, shuffle=True)
testloader = torch.utils.data.DataLoader(valid_set, batch_size=100,shuffle=True)

def test(epoch):
    global best_acc
    vit.eval()
    test_loss = 0
    correct = 0
    total = 0
    with torch.no_grad():
        for batch_idx, (inputs, targets) in enumerate(testloader):
            inputs, targets = inputs.to(device), targets.to(device)
            outputs = vit(torch.squeeze(inputs, 1))
            loss = criterion(outputs, targets)
            test_loss += loss.item()
            _, predicted = outputs.max(1)
            total += targets.size(0)
            correct += predicted.eq(targets).sum().item()
            if (batch_idx+1)%50==0:
                print(batch_idx+1,'/',len(testloader),'epoch: %d'% epoch, 'Loss: %.3f | Acc: %.3f%% (%d/%d)'
                    % (test_loss/(batch_idx+1), 100.*correct/total, correct, total))

vit = vit.to(device)
criterion = nn.CrossEntropyLoss()
optimizer = torch.optim.SGD(vit.parameters(), lr=0.0008, momentum=0.9)

test(0)

[rwightman]

@Vigelos I'd recommend going throught the included validate.py script in detail, including the loader / transform setup ...

[Vigelos]

Thanks, after reading the script and relevent meterials,I guess that I missed the normalization step in loading data, which should be applied to models with convolution layers like ResNet, but should not apply to models without convolution layers like ViT.