Request for the FID Evaluation Code in MAISI

[mar-cry]

I am currently developing a CT image generation algorithm and would like to compare its performance with the results reported in your work, MAISI. However, we have observed a significant discrepancy between the FID values calculated using our code and those reported in your paper.

To ensure fairness in the comparison, I would like to kindly request the code or implementation details used in MAISI for calculating the FID between generated and real images. This would greatly help in maintaining consistency and accuracy in our evaluations.

[aylward]

@guopengf ?

[guopengf]

We are using the FIDMetric in MONAI and the feature extractor is defined as follows:
feature_network = torch.hub.load( "Warvito/radimagenet-models", model="radimagenet_resnet50", verbose=True, trust_repo=True, ).

@dongyang0122 Shall we release the FID evaluation script in the MAISI MONAI Tutorial Repo?

[mar-cry]

Thank you very much for providing the testing method. I will first try to calculate the FID using the method you provided. Additionally, I would greatly appreciate it if you could share the complete FID testing script.

[danielemolino]

@mar-cry were you able to compute the FID score?

[Can-Zhao]

@dongyang0122 This was the discussion we mentioned last week. Thank you!

[mazniashvili]

Hello,
is there any update on this?
Is a global spatial average applied on the extracted features for flattening?
Also, would it be possible to share the SSIM code?

[dongyang0122]

You may refer this part for FID computation now.
https://github.com/Project-MONAI/tutorials/tree/main/generation/maisi#4-fid-score-computation

[dongyang0122]

@mar-cry Thanks for your interest. We will be adding the FID computation script to this tutorial in the following days.

[danielemolino]

@dongyang0122 Any news?

[dongyang0122]

You may refer this part for FID computation now.
https://github.com/Project-MONAI/tutorials/tree/main/generation/maisi#4-fid-score-computation

[dongyang0122]

I am currently developing a CT image generation algorithm and would like to compare its performance with the results reported in your work, MAISI. However, we have observed a significant discrepancy between the FID values calculated using our code and those reported in your paper.

To ensure fairness in the comparison, I would like to kindly request the code or implementation details used in MAISI for calculating the FID between generated and real images. This would greatly help in maintaining consistency and accuracy in our evaluations.

You may refer this part for FID computation now.
https://github.com/Project-MONAI/tutorials/tree/main/generation/maisi#4-fid-score-computation