Verification about support for Granite models in the HELM leaderboard

[RonalddMatias]

Hello, team.

I recently made two contributions to add support for the IBM Granite 3.0 and 3.1 models. After these changes, I would like to know if there's anything additional I need to do to have these models appear in the HELM leaderboard, or if they will be automatically included in the next release of the framework.

Thank you in advance for your help!

Best regards,
Ronaldd Matias.

[yifanmai]

Hi, generally evaluation results need to be manually run internally. We are hoping to use the watsonx version of the Granite models (because we generally consider the first party API version to be the "official" version, and there are slight differences in how the chat template is applied between the API version and the Hugging Face version), but this will take a while to set up. I'm planning to have Granite results in a few weeks.