couldn't find it in the cached files and it looks like Elron/bleurt-tiny-512, how to set the model path?

[bannima]

How to set the eval model path?

Eval

when I use the script to eval model with MATH-500

`NUM_GPUS=8 # Set to 8 for 32B and 70B models
MODEL=Deepseek_R1_distill/Qwen2.5-32B-Open-R1-Distill/
MODEL_ARGS="pretrained=$MODEL,dtype=bfloat16,max_model_length=32768,gpu_memory_utilisation=0.8,tensor_parallel_size=$NUM_GPUS"
OUTPUT_DIR=data/evals/Qwen2.5-32B-Open-R1-Distill

lighteval vllm $MODEL_ARGS "custom|math_500|0|0"
--custom-tasks src/open_r1/evaluate.py
--use-chat-template
--output-dir $OUTPUT_DIR
`

Error

Error: We couldn't connect to 'https://huggingface.co' to load this file, couldn't find it in the cached files and it looks like Elron/bleurt-tiny-512 is not the path to a directory containing a file named
config.json.

Where to set the eval model path in the script?

[hynky1999]

You shouldn't need the model, it should be lazy loaded now #536