llama-bench backend/device selection?

[etasnadi]

Hi,

Maybe I am missing something, but I do not see the option to tell llama-bench which device/backend to use. I have multiple backends compiled and I would like to compare e.g. Vulkan and CUDA performance for example. I managed to do it by adding a few lines of code but I'm curious if there is a better way to do it.

Also, in the report, I see all the backends in the backend column not the one that is assigned to the model used in the test that is not surprising since the get_backend function simply returns all the discovered backends. Is it the expected behavior?

Thanks.

[slaren]

The short answer is that llama-bench was written before the functionality to use a custom list of devices was added to llama.cpp, and has not been updated yet.