Issues with torch version other than 2.2.2

[Nevermetyou65]

I got an error using lighteval with torch version 2.3.0, 2.3.1, 2.4.0, 2.5.1.
Basically, I can only use lighteval with torch 2.2.2.
The error I got is

RuntimeError: r.nvmlDeviceGetNvLinkRemoteDeviceType_ INTERNAL ASSERT FAILED at "../c10/cuda/driver_api.cpp":27, please report a bug to 
PyTorch. Can't find nvmlDeviceGetNvLinkRemoteDeviceType: /lib/x86_64-linux-gnu/libnvidia-ml.so.1: undefined symbol: 
nvmlDeviceGetNvLinkRemoteDeviceType

Here is my GPU machines