Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[tests] Fix bnb cpu error #3351

Merged
merged 15 commits into from
Feb 6, 2025
7 changes: 5 additions & 2 deletions src/accelerate/accelerator.py
Original file line number Diff line number Diff line change
Expand Up @@ -1471,8 +1471,11 @@ def prepare_model(self, model: torch.nn.Module, device_placement: bool = None, e
"You can't train a model that has been loaded in 8-bit or 4-bit precision on a different device than the one "
"you're training on. Make sure you loaded the model on the correct device using for example `device_map={'':torch.cuda.current_device()}` or `device_map={'':torch.xpu.current_device()}`"
)

if ("cpu" in model_devices and not is_bitsandbytes_multi_backend_available()) or "disk" in model_devices:
if (
("cpu" in model_devices and not is_bitsandbytes_multi_backend_available())
or ("cpu" in model_devices and is_xpu_available())
or "disk" in model_devices
):
Comment on lines +1474 to +1478
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this mean that with xpu, it doesn't work if the model in on cpu even if bnb multi-backend is installed ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, cpu and xpu share the same bnb installation method, namely the multi-backend support.

raise ValueError(
"You can't train a model that has been loaded in 8-bit or 4-bit precision with CPU or disk offload. "
"If you want train the 8-bit or 4-bit model in CPU, please install bitsandbytes with multi-backend, see https://huggingface.co/docs/bitsandbytes/main/en/installation#multi-backend"
Expand Down
2 changes: 1 addition & 1 deletion tests/test_accelerator.py
Original file line number Diff line number Diff line change
Expand Up @@ -470,7 +470,7 @@ def test_accelerator_bnb(self):
# This should work
model = accelerator.prepare(model)

@require_cuda
@require_cuda_or_xpu
@slow
@require_bnb
def test_accelerator_bnb_cpu_error(self):
Expand Down