Skip to content

Can't run tests with array_api_strict 2.6 without a CUDA device #34456

Description

@betatim

Warning

This issue is not yet ready for a PR. If you are interested in contributing to scikit-learn, please have a look at our contributing guidelines, and in particular the sections for new contributors and the "Needs triage" label.

I was trying to run the array API tests using array_api_strict (version 2.6, newly released), as you might if you want to run the tests and don't have a MPS or CUDA device locally. I used pytest -k array_api_strict to do that. This works but also selects tests like "sklearn/metrics/tests/test_common.py::test_mixed_array_api_namespace_input_compliance[accuracy_score-array_api_strict to torch cpu]", which makes sense as I have torch installed. However the test fails with the following because my torch was compiled without CUDA support:

<snip>
sklearn/metrics/_classification.py:129: in _check_targets
    y_true, sample_weight = move_to(y_true, sample_weight, xp=xp, device=device)
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
sklearn/utils/_array_api.py:598: in move_to
    array_converted = xp.from_dlpack(array, device=device)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
../../miniconda/envs/sklearn-py3.13/lib/python3.13/site-packages/torch/utils/dlpack.py:169: in from_dlpack
    stream = torch.cuda.current_stream(f'cuda:{ext_device[1]}')
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
../../miniconda/envs/sklearn-py3.13/lib/python3.13/site-packages/torch/cuda/__init__.py:1133: in current_stream
    _lazy_init()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    def _lazy_init():
        global _initialized, _queued_calls
        if is_initialized() or hasattr(_tls, "is_initializing"):
            return
        with _initialization_lock:
            # We be double-checked locking, boys!  This is OK because
            # the above test was GIL protected anyway.  The inner test
            # is for when a thread blocked on some other thread which was
            # doing the initialization; when they get the lock, they will
            # find there is nothing left to do.
            if is_initialized():
                return
            # It is important to prevent other threads from entering _lazy_init
            # immediately, while we are still guaranteed to have the GIL, because some
            # of the C calls we make below will release the GIL
            if _is_in_bad_fork():
                raise RuntimeError(
                    "Cannot re-initialize CUDA in forked subprocess. To use CUDA with "
                    "multiprocessing, you must use the 'spawn' start method"
                )
            if not hasattr(torch._C, "_cuda_getDeviceCount"):
>               raise AssertionError("Torch not compiled with CUDA enabled")
E               AssertionError: Torch not compiled with CUDA enabled

../../miniconda/envs/sklearn-py3.13/lib/python3.13/site-packages/torch/cuda/__init__.py:417: AssertionError

This happens because the array_api_strict device device1 uses CUDA as the device ID (data-apis/array-api-strict#212).

I'm not sure yet what we should do about this. Revert data-apis/array-api-strict#212, live with this, tell people to uninstall torch, something else. With array-api-strict 2.5 I think this isn't a problem.

I thought I'd create the issue already so others don't run into the same problem/spend time debugging this.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions