Skip to content

ENH: Simplify CuPy asarray and to_device #314

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Apr 24, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 17 additions & 31 deletions array_api_compat/common/_helpers.py
Original file line number Diff line number Diff line change
Expand Up @@ -775,42 +775,28 @@ def _cupy_to_device(
/,
stream: int | Any | None = None,
) -> _CupyArray:
import cupy as cp # pyright: ignore[reportMissingTypeStubs]
from cupy.cuda import Device as _Device # pyright: ignore
from cupy.cuda import stream as stream_module # pyright: ignore
from cupy_backends.cuda.api import runtime # pyright: ignore
import cupy as cp

if device == x.device:
return x
elif device == "cpu":
if device == "cpu":
# allowing us to use `to_device(x, "cpu")`
# is useful for portable test swapping between
# host and device backends
return x.get()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whoa. I'm confused: this returns a numpy array if x was a cupy array. This might be intended, but then the return annotation is misleading? Same for the device argument annotation. So maybe add a comment if a numpydoc docstring with Parameters and Returns sections is an overkill.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

None of this has changed. See also #87.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My point is that gh-87 is still open, and there are no tests, so if we touch this, great, let's take the opportunity to also improve testing.

elif not isinstance(device, _Device):
raise ValueError(f"Unsupported device {device!r}")
else:
# see cupy/cupy#5985 for the reason how we handle device/stream here
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Issue fixed in 2021

prev_device: Any = runtime.getDevice() # pyright: ignore[reportUnknownMemberType]
prev_stream = None
if stream is not None:
prev_stream: Any = stream_module.get_current_stream() # pyright: ignore
# stream can be an int as specified in __dlpack__, or a CuPy stream
if isinstance(stream, int):
stream = cp.cuda.ExternalStream(stream) # pyright: ignore
elif isinstance(stream, cp.cuda.Stream): # pyright: ignore[reportUnknownMemberType]
pass
else:
raise ValueError("the input stream is not recognized")
stream.use() # pyright: ignore[reportUnknownMemberType]
try:
runtime.setDevice(device.id) # pyright: ignore[reportUnknownMemberType]
arr = x.copy()
finally:
runtime.setDevice(prev_device) # pyright: ignore[reportUnknownMemberType]
if stream is not None:
prev_stream.use()
return arr
if not isinstance(device, cp.cuda.Device):
raise TypeError(f"Unsupported device type {device!r}")

if stream is None:
with device:
return cp.asarray(x)

# stream can be an int as specified in __dlpack__, or a CuPy stream
if isinstance(stream, int):
stream = cp.cuda.ExternalStream(stream)
elif not isinstance(stream, cp.cuda.Stream):
raise TypeError(f"Unsupported stream type {stream!r}")

with device, stream:
return cp.asarray(x)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have test coverage for this part? TBH I'm ready to believe it's totally correct but would prefer to have some tests to ensure that it really is.
Also I wouldn't be totally surprised if this is cupy version dependent, in which case a decent test coverage would be really helpful for diagnosing/debugging.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

None of this is covered by array-api-tests data-apis/array-api-tests#302.
I've tested this PR manually against the latest CuPy.

Also I wouldn't be totally surprised if this is cupy version dependent

The previous design cites an upstream issue resolved in 2021.
I have not tested against old versions. How old is too old to care? At the moment I understand that the policy is "don't willfully break backwards compatibility", but there are no tests whatsoever for old versions of any backend other than numpy.

in which case a decent test coverage would be really helpful for diagnosing/debugging.

Fully agree, and the place for it is data-apis/array-api-tests#302

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, testing it in array-api-tests would be great. Two things to consider though:

  • Given that there are no tests at all, either here (cupy-specific) or generic, I think this PR is a great opportunity to add some testing
  • Given how underspecified the device story is in the standard (for better or worse), my impression is that array-api-tests would be somewhat limited (as we cannot test implementation-dependent behaviors in there). Thus, cupy-specific tests here would be great.

Re: how old is a version we care? Might want some input from CuPy devs, too. Off the cuff, I'd say we definitely care about 13.4; previous 13.x versions with x < 4, not necessary;
Whether we care about 13.x when 14.x stops being alpha... Maybe not unless there are strong reasons?



def _torch_to_device(
Expand Down
30 changes: 8 additions & 22 deletions array_api_compat/cupy/_aliases.py
Original file line number Diff line number Diff line change
Expand Up @@ -64,8 +64,6 @@
finfo = get_xp(cp)(_aliases.finfo)
iinfo = get_xp(cp)(_aliases.iinfo)

_copy_default = object()


# asarray also adds the copy keyword, which is not present in numpy 1.0.
def asarray(
Expand All @@ -79,7 +77,7 @@ def asarray(
*,
dtype: Optional[DType] = None,
device: Optional[Device] = None,
copy: Optional[bool] = _copy_default,
copy: Optional[bool] = None,
**kwargs,
) -> Array:
"""
Expand All @@ -89,25 +87,13 @@ def asarray(
specification for more details.
"""
with cp.cuda.Device(device):
# cupy is like NumPy 1.26 (except without _CopyMode). See the comments
# in asarray in numpy/_aliases.py.
if copy is not _copy_default:
# A future version of CuPy will change the meaning of copy=False
# to mean no-copy. We don't know for certain what version it will
# be yet, so to avoid breaking that version, we use a different
# default value for copy so asarray(obj) with no copy kwarg will
# always do the copy-if-needed behavior.

# This will still need to be updated to remove the
# NotImplementedError for copy=False, but at least this won't
# break the default or existing behavior.
if copy is None:
copy = False
elif copy is False:
raise NotImplementedError("asarray(copy=False) is not yet supported in cupy")
kwargs['copy'] = copy

return cp.array(obj, dtype=dtype, **kwargs)
if copy is None:
return cp.asarray(obj, dtype=dtype, **kwargs)
else:
res = cp.array(obj, dtype=dtype, copy=copy, **kwargs)
if not copy and res is not obj:
raise ValueError("Unable to avoid copy while creating an array as requested")
return res
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This part seems to streamline and remove workarounds for presumably older cupy versions.

  1. Do we have test coverage?
  2. Which cupy version made the change which renders the removed workaround obsolete?
  3. Do we then need to formulate some sort of policy for a minimum supported CuPy version? I personally would be OK with any sort of policy ("the currently most recent released version" is fine by me), as long as it's spelled out and lower capped in pyproject.toml.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope, the latest cupy still has the same exact issues as the old ones.
The API I'm using hasn't changed since CuPy 1.0.

Do we then need to formulate some sort of policy for a minimum supported CuPy version?

We need to formulate policy for a minimum supported version of all backends.
So far it has not been done deliberately in order to reduce attrition (much appreciated).
This is a matter of discussion for the Committee.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The latest cupy you mean 13.4 or 14.0.a1?

One thing to bear in mind with CuPy specifically: IIUC they are still working through the numpy 2.0 transition. 13.4 is sort-of compatible (one goal was to make it import with both numpy 1 and numpy 2), 14.x series is supposed to complete the transition.
Which is partly a motivation for my questions/comments: if it's a moving target now, we should have test coverage.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

13.4.1.



def astype(
Expand Down
3 changes: 0 additions & 3 deletions cupy-xfails.txt
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,6 @@ array_api_tests/test_array_object.py::test_scalar_casting[__index__(int64)]
# testsuite bug (https://github.com/data-apis/array-api-tests/issues/172)
array_api_tests/test_array_object.py::test_getitem

# copy=False is not yet implemented
array_api_tests/test_creation_functions.py::test_asarray_arrays

# attributes are np.float32 instead of float
# (see also https://github.com/data-apis/array-api/issues/405)
array_api_tests/test_data_type_functions.py::test_finfo[float32]
Expand Down
24 changes: 14 additions & 10 deletions tests/test_common.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
from array_api_compat import (
device, is_array_api_obj, is_lazy_array, is_writeable_array, size, to_device
)
from array_api_compat.common._helpers import _DASK_DEVICE
from ._helpers import all_libraries, import_, wrapped_libraries, xfail


Expand Down Expand Up @@ -189,23 +190,26 @@ class C:


@pytest.mark.parametrize("library", all_libraries)
def test_device(library, request):
def test_device_to_device(library, request):
if library == "ndonnx":
xfail(request, reason="Needs ndonnx >=0.9.4")
xfail(request, reason="Stub raises ValueError")
if library == "sparse":
xfail(request, reason="No __array_namespace_info__()")

xp = import_(library, wrapper=True)
devices = xp.__array_namespace_info__().devices()

# We can't test much for device() and to_device() other than that
# x.to_device(x.device) works.

# Default device
x = xp.asarray([1, 2, 3])
dev = device(x)

x2 = to_device(x, dev)
assert device(x2) == device(x)

x3 = xp.asarray(x, device=dev)
assert device(x3) == device(x)
for dev in devices:
if dev is None: # JAX >=0.5.3
continue
if dev is _DASK_DEVICE: # TODO this needs a better design
continue
y = to_device(x, dev)
assert device(y) == dev


@pytest.mark.parametrize("library", wrapped_libraries)
Expand Down
22 changes: 22 additions & 0 deletions tests/test_cupy.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
import pytest
from array_api_compat import device, to_device

xp = pytest.importorskip("array_api_compat.cupy")
from cupy.cuda import Stream


def test_to_device_with_stream():
devices = xp.__array_namespace_info__().devices()
streams = [
Stream(),
Stream(non_blocking=True),
Stream(null=True),
Stream(ptds=True),
123, # dlpack stream
]

a = xp.asarray([1, 2, 3])
for dev in devices:
for stream in streams:
b = to_device(a, dev, stream=stream)
assert device(b) == dev
Loading