Add pyproject.toml #255

NeilGirdhar · 2025-02-20T21:22:45Z

No description provided.

ev-br · 2025-02-20T23:26:58Z

Two quick notes:

Am going to look at it only after 1.11 is out. So a delay here is not the lack of interest.
Hatchling? Would actually prefer something "boring" and robust. We need dynamic versioning, so flit is out. Can we keep it on setuptools+setuptools_scm for the dynamic version, please.

NeilGirdhar · 2025-02-21T08:34:54Z

Absolutely no rush. I only proposed this because I tried to create a venv and uv wouldn't let me.

Happy to use setuptools. I don't understand a lot about backends. It seems like every project I look at uses a different backend! I just used hatch because it did the conversion for me. I'll make the change to setuptools now.

Thanks for looking at this.

ev-br · 2025-02-21T10:39:47Z

Thanks for looking into int!

It seems like every project I look at uses a different backend!

Exactly. Which is why I'd prefer to have the same backend here and in array-api-strict, https://github.com/data-apis/array-api-strict/blob/main/pyproject.toml

tried to create a venv and uv wouldn't let me

So uv forces users to not use setup.py? Interesting.

NeilGirdhar · 2025-02-21T11:33:38Z

So uv forces users to not use setup.py? Interesting.

Well, uv venv does, yes. The other uv commands don't care AFAIK.

ev-br · 2025-02-27T15:17:39Z

Needs a rebase now that 1.11 is out.

NeilGirdhar · 2025-02-27T15:26:27Z

Done.

NeilGirdhar · 2025-02-27T16:13:07Z

I wish I knew how to set up uv for CI, but it might be worth doing instead of regular pip. I'm also not sure how to get uv to work with this project because cupy won't build on my system (unless I install more things?)

ev-br · 2025-02-27T18:26:25Z

pyproject.toml

+repository = "https://github.com/data-apis/array-api-compat/"
+
+[tool.setuptools.dynamic]
+version = {attr = "array_api_compat.__version__"}


Note to self: need to make sure this keeps working from both a regular commit and a git tag for a release:

a "regular" way: python -m bulid generates a version with a dynamic tag

from a git tag / in a release workflow: the version is from __init__.py, as set manually

ev-br · 2025-02-27T18:46:29Z

Re: uv or other bright and shiny tools: pixi was recommended in #251 to simplify working with cupy.

Making these new tools work is cool of course, and I'm mildly curious if you can make headway in that space.
Moving the CI to pixi might be worth a shot.

I will however have to make sure my regular ways keep working. For instance, I regularly need to mix and match editable installs of branches of -compat, -tests and -strict locally.
(And a modern heavy automation actually makes the need stronger. As a data point: just this morning I had to rely on testing things locally before triggering a release build/upload --- the combination of branch protection and trusted publishing simply does not allow testing things on CI before making a release :-)).

NeilGirdhar · 2025-02-27T18:49:51Z

Oh, no worries, please just keep doing whatever's easiest. I just don't know Pixi yet.

ev-br · 2025-03-21T00:22:41Z

pyproject.toml

+license = {file = "LICENSE"}
+authors = [{name = "Consortium for Python Data API Standards"}]
+classifiers = [
+    "License :: OSI Approved :: MIT License",


I had to upper cap setuptools both here and in array-api-strict : newer versions complained about this way of specifying the license being not SPDX-compliant (whatever that is).

You mean you had to upper cap it just now, or before? I was getting errors too when I had both the license text and the license file. Anyway, I'm okay with the upper cap if you are.

I mean earlier today (ok, yesterday, March 20), both here (setup.py) and in array-api-compat (pyproject.toml). The upper cap is a band-aid, so I was hoping you know about this SPDX thing. If not, then not, we'll upper cap for the time being.

And BTW, I'm planning to take a closer look at this PR, just not this week already.

And BTW, I'm planning to take a closer look at this PR, just not this week already.

No rush, and thank you!

I was hoping you know about this SPDX thing.

I'll look into it. Do you know which command gave you the error? I use uv pip now, so I didn't see any error. (It did give me an error when I had both the license text and file though.)

Do you know which command gave you the error?

This was on CI, so $ python -m build, I'd think. Not sure if this matters, as long as setuptools is uber-recent

Perfect, thanks! Should be fixed now. If you want to support a particular version of MIT (or versions) of MIT, please let me know. The spec is here.

ev-br

Trying the usual testing workflow:

$ pip uninstall array_api_compat
$ pip install -e .
$ cd ../array-api-tests
$ ARRAY_API_TESTS_MODULE=array_api_compat.numpy pytest array_api_tests/ -v 
ImportError while loading conftest '/home/br/repos/array-api-tests/conftest.py'.
conftest.py:11: in <module>
    from array_api_tests import _array_module as xp
array_api_tests/__init__.py:36: in <module>
    xp = import_module(xp_name)
../array-api-compat/array_api_compat/numpy/__init__.py:7: in <module>
    from ._aliases import * # noqa: F403
../array-api-compat/array_api_compat/numpy/_aliases.py:5: in <module>
    from .._internal import get_xp
E   ModuleNotFoundError: No module named 'array_api_compat._internal'

ev-br

A second problem:

$ pip uninstall array_api_compat
$ python -m build .
$ cd dist
$ ll
# ...
-rw-rw-r--  1 br br 52084 мар 23 10:17 array_api_compat-1.12.dev0-py3-none-any.whl
-rw-rw-r--  1 br br 95333 мар 23 10:17 array_api_compat-1.12.dev0.tar.gz
$
$ pip install array_api_compat-1.12.dev0-py3-none-any.whl
$ ipython
Python 3.9.19 | packaged by conda-forge | (main, Mar 20 2024, 12:50:21) 
Type 'copyright', 'credits' or 'license' for more information
IPython 8.18.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import array_api_compat

In [2]: array_api_compat.__version__
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[2], line 1
----> 1 array_api_compat.__version__

AttributeError: module 'array_api_compat' has no attribute '__version__'

NeilGirdhar · 2025-03-23T10:06:26Z

Trying the usual testing workflow:

Hmm, I'm not sure what exactly is wrong because it works fine for me. I use uv for my virtual environment though, so I couldn't follow your instructions exactly.

NeilGirdhar · 2025-03-23T10:09:14Z

Second one works fine for me as written:

❯ cd array-api-compat/
Switching virtualenv .venv [Python 3.13.2]
❯ pip install --config-settings editable_mode=compat -e .
Obtaining file:///home/neil/third/array-api-compat
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Preparing editable metadata (pyproject.toml) ... done
Building wheels for collected packages: array-api-compat
  Building editable for array-api-compat (pyproject.toml) ... done
  Created wheel for array-api-compat: filename=array_api_compat-1.12.dev0-0.editable-py3-none-any.whl size=2999 sha256=fda067b2ed5fd74d694500c33bf939fb078a3632668895c167d1afd0169ff38d
  Stored in directory: /tmp/pip-ephem-wheel-cache-5n0jqyib/wheels/50/66/92/e580815f74b079b73718bd3be84019aa7fa0f0c7936b2cefe5
Successfully built array-api-compat
Installing collected packages: array-api-compat
  Attempting uninstall: array-api-compat
    Found existing installation: array-api-compat 1.12.dev0
    Uninstalling array-api-compat-1.12.dev0:
      Successfully uninstalled array-api-compat-1.12.dev0
Successfully installed array-api-compat-1.12.dev0
❯ python -m build .
* Creating isolated environment: venv+pip...
[snip]
Successfully built array_api_compat-1.12.dev0.tar.gz and array_api_compat-1.12.dev0-py3-none-any.whl
❯ cd dist
Switching virtualenv .venv [Python 3.13.2]
❯ eza -l --git --icons=always --no-permissions --no-user -s=type
52k 23 Mar 06:06 -I  array_api_compat-1.12.dev0-py3-none-any.whl
95k 23 Mar 06:06 -I  array_api_compat-1.12.dev0.tar.gz
❯ pip install array_api_compat-1.12.dev0-py3-none-any.whl
Processing ./array_api_compat-1.12.dev0-py3-none-any.whl
Installing collected packages: array-api-compat
  Attempting uninstall: array-api-compat
    Found existing installation: array-api-compat 1.12.dev0
    Uninstalling array-api-compat-1.12.dev0:
      Successfully uninstalled array-api-compat-1.12.dev0
Successfully installed array-api-compat-1.12.dev0
❯ jupyter console 
Jupyter console 6.6.3
Python 3.13.2 (main, Feb 12 2025, 14:51:17) [Clang 19.1.6 ]
Type 'copyright', 'credits' or 'license' for more information
IPython 9.0.0 -- An enhanced Interactive Python. Type '?' for help.
Tip: Use `F2` or %edit with no arguments to open an empty editor with a temporary file.
[utM] In [1]: import array_api_compat
[utM] In [2]: array_api_compat.__version__
Out[2]: '1.12.dev0'

NeilGirdhar · 2025-03-23T10:11:54Z

My guess is there my be something wrong with your venv? Maybe delete .venv (or wherever you're storing it) and create a new one?

ev-br · 2025-03-23T10:37:20Z

Ok, here's a fresh env:

$ mamba deactivate
$ mamba create -n array-api2 python==3.12 -y
# ....
$ mamba activate array-api2
$ rm -rf dist
$ python -m build .
# ...
$ cd dist/
$ ll
total 156
drwxrwxr-x  2 br br  4096 мар 23 11:32 ./
drwxrwxr-x 12 br br  4096 мар 23 11:32 ../
-rw-rw-r--  1 br br 52084 мар 23 11:32 array_api_compat-1.12.dev0-py3-none-any.whl
-rw-rw-r--  1 br br 95395 мар 23 11:32 array_api_compat-1.12.dev0.tar.gz
$ pip install array_api_compat-1.12.dev0-py3-none-any.whl
Processing ./array_api_compat-1.12.dev0-py3-none-any.whl
Installing collected packages: array-api-compat
Successfully installed array-api-compat-1.12.dev0
$ python
Python 3.12.0 | packaged by conda-forge | (main, Oct  3 2023, 08:43:22) [GCC 12.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import array_api_compat
>>> array_api_compat.__version__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'array_api_compat' has no attribute '__version__'
>>> dir(array_api_compat)
['__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__']

NeilGirdhar · 2025-03-23T11:33:54Z

Can you import anything else from array_api_compat? Because I didn't even change the version attribute.

ev-br · 2025-03-23T11:59:03Z

No, the install is broken:

$ ll /home/br/miniforge3/envs/array-api2/lib/python3.12/site-packages/array_api_compat/
total 28
drwxrwxr-x  7 br br 4096 мар 23 11:46 ./
drwxrwxr-x 18 br br 4096 мар 23 11:46 ../
drwxrwxr-x  3 br br 4096 мар 23 11:46 common/
drwxrwxr-x  3 br br 4096 мар 23 11:46 cupy/
drwxrwxr-x  4 br br 4096 мар 23 11:46 dask/
drwxrwxr-x  3 br br 4096 мар 23 11:46 numpy/
drwxrwxr-x  3 br br 4096 мар 23 11:46 torch/

__version__ is set in array_api_compat/__init__.py which is missing

NeilGirdhar · 2025-03-23T12:26:47Z

Thank you. Would you mind testing the version I just pushed to see if it works?

ev-br · 2025-03-23T12:53:02Z

Thanks! This works now.
Out of curiosity: what was the culprit?
I'll do some more extensive testing next week.

NeilGirdhar · 2025-03-23T12:55:45Z

Great! Thank you very much for your patience with this!

My mistake was that I had

[tool.setuptools.packages.find]
include = ["array_api_compat.*"]

(extra period) instead of

[tool.setuptools.packages.find]
include = ["array_api_compat*"]

Also, I wasn't reproducing it properly because my venv was messed (not sure how).

ev-br · 2025-03-24T11:18:50Z

extra period) instead of

Ouch. This stuff is fiddly indeed :-(.

Could you resolve the conflict please? It is otherwise ready, I tested a bit locally and it all seems to work.

NeilGirdhar · 2025-03-25T02:08:32Z

Rebased, and verified that py.typed is in the wheel.

ev-br · 2025-03-25T12:16:31Z

Okay, the CI failures are known and have fixes in the pipeline (gh-286); local testing does now show new problems, so let's land this. Thank you @NeilGirdhar for your patience and for your persistence!

NeilGirdhar · 2025-03-25T12:17:23Z

Thank you!

NeilGirdhar force-pushed the pyproject branch from 7f42022 to c968dbe Compare February 21, 2025 08:41

NeilGirdhar force-pushed the pyproject branch 6 times, most recently from b96565a to 91d87b4 Compare February 27, 2025 12:01

ev-br added this to the 1.12 milestone Feb 27, 2025

NeilGirdhar force-pushed the pyproject branch from 91d87b4 to c15bdca Compare February 27, 2025 15:25

NeilGirdhar force-pushed the pyproject branch 2 times, most recently from 24fd987 to 1a17fc2 Compare February 27, 2025 16:10

ev-br reviewed Feb 27, 2025

View reviewed changes

NeilGirdhar mentioned this pull request Mar 2, 2025

TYP: Type annotations overhaul, part 1 #257

Merged

ev-br modified the milestones: 1.11.1, 1.12 Mar 4, 2025

NeilGirdhar force-pushed the pyproject branch 3 times, most recently from 6bfa429 to 6fbb863 Compare March 9, 2025 17:17

ev-br modified the milestones: 1.11.2, 1.12 Mar 20, 2025

ev-br mentioned this pull request Mar 20, 2025

REL: release 1.11.2 branch #280

Merged

ev-br reviewed Mar 21, 2025

View reviewed changes

NeilGirdhar force-pushed the pyproject branch 2 times, most recently from 8ec4eef to 4f652f7 Compare March 21, 2025 11:02

lucascolley mentioned this pull request Mar 22, 2025

migrate from setup.py to pyproject.toml #287

Closed

ev-br requested changes Mar 23, 2025

View reviewed changes

NeilGirdhar force-pushed the pyproject branch from 4f652f7 to ab9264a Compare March 23, 2025 12:26

ev-br approved these changes Mar 24, 2025

View reviewed changes

Add pyprject.toml

f19256e

NeilGirdhar force-pushed the pyproject branch from ab9264a to f19256e Compare March 25, 2025 02:04

ev-br merged commit 7e1b689 into data-apis:main Mar 25, 2025
34 of 40 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add pyproject.toml #255

Add pyproject.toml #255

NeilGirdhar commented Feb 20, 2025

ev-br commented Feb 20, 2025

NeilGirdhar commented Feb 21, 2025

ev-br commented Feb 21, 2025

NeilGirdhar commented Feb 21, 2025

ev-br commented Feb 27, 2025

NeilGirdhar commented Feb 27, 2025 •

edited

Loading

NeilGirdhar commented Feb 27, 2025

ev-br Feb 27, 2025

ev-br commented Feb 27, 2025

NeilGirdhar commented Feb 27, 2025

ev-br Mar 21, 2025

NeilGirdhar Mar 21, 2025

ev-br Mar 21, 2025

NeilGirdhar Mar 21, 2025

ev-br Mar 21, 2025

NeilGirdhar Mar 21, 2025 •

edited

Loading

ev-br left a comment

ev-br left a comment

NeilGirdhar commented Mar 23, 2025

NeilGirdhar commented Mar 23, 2025 •

edited

Loading

NeilGirdhar commented Mar 23, 2025 •

edited

Loading

ev-br commented Mar 23, 2025

NeilGirdhar commented Mar 23, 2025

ev-br commented Mar 23, 2025

NeilGirdhar commented Mar 23, 2025

ev-br commented Mar 23, 2025

NeilGirdhar commented Mar 23, 2025 •

edited

Loading

ev-br commented Mar 24, 2025

NeilGirdhar commented Mar 25, 2025

ev-br commented Mar 25, 2025

NeilGirdhar commented Mar 25, 2025

Add pyproject.toml #255

Add pyproject.toml #255

Conversation

NeilGirdhar commented Feb 20, 2025

ev-br commented Feb 20, 2025

NeilGirdhar commented Feb 21, 2025

ev-br commented Feb 21, 2025

NeilGirdhar commented Feb 21, 2025

ev-br commented Feb 27, 2025

NeilGirdhar commented Feb 27, 2025 • edited Loading

NeilGirdhar commented Feb 27, 2025

ev-br Feb 27, 2025

Choose a reason for hiding this comment

ev-br commented Feb 27, 2025

NeilGirdhar commented Feb 27, 2025

ev-br Mar 21, 2025

Choose a reason for hiding this comment

NeilGirdhar Mar 21, 2025

Choose a reason for hiding this comment

ev-br Mar 21, 2025

Choose a reason for hiding this comment

NeilGirdhar Mar 21, 2025

Choose a reason for hiding this comment

ev-br Mar 21, 2025

Choose a reason for hiding this comment

NeilGirdhar Mar 21, 2025 • edited Loading

Choose a reason for hiding this comment

ev-br left a comment

Choose a reason for hiding this comment

ev-br left a comment

Choose a reason for hiding this comment

NeilGirdhar commented Mar 23, 2025

NeilGirdhar commented Mar 23, 2025 • edited Loading

NeilGirdhar commented Mar 23, 2025 • edited Loading

ev-br commented Mar 23, 2025

NeilGirdhar commented Mar 23, 2025

ev-br commented Mar 23, 2025

NeilGirdhar commented Mar 23, 2025

ev-br commented Mar 23, 2025

NeilGirdhar commented Mar 23, 2025 • edited Loading

ev-br commented Mar 24, 2025

NeilGirdhar commented Mar 25, 2025

ev-br commented Mar 25, 2025

NeilGirdhar commented Mar 25, 2025

NeilGirdhar commented Feb 27, 2025 •

edited

Loading

NeilGirdhar Mar 21, 2025 •

edited

Loading

NeilGirdhar commented Mar 23, 2025 •

edited

Loading

NeilGirdhar commented Mar 23, 2025 •

edited

Loading

NeilGirdhar commented Mar 23, 2025 •

edited

Loading