Repository navigation
Support pre-release python versions for python-build-standalone toolchains #2837
Description
Jax (among others) likes to use Python runtimes from python-build-standalone that are pre-release versions to help verify upcoming behavior and try out new Python features. e.g. right now, they carry a patch to map 3.14 to 3.14.0a6.
The underlying versions.bzl file doesn't support pre-release versions, though, so they patch 3.14.0 to point to a 3.14.0a6 URL. This works, but it's a bit misleading. It's also problematic if one wanted to have configuration for two versions at once, differentiated by their pre-release version.
(Using patches isn't ideal, but I think they're using an older version that doesn't yet have the TOOL_VERSIONS arg/override -- regardless, the same issues are present).
While #2081 would also address this, I figured a more specific feature request is warranted (this is more narrowly scoped and well defined than 2081).
In any case, the desired outcome should be:
- A user can set
--python_version=3.14.0.a6on the command line, and have the appropriate runtime used - A user can have more than one pre-release configured in e.g. TOOL_VERSIONS
This should be fairly easy to implement and test:
- Use py_reconfig_test to set the python_version and customize toolchains
- Have the test assert sys.version
- Start fixing the errors and misbehaviors that ensue.
cc @kanglant
Activity
aignas commentedon Apr 29, 2025
Can we limit this support only to
bzlmodor do we need to supportWORKSPACEas well?rickeylev commentedon Apr 29, 2025
It can definitely start with bzlmod.
After skimming the code, also adding workspace may not be that hard -- when py_freethreaded got added, the TOOL_VERSIONS structs have some extra fields to carry more arbitrary config settings. So there's a least a place to put those right now. Users might actually be able to set those themselves via python_register_toolchains ?
pypi integration is the other big half. I don't think that handles below minor versions? That part might have to wait until deps-selection is moved to analysis phase. And possibly sdist building is moved to build phase. Not sure.
aignas commentedon Apr 29, 2025
PyPI integration now handles the full version (
X.Y.Z, but supportingrcversions might be doable, we should just get it from the interpreter itself).aignas commentedon May 15, 2025
#2874 should help with making our code much more robust at parsing versions the users provide. Attempting to register multiple pre-release versions should be doable as well. If it is not, I am happy to mentor someone to help implementing the remaining of the changes needed for this feature.
refactor: consolidate version parsing (#2874)
fix: support pre-release versions and add new toolchain versions (#2969)