poetry install `python X.Y` fails with "No pre-built available..."

[rotu]

Most major versions of python give an error like:

proto run python 3.11
Error: plugin::call_func::failed                                                                                        
  × No pre-built available for version 3.11.9 (via https://github.com/indygreg/python-build-standalone)! Try
  │ installing another version for the time being.

This is on Windows 11 and happens for every major version I've tried except 3.10.
The current best workaround is to specify a previous patch version.

[rotu]

Recurrence of #11

[milesj]

@rotu Can you paste logs? This plugin hasn't really changed in a long time.

All the available releases are here: https://github.com/moonrepo/python-plugin/blob/master/releases.json

[rotu]

Here's an example on my Mac. I don't think it's a problem in the source code of this plugin, but that the versions are out of sync with what's provided by indygreg.

> proto run python 3.11 --log trace          
[DEBUG 2024-04-12 22:23:59.488] proto  Running proto v0.34.3  bin="proto" args=["run", "python", "3.11", "--log", "trace"] pid=48906
[TRACE 22:23:59.491] starbase::app  Running startup phase with 2 systems 
[DEBUG 22:23:59.493] proto_core::proto:detect_proto_env  Creating proto environment, detecting store  store="/Users/dan/.proto"
[TRACE 22:23:59.494] starbase::app  Running analyze phase with 3 systems 
[TRACE 22:23:59.494] proto_core::proto_config:load_proto_configs  Traversing upwards and loading .prototools files 
[DEBUG 22:23:59.495] proto_core::proto_config:load_proto_configs  Loading .prototools  file="/Users/dan/.proto/.prototools"
[TRACE 22:23:59.495] starbase_utils::fs:load_proto_configs  Opening file  file="/Users/dan/.proto/.prototools"
[TRACE 22:23:59.495] starbase_utils::fs_lock:load_proto_configs  Locking file  file="/Users/dan/.proto/.prototools"
[TRACE 22:23:59.496] starbase_utils::fs_lock:load_proto_configs  Unlocking file  file="/Users/dan/.proto/.prototools"
[TRACE 22:23:59.496] schematic::config::loader:load_proto_configs  Loading partial configuration  config="ProtoConfig"
[TRACE 22:23:59.496] schematic::config::loader:load_proto_configs  Creating layer from source  config="ProtoConfig" source="<code>"
[TRACE 22:23:59.500] schematic::config::loader:load_proto_configs  Merging partial layers into a final result  config="ProtoConfig"
[DEBUG 22:23:59.500] proto_core::proto_config:load_proto_configs  Merging loaded configs 
[DEBUG 22:23:59.500] proto_core::proto_config:load_proto_configs  Merged 1 configs 
[TRACE 22:23:59.500] starbase::app  Running execute phase with 2 systems 
[DEBUG 22:23:59.500] proto_core::tool_loader:run  Finding a configured plugin  tool="python"
[DEBUG 22:23:59.501] proto_core::tool_loader:run  Using a built-in plugin  plugin="source:https://github.com/moonrepo/python-plugin/releases/download/v0.9.0/python_plugin.wasm"
[TRACE 22:23:59.501] warpgate::loader:run  Creating plugin loader  cache_dir="/Users/dan/.proto/plugins"
[TRACE 22:23:59.501] warpgate::loader:run  Loading plugin python  id="python"
[TRACE 22:23:59.502] starbase_utils::fs:run  Reading file metadata  file="/Users/dan/.proto/plugins/python-b9493bb66417130c3a39c64107dfeb41bbdcaba202b3f50466b93ef70c7c4e3e.wasm"
[TRACE 22:23:59.503] warpgate::loader:run  Plugin already downloaded and cached  id="python" path="/Users/dan/.proto/plugins/python-b9493bb66417130c3a39c64107dfeb41bbdcaba202b3f50466b93ef70c7c4e3e.wasm"
[DEBUG 22:23:59.503] proto_core::tool_loader:run  Loading WASM plugin  source="/Users/dan/.proto/plugins/python-b9493bb66417130c3a39c64107dfeb41bbdcaba202b3f50466b93ef70c7c4e3e.wasm"
[TRACE 22:23:59.503] warpgate::plugin:run  Storing plugin identifier  id="python"
[TRACE 22:23:59.503] warpgate::plugin:run  Storing host environment  env={"arch":"arm64","libc":"gnu","os":"macos","home_dir":{"path":"/userhome/","virtual_prefix":"/userhome","real_prefix":"/Users/dan"}}
[DEBUG 22:23:59.503] proto_core::tool:run  Creating tool python and instantiating plugin 
[TRACE 22:23:59.542] warpgate::plugin:run  Created plugin container  id="python" plugin="02d55cbf-5ce9-45cc-9620-615c157a2223"
[DEBUG 22:23:59.542] proto_core::tool:run  Created tool python and its WASM runtime 
[TRACE 22:23:59.542] warpgate::plugin:run  Calling plugin function register_tool  id="python" plugin="02d55cbf-5ce9-45cc-9620-615c157a2223" input={"id":"python"}
[TRACE 22:23:59.544] warpgate::plugin:run  Called plugin function register_tool  id="python" plugin="02d55cbf-5ce9-45cc-9620-615c157a2223" output={"inventory":{},"name":"Python","plugin_version":"0.9.0","type":"Language"}
[DEBUG 22:23:59.544] proto_core::tool_manifest:run  Loading manifest.json  file="/Users/dan/.proto/tools/python/manifest.json"
[TRACE 22:23:59.544] starbase_utils::fs:run  Opening file  file="/Users/dan/.proto/tools/python/manifest.json"
[TRACE 22:23:59.544] starbase_utils::fs_lock:run  Locking file  file="/Users/dan/.proto/tools/python/manifest.json"
[TRACE 22:23:59.544] starbase_utils::fs_lock:run  Unlocking file  file="/Users/dan/.proto/tools/python/manifest.json"
[DEBUG 22:23:59.545] proto_core::version_detector:run  Using explicit version passed on the command line  tool="python" version=Req(VersionReq { comparators: [Comparator { op: Tilde, major: 3, minor: Some(11), patch: None, pre: Prerelease("") }] })
[DEBUG 22:23:59.545] proto_core::tool:run  Resolving a semantic version or alias  tool="python" initial_version="~3.11"
[DEBUG 22:23:59.545] proto_core::tool:run  Loading available versions  tool="python"
[TRACE 22:23:59.545] starbase_utils::fs:run  Reading file metadata  file="/Users/dan/.proto/tools/python/remote-versions.json"
[TRACE 22:23:59.545] starbase_utils::fs:run  Reading file  file="/Users/dan/.proto/tools/python/remote-versions.json"
[TRACE 22:23:59.546] starbase_utils::json:run  Reading JSON file  file="/Users/dan/.proto/tools/python/remote-versions.json"
[TRACE 22:23:59.546] proto_core::version_resolver:run  Resolving a version with manifest  candidate="~3.11"
[TRACE 22:23:59.546] proto_core::version_resolver:run  Found a requirement, resolving further  requirement="~3.11"
[TRACE 22:23:59.546] proto_core::version_resolver:run  Resolved to remote available version  version="3.11.9"
[DEBUG 22:23:59.546] proto_core::tool:run  Resolved to 3.11.9  tool="python" version="3.11.9"
[DEBUG 22:23:59.546] proto_core::tool:run  Checking if tool is installed  tool="python" install_dir="/Users/dan/.proto/tools/python/3.11.9"
[DEBUG 22:23:59.546] proto_core::tool:run  Tool has not been installed  tool="python"
[DEBUG 22:23:59.547] proto::commands::run:run  Auto-install setting is configured, attempting to install 
[DEBUG 22:23:59.547] proto_core::tool:run  Checking if tool is installed  tool="python" install_dir="/Users/dan/.proto/tools/python/3.11.9"
[DEBUG 22:23:59.547] proto_core::tool:run  Tool has not been installed  tool="python"
[DEBUG 22:23:59.547] proto::commands::install:run  Installing Python with version 3.11.9 (from 3.11.9) 
⠁ Installing Python 3.11.9                                                      [TRACE 22:23:59.548] starbase_utils::net:run:is_offline  Checking for an internet connection  timeout=750
━━         Installing Python 3.11.9                                             [TRACE 22:23:59.548] starbase_utils::net::offline  Resolving 1.1.1.1:53 
[TRACE 22:23:59.574] starbase_utils::net:run:is_offline  Online! 
[TRACE 22:23:59.574] starbase_utils::fs_lock:run  Locking directory  dir="/Users/dan/.proto/tools/python/3.11.9"
[TRACE 22:23:59.574] starbase_utils::fs:run  Creating file without truncating  file="/Users/dan/.proto/tools/python/3.11.9/.lock"
[TRACE 22:23:59.574] starbase_utils::fs_lock:run  Waiting to acquire lock on directory  lock="/Users/dan/.proto/tools/python/3.11.9/.lock"
[TRACE 22:23:59.575] starbase_utils::fs_lock:run  Acquired lock on directory, writing PID  lock="/Users/dan/.proto/tools/python/3.11.9/.lock" pid=48906
[DEBUG 22:23:59.575] proto_core::tool:run  Installing tool from a pre-built archive  tool="python"
[DEBUG 22:23:59.575] warpgate::client:run  Creating HTTP client 
━━━━       Installing Python 3.11.9                                             [DEBUG 22:23:59.783] warpgate::client:run  Created HTTP client 
[TRACE 22:23:59.784] warpgate::plugin:run  Calling plugin function download_prebuilt  id="python" plugin="02d55cbf-5ce9-45cc-9620-615c157a2223" input={"context":{"proto_version":"0.34.3","tool_dir":{"path":"/proto/tools/python/3.11.9","virtual_prefix":"/proto","real_prefix":"/Users/dan/.proto"},"version":"3.11.9"},"install_dir":{"path":"/proto/tools/python/3.11.9","virtual_prefix":"/proto","real_prefix":"/Users/dan/.proto"}}
[DEBUG 22:23:59.786] extism::pdk:run  Fetching https://raw.githubusercontent.com/moonrepo/python-plugin/master/releases.json  plugin="02d55cbf-5ce9-45cc-9620-615c157a2223"
━━━━━      Installing Python 3.11.9                                             [TRACE 22:23:59.896] starbase_utils::fs_lock:run  Unlocking directory  dir="/Users/dan/.proto/tools/python/3.11.9"
[TRACE 22:23:59.896] starbase_utils::fs:run  Removing file  file="/Users/dan/.proto/tools/python/3.11.9/.lock"
[TRACE 22:23:59.899] starbase::app  Running shutdown phase with 1 systems 
Error: plugin::call_func::failed

  × No pre-built available for version 3.11.9 (via https://github.com/
  │ indygreg/python-build-standalone)! Try installing another version for the
  │ time being.

[milesj]

The prebuilts don't have 3.11.9 yet (at least I don't see it), only 3.11.8, so this error is correct.

[rotu]

Yes, the error is correct. It's still not-so-great a user experience when nearly every 3.X version fails. Maybe proto should ignore or merely warn on unavailable releases?

> proto run python 3.11
Error: plugin::call_func::failed

  × No pre-built available for version 3.11.9 (via https://github.com/
  │ indygreg/python-build-standalone)! Try installing another version for the
  │ time being.

> proto run python 3.10
Error: plugin::call_func::failed

  × No pre-built available for version 3.10.14 (via https://github.com/
  │ indygreg/python-build-standalone)! Try installing another version for the
  │ time being.

> proto run python 3.9 
Error: plugin::call_func::failed

  × No pre-built available for version 3.9.19 (via https://github.com/
  │ indygreg/python-build-standalone)! Try installing another version for the
  │ time being.

> proto run python 3.8
Error: plugin::call_func::failed

  × No pre-built available for version 3.8.19 (via https://github.com/
  │ indygreg/python-build-standalone)! Try installing another version for the
  │ time being.

[milesj]

The version resolution is isolated from the download, so having the download somehow downlevel the version isn't possible, and would just result in some weirdness. I'm not sure what the best solution is here. Should probably just figure out python build.