SNOW-2174388: Remove upper bound for python from package metadata

[a-recknagel]

1. What version of Python are you using?

   Python 3.12.1 (main, Jan 26 2024, 16:46:43) [GCC 11.3.0]

2. What operating system and processor architecture are you using?

   Linux-6.6.87.2-microsoft-standard-WSL2-x86_64-with-glibc2.35

3. What did you do?

   I'm using poetry to manage my python environment. I tried to install snowflake-snowpark-python into a project that has, among other things, the following section in its pyproject.toml

[tool.poetry.dependencies]
python = ">=3.10"

When I run poetry add snowflake-snowpark-python, I'm getting a dependency resolution error:

Resolving dependencies... (5.5s)

The current project's supported Python range (>=3.10) is not compatible with some of the required packages Python requirement:
  - snowflake-snowpark-python requires Python <3.13,>=3.9, so it will not be installable for Python >=3.13

Because no versions of snowflake-snowpark-python match >1.33.0,<2.0.0
 and snowflake-snowpark-python (1.33.0) requires Python <3.13,>=3.9, snowflake-snowpark-python is forbidden.
So, because auxiliary depends on snowflake-snowpark-python (^1.33.0), version solving failed.

  * Check your dependencies Python requirement: The Python requirement can be specified via the `python` or `markers` properties

    For snowflake-snowpark-python, a possible solution would be to set the `python` property to ">=3.10,<3.13"

5. What did you expect to see?

   Given that I'm running python 3.12 which works for both my versions constraints as well as snowpark's, I expected a valid version to get installed.

I'm not entirely sure if this issue is on poetry end or on yours, but either way, putting an upper bound for allowed versions on a package which may be used as a library in other distributions is an antipattern. Would it be possible to remove it? It's not like it does anything useful.

[a-recknagel]

Just in case, the reason why I'm not using the fix suggested by poetry of aligning my project with your version constraint is that I don't want to. I'd need to update a bunch of downstream apps and configs as well, and I'd just rather not.

[sfc-gh-sghosh]

Hello @a-recknagel ,

Thanks for raising the issue.
Could you check the compatibility using conda command

conda create --name py310_env --override-channels -c https://repo.anaconda.com/pkgs/snowflake python=3.10 snowflake-snowpark-python poetry

Its using following for python 3.10

snowflake-snowpark-python 1.32.0
poetry 1.1.7
poetry-core 1.5.2

For other python version such as 3.12, poetry is not compatible.

You can try pip if you want to use python 3.12
Example
conda create --name py312_env --override-channels -c https://repo.anaconda.com/pkgs/snowflake python=3.12 snowflake-snowpark-python
conda activate py312_env
pip install poetry

[a-recknagel]

Hi @sfc-gh-sghosh , thanks for answering. I don't mean to install poetry as a dependency, I'm using it as my package manager instead of conda/pip in order to set up my project, and is not installed into the same site-packages as snowflake-snowpark-python.

If it helps, building this Dockerfile reproduces my issue at hand:

FROM python:3.12

RUN curl -sSL https://install.python-poetry.org | python -        # installing poetry
RUN /root/.local/bin/poetry init -n --name foo --python '>=3.10'  # setting up a generic python project 
RUN /root/.local/bin/poetry add snowflake-snowpark-python         # adding snowpark as a depedency

I'm getting

docker build -t snow-test .                                                                                      
[+] Building 3.2s (7/7) FINISHED                                                                         docker:default
 => [internal] load build definition from Dockerfile                                                               0.0s
 => => transferring dockerfile: 239B                                                                               0.0s
 => [internal] load metadata for docker.io/library/python:3.12                                                     0.0s
 => [internal] load .dockerignore                                                                                  0.0s
 => => transferring context: 2B                                                                                    0.0s
 => [1/4] FROM docker.io/library/python:3.12                                                                       0.0s
 => CACHED [2/4] RUN curl -sSL https://install.python-poetry.org | python -                                        0.0s
 => [3/4] RUN /root/.local/bin/poetry init -n --name foo --python '>=3.10'                                         0.9s
 => ERROR [4/4] RUN /root/.local/bin/poetry add snowflake-snowpark-python                                          2.1s
------
 > [4/4] RUN /root/.local/bin/poetry add snowflake-snowpark-python:
0.867 Creating virtualenv foo-il7asoJj-py3.12 in /root/.cache/pypoetry/virtualenvs
1.732 Using version ^1.33.0 for snowflake-snowpark-python
1.733
1.733 Updating dependencies
1.734 Resolving dependencies...
1.836
1.836 The current project's supported Python range (>=3.10) is not compatible with some of the required packages Python requirement:
1.836   - snowflake-snowpark-python requires Python <3.13,>=3.9, so it will not be installable for Python >=3.13
1.836
1.836 Because no versions of snowflake-snowpark-python match >1.33.0,<2.0.0
1.836  and snowflake-snowpark-python (1.33.0) requires Python <3.13,>=3.9, snowflake-snowpark-python is forbidden.
1.836 So, because foo depends on snowflake-snowpark-python (^1.33.0), version solving failed.
1.836
1.836   * Check your dependencies Python requirement: The Python requirement can be specified via the `python` or `markers` properties
1.836
1.836     For snowflake-snowpark-python, a possible solution would be to set the `python` property to ">=3.10,<3.13"
1.836
1.836     https://python-poetry.org/docs/dependency-specification/#python-restricted-dependencies,
1.836     https://python-poetry.org/docs/dependency-specification/#using-environment-markers
1.836
------
Dockerfile:5
--------------------
   3 |     RUN curl -sSL https://install.python-poetry.org | python -
   4 |     RUN /root/.local/bin/poetry init -n --name foo --python '>=3.10'
   5 | >>> RUN /root/.local/bin/poetry add snowflake-snowpark-python
   6 |
--------------------
ERROR: failed to solve: process "/bin/sh -c /root/.local/bin/poetry add snowflake-snowpark-python" did not complete successfully: exit code: 1

[sfc-gh-sghosh]

Hi @a-recknagel ,

Thanks for the update, could you please use the following dockerfile and try again, its working fine, no error.

dockerfile

`FROM python:3.12

RUN curl -sSL https://install.python-poetry.org | python3 -

ENV PATH="/root/.local/bin:$PATH"

WORKDIR /app

RUN touch README.md

RUN poetry init --name snowpark_project
--dependency "snowflake-snowpark-python>=1.32.0,<2.0.0"
--python ">=3.12,<3.13"
--no-interaction

RUN poetry install --no-root
`

View build details: docker-desktop://dashboard/build/desktop-linux/desktop-linux/b62hvy5pmeetzh6n623ov6dcl
(base) sghosh@C02ZM0X4LVCG my-snowpark-docker_poetry % docker build -t my-snowpark-poetry .
[+] Building 23.8s (11/11) FINISHED docker:desktop-linux
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 859B 0.0s
=> [internal] load metadata for docker.io/library/python:3.12 3.0s
=> [auth] library/python:pull token for registry-1.docker.io 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [1/6] FROM docker.io/library/python:3.12@sha256:77a36ff63e657d8ec7cd4e86e452f4cd23b6c92811696b0735226fbc0660a5b8 0.0s
=> CACHED [2/6] RUN curl -sSL https://install.python-poetry.org | python3 - 0.0s
=> CACHED [3/6] WORKDIR /app 0.0s
=> CACHED [4/6] RUN touch README.md 0.0s
=> [5/6] RUN poetry init --name snowpark_project --dependency "snowflake-snowpark-python>=1.32.0,<2.0.0" --python ">= 2.1s
=> [6/6] RUN poetry install --no-root 16.5s
=> exporting to image 2.1s
=> => exporting layers 2.1s
=> => writing image sha256:af811b8eb93452e49d2b6f9ca2c9cd67adf1ba204d2bbba1f8e1c491d5779fee 0.0s
=> => naming to docker.io/library/my-snowpark-poetry 0.0s

View build details: docker-desktop://dashboard/build/desktop-linux/desktop-linux/vk92i0qz76cpgcqrftwydujwj

[a-recknagel]

Hi @sfc-gh-sghosh, and sorry, I must have missed your last post. As I said before

Just in case, the reason why I'm not using the fix suggested by poetry of aligning my project with your version constraint is that I don't want to. I'd need to update a bunch of downstream apps and configs as well, and I'd just rather not.

So changing the python version from 3.10 to 3.12 does not solve my issue. I want the minimum requirement to be 3.10, it's and important part of the project, even if I'm developing on 3.12. The only reason why I can't do that is because you guys are putting an upper bound on your library for no apparent gain. If you change the line

REQUIRED_PYTHON_VERSION = ">=3.9, <3.13"

in your setup.py to

REQUIRED_PYTHON_VERSION = ">=3.9"

I wouldn't be having that issue. Which is also something you have to do for your runtime tests anyway.

I don't have the permissions to reopen the issue. Should I create a new one?