PEP 796: Relative Virtual Environments

.github/CODEOWNERS

@@ -673,6 +673,7 @@ peps/pep-0791.rst  @vstinner
 peps/pep-0792.rst  @dstufft
 peps/pep-0793.rst  @encukou
 peps/pep-0794.rst  @brettcannon
+peps/pep-0796.rst  @ncoghlan
 peps/pep-0798.rst  @JelleZijlstra
 peps/pep-0799.rst  @pablogsal
 peps/pep-0800.rst  @JelleZijlstra

peps/pep-0796.rst

@@ -0,0 +1,272 @@
+PEP: 796
+Title: Relative Home Path in Virtual Environments
+Author: Richard Levasseur <[email protected]>
+Sponsor: Alyssa Coghlan <[email protected]>
+Discussions-To: Pending
+Status: Draft
+Type: Standards Track
+Created: 02-Jul-2025
+Python-Version: 3.15
+
+
+Abstract
+========
+
+This PEP formally specifies the use of a relative path for ``home``
+in a Python virtual environment's :file:`pyvenv.cfg` file.
+
+Specifically, we will discuss how such relative paths are understood
+by the Python startup process, including their conversion to absolute
+paths for use by the runtime.
+This is a fundamental building block for virtual environments to
+become more portable.
+
+Motivation
+==========
+
+There are two main motivations for allowing relative paths in ``pyvenv.cfg``.
+
+First, it is currently prescribed that the ``home`` value in ``pyvenv.cfg`` be
+an absolute path (`gh-135773`). The behavior of relative paths is unspecified. While
+techniques exist to work around this for every other sub-part of a virtual
+environment, the one remaining part without a tenable solution is how the
+Python runtime itself finds ``PYTHONHOME``. This is because, currently, the
+startup process requires absolute paths be used for the ``home`` key in
+``pyvenv.cfg``. If a relative path is used, behavior is unspecified (the
+current implementation ends up making it relative to the process's current
+working directory, making it untenable to use).
+
+This requirement is overly proscriptive and restrictive because, given a known
+anchor point, it's easy to transform a relative path to an absolute path and
+still retain predictable and reliable behavior. Thus, the absolute path
+requirement should be relaxed and relative path behavior allowed and defined.
+
+Second, such relative paths are a building block to enable portable virtual
+environments, i.e. copying a virtual environment as-is between hosts of
+compatible platforms. For example, by pointing to a parent directory, the
+virtual environment becomes independent of path prefix differences between
+hosts (e.g. ``/usr/local/`` in a container vs
+``/home/user/.pyenv/versions/3.12.0/bin`` in a user's dev environment).
+
+Portable virtual environments are appealing because virtual environments are a
+popular mechanism for running Python applications. This provides several
+benefits:
+
+* The closer the development environment is to the non-development environment,
+  environment-specific issues are less likely, and the easier it is to
+  reproduce issues.
+* The simpler the process of re-creating the environment, environment-specific
+  issues are less likely, and the faster the process can be.
+
+Making it simpler to copy a virtual environment from one host to another
+mitigates these categories of problems. Additionally, the development tools to
+create a virtual environment and install its dependencies aren't needed on the
+host that intends to run the program.
+
+When the virtual environment doesn't require modifications to be usable, it
+also allows more advanced deployment mechanisms, e.g. remote mounting and
+caching of artifacts. While this PEP on its own isn't sufficient to enable
+that, it allows tools like Bazel or venvstacks to more easily prepare
+constrained environments that allow for such use cases.
+
+Rationale
+=========
+
+The reason support for relative virtual environments needs to be
+in the interpreter itself is because locating ``PYTHONHOME`` happens
+very early in the interpreter startup process, which limits the options for
+customizing how it's computed. Without the ability to specify where the
+supporting Python runtime files are, the interpreter can't finish startup,
+so other hook points (e.g. ``site`` initialization) never trigger.
+
+Tools that currently look to enable virtual environment portability across
+machines do so either by relying on undocumented interpreter behaviour
+(Bazel, omitting the ``home`` key entirely to trigger an implementation
+dependent fallback to resolving via a symlinked interpreter binary on
+non-Windows systems, see `gh-135773`) or by requiring a post-installation script to be executed
+after the environment is placed in its target location (venvstacks).
+
+Specification
+=============
+
+The ``home`` value in ``pyvenv.cfg`` is permitted to use a relative path value.
+These may contain parent-directory references outside of the virtual environment root
+directory.
+For example:
+
+* ``subdir/whatever/bin`` (a directory within the virtual environment)
+* ``./subdir/whatever/bin`` (same as above)
+* ``../../../../../elsewhere/runtime/bin`` (a directory outside the virtual
+  environment)
+
+Relative paths are relative to the directory containing :file:`pyvenv.cfg`.
+During interpreter startup (i.e. :file:`getpath.py`), the relative path is joined to the
+directory containing ``pyvenv.cfg`` to form an absolute path.
+Parent-directory references (``../``) and current
+directory references (``./``) are resolved syntactically (i.e. not resolving
+symlinks). Symlinks are *not* resolved prior to construction of the absolute
+path to ensure semantics between a relative path and absolute path remain the
+same.
+
+For example, given
+``/home/user/venv/bin/pyvenv.cfg`` with
+``home = ../../runtime/./bin``, the result is ``home = /home/user/runtime/bin``,
+i.e. it's equivalent to using that value verbatim in ``pyvenv.cfg``.
+
+
+CPython Runtime Changes
+-----------------------
+
+The CPython runtime itself *almost* already supports relative paths. The
+primitives are there, so the only change needed is to define how it resolves
+relative paths for ``home`` in ``pyvenv.cfg``.
+
+Currently, relative paths resolve relative to the process's current working
+directory. Because the current working directory isn't knowable in advance, it
+makes relative paths today effectively impossible.
+
+Instead, the paths should be relative to the location of the ``pyvenv.cfg``
+file. This file is chosen as the anchor point because the tool that creates the
+file also has to know where the Python runtime is, so can easily calculate the
+correct relative path. For tools that read the ``pyvenv.cfg``, it is also easy
+to simply join the directory name of where ``pyvenv.cfg`` was found with the
+path in the config file. When a person reads the config file, they can do
+something similar, which results in a lower cognitive burden and helps avoid
+the question of "relative to what?"
+
+This change is only a couple of lines in the startup code. Specifically, when
+parsing the ``pyvenv.cfg`` file and finding the ``home`` value, it just needs
+to be checked if it's already absolute. If not, then join it to the directory
+name of the ``pyvenv.cfg`` file. The code already knows the directory and has
+helpers already exist for checking if a path is absolute and joining two
+paths.
+
+A proof-of-concept of this is implemented in the author's branch,
+`rickeylev/feat.relative.pyvenv.home <https://github.com/python/cpython/compare/main...rickeylev:cpython:feat.relative.pyvenv.home>`__.
+
+Backwards Compatibility
+=======================
+
+Tools that work around the absolute ``home`` key limitation the way Bazel
+and venvstacks currently do (omitting the ``home`` key, or editing it after
+moving the environment) will be unaffected.
+
+While the PEP author and sponsor aren't aware of any projects that work around
+the limitation by carefully controlling the current working directory used to
+launch the deployed Python environments on target systems, any such projects
+would be unaffected if they already ensured the working directory was set to
+the folder containing ``pyvenv.cfg`` (which seems like a plausible choice,
+since that is typically the root directory of the virtual environment). In the
+even more unlikely case where that assumption doesn't hold, tools generating
+relative virtual environment paths will typically be aware of the underlying
+base runtime Python version, and hence able to update the emitted relative path
+accordingly.
+
+
+How to Teach This
+=================
+
+Teaching this should be straightforward: if you use a relative path in
+``pyvenv.cfg``, then it's relative to the directory containing the
+``pyvenv.cfg`` file. This is simple to explain and easy to understand for
+anyone that is already familiar with handling relative filesystem paths.
+
+
+Reference Implementation
+========================
+
+A reference implementation is available by using the combination of:
+
+* Python runtime from `rickeylev/feat.relative.pyvenv.home <https://github.com/python/cpython/compare/main...rickeylev:cpython:feat.relative.pyvenv.home>`__
+* Relative venv from `rickeylev/relvenv <https://github.com/rickeylev/relvenv>`__
+
+And following the
+`relvenv README <https://github.com/rickeylev/relvenv/blob/main/README.md>`__.
+
+Open Issues
+===========
+
+This PEP does not specify how to create a ``pyvenv.cfg`` with a relative path,
+nor how downstream tools (e.g. installers) should identify them or process
+them. These questions are best addressed separately by tool owners.
+
+References
+==========
+
+* `rules_python <https://github.com/bazel-contrib/rules_python>`__: implements
+  host-relocatable virtual environments.
+* `rules_py <https://github.com/aspect-build/rules_py>`__: implements
+  host-relocatable virtual environments.
+* `python-build-standalone <https://github.com/astral-sh/python-build-standalone>`__
+* `venvstacks <https://pypi.org/project/venvstacks/>`__: a tool for creating
+  reproducible distribution artifacts from virtual environments A relocatable
+  Python runtime.
+* `PoC for relative home in Python startup <https://github.com/python/cpython/compare/main...rickeylev:cpython:feat.relative.pyvenv.home>`__
+* `Python Ideas "Making venvs relocatable friendly" discussion <https://discuss.python.org/t/making-venvs-relocatable-friendly/96177>`__
+* `gh-136051: relative pyvenv.cfg home <https://github.com/python/cpython/issues/136051>`__
+
+Rejected Ideas
+==============
+
+Relative to virtual env root
+----------------------------
+
+Having the ``home`` value in ``pyvenv.cfg`` relative to the virtual
+environment's root directory would work just as well, but this idea is rejected
+because it requires additional effort to compute the virtual env root.
+
+Unspecified home means to dynamically compute home
+----------------------------------------------------
+
+Today, if a ``pyvenv.cfg`` file doesn't set ``home``, the runtime will try to
+dynamically compute it by checking if the current executable (which is
+typically the venv's ``bin/python3`` symlink) is a symlink and, if so, use
+where that points as ``PYTHONHOME``.
+
+While currently used as a workaround by some tools, *standardising* this
+behavior is undesirable for a couple reasons:
+
+1. It presents platform-specific issues, namely with Windows. Windows does
+   support symlinks, but not by default, and it can require special
+   permissions to do so.
+2. It *requires* that a symlink be used, which precludes using otherwise
+   equivalent mechanisms for creating an executable (e.g. a wrapper script,
+   hard links, etc).
+
+In general, symlinks work best when they aren't special cased by consumers.
+
+Using the term "relocatable"
+----------------------------
+
+Discussions pointed out that the term "relocatable" is somewhat ambiguous and
+misleading for a couple reasons.
+
+First, absolute paths make a venv arbitrarily relocatable *within* a host, but
+not between hosts, so "relocatable" requires *some* qualification for
+clarity.
+
+Second, when using relative paths that point outside the venv, the venv is only
+relocatable insofar as those external artifacts are also relocated. This is an
+additional nuance that requires qualification of the term.
+
+To better avoid this confusion, "relative" is chosen, which more naturally
+invites the question *"Relative to what?"*.
+
+
+Using PYTHONHOME at runtime to specify home
+-------------------------------------------
+
+Using the ``PYTHONHOME`` environment variable (or any environment variable) is
+problematic because it's difficult to know and control when an environment
+variable should or shouldn't be inherited by subprocesses. In some cases, it's
+not feasible because of how layers of programs calling programs interact.
+
+Code generally assumes that any virtual environment will be
+automatically detected and activated by the presence of ``pyvenv.cfg``, so
+things work better when alterations to the environment aren't a concern.
+
+Copyright
+=========
+
+This document is placed in the public domain or under the
+CC0-1.0-Universal license, whichever is more permissive.