Skip to content

bpo-41011: venv -- add more variables to pyvenv.cfg (GH-30382) #30382

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 17 commits into from
Jan 7, 2022
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 37 additions & 1 deletion Lib/test/test_venv.py
Original file line number Diff line number Diff line change
@@ -19,7 +19,7 @@
from test.support.os_helper import (can_symlink, EnvironmentVarGuard, rmtree)
import unittest
import venv
from unittest.mock import patch
from unittest.mock import patch, Mock

try:
import ctypes
@@ -114,13 +114,49 @@ def test_defaults(self):
executable = sys._base_executable
path = os.path.dirname(executable)
self.assertIn('home = %s' % path, data)
self.assertIn('executable = %s' %
os.path.realpath(sys.executable), data)
copies = '' if os.name=='nt' else ' --copies'
cmd = f'command = {sys.executable} -m venv{copies} --without-pip {self.env_dir}'
self.assertIn(cmd, data)
fn = self.get_env_file(self.bindir, self.exe)
if not os.path.exists(fn): # diagnostics for Windows buildbot failures
bd = self.get_env_file(self.bindir)
print('Contents of %r:' % bd)
print(' %r' % os.listdir(bd))
self.assertTrue(os.path.exists(fn), 'File %r should exist.' % fn)

def test_config_file_command_key(self):
attrs = [
(None, None),
('symlinks', '--copies'),
('with_pip', '--without-pip'),
('system_site_packages', '--system-site-packages'),
('clear', '--clear'),
('upgrade', '--upgrade'),
('upgrade_deps', '--upgrade-deps'),
('prompt', '--prompt'),
]
for attr, opt in attrs:
rmtree(self.env_dir)
if not attr:
b = venv.EnvBuilder()
else:
b = venv.EnvBuilder(
**{attr: False if attr in ('with_pip', 'symlinks') else True})
b.upgrade_dependencies = Mock() # avoid pip command to upgrade deps
b._setup_pip = Mock() # avoid pip setup
self.run_with_capture(b.create, self.env_dir)
data = self.get_text_file_contents('pyvenv.cfg')
if not attr:
for opt in ('--system-site-packages', '--clear', '--upgrade',
'--upgrade-deps', '--prompt'):
self.assertNotRegex(data, rf'command = .* {opt}')
elif os.name=='nt' and attr=='symlinks':
pass
else:
self.assertRegex(data, rf'command = .* {opt}')

def test_prompt(self):
env_name = os.path.split(self.env_dir)[1]

24 changes: 24 additions & 0 deletions Lib/venv/__init__.py
Original file line number Diff line number Diff line change
@@ -51,6 +51,7 @@ def __init__(self, system_site_packages=False, clear=False,
self.symlinks = symlinks
self.upgrade = upgrade
self.with_pip = with_pip
self.orig_prompt = prompt
if prompt == '.': # see bpo-38901
prompt = os.path.basename(os.getcwd())
self.prompt = prompt
@@ -178,6 +179,29 @@ def create_configuration(self, context):
f.write('version = %d.%d.%d\n' % sys.version_info[:3])
if self.prompt is not None:
f.write(f'prompt = {self.prompt!r}\n')
f.write('executable = %s\n' % os.path.realpath(sys.executable))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(I appreciate that this PR is long since merged, but just following-up on a surprising pyvenv.cfg file that I saw recently, and git-blamed it back to here)

I'm not at all sure why realpath is important here - the sys.executable should be good enough when being called directly. When called through the EnvBuilder, I don't see a reason for this either. 🤔

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, it is a difference when you're not on Windows. But thinking about it, you could view the config as recording the actual interpreter used instead of the path that could change on you in the future (e.g., some symlink later gets changed to point to another interpreter).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this perspective (I hadn't considered it)! I'm not sure that using a config file to log information about how a venv was created is a good idea, but can appreciate the idea. Since this is a machine readable file, I would be concerned that it eventually gets used for non-log/audit purposes.

I've been digging into symlink issues with venv a lot recently, and think calling realpath is quite problematic in other context (i.e. when determining home). More details of that in #106045.

No follow-up needed here - just flagging it, and depending on the outcome of the linked issue, perhaps in the future I may request that this information become non realpath, or under a "history of how this venv was created" section of the config (perhaps in the form of a comment).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

args = []
nt = os.name == 'nt'
if nt and self.symlinks:
args.append('--symlinks')
if not nt and not self.symlinks:
args.append('--copies')
if not self.with_pip:
args.append('--without-pip')
if self.system_site_packages:
args.append('--system-site-packages')
if self.clear:
args.append('--clear')
if self.upgrade:
args.append('--upgrade')
if self.upgrade_deps:
args.append('--upgrade-deps')
if self.orig_prompt is not None:
args.append(f'--prompt="{self.orig_prompt}"')

args.append(context.env_dir)
args = ' '.join(args)
f.write(f'command = {sys.executable} -m venv {args}\n')

if os.name != 'nt':
def symlink_or_copy(self, src, dst, relative_symlinks_ok=False):
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Added two new variables to *pyvenv.cfg* which is generated by :mod:`venv`
module: *executable* for the executable and *command* for the command line used
to create the environment.