Skip to content

Use flit-core to build pip distributions #13473

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Aug 8, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ jobs:
- run: pip install nox
- run: nox -s prepare-release -- 99.9
- run: nox -s build-release -- 99.9
- run: pipx run check-manifest
- run: pipx run check-sdist

vendoring:
name: vendoring
Expand Down
34 changes: 0 additions & 34 deletions MANIFEST.in

This file was deleted.

2 changes: 1 addition & 1 deletion build-project/build-requirements.in
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
build
setuptools
flit-core
16 changes: 7 additions & 9 deletions build-project/build-requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -8,17 +8,15 @@ build==1.3.0 \
--hash=sha256:698edd0ea270bde950f53aed21f3a0135672206f3911e0176261a31e0e07b397 \
--hash=sha256:7145f0b5061ba90a1500d60bd1b13ca0a8a4cebdd0cc16ed8adf1c0e739f43b4
# via -r build-requirements.in
packaging==24.2 \
--hash=sha256:09abb1bccd265c01f4a3aa3f7a7db064b36514d2cba19a2f694fe6150451a759 \
--hash=sha256:c228a6dc5e932d346bc5739379109d49e8853dd8223571c7c5b55260edc0b97f
flit-core==3.12.0 \
--hash=sha256:18f63100d6f94385c6ed57a72073443e1a71a4acb4339491615d0f16d6ff01b2 \
--hash=sha256:e7a0304069ea895172e3c7bb703292e992c5d1555dd1233ab7b5621b5b69e62c
# via -r build-requirements.in
packaging==25.0 \
--hash=sha256:29572ef2b1f17581046b3a2227d5c611fb25ec70ca1ba8554b24b0e69331a484 \
--hash=sha256:d443872c98d677bf60f6a1f2f8c1cb748e8fe762d2bf9d3148b5599295b0fc4f
# via build
pyproject-hooks==1.2.0 \
--hash=sha256:1e859bd5c40fae9448642dd871adf459e5e2084186e8d2c2a79a824c970da1f8 \
--hash=sha256:9e5c6bfa8dcc30091c74b0cf803c81fdd29d94f01992a7707bc97babb1141913
# via build

# The following packages are considered to be unsafe in a requirements file:
setuptools==80.9.0 \
--hash=sha256:062d34222ad13e0cc312a4c02d73f059e86a4acbfbdea8f8f76b28c99f306922 \
--hash=sha256:f36b47402ecde768dbfafc46e8e4207b4360c654f1f3bb84475f0a28628fb19c
# via -r build-requirements.in
1 change: 0 additions & 1 deletion docs/html/development/architecture/anatomy.rst
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,6 @@ The ``README``, license, ``pyproject.toml``, and so on are in the top level.

* ``AUTHORS.txt``
* ``LICENSE.txt``
* ``MANIFEST.in``
* ``NEWS.rst``
* ``pyproject.toml``
* ``README.rst``
Expand Down
2 changes: 2 additions & 0 deletions news/13743.feature.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Building pip itself from source now uses flit-core instead of setuptools.
This does not affect how pip installs or builds packages you use.
59 changes: 34 additions & 25 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
[project]
dynamic = ["version"]

name = "pip"
description = "The PyPA recommended tool for installing Python packages."
readme = "README.rst"
Expand Down Expand Up @@ -46,12 +45,13 @@ Source = "https://github.com/pypa/pip"
Changelog = "https://pip.pypa.io/en/stable/news/"

[build-system]
requires = ["setuptools>=77"]
build-backend = "setuptools.build_meta"
requires = ["flit-core >=3.11,<4"]
build-backend = "flit_core.buildapi"

[dependency-groups]
test = [
"cryptography",
"flit-core >= 3.11, < 4",
"freezegun",
"installer",
# pytest-subket requires 7.0+
Expand All @@ -73,6 +73,7 @@ test = [
]

test-common-wheels = [
"flit-core >= 3.11, < 4",
# We pin setuptools<80 because our test suite currently
# depends on setup.py develop to generate egg-link files.
"setuptools >= 40.8.0, != 60.6.0, <80",
Expand All @@ -82,28 +83,24 @@ test-common-wheels = [
"pytest-subket >= 0.8.1",
]

[tool.setuptools]
package-dir = {"" = "src"}
include-package-data = false

[tool.setuptools.dynamic]
version = {attr = "pip.__version__"}

[tool.setuptools.packages.find]
where = ["src"]
exclude = ["contrib", "docs", "tests*", "tasks"]

[tool.setuptools.package-data]
"pip" = ["py.typed"]
"pip._vendor" = ["vendor.txt"]
"pip._vendor.certifi" = ["*.pem"]
"pip._vendor.distlib" = [
"t32.exe",
"t64.exe",
"t64-arm.exe",
"w32.exe",
"w64.exe",
"w64-arm.exe",
[tool.flit.sdist]
include = [
"NEWS.rst",
"SECURITY.md",
"build-project/.python-version",
"build-project/build-project.py",
"build-project/build-requirements.in",
"build-project/build-requirements.txt",
"docs/requirements.txt",
"docs/**/*.css",
"docs/**/*.dot",
"docs/**/*.md",
"docs/**/*.png",
"docs/**/*.py",
"docs/**/*.rst",
]
exclude = [
"src/pip/_vendor/**/*.pyi",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why excluding this from the sdist?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That pattern was added in #4545

It's probably not doing anything anymore? I'll take a look later and remove it if redundant.

Copy link
Member Author

@notatallshaw notatallshaw Aug 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Vendoring generates stub files for packages which don't include a py.typed: https://github.com/pradyunsg/vendoring/blob/1.2.0/src/vendoring/tasks/stubs.py#L43-L52

Which puts pyi files here: https://github.com/pypa/pip/tree/25.2/src/pip/_vendor

What's interesting about this, is it doesn't seem to work, at least mypy doesn't find the correct type hints for requests: #13476

I will try and investigate this, but I think the solution will be that we need to update vendoring to stop doing this, or do something more useful?

Copy link
Member Author

@notatallshaw notatallshaw Aug 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Vendoring does have some config that allows us to remove or rename these type stub files, which I may take advantage of in a different PR: https://github.com/pypa/pip/blob/25.2/pyproject.toml#L182

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the investigation. This does not sound blocking for this PR anyway.

]

######################################################################################
Expand Down Expand Up @@ -362,3 +359,15 @@ exclude_also = [
# This excludes typing-specific code, which will be validated by mypy anyway.
"if TYPE_CHECKING",
]

[tool.check-sdist]
git-only = [
"tests/**",
"tools/**",
"news/.gitignore",
".gitattributes",
".gitignore",
".git-blame-ignore-revs",
".mailmap",
".readthedocs-custom-redirects.yml"
]
41 changes: 18 additions & 23 deletions tests/functional/test_freeze.py
Original file line number Diff line number Diff line change
Expand Up @@ -99,38 +99,33 @@ def test_freeze_with_pip(script: PipTestEnvironment) -> None:

def test_freeze_with_setuptools(script: PipTestEnvironment) -> None:
"""
Test that pip shows setuptools only when --all is used
or _should_suppress_build_backends() returns false
Test that pip shows setuptools only when --all is used on Python < 3.12,
otherwise it should be shown in default freeze output.
"""

result = script.pip("freeze", "--all")
assert "setuptools==" in result.stdout

(script.site_packages_path / "mock.pth").write_text("import mock\n")

(script.site_packages_path / "mock.py").write_text(
textwrap.dedent(
"""\
import pip._internal.commands.freeze as freeze
freeze._should_suppress_build_backends = lambda: False
"""
)
)

# Test the default behavior (without --all)
result = script.pip("freeze")
assert "setuptools==" in result.stdout

(script.site_packages_path / "mock.py").write_text(
textwrap.dedent(
"""\
import pip._internal.commands.freeze as freeze
freeze._should_suppress_build_backends = lambda: True
"""
should_suppress = sys.version_info < (3, 12)
if should_suppress:
# setuptools should be hidden in default freeze output
assert "setuptools==" not in result.stdout, (
f"setuptools should be suppressed in Python {sys.version_info[:2]} "
f"but was found in freeze output: {result.stdout}"
)
else:
# setuptools should be shown in default freeze output
assert "setuptools==" in result.stdout, (
f"setuptools should be shown in Python {sys.version_info[:2]} "
f"but was not found in freeze output: {result.stdout}"
)
)

result = script.pip("freeze")
assert "setuptools==" not in result.stdout
# --all should always show setuptools regardless of version
result_all = script.pip("freeze", "--all")
assert "setuptools==" in result_all.stdout


def test_exclude_and_normalization(script: PipTestEnvironment, tmpdir: Path) -> None:
Expand Down
6 changes: 5 additions & 1 deletion tests/functional/test_self_update.py
Original file line number Diff line number Diff line change
@@ -1,13 +1,17 @@
# Check that pip can update itself correctly

from pathlib import Path
from typing import Any


def test_self_update_editable(script: Any, pip_src: Any) -> None:
def test_self_update_editable(script: Any, pip_src: Any, common_wheels: Path) -> None:
# Test that if we have an environment with pip installed in non-editable
# mode, that pip can safely update itself to an editable install.
# See https://github.com/pypa/pip/issues/12666 for details.

# Install flit-core (build backend) since we use --no-build-isolation
script.pip("install", "--no-index", "-f", common_wheels, "flit-core")

# Step 1. Install pip as non-editable. This is expected to succeed as
# the existing pip in the environment is installed in editable mode, so
# it only places a .pth file in the environment.
Expand Down
Loading