Merge branch 'main' into typewatch

* main:
  pythonGH-97002: Prevent `_PyInterpreterFrame`s from backing more than one `PyFrameObject` (pythonGH-97996)
  pythongh-97973: Return all necessary information from the tokenizer (pythonGH-97984)
  fixes pythongh-96078: os.sched_yield release the GIL while calling sched_yield(2). (pythongh-97965)
  pythongh-65961: Do not rely solely on `__cached__` (pythonGH-97990)
  pythongh-97850: Remove the open issues section from the import reference (python#97935)
  Docs: pin sphinx-lint (pythonGH-97992)
  pythongh-94590: add signatures to operator itemgetter, attrgetter, methodcaller (python#94591)
  Add Pynche's move to the What's new in 3.11 (python#97974)
  pythongh-97781: Apply changes from importlib_metadata 5. (pythonGH-97785)
  pythongh-86482: Document assignment expression need for ()s (python#23291)
  pythongh-97943: PyFunction_GetAnnotations should return a borrowed reference. (python#97949)

Author: carljm

Doc/c-api/import.rst

@@ -150,6 +150,11 @@ Importing Modules
    See also :c:func:`PyImport_ExecCodeModuleEx` and
    :c:func:`PyImport_ExecCodeModuleWithPathnames`.
 
+   .. versionchanged:: 3.12
+      The setting of :attr:`__cached__` and :attr:`__loader__` is
+      deprecated. See :class:`~importlib.machinery.ModuleSpec` for
+      alternatives.
+
 
 .. c:function:: PyObject* PyImport_ExecCodeModuleEx(const char *name, PyObject *co, const char *pathname)
 
@@ -167,6 +172,10 @@ Importing Modules
 
    .. versionadded:: 3.3
 
+   .. versionchanged:: 3.12
+      Setting :attr:`__cached__` is deprecated. See
+      :class:`~importlib.machinery.ModuleSpec` for alternatives.
+
 
 .. c:function:: PyObject* PyImport_ExecCodeModuleWithPathnames(const char *name, PyObject *co, const char *pathname, const char *cpathname)
 

Doc/library/importlib.metadata.rst

@@ -13,21 +13,39 @@
 
 **Source code:** :source:`Lib/importlib/metadata/__init__.py`
 
-``importlib.metadata`` is a library that provides access to installed
-package metadata, such as its entry points or its
-top-level name.  Built in part on Python's import system, this library
+``importlib_metadata`` is a library that provides access to
+the metadata of an installed `Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_,
+such as its entry points
+or its top-level names (`Import Package <https://packaging.python.org/en/latest/glossary/#term-Import-Package>`_\s, modules, if any).
+Built in part on Python's import system, this library
 intends to replace similar functionality in the `entry point
 API`_ and `metadata API`_ of ``pkg_resources``.  Along with
 :mod:`importlib.resources`,
 this package can eliminate the need to use the older and less efficient
 ``pkg_resources`` package.
 
-By "installed package" we generally mean a third-party package installed into
-Python's ``site-packages`` directory via tools such as `pip
-<https://pypi.org/project/pip/>`_.  Specifically,
-it means a package with either a discoverable ``dist-info`` or ``egg-info``
-directory, and metadata defined by :pep:`566` or its older specifications.
-By default, package metadata can live on the file system or in zip archives on
+``importlib_metadata`` operates on third-party *distribution packages*
+installed into Python's ``site-packages`` directory via tools such as
+`pip <https://pypi.org/project/pip/>`_.
+Specifically, it works with distributions with discoverable
+``dist-info`` or ``egg-info`` directories,
+and metadata defined by the `Core metadata specifications <https://packaging.python.org/en/latest/specifications/core-metadata/#core-metadata>`_.
+
+.. important::
+
+   These are *not* necessarily equivalent to or correspond 1:1 with
+   the top-level *import package* names
+   that can be imported inside Python code.
+   One *distribution package* can contain multiple *import packages*
+   (and single modules),
+   and one top-level *import package*
+   may map to multiple *distribution packages*
+   if it is a namespace package.
+   You can use :ref:`package_distributions() <package-distributions>`
+   to get a mapping between them.
+
+By default, distribution metadata can live on the file system
+or in zip archives on
 :data:`sys.path`.  Through an extension mechanism, the metadata can live almost
 anywhere.
 
@@ -37,12 +55,19 @@ anywhere.
    https://importlib-metadata.readthedocs.io/
       The documentation for ``importlib_metadata``, which supplies a
       backport of ``importlib.metadata``.
+      This includes an `API reference
+      <https://importlib-metadata.readthedocs.io/en/latest/api.html>`__
+      for this module's classes and functions,
+      as well as a `migration guide
+      <https://importlib-metadata.readthedocs.io/en/latest/migration.html>`__
+      for existing users of ``pkg_resources``.
 
 
 Overview
 ========
 
-Let's say you wanted to get the version string for a package you've installed
+Let's say you wanted to get the version string for a
+`Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_ you've installed
 using ``pip``.  We start by creating a virtual environment and installing
 something into it:
 
@@ -151,19 +176,19 @@ for more information on entry points, their definition, and usage.
 The "selectable" entry points were introduced in ``importlib_metadata``
 3.6 and Python 3.10. Prior to those changes, ``entry_points`` accepted
 no parameters and always returned a dictionary of entry points, keyed
-by group. For compatibility, if no parameters are passed to entry_points,
-a ``SelectableGroups`` object is returned, implementing that dict
-interface. In the future, calling ``entry_points`` with no parameters
-will return an ``EntryPoints`` object. Users should rely on the selection
-interface to retrieve entry points by group.
+by group. With ``importlib_metadata`` 5.0 and Python 3.12,
+``entry_points`` always returns an ``EntryPoints`` object. See
+`backports.entry_points_selectable <https://pypi.org/project/backports.entry_points_selectable>`_
+for compatibility options.
 
 
 .. _metadata:
 
 Distribution metadata
 ---------------------
 
-Every distribution includes some metadata, which you can extract using the
+Every `Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_ includes some metadata,
+which you can extract using the
 ``metadata()`` function::
 
     >>> wheel_metadata = metadata('wheel')  # doctest: +SKIP
@@ -201,7 +226,8 @@ all the metadata in a JSON-compatible form per :PEP:`566`::
 Distribution versions
 ---------------------
 
-The ``version()`` function is the quickest way to get a distribution's version
+The ``version()`` function is the quickest way to get a
+`Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_'s version
 number, as a string::
 
     >>> version('wheel')  # doctest: +SKIP
@@ -214,7 +240,8 @@ Distribution files
 ------------------
 
 You can also get the full set of files contained within a distribution.  The
-``files()`` function takes a distribution package name and returns all of the
+``files()`` function takes a `Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_ name
+and returns all of the
 files installed by this distribution.  Each file object returned is a
 ``PackagePath``, a :class:`pathlib.PurePath` derived object with additional ``dist``,
 ``size``, and ``hash`` properties as indicated by the metadata.  For example::
@@ -259,19 +286,24 @@ distribution is not known to have the metadata present.
 Distribution requirements
 -------------------------
 
-To get the full set of requirements for a distribution, use the ``requires()``
+To get the full set of requirements for a `Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_,
+use the ``requires()``
 function::
 
     >>> requires('wheel')  # doctest: +SKIP
     ["pytest (>=3.0.0) ; extra == 'test'", "pytest-cov ; extra == 'test'"]
 
 
-Package distributions
----------------------
+.. _package-distributions:
+.. _import-distribution-package-mapping:
+
+Mapping import to distribution packages
+---------------------------------------
 
-A convenience method to resolve the distribution or
-distributions (in the case of a namespace package) for top-level
-Python packages or modules::
+A convenience method to resolve the `Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_
+name (or names, in the case of a namespace package)
+that provide each importable top-level
+Python module or `Import Package <https://packaging.python.org/en/latest/glossary/#term-Import-Package>`_::
 
     >>> packages_distributions()
     {'importlib_metadata': ['importlib-metadata'], 'yaml': ['PyYAML'], 'jaraco': ['jaraco.classes', 'jaraco.functools'], ...}
@@ -285,7 +317,8 @@ Distributions
 
 While the above API is the most common and convenient usage, you can get all
 of that information from the ``Distribution`` class.  A ``Distribution`` is an
-abstract object that represents the metadata for a Python package.  You can
+abstract object that represents the metadata for
+a Python `Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_.  You can
 get the ``Distribution`` instance::
 
     >>> from importlib.metadata import distribution  # doctest: +SKIP
@@ -305,14 +338,16 @@ instance::
     >>> dist.metadata['License']  # doctest: +SKIP
     'MIT'
 
-The full set of available metadata is not described here.  See :pep:`566`
-for additional details.
+The full set of available metadata is not described here.
+See the `Core metadata specifications <https://packaging.python.org/en/latest/specifications/core-metadata/#core-metadata>`_ for additional details.
 
 
 Distribution Discovery
 ======================
 
-By default, this package provides built-in support for discovery of metadata for file system and zip file packages. This metadata finder search defaults to ``sys.path``, but varies slightly in how it interprets those values from how other import machinery does. In particular:
+By default, this package provides built-in support for discovery of metadata
+for file system and zip file `Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_\s.
+This metadata finder search defaults to ``sys.path``, but varies slightly in how it interprets those values from how other import machinery does. In particular:
 
 - ``importlib.metadata`` does not honor :class:`bytes` objects on ``sys.path``.
 - ``importlib.metadata`` will incidentally honor :py:class:`pathlib.Path` objects on ``sys.path`` even though such values will be ignored for imports.
@@ -321,15 +356,18 @@ By default, this package provides built-in support for discovery of metadata for
 Extending the search algorithm
 ==============================
 
-Because package metadata is not available through :data:`sys.path` searches, or
-package loaders directly, the metadata for a package is found through import
-system :ref:`finders <finders-and-loaders>`.  To find a distribution package's metadata,
+Because `Distribution Package <https://packaging.python.org/en/latest/glossary/#term-Distribution-Package>`_ metadata
+is not available through :data:`sys.path` searches, or
+package loaders directly,
+the metadata for a distribution is found through import
+system `finders`_.  To find a distribution package's metadata,
 ``importlib.metadata`` queries the list of :term:`meta path finders <meta path finder>` on
 :data:`sys.meta_path`.
 
-The default ``PathFinder`` for Python includes a hook that calls into
-``importlib.metadata.MetadataPathFinder`` for finding distributions
-loaded from typical file-system-based paths.
+By default ``importlib_metadata`` installs a finder for distribution packages
+found on the file system.
+This finder doesn't actually find any *distributions*,
+but it can find their metadata.
 
 The abstract class :py:class:`importlib.abc.MetaPathFinder` defines the
 interface expected of finders by Python's import system.
@@ -358,4 +396,4 @@ a custom finder, return instances of this derived ``Distribution`` in the
 
 .. _`entry point API`: https://setuptools.readthedocs.io/en/latest/pkg_resources.html#entry-points
 .. _`metadata API`: https://setuptools.readthedocs.io/en/latest/pkg_resources.html#metadata-api
-.. _`importlib_resources`: https://importlib-resources.readthedocs.io/en/latest/index.html
+.. _`finders`: https://docs.python.org/3/reference/import.html#finders-and-loaders

Doc/library/runpy.rst

@@ -93,6 +93,11 @@ The :mod:`runpy` module provides two functions:
       run this way, as well as ensuring the real module name is always
       accessible as ``__spec__.name``.
 
+   .. versionchanged:: 3.12
+      The setting of ``__cached__``, ``__loader__``, and
+      ``__package__`` are deprecated. See
+      :class:`~importlib.machinery.ModuleSpec` for alternatives.
+
 .. function:: run_path(path_name, init_globals=None, run_name=None)
 
    .. index::
@@ -163,6 +168,10 @@ The :mod:`runpy` module provides two functions:
       case where ``__main__`` is imported from a valid sys.path entry rather
       than being executed directly.
 
+   .. versionchanged:: 3.12
+      The setting of ``__cached__``, ``__loader__``, and
+      ``__package__`` are deprecated.
+
 .. seealso::
 
    :pep:`338` -- Executing modules as scripts

Doc/reference/expressions.rst

@@ -1766,6 +1766,13 @@ Or, when processing a file stream in chunks:
    while chunk := file.read(9000):
        process(chunk)
 
+Assignment expressions must be surrounded by parentheses when used
+as sub-expressions in slicing, conditional, lambda,
+keyword-argument, and comprehension-if expressions
+and in ``assert`` and ``with`` statements.
+In all other places where they can be used, parentheses are not required,
+including in ``if`` and ``while`` statements.
+
 .. versionadded:: 3.8
    See :pep:`572` for more details about assignment expressions.
 

Doc/reference/import.rst

@@ -1025,25 +1025,6 @@ and ``__main__.__spec__`` is set accordingly, they're still considered
 to populate the ``__main__`` namespace, and not during normal import.
 
 
-Open issues
-===========
-
-XXX It would be really nice to have a diagram.
-
-XXX * (import_machinery.rst) how about a section devoted just to the
-attributes of modules and packages, perhaps expanding upon or supplanting the
-related entries in the data model reference page?
-
-XXX runpy, pkgutil, et al in the library manual should all get "See Also"
-links at the top pointing to the new import system section.
-
-XXX Add more explanation regarding the different ways in which
-``__main__`` is initialized?
-
-XXX Add more info on ``__main__`` quirks/pitfalls (i.e. copy from
-:pep:`395`).
-
-
 References
 ==========
 

Doc/requirements.txt

@@ -10,7 +10,7 @@ blurb
 # sphinx-lint 0.6.2 yields many default role errors due to the new regular
 # expression used for default role detection, so we don't use the version
 # until the errors are fixed.
-sphinx-lint<1,!=0.6.2
+sphinx-lint==0.6.1
 
 # The theme used by the documentation is stored separately, so we need
 # to install that as well.

Doc/whatsnew/3.11.rst

@@ -1667,6 +1667,10 @@ Removed
   Python's test suite."
   (Contributed by Victor Stinner in :issue:`46852`.)
 
+* Pynche --- The Pythonically Natural Color and Hue Editor --- has been moved out
+  of ``Tools/scripts`` and is `being developed independently
+  <https://gitlab.com/warsaw/pynche/-/tree/main>`_ from the Python source tree.
+
 Porting to Python 3.11
 ======================
 

Doc/whatsnew/3.12.rst

@@ -280,8 +280,8 @@ Pending Removal in Python 3.14
 * Creating :c:data:`immutable types <Py_TPFLAGS_IMMUTABLETYPE>` with mutable
   bases using the C API.
 
-* ``__package__`` will cease to be set or taken into consideration by
-  the import system (:gh:`97879`).
+* ``__package__`` and ``__cached__`` will cease to be set or taken
+  into consideration by the import system (:gh:`97879`).
 
 
 Pending Removal in Future Versions

Lib/cProfile.py

@@ -7,6 +7,7 @@
 __all__ = ["run", "runctx", "Profile"]
 
 import _lsprof
+import importlib.machinery
 import profile as _pyprofile
 
 # ____________________________________________________________
@@ -169,9 +170,12 @@ def main():
             sys.path.insert(0, os.path.dirname(progname))
             with open(progname, 'rb') as fp:
                 code = compile(fp.read(), progname, 'exec')
+            spec = importlib.machinery.ModuleSpec(name='__main__', loader=None,
+                                                  origin=progname)
             globs = {
-                '__file__': progname,
-                '__name__': '__main__',
+                '__spec__': spec,
+                '__file__': spec.origin,
+                '__name__': spec.name,
                 '__package__': None,
                 '__cached__': None,
             }