Merge branch 'main' into gh-145678-grouper-uaf

Author: sampsonc

.github/CODEOWNERS

@@ -132,7 +132,9 @@ Tools/c-analyzer/             @ericsnowcurrently
 Tools/check-c-api-docs/       @ZeroIntensity
 
 # Fuzzing
-Modules/_xxtestfuzz/          @ammaraskar
+Modules/_xxtestfuzz/                    @python/fuzzers
+Lib/test/test_xxtestfuzz.py             @python/fuzzers
+.github/workflows/reusable-cifuzz.yml   @python/fuzzers
 
 # Limited C API & Stable ABI
 Doc/c-api/stable.rst          @encukou

Doc/Makefile

@@ -88,6 +88,7 @@ htmlhelp: build
 	      "build/htmlhelp/pydoc.hhp project file."
 
 .PHONY: latex
+latex: _ensure-sphinxcontrib-svg2pdfconverter
 latex: BUILDER = latex
 latex: build
 	@echo "Build finished; the LaTeX files are in build/latex."
@@ -231,7 +232,7 @@ dist-text:
 	@echo "Build finished and archived!"
 
 .PHONY: dist-pdf
-dist-pdf:
+dist-pdf: _ensure-sphinxcontrib-svg2pdfconverter
 	# archive the A4 latex
 	@echo "Building LaTeX (A4 paper)..."
 	mkdir -p dist
@@ -292,6 +293,10 @@ _ensure-pre-commit:
 _ensure-sphinx-autobuild:
 	$(MAKE) _ensure-package PACKAGE=sphinx-autobuild
 
+.PHONY: _ensure-sphinxcontrib-svg2pdfconverter
+_ensure-sphinxcontrib-svg2pdfconverter:
+	$(MAKE) _ensure-package PACKAGE=sphinxcontrib-svg2pdfconverter
+
 .PHONY: check
 check: _ensure-pre-commit
 	$(VENVDIR)/bin/python3 -m pre_commit run --all-files

Doc/c-api/arg.rst

@@ -524,7 +524,7 @@ API Functions
    Returns true on success; on failure, it returns false and raises the
    appropriate exception.
 
-   .. versionadded:: next
+   .. versionadded:: 3.15
 
 
 .. c:function:: int PyArg_ParseArrayAndKeywords(PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames, const char *format, const char * const *kwlist, ...)
@@ -535,7 +535,7 @@ API Functions
    Returns true on success; on failure, it returns false and raises the
    appropriate exception.
 
-   .. versionadded:: next
+   .. versionadded:: 3.15
 
 
 .. c:function:: int PyArg_UnpackTuple(PyObject *args, const char *name, Py_ssize_t min, Py_ssize_t max, ...)

Doc/c-api/dict.rst

@@ -495,7 +495,7 @@ Dictionary view objects
 Frozen dictionary objects
 ^^^^^^^^^^^^^^^^^^^^^^^^^
 
-.. versionadded:: next
+.. versionadded:: 3.15
 
 
 .. c:var:: PyTypeObject PyFrozenDict_Type

Doc/c-api/exceptions.rst

@@ -716,7 +716,7 @@ Signal Handling
       This function may now execute a remote debugger script, if remote
       debugging is enabled.
 
-   .. versionchanged:: next
+   .. versionchanged:: 3.15
       The exception set by :c:func:`PyThreadState_SetAsyncExc` is now raised.
 
 

Doc/c-api/float.rst

@@ -224,11 +224,6 @@ endian processor, or ``0`` on little endian processor.
 Return value: ``0`` if all is OK, ``-1`` if error (and an exception is set,
 most likely :exc:`OverflowError`).
 
-There are two problems on non-IEEE platforms:
-
-* What this does is undefined if *x* is a NaN or infinity.
-* ``-0.0`` and ``+0.0`` produce the same bytes string.
-
 .. c:function:: int PyFloat_Pack2(double x, char *p, int le)
 
    Pack a C double as the IEEE 754 binary16 half-precision format.
@@ -256,9 +251,6 @@ Return value: The unpacked double.  On error, this is ``-1.0`` and
 :c:func:`PyErr_Occurred` is true (and an exception is set, most likely
 :exc:`OverflowError`).
 
-Note that on a non-IEEE platform this will refuse to unpack a bytes string that
-represents a NaN or infinity.
-
 .. c:function:: double PyFloat_Unpack2(const char *p, int le)
 
    Unpack the IEEE 754 binary16 half-precision format as a C double.

Doc/c-api/import.rst

@@ -350,14 +350,14 @@ Importing Modules
 
    Gets the current lazy imports mode.
 
-   .. versionadded:: next
+   .. versionadded:: 3.15
 
 .. c:function:: PyObject* PyImport_GetLazyImportsFilter()
 
    Return a :term:`strong reference` to the current lazy imports filter,
    or ``NULL`` if none exists. This function always succeeds.
 
-   .. versionadded:: next
+   .. versionadded:: 3.15
 
 .. c:function:: int PyImport_SetLazyImportsMode(PyImport_LazyImportsMode mode)
 
@@ -366,7 +366,7 @@ Importing Modules
 
    This function always returns ``0``.
 
-   .. versionadded:: next
+   .. versionadded:: 3.15
 
 .. c:function:: int PyImport_SetLazyImportsFilter(PyObject *filter)
 
@@ -377,7 +377,7 @@ Importing Modules
 
    Return ``0`` on success and ``-1`` with an exception set otherwise.
 
-   .. versionadded:: next
+   .. versionadded:: 3.15
 
 .. c:type:: PyImport_LazyImportsMode
 
@@ -396,7 +396,7 @@ Importing Modules
       Disable lazy imports entirely. Even explicit ``lazy`` statements become
       eager imports.
 
-   .. versionadded:: next
+   .. versionadded:: 3.15
 
 .. c:function:: PyObject* PyImport_CreateModuleFromInitfunc(PyObject *spec, PyObject* (*initfunc)(void))
 

Doc/c-api/memory.rst

@@ -204,8 +204,11 @@ The following function sets, modeled after the ANSI C standard, but specifying
 behavior when requesting zero bytes, are available for allocating and releasing
 memory from the Python heap.
 
-The :ref:`default memory allocator <default-memory-allocators>` uses the
-:ref:`pymalloc memory allocator <pymalloc>`.
+In the GIL-enabled build (default build) the
+:ref:`default memory allocator <default-memory-allocators>` uses the
+:ref:`pymalloc memory allocator <pymalloc>`, whereas in the
+:term:`free-threaded build`, the default is the
+:ref:`mimalloc memory allocator <mimalloc>` instead.
 
 .. warning::
 
@@ -215,6 +218,11 @@ The :ref:`default memory allocator <default-memory-allocators>` uses the
 
    The default allocator is now pymalloc instead of system :c:func:`malloc`.
 
+.. versionchanged:: 3.13
+
+   In the :term:`free-threaded <free threading>` build, the default allocator
+   is now :ref:`mimalloc <mimalloc>`.
+
 .. c:function:: void* PyMem_Malloc(size_t n)
 
    Allocates *n* bytes and returns a pointer of type :c:expr:`void*` to the
@@ -340,7 +348,9 @@ memory from the Python heap.
     the :ref:`Customize Memory Allocators <customize-memory-allocators>` section.
 
 The :ref:`default object allocator <default-memory-allocators>` uses the
-:ref:`pymalloc memory allocator <pymalloc>`.
+:ref:`pymalloc memory allocator <pymalloc>`.  In the
+:term:`free-threaded <free threading>` build, the default is the
+:ref:`mimalloc memory allocator <mimalloc>` instead.
 
 .. warning::
 
@@ -420,23 +430,24 @@ Default Memory Allocators
 
 Default memory allocators:
 
-===============================  ====================  ==================  =====================  ====================
-Configuration                    Name                  PyMem_RawMalloc     PyMem_Malloc           PyObject_Malloc
-===============================  ====================  ==================  =====================  ====================
-Release build                    ``"pymalloc"``        ``malloc``          ``pymalloc``           ``pymalloc``
-Debug build                      ``"pymalloc_debug"``  ``malloc`` + debug  ``pymalloc`` + debug   ``pymalloc`` + debug
-Release build, without pymalloc  ``"malloc"``          ``malloc``          ``malloc``             ``malloc``
-Debug build, without pymalloc    ``"malloc_debug"``    ``malloc`` + debug  ``malloc`` + debug     ``malloc`` + debug
-===============================  ====================  ==================  =====================  ====================
+===================================  =======================  ====================  ======================  ======================
+Configuration                        Name                     PyMem_RawMalloc       PyMem_Malloc            PyObject_Malloc
+===================================  =======================  ====================  ======================  ======================
+Release build                        ``"pymalloc"``           ``malloc``            ``pymalloc``            ``pymalloc``
+Debug build                          ``"pymalloc_debug"``     ``malloc`` + debug    ``pymalloc`` + debug    ``pymalloc`` + debug
+Release build, without pymalloc      ``"malloc"``             ``malloc``            ``malloc``              ``malloc``
+Debug build, without pymalloc        ``"malloc_debug"``       ``malloc`` + debug    ``malloc`` + debug      ``malloc`` + debug
+Free-threaded build                  ``"mimalloc"``           ``mimalloc``          ``mimalloc``            ``mimalloc``
+Free-threaded debug build            ``"mimalloc_debug"``     ``mimalloc`` + debug  ``mimalloc`` + debug    ``mimalloc`` + debug
+===================================  =======================  ====================  ======================  ======================
 
 Legend:
 
 * Name: value for :envvar:`PYTHONMALLOC` environment variable.
 * ``malloc``: system allocators from the standard C library, C functions:
   :c:func:`malloc`, :c:func:`calloc`, :c:func:`realloc` and :c:func:`free`.
 * ``pymalloc``: :ref:`pymalloc memory allocator <pymalloc>`.
-* ``mimalloc``: :ref:`mimalloc memory allocator <mimalloc>`.  The pymalloc
-  allocator will be used if mimalloc support isn't available.
+* ``mimalloc``: :ref:`mimalloc memory allocator <mimalloc>`.
 * "+ debug": with :ref:`debug hooks on the Python memory allocators
   <pymem-debug-hooks>`.
 * "Debug build": :ref:`Python build in debug mode <debug-build>`.
@@ -733,9 +744,27 @@ The mimalloc allocator
 
 .. versionadded:: 3.13
 
-Python supports the mimalloc allocator when the underlying platform support is available.
-mimalloc "is a general purpose allocator with excellent performance characteristics.
-Initially developed by Daan Leijen for the runtime systems of the Koka and Lean languages."
+Python supports the `mimalloc <https://github.com/microsoft/mimalloc/>`__
+allocator when the underlying platform support is available.
+mimalloc is a general purpose allocator with excellent performance
+characteristics, initially developed by Daan Leijen for the runtime systems
+of the Koka and Lean languages.
+
+Unlike :ref:`pymalloc <pymalloc>`, which is optimized for small objects (512
+bytes or fewer), mimalloc handles allocations of any size.
+
+In the :term:`free-threaded <free threading>` build, mimalloc is the default
+and **required** allocator for the :c:macro:`PYMEM_DOMAIN_MEM` and
+:c:macro:`PYMEM_DOMAIN_OBJ` domains.  It cannot be disabled in free-threaded
+builds.  The free-threaded build uses per-thread mimalloc heaps, which allows
+allocation and deallocation to proceed without locking in most cases.
+
+In the default (non-free-threaded) build, mimalloc is available but not the
+default allocator.  It can be selected at runtime using
+:envvar:`PYTHONMALLOC`\ ``=mimalloc`` (or ``mimalloc_debug`` to include
+:ref:`debug hooks <pymem-debug-hooks>`).  It can be disabled at build time
+using the :option:`--without-mimalloc` configure option, but this option
+cannot be combined with :option:`--disable-gil`.
 
 tracemalloc C API
 =================

Doc/c-api/object.rst

@@ -817,4 +817,4 @@ Object Protocol
    Returns 1 if the object was made immortal and returns 0 if it was not.
    This function cannot fail.
 
-   .. versionadded:: next
+   .. versionadded:: 3.15

Doc/c-api/structures.rst

@@ -48,6 +48,19 @@ under :ref:`reference counting <countingrefs>`.
       Do not use this field directly; use :c:macro:`Py_TYPE` and
       :c:func:`Py_SET_TYPE` instead.
 
+   .. c:member:: PyMutex ob_mutex
+
+      A :ref:`per-object lock <per-object-locks>`, present only in the :term:`free-threaded <free threading>`
+      build (when :c:macro:`Py_GIL_DISABLED` is defined).
+
+      This field is **reserved for use by the critical section API**
+      (:c:macro:`Py_BEGIN_CRITICAL_SECTION` / :c:macro:`Py_END_CRITICAL_SECTION`).
+      Do **not** lock it directly with ``PyMutex_Lock``; doing so can cause
+      deadlocks.  If you need your own lock, add a separate :c:type:`PyMutex`
+      field to your object struct.
+
+      .. versionadded:: 3.13
+
 
 .. c:type:: PyVarObject