Skip to content

Commit 7195b8f

Browse files
authored
Merge branch '3.14' into backport-3.14-sort-help-env
2 parents 80297ba + 36805f6 commit 7195b8f

File tree

12 files changed

+160
-17
lines changed

12 files changed

+160
-17
lines changed

Doc/library/argparse.rst

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -691,6 +691,8 @@ The add_argument() method
691691

692692
* deprecated_ - Whether or not use of the argument is deprecated.
693693

694+
The method returns an :class:`Action` object representing the argument.
695+
694696
The following sections describe how each of these are used.
695697

696698

Doc/library/stdtypes.rst

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3478,6 +3478,11 @@ The representation of bytearray objects uses the bytes literal format
34783478
``bytearray([46, 46, 46])``. You can always convert a bytearray object into
34793479
a list of integers using ``list(b)``.
34803480

3481+
.. seealso::
3482+
3483+
For detailed information on thread-safety guarantees for :class:`bytearray`
3484+
objects, see :ref:`thread-safety-bytearray`.
3485+
34813486

34823487
.. _bytes-methods:
34833488

Doc/library/threadsafety.rst

Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -447,3 +447,104 @@ atomic:
447447
448448
Consider external synchronization when sharing :class:`set` instances
449449
across threads. See :ref:`freethreading-python-howto` for more information.
450+
451+
452+
.. _thread-safety-bytearray:
453+
454+
Thread safety for bytearray objects
455+
===================================
456+
457+
The :func:`len` function is lock-free and :term:`atomic <atomic operation>`.
458+
459+
Concatenation and comparisons use the buffer protocol, which prevents
460+
resizing but does not hold the per-object lock. These operations may
461+
observe intermediate states from concurrent modifications:
462+
463+
.. code-block::
464+
:class: maybe
465+
466+
ba + other # may observe concurrent writes
467+
ba == other # may observe concurrent writes
468+
ba < other # may observe concurrent writes
469+
470+
All other operations from here on hold the per-object lock.
471+
472+
Reading a single element or slice is safe to call from multiple threads:
473+
474+
.. code-block::
475+
:class: good
476+
477+
ba[i] # bytearray.__getitem__
478+
ba[i:j] # slice
479+
480+
The following operations are safe to call from multiple threads and will
481+
not corrupt the bytearray:
482+
483+
.. code-block::
484+
:class: good
485+
486+
ba[i] = x # write single byte
487+
ba[i:j] = values # write slice
488+
ba.append(x) # append single byte
489+
ba.extend(other) # extend with iterable
490+
ba.insert(i, x) # insert single byte
491+
ba.pop() # remove and return last byte
492+
ba.pop(i) # remove and return byte at index
493+
ba.remove(x) # remove first occurrence
494+
ba.reverse() # reverse in place
495+
ba.clear() # remove all bytes
496+
497+
Slice assignment locks both objects when *values* is a :class:`bytearray`:
498+
499+
.. code-block::
500+
:class: good
501+
502+
ba[i:j] = other_bytearray # both locked
503+
504+
The following operations return new objects and hold the per-object lock
505+
for the duration:
506+
507+
.. code-block::
508+
:class: good
509+
510+
ba.copy() # returns a shallow copy
511+
ba * n # repeat into new bytearray
512+
513+
The membership test holds the lock for its duration:
514+
515+
.. code-block::
516+
:class: good
517+
518+
x in ba # bytearray.__contains__
519+
520+
All other bytearray methods (such as :meth:`~bytearray.find`,
521+
:meth:`~bytearray.replace`, :meth:`~bytearray.split`,
522+
:meth:`~bytearray.decode`, etc.) hold the per-object lock for their
523+
duration.
524+
525+
Operations that involve multiple accesses, as well as iteration, are never
526+
atomic:
527+
528+
.. code-block::
529+
:class: bad
530+
531+
# NOT atomic: check-then-act
532+
if x in ba:
533+
ba.remove(x)
534+
535+
# NOT thread-safe: iteration while modifying
536+
for byte in ba:
537+
process(byte) # another thread may modify ba
538+
539+
To safely iterate over a bytearray that may be modified by another
540+
thread, iterate over a copy:
541+
542+
.. code-block::
543+
:class: good
544+
545+
# Make a copy to iterate safely
546+
for byte in ba.copy():
547+
process(byte)
548+
549+
Consider external synchronization when sharing :class:`bytearray` instances
550+
across threads. See :ref:`freethreading-python-howto` for more information.

Lib/test/libregrtest/utils.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -150,7 +150,7 @@ def setup_unraisable_hook() -> None:
150150
sys.unraisablehook = regrtest_unraisable_hook
151151

152152

153-
orig_threading_excepthook: Callable[..., None] | None = None
153+
orig_threading_excepthook: Callable[..., object] | None = None
154154

155155

156156
def regrtest_threading_excepthook(args) -> None:

Lib/test/test_cmd_line.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,9 @@ def test_help_env(self):
7373
def test_help_xoptions(self):
7474
out = self.verify_valid_flag('--help-xoptions')
7575
self.assertIn(b'-X dev', out)
76+
options = re.findall(rb'^-X (\w+)', out, re.MULTILINE)
77+
self.assertEqual(options, sorted(options),
78+
"options should be sorted alphabetically")
7679

7780
@support.cpython_only
7881
def test_help_all(self):

Lib/test/test_pyexpat.py

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -689,6 +689,25 @@ def test_trigger_leak(self):
689689
parser.ElementDeclHandler = lambda _1, _2: None
690690
self.assertRaises(TypeError, parser.Parse, data, True)
691691

692+
@support.skip_if_unlimited_stack_size
693+
@support.skip_emscripten_stack_overflow()
694+
@support.skip_wasi_stack_overflow()
695+
def test_deeply_nested_content_model(self):
696+
# This should raise a RecursionError and not crash.
697+
# See https://github.com/python/cpython/issues/145986.
698+
N = 500_000
699+
data = (
700+
b'<!DOCTYPE root [\n<!ELEMENT root '
701+
+ b'(a, ' * N + b'a' + b')' * N
702+
+ b'>\n]>\n<root/>\n'
703+
)
704+
705+
parser = expat.ParserCreate()
706+
parser.ElementDeclHandler = lambda _1, _2: None
707+
with support.infinite_recursion():
708+
with self.assertRaises(RecursionError):
709+
parser.Parse(data)
710+
692711
class MalformedInputTest(unittest.TestCase):
693712
def test1(self):
694713
xml = b"\0\r\n"
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
``python --help-xoptions`` is now sorted by ``-X`` option name.
Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
:mod:`xml.parsers.expat`: Fixed a crash caused by unbounded C recursion when
2+
converting deeply nested XML content models with
3+
:meth:`~xml.parsers.expat.xmlparser.ElementDeclHandler`.
4+
This addresses :cve:`2026-4224`.

Modules/pyexpat.c

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@
33
#endif
44

55
#include "Python.h"
6+
#include "pycore_ceval.h" // _Py_EnterRecursiveCall()
67
#include "pycore_import.h" // _PyImport_SetModule()
78
#include "pycore_pyhash.h" // _Py_HashSecret
89
#include "pycore_traceback.h" // _PyTraceback_Add()
@@ -603,6 +604,10 @@ static PyObject *
603604
conv_content_model(XML_Content * const model,
604605
PyObject *(*conv_string)(void *))
605606
{
607+
if (_Py_EnterRecursiveCall(" in conv_content_model")) {
608+
return NULL;
609+
}
610+
606611
PyObject *result = NULL;
607612
PyObject *children = PyTuple_New(model->numchildren);
608613
int i;
@@ -614,14 +619,16 @@ conv_content_model(XML_Content * const model,
614619
conv_string);
615620
if (child == NULL) {
616621
Py_XDECREF(children);
617-
return NULL;
622+
goto done;
618623
}
619624
PyTuple_SET_ITEM(children, i, child);
620625
}
621626
result = Py_BuildValue("(iiO&N)",
622627
model->type, model->quant,
623628
conv_string, model->name, children);
624629
}
630+
done:
631+
_Py_LeaveRecursiveCall();
625632
return result;
626633
}
627634

Python/initconfig.c

Lines changed: 12 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -301,9 +301,15 @@ arg ...: arguments passed to program in sys.argv[1:]\n\
301301

302302
static const char usage_xoptions[] = "\
303303
The following implementation-specific options are available:\n\
304+
-X context_aware_warnings=[0|1]: if true (1) then the warnings module will\n\
305+
use a context variables; if false (0) then the warnings module will\n\
306+
use module globals, which is not concurrent-safe; set to true for\n\
307+
free-threaded builds and false otherwise; also\n\
308+
PYTHON_CONTEXT_AWARE_WARNINGS\n\
304309
-X cpu_count=N: override the return value of os.cpu_count();\n\
305310
-X cpu_count=default cancels overriding; also PYTHON_CPU_COUNT\n\
306311
-X dev : enable Python Development Mode; also PYTHONDEVMODE\n\
312+
-X disable-remote-debug: disable remote debugging; also PYTHON_DISABLE_REMOTE_DEBUG\n\
307313
-X faulthandler: dump the Python traceback on fatal errors;\n\
308314
also PYTHONFAULTHANDLER\n\
309315
-X frozen_modules=[on|off]: whether to use frozen modules; the default is \"on\"\n\
@@ -323,7 +329,6 @@ The following implementation-specific options are available:\n\
323329
-X perf: support the Linux \"perf\" profiler; also PYTHONPERFSUPPORT=1\n\
324330
-X perf_jit: support the Linux \"perf\" profiler with DWARF support;\n\
325331
also PYTHON_PERF_JIT_SUPPORT=1\n\
326-
-X disable-remote-debug: disable remote debugging; also PYTHON_DISABLE_REMOTE_DEBUG\n\
327332
"
328333
#ifdef Py_DEBUG
329334
"-X presite=MOD: import this module before site; also PYTHON_PRESITE\n"
@@ -338,21 +343,17 @@ The following implementation-specific options are available:\n\
338343
"\
339344
-X showrefcount: output the total reference count and number of used\n\
340345
memory blocks when the program finishes or after each statement in\n\
341-
the interactive interpreter; only works on debug builds\n"
346+
the interactive interpreter; only works on debug builds\n\
347+
-X thread_inherit_context=[0|1]: enable (1) or disable (0) threads inheriting\n\
348+
context vars by default; enabled by default in the free-threaded\n\
349+
build and disabled otherwise; also PYTHON_THREAD_INHERIT_CONTEXT\n\
350+
"
342351
#ifdef Py_GIL_DISABLED
343352
"-X tlbc=[0|1]: enable (1) or disable (0) thread-local bytecode. Also\n\
344353
PYTHON_TLBC\n"
345354
#endif
346355
"\
347-
-X thread_inherit_context=[0|1]: enable (1) or disable (0) threads inheriting\n\
348-
context vars by default; enabled by default in the free-threaded\n\
349-
build and disabled otherwise; also PYTHON_THREAD_INHERIT_CONTEXT\n\
350-
-X context_aware_warnings=[0|1]: if true (1) then the warnings module will\n\
351-
use a context variables; if false (0) then the warnings module will\n\
352-
use module globals, which is not concurrent-safe; set to true for\n\
353-
free-threaded builds and false otherwise; also\n\
354-
PYTHON_CONTEXT_AWARE_WARNINGS\n\
355-
-X tracemalloc[=N]: trace Python memory allocations; N sets a traceback limit\n \
356+
-X tracemalloc[=N]: trace Python memory allocations; N sets a traceback limit\n\
356357
of N frames (default: 1); also PYTHONTRACEMALLOC=N\n\
357358
-X utf8[=0|1]: enable (1) or disable (0) UTF-8 mode; also PYTHONUTF8\n\
358359
-X warn_default_encoding: enable opt-in EncodingWarning for 'encoding=None';\n\

0 commit comments

Comments
 (0)