-
-
Notifications
You must be signed in to change notification settings - Fork 32.3k
gh-85222: Document the side effect in multiprocessing #136426
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
646225c
1b7a0ea
29b5cf3
7962657
ed60127
881a347
de61926
982c7c2
c85562f
8a9ae53
a62a4fe
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||||||
---|---|---|---|---|---|---|---|---|
|
@@ -511,6 +511,21 @@ | |||||||
The :mod:`multiprocessing` package mostly replicates the API of the | ||||||||
:mod:`threading` module. | ||||||||
|
||||||||
.. _global-start-method: | ||||||||
|
||||||||
Global start method | ||||||||
^^^^^^^^^^^^^^^^^^^ | ||||||||
|
||||||||
Python supports several ways to create and initialize a process. | ||||||||
The global start method sets the default mechanism for creating a process. | ||||||||
|
||||||||
Several multiprocessing functions and methods that may also instantiate | ||||||||
certain objects will implicitly set the global start method to the system's default, | ||||||||
if it hasn’t been set already. The global start method can only be set once. | ||||||||
If you need to change the start method from the system default, you must | ||||||||
proactively set the global start method before calling functions or methods, | ||||||||
or creating these objects. | ||||||||
|
||||||||
|
||||||||
:class:`Process` and exceptions | ||||||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ||||||||
|
@@ -867,6 +882,9 @@ | |||||||
locks/semaphores. When a process first puts an item on the queue a feeder | ||||||||
thread is started which transfers objects from a buffer into the pipe. | ||||||||
|
||||||||
Instantiating this class may set the global start method. See | ||||||||
:ref:`global-start-method` for more details. | ||||||||
|
||||||||
The usual :exc:`queue.Empty` and :exc:`queue.Full` exceptions from the | ||||||||
standard library's :mod:`queue` module are raised to signal timeouts. | ||||||||
|
||||||||
|
@@ -977,6 +995,9 @@ | |||||||
|
||||||||
It is a simplified :class:`Queue` type, very close to a locked :class:`Pipe`. | ||||||||
|
||||||||
Instantiating this class may set the global start method. See | ||||||||
:ref:`global-start-method` for more details. | ||||||||
|
||||||||
.. method:: close() | ||||||||
|
||||||||
Close the queue: release internal resources. | ||||||||
|
@@ -1007,6 +1028,9 @@ | |||||||
:class:`JoinableQueue`, a :class:`Queue` subclass, is a queue which | ||||||||
additionally has :meth:`task_done` and :meth:`join` methods. | ||||||||
|
||||||||
Instantiating this class may set the global start method. See | ||||||||
:ref:`global-start-method` for more details. | ||||||||
|
||||||||
.. method:: task_done() | ||||||||
|
||||||||
Indicate that a formerly enqueued task is complete. Used by queue | ||||||||
|
@@ -1119,8 +1143,8 @@ | |||||||
:mod:`multiprocessing` module. | ||||||||
|
||||||||
If *method* is ``None`` then the default context is returned. Note that if | ||||||||
the global start method has not been set, this will set it to the | ||||||||
default method. | ||||||||
the global start method has not been set, this will set it to the system default | ||||||||
See :ref:`global-start-method` for more details. | ||||||||
Otherwise *method* should be ``'fork'``, ``'spawn'``, | ||||||||
``'forkserver'``. :exc:`ValueError` is raised if the specified | ||||||||
start method is not available. See :ref:`multiprocessing-start-methods`. | ||||||||
|
@@ -1131,10 +1155,9 @@ | |||||||
|
||||||||
Return the name of start method used for starting processes. | ||||||||
|
||||||||
If the global start method has not been set and *allow_none* is | ||||||||
``False``, then the start method is set to the default and the name | ||||||||
is returned. If the start method has not been set and *allow_none* is | ||||||||
``True`` then ``None`` is returned. | ||||||||
If the global start method is not set and *allow_none* is ``False``, the global start | ||||||||
method is set to the default, and its name is returned. See | ||||||||
:ref:`global-start-method` for more details. | ||||||||
|
||||||||
The return value can be ``'fork'``, ``'spawn'``, ``'forkserver'`` | ||||||||
or ``None``. See :ref:`multiprocessing-start-methods`. | ||||||||
|
@@ -1361,13 +1384,19 @@ | |||||||
|
||||||||
A barrier object: a clone of :class:`threading.Barrier`. | ||||||||
|
||||||||
Instantiating this class may set the global start method. See | ||||||||
:ref:`global-start-method` for more details. | ||||||||
|
||||||||
.. versionadded:: 3.3 | ||||||||
|
||||||||
.. class:: BoundedSemaphore([value]) | ||||||||
|
||||||||
A bounded semaphore object: a close analog of | ||||||||
:class:`threading.BoundedSemaphore`. | ||||||||
|
||||||||
Instantiating this class may set the global start method. See | ||||||||
:ref:`global-start-method` for more details. | ||||||||
|
||||||||
A solitary difference from its close analog exists: its ``acquire`` method's | ||||||||
first argument is named *block*, as is consistent with :meth:`Lock.acquire`. | ||||||||
|
||||||||
|
@@ -1388,13 +1417,18 @@ | |||||||
If *lock* is specified then it should be a :class:`Lock` or :class:`RLock` | ||||||||
object from :mod:`multiprocessing`. | ||||||||
|
||||||||
Instantiating this class may set the global start method. See | ||||||||
:ref:`global-start-method` for more details. | ||||||||
|
||||||||
.. versionchanged:: 3.3 | ||||||||
The :meth:`~threading.Condition.wait_for` method was added. | ||||||||
|
||||||||
.. class:: Event() | ||||||||
|
||||||||
A clone of :class:`threading.Event`. | ||||||||
|
||||||||
Instantiating this class may set the global start method. See | ||||||||
:ref:`global-start-method` for more details. | ||||||||
|
||||||||
.. class:: Lock() | ||||||||
|
||||||||
|
@@ -1410,6 +1444,9 @@ | |||||||
instance of ``multiprocessing.synchronize.Lock`` initialized with a | ||||||||
default context. | ||||||||
|
||||||||
Instantiating this class may set the global start method. See | ||||||||
:ref:`global-start-method` for more details. | ||||||||
|
||||||||
:class:`Lock` supports the :term:`context manager` protocol and thus may be | ||||||||
used in :keyword:`with` statements. | ||||||||
|
||||||||
|
@@ -1467,6 +1504,9 @@ | |||||||
instance of ``multiprocessing.synchronize.RLock`` initialized with a | ||||||||
default context. | ||||||||
|
||||||||
Instantiating this class may set the global start method. See | ||||||||
:ref:`global-start-method` for more details. | ||||||||
|
||||||||
:class:`RLock` supports the :term:`context manager` protocol and thus may be | ||||||||
used in :keyword:`with` statements. | ||||||||
|
||||||||
|
@@ -1526,6 +1566,9 @@ | |||||||
|
||||||||
A semaphore object: a close analog of :class:`threading.Semaphore`. | ||||||||
|
||||||||
Instantiating this class may set the global start method. See | ||||||||
:ref:`global-start-method` for more details. | ||||||||
|
||||||||
A solitary difference from its close analog exists: its ``acquire`` method's | ||||||||
first argument is named *block*, as is consistent with :meth:`Lock.acquire`. | ||||||||
|
||||||||
|
@@ -1660,7 +1703,7 @@ | |||||||
attributes which allow one to use it to store and retrieve strings -- see | ||||||||
documentation for :mod:`ctypes`. | ||||||||
|
||||||||
.. function:: Array(typecode_or_type, size_or_initializer, *, lock=True) | ||||||||
.. function:: Array(typecode_or_type, size_or_initializer, *, lock=True, ctx=None) | ||||||||
|
||||||||
The same as :func:`RawArray` except that depending on the value of *lock* a | ||||||||
process-safe synchronization wrapper may be returned instead of a raw ctypes | ||||||||
|
@@ -1674,9 +1717,13 @@ | |||||||
automatically protected by a lock, so it will not necessarily be | ||||||||
"process-safe". | ||||||||
|
||||||||
Note that *lock* is a keyword-only argument. | ||||||||
*ctx* is a context object, or ``None`` (use the current context). If ``None``, | ||||||||
calling this may set the global start method. See | ||||||||
:ref:`global-start-method` for more details. | ||||||||
|
||||||||
.. function:: Value(typecode_or_type, *args, lock=True) | ||||||||
Note that *lock* and *ctx* are keyword-only parameters. | ||||||||
|
||||||||
.. function:: Value(typecode_or_type, *args, lock=True, ctx=None) | ||||||||
|
||||||||
The same as :func:`RawValue` except that depending on the value of *lock* a | ||||||||
process-safe synchronization wrapper may be returned instead of a raw ctypes | ||||||||
|
@@ -1689,20 +1736,28 @@ | |||||||
automatically protected by a lock, so it will not necessarily be | ||||||||
"process-safe". | ||||||||
|
||||||||
Note that *lock* is a keyword-only argument. | ||||||||
*ctx* is a context object, or ``None`` (use the current context). If ``None``, | ||||||||
calling this may set the global start method. See | ||||||||
:ref:`global-start-method` for more details. | ||||||||
|
||||||||
Note that *lock* and *ctx* are keyword-only parameters. | ||||||||
|
||||||||
.. function:: copy(obj) | ||||||||
|
||||||||
Return a ctypes object allocated from shared memory which is a copy of the | ||||||||
ctypes object *obj*. | ||||||||
|
||||||||
.. function:: synchronized(obj[, lock]) | ||||||||
.. function:: synchronized(obj, lock=None, ctx=None) | ||||||||
|
||||||||
Return a process-safe wrapper object for a ctypes object which uses *lock* to | ||||||||
synchronize access. If *lock* is ``None`` (the default) then a | ||||||||
:class:`multiprocessing.RLock` object is created automatically. | ||||||||
|
||||||||
*ctx* is a context object, or ``None`` (use the current context). If ``None``, | ||||||||
calling this may set the global start method. See | ||||||||
:ref:`global-start-method` for more details. | ||||||||
|
||||||||
A synchronized wrapper will have two methods in addition to those of the | ||||||||
Check warning on line 1760 in Doc/library/multiprocessing.rst
|
||||||||
object it wraps: :meth:`get_obj` returns the wrapped object and | ||||||||
:meth:`get_lock` returns the lock object used for synchronization. | ||||||||
|
||||||||
|
@@ -1819,8 +1874,9 @@ | |||||||
*serializer* must be ``'pickle'`` (use :mod:`pickle` serialization) or | ||||||||
``'xmlrpclib'`` (use :mod:`xmlrpc.client` serialization). | ||||||||
|
||||||||
*ctx* is a context object, or ``None`` (use the current context). See the | ||||||||
:func:`get_context` function. | ||||||||
*ctx* is a context object, or ``None`` (use the current context). If ``None``, | ||||||||
calling this may set the global start method. See | ||||||||
:ref:`global-start-method` for more details. | ||||||||
|
||||||||
*shutdown_timeout* is a timeout in seconds used to wait until the process | ||||||||
used by the manager completes in the :meth:`shutdown` method. If the | ||||||||
|
@@ -2309,11 +2365,13 @@ | |||||||
unused resources to be freed. The default *maxtasksperchild* is ``None``, which | ||||||||
means worker processes will live as long as the pool. | ||||||||
|
||||||||
*context* can be used to specify the context used for starting | ||||||||
Check warning on line 2368 in Doc/library/multiprocessing.rst
|
||||||||
the worker processes. Usually a pool is created using the | ||||||||
function :func:`multiprocessing.Pool` or the :meth:`Pool` method | ||||||||
of a context object. In both cases *context* is set | ||||||||
appropriately. | ||||||||
appropriately. If ``None``, calling this function will have the side effect | ||||||||
of setting the current global start method if it has not been set already. | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
More small suggestions to improve reading flow Up to you if you wanna take this one |
||||||||
See the :func:`get_context` function. | ||||||||
|
||||||||
Note that the methods of the pool object should only be called by | ||||||||
the process which created the pool. | ||||||||
|
Uh oh!
There was an error while loading. Please reload this page.