v0.9.0b1

This release adds two new constructor parameters:

• error_callback: a function which will be called with the traceback (as a string) of an exception which occurs inside a process.
• only_threads: when True, all tasks will be run as threads.

v0.8.0b1

This release adds a new constructor parameter, capture_stdout=False. When capture_stdout is True, stdout from processes will be captured and written to the main process' stdout.

This can be useful for logging, etc.

Note: stderr is also written to stdout, but this may be tweaked in a later release.

v0.7.0b1

Scheduler now accepts a new constructor parameter, raise_exceptions=False. If True, a TaskFailedException will be raised in the main process if a Scheduler task fails. This Exception will contain the traceback from the task's exception.

v0.6.0b1

Scheduler's constructor now accepts a new parameter, run_in_thread=False. When run_in_thread is True, the first task will be run in a thread instead of a process.

This is intended to improve performance when the OS does not allow processes to be forked, since it removes some of the overhead of spawning processes.

v0.5.0b1

New features

• With the new map() and map_blocking() functions, a scheduler can be called with a single line of code.
• Shared memory is now supported for Numpy arrays on Python 3.8. This can provide substantial speed improvements when working with large arrays.

Mapping functions

See the documentation.

Shared memory

This feature is new, and may not be fully stable until a future release.

To enable it, use shared_memory=True when creating a Scheduler:

scheduler = Scheduler(shared_memory=True)

On Python 3.8, any Numpy arrays returned from a function added to a Scheduler will automatically be passed via shared memory if and only if shared memory is enabled and their size (arr.size) is greater than the shared_memory_threshold (which defaults to 1e7).

If multiple values are returned from a function, each value which is a Numpy array will be passed via shared memory if the above conditions are met for each array. Numpy arrays returned within a data structure, e.g. a list or dictionary, will not be passed via shared memory.

On Python 3.7 and below, Scheduler will fall back to the traditional Queue-based implementation even if shared memory is enabled.

v0.4.0b1

Normal functions can now be added to a scheduler with Scheduler.add().

v0.3.0b1

Scheduler now returns an empty list when terminated.

v0.2.0b1

Added Scheduler.add(), Scheduler.optimal_process_count(), Scheduler.is_running().

v0.1.0b1

First beta release.