-
Notifications
You must be signed in to change notification settings - Fork 212
Description
Is your feature request related to a problem? Please describe.
The interchange is launched using Python multiprocessing but mostly wants to be its own freestanding process: for example, it awkwardly works around inheriting the main process logger by avoiding the "parsl" logging topic; and it communicates with the main process mostly via ZMQ (with only a small bit of multiprocessing-based communication near the start).
Multiprocessing-fork is a Bad Thing (see issue #2343) so launching the interchange this way should go away.
Launching as a fresh process would also allow more normal logging (for example, logging everything to the interchange log file rather than carefully avoiding what was inherited).
The only use of multiprocessing queues, which is to communicate the ZMQ port numbers to be used by workers, could be transmitted in a different way: for example add a WORKER_PORTS command to the command channel.
Describe the solution you'd like
Use normal fork/exec so this behaves like a freestanding process