Better process isolation

[DRMacIver]

Currently if you want to have process isolation for your tests you can use the forking executor.

But you probably shouldn’t. It has a number of weird limitations: It may (usually will) interact poorly with how test runners capture output, and for reasons that will be completely opaque to you but make sense I promise some examples will not minimize correctly.

I’d like to make this better and integrate it more thoroughly into Hypothesis, so it’s just a config item to get process isolation and it should transparently work with things. Ideally this would give built in parallel test execution too.

This may end up dependent on conjecture integration

[Zac-HD]

Closing this, as Conjecture has been part of Hypothesis since 3.0 and we run our own tests with multiple processes so it must be working!

[DRMacIver]

Nope. This is specifically about running each example in its own subprocess, not whole tests.

[Zac-HD]

Ooh, okay. Sounds expensive but I can see that it would make sense sometimes.

[Zac-HD]

In #2340, it took a while to work out that the code under test was hanging, and so the engine code could neither see that settings.deadline was exceeded nor abort the test. Running each test case in a separate process would let us avoid this classic failure mode of cooperative task scheduling.