(maybe) put hypothesis back

[glyph]

#586 removed hypothesis, but left the door open for re-integrating it if we can address a couple of issues:

• remove random generation from gating PR CI
  • ideally, run any random stuff in a timed action of some kind, that we can peruse to discover new tests
• fix the bug in Hyperlink's strategies that generates bogus URLs Hypothesis: builtins.UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte python-hyper/hyperlink#153
• reduce the selection of sample data being used so that its doesn't double the amount of time we spend in CI. A little overhead might be worth it for the functionality that Hypothesis is giving us, but not multiple minutes per run.

[exarkun]

fix the bug in Hyperlink's strategies that generates bogus URLs python-hyper/hyperlink#153

I fixed this almost a year ago with absolutely no response from the maintainers. Any ideas how to actually get any changes in to Hyperlink?

[Zac-HD]

👋 I can't do much about the hyperlink issue, but happy to help you find a hypothesis workflow you like. It sounds like this suggestion from @nelhage might work well:

• use Hypothesis' settings to run only the explicit phase, i.e. inputs in @example(...) decorators. This is fully deterministic, and basically equivalent to @pytest.mark.parametrize().
• have a separate cronjob in CI which runs just the Hypothesis tests on trunk with all phases, to find new failures which you can add to those explicit decorators.
  • see GitHubArtifactDatabase for easy local replay
  • since last week, Hypothesis can dump patch files to auto-add failing examples to your code - this is automatic but requires optional deps to be installed
  • or try HypoFuzz to get coverage-guided fuzzing in this step - patching in covering @example()s is planned (but no eta)

Alternatively, using e.g. max_examples=20, derandomize=True in your CI settings might be sufficient to meet your goals.

[glyph]

fix the bug in Hyperlink's strategies that generates bogus URLs python-hyper/hyperlink#153

I fixed this almost a year ago with absolutely no response from the maintainers. Any ideas how to actually get any changes in to Hyperlink?

Hmm. Apparently I'm a maintainer so maybe I can do something about that. Are you referring to this? python-hyper/hyperlink#178 Was it ever passing CI?

[exarkun]

fix the bug in Hyperlink's strategies that generates bogus URLs python-hyper/hyperlink#153

I fixed this almost a year ago with absolutely no response from the maintainers. Any ideas how to actually get any changes in to Hyperlink?

Hmm. Apparently I'm a maintainer so maybe I can do something about that. Are you referring to this? python-hyper/hyperlink#178 Was it ever passing CI?

That's the one, yes. I don't remember any of the details.