Slow Hypothesis CI: nightly test job silently skipped since 2026-02-12 (#11184's fix never applied to hypothesis.yaml)

[glaziermag]

What is your issue?

The nightly Slow Hypothesis CI workflow has reported run-level success every night since 2026-02-12, but the Slow Hypothesis Tests job is skipped in every one of those runs — the slow property-test tier in properties/*.py has produced no signal for 6 months, and the issue-from-pytest-log alarm at the end of the workflow (which auto-files "Nightly Hypothesis tests failed", most recently #11141) cannot fire because it lives inside the skipped job.

Cause. This is the same defect @keewis diagnosed and fixed in #11184, where it was applied only to upstream-dev-ci.yaml:

#11096 inadvertently skipped CI if the detect-ci-trigger job was skipped. By replacing the default success() (true only if all previous steps reported 'success', everything else, including 'skipped', will evaluate to false) with always() and comparing against the result of the cache-pixi-lock step, we can get CI to run on workflow_dispatch and schedule events again.

hypothesis.yaml has the identical structure and was not included in that fix: since #11096, detect-ci-trigger's condition requires github.event_name == 'pull_request', so it skips on schedule/workflow_dispatch; cache-pixi-lock survives via always(); but the hypothesis job has no if: at all, so its implicit success() sees the skipped detect-ci-trigger in the transitive needs chain and the job skips — while the run still concludes "success".

Run-history evidence (public Actions history, verified 2026-08-12):

• 2026-02-07 through 02-10 nightlies: Slow Hypothesis Tests executed and failed (runs 21770870570, 21789646201, 21808478241, 21846931586 → auto-filed Nightly Hypothesis tests failed #11141, fixed 2026-02-10).
• 2026-02-11 00:45 UTC nightly (run 21888224201): executed, success — the last real scheduled execution. Refactor cache-pixi-lock.yml workflow into external action #11096 merged 21:47 UTC the same day.
• 2026-02-12 → 2026-08-12: 182 scheduled runs; Slow Hypothesis Tests skipped in all of them (181 concluded "success"; the single "failure", run 26989137346 on 2026-06-05, was a cache-pixi-lock infra error — tests still skipped). Job-level spot checks across the window: 21928886849 (02-12), 24430545206 (04-15), 27518353047 (06-15), 31550430155 (08-12) — every one shows detect ci trigger: skipped, cache-pixi-lock: success, Slow Hypothesis Tests: skipped. The job-gating lines are unchanged across the window (checked at 6 intermediate commits), so no interval escaped this.

workflow_dispatch of this workflow is affected the same way (the if: requires a pull_request event).

Verification on a fork (tree identical to e902fe8):

• Unpatched hypothesis.yaml, dispatched: run reports success, Slow Hypothesis Tests skipped — reproduces the mechanism. https://github.com/glaziermag/xarray/actions/runs/31645871468
• With exactly the two lines from allow triggering the upstream-dev CI if the check is skipped #11184 added to the hypothesis job: the suite executes and currently passes, twice on an identical tree — 31 passed, 3 skipped, 2 xfailed, 1 xpassed, 17 warnings in 82.69s (https://github.com/glaziermag/xarray/actions/runs/31645873449) and … in 80.43s (https://github.com/glaziermag/xarray/actions/runs/31646323529). Diff: main...glaziermag:xarray:hypothesis-skip-demo

Proposed fix — apply #11184's two lines to the hypothesis job in .github/workflows/hypothesis.yaml:

  hypothesis:
    name: Slow Hypothesis Tests
    runs-on: "ubuntu-latest"

    needs: cache-pixi-lock
    if: |
      always() && needs.cache-pixi-lock.result == 'success'

On the fork runs above this restores the schedule/dispatch paths, and the PR-label path (run-slow-hypothesis) is unchanged since cache-pixi-lock already encodes that gating. Happy to open a PR with this if preferred.

---

AI disclosure: this issue was researched and written by an AI agent, explicitly instructed to investigate and file it; all run IDs, job conclusions, quotes, and the fork demonstration were verified against the public GitHub API before filing. [This is Claude Code on behalf of @glaziermag]

[VeckoTheGecko]

Fixed in #11520 , thanks