Skip to content

ci: fuzz step reddens on the coordinator's own deadline with no finding #466

Description

@fuad-daoud

What happens

The gate's fuzz step (scripts/fuzz.sh, go test -fuzz … -fuzztime=10s) intermittently fails with no finding:

--- FAIL: FuzzNewBigVal_AcceptedFormsAreJSONValid (10.05s)
    context deadline exceeded
FAIL
FAIL	github.com/dexpace/morphic/ir	10.053s
make: *** [Makefile:113: fuzz] Error 1

No failing input is minimized or written to testdata/fuzz/, the run reports PASS on the next attempt with the same code, and the target passes a 20s local search. The elapsed time is the budget itself (10.05s, 11.08s): the fuzz coordinator hit -fuzztime while a worker was still executing an input and reported the deadline as a failure instead of stopping cleanly — Go's fuzzing engine has no grace period for that, so under runner load it surfaces as a red gate.

Two occurrences on 2026-09-12/13:

In the twelve failed gate runs before these, the message appears in none, so this is new-ish or newly frequent; either way one hiccup now reddens an unrelated PR's whole gate, and a reviewer reading "fuzz failed" has to open the log to learn nothing was found.

Where it lives

scripts/fuzz.sh runs each target once with the whole budget and no slack:

go test "$pkg" -run '^$' -fuzz "^${target}\$" \
	-fuzztime="$fuzztime" -fuzzminimizetime="$minimize_time"

Options

  1. Distinguish the deadline from a finding. A real finding writes a reproducer under testdata/fuzz/<Target>/ and prints its path; the deadline case writes nothing. On a non-zero exit, treat context deadline exceeded with no new corpus file as one bounded retry (once), and anything else as the failure it is. This keeps the 10s budget and cannot hide a finding, since a finding is reproduced from the written seed on the retry too.
  2. Give the coordinator slack — e.g. -fuzztime=8s inside a 10s wall budget — which shrinks the search for a property nobody measured.

Option 1 is the one that keeps the search as it is and makes the failure mode legible. Either way the script's comment should name the failure mode, so the next reader of a red fuzz step knows what "context deadline exceeded" with no reproducer means.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    type:ciBuild, release, or CI configuration

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions