Skip to content

audit_lut_table.py is genuinely over the ratchet's 120 s on the runner, not flaky #736

Description

@gHashTag

The measurement

audit_lut_table.py does not finish inside the ratchet's 120 s on ubuntu-latest.

where cores time result
laptop 10, run alone 65.6 s exit 0, clean
ubuntu-latest 2, --jobs 2 >120 s TIMEOUT

Confirmed by a deliberate re-measurement on the runner (the rebaseline dispatch added in #729), not inferred from a failing PR. It had been failing every PR for days, and I twice reported it as possibly flaky — it passed on one run and failed on others, which reads like noise. It is not noise: it is over the line on the machine that scores it, and close enough to the line that scheduling occasionally hides that.

Why this is an issue and not a timeout bump

gate_status_ratchet.py says it plainly:

The timeout is part of the MEASUREMENT, not a convenience knob: a script that is TIMEOUT at 90s and clean at 300s has not changed.

Raising the limit would turn this green with nothing repaired. The baseline now records TIMEOUT (#735) — an honest statement of what the instrument reads — so the ratchet stops reporting a phantom degradation on every unrelated PR. That is bookkeeping, not a fix.

What would actually resolve it

One of:

  1. Make it faster. 65.6 s single-threaded for a LUT table audit suggests re-derivation that could be cached or narrowed. This is the only option that improves anything.
  2. Split it. If it audits several independent tables, several gates each under the limit are strictly better than one over it — and they fail more precisely.
  3. Decide the runner should have more time, deliberately, and re-measure everything at the new limit via the rebaseline dispatch so the whole column stays comparable. Not by editing the number and leaving the old statuses in place.

The last option is a decision about what the ratchet's timeout means and belongs to whoever owns that contract.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions