Skip to content

Add a CI workflow that runs the skill-harness suite — the guards from #28 currently only run locally #33

Description

@oschaaf

Raised in review of #28: after that PR, pnpm test runs the schema-digest drift check, the coverage assertions, and the vendored-artifact pins — but nothing automated runs pnpm test. The repo has no .github/workflows/ (the only checks on PRs are org-level CodeQL default setup), so every guard added in #28 holds only when a human runs the suite locally.

Suggested work

A small workflow (PR + push to main) that runs, from skill-harness/:

  1. pnpm install --frozen-lockfile
  2. the unit suite (node --import tsx --test 'src/__tests__/**/*.test.ts' until pnpm test works again)
  3. biome check --error-on-warnings and tsc --noEmit
  4. node scripts/generate-schema-digest.mjs --check from the repo root (also covered by the suite, but a named check line is worth having)

Prerequisite: #30. pnpm install --frozen-lockfile currently fails on pnpm 11 (overrides placement), and a CI that installs unfrozen defeats part of the point. Fix #30 first or in the same PR, then wire the workflow.

Explicitly out of scope: the Tier-2 bench (pnpm bench) — it calls a model API and needs a key + cost decision; the workflow should run the offline suite only.

Once green, update README.md / CLAUDE.md wording from "pnpm test is what enforces it" to point at the workflow.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions