Skip to content

feat(runner): evaluate exit-code install-gate policy into artifact verdict - #24

Merged
jesse-merhi merged 2 commits into
mainfrom
pr-a2-gate
Jul 24, 2026
Merged

feat(runner): evaluate exit-code install-gate policy into artifact verdict#24
jesse-merhi merged 2 commits into
mainfrom
pr-a2-gate

Conversation

@jesse-merhi

@jesse-merhi jesse-merhi commented Jul 23, 2026

Copy link
Copy Markdown
Member

New behavior

Config-defined scanners can turn their process exit code into a recorded
pass, warn, or block gate verdict. ClawScan writes the verdict to the run
artifact and prints it in the human scan summary.

  • blockOnExitCode and warnOnExitCode accept one code, a list, or nonzero.
  • Explicit codes are limited to 0–124. Shell, container-runtime, signal, and
    timeout statuses are not treated as scanner verdicts.
  • block wins over warn, and no fired rule records pass.
  • Repeated scanner selections execute once, so a later duplicate cannot erase
    an earlier result.
  • A CLI --scanner override drops gate policies for excluded profile scanners.
  • Invalid, null, empty, aliased-null, or overlapping gate policies fail during
    config resolution.

Gate actions are record-only: block does not stop later scanners or the judge,
and it does not change ClawScan's process exit status. Enforcement belongs to
the caller that consumes the artifact or human summary.

Observable contract

Scanner outcome Recorded result
Exit code matches blockOnExitCode gate: block plus one gateRules entry
Exit code matches warnOnExitCode gate: warn plus one gateRules entry
No rule matches gate: pass, gateRules: []
Timeout, signal, or exit 125+ with valid JSON Raw evidence is preserved; exitCode is omitted and no rule fires

For a single run, inspect gate and gateRules. Batch artifacts expose
runs[].gate and runs[].gateRules; benchmark artifacts expose
cases[].run.gate and cases[].run.gateRules. The human scan summary
aggregates the strongest batch action.

How to verify

Create a config-defined scanner that emits JSON and exits 3:

version: 1
profiles:
  demo:
    scanners:
      - id: demo-scanner
        command: |
          test -d {{target}}
          printf '{"findings":1}\n'
          exit 3
        gate:
          blockOnExitCode: 3

Run it without Docker or credentials:

mkdir -p /tmp/clawscan-gate-demo
printf '# demo\n' > /tmp/clawscan-gate-demo/SKILL.md
go run ./cmd/clawscan /tmp/clawscan-gate-demo \
  --config ./with-gate.yml \
  --profile demo \
  --sandbox off \
  --json

The artifact contains:

{
  "gate": "block",
  "gateRules": [
    {
      "scanner": "demo-scanner",
      "rule": "blockOnExitCode",
      "exitCode": 3,
      "action": "block"
    }
  ]
}

Running without --json prints:

gate: block (demo-scanner exit 3 -> block)

The command still exits 0 because this PR records but does not enforce the
verdict.

Checks

  • Focused profile, runner, benchmark, and CLI gate regressions passed.
  • go test -count=1 ./internal/runner -skip 'TestResolveTargetClassifiesPlugin(Directory|ManifestFile)$' passed.
  • All other Go packages passed.
  • go vet ./... passed.
  • go build ./... passed.
  • make docs-site passed in an isolated validation copy.
  • The unfiltered suite reaches two unchanged macOS assertions that compare
    equivalent /var/... and /private/var/... paths; the affected production
    and test files are unchanged from main.

Copilot AI review requested due to automatic review settings July 23, 2026 03:31

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Object-form scanner entries accept a nested gate with blockOnExitCode /
warnOnExitCode (int, int list, or "nonzero"). The runner captures
user-defined scanner exit codes, evaluates rules after all scanners
complete, and records gate: pass|warn|block plus fired gateRules in the
artifact. Rules referencing unrequested scanners fail before scanning.
Abnormal termination (signal/timeout) never fires exit-code rules.
@jesse-merhi
jesse-merhi changed the base branch from pr-a1-byos to main July 24, 2026 06:44
@clawsweeper

clawsweeper Bot commented Jul 24, 2026

Copy link
Copy Markdown

ClawSweeper status: review started.

I am starting a fresh review of this pull request: feat(runner): evaluate exit-code install-gate policy into artifact verdict This is item 1/1 in the current shard. Shard 0/1.

This placeholder means the worker is alive and reading the current context. I will edit this same comment with the actual review when the claws are done clicking.

Crustacean status: shell secured, claws on keyboard, evidence pebbles being sorted.

@jesse-merhi
jesse-merhi merged commit b5d45b7 into main Jul 24, 2026
4 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants