fix(runtime): upgrade SkillSpector to 2.11.2

[Patrick-Erichsen]

Summary

Upgrade the Docker runtime's immutable SkillSpector pin to 2.11.2 (69dcdfb74487d361ba4c811d088cfdea2ff3a9dc). This includes the configurable aggregate deadline with a 600-second default from NVIDIA/SkillSpector#468, plus the 2.11.2 reference-accounting and parser fixes. This is the only change intended for ClawScan 0.1.8.

Scope

• Release/CI/repo automation

Security / Trust Impact

• Security/trust impact explained

The scanner receives more time to inspect large skills. Raw findings and incomplete-analysis evidence are preserved; no findings are suppressed.

Verification

• go test -count=1 ./...
• go vet ./...
• node --test npm/clawscan/test/*.test.mjs scripts/build-npm-package.test.mjs
• node scripts/build-npm-package.mjs --version v0.1.8 --pack --smoke
• Structured autoreview: no actionable findings.
• Docs site: N/A (no documentation changes).

Live proof on byungkyu/api-gateway@1.2.0, downloaded from the owner-qualified ClawHub API. Every published file matched its API SHA-256. The current export has 236 published files (the original 235 plus a generated skill-card.md); export-only _meta.json was excluded. All scans used --no-llm and an isolated environment with no model credentials.

Run	Wall time	Coverage	Runtime-limit exceptions	AE1 findings
SkillSpector 2.11.0 default	27.17s	99.6%	0	0
SkillSpector 2.11.2, forced 1s budget	3.42s	0%	441	98
SkillSpector 2.11.2 default 600s budget	67.04s	99.2%	0	0

The timeout override works, and the default-budget run avoids runtime-limit/AE1 findings. The original 2.11.0 timeout did not reproduce on this Mac. The latest report remains partial with 186 unresolved-reference exceptions, one reference-extraction limit, and two obfuscated-text exceptions; it still reports 599 findings and CRITICAL / DO_NOT_INSTALL. This update does not claim to fix those independent scanner limits or ClawHub's completeness-reporting bug.

Packaged Linux/arm64 verification also passed using the built ClawScan 0.1.8 candidate and the complete runtime Dockerfile. The wrapper recorded SkillSpector 2.11.2 as completed, preserving its raw partial report (99.2% coverage, no runtime-limit exceptions) in 65.94s. The same container stack with only SkillSpector reverted to 2.11.0 completed in 17.73s with 99.6% coverage and no runtime-limit exceptions. Both wrapper runs exited 0; SkillSpector itself exited 1 for findings. Neither result is presented as a clean or complete security scan.

[clawsweeper]

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

[clawsweeper]

Codex review: blocked before merge. Reviewed September 10, 2026, 12:27 AM ET / 04:27 UTC (Revision 2).

ClawSweeper review

What this changes

Updates the Docker runtime’s pinned SkillSpector scanner to 2.11.2, bringing a longer analysis deadline and parser and reference-accounting fixes.

Merge readiness

⛔ Blocked before merge - 3 items remain

This remains useful work: main and v0.1.7 retain the older pin, and the packaged scan evidence supports compatibility with ClawScan’s adapter. No actionable patch defect was found; the repository’s benchmark-based runtime upgrade gate remains outstanding.

Priority: P2
Reviewed head: 3e3154039dacf95f8e0fd07a46edd4bc0cdb861c

Review scores

Measure	Result	What it means
Overall readiness	🐚 platinum hermit (4/6)	A focused dependency update with meaningful packaged runtime evidence and no identified code defect, pending benchmark compatibility validation.
Proof confidence	🐚 platinum hermit (4/6)	Sufficient (live_output): The captured Linux/arm64 results exercise the packaged ClawScan adapter and complete candidate Docker runtime against a real skill, showing completed execution with partial raw evidence preserved; separate forced-budget results exercise deadline behavior. This supports the changed integration but does not replace the benchmark upgrade gate.
Patch quality	🐚 platinum hermit (4/6)	No actionable review findings were identified.

Verification

Check	Result	Evidence
Real behavior	Verified	Sufficient (live_output): The captured Linux/arm64 results exercise the packaged ClawScan adapter and complete candidate Docker runtime against a real skill, showing completed execution with partial raw evidence preserved; separate forced-budget results exercise deadline behavior. This supports the changed integration but does not replace the benchmark upgrade gate.
Evidence reviewed	8 items	Exact introduced change and dependency boundary: The verified base-to-head diff changes only the immutable SkillSpector revision. Dockerfile line 24 installs that revision directly from NVIDIA/SkillSpector, establishing the dependency contract relevant to this review. Current main and released baseline: The pinned main revision and v0.1.7 retain SkillSpector revision 539c1150b7c6e9a8935679f5ed5108bafdc68515. GitHub identifies v0.1.7 as the latest release; the requested upgrade is not already implemented there. Adapter compatibility: The adapter invokes scan with JSON output, selects --no-llm without provider credentials, defaults to a 20-minute outer timeout, and preserves valid JSON even when the scanner exits with findings. TestSkillSpectorReportWithNonZeroExitIsCompletedEvidence covers that existing contract.
Findings	None	None.
Security	None	None.

How this fits together

ClawScan invokes SkillSpector inside its default Docker runtime to inspect skill files. The adapter preserves the scanner’s JSON evidence for reports, configured gates, and optional external judges.

flowchart LR
  A[Skill files] --> B[ClawScan scanner adapter]
  B --> C[Docker runtime]
  C --> D[SkillSpector analysis]
  D --> E[Raw JSON evidence]
  E --> F[Reports and configured gates]
  E --> G[Optional external judge]

Loading

Before merge

• Resolve merge risk (P1) - The single-skill comparison does not establish benchmark verdict or failure-rate compatibility for existing profiles; the upstream update also changes finding classification and LLM concurrency.
• Resolve merge risk (P1) - The packaged sample increased from 17.73 to 65.94 seconds, while the upstream aggregate deadline increases tenfold; broader runtime impact remains unmeasured.
• Complete next step (P2) - Attach candidate-versus-current SkillTrustBench and applicable Security Signals benchmark evidence, explaining verdict, failure-rate, and runtime changes required by the runtime upgrade checklist.

Agent review details

Security

None.

Review metrics

Metric	Value	Why it matters
Packaged scan time	17.73s → 65.94s, approximately 3.7×	The supplied comparison establishes a concrete runtime increase worth checking across benchmark cases.

Merge-risk options

Maintainer options:

1. Complete the runtime comparison (recommended)
   Attach candidate-versus-current SkillTrustBench and applicable Security Signals results, explaining changed verdicts, failures, and timings before merge.

Technical review

Best possible solution:

Retain the focused immutable-pin upgrade with benchmark evidence showing acceptable verdict, failure-rate, and runtime changes for existing profiles.

Do we have a high-confidence way to reproduce the issue?

No high-confidence reproduction of the original timeout was established: the supplied baseline runs completed without runtime-limit exceptions. The forced-budget run demonstrates the deadline mechanism, rather than reproducing the original failure.

Is this the best way to solve the issue?

Yes, updating the existing immutable dependency pin is the narrowest integration path; the remaining acceptance work is the repository’s runtime benchmark comparison.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning medium; reviewed against 6432c40f2340.

Labels

Label justifications:

• P2: This is a bounded scanner maintenance update with useful fixes and no demonstrated urgent ClawScan outage.
• merge-risk: 🚨 compatibility: The dependency changes scanner evidence and execution budgets consumed by existing profiles, with benchmark compatibility still unverified.
• rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
• status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (live_output): The captured Linux/arm64 results exercise the packaged ClawScan adapter and complete candidate Docker runtime against a real skill, showing completed execution with partial raw evidence preserved; separate forced-budget results exercise deadline behavior. This supports the changed integration but does not replace the benchmark upgrade gate.
• proof: sufficient: Contributor real behavior proof is sufficient. The captured Linux/arm64 results exercise the packaged ClawScan adapter and complete candidate Docker runtime against a real skill, showing completed execution with partial raw evidence preserved; separate forced-budget results exercise deadline behavior. This supports the changed integration but does not replace the benchmark upgrade gate.

Evidence

What I checked:

• Exact introduced change and dependency boundary: The verified base-to-head diff changes only the immutable SkillSpector revision. Dockerfile line 24 installs that revision directly from NVIDIA/SkillSpector, establishing the dependency contract relevant to this review. (docker/clawscan-runtime/Dockerfile:3, 3e3154039dac)
• Current main and released baseline: The pinned main revision and v0.1.7 retain SkillSpector revision 539c1150b7c6e9a8935679f5ed5108bafdc68515. GitHub identifies v0.1.7 as the latest release; the requested upgrade is not already implemented there. (docker/clawscan-runtime/Dockerfile:3, 6432c40f2340)
• Adapter compatibility: The adapter invokes scan with JSON output, selects --no-llm without provider credentials, defaults to a 20-minute outer timeout, and preserves valid JSON even when the scanner exits with findings. TestSkillSpectorReportWithNonZeroExitIsCompletedEvidence covers that existing contract. (internal/runner/runner.go:2078, 3e3154039dac)
• Runtime upgrade acceptance criteria: The runtime-update workflow explicitly requires candidate-versus-current benchmark artifacts, SkillTrustBench evidence, Security Signals evidence when ClawHub profile behavior is affected, and explanations of verdict, failure-rate, or runtime changes. (.github/workflows/runtime-tool-updates.yml:80, 3e3154039dac)
• Captured production-path proof: The supplied complete PR body, captured under sourceRevision 51718031186242fd01be42f8c3bcc9c3e516375d59cb66f4d2371887f3f7cc08, reports a packaged Linux/arm64 ClawScan candidate using the complete Dockerfile against a hash-verified 236-file skill. The new scanner completed in 65.94 seconds, preserved partial JSON with 99.2% coverage, and recorded no runtime-limit exceptions; the reverted scanner completed in 17.73 seconds. Separate Mac runs exercised a forced one-second budget and the default budget. The body explicitly disclaims reproducing the original timeout and does not claim complete or clean analysis. (3e3154039dac)
• Verified upstream implementation: GitHub verified repository ownership and the 21-commit comparison between the two pins. The inspected production patches raise the default aggregate deadline from 60 to 600 seconds, preserve classification during deduplication, repair reference accounting, adjust bounded shell parsing, and share LLM concurrency limits across analyzers. Package requirements and third-party lockfile versions are unchanged. (src/skillspector/state.py:45, 69dcdfb74487)

Likely related people:

• Patrick-Erichsen: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)
• steipete: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)

Rank-up moves

Optional improvements that raise the rating; they are not merge blockers.

• Provide the repository-required candidate-versus-current benchmark comparison and explain verdict, failure-rate, and runtime changes.

Rating scale

Score	Internal tier	Crab rank	Meaning
6/6	S	🦀 challenger crab	Exceptional readiness
5/6	A	🦞 diamond lobster	Very strong readiness
4/6	B	🐚 platinum hermit	Good normal PR; ordinary maintainer review
3/6	C	🦐 gold shrimp	Useful, but confidence is limited
2/6	D	🦪 silver shellfish	Proof or implementation needs work
1/6	F	🧂 unranked krab	Not merge-ready
N/A	NA	🌊 off-meta tidepool	Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

• ClawSweeper keeps one durable marker-backed review comment per issue or PR.
• Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
• A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
• PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
• Maintainers can also comment @clawsweeper review to request a fresh review only.
• Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
• Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
• Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (1 earlier review cycle)

• reviewed 2026-09-10T04:18:21.932Z sha 3e31540 :: needs maintainer review before merge. :: none