Skip to content

[minor] MLAI-1288 - Enforce VS Code skill governance through agent-guard - #77

Draft
shmuelqwak wants to merge 2 commits into
mainfrom
feature/MLAI-1288-vscode-skill-governance
Draft

[minor] MLAI-1288 - Enforce VS Code skill governance through agent-guard#77
shmuelqwak wants to merge 2 commits into
mainfrom
feature/MLAI-1288-vscode-skill-governance

Conversation

@shmuelqwak

Copy link
Copy Markdown
Collaborator

Adds JFrog skill governance to the VS Code / GitHub Copilot Chat plugin, by carrying the hook event to agent-guard --enforce-skill --client vscode. Part of MLAI-1288, alongside claude-plugin #66 and cursor-plugin #81.

Important

Draft: must not merge until an agent-guard release contains --client vscode.
Released latest is v1.13.0 and it rejects the flag — both v1.12.0 and v1.13.0 were cut before agent-guard #208 merged, so the support is on master and unreleased. The governed hooks resolve latest on purpose and never pin, so merging today makes every governed VS Code event hard-error on an unknown --client value. Nothing in this PR needs to change for it — it needs a release cut from agent-guard master.

What the plugin does, and does not do

No governance logic lives here. Two hooks pipe the raw event to agent-guard on stdin and forward its stdout verbatim. Fingerprinting, resolution, evaluation, the block card and the waiver flow are all agent-guard's.

hook what it governs
UserPromptSubmit a typed /<skill-name>
PreToolUse the skill tool, and a read_file of a SKILL.md

Three VS Code specifics worth knowing while reviewing

github.copilot.chat.skillTool.enabled defaults to false. With it off Copilot registers no skill tool and instead instructs the model to "use the read_file tool to read the corresponding SKILL.md files" — read from the shipped Copilot extension and confirmed by hook-level measurement. So on a stock install the read_file surface, not the skill tool, is what governs a model-discovered skill. agent-guard #208 governs both.

VS Code parses hooks.json matchers and then ignores them — stated in its own docs. Every tool call in the session therefore starts the process, and the tool-name check inside agent-guard is the only filter. It also means governing read_file costs no extra invocations: those events already arrive.

There is no async hook field on any event, so no session-start pre-warm is possible. That is why the two governed hooks split the npx cache: UserPromptSubmit fires once per prompt and revalidates (dropping --prefer-offline), and PreToolUse reads what that left. Without the split, agent-guard freezes at whatever version a machine first fetched — not theoretical, a cached 1.10.0 once served verdicts a released 1.11.0 had already fixed.

The hook command

npm_config_fetch_retries=0 npm_config_fetch_timeout=10000 \
JF_AGENT_GUARD_ENFORCE_DEADLINE="$(_JFAG_NOW=$(date +%s 2>/dev/null); echo ${_JFAG_NOW:+$((_JFAG_NOW + 25))})" \
npx --yes [--prefer-offline] --registry "${JFROG_AGENT_GUARD_REPO:-…}" @jfrog/agent-guard --enforce-skill --client vscode

Two deliberate details:

  • One simple command, with the ; scoped inside $( ). On Cursor a top-level ; severed the payload pipeline and silently allowed every skill (MLAI-1310); VS Code delivers on real stdin so it is unaffected, but these strings are kept identical across the three plugins and a copy carrying the defect would reintroduce it.
  • The deadline degrades to EMPTY when date(1) cannot be read — agent-guard ignores an empty deadline and uses its own budget, whereas a garbage epoch floors that budget at 500 ms and blocks everything.

The version is deliberately never pinned, unlike the MCP-align hook which pins on purpose, so a shipped GA fix reaches users without a plugin release. The validator asserts nothing can pin it.

Validation

scripts/validate-skill-governance.mjs38 checks — asserts the shape and then executes the real command string from hooks.json against a stub agent-guard: stdin forwarded verbatim, a deny forwarded at exit 0, an allow as explicit {"continue": true}, invalid JSON and empty stdout forwarded unchanged, an agent-guard failure and a missing npx both failing open, agent-guard's exit 2 still blocking, the fetch bounds by value, the deadline fresh against a stale inherited one, and the cache split.

It also runs in release.yml, not only on PRs — a PR-only workflow never sees the merge commit a release is cut from. That replaces the # NOTE: no plugin-layout validator exists in this repo to gate the release on comment.

validate-package-resolution-hook.mjs was converted from index-based to content-based lookup, so adding events cannot trip it.

All gates green on this branch: skill-governance 38/38, package-resolution-hook pass, vendored-modules pass, 38 unit tests pass.

Known gaps, all failing in the allow direction

  • Windows is not supported by these hooks. VS Code spawns hooks with cmd.exe /c on Windows (Qn === 1cmd.exe), and this command string is POSIX. hooks.json does support per-platform windows / osx / linux command overrides, so the fix belongs here — tracked separately rather than guessed at without a Windows machine to verify on.
  • With skillTool.enabled off, the skill tool never fires; coverage comes from read_file and the slash surface.
  • runSubagent, run_in_terminal and grep_search can reach a skill's content without a governed surface.
  • Roots added via chat.agentSkillsLocations are not read — no resolution API.
  • Extension-contributed skills (contributes.chatSkills) are never seen.
  • Hooks are Preview and can be disabled org-side with no signal.

Version

1.0.261.1.0 in both carriers (plugin/.claude-plugin/plugin.json and marketplace.json .plugins[0].version). The AX-2134 release trigger refuses a version equal to or older than the latest tag, and v1.0.26 is tagged.

shmuelqwak and others added 2 commits September 9, 2026 17:56
Adds skill governance to the Copilot Chat plugin by calling
@jfrog/agent-guard --enforce-skill --client vscode from two hooks. No
governance logic lands here: fingerprinting, the governance round trip,
the block card and the waiver all live in agent-guard, as they do for
the Claude Code and Cursor plugins.

UserPromptSubmit covers "/<skill-name>"; PreToolUse covers Copilot's
skill tool. Neither carries a matcher: VS Code parses matchers for
Claude Code compatibility but ignores their values, so one here would
read as protection that does not exist. The tool-name filter lives in
agent-guard, which is also why every tool call reaching that hook is
answered rather than dropped.

Infrastructure failure fails OPEN — npx missing, a failed install, an
unreachable registry or no configured server let the skill through. A
policy denial still blocks, in the JSON payload.

The version is deliberately NOT pinned, unlike the MCP-align hook in
this same repo, so a governance fix reaches users without a plugin
release. The README says why the two differ.

VS Code documents no async hook, so there is no session-start pre-warm
to keep --prefer-offline safe. Instead UserPromptSubmit revalidates
against the registry once per prompt and PreToolUse reads that from
cache. Without the split, agent-guard freezes at whatever version a
machine first fetched — measured on Cursor, where a cached 1.10.0 kept
serving verdicts a released 1.11.0 had already fixed.

release.yml now runs the validator too. A pull-request workflow never
sees the merge commit a release is cut from, which CONTRIBUTING.md
already required and a NOTE in release.yml asked for. The
package-resolution validator located its SessionStart hooks by index
and asserted a list length; both are now by content, so appending a
hook cannot break it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ivery per client

Hardens the two governed hooks into a single simple command by computing the deadline
inside a command substitution, matching the cursor-plugin fix.

VS Code is not affected by MLAI-1310: it writes the event to the child's real stdin
(`stdin.write(JSON.stringify(...))` with `stdio: ["pipe", ...]` in agentHostMain.js), so
a top-level `;` is harmless here. It is still removed, for two reasons. These command
strings are kept deliberately identical across the three plugins, and on Cursor the same
`;` severs the pipeline Cursor wraps around the command, silently allowing every skill —
a string copied from here would carry that defect. And a one-simple-command hook works
under every delivery model, whereas one that relies on inheriting the shell's stdin only
works under some.

The validator gains the static top-level-operator check, and runHook now documents that
the delivery model is client-specific and verified per client rather than assumed. That
assumption is exactly what let MLAI-1310 ship: cursor-plugin's validator fed the payload
on the shell's stdin the way VS Code and Claude Code do, so all 34 checks passed against
a hook that delivered nothing at all.

Verified: validate-skill-governance, validate-package-resolution-hook,
check-vendored-modules, and 38 unit tests all pass.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant