Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .agents/skills/contribute-docs/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,10 @@ Use this skill for docs-only or example-heavy changes.
- Keep stable user-facing wrappers at `scripts/` root in docs and examples;
only point at namespaced helper paths when documenting internal maintenance
work
- Dynamic plugin docs must keep Rust native plugin examples, Python worker
plugin examples, and `grpc-v1` protocol details on separate pages.
- Dynamic plugin manifests in docs/examples should use
`compat.relay = ">=0.5,<1.0"` unless deliberately narrower.
- In MDX files, top-of-file comments must use JSX comment delimiters:
`{/*` to open and `*/}` to close. Do not use HTML comments for MDX SPDX
headers.
Expand All @@ -36,6 +40,8 @@ Use this skill for docs-only or example-heavy changes.
- [ ] Relevant getting-started or reference docs updated
- [ ] Example commands still match current package names and paths
- [ ] Relevant package or crate `README.md` files updated when examples or binding guidance changed
- [ ] Dynamic plugin entry pages link to native, worker, Rust example, Python
example, and protocol pages when those pages exist
- [ ] New or regenerated MDX files use `{/* ... */}` for top-of-file SPDX comments
- [ ] Release-policy docs still point to GitHub Releases as the only release-history source of truth
- [ ] Run `just docs` when the docs site changed; `./scripts/build-docs.sh html` remains the compatibility wrapper
Expand Down
9 changes: 9 additions & 0 deletions .agents/skills/maintain-ci/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,10 @@ reliability, or reproducibility.
pipeline has tag-based publish behavior.
- Keep release-tag policy aligned with `RELEASING.md`: raw SemVer tags only,
no leading `v`.
- Keep Codecov component paths aligned with new crates, packages, and generated
outputs. Dynamic plugin SDK/protocol paths belong in the plugin component.
- Keep pure-Python plugin SDK packaging as a single wheel artifact instead of
duplicating it across every platform matrix entry.

## Permission Model

Expand Down Expand Up @@ -61,6 +65,10 @@ reliability, or reproducibility.
- [ ] Every external action is pinned to a full SHA
- [ ] Cache settings are tied to lockfiles, manifests, or explicit tool versions
- [ ] Secrets are only passed to the jobs that consume them
- [ ] Codecov upload counts match `codecov.yml` after adding or removing upload
jobs
- [ ] Package artifacts include any first-class SDK packages introduced by the
change
- [ ] Concurrency, branch filters, and publish guards still reflect release intent
- [ ] Artifact upload, download, and Pages deploy steps have matching permissions
- [ ] Tag-triggered release workflows fail early when a tag violates repo policy
Expand Down Expand Up @@ -92,3 +100,4 @@ source instead of assuming local success proves remote success.
- `.pre-commit-config.yaml`
- `maintain-packaging`
- `validate-change`
- `maintain-dynamic-plugins`
83 changes: 83 additions & 0 deletions .agents/skills/maintain-dynamic-plugins/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
---
name: maintain-dynamic-plugins
description: Maintain NeMo Relay dynamic plugin loaders, manifests, Rust native SDKs, gRPC worker protocol, Python worker SDK, docs, tests, and release workflow coverage
author: NVIDIA Corporation and Affiliates
license: Apache-2.0
---

# Maintain Dynamic Plugins

## Companion Guidance

Use `karpathy-guidelines`, `validate-change`, `maintain-packaging`, and
`contribute-docs` alongside this skill when implementation, packaging, CI, or
documentation changes are involved.

Use this skill for `plugin.kind = "rust_dynamic"`, `plugin.kind = "worker"`,
`nemo-relay-plugin`, `nemo-relay-worker`, `nemo-relay-worker-proto`,
`nemo-relay-types`, and the Python `nemo-relay-plugin` package.

## Rules

- Keep the stable boundary explicit: native plugins cross a C ABI; worker
plugins cross `grpc-v1`.
- Do not pass Rust runtime types, trait objects, futures, or allocator-owned
strings across the native dynamic-library boundary.
- Keep worker protocol DTOs in `JsonEnvelope`; protobuf owns control flow, not
duplicated Relay data models.
- Keep `relay-plugin.toml` dynamic records separate from generic runtime
components. Enabled dynamic records may synthesize internal component specs;
disabled records stay inspectable but unloaded.
- Treat plugin Relay compatibility as normal SemVer. Use `>=0.5,<1.0` in
examples unless a plugin intentionally declares a narrower range.
- Do not add tests under `src`; Rust tests belong in crate `tests/` trees and
Python SDK tests belong under `python/tests`.
- Native and worker plugins are trusted extensions. Document that native plugins
are in-process and unsandboxed; worker plugins provide process isolation but
not a security sandbox.

## Checklist

- [ ] Manifest validation covers kind, compatibility, load contract, integrity,
capability mismatch, and disabled-plugin behavior.
- [ ] Native loader keeps libraries alive until registered callbacks are cleared
and deregisters plugin kinds before unload.
- [ ] Worker activation covers process launch, token auth, handshake, validation,
declarative registration, proxy rollback, cancellation, and shutdown.
- [ ] Rust and Python SDKs expose every supported registration surface.
- [ ] Runtime helpers cover marks, scopes, continuations, and isolated scope
stacks.
- [ ] CLI `plugins list/inspect/validate/doctor` diagnostics include native ABI
and worker protocol status without leaking secret config.
- [ ] Docs include separate Rust native, Python worker, and `grpc-v1` protocol
pages.
- [ ] `justfile`, Codecov, and CI package/test workflows include new plugin
crates and packages.

## Validation

```bash
cargo test -p nemo-relay-types
cargo test -p nemo-relay-plugin
cargo test -p nemo-relay-worker-proto
cargo test -p nemo-relay-worker
cargo test -p nemo-relay --test native_plugin_tests --test worker_plugin_tests
uv run pytest python/tests/plugin
just test-rust
just test-python
just docs
```

For broad runtime or public API changes, run the full `validate-change` matrix.

## References

- `crates/core/src/plugin/dynamic/`
- `crates/plugin`
- `crates/worker`
- `crates/worker-proto`
- `crates/types`
- `python/plugin`
- `examples/rust-native-plugin`
- `docs/build-plugins`
- `examples/python-grpc-worker-plugin`
7 changes: 7 additions & 0 deletions .agents/skills/maintain-packaging/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,15 @@ consumed outside the source tree.

- Rust `Cargo.toml` package names and workspace metadata
- Python packaging in `pyproject.toml`
- Python worker plugin SDK packaging in `python/plugin/pyproject.toml`
- Go module path in `go/nemo_relay/go.mod`
- Node workspace metadata in root `package.json` and `package-lock.json`
- Node package metadata in `crates/node/package.json`
- WebAssembly package naming and generated package expectations
- FFI header and library naming
- CI workflows, install commands, and example commands
- `justfile` build, test, clean, version, and package recipes for plugin crates
and packages
- Release tags, release-note surfaces, and registry-facing version translation

## Checklist
Expand All @@ -36,12 +39,15 @@ consumed outside the source tree.
- [ ] Docs and examples use the current install/import/build commands
- [ ] CI references the same package names as local workflows
- [ ] Public packaging changes are reflected in release-facing docs
- [ ] `nemo-relay-plugin` Rust and Python packages track the project SemVer
policy and Python wheels use valid PEP 440 translation
- [ ] Release tags still use raw SemVer without a leading `v`
- [ ] Release history and release notes still point to GitHub Releases, not `CHANGELOG.md` or docs pages

## References

- `pyproject.toml`
- `python/plugin/pyproject.toml`
- `go/nemo_relay/go.mod`
- `package.json`
- `package-lock.json`
Expand All @@ -50,3 +56,4 @@ consumed outside the source tree.
- `.github/workflows/ci_pipe.yml`
- `.github/workflows/ci.yaml`
- `.gitlab-ci.yml`
- `maintain-dynamic-plugins`
19 changes: 15 additions & 4 deletions .agents/skills/test-python-binding/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,18 +15,20 @@ work. Keep changes scoped, surface assumptions, and define focused validation
before editing.

Use this skill when the change is primarily in `python/nemo_relay`,
`python/tests`, `crates/python`, or Python-facing docs/examples.
`python/plugin`, `python/tests`, `crates/python`, or Python-facing
docs/examples.

## Default Path

1. Format changed Python wrapper and test files with `uv run ruff format python`.
1. Format changed Python wrapper and test files with `uv run ruff format python python/plugin`.
2. Run focused `pytest` first when you know the affected area.
3. Run the full Python suite with `just test-python` before review.
4. If any Rust files changed as part of the Python work, also run
`cargo fmt --all`, `just test-rust`, and
`cargo clippy --workspace --all-targets -- -D warnings`.
5. Use `just build-python` when you want an explicit build-only pass.
6. If the native Rust bridge changed, add the Rust crate tests for
6. Use `just build-python-plugin` when the Python worker SDK changed.
7. If the native Rust bridge changed, add the Rust crate tests for
`nemo-relay-python`.

## Python Test Style
Expand All @@ -53,9 +55,10 @@ Use this skill when the change is primarily in `python/nemo_relay`,
```bash
# Focused test loop
uv run pytest -k "<pattern>"
uv run pytest python/tests/plugin

# Format Python files
uv run ruff format python
uv run ruff format python python/plugin

# Full Python suite
just test-python
Expand All @@ -68,6 +71,9 @@ cargo clippy --workspace --all-targets -- -D warnings
# Rebuild the editable package plus native extension
just build-python

# Rebuild/install the Python worker plugin SDK
just build-python-plugin

# Native extension crate when crates/python changed
cargo test -p nemo-relay-python
```
Expand All @@ -76,6 +82,8 @@ cargo test -p nemo-relay-python

- If `crates/core`, `crates/adaptive`, or shared runtime semantics changed,
also use `validate-change`.
- If `python/plugin` or worker protocol behavior changed, also use
`maintain-dynamic-plugins`.
- If the change is actually about docs only, prefer `contribute-docs`
plus targeted command checks.

Expand All @@ -85,6 +93,9 @@ cargo test -p nemo-relay-python
- `crates/python/Cargo.toml`
- `crates/python/README.md`
- `python/nemo_relay/README.md`
- `python/plugin/pyproject.toml`
- `python/plugin/src/nemo_relay_plugin`
- `python/tests/plugin`
- `docs/getting-started/python.md`
- `docs/contribute/testing-and-docs.md`
- `validate-change`
13 changes: 12 additions & 1 deletion .agents/skills/test-rust-core/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ work. Keep changes scoped, surface assumptions, and define focused validation
before editing.

Use this skill when a change is primarily in `crates/core`, `crates/adaptive`,
or shared Rust runtime semantics.
dynamic plugin Rust crates, or shared Rust runtime semantics.

## Default Path

Expand Down Expand Up @@ -46,6 +46,13 @@ cargo test -p nemo-relay
# Adaptive crate when touched
cargo test -p nemo-relay-adaptive

# Dynamic plugin crates when touched
cargo test -p nemo-relay-types
cargo test -p nemo-relay-plugin
cargo test -p nemo-relay-worker-proto
cargo test -p nemo-relay-worker
cargo test -p nemo-relay --test native_plugin_tests --test worker_plugin_tests

# Compile sweep
just build-rust

Expand All @@ -57,6 +64,9 @@ just ci=true test-rust

- If a public API, event shape, middleware behavior, plugin semantics, or any
`crates/core`/`crates/adaptive` behavior changed, also use `validate-change`.
- If native dynamic plugins, gRPC workers, `nemo-relay-plugin`,
`nemo-relay-worker`, `nemo-relay-worker-proto`, or `nemo-relay-types` changed,
also use `maintain-dynamic-plugins`.
- If the change is isolated to one binding wrapper on top of unchanged Rust
semantics, prefer that binding's build/test skill instead.

Expand All @@ -69,3 +79,4 @@ just ci=true test-rust
- `crates/adaptive/README.md`
- `docs/contribute/testing-and-docs.md`
- `validate-change`
- `maintain-dynamic-plugins`
13 changes: 13 additions & 0 deletions .agents/skills/validate-change/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,9 @@ surfaces touched by a change.
Python, Go, Node.js, and WebAssembly.
- If a language surface changed, always run that language's test target even when
Rust core did not change.
- If dynamic plugin behavior changed, use `maintain-dynamic-plugins` and include
the native SDK, worker protocol, Python SDK, docs, packaging, and Codecov
surfaces in the validation plan.
- If code changes alter APIs, bindings, commands, paths, packaging behavior,
observability/adaptive semantics, or documented best practices, update any
dependent maintainer or consumer skills in the same branch.
Expand All @@ -50,6 +53,10 @@ surfaces touched by a change.
Use `test-wasm-binding`.
- **FFI surface change**
Use `test-ffi-surface`.
- **Dynamic plugin loader, SDK, or protocol change**
Use `maintain-dynamic-plugins`. Run the targeted plugin crates and
`python/tests/plugin` first, then escalate to the core validation matrix when
runtime behavior or `crates/core` changed.
- **Third-party integration or patch change**
Run patch validation with `./scripts/apply-patches.sh --check` and the relevant
integration tests. Keep the root `./scripts/*.sh` wrappers for third-party
Expand Down Expand Up @@ -82,9 +89,11 @@ cargo clippy --workspace --all-targets -- -D warnings

# Python
just build-python
just build-python-plugin
just test-python
uv run ruff format python
uv run pytest -k "<pattern>"
uv run pytest python/tests/plugin

# Go
just build-go
Expand Down Expand Up @@ -118,6 +127,7 @@ just docs-linkcheck
- `test-node-binding`
- `test-wasm-binding`
- `test-ffi-surface`
- `maintain-dynamic-plugins`

## Pre-commit Semantics

Expand Down Expand Up @@ -163,6 +173,8 @@ If the change is large or public-facing, also verify:
- README and docs entry points still match current package names and paths
- Examples still run with the documented commands
- Any renamed public surfaces are reflected consistently in manifests and docs
- Dynamic plugin examples use `compat.relay = ">=0.5,<1.0"` unless deliberately
narrower.

## References

Expand All @@ -171,3 +183,4 @@ If the change is large or public-facing, also verify:
- Build and test dispatchers: `justfile`
- Patch helpers: `scripts/apply-patches.sh`, `scripts/generate-patches.sh`
- Third-party script implementations: `scripts/third-party/`
- Dynamic plugin guidance: `maintain-dynamic-plugins`
18 changes: 18 additions & 0 deletions .github/workflows/ci_python.yml
Original file line number Diff line number Diff line change
Expand Up @@ -268,6 +268,24 @@ jobs:
path: ${{ env.NEMO_RELAY_CI_WORKSPACE_TMP }}/wheels/*.whl
if-no-files-found: error

- name: Package Python plugin SDK wheel
if: ${{ matrix.platform == 'linux-amd64' }}
working-directory: ${{ env.NEMO_RELAY_CI_WORKSPACE }}
run: |
set -e
just \
--set output_dir "${{ env.NEMO_RELAY_CI_WORKSPACE_TMP }}" \
--set ref_name "${NEMO_RELAY_PACKAGE_VERSION}" \
package-python-plugin

- name: Upload Python plugin SDK wheel artifact
if: ${{ matrix.platform == 'linux-amd64' }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: python-plugin-wheel
path: ${{ env.NEMO_RELAY_CI_WORKSPACE_TMP }}/plugin-wheels/*.whl
if-no-files-found: error

- name: Prune uv cache
working-directory: ${{ env.NEMO_RELAY_CI_WORKSPACE }}
run: uv cache prune --ci
9 changes: 8 additions & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ repos:
hooks:
- id: ty
name: ty (type check)
entry: uv run ty check . --exclude docs/** --exclude fern/** --exclude third_party/** --exclude ./examples/** --exclude .cache/** --exclude .claude/**
entry: uv run ty check . --extra-search-path python/plugin/src --exclude docs/** --exclude fern/** --exclude third_party/** --exclude ./examples/** --exclude .cache/** --exclude .claude/** --exclude python/plugin/src/nemo_relay_plugin/_proto/**
language: system
types: [python]
pass_filenames: false
Expand Down Expand Up @@ -105,6 +105,13 @@ repos:
files: '^(pyproject\.toml|uv\.lock)$'
pass_filenames: false

- id: python-worker-proto-check
name: Python worker protobuf stubs are up to date
entry: just check-python-worker-proto
language: system
files: '^(crates/worker-proto/proto/nemo/relay/worker/v1/plugin_worker\.proto|python/plugin/src/nemo_relay_plugin/_proto/plugin_worker_pb2(_grpc)?\.py|justfile)$'
pass_filenames: false

- id: node-lockfile-check
name: package-lock.json is up to date
entry: bash -c 'npm install --package-lock-only --ignore-scripts --audit=false --fund=false'
Expand Down
Loading
Loading