refactor(compute): decouple gateway driver composition

[drew]

Summary

Move first-party compute-driver composition and VM process ownership into openshell-gateway, leaving openshell-server independent of compute backend implementations. The gateway can build without compute drivers, compile first-party drivers through generic registration, or connect to drivers through the public compute-driver RPC.

Related Issue

Part of #2417.

Changes

• add openshell-gateway as the binary and first-party driver composition crate
• gate compiled first-party registrations behind in-tree-compute-drivers
• keep openshell-server free of compute-driver implementation dependencies and construction logic
• move VM subprocess launch and first-party registration to the gateway composition boundary
• support driver-free gateway builds and external Docker, Podman, Kubernetes, and VM driver binaries
• bound compute-driver telemetry categories without backend-specific server logic
• configure sandbox token bootstrap under the gateway-owned [openshell.gateway.sandbox_token_bootstrap] table
• retain existing compiled Kubernetes configuration through a generic registration compatibility default while unrelated external drivers require no Kubernetes table
• carry user_namespaces as a typed compute-driver field and perform Kubernetes hostUsers translation in the Kubernetes driver
• accept the legacy Kubernetes platform_config.host_users encoding for rolling upgrades
• update architecture, gateway configuration, Helm rendering/tests, packaging, deployment, and troubleshooting guidance
• remove stale backend-specific comments and obsolete build references

Testing

• mise run pre-commit
• OPENSHELL_SYSTEM_GATEWAY_DIR=/tmp/openshell-empty-system-config-codex-pr2823 mise run test
• mise run helm:test (106 tests)
• cargo check -p openshell-gateway --no-default-features
• cargo check -p openshell-gateway --features in-tree-compute-drivers
• focused server, gateway registration, and Kubernetes user-namespace tests
• mise run e2e:gateway:no-compute-drivers
• external-driver E2E matrix runs in branch CI

Checklist

• Changes are scoped to accepted issue refactor(compute): define driver shutdown and restart lifecycle contract #2417
• Follows Conventional Commits
• Commits are signed off (DCO)
• Architecture documentation is updated
• Public gateway configuration documentation is updated
• Public compute-driver changes are additive and retain legacy Kubernetes input compatibility
• PR remains draft

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[github-actions]

🌿 Preview your docs: https://nvidia-preview-pr-2823.docs.buildwithfern.com/openshell

[drew]

/ok to test

[drew]

/ok to test

[drew]

/ok to test

[drew]

/ok to test

[elezar]

@drew This branch update is rebase-only: PR #2823 was rebased onto the restored PR #2822 head. No intentional feature or scope changes were added to this PR; the conflict resolution preserves the parent stack behavior while retaining the existing gateway-composition and telemetry commits.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[drew]

/ok to test

[drew]

/ok to test

[drew]

/ok to test