fix(cloudxr): tear down on signal and join main on Orin

[nv-mhaselton]

Description

Signalled shutdown of an embedding process previously left the CloudXR runtime running. The runtime is spawned with start_new_session=True, and Python does not run atexit handlers on SIGTERM/SIGINT, so the next start could fail with PORT_UNAVAILABLE. This change installs SIGTERM/SIGINT handlers (and atexit) after Popen so the session-scoped runtime is torn down cleanly, then invokes any previously installed handler so the embedding app keeps its normal shutdown behavior. Teardown keeps those handlers installed until stop finishes, guards re-entrant stop(), and restores a prior handler only when the launcher wrapper is still installed.

Separately, joining nv_cxr_service_join on a worker thread can abort on Orin with _PyGILState_NoteThreadState: Couldn't create autoTSSkey mapping after V4L2 encoder teardown. Main-thread join is selected automatically on Orin. Because Python handlers cannot run while that join blocks, SIGINT/SIGTERM are handled via sigwait so nv_cxr_service_stop can still run. Override with ISAAC_TELEOP_CLOUDXR_JOIN_MAIN=0|1 (unset keeps the platform default). Launcher overrides (ISAAC_TELEOP_CLOUDXR_EXP, ISAAC_TELEOP_CLOUDXR_JOIN_MAIN) are documented as Orin auto / override-only in the CloudXR reference, with a short note under the quick-start env table.

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation update

Testing

• Unit: ctest --test-dir build --output-on-failure -R 'cloudxr' (launcher + runtime, including join-main sigwait coverage).
• Smoke on Orin: join auto-selected; start → SIGTERM → restart with no orphans / no PORT_UNAVAILABLE.
• Smoke on Thor: join not auto-selected; same teardown/restart path ok.

Checklist

• I have read and understood the contribution guidelines
• I have run the linter and formatter with SKIP=check-copyright-year pre-commit run --all-files
• I have made corresponding changes to the documentation
• I have added tests that prove my fix/feature works (or explained why not)
• I have signed off all my commits (git commit -s) per the DCO

[github-actions]

📝 Docs preview is not auto-deployed for fork PRs.

A maintainer with write access to NVIDIA/IsaacTeleop can deploy a preview by
commenting /preview-docs on this PR. Once deployed, the preview
will live at:

https://nvidia.github.io/IsaacTeleop/preview/pr-816/

[coderabbitai]

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: cdc89696-19e8-41cf-94b5-8d6cb87e6d30

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

📝 Walkthrough

Walkthrough

CloudXR runtime shutdown can now join and destroy the native service on the main thread or a worker thread based on ISAAC_TELEOP_CLOUDXR_JOIN_MAIN, with Orin-based default selection. Documentation and tests cover the setting. CloudXRLauncher now registers exit cleanup earlier, handles SIGTERM and SIGINT, chains existing handlers, and restores them during shutdown.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Environment
  participant CloudXRRuntime
  participant NativeCloudXRService
  participant WorkerThread
  Environment->>CloudXRRuntime: Provide join policy
  CloudXRRuntime->>CloudXRRuntime: Resolve main-thread or worker-thread path
  alt Main-thread path
    CloudXRRuntime->>NativeCloudXRService: Join and destroy on main thread
  else Worker-thread path
    CloudXRRuntime->>WorkerThread: Run blocking join and destroy
    WorkerThread->>NativeCloudXRService: Join and destroy
  end

Loading

sequenceDiagram
  participant OS
  participant CloudXRLauncher
  participant WSSProxy
  participant CloudXRRuntime
  OS->>CloudXRLauncher: SIGTERM or SIGINT
  CloudXRLauncher->>CloudXRLauncher: Restore previous handlers
  CloudXRLauncher->>WSSProxy: Stop proxy
  CloudXRLauncher->>CloudXRRuntime: Terminate runtime
  CloudXRLauncher->>OS: Chain previous handler

Loading

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly matches the main changes: signal-based teardown and main-thread CloudXR join on Orin.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/core/cloudxr/python/launcher.py`:
- Line 460: Update the launcher teardown flow around _restore_signal_handlers so
signal handlers remain active while _stop_wss_proxy() and _terminate_runtime()
complete. Move handler restoration into a finally block after both teardown
operations, and add a re-entry guard to prevent repeated signal handling during
teardown.
- Around line 543-550: Update _restore_signal_handlers() to inspect the current
handler before restoring each saved signal, and only call signal.signal() when
it is still the launcher-installed wrapper. Remove each entry from
_prev_signal_handlers only after a successful restore, retaining it when
restoration fails so a worker-thread stop cannot discard bookkeeping or
overwrite a host-installed handler.

In `@src/core/cloudxr/python/runtime.py`:
- Around line 340-356: Add an out-of-band shutdown mechanism for the
_should_join_main() branch so SIGINT/SIGTERM can invoke nv_cxr_service_stop(svc)
while the main thread is blocked in nv_cxr_service_join(). Preserve the existing
main-thread join and destruction flow, and add an Orin-specific regression test
verifying SIGTERM stops and cleans up the service successfully.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: dc852a71-f2d4-48a1-b097-19e0b3fa38eb

📥 Commits

Reviewing files that changed from the base of the PR and between 6258095 and 217c5f2.

📒 Files selected for processing (4)

• docs/source/references/cloudxr.rst
• src/core/cloudxr/python/launcher.py
• src/core/cloudxr/python/runtime.py
• src/core/cloudxr_tests/python/test_runtime.py

[aristarkhovNV]

Is it possible to converge on the same Main-thread join code path on all platforms?

[nv-mhaselton]

Is it possible to converge on the same Main-thread join code path on all platforms?

The thought was to keep this workaround Orin-specific. On other platforms the worker-thread join is useful so the main thread can still run Python signal handlers (Ctrl+C) while blocked in native nv_cxr_service_join. Converging everyone onto main-thread join would give up that behavior and rely on launcher process teardown everywhere. I figured we'd rather keep the worker-thread join as the long-term path and move Orin over to it once the abort is gone.

On Orin, starting any new Python thread after the native service is up aborts with _PyGILState_NoteThreadState: Couldn't create autoTSSkey mapping (including a join worker and a sigwait helper we tried), so join stays on the main thread there.

I've added more clarification around the workaround in runtime.py. Happy to revisit a single (worker-thread) path once the Orin abort is gone.