Skip to content

Conversation

@hellomypastor
Copy link
Contributor

Summary

  • Added a new examples/rl-training example demonstrating RL training (CartPole + DQN) inside OpenSandbox, including dependency installation, training, checkpointing, and summary output. Also updated
    the examples index and aligned dependencies with the default opensandbox/code-interpreter:latest image. Resolves Add Reinforcement Learning (RL) Sandbox Example #29.

Testing

  • Not run (example is environment-dependent; requires running OpenSandbox server and sandbox image)
  • Unit tests
  • Integration tests
  • e2e / manual verification

Breaking Changes

  • None
  • Yes (describe impact and migration path)

Checklist

  • Linked Issue or clearly described motivation
  • Added/updated docs (if needed)
  • Added/updated tests (if needed)
  • Security impact considered
  • Backward compatibility considered

@jwx0925
Copy link
Collaborator

jwx0925 commented Dec 25, 2025

@codex review

Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +156 to +158
if not await _ensure_pip(sandbox):
print("Failed to bootstrap pip inside the sandbox.")
return

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Early returns leave sandbox running on failure paths

If pip bootstrap, dependency installation, or training fails, the function returns from inside the async with sandbox block before reaching the kill call at the end, while Sandbox.__aexit__ only closes client resources and does not terminate the remote instance. This leaves an orphaned sandbox running until an external timeout, consuming resources in failure scenarios; wrapping the block in a try/finally that always calls sandbox.kill() would avoid leaks.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add Reinforcement Learning (RL) Sandbox Example

2 participants