Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 55 additions & 8 deletions docs/auth.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,20 +72,22 @@ A repo whose `agent_cli` is `codex` runs the `codex` CLI in its task containers
and codex authenticates differently: it reads credentials from `$CODEX_HOME/auth.json`, not from an
env var directly, and otherwise reaches for an OS keyring the container doesn't have. The container
adapter bridges this — it pins codex to the **file** credential store and, on the container's first
launch, materializes `auth.json` from whatever key you put in the repo's env-file.
launch, materializes `auth.json` from whatever credentials you configure.

So setup is the same shape as claude's: **add one line to the repo's `env_file`** (see *Add it to the
repo's env-file* above for how to create one and point the repo at it). Choose **one**:
Choose **one** of the three tiers:

- **API key** — `OPENAI_API_KEY=sk-…` (or `CODEX_API_KEY=sk-…`; both spellings are accepted). The
standard API-billed key from platform.openai.com. On first launch the adapter writes
`$CODEX_HOME/auth.json` as `{"auth_mode": "apikey", "OPENAI_API_KEY": "…"}` (mode `0600`).
standard API-billed key from platform.openai.com. Add it to the repo's `env_file` (see *One-time
setup* above for how to create one). On first launch the adapter writes `$CODEX_HOME/auth.json`
as `{"auth_mode": "apikey", "OPENAI_API_KEY": "…"}` (mode `0600`).
- **ChatGPT workspace token** — `CODEX_ACCESS_TOKEN=…`, a ChatGPT Business/Enterprise workspace
access token (minted at chatgpt.com/admin → access tokens), the analog of `claude setup-token`.
Codex reads it straight from the env; no file is written.
Add it to the repo's `env_file`. Codex reads it straight from the env; no file is written.
- **ChatGPT Plus/Pro subscription** — rotating tokens that must be shared across containers; see
*Plus/Pro subscription (`credential_dir`)* below.

That's it — new codex task containers for that repo now authenticate from the env-file. There is no
`setup-repo` equivalent for codex yet, so add the line by hand.
That's it for the first two — new codex task containers for that repo now authenticate from the
env-file. There is no `setup-repo` equivalent for codex yet, so add the line by hand.

Notes specific to codex:

Expand All @@ -100,6 +102,51 @@ Notes specific to codex:
live one, clear its `auth.json` from the per-task volume before respawn. (`CODEX_ACCESS_TOKEN`,
read from the env, has no such caching — respawn picks up a change.)

### Plus/Pro subscription (`credential_dir`)

ChatGPT Plus/Pro refresh tokens rotate with reuse detection: every task container on a host must
share the same `auth.json` rather than each holding a stale copy. The `credential_dir` field on the
repo points at a directory in the secrets dir that holds this shared file.

**One-time setup on each host:**

```sh
codex login # or: codex login --device-auth (headless / no browser)
mkdir -p ~/.config/panopticon/secrets/openai.d
cp ~/.codex/auth.json ~/.config/panopticon/secrets/openai.d/
chmod 0600 ~/.config/panopticon/secrets/openai.d/auth.json
```

Then set the repo's `credential_dir` to `openai.d` in the dashboard repo form, or via the API:

```sh
curl -X PATCH "$PANOPTICON_SERVICE_URL/repos/<repo-id>" \
-H 'content-type: application/json' \
-d '{"credential_dir": "openai.d"}'
```

The runner mounts the directory **read-write and shared** into every task container for that repo
at `/panopticon/credentials`; the adapter symlinks `auth.json` from there into each task's
`$CODEX_HOME` on start.

**Why symlinks work here (but not for Claude):** codex opens `auth.json` in-place with
truncate-and-write (`FileAuthStorage::save` in the open-source
`codex-rs/login/src/auth/storage.rs`), following the symlink to the shared host file. The
refreshed token therefore lands in the shared directory and all concurrent containers on the host
stay in sync. Claude's OAuth refresh is an atomic temp-file rename that *replaces* the symlink
with a container-local regular file.

**Important constraints:**

- **Do not copy the same `auth.json` to a second host.** OpenAI's reuse detection fires when
the same file is used from two different IPs. Log in per host (`codex login` on each), or use
an access token (tier 2 above) for multi-host setups.
- **Invalidated chain** (re-login elsewhere, revocation): tasks will fail with a lifecycle detail
describing the fix. Re-run `codex login` on the host and copy the new `auth.json` to the
credential dir.
- **Rate limits** (not auth) cap concurrent Plus/Pro Codex throughput. An API key (tier 1)
avoids the subscription concurrency constraints.

## Notes (both CLIs)

- **The env-file lives on the host that spawns the container.** Because `env_file` is stored as a
Expand Down
2 changes: 2 additions & 0 deletions src/panopticon/client.py
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,7 @@ def create_repo(
default_base: str = "main",
*,
env_file: str | None = None,
credential_dir: str | None = None,
image_layer_file: str | None = None,
hook_file: str | None = None,
capabilities: dict[str, Any] | None = None,
Expand All @@ -142,6 +143,7 @@ def create_repo(
"git_url": git_url,
"default_base": default_base,
"env_file": env_file,
"credential_dir": credential_dir,
"image_layer_file": image_layer_file,
"hook_file": hook_file,
"enabled_workflows": enabled_workflows or [],
Expand Down
35 changes: 25 additions & 10 deletions src/panopticon/container/cli/codex.py
Original file line number Diff line number Diff line change
Expand Up @@ -230,17 +230,22 @@ def auth_missing_detail(self, env: Mapping[str, str], config_dir: Path) -> str |
which :meth:`write_credentials` materializes into ``auth.json``) or a ChatGPT workspace access
token (``CODEX_ACCESS_TOKEN``, read straight from the env) — **or** a pre-existing
``auth.json`` on the per-task config volume (a container already logged in, e.g. carried
across respawn — which a bare env check would wrongly fail). Presence checks only: we don't
validate the key shape (OpenAI's format isn't ours to pin); an invalid credential surfaces at
codex's first call.
across respawn) — **or** a ``PANOPTICON_CREDENTIALS`` mount holding ``auth.json`` (ChatGPT
Plus/Pro subscription; :meth:`write_credentials` symlinks it into the config dir). Presence
checks only: we don't validate the key shape; an invalid credential surfaces at codex's first
call.
"""
if any(env.get(var) for var in (*self.API_KEY_VARS, self.ACCESS_TOKEN_VAR)):
return None
if (config_dir / self.AUTH_FILE).exists():
return None
creds = env.get("PANOPTICON_CREDENTIALS")
if creds and (Path(creds) / self.AUTH_FILE).exists():
return None
return (
"No codex auth — set OPENAI_API_KEY (or CODEX_API_KEY / CODEX_ACCESS_TOKEN) in the "
"repo's env_file (see docs/auth.md)"
"repo's env_file, or give the repo a credential_dir holding a ChatGPT auth.json "
"(see docs/auth.md)"
)

def write_credentials(self, config_dir: Path, env: Mapping[str, str]) -> Path | None:
Expand All @@ -253,21 +258,31 @@ def write_credentials(self, config_dir: Path, env: Mapping[str, str]) -> Path |
- set ``cli_auth_credentials_store = "file"`` (top-level ``config.toml``) so codex reads
credentials from the file, never a keyring — done unconditionally, so it also governs a
pre-existing ``auth.json`` carried across respawn;
- when ``auth.json`` is absent, render it from ``CODEX_API_KEY`` or ``OPENAI_API_KEY`` in the
- when ``auth.json`` is absent, try the credential-dir mount first: if
``PANOPTICON_CREDENTIALS`` points at a directory containing ``auth.json``, create a
**symlink** from the config dir into that shared host path. Codex opens ``auth.json``
in-place with truncate-and-write (``FileAuthStorage::save`` in the open-source
``codex-rs/login/src/auth/storage.rs``), following the symlink to the shared file, so
refreshed tokens propagate back and all concurrent containers on the host stay consistent.
- otherwise, render ``auth.json`` from ``CODEX_API_KEY`` or ``OPENAI_API_KEY`` in the
exact shape ``codex login --with-api-key`` writes — ``{"auth_mode": "apikey",
"OPENAI_API_KEY": <key>}`` — at mode ``0600``.

**Idempotent: an existing ``auth.json`` is never clobbered**, so a container already logged in
keeps its credentials. Returns the ``auth.json`` path when written, else ``None`` (no API key,
or one already present). A workspace access token (``CODEX_ACCESS_TOKEN``) needs no file —
codex reads it from the env — so it doesn't trigger a write here (the auth gate accepts it).
**Idempotent: an existing ``auth.json`` (or symlink, even dangling) is never clobbered**,
so a container already logged in keeps its credentials. Returns the ``auth.json`` path when
written or symlinked, else ``None``. A workspace access token (``CODEX_ACCESS_TOKEN``) needs
no file — codex reads it from the env — so it doesn't trigger a write here.
"""
config = config_dir / self.CONFIG_FILE
with update_toml_config(config) as data:
data["cli_auth_credentials_store"] = "file"
auth = config_dir / self.AUTH_FILE
if auth.exists():
if auth.exists() or auth.is_symlink(): # is_symlink catches a dangling symlink
return None
creds = env.get("PANOPTICON_CREDENTIALS")
if creds and (Path(creds) / self.AUTH_FILE).exists():
auth.symlink_to(Path(creds) / self.AUTH_FILE)
return auth
key = next((env[var] for var in self.API_KEY_VARS if env.get(var)), None)
if not key:
return None
Expand Down
43 changes: 43 additions & 0 deletions src/panopticon/core/dirs.py
Original file line number Diff line number Diff line change
Expand Up @@ -148,6 +148,49 @@ def secrets_file_path(name: str | None, *, secrets_dir: str | Path | None = None
return str(path)


def credential_dir_path(name: str | None, *, secrets_dir: str | Path | None = None) -> str | None:
"""Resolve a stored ``credential_dir`` *name* to an absolute path under the secrets dir.

Mirrors :func:`secrets_file_path` but for a directory reference. ``name`` is a path relative
to the secrets dir (see :data:`SECRETS_DIR`); ``secrets_dir`` defaults to this host's.
Returns ``None`` for a falsy name; raises :class:`ValueError` on path escape (``..`` or an
absolute name). The runner calls this to build the ``docker run --volume`` argument for the
shared credential-dir mount (``/panopticon/credentials``).
"""
if not name:
return None
root = (Path(secrets_dir) if secrets_dir is not None else _secrets_dir()).resolve()
path = (root / name).resolve()
if path != root and root not in path.parents:
raise ValueError(f"credential_dir name {name!r} escapes the secrets dir")
return str(path)


def relativize_credential_dir(path: str, *, secrets_dir: str | Path | None = None) -> str:
"""Normalize a user-entered credential-dir ``path`` to a name relative to the secrets dir.

The ``credential_dir`` analogue of :func:`relativize_secrets_file`. Accepts an absolute or
relative path and always yields a stored *relative name*:

- a path inside the secrets dir → its subpath relative to the dir;
- any other absolute path → its basename;
- a relative path → returned unchanged.

An empty/whitespace input yields ``""``. ``secrets_dir`` defaults to this host's.
"""
path = path.strip()
if not path:
return ""
p = Path(path)
if p.is_absolute():
root = (Path(secrets_dir) if secrets_dir is not None else _secrets_dir()).resolve()
resolved = p.resolve()
if resolved == root or root in resolved.parents:
return str(resolved.relative_to(root))
return p.name
return path


def _hooks_dir() -> Path:
"""The hooks dir, resolved dynamically (so ``$PANOPTICON_CONFIG``/XDG overrides take effect).

Expand Down
11 changes: 10 additions & 1 deletion src/panopticon/core/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -170,13 +170,21 @@ class Tool:
class Repo:
"""A repository tasks operate on.

Holds a *reference* to its per-repo secrets (ADR 0007), never the values: ``env_file`` is a
Holds *references* to its per-repo secrets (ADR 0007), never the values: ``env_file`` is a
**name relative to the secrets dir** (``$PANOPTICON_CONFIG/secrets``) naming an env-file of
API-key-style secrets, injected into the task container at launch (``--env-file``), so secrets
stay out of the DB, artifacts, and image layers. The runner resolves it against its **own**
host's secrets dir, so a remote runner uses its own secrets and the value stays host-agnostic;
the file's content never crosses the wire.

``credential_dir`` is also a **name relative to the secrets dir**, but naming a *directory*
that holds rotating credential files (e.g. ``openai.d/`` containing ``auth.json`` for a Codex
ChatGPT-subscription login). The runner mounts it **read-write** at ``/panopticon/credentials``
and exports ``PANOPTICON_CREDENTIALS`` so the in-container adapter can find it. The mount is
shared across all containers for the same repo on the same host, letting codex write refreshed
tokens back through a symlink (see ``docs/auth.md`` and ADR 0012 for why symlinks work for
codex but not Claude).

``image_layer_file`` *references* the repo's Dockerfile fragment (ADR 0005's repo tier): a file
name resolved relative to the task service's layers directory, not inline content. The task
service reads it to serve over REST (``GET /repos/{id}/image-layer``) and the runner composes
Expand All @@ -202,6 +210,7 @@ class Repo:
git_url: str
default_base: str = "main"
env_file: str | None = None
credential_dir: str | None = None
image_layer_file: str | None = None
capabilities: dict[str, Any] = field(default_factory=dict)
hook_file: str | None = None
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
"""add repo.credential_dir

Revision ID: f634214d376c
Revises: 41456592aa95
Create Date: 2026-08-24 15:09:28.031706
"""

from __future__ import annotations

from collections.abc import Sequence

import sqlalchemy as sa
from alembic import op

# revision identifiers, used by Alembic.
revision: str = "f634214d376c"
down_revision: str | None = "41456592aa95"
branch_labels: str | Sequence[str] | None = None
depends_on: str | Sequence[str] | None = None


def upgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
with op.batch_alter_table("repo", schema=None) as batch_op:
batch_op.add_column(sa.Column("credential_dir", sa.String(), nullable=True))

# ### end Alembic commands ###


def downgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
with op.batch_alter_table("repo", schema=None) as batch_op:
batch_op.drop_column("credential_dir")

# ### end Alembic commands ###
8 changes: 7 additions & 1 deletion src/panopticon/sessionservice/local_runner.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
from pathlib import Path
from typing import Protocol

from panopticon.core.dirs import secrets_file_path
from panopticon.core.dirs import credential_dir_path, secrets_file_path
from panopticon.core.models import DEFAULT_AGENT_CLI, LifecyclePhase
from panopticon.sessionservice.runner import Runner

Expand Down Expand Up @@ -158,6 +158,7 @@ def spawn(
task_id: str,
*,
env_file: str | None = None,
credential_dir: str | None = None,
workspace: str | None = None,
image: str | None = None,
docker_in_docker: bool = False,
Expand Down Expand Up @@ -241,6 +242,11 @@ def _report(phase: LifecyclePhase) -> None:
env["PANOPTICON_DOCKER_IN_DOCKER"] = "1"
if env_path := secrets_file_path(env_file, secrets_dir=self._secrets_dir):
docker_run += ["--env-file", env_path] # per-repo secrets, resolved host-locally
if cred_path := credential_dir_path(credential_dir, secrets_dir=self._secrets_dir):
# Shared read-write mount: rotating tokens (e.g. codex auth.json) write back through
# the in-container symlink to this host directory, keeping concurrent tasks in sync.
docker_run += ["--volume", f"{cred_path}:/panopticon/credentials:rw"]
env["PANOPTICON_CREDENTIALS"] = "/panopticon/credentials"
if workspace: # the per-task clone — the agent's writable working dir (ADR 0011)
docker_run += [
"--volume",
Expand Down
1 change: 1 addition & 0 deletions src/panopticon/sessionservice/spawner.py
Original file line number Diff line number Diff line change
Expand Up @@ -230,6 +230,7 @@ def _spawn_container(self, task: JsonObj, repo: JsonObj) -> str:
return self._runner.spawn(
task_id,
env_file=repo.get("env_file"),
credential_dir=repo.get("credential_dir"),
workspace=workspace,
image=image,
docker_in_docker=bool((repo.get("capabilities") or {}).get("docker_in_docker")),
Expand Down
26 changes: 25 additions & 1 deletion src/panopticon/taskservice/service.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
from typing import Any

from panopticon.core.artifacts import ArtifactStore, decode_b64_artifact
from panopticon.core.dirs import secrets_file_path
from panopticon.core.dirs import credential_dir_path, secrets_file_path
from panopticon.core.layers import LayerStore
from panopticon.core.models import (
Actor,
Expand Down Expand Up @@ -147,6 +147,7 @@ async def init(self) -> None:

async def create_repo(self, repo: Repo) -> Repo:
await self._validate_env_file(repo.env_file)
await self._validate_credential_dir(repo.credential_dir)
await self._store.create_repo(repo)
return repo

Expand All @@ -172,6 +173,27 @@ async def _validate_env_file(self, env_file: str | None) -> None:
if not await asyncio.to_thread(os.path.isfile, path):
raise ValueError(f"env_file {env_file!r} does not exist under the secrets dir")

async def _validate_credential_dir(self, credential_dir: str | None) -> None:
"""Reject a repo whose credential-dir reference points at a missing directory.

``credential_dir`` is a *name* relative to the secrets dir — the same root as
``env_file`` (ADR 0007). Validated on create/update so a bad reference surfaces at
registration rather than as an obscure ``--volume`` failure at spawn. ``None`` (no
credential dir) is valid. Raises :class:`ValueError` for a name that escapes the secrets
dir or one that resolves to a missing or non-directory path.

NOTE(M5): resolved against *this host's* secrets dir (same caveat as
:meth:`_validate_env_file`).
"""
path = credential_dir_path(credential_dir) # None for no reference; raises on escape
if path is None:
return
if not await asyncio.to_thread(os.path.isdir, path):
raise ValueError(
f"credential_dir {credential_dir!r} does not exist or is not a directory"
" under the secrets dir"
)

async def get_repo(self, repo_id: str) -> Repo:
repo = await self._store.get_repo(repo_id)
if repo is None:
Expand All @@ -196,6 +218,8 @@ async def update_repo(self, repo_id: str, changes: Mapping[str, Any]) -> Repo:
await self._validate_env_file(
updated.env_file
) # so an unrelated patch never fails on it
if "credential_dir" in changes:
await self._validate_credential_dir(updated.credential_dir)
await self._store.update_repo(updated)
return updated

Expand Down
Loading
Loading