Skip to content

feat: Rust port of the data/UI half (export, push, scan, inspect)#30

Merged
dacorvo merged 1 commit into
mainfrom
rust-port-data-ui
Jun 24, 2026
Merged

feat: Rust port of the data/UI half (export, push, scan, inspect)#30
dacorvo merged 1 commit into
mainfrom
rust-port-data-ui

Conversation

@dacorvo

@dacorvo dacorvo commented Jun 24, 2026

Copy link
Copy Markdown
Collaborator

What

Phase 1 of porting agentcap from Python to a single static Rust binary: the data/UI halfexport, HF push, the trufflehog scan gate, ls, and the inspect terminal UI. It builds alongside the existing Python package (src/agentcap/), which stays working and is removed per-command as each reaches parity.

The capture/runtime half (run, the HTTP capture proxy, podman sandbox, agent drivers, orchestrator) is out of scope here — a separate later phase. Until then the Python run produces the captures this half consumes; the on-disk .agentcap/ layout is the contract between the two.

Modules

  • export — render each capture dir to parquet (exact 13-column schema, with agent/model/tasks in the key-value metadata), then push paired <owner>/<base>-captures + per-agent <owner>/<base>-<agent>-traces datasets and group them in a Collection. One commit per push.
  • scan — trufflehog over captures/ traces/ sessions/: verified hits abort, unverified are surfaced but non-blocking; results cached to scan.json; --no-scan to skip.
  • inspect — in-process ratatui + nucleo picker chain (run → request → message, plus local-parquet and hf:// flows) replacing the whole fzf/--listen/preview-subprocess pipeline. Live fuzzy filter, message-diff preview with query highlighting, Esc walk-back. The HF flow streams a footer-read picker (ranged reads, no full download) that fills in as metadata arrives.
  • ls — workspace run listing.

Design notes

  • Synchronous throughout — uses hf-hub's blocking API (each op wraps its own runtime), so there's no tokio to manage; inspect's footer concurrency uses std threads + an mpsc channel feeding the UI.
  • hf-hub only for the Hub (auth, ranged footer reads, commits); Collections go through a small reqwest::blocking REST call (no hf-hub Collections API).
  • Captures live in HF Dataset repos (hf://datasets/owner/name).
  • inspect parses each parquet once into an in-memory map and caches the preview, so navigation stays instant even on large files.

Verification

  • cargo fmt --check, cargo clippy --all-targets -- -D warnings, and cargo test (52 tests) all green. The pure helpers (SSE decode, message diff, fzf-term parsing, rid resolution, parquet round-trip) are ported with their Python tests.
  • Cross-impl: pyarrow reads a Rust-written parquet — schema/column-order, agent/model/tasks KV, non-ASCII content, and the streamed-response wrapper all match the Python exporter's output, so existing Python consumers load Rust-produced datasets unchanged.
  • Manual: ls, the three inspect flows, and a live export --push round-trip exercised end-to-end.

🤖 Generated with Claude Code

Port the read/publish half of agentcap from Python to a single static Rust
binary, alongside the still-present Python package (removed per-command as
parity lands). Phase 1 of a two-phase port; the capture/runtime half (run,
proxy, sandbox, drivers) stays in Python for now and produces the captures
this half consumes.

- export: render capture dirs to parquet (schema + agent/model/tasks KV) and
  push paired captures + per-agent traces datasets, grouped in a Collection.
- scan: trufflehog gate (verified aborts, unverified surfaced; scan.json cache).
- inspect: in-process ratatui + nucleo picker chain (run/request/message,
  parquet, and hf:// flows) replacing the fzf pipeline, with a streaming HF
  footer picker over ranged reads. Parquet rows are parsed once into an
  in-memory map and the preview is cached, so navigation stays instant.
- ls: workspace run listing.

Synchronous throughout (hf-hub blocking API, no tokio); inspect concurrency
uses std threads. Verified: cargo fmt/clippy/test green (52 tests) and pyarrow
reads Rust-written parquets (schema/KV/rows match the Python exporter).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@dacorvo dacorvo merged commit 1a47607 into main Jun 24, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant