feat: Rust port of the data/UI half (export, push, scan, inspect)#30
Merged
Conversation
Port the read/publish half of agentcap from Python to a single static Rust binary, alongside the still-present Python package (removed per-command as parity lands). Phase 1 of a two-phase port; the capture/runtime half (run, proxy, sandbox, drivers) stays in Python for now and produces the captures this half consumes. - export: render capture dirs to parquet (schema + agent/model/tasks KV) and push paired captures + per-agent traces datasets, grouped in a Collection. - scan: trufflehog gate (verified aborts, unverified surfaced; scan.json cache). - inspect: in-process ratatui + nucleo picker chain (run/request/message, parquet, and hf:// flows) replacing the fzf pipeline, with a streaming HF footer picker over ranged reads. Parquet rows are parsed once into an in-memory map and the preview is cached, so navigation stays instant. - ls: workspace run listing. Synchronous throughout (hf-hub blocking API, no tokio); inspect concurrency uses std threads. Verified: cargo fmt/clippy/test green (52 tests) and pyarrow reads Rust-written parquets (schema/KV/rows match the Python exporter). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Phase 1 of porting agentcap from Python to a single static Rust binary: the data/UI half —
export, HF push, the trufflehogscangate,ls, and theinspectterminal UI. It builds alongside the existing Python package (src/agentcap/), which stays working and is removed per-command as each reaches parity.The capture/runtime half (
run, the HTTP capture proxy, podman sandbox, agent drivers, orchestrator) is out of scope here — a separate later phase. Until then the Pythonrunproduces the captures this half consumes; the on-disk.agentcap/layout is the contract between the two.Modules
export— render each capture dir to parquet (exact 13-column schema, withagent/model/tasksin the key-value metadata), then push paired<owner>/<base>-captures+ per-agent<owner>/<base>-<agent>-tracesdatasets and group them in a Collection. One commit per push.scan— trufflehog overcaptures/ traces/ sessions/: verified hits abort, unverified are surfaced but non-blocking; results cached toscan.json;--no-scanto skip.inspect— in-process ratatui + nucleo picker chain (run → request → message, plus local-parquet andhf://flows) replacing the whole fzf/--listen/preview-subprocess pipeline. Live fuzzy filter, message-diff preview with query highlighting, Esc walk-back. The HF flow streams a footer-read picker (ranged reads, no full download) that fills in as metadata arrives.ls— workspace run listing.Design notes
blockingAPI (each op wraps its own runtime), so there's no tokio to manage;inspect's footer concurrency uses std threads + an mpsc channel feeding the UI.reqwest::blockingREST call (no hf-hub Collections API).hf://datasets/owner/name).inspectparses each parquet once into an in-memory map and caches the preview, so navigation stays instant even on large files.Verification
cargo fmt --check,cargo clippy --all-targets -- -D warnings, andcargo test(52 tests) all green. The pure helpers (SSE decode, message diff, fzf-term parsing, rid resolution, parquet round-trip) are ported with their Python tests.agent/model/tasksKV, non-ASCII content, and the streamed-response wrapper all match the Python exporter's output, so existing Python consumers load Rust-produced datasets unchanged.ls, the threeinspectflows, and a liveexport --pushround-trip exercised end-to-end.🤖 Generated with Claude Code