Skip to content

Bound SDK event logs while preserving screenshot evidence - #2899

Open
miguelg719 wants to merge 9 commits into
evals/consolidation-08-native-routingfrom
evals/consolidation-09-event-retention
Open

Bound SDK event logs while preserving screenshot evidence#2899
miguelg719 wants to merge 9 commits into
evals/consolidation-08-native-routingfrom
evals/consolidation-09-event-retention

Conversation

@miguelg719

@miguelg719 miguelg719 commented Sep 8, 2026

Copy link
Copy Markdown
Collaborator

Bounds retained SDK event logs while keeping actionable diagnostics and trajectory evidence. Pi screenshot compaction and its Buffer-aware trajectory consumer land together, preventing intermediate screenshot loss.

Validation: 118 SDK/event tests, five Pi adapter/pipeline tests, 12 build tasks, and SDK/core/eval typechecks passed. The pipeline test imports the actual built Pi compactor and checks retained image bytes for compacted Buffer and historical base64 events.

Stack position: 9/17. Base: evals/consolidation-08-native-routing. Each PR contains the change relative to its immediate predecessor.

Reviewer entry points:

Readiness: local validation is described above; GitHub CI and automated review feedback are pending. This PR is open as non-draft so automated reviewers can run.


Summary by cubic

Bounds retained SDK event logs so they stop growing with stream noise and repeated payloads, while keeping error visibility and screenshot evidence for trajectories.

  • Event log classification: Added harnessEventLogLevel in @browserbasehq/stagehand-integrations/harness to drop stream deltas and lifecycle markers, keep errors at level 1, and demote everything else to debug. Applied across Claude, Codex, DeepAgents, Eve, Fx, Mastra, and Pi sessions.
  • Screenshot retention: Pi screenshots are decoded to a single Buffer on tool_execution_end; base64 is stripped from logs and non-assistant messages, and the trajectory adapter accepts both Buffer and base64. Images over 8 MiB or past the 64 MiB per-run budget become explicit text omission markers. Mastra step-finish/finish payloads are compacted to stop quadratic growth from repeated request bodies.

Written for commit caad20b. Summary will update on new commits.

Review in cubic

@changeset-bot

changeset-bot Bot commented Sep 8, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: caad20b

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 21 files

Architecture diagram
sequenceDiagram
    participant SDK as SDK Agent
    participant Session as Session Runner
    participant Classifier as harnessEventLogLevel
    participant Logger as Harness Logger
    participant Pi as Pi Session Compactor
    participant Adapter as Pi Adapter
    
    Note over SDK,Adapter: Event ingestion and logging classification
    SDK->>Session: Emit raw event
    Session->>Classifier: Classify event type
    alt Stream delta or bare lifecycle
        Classifier-->>Session: undefined (drop)
        Session->>Logger: Skip logging
    else Error event
        Classifier-->>Session: level 1
        Session->>Logger: Log with level 1
    else Content event
        Classifier-->>Session: level 2
        Session->>Logger: Log with level 2
    end
    
    Note over Session,Adapter: Pi screenshot evidence pipeline
    SDK->>Session: tool_execution_end with base64 image
    Session->>Pi: compactPiEvent(event)
    Pi->>Pi: Decode image block to Buffer
    Pi-->>Session: Compacted event with bytes
    
    Session->>Adapter: Pass compacted trajectory events
    alt Buffer image block
        Adapter->>Adapter: Use bytes directly
        Adapter-->>Session: Image modality with Buffer
    else Legacy base64 block
        Adapter->>Adapter: Decode base64 to Buffer
        Adapter-->>Session: Image modality with Buffer
    end
    
    Note over Session,Logger: Log sanitization
    Session->>Session: withoutImageData(event)
    Session->>Logger: Log with [image N bytes] placeholder
    Logger-->>Session: Sanitized log entry
    
    Note over Session,Adapter: Mastra event compaction
    Session->>Session: compactMastraEvent(event)
    alt Retained event type
        Session-->>Session: Keep full event
    else step-finish/finish
        Session->>Session: Strip request/response bodies
        Session-->>Session: Keep only stepResult and usage
    end
Loading

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread packages/integrations/pi-sdk/src/session.ts Outdated
Comment thread packages/integrations/claude-agent-sdk/src/session.ts Outdated
Comment thread packages/integrations/eve-sdk/src/session.ts Outdated
Comment thread packages/integrations/codex-sdk/src/session.ts Outdated
Comment thread packages/integrations/mastra-sdk/src/session.ts
Comment thread packages/integrations/fx-sdk/src/session.ts Outdated
Comment thread packages/integrations/pi-sdk/src/session.ts Outdated
Comment thread packages/integrations/pi-sdk/src/session.ts Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants