Repository navigation
Conversation
…e, screenshots from live runs
DABH
marked this pull request as draft
September 11, 2026 17:56
This was referenced Sep 11, 2026
The OpenAI Agents plugin now ships as the standalone temporalio-openai-agents package (import path temporalio.openai_agents); temporalio 1.33's openai-agents extra pins an older openai-agents and no longer resolves alongside it. Switch the dependency group and imports, require temporalio 1.33, and raise the OpenInference agents instrumentation floor to 2.x. The single-process span-parenting bug (sdk-python#1852) is fixed in ai-integrations#26 but 1.0.0 predates it, so the --role split stays; reword the notes accordingly. Also bump Phoenix to 20.16.0 and state that the Arize AX exporter path was not exercised live.
DABH
marked this pull request as ready for review
October 1, 2026 16:32
temporalio-openai-agents 1.1.0 carries the span-parenting fix for running workflows and activities in one worker, but it requires openai 3, and the root lock stays on openai 2 for the LiteLLM samples (litellm caps openai<3). Give the sample its own uv project, like openai_agents/mcp_v2, and add it to the standalone CI matrix. With the fix in place, drop the --role split, the workflow-only plugin switch, the detach-context log filter, and the known-issue notes; update the trace diagrams and the screenshot from a single-process run.
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
The activity heartbeat configuration can cause duplicate LLM calls, and the verifier can report unreliable ingestion and deduplication results.
Review effort: Balanced
Findings: 1
Open (3)
What changed in this PR
Adds a standalone Arize tracing sample demonstrating replay-safe Temporal traces for standard OpenAI activities and the OpenAI Agents SDK.
Changes:
- Adds shared OpenTelemetry/OpenInference configuration and Phoenix verification tooling.
- Adds two ticket-triage scenarios with offline integration tests.
- Integrates the sample into CI, documentation, and ownership configuration.
| File | Description |
|---|---|
.github/CODEOWNERS |
Assigns sample ownership. |
.github/workflows/ci.yml |
Adds standalone sample CI. |
README.md |
Lists the Arize sample. |
uv.lock |
Updates OpenInference packages. |
arize_tracing/.env.example |
Documents environment configuration. |
arize_tracing/README.md |
Provides the main runbook. |
arize_tracing/pyproject.toml |
Defines the standalone project. |
arize_tracing/uv.lock |
Locks standalone dependencies. |
arize_tracing/telemetry.py |
Configures tracing and enrichment. |
arize_tracing/verify_trace.py |
Verifies Phoenix traces. |
arize_tracing/phoenix/docker-compose.yml |
Runs local Phoenix. |
arize_tracing/phoenix-retry-attempts.png |
Shows retry tracing. |
arize_tracing/phoenix-ticket-triage.png |
Shows activity-based tracing. |
arize_tracing/phoenix-ticket-triage-agents.png |
Shows agent tracing. |
arize_tracing/ticket_triage/__init__.py |
Marks the scenario package. |
arize_tracing/ticket_triage/README.md |
Documents the activity scenario. |
arize_tracing/ticket_triage/activities.py |
Implements triage activities. |
arize_tracing/ticket_triage/starter.py |
Starts and traces workflows. |
arize_tracing/ticket_triage/worker.py |
Runs the activity worker. |
arize_tracing/ticket_triage/workflows.py |
Implements the triage workflow. |
arize_tracing/ticket_triage_agents/__init__.py |
Marks the agents package. |
arize_tracing/ticket_triage_agents/README.md |
Documents the agents scenario. |
arize_tracing/ticket_triage_agents/plugin.py |
Configures the Agents plugin. |
arize_tracing/ticket_triage_agents/starter.py |
Starts agent workflows. |
arize_tracing/ticket_triage_agents/worker.py |
Runs the agents worker. |
arize_tracing/ticket_triage_agents/workflows.py |
Implements agent-based triage. |
arize_tracing/tests/__init__.py |
Marks the test package. |
arize_tracing/tests/conftest.py |
Defines standalone fixtures. |
arize_tracing/tests/helpers.py |
Provides span-tree helpers. |
arize_tracing/tests/test_ticket_triage.py |
Tests activity tracing and replay. |
arize_tracing/tests/test_ticket_triage_agents.py |
Tests agent tracing and replay. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
classify_ticket heartbeated only in the --slow-classify loop, so a model response longer than the 10-second heartbeat timeout would have been retried while the first request was still in flight. Run the call through a helper that heartbeats until it completes. verify_trace.py now waits for the root span plus three consecutive polls with an unchanged span count before checking, and no longer claims to detect duplicate exports: Phoenix drops same-id inserts, so only the in-memory tests can observe re-emission.
brianstrauch
reviewed
Oct 6, 2026
moetemp
approved these changes
Oct 7, 2026
|
AX test results and one reset issue are on the docs PR: temporalio/documentation#5306 (comment) |
# Conflicts: # .github/CODEOWNERS
Reuse one lazily created OpenAI client per worker and close it on shutdown instead of building an unclosed client per attempt. Rename the verifier's --expect-attempt flag to --surviving-attempt, validate ACTIVITY=N values, and only finish polling once the root span and one completed-run span per expected run are present, retrying the checks until the deadline. Document what Arize observed against an AX space: a retried activity marks the trace as an error there; stopping a worker mid-workflow logs a harmless "Failed to detach context" traceback; and after a reset that reapplies the update, the reapplied HandleUpdate span reuses the original run's RunWorkflow span id, which Phoenix drops and AX shows as a duplicate subtree. The verifier now warns about the dropped span. Drop the note that the AX path was not exercised.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
arize_tracing/, a sample that sends Temporal workflow traces to Arize Phoenix (self-hosted, one container) or Arize AX through the OpenTelemetry plugin, using the OpenInference conventions Arize renders. Companion to the Langfuse sample and to Arize's own Temporal guide.Two scenarios share one telemetry module.
ticket_triage/keeps LLM calls in activities with the OpenInference OpenAI instrumentation, a custom span in workflow code, and a human approval via update.ticket_triage_agents/runs the same flow on the OpenAI Agents SDK throughOpenAIAgentsPlugin(use_otel_instrumentation=True)fromtemporalio-openai-agents. A small span processor gives Temporal's spans an OpenInference kind, sets the Arize session to the workflow ID, and records the activity attempt number.verify_trace.pychecks a trace through the Phoenix REST API: tree shape with span kinds, no missing parents, enrichment attributes, LLM token usage, and activity attempts; the in-memory tests cover replay re-emission, since Phoenix deduplicates by span ID. The README runbook reproduces replay stress, worker restart, activity retry, worker crash mid-activity, and reset; every case was verified live against Phoenix with real model calls, most recently ontemporalio-openai-agents1.1.0, openai 3, and Phoenix 20.16.Arize validated the sample against an Arize AX space. The one defect they found, a reapplied update's
HandleUpdatespan reusing the original run'sRunWorkflowspan id after a reset, is an SDK issue (temporalio/sdk-python#2070); the README and the verifier describe it.The sample is its own uv project (like
openai_agents/mcp_v2) and is in the standalone CI matrix: it pinstemporalio-openai-agents1.1.0, which needs openai 3, while the root lock stays on openai 2 for the LiteLLM samples. 1.1.0 carries the fix for temporalio/sdk-python#1852, so the agents scenario runs workflows and activities in one worker with no workaround.Tests run from the sample directory (
uv run poe test) without Phoenix, Docker, or an API key, including under the time-skipping environment.