Replaying a Semantic Kernel run with no provider reachable — and why OPENAI_BASE_URL is the only lever

[Xizhuo (xizhuomengcontin)]

I record agent and pipeline runs at the HTTP boundary to the model provider — from outside the process — and serve the recording back so the run happens again with no provider called. I checked it against Semantic Kernel tonight. It works with zero configuration, and while finding out why, I ran into a Python-connector detail that seems worth writing down on its own.

The detail

OpenAIChatCompletion.__init__ takes:

['self', 'ai_model_id', 'service_id', 'api_key', 'org_id',
 'default_headers', 'async_client', 'env_file_path', 'env_file_encoding', 'instruction_role']

There is no base_url parameter. And in open_ai_config_base.py the client is built as:

client = AsyncOpenAI(
    api_key=api_key,
    organization=org_id,
    default_headers=merged_headers,
)

base_url is never passed, so the OpenAI SDK's own fallback governs — it reads OPENAI_BASE_URL from the environment. (The Azure path is different: AzureOpenAISettings does expose base_url / AZURE_OPENAI_BASE_URL.)

The practical consequence for anyone pointing SK at an OpenAI-compatible gateway, vLLM, or a local server: for the non-Azure OpenAIChatCompletion there are exactly two routes — set OPENAI_BASE_URL, or construct your own AsyncOpenAI and pass it as async_client. There is no constructor argument for it, which is not obvious from the signature until you go looking. If that is intentional, it might be worth a line in the docstring; if the asymmetry with the Azure connector is accidental, that is a small fix.

What that makes possible

Because the origin comes from the environment, a recorder that launches your script as a child process with that one variable redirected captures SK with no code change at all:

kernel = Kernel()
kernel.add_service(OpenAIChatCompletion(ai_model_id="gpt-4o-mini", service_id="oai"))
service = kernel.get_service("oai")
history = ChatHistory()
history.add_user_message("Say hello in three words.")
reply = await service.get_chat_message_content(
    chat_history=history, settings=OpenAIChatPromptExecutionSettings()
)

$ orca record generic-openai -- python app.py
SK replied: the stub answered
info recorded run=run_639bdbfdcc46 events=6 exit=0

Then, with the origin process killed so there is nothing upstream to reach:

$ orca replay last --worktree
info replaying exchanges=1 egress=blocked
SK replied: the stub answered
info replay.done reused=1/1 exact=1 divergences=0 unmatched=0 exit=0

exact=1 means the replayed request matched the recording byte for byte — chat history assembly, execution settings, headers and all. The kernel, the service resolution and your own code run for real; only the model's answer comes from the trace. Nothing is installed into SK: no filter, no telemetry hook, no patched client.

Two things that buys you: a misbehaving run becomes a file a colleague can re-run without your key and without spending tokens, and the same file pins a kernel configuration in CI — which is the awkward part of testing anything with a chat completion in it, since real calls cost money and do not answer the same way twice.

Tested with semantic-kernel 1.44.1 on Python 3.12 against a deterministic local origin standing in for a provider, so "byte for byte" is a real comparison rather than a model happening to repeat itself.

Limits, stated rather than buried

• Blocked egress means model-provider egress, not network isolation. Replay calls no provider but re-executes everything else for real — a plugin that fetches a URL or writes to a store does it again. Not a sandbox.
• A matching replay is not a determinism result. The model is not re-asked; its recorded answers are served back.
• Embedding calls are not captured (/v1/embeddings is not one of the claimed paths), so a memory/RAG-heavy kernel that embeds at query time cannot be fully replayed today. Replay refuses those calls with an explicit error rather than serving a wrong vector. I mention it because SK's memory connectors make this a realistic case, and I would rather state it than have someone find out at replay time.
• I have not tested the Azure connector, .NET, or Java. The reasoning above is specific to the Python OpenAIChatCompletion.

Tool is OrcaReplay (Apache-2.0, npm, Node 20+); I maintain it, so weigh the post accordingly. The connector observation at the top stands on its own — it is about SK's signature and client construction, not about my tool.