add files

Author: nm-openai

.gitignore

@@ -0,0 +1,40 @@
+# dependencies
+/node_modules
+/.pnp
+.pnp.*
+.yarn/*
+!.yarn/patches
+!.yarn/plugins
+!.yarn/releases
+!.yarn/versions
+
+# testing
+/coverage
+
+# next.js
+/.next/
+/out/
+
+# production
+/build
+
+# misc
+.DS_Store
+*.pem
+
+# debug
+npm-debug.log*
+yarn-debug.log*
+yarn-error.log*
+.pnpm-debug.log*
+
+# env files (can opt-in for committing if needed)
+.env*
+
+# vercel
+.vercel
+
+# typescript
+*.tsbuildinfo
+next-env.d.ts
+todo.md

LICENSE

@@ -0,0 +1,9 @@
+MIT License
+
+Copyright (c) 2025 OpenAI
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

README.md

@@ -0,0 +1,72 @@
+# Realtime API Agents Demo
+
+This is a simple demonstration of more advanced, agentic patterns built on top of the Realtime API. In particular, this demonstrates:
+- Sequential agent handoffs according to a defined agent graph (taking inspiration from [OpenAI Swarm](https://github.com/openai/swarm))
+- Background escalation to more intelligent models like o1-mini for high-stakes decisions
+- Prompting models to follow a state machine, for example to accurately collect things like names and phone numbers with confirmation character by character to authenticate a user.
+
+You should be able to use this repo to prototype your own multi-agent realtime voice app in less than 20 minutes!
+
+![Screenshot of the Realtime API Agents Demo](/public/screenshot.png)
+
+## Setup
+
+- This is a Next.js typescript app
+- Install dependencies with `npm i`
+- Add your `OPENAI_API_KEY` to your env
+- Start the server with `npm run dev`
+- Open your browser to [http://localhost:3000](http://localhost:3000) to see the app. It should automatically connect to the `simpleExample` Agent Set.
+
+## Configuring Agents
+Configuration in `src/app/agentConfigs/simpleExample.ts`
+```javascript
+import { AgentConfig } from "@/app/types";
+import { injectTransferTools } from "./utils";
+
+// Define agents
+const haiku: AgentConfig = {
+  name: "haiku",
+  publicDescription: "Agent that writes haikus.", // Context for the agent_transfer tool
+  instructions:
+    "Ask the user for a topic, then reply with a haiku about that topic.",
+  tools: [],
+};
+
+const greeter: AgentConfig = {
+  name: "greeter",
+  publicDescription: "Agent that greets the user.",
+  instructions:
+    "Please greet the user and ask them if they'd like a Haiku. If yes, transfer them to the 'haiku' agent.",
+  tools: [],
+  downstreamAgents: [haiku],
+};
+
+// add the transfer tool to point to downstreamAgents
+const agents = injectTransferTools([greeter, haiku]);
+
+export default agents;
+```
+
+This fully specifies the agent set that was used in the interaction shown in the screenshot above.
+
+### Next steps
+- Check out the configs in `src/app/agentConfigs`. The example above is a minimal demo that illustrates the core concepts.
+- [frontDeskAuthentication](src/app/agentConfigs/frontDeskAuthentication) Guides the user through a step-by-step authentication flow, confirming each value character-by-character, authenticates the user with a tool call, and then transfers to another agent. Note that the second agent is intentionally "bored" to show how to prompt for personality and tone.
+- [customerServiceRetail](src/app/agentConfigs/customerServiceRetail) Also guides through an authentication flow, reads a long offer from a canned script verbatim, and then walks through a complex return flow which requires looking up orders and policies, gathering user context, and checking with `o1-mini` to ensure the return is eligible. To test this flow, say that you'd like to return your snowboard and go through the necessary prompts!
+
+### Defining your own agents
+- You can copy these to make your own multi-agent voice app! Once you make a new agent set config, add it to `src/app/agentConfigs/index.ts` and you should be able to select it in the UI in the "Scenario" dropdown menu.
+- To see how to define tools and toolLogic, including a background LLM call, see [src/app/agentConfigs/customerServiceRetail/returns.ts](src/app/agentConfigs/customerServiceRetail/returns.ts)
+- To see how to define a detailed personality and tone, and use a prompt state machine to collect user information step by step, see [src/app/agentConfigs/frontDeskAuthentication/authentication.ts](src/app/agentConfigs/frontDeskAuthentication/authentication.ts)
+- To see how to wire up Agents into a single Agent Set, see [src/app/agentConfigs/frontDeskAuthentication/index.ts](src/app/agentConfigs/frontDeskAuthentication/index.ts)
+- If you want help creating your own prompt using these conventions, we've included a metaprompt [here](src/app/agentConfigs/voiceAgentMetaprompt.txt), or you can use our [Voice Agent Metaprompter GPT](https://chatgpt.com/g/g-678865c9fb5c81918fa28699735dd08e-voice-agent-metaprompt-gpt)
+
+## UI
+- You can select agent scenarios in the Scenario dropdown, and automatically switch to a specific agent with the Agent dropdown.
+- The conversation transcript is on the left, including tool calls, tool call responses, and agent changes. Click to expand non-message elements.
+- The event log is on the right, showing both client and server events. Click to see the full payload.
+- On the bottom, you can disconnect, toggle between automated voice-activity detection or PTT, turn off audio playback, and toggle logs.
+
+## Core Contributors
+- Noah MacCallum - [noahmacca](https://x.com/noahmacca)
+- Ilan Bigio - [ibigio](https://github.com/ibigio)

eslint.config.mjs

@@ -0,0 +1,22 @@
+import { dirname } from "path";
+import { fileURLToPath } from "url";
+import { FlatCompat } from "@eslint/eslintrc";
+
+const __filename = fileURLToPath(import.meta.url);
+const __dirname = dirname(__filename);
+
+const compat = new FlatCompat({
+  baseDirectory: __dirname,
+});
+
+const eslintConfig = [
+  ...compat.extends("next/core-web-vitals", "next/typescript"),
+  {
+    rules: {
+      "@typescript-eslint/no-explicit-any": "off",
+      "react-hooks/exhaustive-deps": "off"
+    },
+  },
+];
+
+export default eslintConfig;

next.config.ts

@@ -0,0 +1,7 @@
+import type { NextConfig } from "next";
+
+const nextConfig: NextConfig = {
+  /* config options here */
+};
+
+export default nextConfig;