Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions fern/assistants/background-speech-denoising.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -393,6 +393,8 @@ Fourier Denoising should never be used alone. It's designed to complement Smart
4. Monitor user feedback
5. A/B test different configurations

Denoising also applies when a simulation's AI tester plays [background sound](/observability/simulations-configure-ai-tester#background-sound), so the assistant hears the call the way it would in production.

## Troubleshooting fourier denoising

<AccordionGroup>
Expand Down
58 changes: 57 additions & 1 deletion fern/observability/simulations-configure-ai-tester.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ The available settings depend on whether you configure the AI tester in the Dash
| Area | Dashboard settings | Additional API settings |
| -- | -- | -- |
| **Scenario** | Name and intent | Overrides for the assistant or squad under test, webhook destinations, transcript and message inclusion, and recording inclusion |
| **Behavior** | Behavior and who starts first | First message, start-speaking plan, stop-speaking plan, duration limit, background sound, and background-speech denoising |
| **Behavior** | Behavior, who starts first, and [background sound](#background-sound) | First message, start-speaking plan, stop-speaking plan, duration limit, and background-speech denoising |
| **Model** | Provider and model | Temperature, maximum tokens, fallback models, tools, prompt caching, knowledge base, and model-specific reasoning |
| **Transcriber** | Provider, model, and language | Language detection and hints, endpoint delay, vocabulary, contextual hints, and fallback transcribers |
| **Voice** | Provider and voice ID | Speed, version, language, pronunciation dictionaries, chunking, formatting, and caching |
Expand Down Expand Up @@ -186,6 +186,62 @@ The available fields vary by provider. See [**Update Personality**](/api-referen
Editing a built-in **(Default)** personality in the Dashboard saves it as a new personality you own. The built-in personality remains unchanged.
</Note>

## Background sound

Background sound is an audio file that the AI tester plays throughout a voice simulation, mixed with its speech. Use it to test how the assistant or squad you're testing handles a caller in a noisy place. Chat simulations ignore background sound.

<Warning>
[GPT-Live](/gpt-live/overview) AI testers add background sound only while they're speaking, not during silence. They also don't play the **Office** preset, so choose another one.
</Warning>

In the Dashboard, open the **Personality** tab, expand **Advanced settings**, and turn on **Background sound**. Turning it on selects **White noise**. To change it, select another preset or **Custom URL**.

| Preset | Sound | API value |
| -- | -- | -- |
| **Office** | Office chatter and typing | `office` |
| **White noise** | Steady static | `white-noise` |
| **City** | Street traffic | `city-traffic` |
| **Crowd** | Many people talking | `crowd-conversation` |
| **TV** | A television playing | `tv` |
| **Farm** | Chickens and roosters | `chickens-roosters` |

Office is the quietest preset, at about 30 dB below the AI tester's speech. The others play about 15 dB below it.

With **Custom URL**, enter a public HTTPS URL of up to 1,000 characters that points to an audio file of 25 MB or less. The file loops for the whole call. Use WAV when you can, because an MP3 file can leave a short gap at the loop point.

Through the API, set `backgroundSound` in the personality's `assistant` to a preset's API value or your own URL:

```json
{
"assistant": {
"backgroundSound": "white-noise"
}
}
```

Include the rest of the personality's assistant configuration in the same request, as described in [Configure the AI tester](#configure-the-ai-tester).

<Note>
Comment thread
scott-lowe-vapi marked this conversation as resolved.
Resolved
The assistant or squad you're testing denoises incoming audio before it transcribes and records the call. Smart Denoising is on by default, so it removes most of the background sound, and the recording may sound clean. A clean recording is expected, because the simulation tests your production setup.
</Note>

### Hear the raw noise

To hear the raw noise, turn off denoising in the scenario's `targetOverrides`. Also set `backgroundDenoisingEnabled` to `false`, because this legacy flag takes precedence when the assistant or squad you're testing sets it:

```json
{
"targetOverrides": {
"backgroundDenoisingEnabled": false,
"backgroundSpeechDenoisingPlan": {
"smartDenoisingPlan": { "enabled": false }
}
}
}
```

If the assistant or squad you're testing enables [Fourier denoising](/documentation/assistants/conversation-behavior/background-speech-denoising#fourier-denoising-experimental), also set `backgroundSpeechDenoisingPlan.fourierDenoisingPlan.enabled` to `false`. Updating a scenario replaces its `targetOverrides`, so include any existing overrides you want to keep.

## Next steps

<CardGroup cols={2}>
Expand Down
2 changes: 2 additions & 0 deletions fern/test/simulations-best-practices.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,8 @@ This creates useful variation without changing the business requirement.

Build and debug a scenario in chat mode, then run important paths in voice mode before launch. Also use voice mode after changing the transcriber, voice, or turn-taking configuration.

- Add a noisy-caller scenario for important paths. Turn on [background sound](/observability/simulations-configure-ai-tester#background-sound) in the AI tester's personality to check that the agent still understands the caller and completes the task.

<Note>
Synthetic callers do not fully represent real callers, devices, or environments.
Supplement voice Simulations with manual calls and production-call review for
Expand Down
Loading