> ## Documentation Index
> Fetch the complete documentation index at: https://rasa.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Test and monitor voice

> Listen to your agent in the Inspector, then watch latency, interruptions, and failures with traces and metrics.

Most of what makes a voice agent right is decided in the skill, and you can
test that over chat. What chat cannot show you is how the agent sounds and how
fast it answers. This page covers testing those during development and
watching them in production.

## Talk to the agent in the Inspector

The Inspector has a voice mode, so you can test with your own voice before
connecting a phone line. Configure it with the same speech providers
as your production line, as in
[Getting Started](https://rasa.com/docs/mantle/getting-started.md#3-set-your-environment-variables). See
[Speech providers](https://rasa.com/docs/mantle/reference/speech-providers.md).

1. Set your speech provider's API key in your environment or `.env` (`DEEPGRAM_API_KEY` for Deepgram).
2. Run `rasa inspect`.
3. Leave the message box empty and select **Start voice conversation** (the
   waveform button). Allow microphone access when the browser asks.
4. Select **End conversation** to hang up.

During the call, the Inspector shows the same skill, memory, and tool events as
a chat conversation, alongside voice events:

| What you see | What it tells you |
| - | - |
| **Interruption events** | The caller spoke over the agent, and whether playback was in progress or still queued |
| **Suppressed input** | Short caller speech that the classifier scored as a backchannel and that did not interrupt |
| **Silence timeouts** | The caller was quiet for the configured timeout and the silence skill ran |
| **Latency per turn** | Time to the first model token, time to the first audio byte, total speech generation time, and whether the audio came from cache |

### What to listen for

Test each skill out loud at least once. Things that read fine in chat often
sound wrong on a call:

* **Long replies.** More than a few sentences is hard to follow by ear.
* **Lists.** Several items read in a row are hard to remember.
* **Numbers and identifiers.** Check that order numbers, card digits, dates, and
  amounts are read the way a caller expects.
* **Dead air.** A long silence before a reply usually means a slow tool or
  model call. Check the turn's latency.
* **Interruptions.** Talk over the agent mid-sentence and check it stops, and
  that any tool with side effects did not run twice.
* **The opening and the close.** Check what the caller hears first, and that the
  agent says goodbye once.

[Designing for voice](https://rasa.com/docs/mantle/voice/designing-for-voice.md) covers the fix for each.

## Test the speech path separately

[Simulations](https://rasa.com/docs/mantle/testing/simulating-and-evaluating.md) help you check skill behavior
using text. They do not establish that a caller's audio was recognized correctly,
that a reference sounded right, or that interruptions worked on your phone line.
Keep a small set of repeatable spoken cases alongside your text scenarios:

| Case | What to check |
| - | - |
| A short *yes* or *no* | A final user input reaches the agent, or a [suppressed-input event](https://rasa.com/docs/mantle/reference/events.md#suppressed-user-input) explains why it did not |
| A name, amount, or reference | The transcript and resolved record agree with the intended value; a failed lookup asks for correction |
| A correction before confirmation | The confirmation restates the corrected facts, and the tool uses those facts |
| A denied confirmation | The write tool does not run |
| An interruption during a slow tool | Playback and turn behavior match the channel; an external operation is reconciled and not duplicated |
| A pause inside a reference | The recognizer does not split the value into unintended user turns |
| Silence and the end of a call | The configured check-ins and goodbye occur, and the call closes as intended |

Start with the same utterances and expected tool outcomes for each configuration.
Record the Rasa version, speech provider, model, voice, language, channel, and test
date. Change one setting at a time. Include the names and pronunciations your
callers use; a good overall transcription score can hide errors in the few
fields that drive a lookup. A provider change also needs a test on the target
phone line, since browser and telephony audio paths differ.

If a short answer produces no user input, inspect recognition and suppression
before changing the skill prompt. The community
[short-reply investigation](https://rasa.community/library/guides/rasa-deepgram-flux-drops-one-word-replies/)
shows this diagnostic on specific Rasa versions. It is a versioned example,
not a claim about every recognizer or release. For pauses that split speech or
delay a turn, start with
[Deepgram Flux settings](https://rasa.com/docs/mantle/reference/integrations-yml.md#deepgram-flux-for-speech-recognition)
when using Flux; other recognizers have their own turn-completion settings.

## Monitor calls in production

Voice calls emit OpenTelemetry traces and metrics. Configure an exporter in
`integrations.yml`:

```yaml integrations.yml theme={null}
tracing:
  type: otlp
  endpoint: <your-otel-collector-endpoint>
  insecure: true

metrics:
  type: otlp
  endpoint: <your-otel-collector-endpoint>
  insecure: true
```

### Traces

Each call produces one trace that follows the provider connections, each
transcription, each agent turn, speech synthesis, and audio playback. Use traces to answer "what happened on this call":
why a turn was slow, why an interruption was accepted or ignored, or where a
provider failed.

Transcripts and agent text are not captured by default. See
[Content capture](https://rasa.com/docs/mantle/reference/observability.md#content-capture) before turning it on.
The full span catalog is on [Observability](https://rasa.com/docs/mantle/reference/observability.md#voice-traces).

### Metrics

Metrics answer "how are calls going overall". Start with how long callers wait
before they hear each reply (time to first audio) and how long each turn takes.
Then watch how often callers interrupt, how often they go quiet, which often
points to a confusing question, and the failure counts for speech recognition,
speech synthesis, and playback.

### Separate first audio from the answer

An acknowledgement such as "Let me check" gives the caller feedback, but is not
the result they asked for. Track the wait to that first sound separately from
the wait to the useful answer, such as the order status or confirmation:

* [`rasa.voice.tts.time_to_first_audio`](https://rasa.com/docs/mantle/reference/metrics#rasavoicettstime_to_first_audio)
  measures one message's synthesis from a provider request or cache lookup. It
  excludes speech recognition, the model, and tool work before that message.
* [`rasa.turn.time_to_first_output`](https://rasa.com/docs/mantle/reference/metrics#rasaturntime_to_first_output)
  starts at accepted input. For speech, that is the final transcript, not when the caller stopped
  speaking. It ends at the first TTS audio chunk, which may be an acknowledgement.
* To measure the caller's wait to the answer, instrument your test client from
  the end of the caller's audio to the playback of the response that supplies
  the answer. This is an additional measurement, not a built-in answer-latency
  metric.

Keep cache hits and misses separate, and retain failures and turns with no
audio rather than treating a missing metric as a zero. The community guide
[How to measure voice agent latency to the answer](https://rasa.community/library/guides/voice-latency-times-the-filler/)
walks through recorded examples and a test harness. The
[companion voice driver](https://github.com/RasaHQ/rasa-community-resources/blob/4aa0c4419dc193fef7a969c12d59edcf720f2606/scripts/case_builds/voice_driver.py)
shows the client-side audio and playback instrumentation used by those examples.
Read the example's setup before running it: live calls use provider credentials
and can incur charges. Its recorded timings are not a latency guarantee for your
deployment.

Every metric, with its dimensions and precise timing boundaries, is on
[Metrics](https://rasa.com/docs/mantle/reference/metrics.md).

## Next steps

> **[Designing for voice](https://rasa.com/docs/mantle/voice/designing-for-voice.md)**
>
> Fix what you heard: lists, numbers, confirmations, and endings.

> **[Connect a phone line](https://rasa.com/docs/mantle/voice/telephony.md)**
>
> Put the agent behind a real phone number.


## Related topics

- [Designing for voice](https://rasa.com/docs/mantle/voice/designing-for-voice.md)
- [Voice agents](https://rasa.com/docs/mantle/voice/overview.md)
- [Build a voice agent](https://rasa.com/docs/mantle/voice/tutorial.md)
- [Connect a phone line](https://rasa.com/docs/mantle/voice/telephony.md)
- [Speech providers](https://rasa.com/docs/mantle/reference/speech-providers.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.