> ## Documentation Index
> Fetch the complete documentation index at: https://rasa.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Speech providers

> Supported speech recognition and speech synthesis providers, with their API keys, defaults, and settings.

Every voice channel turns the caller's audio into text with a speech recognition
(ASR) provider, and turns the agent's replies into audio with a speech synthesis
(TTS) provider. You choose both per channel in `integrations.yml`, under the
channel's `asr:` and `tts:` keys. Recognition and synthesis are chosen separately,
so you can mix providers. To use a provider that is not built in, see
[Custom speech providers](https://rasa.com/docs/mantle/reference/custom-speech-providers.md).

## Choose a provider

Set `name` on `asr:` and `tts:`, and choose models and voices per language in
[`language_map`](https://rasa.com/docs/mantle/reference/integrations-yml#language_map):

```yaml integrations.yml theme={null}
channels:
  inspector:
    enabled: true
    asr:
      name: deepgram
      language_map:
        en:
          language: en
          model: flux-general-en
    tts:
      name: cartesia
      language_map:
        en:
          language: en
          voice: f786b574-daa5-4673-aa0c-cbe3e8534c02
```

| Provider | `asr.name` | `tts.name` | API key environment variable |
| - | - | - | - |
| Deepgram | `deepgram` | `deepgram` | `DEEPGRAM_API_KEY` |
| Deepgram Flux | `deepgram` with a `flux-` model | `deepgram_flux` | `DEEPGRAM_API_KEY` |
| Azure AI Speech | `azure` | `azure` | `AZURE_SPEECH_API_KEY` |
| Cartesia | — | `cartesia` | `CARTESIA_API_KEY` |
| Rime | — | `rime` | `RIME_API_KEY` |
| ElevenLabs | `elevenlabs` | `elevenlabs` | `ELEVENLABS_API_KEY` |

Set the API key in your environment or `.env`. If a key is missing, Mantle logs
the missing variable and the voice conversation can't start. In the Inspector, you
see a "Voice isn't set up yet" warning.

## Defaults

If the Inspector has no `asr:` or `tts:` keys, it still works with only
`DEEPGRAM_API_KEY`, using Deepgram's earlier models:

| | Provider | Model |
| - | - | - |
| Speech recognition | `deepgram` | `nova-3`, English |
| Speech synthesis | `deepgram` | `aura-2-thalia-en` |

With no `asr:` or `tts:` keys, the Inspector uses English Deepgram models, even if
`agent.language` is another language. For another language, set
[`language_map`](https://rasa.com/docs/mantle/reference/integrations-yml#language_map). Telephony channels have
no defaults, so you must set both `asr:` and `tts:`.

## Speech recognition settings

Settings other than `name` and `language_map` go next to them, under `asr:`.

| Provider | Settings |
| - | - |
| `deepgram` | `endpoint` (host name, default `api.deepgram.com`), `endpointing` (ms of silence before a transcript is final, default `400`), `smart_format` (default `true`), `utterance_end_ms` (default `1000`) |
| `deepgram` with a `flux-` model | `endpoint`, `eot_threshold` (0.5–0.9, default `0.7`), `eot_timeout_ms` (500–10000, default `5000`). See [Deepgram Flux for speech recognition](https://rasa.com/docs/mantle/reference/integrations-yml.md#deepgram-flux-for-speech-recognition). |
| `azure` | One of `speech_region`, `speech_host`, or `speech_endpoint` (required) |
| `elevenlabs` | `endpoint` (full WebSocket URL), `commit_strategy` (`vad` or `manual`, default `vad`), `include_timestamps` (default `false`), `include_language_detection` (default `false`) |

## Speech synthesis settings

Settings other than `name` and `language_map` go next to them, under `tts:`.

| Provider | Settings |
| - | - |
| `deepgram` | `endpoint` (full URL, default `wss://api.deepgram.com/v1/speak`) |
| `deepgram_flux` | `endpoint` (default `wss://api.deepgram.com/v2/speak`), `speed` (0.5–1.5 in 0.05 steps), `expressivity` (-2 to 2, default `0`). Every `language_map` entry needs a `flux-` model; the default is `flux-alexis-en`. |
| `azure` | `speech_region` (default `eastus`), `endpoint`, `ws_endpoint`. Default voice `en-US-JennyNeural`. |
| `cartesia` | `model_id` (default `sonic-3.6`), `version`, `endpoint` |
| `rime` | `model_id` (default `coda`), `endpoint`, `speed_alpha` (default `1.0`), `segment` (default `immediate`), `no_text_normalization` (default `false`). Default voice `astra`. |
| `elevenlabs` | `endpoint` (base WebSocket URL, default `wss://api.elevenlabs.io/v1/text-to-speech`), `model_id` (default `eleven_multilingual_v2`), `stability`, `similarity_boost`. Each `language_map` entry must include an ElevenLabs voice ID the API key can access; streaming synthesis includes word-level timestamps. |

All speech synthesis engines also accept `timeout` (seconds, default `30`).
Synthesized replies are cached in memory, so a repeated line, such as a
greeting, plays without a new provider request.

## Regional endpoints

To keep audio in a region, point `endpoint` at the provider's regional host.
For Deepgram, the speech recognition `endpoint` is a host name and the speech
synthesis `endpoint` is a full URL:

```yaml theme={null}
asr:
  name: deepgram
  endpoint: api.eu.deepgram.com
tts:
  name: deepgram_flux
  endpoint: wss://api.eu.deepgram.com/v2/speak
```

Check your provider's documentation for the hosts available to your account.

## Troubleshoot setup

| Symptom | Check first |
| - | - |
| The Inspector says "Voice isn't set up yet" | Usually missing ASR or TTS keys: set them in the environment of the process running `rasa inspect`, or in the project's `.env`, and restart. Any other voice startup error shows the same warning, so check the server log for the cause. |
| Chat works, but the caller is not heard | Check microphone permission and ASR connection errors, then whether a final transcript reaches the agent. Chat does not test the speech provider. |
| The agent produces text but no speech | Check the TTS connection, model or voice identifier, and provider access for your account. |
| A language uses the wrong voice or recognizer | Match `language_map` keys to the agent's language codes and set the provider's `language`, `model`, and `voice` values explicitly where required. The Inspector's implicit defaults are English. |

Keep keys out of `integrations.yml`, shared logs, and source control. When
switching providers, preserve the rest of the file, including `model_groups`.
Speech settings belong under the selected channel, not at the file root. Channel
settings are read from the live project; restart to pick them up. Changing
skills, or which model group `agent.yml` names, needs `rasa train`. See
[`integrations.yml`](https://rasa.com/docs/mantle/reference/integrations-yml) for configuration lifecycles.

Use [Test and monitor voice](https://rasa.com/docs/mantle/voice/testing.md) to distinguish recognition, turn
handling, tool, and playback failures before tuning a provider.

## See also

* [`integrations.yml` → Voice channels](https://rasa.com/docs/mantle/reference/integrations-yml#voice-channels): `language_map`, `interruptions`, and `silence_timeout`
* [Custom speech providers](https://rasa.com/docs/mantle/reference/custom-speech-providers.md): use a provider that is not built in
* [Rasa Community voice tuning walkthrough](https://rasa.community/library/tutorials/voice-ai-agent/09-voice-deep-dive/): a worked Deepgram Flux example with spoken formatting and recognition recovery; check its tested version before reusing its settings


## Related topics

- [Custom speech providers](https://rasa.com/docs/mantle/reference/custom-speech-providers.md)
- [Voice agents](https://rasa.com/docs/mantle/voice/overview.md)
- [Connect a phone line](https://rasa.com/docs/mantle/voice/telephony.md)
- [Test and monitor voice](https://rasa.com/docs/mantle/voice/testing.md)
- [Build a voice agent](https://rasa.com/docs/mantle/voice/tutorial.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.