> ## Documentation Index > Fetch the complete documentation index at: https://rasa.com/docs/llms.txt > Use this file to discover all available pages before exploring further. # TTS Integrations > Configure the built-in text-to-speech providers for Rasa Voice Stream channels. ## Text To Speech (TTS) This section describes the built-in text-to-speech (TTS) integrations for Voice Stream channels. All built-in TTS engines support streamed text input, so Rasa can send generated response text to the provider as it arrives. Every engine accepts `timeout` (seconds, default `30`) and `language_map`. Each map key must match a language configured in `config.yml`; entries can contain provider-specific `language`, `voice`, and `model` values. The top-level `language`, `voice`, and `model` options are deprecated and cannot be combined with `language_map`. ### Azure TTS The API Key can be set with the environment variable `AZURE_SPEECH_API_KEY`. Sample configuration looks as follow: ```yaml title="credentials.yml" theme={null} browser_audio: # ... other configuration tts: name: azure ``` Azure streams plain-text responses when the Azure Speech SDK is available. When a response contains Speech Synthesis Markup Language (SSML), or the SDK is unavailable, Rasa uses Azure's request-based synthesis path. #### Configuration parameters * `speech_region`: Optional, defaults to `eastus` - The region identifier for Azure Speech. Ensure that it matches your subscription. * `endpoint`: Optional - A custom Azure Speech REST endpoint. By default, Rasa builds the endpoint from `speech_region`. * `ws_endpoint`: Optional - A custom Azure Speech WebSocket endpoint for streamed text input. * `language_map`: Optional. By default, Rasa maps the assistant language to `en-US` and voice `en-US-JennyNeural`. Each entry supports `language` and `voice`. > [!WARNING] > > **Language Configuration** > > The deprecated top-level `language` and `voice` options cannot be combined with `language_map`. Put those values inside each language entry instead. #### Azure TTS Multilingual Support > [!NOTE] > > New in Rasa 3.16.0 To configure Azure TTS for multilingual agent use `language_map` property. Keys listed in the map must match languages (`language` and `additional_languages`) mentioned in `config.yml`. Azure TTS supports `language` and `voice` as configuration parameters inside `language_map`. ```yaml title="config.yml" theme={null} recipe: default.v1 language: en additional_languages: - de-DE pipeline: - name: SingleStepLLMCommandGenerator policies: - name: FlowPolicy assistant_id: 20240418-073244-narrow-archive ``` ```yaml title="credentials.yml" theme={null} browser_audio: tts: name: azure speech_region: germanywestcentral language_map: en: language: en-US voice: en-US-JennyMultilingualNeural de-DE: language: de-DE voice: de-DE-KatjaNeural ``` To see the full list of supported languages and models for Azure TTS visit [Azure TTS Languages](https://learn.microsoft.com/en-us/azure/ai-services/speech-service/language-support?tabs=tts). ### Cartesia TTS Use the environment variable `CARTESIA_API_KEY` for Cartesia API Key. The API Key requires a [Cartesia](https://www.cartesia.ai/) account. It can be configured in a Voice Stream channel as follows, ```yaml title="credentials.yml" theme={null} browser_audio: # ... other configuration tts: name: cartesia ``` #### Configuration parameters * `timeout`: Defaults to `30` seconds. See the shared TTS options above. * `model_id`: Optional, defaults to `sonic-3.6` - The model ID to be used for the text-to-speech conversion. * `version`: Optional, defaults to `2026-03-01` - The version of the model to be used for the text-to-speech conversion. * `endpoint`: Optional, defaults to `wss://api.cartesia.ai/tts/websocket` - The WebSocket endpoint for the Cartesia API. * `language_map`: Optional. By default, Rasa maps the assistant language to Cartesia language `en` and its default voice. Each entry supports `language` and `voice`. > [!WARNING] > > **Language Configuration** > > `language` and `language_map` are mutually exclusive properties. #### Cartesia Multilingual Support > [!NOTE] > > New in Rasa 3.16.0 To configure Cartesia for multilingual agent use `language_map` property. Keys listed in the map must match languages (`language` and `additional_languages`) mentioned in `config.yml`. Cartesia supports `language` as configuration parameter inside `language_map`. ```yaml title="config.yml" theme={null} recipe: default.v1 language: en additional_languages: - de-DE pipeline: - name: SingleStepLLMCommandGenerator policies: - name: FlowPolicy assistant_id: 20240418-073244-narrow-archive ``` ```yaml title="credentials.yml" theme={null} browser_audio: tts: name: cartesia language_map: en: language: en de-DE: language: de ``` To see the full list of supported languages and models for Cartesia TTS visit [Cartesia Supported Languages](https://docs.cartesia.ai/build-with-cartesia/tts-models/latest#language-support). ### Deepgram TTS Use the environment variable `DEEPGRAM_API_KEY` for Deepgram API Key. You can request a key from [Deepgram](https://deepgram.com/). It can be configured in a Voice Stream channel as follows: ```yaml title="credentials.yml" theme={null} browser_audio: # ... other configuration tts: name: deepgram ``` #### Configuration parameters Deepgram uses the `model` value in each `language_map` entry to select a voice model. The default model is `aura-2-thalia-en`; see [Deepgram documentation](https://developers.deepgram.com/docs/tts-models) for available models. * `endpoint`: Optional, defaults to `wss://api.deepgram.com/v1/speak` - The endpoint URL for the Deepgram API. * `timeout`: Defaults to `30` seconds. See the shared TTS options above. * `language_map`: Optional. By default, Rasa maps the assistant language to model `aura-2-thalia-en`. Set `model` for each language to select another Deepgram voice model. > [!WARNING] > > **Language Configuration** > > The top-level `model` option is deprecated and cannot be combined with `language_map`. Set the model inside each language entry instead. The Deepgram configuration model still accepts a `model_id` field, but the engine does not use it to select a model. Use `language_map..model` instead. #### Deepgram TTS Multilingual Support > [!NOTE] > > New in Rasa 3.16.0 To configure Deepgram TTS for multilingual agent use `language_map` property. Keys listed in the map must match languages (`language` and `additional_languages`) mentioned in `config.yml`. Deepgram TTS supports `model` as configuration parameter inside `language_map`. ```yaml title="config.yml" theme={null} recipe: default.v1 language: en additional_languages: - de-DE pipeline: - name: SingleStepLLMCommandGenerator policies: - name: FlowPolicy assistant_id: 20240418-073244-narrow-archive ``` ```yaml title="credentials.yml" theme={null} browser_audio: tts: name: deepgram language_map: en: model: aura-2-thalia-en de-DE: model: aura-2-julius-de ``` To see the full list of supported languages and models for Deepgram TTS visit [Deepgram TTS Languages](https://developers.deepgram.com/docs/tts-models#language-support). ### Deepgram Flux TTS Deepgram Flux TTS uses the `deepgram_flux` engine and the `DEEPGRAM_API_KEY` environment variable. Its default endpoint is `wss://api.deepgram.com/v2/speak`. ```yaml title="credentials.yml" theme={null} browser_audio: tts: name: deepgram_flux language_map: en: model: flux-alexis-en ``` #### Configuration parameters * `endpoint`: Optional, defaults to `wss://api.deepgram.com/v2/speak`. * `speed`: Optional. Set a value from `0.5` to `1.5` in increments of `0.05`. * `expressivity`: Optional, defaults to `0`. Set an integer from `-2` to `2`. * `mip_opt_out`: Optional, defaults to `false`. Controls the provider's MIP setting. * `tag`: Optional. Adds a request tag. * `language_map`: Optional. By default, Rasa maps the assistant language to model `flux-alexis-en`. Each entry must include a `model` whose name starts with `flux-`. ### ElevenLabs TTS Set the `ELEVENLABS_API_KEY` environment variable. The default model is `eleven_multilingual_v2` and the default endpoint is `wss://api.elevenlabs.io/v1/text-to-speech`. ```yaml title="credentials.yml" theme={null} browser_audio: tts: name: elevenlabs language_map: en: voice: "" ``` #### Configuration parameters * `endpoint`: Optional. The ElevenLabs WebSocket endpoint. * `model_id`: Optional. The default model ID is `eleven_multilingual_v2`. You can also set a model for an individual language under `language_map` with `model`. * `stability`: Optional float from `0` to `1`. Sets voice stability. * `similarity_boost`: Optional float from `0` to `1`. Sets voice similarity boost. * `language_map`: Required for ElevenLabs TTS. Each entry must include a `voice` ID and may include a `model`. > [!NOTE] > > ElevenLabs v3 and v4 models are not supported by this streaming integration. ### Rime TTS Use the environment variable `RIME_API_KEY` for Rime API Key. You can request a key from [Rime](https://rime.ai/). It can be configured in a Voice Stream channel as follows: ```yaml title="credentials.yml" theme={null} browser_audio: # ... other configuration tts: name: rime ``` #### Configuration parameters Rasa uses the Rime [Websockets JSON API](https://docs.rime.ai/api-reference/endpoint/websockets-json), * `model_id`: Optional, defaults to `coda` - The model ID to be used for text-to-speech conversion. * `endpoint`: Optional, defaults to `wss://users-ws.rime.ai/ws3` - The endpoint URL for the Rime API. * `speed_alpha`: Optional, defaults to `1.0` - Controls the speed of speech synthesis. * `segment`: Optional, defaults to `immediate` - Segment mode for synthesis. Use "immediate" for low latency. * `no_text_normalization`: Optional, defaults to `False` - Turns off text normalization to reduce the amount of computation needed to prepare input text for TTS inference. Rime recommends [enabling it to reduce response latency](https://docs.rime.ai/api-reference/latency#recommendations-for-reducing-response-time). * `language_map`: Optional. By default, Rasa maps the assistant language to language `en` and voice `astra`. Each entry supports `language` and `voice`. * `timeout`: Defaults to `30` seconds. See the shared TTS options above. > [!WARNING] > > **Voice Configuration** > > Set the Rime speaker through `language_map..voice`. The top-level `speaker` field does not change the selected voice. #### Rime TTS Multilingual Support > [!NOTE] > > New in Rasa 3.16.0 To configure Rime TTS for multilingual agent use `language_map` property. Keys listed in the map must match languages (`language` and `additional_languages`) mentioned in `config.yml`. Rime TTS supports `language` and `voice` as configuration parameters inside `language_map`. ```yaml title="config.yml" theme={null} recipe: default.v1 language: en additional_languages: - de-DE pipeline: - name: SingleStepLLMCommandGenerator policies: - name: FlowPolicy assistant_id: 20240418-073244-narrow-archive ``` ```yaml title="credentials.yml" theme={null} browser_audio: tts: name: rime language_map: en: language: eng voice: abbie de-DE: language: ger voice: amalia ``` To see the full list of supported languages and voices for Rime TTS visit [Rime TTS Languages](https://users.rime.ai/data/voices/all-v2.json). ### Others Looking for integration with a different TTS service? You can [create your own custom TTS component](#custom-tts). ## Custom TTS > [!TIP] > > **Create Custom TTS with Skills** > > [Rasa MCP Tools](https://rasa.com/docs/pro/installation/rasa-mcp-tools.md) includes AI Skills to scaffold and audit your custom TTS component with your IDE copilot. > Run `rasa tools init` in your project root to install the agent skills. You can implement your own custom TTS component as a Python class to integrate with any third-party text-to-speech service. A custom TTS component must subclass the `TTSEngine` class from `rasa.core.channels.voice_stream.tts.tts_engine`. Your custom TTS component must output audio in the [RasaAudioBytes format](https://rasa.com/docs/reference/integrations/speech-integrations.md#audio-format) and convert it using the `engine_bytes_to_rasa_audio_bytes` method. ### Required Methods Your custom TTS component must implement the following methods: * `synthesize(text: str, config: Optional[T])`: Generate speech from text, returning an async iterator of `RasaAudioBytes` chunks * `engine_bytes_to_rasa_audio_bytes(chunk: bytes)`: Convert your engine's audio format to Rasa's audio format * `from_config_dict(config: Dict)`: Class method to create an instance from configuration dictionary * `get_default_config()`: Static method that returns the default configuration for your component ### Optional Methods You may also override these methods as needed: * `connect(config: Optional[T])`: Establish connection to the TTS engine if necessary. Default implementation does nothing. * `close_connection()`: Custom cleanup when closing connections (e.g., closing websockets or HTTP sessions). Default implementation does nothing. * `send_text_chunk(text: str)`: Send text chunks to the TTS system for streaming synthesis. Used with `stream_audio()` for real-time streaming. * `signal_text_done()`: Signal TTS engine to process any remaining buffered text and prepare to end the stream. * `stream_audio()`: Stream audio output from the TTS engine. Continuously yields audio chunks as they are produced by the engine. ### Streaming vs Non-Streaming TTS The TTSEngine supports both streaming and non-streaming modes: * **Non-streaming**: Implement only `synthesize()` method for simple request-response synthesis * **Streaming**: Set `streaming_input = True` and implement `send_text_chunk()`, `signal_text_done()`, and `stream_audio()` methods for real-time streaming synthesis ### Required Class Attributes You can optionally define these class attributes: * `required_env_vars`: Tuple of required environment variable names * `required_packages`: Tuple of required Python package names * `streaming_input`: Boolean indicating if the engine supports streaming input (defaults to `False`) See [Configure a custom TTS component](#configure-a-custom-tts-component) for the credentials configuration. ### Example Implementation Here's an example of an implementation for TTS streaming: ```python title="custom_tts.py" theme={null} import os from dataclasses import dataclass from typing import AsyncIterator, Dict, Optional from urllib.parse import urlencode import aiohttp from aiohttp import ClientTimeout, WSMsgType from rasa.core.channels.voice_stream.audio_bytes import RasaAudioBytes from rasa.core.channels.voice_stream.tts.tts_engine import ( TTSEngine, TTSEngineConfig, TTSError, ) @dataclass class MyTTSConfig(TTSEngineConfig): endpoint: str = "wss://api.example.com/v1/speak" model_id: str = "en-US-standard" class MyTTS(TTSEngine[MyTTSConfig]): session: Optional[aiohttp.ClientSession] = None required_env_vars = ("MY_TTS_API_KEY",) # Optional: required environment variables required_packages = ("aiohttp",) # Optional: required Python packages streaming_input = True ws: Optional[aiohttp.ClientWebSocketResponse] = None def __init__(self, config: Optional[MyTTSConfig] = None): super().__init__(config) timeout = ClientTimeout(total=self.config.timeout) # All class instances share the same session if self.__class__.session is None or self.__class__.session.closed: self.__class__.session = aiohttp.ClientSession(timeout=timeout) async def connect(self, config: Optional[MyTTSConfig] = None) -> None: """Establish WebSocket connection to the TTS engine.""" headers = { "Authorization": f"Bearer {os.environ['MY_TTS_API_KEY']}", } query_params = { "model": merged_config.model_id, "language": merged_config.language, "voice": merged_config.voice, "encoding": "mulaw", "sample_rate": "8000", } ws_url = f"{merged_config.endpoint}?{urlencode(query_params)}" try: self.ws = await self.session.ws_connect( ws_url, headers=headers, timeout=float(self.config.timeout) if self.config.timeout else 30, ) except Exception as e: raise TTSError(f"Failed to connect to TTS service: {e}") async def close_connection(self) -> None: """Close WebSocket connection if it exists.""" if self.ws and not self.ws.closed: await self.ws.close() self.ws = None async def send_text_chunk(self, text: str) -> None: """Send text chunk to TTS engine for streaming synthesis.""" await self.ws.send_json({"text": text}) async def signal_text_done(self) -> None: """Signal TTS engine that all text has been sent.""" await self.ws.send_json({"operation": "flush"}) async def stream_audio(self) -> AsyncIterator[RasaAudioBytes]: """Stream audio output from the TTS engine.""" try: async for msg in self.ws: if msg.type == WSMsgType.TEXT: data = msg.json() if data.get("type") == "audio": # Assume base64 encoded audio data import base64 audio_bytes = base64.b64decode(data.get("data", "")) if audio_bytes: yield self.engine_bytes_to_rasa_audio_bytes(audio_bytes) elif data.get("type") == "error": raise TTSError(f"TTS error: {data.get('message')}") elif msg.type == WSMsgType.CLOSED: break elif msg.type == WSMsgType.ERROR: raise TTSError("WebSocket error during audio streaming") except Exception as e: raise TTSError(f"Error during audio streaming: {e}") async def synthesize( self, text: str, config: Optional[MyTTSConfig] = None ) -> AsyncIterator[RasaAudioBytes]: """Generate speech from text using streaming WebSocket.""" await self.connect(config) try: await self.send_text_chunk(text) await self.signal_text_done() async for audio_chunk in self.stream_audio(): yield audio_chunk finally: await self.close_connection() def engine_bytes_to_rasa_audio_bytes(self, chunk: bytes) -> RasaAudioBytes: """Convert the generated TTS audio bytes into Rasa audio bytes.""" # If your service returns audio in mulaw format already, return directly return RasaAudioBytes(chunk) # If your service returns audio in a different format (e.g., PCM linear), # you'll need to convert it. For example, to convert from linear PCM: # import audioop # mulaw_data = audioop.lin2ulaw(chunk, 2) # 2 = 16-bit samples # return RasaAudioBytes(mulaw_data) @staticmethod def get_default_config() -> MyTTSConfig: return MyTTSConfig( endpoint="wss://api.example.com/v1/speak", model_id="en-US-standard", language="en-US", voice="female-1", timeout=30, ) @classmethod def from_config_dict(cls, config: Dict) -> "MyTTS": return cls(MyTTSConfig.from_dict(config)) ``` > [!TIP] > > **Non-Streaming TTS** > > If your TTS service doesn't support continuous input streaming (i.e., you need to send all text at once), set `streaming_input = False` and only implement the `synthesize()` method. You can skip implementing `connect()`, `send_text_chunk()`, `signal_text_done()`, and `stream_audio()` methods. The `synthesize()` method should handle the entire synthesis process from text to audio. ## Configure a custom TTS component To use a custom TTS component, configure its module path and settings in your `credentials.yml` file. The module path follows the format `path.to.module.ClassName`. For example, a class `MyTTS` in `addons/custom_tts.py` has module path `addons.custom_tts.MyTTS`. ### Custom TTS Configuration Example ```yaml title="credentials.yml" theme={null} browser_audio: # ... other configuration tts: name: addons.custom_tts.MyTTS api_key: "your_api_key" endpoint: "wss://api.example.com/v1/speak" language: "en-US" voice: "en-US-JennyNeural" timeout: 30 # any other custom parameters your TTS needs ``` Any custom parameters you define in your configuration class (e.g., `MyASRConfig` or `MyTTSConfig`) can be passed through the credentials file and will be available in your component via `self.config`. ## Related topics - [Speech Integrations](https://rasa.com/docs/reference/integrations/speech-integrations.md) - [Rasa Tutorial](https://rasa.com/docs/pro/tutorial.md) - [Genesys Cloud](https://rasa.com/docs/reference/channels/genesys-cloud-voice.md) - [Voice Assistants](https://rasa.com/docs/pro/build/voice-assistants.md) - [Custom Connectors](https://rasa.com/docs/reference/channels/custom-connectors.md) This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.