Skip to main content

Audio Format

Rasa uses a common intermediate audio format called RasaAudioBytes that acts as a standard data format to prevent complexity between different channels, ASR engines, and TTS engines. Rasa supports the following audio formats:
  • 8 bit 8KHz μ-law (mulaw) encoding mono channel
  • Linear 16 bit 16KHz PCM mono channel
  • Linear 16 bit 24KHz PCM mono channel
  • Linear 16 bit 48KHz PCM mono channel
Rasa will automatically try to negotiate Linear 16 bit 24KHz PCM mono audio format for AudioCodes, Jambonz, Signalwire and Vonage channels. Other channels, except for browser channel, will use 8 bit 8KHz μ-law (mulaw) encoding mono channel. Browser channel supports all four mentioned formats. By default, browser channel will use Linear 16 bit 48KHz PCM mono. Choosing the audio format is done at Rasa start by supplying sample_rate config property in credentials.yml:
credentials.yml
The browser_audio channel used by the Rasa Inspector and by voice-stream browser clients also accepts the sample_rate property. Rasa passes Linear PCM sample rates (16 kHz, 24 kHz, 48 kHz) through end-to-end without server-side resampling.
Rime TTS and Browser channelRime TTS does not support 48KHz sample rate. If Browser channel is set to 48KHz sample rate, Rasa will resample audio received from Rime from 24KHz to 48KHz.
Rasa uses the library audioop-lts for conversion between audio encodings (functions like ulaw2lin() or lin2ulaw() and ratecv).

Speech service integrations

Rasa configures automatic speech recognition (ASR) and text-to-speech (TTS) providers on Voice Stream channels. See ASR Integrations and TTS Integrations for supported providers and configuration options.

Automatic Speech Recognition (ASR)

Provider setup and the custom ASR component guide are now on the ASR Integrations page.

Text To Speech (TTS)

Provider setup and the custom TTS component guide are now on the TTS Integrations page.

Custom ASR

See Custom ASR.

Custom TTS

See Custom TTS.

Configuration for Custom Components

See the custom ASR component guide and custom TTS component guide.