> ## Documentation Index
> Fetch the complete documentation index at: https://rasa.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Observability Metrics

> The custom metrics Rasa emits via OpenTelemetry for LLM usage and performance, and how to enable them alongside tracing.

Metrics are runtime measurements that capture indicators of a service’s availability and performance. Unlike tracing—which helps you understand the sequence of operations for a single request—metrics provide an aggregated statistical view across multiple requests or conversations. Typical examples include **average response time**, **throughput**, and **CPU/memory consumption**. Monitoring these helps you:

* Track the health of your service.
* Quickly detect and alert on outages or anomalies.
* Quantify the impact of code or infrastructure changes on performance.

Tracing must also be enabled for metrics to be recorded. Metric measurements are emitted from Rasa's instrumentation layer, which is only activated when tracing is configured. With both enabled, metrics and traces together give you a complete view of your deployment behavior, making it easier to debug issues and optimize resource usage.

## How To Use Metrics

### Enabling Metrics in Rasa

Rasa uses an [OpenTelemetry (OTEL) Collector](https://opentelemetry.io/docs/collector/) to collect metrics and send them to your desired backend (e.g., Prometheus, Datadog, etc.).

> [!WARNING]
>
> **Tracing is required**
>
> Configuring the `metrics` block alone does not activate metric collection. Rasa records measurements through the same instrumentation layer that powers tracing, and that layer starts only when a `tracing` block is present in your endpoints configuration. You must enable [tracing](https://rasa.com/docs/pro/improve/tracing.md) in addition to metrics.

1. **Enable tracing in your endpoints file (or Helm values):**

   Add a `tracing` block to activate Rasa's instrumentation layer. For example, using an OTLP collector:

```yaml title="endpoints.yml" theme={null}
tracing:
  - type: otlp
    endpoint: my-otlp-host:4318
    insecure: false
    service_name: rasa
    root_certificates: ./path/to/ca.pem
```

See [Tracing](https://rasa.com/docs/pro/improve/tracing.md) for other supported backends (Jaeger, Langfuse, and more).
2\. **Configure metrics in the same endpoints file (or Helm values):**

```yaml title="endpoints.yml" theme={null}
metrics:
  type: otlp
  endpoint: my-otlp-host:4318
  insecure: false
  service_name: rasa
  root_certificates: ./path/to/ca.pem
```

* `type: otlp` indicates you are using OpenTelemetry’s OTLP format.
* `endpoint` is the URL of the OTEL Collector or metrics backend.
* `service_name` is an identifier for your Rasa Pro service.
* `insecure`/`root_certificates` specify how TLS is handled.

Tracing surfaces the sequence of internal method calls for individual requests; metrics aggregate their performance across many requests. Enabling both gives you the most complete observability picture.

### Custom Metrics Collected by Rasa

Once tracing and metrics are both configured, Rasa automatically collects several custom metrics relevant to large language model (LLM) usage and overall assistant performance:

* **CPU and Memory Usage** of any LLM-based command generator (e.g., `CompactLLMCommandGenerator`, `SearchReadyLLMCommandGenerator`) at the time of making an LLM call.
* **Prompt Token Usage** for LLM-based command generators, provided the `trace_prompt_tokens` config property is enabled.
* **Method Call Durations** for LLM-specific components, such as:
  * `EnterpriseSearchPolicy`
  * `ContextualResponseRephraser`
  * `CompactLLMCommandGenerator`
  * `SearchReadyLLMCommandGenerator`
* **HTTP Request Metrics** for the Rasa client:
  * Duration of requests to external services (action server, NLG server, etc.).
  * Request size in bytes.

### Sub-agents (ReAct and A2A)

When a flow uses [autonomous steps](https://rasa.com/docs/reference/primitives/flow-steps.md#autonomous-steps) to hand off to a [ReAct](https://rasa.com/docs/reference/config/agents/react-sub-agents.md) or [A2A](https://rasa.com/docs/reference/config/agents/external-sub-agents.md) sub-agent, Rasa emits extra OpenTelemetry **histograms** (in addition to the spans described in [Tracing](https://rasa.com/docs/pro/improve/tracing.md)):

* **`agent_execution_duration`** — Wall time for each sub-agent run (`_call_agent_with_retry` in the agent executor). Useful for end-to-end latency and error rates per sub-agent. Attribute labels include:

  * **`agent_name`** — Sub-agent id from configuration.
  * **`protocol_type`** — How the sub-agent is connected: `mcp_open` or `mcp_task` for ReAct-style MCP sub-agents, or `a2a` for A2A sub-agents.
  * **`status`** — Final status of the sub-agent result.
* **`mcp_tool_execution_duration`** — Time spent inside an MCP tool call. The **same metric name** is used in two execution paths; use the **`execution_context`** label to distinguish them:

  * **`execution_context` = `flow`** — Tool invoked from a flow [MCP tool step](https://rasa.com/docs/reference/primitives/flow-steps.md#calling-an-mcp-tool). Labels include **`tool_id`**, **`mcp_server`**, and **`success`**.
  * **`execution_context` = `agent`** — Tool invoked while a **ReAct MCP sub-agent** runs (`_execute_tool_call`). Labels include **`tool_name`**, **`agent_name`**, **`protocol_type`**, and **`success`**.
* **ReAct MCP sub-agent LLM usage** — For MCP-based sub-agents, each LLM `send_message` round-trip also records resource-style histograms (CPU and memory sampled like other LLM components, plus estimated prompt size and response duration):

  * `mcp_agent_llm_cpu_usage`
  * `mcp_agent_llm_memory_usage`
  * `mcp_agent_llm_prompt_token_usage`
  * `mcp_agent_llm_response_duration`

By collecting these telemetry metrics, you gain robust insights into how your assistant performs under real-world usage. You can proactively detect issues, understand resource consumption, and tailor your assistant’s architecture to provide the best possible experience for your users.

## Voice metrics

Voice metrics cover calls handled through streaming voice channels.

### `rasa.voice.call.duration`

Records the time from the start of a streaming voice call until the call ends.
Rasa records it once at the end of the call, so ongoing calls are not included.

Use it to compare call durations and outcomes.

| Detail | Value |
| - | - |
| Kind | Histogram |
| Unit | Seconds (`s`) |
| Histogram boundaries | `5`, `15`, `30`, `60`, `120`, `300`, `600`, `1200`, `1800`, `2700`, `3600` |

**Dimensions**

| Dimension | Type | Required | Values |
| - | - | - | - |
| `rasa.voice.outcome` | string | Yes | `completed` — the call ended normally, at the assistant's request, or while Rasa was shutting down; cancellation is not a separate outcome `disconnected` — the connection dropped `failed` — an unhandled error ended the call |

### `rasa.voice.tts.time_to_first_audio`

Records the time to the first audio chunk for one assistant message. For a cache
hit, timing starts when Rasa finds the cached audio. For a cache miss, it starts
when Rasa sends the request to the TTS provider. The metric is not recorded if
no audio is produced or TTS fails, even if fallback audio plays after the
failure. In that case, the
`voice.latency.tts_first_byte_ms` span attribute can still measure the fallback
audio when tracing is enabled.

Use it to compare time to first audio by provider and cache status.

| Detail | Value |
| - | - |
| Kind | Histogram |
| Unit | Seconds (`s`) |
| Histogram boundaries | `0.005`, `0.01`, `0.025`, `0.05`, `0.1`, `0.25`, `0.5`, `1`, `2.5`, `5`, `10`, `30` |

**Dimensions**

| Dimension | Type | Required | Values |
| - | - | - | - |
| `rasa.voice.provider` | string | No | TTS provider name; free-form |
| `rasa.voice.tts.cache_hit` | boolean | No | `true` / `false` |

### `rasa.voice.tts.synthesis.duration`

Records the time to complete TTS for one assistant message. For a cache hit, timing
starts when Rasa finds the cached audio and ends when cached-audio delivery
completes. For a cache miss, it starts when Rasa sends the provider request and
ends when audio generation completes. On failure, timing stops at the first
failure and excludes fallback audio. If TTS neither completes nor fails, no
measurement is recorded.

Use it to compare TTS completion times by outcome, provider, and cache status.

| Detail | Value |
| - | - |
| Kind | Histogram |
| Unit | Seconds (`s`) |
| Histogram boundaries | `0.005`, `0.01`, `0.025`, `0.05`, `0.1`, `0.25`, `0.5`, `1`, `2.5`, `5`, `10`, `30` |

**Dimensions**

| Dimension | Type | Required | Values |
| - | - | - | - |
| `rasa.voice.outcome` | string | Yes | `completed` `error` |
| `rasa.voice.provider` | string | No | TTS provider name; free-form |
| `rasa.voice.tts.cache_hit` | boolean | No | `true` / `false` |

### `rasa.voice.playback.duration`

Records the time from the first delivered audio chunk until playback completes,
is interrupted, is cancelled, or fails. Rasa records it only after at least one
audio chunk is delivered. If playback ends before delivering audio, this metric
is not recorded, but `rasa.voice.operation.failures` can still count an
unexpected failure.

Use it to compare playback times by outcome.

| Detail | Value |
| - | - |
| Kind | Histogram |
| Unit | Seconds (`s`) |
| Histogram boundaries | `0.25`, `0.5`, `1`, `2.5`, `5`, `10`, `20`, `30`, `60`, `120`, `300` |

**Dimensions**

| Dimension | Type | Required | Values |
| - | - | - | - |
| `rasa.voice.outcome` | string | Yes | `completed` `interrupted` — the caller interrupted the assistant `cancelled` — the channel cancelled playback `error` |

### `rasa.voice.operation.failures`

Counts unexpected failures in streaming voice operations. Expected outcomes,
such as an ignored barge-in, a DTMF timeout, or interrupted or cancelled
playback, are not counted.

Use it to identify failures by component, operation, provider, and error type.

| Detail | Value |
| - | - |
| Kind | Counter |
| Unit | Failures (`{failure}`) |

**Dimensions**

| Dimension | Type | Required | Values |
| - | - | - | - |
| `rasa.voice.component` | string | Yes | `asr` `tts` `playback` `composer` `channel` |
| `rasa.voice.operation` | string | Yes | `connect` `disconnect` `transcribe` `synthesize` `play` `process_channel_message` `compose_event` |
| `rasa.voice.provider` | string | No | ASR or TTS provider name, when applicable; free-form |
| `rasa.voice.error_type` | string | Yes | Exception class name or explicit error value; free-form. Exception messages are excluded. |

### `rasa.voice.barge_in.decisions`

Counts one accepted or ignored barge-in decision for each transcript update Rasa
evaluates, including partial transcripts. Decisions are made when interruptions
are enabled and the caller speaks while assistant audio is playing or waiting to
play. One caller interruption can produce several ignored decisions before an
accepted one. Other caller speech that does not result in a decision is not
counted.

Use it to compare accepted and ignored decisions and their reasons.

| Detail | Value |
| - | - |
| Kind | Counter |
| Unit | Decisions (`{decision}`) |

**Dimensions**

| Dimension | Type | Required | Values |
| - | - | - | - |
| `rasa.voice.outcome` | string | Yes | `accepted` / `ignored` |
| `rasa.voice.reason` | string | No | `below_min_words` — the transcript did not reach the configured minimum word count `word_threshold_reached` — the transcript reached the configured minimum word count |

### `rasa.voice.dtmf.decisions`

Counts accepted DTMF input, keypresses received when Rasa is not waiting for
DTMF input, and DTMF input attempts that time out. Keypresses are not counted
individually while Rasa waits for the full input.

Use it to compare keypad interaction outcomes without recording the digits
entered.

| Detail | Value |
| - | - |
| Kind | Counter |
| Unit | Decisions (`{decision}`) |

**Dimensions**

| Dimension | Type | Required | Values |
| - | - | - | - |
| `rasa.voice.outcome` | string | Yes | `accepted` `ignored` `timeout` |
| `rasa.voice.reason` | string | No | Reason for the decision, when provided; free-form. Rasa currently omits this dimension for DTMF decisions. |

### `rasa.voice.silence_timeouts`

Counts silence timers that expire and trigger a turn. Timers cancelled before
they expire are not counted.

Use it to compare counts across configured timeout values.

| Detail | Value |
| - | - |
| Kind | Counter |
| Unit | Timeouts (`{timeout}`) |

**Dimensions**

| Dimension | Type | Required | Values |
| - | - | - | - |
| `rasa.voice.outcome` | string | Yes | `accepted` |
| `rasa.voice.silence.timeout_s` | double | No | Configured timeout in seconds, rounded to one decimal place; this is a configuration value, not elapsed time |

### `rasa.voice.audio.bytes`

Counts audio bytes processed during streaming voice calls. At the end of a call,
Rasa records the total for each direction with more than zero bytes. Directions
with no audio and ongoing calls are not included.

Use it to compare incoming and outgoing audio volume.

| Detail | Value |
| - | - |
| Kind | Counter |
| Unit | Bytes (`By`) |

**Dimensions**

| Dimension | Type | Required | Values |
| - | - | - | - |
| `rasa.voice.audio.direction` | string | Yes | `input` — caller audio received by Rasa `output` — audio sent to the caller |


## Related topics

- [Tracing](https://rasa.com/docs/pro/improve/tracing.md)
- [Studio Environment Variables](https://rasa.com/docs/reference/config/studio-environment-variables.md)
- [Environment Variables](https://rasa.com/docs/reference/config/environment-variables.md)
- [Configuration & Results](https://rasa.com/docs/reference/testing/evals/configuration-and-results.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.