Conversation Repair

Overview#

Flows describe your assistant's business logic. In the tutorial example, your transfer_money flow specifies that your assistant needs to ask the user for the recipient and the amount_of_money.

The "happy path" refers to a conversation where every time the assistant asks the user for information, the user provides an answer which successfully fills the requested slot. Conversation Repair is there to handle all the conversations which differ from the happy path.

For example:

• your assistant asked for an amount of money, but the user says something else.
• the end user interrupts the current flow and switches context to another topic.
• the end user changes their mind about something they said earlier.

Rasa has default patterns for handling each of these cases. Each of these patterns is a pre-built flow. The behaviour can be fully customized by adding a flow with the same name to your assistant.

Conversation Repair Cases#

1. Digressions#

Digressions arise when a user shifts from one flow to another.

Example: During a money transfer, a user might ask about their current balance.

2. Corrections#

Corrections occur when users modify input data or rectify mistakes.

Example: A user might change their mind about the recipient of a transfer.

3. Cancellations#

Cancellations happen when users halt a flow midway.

Example: A user opts out of sending money after initiating the process.

4. Skipping collect steps#

Occurs when a user intends to bypass the collect step either by avoiding to provide the requested information or requesting to move past the current step.

Example: A user avoids to answer the bot question.

5. Chitchat#

Engage in off-topic interactions without impacting a flow.

Example: A user has a casual conversation with the assistant that responds with free-form response.

The default behavior relies on responses defined within the domain that are not part of any flow, with the Intentless Policy determining the appropriate response. If the IntentlessPolicy is not configured, the assistant will activate the Cannot Handle pattern, thus informing the user that their request cannot be handled. This effectively disables Chitchat.

Additionally, you have the option to explicitly disable Chitchat. For instructions on how to do this, see this section.

You can customize the default behavior to enable free-form responses. See this section on how to do it.

6. Completion#

Flows conclude either by achieving user goals or by user abandonment.

Example: User queries about account balance.

7. Clarification#

Clarifications arise when the user request can't clearly be identified and potentially matches multiple flows.

Example: User request can be matched to two flows.

8. Internal Errors#

Errors arise from unexpected system or flow issues:

1. Unavailable actions, unresponsive modules, or errors from dependent services (e.g. timeout response from LLM).
2. User input exceeds predefined limit (if limits are set).
3. User sends empty message.
4. Errors from dependent services, such as a timeout response from LLM.
5. (with Enterprise Search Policy) In case of an error when connecting to the vector store, during document retrieval or from the LLM. Please click here for more details

Below are examples showcasing different internal error scenarios:

Example: Internal error is raised due to unavailable actions or timeout from LLM.

Example: Internal error is raised due to exceeded input limit.

Example: Internal error is raised due to empty user message.

Context Attributes

error_type and info serve as context attributes for managing errors in pattern_internal_error. info attribute is a dictionary meant to contain additional information like max_characters. error_type acts as a switch-case selector. It determines the appropriate error message based on the encountered scenario. The message is then sent to the user.

• rasa_internal_error_user_input_too_long, set when user input exceeds the user_input.max_character limit of the LLMCommandGenerator.
• rasa_internal_error_user_input_empty, set by blank inputs.

These context attributes are not available outside of the pattern_internal_error.

9. Cannot Handle#

This pattern is triggered to gracefully handle the following situations:

• When the LLM-based command generator encounters scenarios where it is unable to predict a valid command (e.g. LLM hallucinates attempting to start a non-existing flow) this mechanism prompts the user to rephrase their request. Additionally, MultiStepLLMCommandGenerator can directly predict the command when the scope of the user message is beyond starting, canceling, or clarifying a flow.

• When the Enterprise Search policy cannot retrieve any relevant documents from the vector store, it triggers this mechanism. In this case, the bot may need to prompt the user to rephrase their request or inform the user that relevant information couldn't be found.

• User indulges in an off-topic conversation (chitchat) and the assistant is configured to respond with predefined responses but it is trained without the IntentlessPolicy in the pipeline.

Context Attributes

The pattern pattern_cannot_handle has the context attribute reason. reason is set to:

• cannot_handle_chitchat, when pattern_chitchat attempts to invoke action_trigger_chitchat without an IntentlessPolicy defined.

The context attribute is not be available outside of pattern_cannot_handle.

10. Human Handoff#

When the user requests to be connected to a human agent or when the assistant cannot handle the user's request, the assistant can handoff the conversation.

Example: User requests to be connected to a human agent.

Configurations#

Default Behavior#

Rasa ships a default behavior for every conversation repair case that works out-of-the-box. Each case is handled through a pattern which is a special flow designed specifically to handle the case:

• pattern_continue_interrupted for digressions.
• pattern_correction for corrections.
• pattern_cancel_flow for cancellations.
• pattern_skip_question for skipping collect steps.
• pattern_chitchat for chitchat.
• pattern_completed for completion.
• pattern_clarification for clarification.
• pattern_internal_error for internal errors.
• pattern_cannot_handle for cannot handle.
• pattern_human_handoff for human handoff.

The syntax for each of these flows is the same as any other flow.

Modifying default behaviour#

It is possible to override the default behaviour of each conversation repair case by creating a flow with the same name as that of the pattern used to handle the corresponding case, like pattern_correction. If the pattern uses a default action which needs to be modified, you can override the implementation of the default action by implementing a new custom action and use that custom action in the flow.

It is possible to add a link step from a pattern to a flow, except for pattern_internal_error, where link steps are not allowed.

Additionally, you can link a pattern to the pattern_human_handoff.

Sample Configuration#

Modify Rasa's response when a flow concludes:

Common Modifications#

Here are some common modifications to the default behavior.

Requiring Confirmation#

You can change the default implementation for a correction and require a confirmation from the user before a slot is updated, e.g. this would result in a conversation like this:

To achieve the above confirmation, create a flow named pattern_correction which is defined as follows:

Also make sure to add the used responses and slots to your domain file:

Implementing a Human Handoff#

Currently, the default behaviour for a human handoff is to inform the user that the assistant cannot help with the request. However, in scenarios where customer service is available, implementing a human handoff becomes relevant. You can implement a human handoff by writing a custom action and overriding the flow named pattern_human_handoff:

Also make sure to add the used actions, responses and slots to your domain file:

React dependent on the current flow#

You can change a pattern's behaviour depending on the flow that was interrupted. This can be done by using the context object in the if condition of a pattern:

In the above example, the inform_user step is only used if the flow that was interrupted is called transfer_money.

Free form generation for chitchat#

By default, chitchat operates via action_trigger_chitchat that invokes the IntentlessPolicy to provide a relevant predefined response.

To switch to free-form generated responses, override the default behaviour of pattern_chitchat by creating a flow named pattern_chitchat which is defined as follows:

Disabling chitchat#

By default, if the Intentless Policy is not configured, the assistant defaults to the Cannot Handle pattern, effectively restricting chitchat by informing the user that the request cannot be processed.

To completely restrict casual conversation, override the default behaviour of pattern_chitchat by creating a flow named pattern_chitchat. Instead of the usual behavior that triggers action_trigger_chitchat, configure it to use a predefined response:

Skipping clarification#

By default, clarification will check on the users intention by asking the user to choose from a list of flows. In some cases, it may be desirable to skip clarification and move directly to starting a flow by adding a link step directly to that flow.

Preventing multiple Clarifications#

By default, clarification will continue to check on the users intentions regardless of the number of times it has asked. This can be avoided by counting the number of clarification requests and linking to the human handoff pattern if this count reaches some threshold.

This would require the custom action action_increase_clarification_count to be implemented and added to the domain.yml along with the clarification_count slot.

Reference: Default Pattern Configuration#

For reference, here is the complete default configuration for conversation repair: