Migrating an NLU-based assistant to CALM

Coexistence allows you to run a single assistant that uses both the Conversational AI with Language Models (CALM) approach and an NLU-based system in parallel for the purpose of migrating from NLU-based assistants to CALM in an iterative fashion. This way you do not need to do a waterfall migration and can instead start gathering confidence in using CALM in production with small steps.

Glossary#

• CALM: CALM (Conversational AI with Language Models), Rasa’s new system for creating powerful yet steerable conversational experiences with large language models fast.
• NLU-based system: NLU-based systems rely on NLU components to process incoming user messages and detect intents and entities. Further, rules and stories are used to steer the next action.
• System: Whenever we use phrases like “the two systems” or “either of the systems” we refer to CALM or the NLU-based system.
• Skill: A small self-contained piece of business logic that lets the user achieve a goal or answer one or multiple questions. Examples of skills are transferring money, checking account balance, or answering frequently asked questions. Oftentimes, skills require the user to provide multiple pieces of information across multiple steps.
• Skill interruption: A skill is interrupted when during its execution another skill is started without stopping the earlier skill. For example, while in the process of sending money and being asked to confirm the transaction, users might do a final check of of their account balance or transaction history before giving the final confirmation. In many cases, it is desirable to return to the interrupted skill after the interrupting skill is finished.
• Topic area: A collection of skills that are closely related and oftentimes happen together and interrupt one another. An example for a topic area would be “investment” containing skills such as providing security quotes, buying and selling securities, portfolio management, providing quarterly and yearly results of a company, etc.

Getting Started#

Identification of Topic Areas#

To get started using coexistence, it is of ample importance that you identify individual skills and at least one topic area in your assistant for migration. Individual skills are useful to identify because they most commonly turn into individual flows in CALM. Topic areas are important because they group skills that should be migrated together and allow for easy skill interruptions in a single conversation session.

For example, if your bot allows for transferring money and you want to move that skill to CALM, you probably also want to move:

• skills such as
  • checking balance
  • seeing transaction history
  • adding contacts (if users first have to add someone as a contact before sending money)
• answering of frequently asked questions around the topic of money transfer

Moving these other skills as well will allow users to start these skills while having started a money transfer. If the other skills would remain in the NLU-based system only, they could only be started after the money transfer is finished or aborted.

In some cases, you might want skills to be available in both systems, in that case you should keep a copy of that skill in the NLU-based system as well. There can be two reasons to keep a skill in the NLU-based system:

• The skill is commonly used by topics in both systems.
• It's a high stakes skill, but because skill interruptions can happen within one system only, you would like the bot to still be able to handle it in both systems.

Once you have identified skills and a first topic area to migrate, you can start writing flows.

In the next steps we take a look at how to integrate your new flows with the remainder of your NLU-based system.

Updating your NLU pipeline and policies#

First of all you need to decide whether the routing mechanism should be based on intents or whether an LLM should decide where incoming messages should be routed to. You can choose between two different router components: IntentBasedRouter and LLMBasedRouter. IntentBasedRouter routes the messages based on the predicted intent of your NLU components. The LLMBasedRouter leverages an LLM to decide whether an incoming message should be routed to the NLU-based system or CALM. For more information on those components, please read the router components documentation.

Once you have decided which router component to use, you need to add it to your config file together with the LLMCommandGenerator and FlowPolicy. For more information on the LLMCommandGenerator or the FlowPolicy, please consult the documentation for flows and dialogue understanding.

After adding the components, your config.yml might look like the following, depending on which router you chose and which NLU-based system components you employ:

Adding the routing slot to your domain#

The routing slot is used to store the routing decision of the router component, e.g. whether a message should be routed to the NLU-based system or CALM.

It’s important that you add the routing slot to your domain like the following:

This needs to be exactly this way. It needs to be a boolean slot to capture the three cases based on its value:

• None: no routing active, router will be engaged
• True: routing to CALM is active
• False: routing to the NLU-based system is active

A routing session is active when the routing slot is either set to True or False. Incoming messages are routed accordingly. If the routing slot is reset and no routing is active, e.g. set to None, the router will be engaged again on the next incoming message. To reset the routing slot the new default action action_reset_routing can be called (see section).

The influence_conversation: false is important to not trip up the NLU-based system policies based on the value of this slot.

If you want to have one of the system active right from the beginning of the conversation, you can give the slot an initial_value. During routing resets, this slot will always be set to None and not to its initial value.

Managing routing resets#

Resetting the routing slot is important to be able to achieve multiple skills in a single session. To reset the routing the new default action action_reset_routing needs to be called.

Resetting when CALM is finished

Resetting once CALM is finished is relatively straightforward. Whenever all started flows are finished, CALM triggers the flow pattern_completed. We can leverage this fact and adjust this pattern slightly as follows:

In the original definition of pattern_completed, we just ask the user whether we can do something else for them. In this version, we also reset the routing. Make sure to add this snippet to your flows to overwrite the default implementation of pattern_completed that comes with CALM.

This setup will take care of all the following cases:

• User starts a flow and finishes or cancels it.
• User starts multiple flows in sequence or simultaneously and finishes or cancels them all.
• User starts a flow and asks knowledge questions or uses chitchat while in a flow and then finishes or cancels the flow.
• User starts a CALM session directly with a knowledge related question.

Resetting when the NLU-based system is finished

In contrast to CALM, where the boundaries of skills are well defined, the NLU-based system policies use more loosely connected fragments of business logic to describe skills. If you want to allow the user to use the CALM part of your assistant after a skill was completed in the NLU-based system, you will have to add the action_reset_routing at the appropriate places in your rules and stories.

Usually, this will be at the end of different stories and rules. Potentially you have some conversation parts that you carry out at the end of each interaction. These parts can be great places at whose end you can reset the routing as in the following:

It is very likely that you have to add the action_reset_routing at the end of multiple stories or rules, whenever you are sure that the user has just finished a skill and nothing else needs to be wrapped up in the NLU-based system such as an interrupted skill.

Managing shared slots between CALM and the NLU-based system#

To prevent slots from being reset during the action_reset_routing you can configure slots with the option shared_for_coexistence: True.

Alternatively, you can store user profile information in a database and retrieve it in different places during your stories and flows and store it in slots temporarily. In this case, you won’t need to worry about the persistence of shared slots as the data is persisted in the database.

Custom Actions#

When moving functionality to CALM you might have to adjust your custom actions to varying degrees. Some concepts, such as dynamic forms have been integrated into the core flow functionality and should be implemented using branches in flows. It is best to reconsider the implementation of all custom actions that are moved to CALM and assess whether they should be adjusted to better play with the new paradigm.

Generally speaking, it is also possible to use the same action both in CALM and the NLU-based system. Both systems share the same action server.

Testing your assistant#

Testing your assistant using both CALM and NLU-based system can be achieved using e2e tests. The definition of these tests is not very dependent on the underlying implementation and should thus not need to change significantly when more and more skills are moved from the NLU-based system to CALM subsequently. e2e tests help you to test skill interruptions and skill triggers across paradigms.

You can still evaluate just the NLU-based system of your assistant using the command rasa test.