Improving an assistant involves making incremental updates both on the Rasa X server and on your local machine. By incorporating automated testing (e.g. End-to-End Testing) into every update, you will immediately know how any changes to your training data affect the performance of your assistant.
Rasa X is not intended to be an all-in-one bot builder; many things are easier to do in a text editor on your local machine, and that’s where you should do them. Rasa X is meant to help you improve some parts of your assistant using a UI, while still keeping it easy to make changes locally.
In order to improve your assistant, you will need to make updates inside and outside of Rasa X based on what you learn from your conversations. There are 3 types of updates you can make to improve your assistant: minor, major, and architectural updates.
The most efficient way to make iterative improvements to your model, such as fixing incorrect NLU / Core predictions or updating response templates, is to do it on the Rasa X server. From there, you can push your changes to the remote Git repository via the Integrated Version Control feature and run automated tests before redeploying.
One example of a minor update is annotating new messages that come from users talking to your assistant. You can improve your assistant by annotating new data from these users. You can classify the data to new or existing intents.
Data that is displayed in the NLU Inbox comes from a few different sources. These sources include:
User conversations from sharing your assistant or via a custom channel like Slack, Twilio, etc.
Conversations you have with the assistant on the Talk to your bot screen.
In the NLU Inbox, you can review & fix intent and entity predictions, mark them as correct, or delete them.
You can also make use of keyboard shortcuts to navigate the NLU Inbox table rows and perform actions.
k keys allow you to navigate down and up respectively. Pressing
s will mark the currently selected item as correct.
d will delete the currently selected item from the inbox.
Once you select Mark Correct, the data will be moved from the NLU Inbox to the Training Data screen.
To annotate entities, you select one or multiple words in a sentence. A menu will pop up where you select the Entity type and an optional Entity Synonym to be used instead of the actual text in the sentence:
In order to use this training data you will need to retrain your assistant via the Train button.
For more disruptive changes to your assistant, like merging two intents into one or changing the logic of stories, whether you will want to use Rasa X or not depends on the situation. Usually smaller changes are easier in Rasa X, while larger changes are often faster in a text editor on your local machine. After you make these changes, you will want to push your updates to the Git repo connected to your Rasa X instance, and let a continuous integration (CI) server ensure that everything works as expected.
A good example of a major update that you can do with Rasa X is the creation of a new story using Interactive Learning mode on the Talk to your bot screen.
With Interactive Learning, you create a story by teaching your assistant as you talk to it. At each step in the conversation, you validate and correct the predictions that are made by your assistant. At the end, the new story can be added directly to the training data. In addition, the same story will be available to use as an end-to-end test.
To start, on the Talk to your bot screen you select the Interactive Learning button:
In this example, when you enter hello, the assistant correctly predicts the intent greet. However, because you are in Interactive Learning mode, the conversation with the assistant pauses until you mark this as a correct prediction by clicking the check box.
As soon as you click the check box, the sentence with it’s prediction will be added to the Training Data screen. If a sentence happens to be already in the Training Data, it will not be added again to avoid duplication.
You can check if a sentence like ‘Yoh!’ also triggers the greet intent, by refreshing the page, switching again to Interactive Learning mode, and typing it in to check the prediction:
In this case, the assistant predicts the intent affirm, which is wrong. You can change the predicted intent in the dropdown to greet, and then click on the checkbox.
After clicking the checkbox, you will see the next predicted Core action. The conversation will again pause until you validate or correct this. In this example, the next action of the assistant is correct, and you click the checkbox.
The conversation is moving along nicely, and after the assistant has completed the utter_great action, you will see the next predicted step in the conversation. It is predicted that the assistant should take as Next Action: action_listen. This is correct, so you click the checkbox.
You continue this flow until you have created a new story that you want to add to the Training Data.
On the right hand side of the screen you can find the Story till now and you can save it in the Training Data by clicking the Save Story button.
This completes the major update of creating a new story via Interactive Learning, and you can now retrain your assistant via the Train button.
To enable End-to-End Testing,
Rasa X also provides the story in an end-to-end story format, which includes
both the NLU output and the original text. If Rasa X is connected to a remote Git
you can save the end-to-end story by clicking on the
Save end-to-end test button.
If the improvement requires altering the structure of your assistant that surrounds and defines your model, such as implementing a custom NLU component or custom action, you will want to use a text editor on your local machine. After you have improved your assistant, you will want to go through the process of pushing your changes to the Git repository and then testing and deploying with CI/CD.