Improving an assistant involves making incremental updates both on the Rasa X server and on your local machine. By incorporating automated testing into every update, you will immediately know how any changes to your training data affect the performance of your assistant.
Rasa X is not intended to be an all-in-one bot builder; many things are easier to do in a text editor on your local machine, and that’s where you should do them. Rasa X is meant to help you improve some parts of your assistant using an interface, while still keeping it easy to make changes locally.
In order to improve your assistant, you will need to make updates inside and outside of Rasa X based on what you learn from your conversations. There are 3 types of updates you can make to improve your assistant: major, minor and architectural updates.
The most efficient way to make iterative improvements to your model, such as fixing incorrect NLU / Core predictions or updating response templates, is to do it on the Rasa X server. From there, you can push your changes to the remote Git repository via the Integrated Version Control feature and run automated tests before redeploying.
One example of a minor update is annotating new messages that come from users talking to your assistant. You can improve your assistant by annotating new data from these users. You can classify the data to new or existing intents. Data that is displayed on the Annotate new data screen of comes from a few different sources. These sources include:
User conversations from sharing your assistant or via a custom channel like Slack, Twilio, etc.
Conversations you have with the assistant on the Talk to your bot screen.
You can use this training data to improve your assistant. For example, maybe you have been testing your assistant on the Talk to your bot screen and have noticed some intents are not being predicted correctly, or you’ve have noticed some conversations from your test users that don’t seem correct. You can easily fix these in bulk on the Annotate new data screen.
These 3 sentences came from conversations directly from testing the bot on the Talk to your bot screen.
From here, you can fix the predictions and then mark them as correct, or delete them.
In order to use this new data you will need to retrain the model via the Train button.
For more disruptive changes to your assistant, like merging two intents into one or changing the logic of stories, whether you will want to use Rasa X or not depends on the situation. Usually smaller changes are easier in Rasa X, while larger changes are often faster in a text editor on your local machine. After you make these changes, you will want to push your updates to the Git repo connected to your Rasa X instance, and let a continuous integration (CI) server ensure that everything works as expected.
If the improvement requires altering the structure of your assistant that surrounds and defines your model, such as implementing a custom NLU component or custom action, you will want to use a text editor on your local machine. After you have improved your assistant, you will once again want to go through the process of pushing your changes to the Git repository and then testing and deploying with CI/CD.
Integrating your Rasa X server with Git enables you and your team to apply best practices from software engineering to the development of AI assistants.
Once you have connected your assistant’s repository to your Rasa X server, Rasa X will automatically pull data from your remote repository and reflect those updates in Rasa X. At this initial stage, Rasa X will have the latest data from your remote repository on the Git server. Rasa X will indicate that you are up to date with the latest changes with a green state in the side bar.
When you make a change to your assistant in Rasa X, the side bar state will transition to an orange state. This means that you have changes that can be pushed to the remote repository when you are ready. During this time, Rasa X won’t fetch new changes from the remote repository to avoid overwriting your changes.
When you are finished making changes and are ready to add them to your Git server, you can commit and push them by clicking on the Git icon in the sidebar. If your remote branch has not changed while you were annotating, you can choose between
pushing changes directly to the remote branch
pushing them to a new branch
If new changes were added to the remote branch while you were annotating, you will only be able to push your changes to a new branch to avoid potential merge conflicts. Rasa X will notify you that there have been changes to the remote branch with a red state on the side bar.
When you are ready, click on the side bar icon and follow its instructions. Once your changes are pushed, Rasa X will return to being in sync with your remote branch and will pull in the latest changes from your remote as they become available.
If you pushed your changes to a new branch, these changes won’t be reflected in Rasa X until you have merged them into the target branch.
You can of course also discard your changes. Rasa X will then be reset to the latest state of your remote branch.
Integrating Rasa X with Git enables your team to use the full power of the Git ecosystem:
Apply a review process for any changes to your model
Automatically evaluate dialogues end-to-end, running through test conversations and making sure that both NLU and Core make correct predictions, as part of a CI pipeline (check out End-to-End Evaluation in the Rasa docs for more information)
Automatically generate reports, a confusion matrix, and a confidence histogram on your new model as part of a CI pipeline (check out Evaluating Models in the Rasa docs for more information)
Minimize the time from change to production with continuous deployment