Improve Assistant

Improving an assistant involves making incremental updates both on the Rasa X server and on your local machine. By incorporating automated testing into every update, you will immediately know how any changes to your training data affect the performance of your assistant.

Rasa X is not intended to be an all-in-one bot builder; many things are easier to do in a text editor on your local machine, and that’s where you should do them. Rasa X is meant to help you improve some parts of your assistant using an interface, while still keeping it easy to make changes locally.

Making Incremental Updates

The most efficient way to make iterative improvements to your model, such as fixing incorrect NLU / Core predictions or updating response templates, is to do it on the Rasa X server. From there, you can push your changes to the remote Git repository via the Integrated Version Control feature and run automated tests before redeploying.

For more disruptive changes to your assistant, like merging two intents into one or changing the logic of stories, whether you will want to use Rasa X or not depends on the situation. Usually smaller changes are easier in Rasa X, while larger changes are often faster in a text editor on your local machine. After you make these changes, you will want to push your updates to the Git repo connected to your Rasa X instance, and let a continuous integration (CI) server ensure that everything works as expected.

If the improvement requires altering the structure of your assistant that surrounds and defines your model, such as implementing a custom NLU component or custom action, you will want to use a text editor on your local machine. After you have improved your assistant, you will once again want to go through the process of pushing your changes to the Git repository and then testing and deploying with CI/CD.

Using Integrated Version Control

Integrating your Rasa X server with Git enables you and your team to apply best practices from software engineering to the development of AI assistants.

Once you have connected your assistant’s repository to your Rasa X server, Rasa X will automatically pull data from your remote repository and reflect those updates in Rasa X. At this initial stage, Rasa X will have the latest data from your remote repository on the Git server. Rasa X will indicate that you are up to date with the latest changes with a green state in the side bar.

Rasa X server in a clean Git state

When you make a change to your assistant in Rasa X, the side bar state will transition to an orange state. This means that you have changes that can be pushed to the remote repository when you are ready. During this time, Rasa X won’t fetch new changes from the remote repository to avoid overwriting your changes.

Rasa X server with changes which can be committed and pushed.

When you are finished making changes and are ready to add them to your Git server, you can commit and push them by clicking on the Git icon in the sidebar. If your remote branch has not changed while you were annotating, you can choose between

  • pushing changes directly to the remote branch
  • pushing them to a new branch

If new changes were added to the remote branch while you were annotating, you will only be able to push your changes to a new branch to avoid potential merge conflicts. Rasa X will notify you that there have been changes to the remote branch with a red state on the side bar.

Rasa X server with changes while remote is ahead.

When you are ready, click on the side bar icon and follow its instructions. Once your changes are pushed, Rasa X will return to being in sync with your remote branch and will pull in the latest changes from your remote as they become available.

Note

If you pushed your changes to a new branch, these changes won’t be reflected in Rasa X until you have merged them into the target branch.

You can of course also discard your changes. Rasa X will then be reset to the latest state of your remote branch.

Leveraging Git

Integrating Rasa X with Git enables your team to use the full power of the Git ecosystem:

  • Apply a review process for any changes to your model
  • Automatically evaluate dialogues end-to-end, running through test conversations and making sure that both NLU and Core make correct predictions, as part of a CI pipeline (check out End-to-End Evaluation in the Rasa docs for more information)
  • Automatically generate reports, a confusion matrix, and a confidence histogram on your new model as part of a CI pipeline (check out Evaluating Models in the Rasa docs for more information)
  • Minimize the time from change to production with continuous deployment