Update: As of September 17, 2020, we've released a Rasa Open Source 2.0 release candidate. You can install the release candidate version using pip install rasa==2.0.0rc2. Check out the 1.x to 2.x migration guide for more details.
We're fast approaching our next major release, and we're excited to give the community an in-depth look at what we've been working on. Rasa Open Source 2.0 is centered on simplicity, especially unifying key concepts and reducing the learning curve for newcomers-while still expanding what advanced Rasa users can build. And with the release of our first alpha version 2.0.0a1, you can now take it for an early test drive. Just run pip install rasa==2.0.0a1 to try it out.
Rasa Open Source 2.0 is a continuation of what we started with the 1.0 release, which merged Rasa NLU and Rasa Core into a single library. This change set us up to break down the division between NLU and dialogue and instead think in terms of whole conversations. It's a path that leads to where we are today: building toward a unified training data format, config, and model handling.
When things make more sense on the backend, they tend to make more sense for the end user as well. This, plus tooling that lets developers learn by doing, means developers can successfully explore Rasa without having to be an expert right away. We want newcomers to get their first assistant running as quickly as possible, and we want them to have a smooth path to becoming advanced users.
We're in early stages, and we expect to build on the 2.0.0a1 release as we receive feedback from the community and iterate. Here's a preview of where we're going with Rasa Open Source 2.0.
Suggested Config
Our CLI already provides these easy-to-remember commands:
- rasa train (train a model)
- rasa shell (talk to your assistant)
Now, when you run those commands, Rasa Open Source will also recommend a config.yml for you if you haven't created one already. As you grow your Rasa knowledge, you can start to customise your config.yml, mixing and matching different components, and tweaking hyperparameters.
Rasa developers love the ability to fully customise their NLU pipeline by mixing and matching components in the config.yml file. However, this can be a little daunting if you're working with Rasa for the first time. Rasa Open Source 2.0 will look at your training data and suggest a pipeline configuration automatically. You can use that as a starting point to customise.
There's no one-size-fits-all configuration that works for every data set, and for teams building assistants, the ability to tweak the configuration and hyperparameters to arrive at their best performance is a big advantage of using Rasa. Teams can still manually fine tune hyperparameters, but configuration doesn't need to be one of the first things you grapple with during development.
From many rule policies, to one
The way dialogue policies work in 2.0 is probably the biggest simplification to the developer experience we've ever made.
In Rasa Open Source 1.x, developers configure a set of dialogue management policies to determine the bot's next action. Typically, this configuration includes 5 or 6 policies in a policy ensemble. Each policy has a different priority, allowing e.g. the mapping policy to override others. This system made sense when we first implemented it, but as new policies were added (the MappingPolicy, FormPolicy, FallbackPolicy, etc.) this became overly complex. Something that might not be obvious, even to developers with high levels of experience with Rasa, is that only a few policies use machine learning (the TED policy and Memoization policy, to be exact). The other policies have now all been merged into a single RulePolicy.
Rule-based dialogue policies help you enforce business logic in your assistant. And that business logic is something that you as a developer (or someone on your team) already knows: that when the user says X, the bot needs to do Y. Machine learning policies on the other hand, account for the conversation turns and user behavior that you can't anticipate. To create a good user experience, you need both rule-based and machine learning-based dialogue policies.
However, exposing multiple rule-based policies in the configuration file is messy and caused a great deal of confusion (thanks, as always, for posting your questions on the forum). The user only needs to worry about two things: the logic they want to implement, and the machine learning-based policy that kicks in whenever you're in uncharted territory. In Rasa Open Source 2.0, we're consolidating all of the rule-based policies into a single policy. We've also simplified the format for specifying these rules, so you can keep them separate from your stories.
Bringing forms into Rasa Open Source
In addition to bundling forms into the new Rule Policy, we're also moving the forms implementation from the Rasa SDK into Rasa Open Source. As a basic feature of Rasa Open Source, forms logically belong in the standard library. This makes our SDK much more lightweight while still allowing full customization.
Training data 2.0
We recently polled the community about which future training data format they'd like to see-and YAML was the clear winner. We're adopting YAML over markdown because it provides three advantages: the ability to support custom metadata in training examples (like the user who added it and timestamp), the ability to divide long files into smaller, more modular ones, and the ability to support rich media in the response selector. It's also a standardized format we can adopt without introducing custom syntax, making it easier for developers to work with. Importantly, it's still human read/writeable and leaves nice diffs!
But we have bigger plans ahead. In Rasa Open Source 2.0, we're introducing an experimental feature that brings NLU into the dialogue management model, allowing the dialogue model to access the text of the user message instead of just the predicted intent. This allows the text of the conversation, not just the intent, to influence the prediction of the next action. The training data for this new approach will look much more like the format for testing end-to-end stories; that is, user messages, intents, entities, and bot actions will all be represented in a single story.
This means we can now do end-to-end learning in Rasa, where we don't have to come up with intent labels for messages at all. And the new format is flexible enough that you can mix and match stories, where some messages have an intent label and others don't. We have the foundation for breaking free from the limitations of intents
Join the alpha
Here's where we need your help!
Rasa Open Source 2.0.0a1 is now available. We recommend installing it on a clean environment, separate from your production projects. If you're interested in sharing your feedback, join the conversation in this dedicated forum thread or sign up here.
Conclusion
With major releases, we tend to focus on what's new, and we don't talk as much about the many small changes that are bundled in: the refactoring, the deprecated code that's removed. There are a lot of changes that are happening under the hood with Rasa Open Source 2.0 to pave the way for even bigger things in future releases.
Directionally, we're moving toward a future where building a robust prototype with Rasa Open Source can be accomplished in minutes, not hours. These are big changes, and we need collaboration with our open source community to get there.
We're excited to share a preview of some of the highlights of Rasa Open Source 2.0 and we can't wait to get the release in the hands of our community to start testing and collecting feedback. Tell us what you think, and help us shape the next major release of Rasa Open Source!