notice

This is documentation for Rasa Documentation v2.x, which is no longer actively maintained.
For up-to-date documentation, see the latest version (3.x).

Version: 2.x

Version Migration Guide

This page contains information about changes between major versions and how you can migrate from one version to another.

Rasa 2.7 to 2.8

caution

This release breaks backward compatibility of machine learning models. It is not possible to load models trained with previous versions of Rasa Open Source. Please re-train your assistant before using this version.

Deprecations

Tracker Featurizers

training_states_actions_and_entities method of TrackerFeaturizer, FullDialogueTrackerFeaturizer and MaxHistoryTrackerFeaturizer classes is deprecated and will be removed in Rasa Open Source 3.0 . If you had a custom tracker featurizer which relied on this method from any of the above classes, please use training_states_labels_and_entities instead.

training_states_actions method of TrackerFeaturizer, FullDialogueTrackerFeaturizer and MaxHistoryTrackerFeaturizer classes is deprecated and will be removed in Rasa Open Source 3.0 . If you had a custom tracker featurizer which relied on this method from any of the above classes, please use training_states_labels instead.

State Featurizer

encode_all_actions method of SingleStateFeaturizer class is deprecated and will be removed in Rasa Open Source 3.0 . It is recommended to use the method encode_all_labels instead.

Incremental Training

Users don't need to specify an additional buffer size for sparse featurizers anymore during incremental training.

Space for new sparse features are created dynamically inside the downstream machine learning models - DIETClassifier, ResponseSelector. In other words, no extra buffer is created in advance for additional vocabulary items and space will be dynamically allocated for them inside the model.

This means there's no need to specify additional_vocabulary_size for CountVectorsFeaturizer or number_additional_patterns for RegexFeaturizer. These parameters are now deprecated.

Before

pipeline:
- name: "WhitespaceTokenizer"
- name: "RegexFeaturizer"
number_additional_patterns: 100
- name: "CountVectorsFeaturizer"
additional_vocabulary_size: {text: 100, response: 20}

Now

pipeline:
- name: "WhitespaceTokenizer"
- name: "RegexFeaturizer"
- name: "CountVectorsFeaturizer"

Machine Learning Components

The option model_confidence=linear_norm is deprecated and will be removed in Rasa Open Source 3.0.0.

Rasa Open Source 2.3.0 introduced linear_norm as a possible value for model_confidence parameter in machine learning components such as DIETClassifier, ResponseSelector and TEDPolicy. Based on user feedback, we have identified multiple problems with this option. Therefore, model_confidence=linear_norm is now deprecated and will be removed in Rasa Open Source 3.0.0. If you were using model_confidence=linear_norm for any of the mentioned components, we recommend to revert it back to model_confidence=softmax and re-train the assistant. After re-training, we also recommend to re-tune the thresholds for fallback components.

Rasa 2.5 to 2.6

Forms

New ignored_intents parameter in Forms

There is a new parameter under Forms called ignored_intents. This parameter can be used to prevent any required slots in a form from being filled with the specified intent or intents. Please see the Forms documentation for examples and more information on how to use it in your domain.yml file.

Before, if a user did not want to fill any slots of a form with a specified intent they would have to define it under the not_intent parameter for every slot mapping as shown in the following example :

domain.yml
forms:
restaurant_form:
cuisine:
- type: from_entity
entity: cuisine
not_intent: chitchat
num_people:
- type: from_entity
entity: number
intent: [inform, request_restaurant]
not_intent: chitchat
feedback:
- type: from_entity
entity: feedback
not_intent: chitchat

By introducing the ignored_intents parameter, we now only need to define it in one place and it will affect all the slots of the form :

domain.yml
forms:
restaurant_form:
ignored_intents: chitchat
required_slots:
cuisine:
- type: from_entity
entity: cuisine
num_people:
- type: from_entity
entity: number
intent: [inform, request_restaurant]
feedback:
- type: from_entity
entity: feedback
- type: from_text

Rasa 2.4 to 2.5

Machine Learning Components

DIET, TED, and ResponseSelector

The former weight_sparsity parameter of the DIETClassifier, TEDPolicy, and the ResponseSelector, is now deprecated and superseded by the new connection_density parameter. The old weight_sparsity is roughly equivalent to 1 - connection_density, except at very low densities (high sparsities).

To avoid deprecation issues, you should set connection_density to 1 - your former weight_sparsity setting throughout the config file. (If you left weight_sparsity at its default setting, you don't need to do anything.)

SpaCy 3.0

Rasa now supports spaCy 3.0. This means that we can support more features for more languages but this also introduced a breaking change. SpaCy 3.0 deprecated the spacy link <language model> command. So from now on you need to use the the full model name in the config.yml file.

Before

Before you could run spacy link en en_core_web_md and then we would be able to pick up the correct model from the language parameter.

language: en
pipeline:
- name: SpacyNLP

Now

This behavior will be deprecated and instead you'll want to be explicit in config.yml.

language: en
pipeline:
- name: SpacyNLP
model: en_core_web_md

Fallback

To make the transition easier, Rasa will try to fall back to a medium spaCy model whenever a compatible language is configured for the entire pipeline in config.yml, even if you don't specify a model. This fallback behavior is temporary and will be deprecated in Rasa Open Source 3.0.0.

We've updated our docs to reflect these changes. All examples now show a direct link to the correct spaCy model. We've also added a warning to the SpaCyNLP docs that explains the fallback behavior.

Rasa 2.3 to Rasa 2.4

Deprecating template for response

NLG Server

  • Changed request format to send response as well as template as a field. The template field will be removed in Rasa Open Source 3.0.0.

rasa.core.agent

  • The terminology template is deprecated and replaced by response. Support for template from the NLG response will be removed in Rasa Open Source 3.0.0. Please see here for more details.

rasa.core.nlg.generator

  • generate() now takes in utter_action as a parameter.
  • The terminology template is deprecated and replaced by response. Support for template in the NaturalLanguageGenerator will be removed in Rasa Open Source 3.0.0.

rasa.shared.core.domain

  • The property templates is deprecated. Use responses instead. It will be removed in Rasa Open Source 3.0.0.
  • retrieval_intent_templates will be removed in Rasa Open Source 3.0.0. Please use retrieval_intent_responses instead.
  • is_retrieval_intent_template will be removed in Rasa Open Source 3.0.0. Please use is_retrieval_intent_response instead.
  • check_missing_templates will be removed in Rasa Open Source 3.0.0. Please use check_missing_responses instead.

Response Selector

  • The field template_name will be deprecated in Rasa Open Source 3.0.0. Please use utter_action instead. Please see here for more details.
  • The field response_templates will be deprecated in Rasa Open Source 3.0.0. Please use responses instead. Please see here for more details.

Rasa 2.3.3 to Rasa 2.3.4

caution

This is a release breaking backwards compatibility of machine learning models. It is not possible to load previously trained models if they were trained with model_confidence=cosine or model_confidence=inner setting. Please make sure to re-train the assistant before trying to use it with this improved version.

Machine Learning Components

Rasa Open Source 2.3.0 introduced the option of using cosine similarities for model confidences by setting model_confidence=cosine. Some post-release experiments revealed that using model_confidence=cosine is wrong as it can change the order of predicted labels. That's why this option was removed in Rasa Open Source version 2.3.4.

model_confidence=inner is deprecated as it produces an unbounded range of confidences which can break the logic of assistants in various other places.

We encourage you to try model_confidence=linear_norm which will produce a linearly normalized version of dot product similarities with each value in the range [0,1]. This can be done with the following config:

- name: DIETClassifier
model_confidence: linear_norm
constrain_similarities: True

If you trained a model with model_confidence=cosine or model_confidence=inner setting using previous versions of Rasa Open Source, please re-train by either removing the model_confidence option from the configuration or setting it to linear_norm.

Rasa 2.2 to Rasa 2.3

General

If you want to use Tensorboard for DIETClassifier, ResponseSelector, or TEDPolicy and log metrics after every (mini)batch, please use 'batch' instead of 'minibatch' as 'tensorboard_log_level'.

Machine Learning Components

A few changes have been made to the loss function inside machine learning (ML) components DIETClassifier, ResponseSelector and TEDPolicy. These include:

  1. Configuration option loss_type=softmax is now deprecated and will be removed in Rasa Open Source 3.0.0. Use loss_type=cross_entropy instead.
  2. The default loss function (loss_type=cross_entropy) can add an optional sigmoid cross-entropy loss of all similarity values to constrain them to an approximate range. You can turn on this option by setting constrain_similarities=True. This should help the models to perform better on real world test sets.

A new option model_confidence has been added to each ML component. It affects how the model's confidence for each label is computed during inference. It can take one of three values:

  1. softmax - Dot product similarities between input and label embeddings are post-processed with a softmax function, as a result of which confidence for all labels sum up to 1.
  2. cosine - Cosine similarity between input and label embeddings. Confidence for each label will be in the range [-1,1].
  3. linear_norm - Dot product similarities between input and label embeddings are post-processed with a linear normalization function. Confidence for each label will be in the range [0,1].

The default value is softmax, but we recommend trying linear_norm. This should make it easier to tune thresholds for triggering fallback. The value of this option does not affect how confidences are computed for entity predictions in DIETClassifier.

We encourage you to try both the above recommendations. This can be done with the following config:

- name: DIETClassifier
model_confidence: linear_norm
constrain_similarities: True
...

Once the assistant is re-trained with the above configuration, users should also tune fallback confidence thresholds.

EDIT: Some post-release experiments revealed that using model_confidence=cosine is wrong as it can change the order of predicted labels. That's why this option was removed in Rasa Open Source version 2.3.4.

Rasa 2.1 to Rasa 2.2

General

TEDPolicy's transformer_size, number_of_transformer_layers, and dense_dimensions parameters have been renamed. Please update your configuration files using the following mapping:

Old Model ParameterNew Model Parameter
transformer_sizedictionary transformer_size with keys
text, action_text, label_action_text, dialogue
number_of_transformer_layersdictionary number_of_transformer_layers with keys
text, action_text, label_action_text, dialogue
dense_dimensiondictionary dense_dimension with keys
text, action_text, label_action_text, intent,
action_name, label_action_name, entities, slots,
active_loop

For example:

config.yml
policies:
- name: TEDPolicy
transformer_size:
text: 128
action_text: 128
label_action_text: 128
dialogue: 128
number_of_transformer_layers:
text: 1
action_text: 1
label_action_text: 1
dialogue: 1
dense_dimension:
text: 128
action_text: 128
label_action_text: 128
intent: 20
action_name: 20
label_action_name: 20
entities: 20
slots: 20
active_loop: 20

Deprecations

Markdown Data

Training and test data in Markdown format is now deprecated. This includes:

  • reading and writing of story files in Markdown format
  • reading and writing of NLU data in Markdown format
  • reading and writing of retrieval intent data in Markdown format

Support for Markdown data will be removed entirely in Rasa Open Source 3.0.0.

Please convert your existing Markdown data by using the commands described here.

Policies

Policies now require a **kwargs argument in their constructor and load method. Policies without **kwargs will be supported until Rasa version 3.0.0. However when using incremental training **kwargs must be included.

Other

  • Domain.random_template_for is deprecated and will be removed in Rasa Open Source 3.0.0. You can alternatively use the TemplatedNaturalLanguageGenerator.
  • Domain.action_names is deprecated and will be removed in Rasa Open Source 3.0.0. Please use Domain.action_names_or_texts instead.

Rasa 2.0 to Rasa 2.1

Deprecations

ConveRTTokenizer is now deprecated. ConveRTFeaturizer now implements its behaviour. To migrate, replace ConveRTTokenizer with any other tokenizer, for e.g.:

pipeline:
- name: WhitespaceTokenizer
- name: ConveRTFeaturizer
model_url: <Remote/Local path to model files>
...

HFTransformersNLP and LanguageModelTokenizer components are now deprecated. LanguageModelFeaturizer now implements their behaviour. To migrate, replace both the above components with any tokenizer and specify the model architecture and model weights as part of LanguageModelFeaturizer, for e.g.:

pipeline:
- name: WhitespaceTokenizer
- name: LanguageModelFeaturizer
model_name: "bert"
model_weights: "rasa/LaBSE"
...

Rasa 1.10 to Rasa 2.0

General

A lot has changed in version 2.0. Make sure you read through this guide thoroughly, to make sure all parts of your bot are updated. A lot of updates can be done automatically with inbuilt commands, others will need some manual conversion. If you have any feedback about these updates or the migration process, please post it in the forum.

Training data files

As of version 2.0, the new default training data format is yaml. Markdown is still supported, but this will be deprecated in Rasa Open Source 3.0.0.

You can convert existing NLU, Stories, and NLG (i.e. responses.md) training data files in the Markdown format to the new YAML format using following commands:

rasa data convert nlu -f yaml --data={SOURCE_DIR} --out={TARGET_DIR}
rasa data convert nlg -f yaml --data={SOURCE_DIR} --out={TARGET_DIR}
rasa data convert core -f yaml --data={SOURCE_DIR} --out={TARGET_DIR}

Converted files will have the same names as the original ones but with a _converted.yml suffix.

If you are using forms or response selectors, some additional changes will need to be made as described in their respective sections.

Policies

With the introduction of rules and the RulePolicy, the following policies are deprecated:

To migrate the policies automatically, you can run the following command:

rasa data convert config

This command will take care of updating your config.yml and domain.yml, while making backups of your existing files using the .bak suffix. It will also add a rules.yml if necessary.

Your forms will still function as normal in the old format after this update, but this command does not convert them into the new format automatically. This should be done manually, as described in the section on forms.

You can also migrate the individual policies manually, if you don't want to use the automatic conversion command.

Manually migrating from the Mapping Policy

If you previously used the Mapping Policy, you can follow the documentation on FAQs to convert your mapped intents to rules. Suppose you previously mapped an intent ask_is_bot as follows:

domain.yml
intents:
- ask_is_bot:
triggers: action_is_bot

This becomes the following rule:

rules.yml
rules:
- rule: Rule to map `ask_is_bot` intent
steps:
- intent: ask_is_bot
- action: action_is_bot

And you can safely remove any triggers: from your domain:

domain.yml
intents:
- ask_is_bot

Finally, you can replace the Mapping Policy with the Rule Policy in your model configuration:

config.yml
policies:
# Other policies
- name: RulePolicy

Manually migrating from the Fallback Policy

If you previously used the Fallback Policy, the following model configuration would translate as follows given a previous configuration like this:

config.yml
policies:
- name: "FallbackPolicy"
nlu_threshold: 0.4
core_threshold: 0.3
fallback_action_name: "action_default_fallback"
ambiguity_threshold: 0.1

The new configuration would then look like:

config.yml
policies:
# Other policies
- name: RulePolicy
core_fallback_threshold: 0.3
core_fallback_action_name: "action_default_fallback"
pipeline:
# Other components
- name: FallbackClassifier
threshold: 0.4
ambiguity_threshold: 0.1

In addition, you need to add a rule to specify which action to run in case of low NLU confidence:

rules.yml
rules:
- rule: Ask the user to rephrase whenever they send a message with low NLU confidence
steps:
- intent: nlu_fallback
- action: utter_please_rephrase

See the documentation on fallback for more information.

Manually migrating from the Two-Stage-Fallback Policy

If you previously used the Two-Stage-Fallback Policy, with a configuration like this for example:

config.yml
policies:
- name: TwoStageFallbackPolicy
nlu_threshold: 0.4
ambiguity_threshold: 0.1
core_threshold: 0.3
fallback_core_action_name: "action_default_fallback"
fallback_nlu_action_name: "action_default_fallback"
deny_suggestion_intent_name: "out_of_scope"

The new configuration would look like this:

config.yml
policies:
# Other policies
- name: RulePolicy
core_fallback_threshold: 0.3
core_fallback_action_name: "action_default_fallback"
pipeline:
# Other components
- name: FallbackClassifier
threshold: 0.4
ambiguity_threshold: 0.1

In addition you need to add a rule to activate the Two-Stage Fallback for messages with low NLU confidence.

rules.yml
rules:
- rule: Implementation of the TwoStageFallbackPolicy
steps:
# This intent is automatically triggered by the `FallbackClassifier` in the NLU
# pipeline in case the intent confidence was below the specified threshold.
- intent: nlu_fallback
# The Fallback is now implemented as a form.
- action: action_two_stage_fallback
- active_loop: action_two_stage_fallback

Note that the previous parameters fallback_nlu_action_name and deny_suggestion_intent_name are no longer configurable and have the fixed values action_default_fallback and out_of_scope.

See the fallback documentation for more information.

Forms

As of version 2.0 the logic for forms has been moved from the Rasa SDK to Rasa Open Source to simplify implementation and make it easier to write action servers in other languages.

This means that forms are no longer implemented using a FormAction, but instead defined in the domain. Any customizations around requesting slots or slot validation can be handled with a FormValidationAction.

Consider a custom form action from 1.x like this:

from typing import Text, List, Any, Dict, Union
from rasa_sdk import Tracker
from rasa_sdk.executor import CollectingDispatcher
from rasa_sdk.forms import FormAction
class RestaurantForm(FormAction):
def name(self) -> Text:
return "restaurant_form"
@staticmethod
def required_slots(tracker: Tracker) -> List[Text]:
return ["cuisine"]
def slot_mappings(self) -> Dict[Text, Union[Dict, List[Dict]]]:
return {
"cuisine": self.from_entity(entity="cuisine", not_intent="chitchat"),
}
@staticmethod
def cuisine_db() -> List[Text]:
"""Database of supported cuisines"""
return ["caribbean", "chinese", "french"]
def validate_cuisine(
self,
value: Text,
dispatcher: CollectingDispatcher,
tracker: Tracker,
domain: Dict[Text, Any],
) -> Dict[Text, Any]:
"""Validate cuisine value."""
if value.lower() in self.cuisine_db():
# validation succeeded, set the value of the "cuisine" slot to value
return {"cuisine": value}
else:
dispatcher.utter_message(template="utter_wrong_cuisine")
# validation failed, set this slot to None, meaning the
# user will be asked for the slot again
return {"cuisine": None}
def submit(
self,
dispatcher: CollectingDispatcher,
tracker: Tracker,
domain: Dict[Text, Any],
) -> List[Dict]:
"""Define what the form has to do
after all required slots are filled"""
# utter submit template
dispatcher.utter_message(template="utter_submit")
return []

Start the migration by removing the FormPolicy and adding the RulePolicy (if not there already) to your model configuration:

config.yml
policies:
# Other policies
# ...
- name: RulePolicy

Then you need to define the form, required slots and their slot mappings in the domain as described in the documentation on forms:

domain.yml
forms:
restaurant_form:
cuisine:
- type: from_entity
entity: cuisine
not_intent: chitchat

If you ran the command to convert your stories, you will have a story that handles form activation and deactivation like this:

stories.yml
stories:
- story: cuisine form
steps:
- intent: request_restaurant
- action: restaurant_form
- active_loop: restaurant_form
- active_loop: null
- action: utter_submit

This will work fine, but the best way to handle form behavior is to remove this story and instead define two separate rules for form activation and submission:

rules.yml
rules:
- rule: Activate form
steps:
- intent: request_restaurant
- action: restaurant_form
- active_loop: restaurant_form
- rule: Submit form
condition:
# Condition that form is active.
- active_loop: restaurant_form
steps:
- action: restaurant_form
- active_loop: null
# The action we want to run when the form is submitted.
- action: utter_submit

The last step is to implement a custom action to validate the form slots. Start by adding the custom action to your domain:

domain.yml
actions:
# Other actions
# ...
- validate_restaurant_form

Then add a custom action which validates the cuisine slot:

from typing import Text, List, Any, Dict, Union
from rasa_sdk import Tracker
from rasa_sdk.executor import CollectingDispatcher
from rasa_sdk import FormValidationAction
from rasa_sdk.types import DomainDict
class RestaurantFormValidator(FormValidationAction):
def name(self) -> Text:
return "validate_restaurant_form"
@staticmethod
def cuisine_db() -> List[Text]:
"""Database of supported cuisines"""
return ["caribbean", "chinese", "french"]
def validate_cuisine(
self,
slot_value: Any,
dispatcher: CollectingDispatcher,
tracker: Tracker,
domain: DomainDict,
) -> Dict[Text, Any]:
"""Validate cuisine value."""
if slot_value.lower() in self.cuisine_db():
# validation succeeded, set the value of the "cuisine" slot to value
return {"cuisine": slot_value}
else:
# validation failed, set this slot to None, meaning the
# user will be asked for the slot again
return {"cuisine": None}

You can also migrate forms from Rasa SDK to Rasa Open Source 2 iteratively. You can for example migrate one form to the Rasa Open Source 2 implementation while continue using the deprecated Rasa SDK implementation for another form. To continue to use the deprecated Rasa SDK FormActions, add a custom action with the name of your form to your domain. Note that you should complete the migration as soon as possible as the deprecated FormAction will be removed from the Rasa SDK in Rasa Open Source 3.

domain.yml
actions:
# Adding a custom action for a form will
# instruct Rasa Open Source to use the
# deprecated Rasa SDK implementation of forms.
- my_form
forms:
my_form:

See the forms documentation for more details.

Response Selectors

Response Selectors are a stable feature as of version 2.0.

The conversion command will automatically convert your responses.md file, stories and nlu training data to the new yaml format. It will also take care of adding the utter_ prefix to your responses. Additionally you will need to rename the respond_ actions in your stories files to use the utter_ prefix instead. Run the following command to apply these changes:

rasa data convert responses --data {SOURCE_DIR} --out={TARGET_DIR}

You can also apply these changes manually. For example:

stories:
- story: chitchat
steps:
- intent: chitchat
- action: respond_chitchat

becomes

stories:
- story: chitchat
steps:
- intent: chitchat
- action: utter_chitchat

and you will need to add the utter_ prefix to the response names in your responses.md as well. For example:

responses:
chitchat/ask_name:
- text: Oh yeah, I am called the retrieval bot.
chitchat/ask_weather:
- text: Oh, it does look sunny right now in Berlin.

becomes

responses:
utter_chitchat/ask_name:
- text: Oh yeah, I am called the retrieval bot.
utter_chitchat/ask_weather:
- text: Oh, it does look sunny right now in Berlin.

Finally, you should remove any actions with the respond_ prefix from the actions list in your domain.

This behavior will work fine when defined as a story, but even better when defined as a rule. You should consider transferring your retrieval stories to rules. More information on what that looks like in the chitchat and FAQs documentation.

Response Selectors are now trained on retrieval intent labels by default instead of the actual response text. For most models, this should improve training time and accuracy of the ResponseSelector.

If you want to revert to the pre-2.0 default behavior, add the use_text_as_label: true parameter to your ResponseSelector component:

pipeline:
# other components
- name: ResponseSelector
use_text_as_label: true

The output schema of ResponseSelector has changed. An example output looks like this:

{
"response_selector": {
"all_retrieval_intents": [
"faq"
],
"default": {
"response": {
"id": 1388783286124362000,
"confidence": 1,
"intent_response_key": "faq/is_legit",
"response_templates": [
{
"text": "absolutely",
"image": "https://i.imgur.com/nGF1K8f.jpg"
},
{
"text": "I think so."
}
]
"template_name": "utter_faq/is_legit"
},
"ranking": [
{
"id": 1388783286124362000,
"confidence": 1,
"intent_response_key": "faq/is_legit"
}
]
}
}
}

As a result of this, if you were previously querying for the key full_retrieval_intent as:

response_selector_output.get("default")
.get("full_retrieval_intent")

you should instead now do this:

response_selector_output.get("default")
.get("response")
.get("intent_response_key")

Unfeaturized Slots

Slots of type unfeaturized are deprecated and will be removed in version 3.0. To ignore slot values during a conversation, set the influence_conversation property of the slot to false.

The following snippet is an example of the deprecated unfeaturized slot usage:

slots:
username:
type: unfeaturized

To update this to the new format, you can specify the expected data type text and define that the slot should be ignored during the conversation.

slots:
username:
type: text
# Set `influence_conversation` to `false`
# to ignore the slot value during the conversation.
influence_conversation: false

If you don't require the slot to have a specific data type, you can use the new slot type any. This slot type is always ignored during a conversation and does not make any assumptions regarding the data type of the slot value.

slots:
username:
type: any

Please see the updated slots documentation for more information.

Conversation sessions

Conversation sessions are now enabled by default if your Domain does not contain a session configuration. Previously a missing session configuration was treated as if conversation sessions were disabled. You can explicitly disable conversation sessions using the following snippet:

domain.yml
session_config:
# A session expiration time of `0`
# disables conversation sessions
session_expiration_time: 0

Dialogue Featurization

This section is only relevant if you explicitly defined featurizers in your policy configuration.

LabelTokenizerSingleStateFeaturizer is deprecated and will be removed in the future. It should be replaced with SingleStateFeaturizer and some changes should be made to the NLU pipeline. Add a Tokenizer with the option intent_tokenization_flag: True and CountVectorsFeaturizer to the NLU pipeline.

For example:

language: en
pipeline:
- name: WhitespaceTokenizer
intent_tokenization_flag: True
- name: CountVectorsFeaturizer
# other components
policies:
# other policies
- name: TEDPolicy
featurizer:
- name: SingleStateFeaturizer

BinarySingleStateFeaturizer is deprecated and will be removed in the future. You should replace it with SingleStateFeaturizer and a NLU pipeline where intent_tokenization_flag of a Tokenizer is set to False.

For example:

language: en
pipeline:
- name: WhitespaceTokenizer
intent_tokenization_flag: False
# other components
policies:
# other policies
- name: TEDPolicy
featurizer:
- name: SingleStateFeaturizer

Deprecations

The deprecated event brokers FileProducer, KafkaProducer, PikaProducer and SQLProducer have been removed. If you used these brokers in your endpoints.yml make sure to use the renamed variants instead:

  • FileProducer became FileEventBroker
  • KafkaProducer became KafkaEventBroker
  • PikaProducer became PikaEventBroker
  • SQLProducer became SQLEventBroker

The deprecated EmbeddingIntentClassifier has been removed. If you used this component in your pipeline configuration (config.yml) you can replace it with DIETClassifier. It accepts the same configuration parameters.

The deprecated KerasPolicy has been removed. If you used this component in your policies configuration (config.yml) you can replace it with TEDPolicy. It accepts the same configuration parameters.

Rasa 1.7 to Rasa 1.8

caution

This is a release breaking backwards compatibility. It is not possible to load previously trained models. Please make sure to retrain a model before trying to use it with this improved version.

General

  • The TED Policy replaced the keras_policy as recommended machine learning policy. New projects generated with rasa init will automatically use this policy. In case you want to change your existing model configuration to use the TED Policy add this to the policies section in your config.yml and remove potentially existing KerasPolicy entries:

    policies:
    # - ... other policies
    - name: TEDPolicy
    max_history: 5
    epochs: 100

    The given snippet specifies default values for the parameters max_history and epochs. max_history is particularly important and strongly depends on your stories. Please see the docs of the TED Policy if you want to customize them.

  • All pre-defined pipeline templates are deprecated. Any templates you use will be mapped to the new configuration, but the underlying architecture is the same. Take a look at Tuning Your Model to decide on what components you should use in your configuration file.

  • The Embedding Policy was renamed to TED Policy. The functionality of the policy stayed the same. Please update your configuration files to use TEDPolicy instead of EmbeddingPolicy.

  • Most of the model options for EmbeddingPolicy, EmbeddingIntentClassifier, and ResponseSelector got renamed. Please update your configuration files using the following mapping:

    Old model optionNew model option
    hidden_layers_sizes_adictionary “hidden_layers_sizes” with key “text”
    hidden_layers_sizes_bdictionary “hidden_layers_sizes” with key “label”
    hidden_layers_sizes_pre_dialdictionary “hidden_layers_sizes” with key “dialogue”
    hidden_layers_sizes_botdictionary “hidden_layers_sizes” with key “label”
    num_transformer_layersnumber_of_transformer_layers
    num_headsnumber_of_attention_heads
    max_seq_lengthmaximum_sequence_length
    dense_dimdense_dimension
    embed_dimembedding_dimension
    num_negnumber_of_negative_examples
    mu_posmaximum_positive_similarity
    mu_negmaximum_negative_similarity
    use_max_sim_neguse_maximum_negative_similarity
    C2regularization_constant
    C_embnegative_margin_scale
    droprate_adroprate_dialogue
    droprate_bdroprate_label
    evaluate_every_num_epochsevaluate_every_number_of_epochs
    evaluate_on_num_examplesevaluate_on_number_of_examples

    Old configuration options will be mapped to the new names, and a warning will be thrown. However, these will be deprecated in a future release.

  • The Embedding Intent Classifier is now deprecated and will be replaced by DIETClassifier in the future. DIETClassfier performs intent classification as well as entity recognition. If you want to get the same model behavior as the current EmbeddingIntentClassifier, you can use the following configuration of DIETClassifier:

    pipeline:
    # - ... other components
    - name: DIETClassifier
    hidden_layers_sizes:
    text: [256, 128]
    number_of_transformer_layers: 0
    weight_sparsity: 0
    intent_classification: True
    entity_recognition: False
    use_masked_language_model: False
    BILOU_flag: False
    scale_loss: True
    use_sparse_input_dropout: False
    use_dense_input_dropout: False
    # ... any other parameters

    See DIETClassifier for more information about the new component. Specifying EmbeddingIntentClassifier in the configuration maps to the above component definition, and results in the same behaviour within the same Rasa version.

  • CRFEntityExtractor is now deprecated and will be replaced by DIETClassifier in the future. If you want to get the same model behavior as the current CRFEntityExtractor, you can use the following configuration:

    pipeline:
    # - ... other components
    - name: LexicalSyntacticFeaturizer
    features: [
    ["low", "title", "upper"],
    [
    "BOS",
    "EOS",
    "low",
    "prefix5",
    "prefix2",
    "suffix5",
    "suffix3",
    "suffix2",
    "upper",
    "title",
    "digit",
    ],
    ["low", "title", "upper"],
    ]
    - name: DIETClassifier
    intent_classification: False
    entity_recognition: True
    use_masked_language_model: False
    number_of_transformer_layers: 0
    # ... any other parameters

    CRFEntityExtractor featurizes user messages on its own, it does not depend on any featurizer. We extracted the featurization from the component into the new featurizer LexicalSyntacticFeaturizer. Thus, in order to obtain the same results as before, you need to add this featurizer to your pipeline before the DIETClassifier. Specifying CRFEntityExtractor in the configuration maps to the above component definition, the behavior is unchanged from previous versions.

  • If your pipeline contains CRFEntityExtractor and EmbeddingIntentClassifier you can substitute both components with DIETClassifier. You can use the following pipeline for that:

    pipeline:
    # - ... other components
    - name: LexicalSyntacticFeaturizer
    features: [
    ["low", "title", "upper"],
    [
    "BOS",
    "EOS",
    "low",
    "prefix5",
    "prefix2",
    "suffix5",
    "suffix3",
    "suffix2",
    "upper",
    "title",
    "digit",
    ],
    ["low", "title", "upper"],
    ]
    - name: DIETClassifier
    number_of_transformer_layers: 0
    # ... any other parameters

Rasa 1.6 to Rasa 1.7

General

  • By default, the EmbeddingIntentClassifier, EmbeddingPolicy, and ResponseSelector will now normalize the top 10 confidence results if the loss_type is "softmax" (which has been default since 1.3, see Rasa 1.2 to Rasa 1.3). This is configurable via the ranking_length configuration parameter; to turn off normalization to match the previous behavior, set ranking_length: 0.

Rasa 1.2 to Rasa 1.3

caution

This is a release breaking backwards compatibility. It is not possible to load previously trained models. Please make sure to retrain a model before trying to use it with this improved version.

General

  • Default parameters of EmbeddingIntentClassifier are changed. See the Components page for details. Architecture implementation is changed as well, so old trained models cannot be loaded. Default parameters and architecture for EmbeddingPolicy are changed. See Policies for details. It uses transformer instead of lstm. Old trained models cannot be loaded. They use inner similarity and softmax loss by default instead of cosine similarity and margin loss (can be set in config file). They use balanced batching strategy by default to counteract class imbalance problem. The meaning of evaluate_on_num_examples is changed. If it is non zero, random examples will be picked by stratified split and used as hold out validation set, so they will be excluded from training data. We suggest to set it to zero (default) if data set contains a lot of unique examples of dialogue turns. Removed label_tokenization_flag and label_split_symbol from component. Instead moved intent splitting to Tokenizer components via intent_tokenization_flag and intent_split_symbol flag.

  • Default max_history for EmbeddingPolicy is None which means it'll use the FullDialogueTrackerFeaturizer. We recommend to set max_history to some finite value in order to use MaxHistoryTrackerFeaturizer for faster training. See Featurizers for details. We recommend to increase batch_size for MaxHistoryTrackerFeaturizer (e.g. "batch_size": [32, 64])

  • Compare mode of rasa train core allows the whole core config comparison. Therefore, we changed the naming of trained models. They are named by config file name instead of policy name. Old naming style will not be read correctly when creating compare plots (rasa test core). Please remove old trained models in comparison folder and retrain. Normal core training is unaffected.

  • We updated the evaluation metric for our NER. We report the weighted precision and f1-score. So far we included no-entity in this report. However, as most of the tokens actually don't have an entity set, this will influence the weighted precision and f1-score quite a bit. From now on we exclude no-entity from the evaluation. The overall metrics now only include proper entities. You might see a drop in the performance scores when running the evaluation again.

  • / is reserved as a delimiter token to distinguish between retrieval intent and the corresponding response text identifier. Make sure you don't include / symbol in the name of your intents.

Rasa NLU 0.14.x and Rasa Core 0.13.x to Rasa 1.0

caution

This is a release breaking backwards compatibility. It is not possible to load previously trained models. Please make sure to retrain a model before trying to use it with this improved version.

General

  • The scripts in rasa.core and rasa.nlu can no longer be executed. To train, test, run, … an NLU or Core model, you should now use the command line interface rasa. The functionality is, for the most part, the same as before. Some changes in commands reflect the combined training and running of NLU and Core models, but NLU and Core can still be trained and used individually. If you attempt to run one of the old scripts in rasa.core or rasa.nlu, an error is thrown that points you to the command you should use instead. See all the new commands at Command Line Interface.

  • If you have written a custom output channel, all send_ methods subclassed from the OutputChannel class need to take an additional \*\*kwargs argument. You can use these keyword args from your custom action code or the templates in your domain file to send any extra parameters used in your channel's send methods.

  • If you were previously importing the Button or Element classes from rasa_core.dispatcher, these are now to be imported from rasa_sdk.utils.

  • Rasa NLU and Core previously used separate configuration files. These two files should be merged into a single file either named config.yml, or passed via the --config parameter.

Script parameters

  • All script parameter names have been unified to follow the same schema. Any underscores (_) in arguments have been replaced with dashes (-). For example: --max_history has been changed to --max-history. You can see all of the script parameters in the --help output of the commands in the Command Line Interface.

  • The --num_threads parameter was removed from the run command. The server will always run single-threaded, but will now run asynchronously. If you want to make use of multiple processes, feel free to check out the Sanic server documentation.

  • To avoid conflicts in script parameter names, connectors in the run command now need to be specified with --connector, as -c is no longer supported. The maximum history in the rasa visualize command needs to be defined with --max-history. Output paths and log files cannot be specified with -o anymore; --out and --log-file should be used. NLU data has been standarized to be --nlu and the name of any kind of data files or directory to be --data.

HTTP API

  • There are numerous HTTP API endpoint changes which can be found here.