Version: Latest

Rasa Pro Change Log

All notable changes to Rasa Pro will be documented in this page. This product adheres to Semantic Versioning starting with version 3.3 (initial version).

Rasa Pro consists of two deployable artifacts: Rasa Pro and Rasa Pro Services. You can read the change log for both artifacts below.

[3.11.1] - 2024-12-13

Rasa Pro 3.11.1 (2024-12-13)

Bugfixes

Add the possibility to pass a transform callable parameter when writing yaml. This allows passing a custom function to transform endpoints before uploading to Studio. This was required to fix the issue where yaml wraps in quotes any string that doesn't start with an alphabetic character such as unexpanded environment variables in the endpoints yml file.
Pass flow human-readable name instead of flow id when the cancel pattern stack frame is pushed during flow policy validation checks of collect steps.
Fixed the accuracy calculation to prevent 100% assertion reporting when a test case fails before any assertions are reached.
Fixed regression on training time for projects with a lot of YAML files.

[3.11.0] - 2024-12-11

Rasa Pro 3.11.0 (2024-12-11)

Deprecations and Removals

Removed UnexpecTEDIntentPolicy from the default config.yml. It is an experimental policy and not suitable for default configuration
The reset_after_flow_ends property of collect steps is now deprecated and will be removed in Rasa Pro 4.0.0. Please use the persisted_slots property at the flow level instead.

Features

Added Twilio Media Streams channel which can be configured to use arbitrary Text-To-Speech and Speech-To-Text services. Added Voice Stream Channel Interface which makes it easier to add voice channels that directly integrate with audio streams. Added support for Deepgram Speech-To-Text and Azure Text-To-Speech in Voice Stream Channels.
Added default action action_hangup it can be used to hang up a phone call from a flow. Added SessionEnded event and SessionEndCommand command Updated Audiocodes, Jambonz and Twilio Voice channels to send /session_end if the phone call is disconnected by user.
Added support for Cartesia Text-To-Speech in Voice Stream Channels.
Implement Rasa Pro native model service that takes care of training and running an assistant model in Studio. To find out more about this service, read more in the Studio documentation.
Added a feature to be able to use voice to interact with the bot in the inspector.
Multi-LLM Routing:
1. Decoupled LLM Configuration from Components
  - The previous integration of LLMs within CALM is closely tied to the components where they are used. However, this is no longer necessary, as we no longer perform training within the individual components that interact with external LLM endpoints.
  - As a result, LLM and embedding client configurations have been moved to endpoints.yml. To define LLM configurations in endpoints.yml, use the model_groups as shown below:
    model_groups:
    - id: gpt-4-direct
    models:
    - provider: openai
    model: gpt-4
    timeout: 7
    temperature: 0.0
    - id: text-embedding-3-small-direct
    models:
    - provider: openai
    model: text-embedding-3-small
  - These model_groups can then be referenced in config.yml as follows:
    pipeline:
    ...
    - name: SingleStepLLMCommandGenerator
    llm:
    model_group: gpt-4-direct
    flow_retrieval:
    embeddings:
    model_group: text-embedding-3-smal-direct
    ...
2. Support for Multiple Subscription Deployments
  - Allows customers to use deployments from different subscriptions for the same provider.
  - Resolved the limitation of API key configuration being tied exclusively to a single environment variable.
  Example configuration in endpoints.yml for Azure deployments:
  model_groups:
  - id: azure-gpt-model-eu
  models:
  - provider: azure
  deployment: azure-eu-deployment
  api_base: https://api.azure-europe.example.com
  api_version: 2024-08-01-preview
  api_key: ${AZURE_API_KEY_EU}
  timeout: 7
  temperature: 0.0
  ...
  - id: azure-gpt-model-us
  models:
  - provider: azure
  deployment: azure-us-deployment
  api_base: https://api.azure-us.example.com
  api_version: 2024-08-01-preview
  api_key: ${AZURE_API_KEY_US}
  timeout: 7
  temperature: 0.0
  ...
  ...
3. Seamless Model Configuration Across Environments Without Retraining
  - Added support for using different model configurations in different environments, such as dev, staging, and prod, without requiring the bot to be retrained for each environment.
  - Extended the ${...} syntax to deployment, api_base, and api_version in model_groups, allowing these values to change dynamically based on the environment.
  model_groups:
  - id: azure-gpt-4
  models:
  - provider: azure
  deployment: ${AZURE_DEPLOYMENT_GPT4}
  api_base: ${AZURE_API_BASE_GPT4}
  api_key: ${AZURE_API_KEY_GPT4}
  ...
  - id: azure-text-embeddings-3-small
  models:
  - provider: azure
  deployment: ${AZURE_DEPLOYMENT_EMBEDDINGS_3_SMALL}
  api_base: ${AZURE_API_BASE_EMBEDDINGS_3_SMALL}
  api_key: ${AZURE_API_EMBEDDINGS_3_SMALL}
  ...
4. Supporting Multiple Deployments for Load Balancing
  - Enabled targeting of multiple LLM deployments for a single Rasa component.
  - Implemented the routing feature that supports load balancing to handle rate limits and improve scalability. When multiple models are defined within a model group, you can specify the router key with a routing_strategy to control how requests are distributed among the models.
  Example configuration in endpoints.yml for Azure deployments with load balancing:
  model_groups:
  - id: azure-gpt-models
  models:
  - provider: azure
  deployment: azure-eu-deployment
  api_base: https://api.azure-europe.example.com
  api_version: 2024-08-01-preview
  api_key: ${AZURE_API_KEY_EU}
  timeout: 7
  temperature: 0.0
  ...
  - provider: azure
  deployment: azure-us-deployment
  api_base: https://api.azure-us.example.com
  api_version: 2024-08-01-preview
  api_key: ${AZURE_API_KEY_US}
  timeout: 7
  temperature: 0.0
  ...
  router:
  routing_strategy: least-busy
  ...
  Example of usage in config.yml:
  pipeline:
  ...
  - name: SingleStepLLMCommandGenerator
  llm:
  model_group: azure-gpt-models
  ...
5. Backward Compatibility
  - Existing configurations that couple LLMs to specific Rasa components remain unaffected by this change.
  - However, this configuration method is now deprecated and scheduled for removal in version 4.0.0.
Added support for Azure Speech-To-Text in Voice Stream Channels.
Added UserSilenceCommand and pattern_user_silence which is triggered by Voice Stream channels when the user is silent for more than a silence timeout. These values are configurable with the newly added slots silence_timeout and consecutive_silence_timeouts. Silence Monitoring is disabled by default and can be enabled using the configuration monitor_silence: true in the relevant Voice Stream Channel configuration.
The inspector is not its own input / output channel anymore. Rather, it can be attached to other channels. This way, it isn't limited to conversations going through the socketio channel anymore, but can be used with other text channels or voice channels.
You can attach it to any channel(s) configured in your credentials.yml by adding a flag to rasa run: rasa run --inspect.
In addition to that, the conenience cli command rasa inspect is retained, which starts the inspector with the socketio channel as usual.

Improvements

In Audiocodes channel, /vaig_event_start is replaced by /session_start. This intent marks the beginning of conversation and it is sent when the phone call is connected.
Introduced the environment variable MAX_NUMBER_OF_PREDICTIONS_CALM to configure the CALM-specific limit for the number of predictions. This variable defaults to 1000, providing a higher prediction limit compared to the default value of 10 for nlu-based assistants.
In Audiocodes and Twilio Voice channel connector, the call metadata received from the providers can be accessed in the slot session_started_metadata. The call metadata parameter names have been standardised with CallParameters dataclass Twilio Voice Channel Connector sends /session_start intent at the beginning of conversation and the channel parameter initial_prompt has been removed
Enable configurability of Vault secret manager's mount point property in the endpoints yaml file or as an environment variable.
In Twilio Media Streams channel connector, call metadata is availble in session_start_metadata slot. It also supports default action action_hangup
Catch API connection errors, and validate the correctness of the values present in model configuration at model training time by making a test API request. This feature is enabled by default and can be disabled by setting the environment variable LLM_API_HEALTH_CHECK to False.
Socketio channel connector now sends the websocket messages tracker_state and rasa_events with each bot response. tracker_state contains the tracker store state at that point in conversation and includes slots, events, stack, latest message and latest action. rasa_events contains a list of new events that have happened since the last message.
Speech-To-Text and Text-To-Speech Services can be configured for Voice Stream Channel Connectors Added tests for voice components and redefined code structure
Add support for Python 3.11
Removed JSON response validation except when HTTP protocol and E2E Stub is used for Custom Action execution.
Optimized JSON response validation by initializing the Draft202012Validator once and caching it.
Add an optional property persisted_slots at the flow level. This property configures whether slots collected or set across any of the flow steps should be persisted after the flow ends. This property expects a list of slot names.
Added support for custom Automatic Speech Recognition (ASR) or Text To Speech (TTS) providers to a Rasa Assistant. This allows developers to bring their own speech providers to Rasa by subclassing classes ASREngine and TTSEngine
If flow retrieval is disabled, a warning is raised only if the number of user flows exceed 20.
Added validation to the TestCase class to issue a warning when duplicate user messages lack metadata or have incorrect metadata. This enhancement provides clear guidance to users on the issue and how to resolve it.
Fixed global should-hangup variable in Voice Stream Channels by moving to a context variable CallState that stores the session variables
Run Rasa Pro data validation before uploading to Studio. This is to avoid uploading invalid assistant data that would raise errors during Rasa Pro model training in Studio.
Added vector_name to Qdrant's configuration to enable customization of the vector field name for storing embeddings.
Enhanced YamlValidationException error messages to include the line number and a relevant YAML snippet showing where the validation error occurred. Line numbers start from 1 (1-based indexing).
The error-handling behavior has been modified so that only one validation error is displayed. This exception is raised when the YAML content does not comply with the defined YAML schema.
Added a new assertion type bot_did_not_utter to allow testing that the bot does not utter specific messages or include certain buttons during conversations.
Ensure that the model service fails properly if the minimum disk space requirement is not met.
Do not expand environment variables when reading yaml files during rasa studio upload execution.
Stream model files to Studio rather than providing full files. Provide a HEAD endpoint for Studio to check if a model is available and what its size is. Add an environment variable to set the port of the model service. This makes the development with Studio easier, previously the port was hard coded making it harder to use a separately deployed model service now that Studio includes that in its development deployment.
Add flag --skip-yaml-validation to skip YAML validation during Rasa run. User can use it to skip domain YAML validation during Rasa run. Do not instantiate multiple instances of TrainingDataImporter class for validation and training.
Introduced a summarize_history flag for the contextual response rephraser, defaulting to True. When set to False, the conversation transcript instead of the summary is included in the prompt of the contextual response rephraser. This saves a separate summarization call to an LLM. The number of conversation turns to be used when summarize_history is set to False can be set via max_historical_turns. By default this value is set to 5.
Example:
nlg:
- type: rephrase
summarize_history: False
max_historical_turns: 5

Bugfixes

Fix OpenAI LLM client ignoring API base and API version arguments if set.
Fix AttributeError with the instrumentation of the run method of the CustomActionExecutor class.
Throw DuplicatedFlowIdException during rasa data validate and rasa train if there are duplicate flows defined.
Replace pickle and joblib with safer alternatives, e.g. json, safetensors, and skops, for serializing components.
Note: This is a model breaking change. Please retrain your model.
If you have a custom component that inherits from one of the components listed below and modified the persist or load method, make sure to update your code. Please contact us in case you encounter any problems.
Affected components:
- CountVectorFeaturizer
- LexicalSyntacticFeaturizer
- LogisticRegressionClassifier
- SklearnIntentClassifier
- DIETClassifier
- CRFEntityExtractor
- TrackerFeaturizer
- TEDPolicy
- UnexpectedIntentTEDPolicy
Avoid filling slots that have ask_before_filling = True and utilize a from_text slot mapping during other steps in the flow. Ensure that the NLUCommandAdapter only fills these types of slots when the flow reaches the designated collection step.
Check for the metadata's step_id and active_flow keys when adding the ActionExecuted event to the flows paths stack.
Fixed a bug on Windows where flow files with names starting with 'u' would fail to load due to improper path escaping in YAML content processing
Fixes OpenAIException - AsyncClient.init() got an unexpected keyword argument 'proxies'
Fix retrieval of model file stored in the cloud storage by the model service. This change consisted in uploading only the model file instead of the full model path during training when --remote-storage CLI flag is used.
Fix issue in e2e testing when customising action_session_start would lead to AttributeError, because the output_channel was not set. This is now fixed by setting the output_channel to CollectingOutputChannel().

Miscellaneous internal changes

Miscellaneous internal changes.

[3.10.14] - 2024-12-04

Rasa Pro 3.10.14 (2024-12-04)

Bugfixes

Avoid filling slots that have ask_before_filling = True and utilize a from_text slot mapping during other steps in the flow. Ensure that the NLUCommandAdapter only fills these types of slots when the flow reaches the designated collection step.
Fixes OpenAIException - AsyncClient.init() got an unexpected keyword argument 'proxies'
Fix validation for LLM/Embedding clients when the api_base is configured in the config itself but not as an environment variable.

[3.10.13] - 2024-11-29

Rasa Pro 3.10.13 (2024-11-29)

Bugfixes

Implement eq and hash functions for ChangeFlowCommand to fix error=unhashable type: 'ChangeFlowCommand' error in MultiStepCommandGenerator.
Fixed an issue on Windows where flow files with names starting with 'u' would fail to load due to improper path escaping in YAML content processing
Store the value of the --disable-verify CLI flag in the disable_verify attribute of the StudioConfig object, so it can be reused across other studio commands.

[3.10.12] - 2024-11-25

Rasa Pro 3.10.12 (2024-11-25)

Bugfixes

Replace pickle and joblib with safer alternatives, e.g. json, safetensors, and skops, for serializing components.
Note: This is a model breaking change. Please retrain your model.
If you have a custom component that inherits from one of the components listed below and modified the persist or load method, make sure to update your code. Please contact us in case you encounter any problems.
Affected components:
- CountVectorFeaturizer
- LexicalSyntacticFeaturizer
- LogisticRegressionClassifier
- SklearnIntentClassifier
- DIETClassifier
- CRFEntityExtractor
- TrackerFeaturizer
- TEDPolicy
- UnexpectedIntentTEDPolicy

[3.10.11] - 2024-11-20

Rasa Pro 3.10.11 (2024-11-20)

Bugfixes

Fix parsing of commands in case the LLM response surrounds flow names, slot names, or slot values with single or double quotes.

Miscellaneous internal changes

Miscellaneous internal changes.

[3.10.10] - 2024-11-14

Rasa Pro 3.10.10 (2024-11-14)

Bugfixes

Check for the metadata's step_id and active_flow keys when adding the ActionExecuted event to the flows paths stack.

[3.10.9] - 2024-11-13

Rasa Pro 3.10.9 (2024-11-13)

Bugfixes

Introduced the environment variable MAX_NUMBER_OF_PREDICTIONS_CALM to configure the CALM-specific limit for the number of predictions. This variable defaults to 1000, providing a higher prediction limit compared to the default value of 10 for nlu-based assistants.
Filter out comments from e2e test input files when writing e2e results to file.
Specified UTF-8 encoding to correctly read test cases on Windows.

[3.10.8] - 2024-10-24

Rasa Pro 3.10.8 (2024-10-24)

Bugfixes

The user message "/restart" is now restarting the session again after adding a proper implementation (stack frame and command) for pattern_restart.
Only infer and set the provider to azure for our LLM clients in case NO provider is specified, but the deployment key is set.
Fix OPENAI_API_KEY authentication error when using self-hosted provider.

Miscellaneous internal changes

Miscellaneous internal changes.

[3.10.7] - 2024-10-17

Rasa Pro 3.10.7 (2024-10-17)

Improvements

Change default response of utter_free_chitchat_response from "placeholder_this_utterance_needs_the_rephraser" to "Sorry, I'm not able to answer that right now.".

Bugfixes

Disallow using the command payload syntax to set slots not filled by any of the active or startable flow(s) collect steps.
Add flow name to error message validator.verify_flows_steps_against_domain.collect_step.
Update e2e test results output files on each test run so that, for example, when all tests pass on subsequent runs after failing previously, the failed results output file is emptied.
Disable strict SSL verification to the Rasa Studio authentication server via the --disable-verify or -x CLI argument added to the rasa studio config command.
Upgrade zipp dependency version to fix a security vulnerability: CVE-2024-5569.

[3.10.6] - 2024-10-04

Rasa Pro 3.10.6 (2024-10-04)

Bugfixes

Fix cleanup of SetSlot commands issued by the LLM-based command generator for slots that define a slot mapping other than the from_llm slot mapping. The command processor now correctly removes the SetSlot command in these scenarios and instead adds a CannotHandleCommand.
Fix UnicodeDecodeError while reading Windows path from yaml files.
Fix model loading from remote storage by correcting the handling of remote storage enum during the creation of the persistor object.

Miscellaneous internal changes

Miscellaneous internal changes.

[3.10.5] - 2024-10-01

Rasa Pro 3.10.5 (2024-10-01)

Bugfixes

Fix the case where IntentlessPolicy is triggered while no e2e stories were written to guide it. In this situation a CannotHandleCommand will be issued.
Update litellm to version 1.45.0 to fix security vulnerability (CVE-2024-6587). Update gitpython to version 3.1.41 to fix security vulnerability (CVE-2024-22190). Update certifi to version 2024.07.04 to fix security vulnerability (CVE-2024-39689).
Prevent invalid domain with incorrectly defined intent from throwing stack trace. Throw InvalidDomain exception and send message to the user instead. The message looks like this:
Detected invalid intent definition: {'intent': 'ask_help'}.
Please make sure all intent definitions are valid.
Support text completions endpoint when using self hosted models.
The use_chat_completions_endpoint parameter is now supported when using self-hosted models. This parameter is used to enable the use of the chat completions endpoint when using a self-hosted model. This parameter is set to True by default. To use the text completions endpoint, set use_chat_completions_endpoint to False in the llm section of the component.
Usage:
llm:
provider: self-hosted
model: meta-llama/Meta-Llama-3-8B
api_base: "https://my-endpoint/v1"
use_chat_completions_endpoint: false
Fixes an issue where the CountVectorsFeaturizer and LogisticRegressionClassifier would throw error during inference when no NLU training data is provided.
Added tracing explicitly to GRPCCustomActionExecutor.run in order to pass the tracing context to the action server.

[3.10.4] - 2024-09-25

Rasa Pro 3.10.4 (2024-09-25)

Bugfixes

Fix failing validation of categorical slots when slot values contain Apostrophe.

[3.10.3] - 2024-09-20

Rasa Pro 3.10.3 (2024-09-20)

No significant changes.

[3.10.2] - 2024-09-19

Rasa Pro 3.10.2 (2024-09-19)

Deprecations and Removals

Dropped support for Python 3.8 ahead of Python 3.8 End of Life in October 2024. In Rasa Pro versions 3.10.0, 3.9.11 and 3.8.13, we needed to pin the TensorFlow library version to 2.13.0rc1 in order to remove critical vulnerabilities; this resulted in poor user experience when installing these versions of Rasa Pro with uv pip. Removing support for Python 3.8 will make it possible to upgrade to a stabler version of TensorFlow.

Improvements

Update Keras and Tensorflow to version 2.14. This will eliminate the need to use the --prerelease allow flag when installing Rasa Pro using uv pip tool.

Bugfixes

Revert the old behavior when loading trained model by supplying a path to the model on the remote storage by using the model path (-m) argument when REMOTE_STORAGE_PATH environment variable is not set. Resulting path on the remote storage will be the same as the model path (-m) argument.
Additionally, entire model path (-m) argument wil be used when trained model is being uploaded to the remote storage with REMOTE_STORAGE_PATH environment variable not set. Resulting path on the remote storage will be the same as the model path (-m) argument.
If REMOTE_STORAGE_PATH environment variable is set, only the file name part of the model path (-m) argument is used in both loading and storage from/to the remote storage. Resulting path on the remote storage will be: REMOTE_STORAGE_PATH + file name part of the model path (-m) argument.
Fixed UnexpecTEDIntentlessPolicy training errors that resulted from a change to batching behavior. Changed the batching behavior back to the original for all components. Made the changed batching behavior accessible in DietClassifier using drop_small_last_batch: True.

Miscellaneous internal changes

Miscellaneous internal changes.

[3.10.1] - 2024-09-11

Rasa Pro 3.10.1 (2024-09-11)

Bugfixes

Fix OpenAI LLM client ignoring API base and API version arguments if set.
Fix FileNotFound error when running rasa studio commands and no pre-existing local assistant project exists.
Fixed telemetry collection for the components Rephraser, LLM Intent Classifier, Intentless Policy and Enterprise Search Policy to ensure that the telemetry data is only collected when it is enabled
Update the default config for E2E test conversion to use the provider key instead of api_type.
Fix inconsistent recording of telemetry events for llm-based command generators.
Throw deprecation warning when REQUESTS_CA_BUNDLE env var is used.

[3.10.0] - 2024-09-04

Rasa Pro 3.10.0 (2024-09-04)

Deprecations and Removals

Remove experimental LLMIntentClassifier. Use Rasa CALM instead.

Features

Implement the shell output of accuracy rate by assertion type as a table when running end-to-end testing with assertions.
Implement E2E testing assertions that measure metrics such as grounded-ness and answer relevance of generative responses issued by either Enterprise Search or the Contextual Response Rephraser.
You must specify a threshold which must be reached for the generative evaluation assertion to pass. In addition, you can also specify ground_truth if you prefer providing this in the E2E test rather than relying on the retrieved context from the vector store (in the case of Enterprise Search) or from the domain (in the case of Contextual Response Rephraser) that is stored in the bot utterance event metadata. For rephrased answers, you must specify utter_name to run the assertion.
These assertions can be specified for user steps only and cannot be used alongside the former E2E test format. You can learn more about this new feature in the documentation sections for grounded and relevant assertion types.
To enable this feature, please set the environment variable RASA_PRO_BETA_E2E_ASSERTIONS to true.
export RASA_PRO_BETA_E2E_ASSERTIONS=true
You can now produce a coverage report of your e2e tests via the following command:
rasa test e2e <e2e-test-folder> --coverage-report [--coverage-output-path <output-folder>]
The coverage report contains the number of steps and the number of tested steps per flow. Untested steps are referenced by line numbers.
Flow Name Coverage Num Steps Missing Steps Line Numbers
flow_1 0.00% 1 1 [10-10]
flow_2 100.00% 4 0 []
Total 80.00% 5 1
Additionally, we also create a histogram of command coverage showing how many and what commands are produced in your e2e tests.
To enable this feature, please set the environment variable RASA_PRO_BETA_FINETUNING_RECIPE to true.
export RASA_PRO_BETA_FINETUNING_RECIPE=true
More information can be found on the documentation of the feature.
Create a self-hosted LLM client compatible with OpenAI format. Users can connect to their own self-hosted LLM server that is compatible with OpenAI format.
Sample basic usage:
llm:
provider: self-hosted
model: <deployment_name>
api_base: <deployment_url>
api_type: openai [Optional]
Add a new CLI command rasa llm finetune prepare-data to create a dataset from e2e tests that can be used to fine-tune a base model for the task of command generation.
To enable this feature, please set the environment variable RASA_PRO_BETA_FINETUNING_RECIPE to true.
export RASA_PRO_BETA_FINETUNING_RECIPE=true
It is now allowed to link to pattern_human_handoff from any pattern and user flow.
Allow links from all patterns to user flows except for pattern_internal_error.
- LiteLLM Integration & Reduced LangChain Reliance:
  - Introduced LLMClient and EmbeddingClient protocols for standardized client interfaces.
  - Created lightweight client wrappers for LiteLLM to streamline model instantiation, management, and inference.
  - Updated llm_factory and embedder_factory to utilize these LiteLLM client wrappers.
  - Added dedicated clients for Azure OpenAI and OpenAI to support both LLMs and embedding models.
  - Added a HuggingFace client to compute embeddings using locally stored transformer models via the sentence-transformers package.
- LangChain Update: Upgraded to the latest version (0.2.x) for improved compatibility and features. To understand the implications on your assistant, please refer to the feature documentation and the migration guide.
Implement as part of E2E testing a new type of evaluation specifically designed to increase confidence in CALM. This evaluation runs assertions on the assistant's actual events and generative responses. New assertions include the ability to check for the presence of specific events, such as:
- flow started, flow completed or flow cancelled events
- whether pattern_clarification was triggered for specific flows
- whether buttons rendered well as part of the bot uttered event
- whether slots were set correctly or not
- whether the bot text response matches a provided regex pattern
- whether the bot response matches a provided domain response name
These assertions can be specified for user steps only and cannot be used alongside the former E2E test format. You can learn more about this new feature in the documentation.
To enable this feature, please set the environment variable RASA_PRO_BETA_E2E_ASSERTIONS to true.
export RASA_PRO_BETA_E2E_ASSERTIONS=true
Configure LLM-as-Judge settings in the llm_as_judge section of the conftest.yml file. These settings will be used to evaluate the groundedness and relevance of generated bot responses. The conftest.yml is discoverable as long as it is in the root directory of the assistant project, at the same level as the config.yml file.
If the conftest.yml file is not present in the root directory, the default LLM judge settings will be used.
Implement automatic E2E test case conversion from sample conversation data.
This feature includes:
- A CLI command to convert sample conversation data (CSV, XLSX) into executable E2E test cases.
- Conversion of sample data using an LLM to generate YAML formatted test cases.
- Export of generated test cases into a specified YAML file.
Usage:
rasa data convert e2e <path>

To enable this feature, please set the environment variable RASA_PRO_BETA_E2E_CONVERSION to true.

export RASA_PRO_BETA_E2E_CONVERSION=true

For more details, please refer to this documentation page.

Improvements

Implemented custom action stubbing for E2E test cases. To define custom action stubs, add stub_custom_actions to the test case file.
Stubs can be defined in two ways:
- Test file level: Define each action by its name (action_name).
- Test case level: Define the stub using the test case ID as a prefix (test_case_id::action_name).
To learn more about this feature, please refer to the documentation.
To enable this feature, set the environment variable RASA_PRO_BETA_STUB_CUSTOM_ACTION to true:
export RASA_PRO_BETA_STUB_CUSTOM_ACTION=true
Add max_messages_in_query parameter to Enterprise Search Policy, it allows controlling the number of past messages that are used in the search query for retrieval
Configure LLM E2E test converter settings in the llm_e2e_test_conversion section of the conftest.yml file.
These settings will be used to configure the LLM used to convert sample conversation data into E2E test cases.
The conftest.yml is discoverable as long as it is in the root directory of the tests output path.
If the conftest.yml file is not present in the root directory, the default LLM settings will be used.
Add the datetime of Rasa Pro license expiry to rasa --version command Add /license API endpoint that also returns the same information
Suppress LiteLLM info and debug log messages in the console.
Cache llm_factory and embedder_factory methods to avoid client instantiation and validation for every user utterance.
Added E2E Test Conversion Completed telemetry event with file type and test case count properties.
Separate writing of failed and passed e2e test results to distinct file paths.
Implement support for evaluating IntentlessPolicy responses with generative response assertions.
Use direct custom action execution in tutorial and CALM templates. Skip action server health check in e2e testing if direct custom action execution is configured.
Modified the type of flows which are included into the import CLI (previously only user flows were enabled, now patterns are included). Use case: This is needed for Studio 1.7, since that release is enabling modification and management of patterns inside Studio, and needs the ability to import patterns from yaml files.
Improve events and responses sub-schemas used by the stub_custom_actions sub-schema of end-to-end testing. The events sub-schema only allows the usage of events which are supported by the rasa-sdk. These are documented in the action server API documentation.
Change default model of conversation rephraser to 'gpt-4o-mini'.
Add file_path to Flow so that we can show the full name, e.g. path/to/flow.py::flow name in the e2e test coverage report.
Introduced remote storage to upload trained model to persistors(AWS, GCP, Azure)
Add ability to download training data from remote storage(gcs, aws, azure)
Allow saving models to and retrieving from sub folders in cloud storage.
Introduced DirectCustomActionExecutor for executing custom actions directly through the assistant.
Introduced actions_module variable under action_endpoint in endpoints.yml to explicitly specify the path to custom actions module.
If actions_module is set, custom actions will be executed directly through the assistant.
Add validation for the values against which categorical and boolean slots are checked in the if conditional steps. An error will be thrown when a slot is compared to an invalid/non-existent value for boolean and categorical slots.
Add user query and retrieved document results to the metadata of action_send_text predicted by EnterpriseSearchPolicy. In addition, add domain ground truth responses to the BotUttered event metadata when rephrasing is enabled. These changes were required to allow evaluations of generative responses against the ground truth stored in the metadata of BotUttered events.

Bugfixes

Fix problem with custom action invocation when model is loaded from remote storage.
Ensure certificates for openai based clients.
Mark the first slot event as seen when the user turn in a E2E test case contains multiple slot events for the same slot. This fixes the issue when the assertion_order_enabled is set to true and the user step in a test case contained multiple slot_was_set assertions for the same slot, the last slot event was marked as seen when the first assertion was running. This caused the test to fail for subsequent slot_was_set assertions for the same slot with error Slot <slot_name> was not set.
Validate the LLM configuration during training for the following components:
- Contextual Response Rephraser
- Enterprise Search Policy
- Intentless Policy
- LLM Based Command Generator
- LLM Based Router
Additionally, update the get_provider_from_config method to retrieve the provider using both the model and model_name configuration parameters.
Fixes throwing the deprecation warning if the setting for Azure OpenAI Embedding Client was not set through the deprecated environment variable.
Fix execution of stub custom actions when they contain test case name and the separator in its provided stub name. Test runner will now correctly execute the correct stub implementation for the same custom action dependent on the test name.
Add validation to conversation rephraser.
Ensure YAML files with datetime-formatted strings are read as plain strings instead of being converted to datetime objects.
Deprecate 'request_timeout' for OpenAI and Azure OpenAI clients in favor of 'timeout'
Forbid stream and n parameters for clients. Having these parameters within llm and embeddings configuration will result in error.
Raise deprecation warning if api_type is set to huggingface instead of huggingface_local for HuggingFace local embeddings.
Fix resolving aliases for deprecated keys when instantiating LLM and embedding clients.
Fix detection of conftest file which contained custom LLM judge configuration.
Fix issue with Rasa Pro Studio download command exporting default flows which had not been customized by the Studio user. Rasa Pro Studio download command only exports user defined flows, customized patterns and user defined domain locally from the Studio instance.
Similarly, fix issue with Rasa Pro Studio upload command importing default flows which had not been customized to Studio. Rasa Pro Studio upload command only imports user defined flows, customized patterns and user defined domain to the Studio instance.
Disable auto-inferring provider from the config. Ensure the provider is explicitly read from the provider key.
Fix writing e2e test cases to disk. slot_was_set and slot_was_not_set are now written down correctly.
The rephraser of the rasa llm finetune data-prepare command now compares the original user message and the user message returned in the LLM output case-insensitive.
[rasa llm finetune prepare-data] Do not rephrase user messages that come from a button payload.
Separate commands in the expected LLM output by newlines.
Fix TypeError in PatternClarificationContainsAssertion hash function by converting sets to lists for successful JSON serialization.
Fix validation in case a link to pattern_human_handoff is used.
[rasa llm finetune prepare-data] Skip paraphrasing module in case num-rephrases is set to 0.
Update the handling of incorrect use of slash syntax. Messages with undefined intents do not automatically trigger pattern_cannot_handle; instead, they are sanitized (prepended slash(es) are removed) and passed through the graph.
Allow suitable patterns to be properly started using nlu triggers
Fix API connection error for bedrock embedding endpoint.

Miscellaneous internal changes

Miscellaneous internal changes.

[3.9.17] - 2024-12-05

Rasa Pro 3.9.17 (2024-12-05)

Bugfixes

Implement eq and hash functions for ChangeFlowCommand to fix error=unhashable type: 'ChangeFlowCommand' error in MultiStepCommandGenerator.

[3.9.16] - 2024-11-26

Rasa Pro 3.9.16 (2024-11-26)

Bugfixes

Replace pickle and joblib with safer alternatives, e.g. json, safetensors, and skops, for serializing components.
Note: This is a model breaking change. Please retrain your model.
If you have a custom component that inherits from one of the components listed below and modified the persist or load method, make sure to update your code. Please contact us in case you encounter any problems.
Affected components:
- CountVectorFeaturizer
- LexicalSyntacticFeaturizer
- LogisticRegressionClassifier
- SklearnIntentClassifier
- DIETClassifier
- CRFEntityExtractor
- TrackerFeaturizer
- TEDPolicy
- UnexpectedIntentTEDPolicy

[3.9.15] - 2024-10-18

Rasa Pro 3.9.15 (2024-10-18)

Improvements

Change default response of utter_free_chitchat_response from "placeholder_this_utterance_needs_the_rephraser" to "Sorry, I'm not able to answer that right now.".

Bugfixes

Fix cleanup of SetSlot commands issued by the LLM-based command generator for slots that define a slot mapping other than the from_llm slot mapping. The command processor now correctly removes the SetSlot command in these scenarios and instead adds a CannotHandleCommand.
Disallow using the command payload syntax to set slots not filled by any of the active or startable flow(s) collect steps.

Miscellaneous internal changes

Miscellaneous internal changes.

[3.9.14] - 2024-10-02

Rasa Pro 3.9.14 (2024-10-02)

No significant changes.

[3.9.13] - 2024-10-01

Rasa Pro 3.9.13 (2024-10-01)

Bugfixes

Fix inconsistent recording of telemetry events for llm-based command generators.
Added tracing explicitly to GRPCCustomActionExecutor.run in order to pass the tracing context to the action server.
Fixes an issue where the CountVectorsFeaturizer and LogisticRegressionClassifier would throw error during inference when no NLU training data is provided.

[3.9.12] - 2024-09-20

Rasa Pro 3.9.12 (2024-09-20)

Deprecations and Removals

Dropped support for Python 3.8 ahead of Python 3.8 End of Life in October 2024. In Rasa Pro versions 3.10.0, 3.9.11 and 3.8.13, we needed to pin the TensorFlow library version to 2.13.0rc1 in order to remove critical vulnerabilities; this resulted in poor user experience when installing these versions of Rasa Pro with uv pip. Removing support for Python 3.8 will make it possible to upgrade to a stabler version of TensorFlow.

Improvements

Update Keras and Tensorflow to version 2.14. This will eliminate the need to use the --prerelease allow flag when installing Rasa Pro using uv pip tool.

Bugfixes

Fix AttributeError with the instrumentation of the run method of the CustomActionExecutor class.
Fixed UnexpecTEDIntentlessPolicy training errors that resulted from a change to batching behavior. Changed the batching behavior back to the original for all components. Made the changed batching behavior accessible in DietClassifier using drop_small_last_batch: True.

Miscellaneous internal changes

Miscellaneous internal changes.

[3.9.11] - 2024-09-13

Rasa Pro 3.9.11 (2024-09-13)

Bugfixes

Update Keras to 2.13.1 and Tensorflow to 2.13.0rc0 to fix critical vulnerability (CVE-2024-3660).

[3.9.10] - 2024-09-12

Rasa Pro 3.9.10 (2024-09-12)

Bugfixes

Fix FileNotFound error when running rasa studio commands and no pre-existing local assistant project exists.
Fixed telemetry collection for the components Rephraser, LLM Intent Classifier, Intentless Policy and Enterprise Search Policy to ensure that the telemetry data is only collected when it is enabled

[3.9.9] - 2024-08-23

Rasa Pro 3.9.9 (2024-08-23)

Bugfixes

Updated behaviour of policies in coexistence:
- CALM policies run in case the routing slot is set to True (routing to CALM).
- Policies of the nlu-based system run in case the routing slot is set to False (routing to NLU-based system) or None (non-sticky routing).
Don't create an instance of FlowRetrieval in the command generators in case no flows exists.
Patterns do not count as active flows in MultiStepLLMCommandGenerator anymore.
Make sure that all e2e test cases in rasa inspector are valid.
Downloading of CALM Assistants from Studio improved:
- Downloading CALM assistants from Studio now includes config and endpoints files
- Downloading CALM assistants from Studio now doesn't require config.yml and data folder to exist

[3.9.8] - 2024-08-21

Rasa Pro 3.9.8 (2024-08-21)

Bugfixes

Fix problem with custom action invocation when model is loaded from remote storage.

[3.9.7] - 2024-08-15

Rasa Pro 3.9.7 (2024-08-15)

Bugfixes

Fix extraction of tracing context from the request headers and injection into the Rasa server tracing context.
YamlValidationException will correctly return line number of the element where the error occurred when line number of that element is not returned by ruamel.yaml (for elements of primitive types, e.g. str, int, etc.), instead of returning the line number of the parent element.
Updated setuptools to fix security vulnerability.
Fix tracing context propagation to work for all external service calls.

Miscellaneous internal changes

Miscellaneous internal changes.

[3.9.6] - 2024-08-07

Rasa Pro 3.9.6 (2024-08-07)

Miscellaneous internal changes

Miscellaneous internal changes.

[3.9.5] - 2024-08-01

Rasa Pro 3.9.5 (2024-08-01)

Improvements

Enabled generative chitchat in the tutorial template with instructions on how to turn it off added to the documentation.

Bugfixes

Update the usage of time.process_time_ns with time.perf_counter_ns to fix the inconsistencies between duration metrics and trace spans duration.

[3.9.4] - 2024-07-25

Rasa Pro 3.9.4 (2024-07-25)

Bugfixes

Fix instrumentation not accounting for kwargs that are passed to NLUCommandAdapter.predict_commands.

Miscellaneous internal changes

Miscellaneous internal changes.

[3.9.3] - 2024-07-18

Rasa Pro 3.9.3 (2024-07-18)

Bugfixes

Refactor the supported remote storage (AWS, GCS, Azure) verification check before downloading Rasa model by fixing the initial implementation which attempted to create the object storage to check existence.
Fix TypeError: InformationRetrieval.search() got an unexpected keyword argument when tracing is enabled with EnterpriseSearchPolicy.
Change warning log level to error log level for Validator methods that verify that forms and actions used in stories and rules are present in the domain.

[3.9.2] - 2024-07-09

Rasa Pro 3.9.2 (2024-07-09)

Bugfixes

Add key-word arguments in the predict_commands method of LLM-based CommandGenerator class to ensure custom components are not impacted by changes to the signature of the base classes.

[3.9.1] - 2024-07-04

Rasa Pro 3.9.1 (2024-07-04)

Bugfixes

Modify the validation to throw an error for a missing associated action/utterance in a collect step only if the slot does not have a defined initial value.
Modify the collect step validation in flow executor to trigger pattern_internal_error for a missing associated action/utterance in a collect step only if the slot does not have a defined initial value.

[3.9.0] - 2024-07-03

Rasa Pro 3.9.0 (2024-07-03)

Features

Introduce a new response button payload format that runs set slot CALM commands directly by skipping the user message processing pipeline.
Added support for Information Retrieval custom components. It allows Enterprise Search Policy to be used with arbitrary search systems. Custom Information Retrievals can be implemented as a subclass of rasa.core.information_retrieval.InformationRetrieval
Enable slot filling in a CALM assistant to be configurable:
- either use NLU-based predefined slot mappings that instructs NLUCommandAdapter to issue SetSlot commands with values extracted from the user input via an entity extractor or intent classifier
- or use the new predefined slot mapping from_llm which enables LLM-based command generators to issue SetSlot commands If no slot mapping is defined, the default behavior is to use the from_llm slot mapping.
In case you had been using custom slot mapping type for slots set with the prediction of the LLM-based command generator, you need to update your assistant configuration to use the new from_llm slot mapping type. Note that even if you have written custom slot validation actions (following the validate_<slot_name> convention) for slots set by the LLM-based command generator, you need to update your assistant configuration to use the new from_llm slot mapping type.
For slots that are set only via the custom action e.g. slots set by external sources only, you need to add the action name to the slot mapping:
slots:
slot_name:
type: text
mappings:
- type: custom
action: custom_action_name
Skip SetSlot commands issued by LLM based command generators for slots with NLU-based predefined slot mappings. Instead, the command processor component will issue CannotHandle command to trigger pattern_cannot_handle if no other valid command is found.
Rasa now supports gRPC protocol for custom actions. This allows users to use gRPC to invoke custom actions. To connect to the action server using gRPC, specify:
endpoints.yml
action_endpoint:
url: "grpc://<rasa-grpc-action-server>:<port>"
Users can use secure (TLS) and insecure connections to communicate over gRPC. To use TLS specify the following in endpoints.yml:
endpoints.yml
action_endpoint:
url: "grpc://<rasa-grpc-action-server>:<port>"
cafile: "<ca_file>"
Add MultiStepLLMCommandGenerator as an alternative LLM based command generator. MultiStepLLMCommandGenerator breaks down the task of dialogue understanding into two steps: handling the flows and filling the slots. The component was designed to enable cheaper and smaller LLMs, such as gpt-3.5-turbo, as viable alternatives to costlier but more powerful models such as gpt-4. To use the MultiStepLLMCommandGenerator add it to your pipeline:
pipeline:
...
- name: MultiStepLLMCommandGenerator
...

Improvements

Improve diagram display in the inspector by adding an horizontal scroll and an auto scroll to the active step.
Create a separate default prompt for Enterprise Search with source citation enabled and revert the default Enterprise Search prompt to that of 3.7.x.
Refactored RemoteAction to utilize a new CustomActionExecutor interface by implementing HTTPCustomActionExecutor to handle HTTP requests for custom actions.
Implemented an optimization to reduce payload size by ensuring the Assistant sends the domain dictionary to the Action Server only once, which the server then stores. If the Action Server responds with a 449 status code indicating a missing domain context, the Assistant will repeat the API request including the domain dictionary in the payload, ensuring the server properly saves this data.
Integrate the capability of testing scenarios that reflect actual operational environments where conversations can be influenced by real-time external data. This is done by injecting metadata when running end-to-end tests.
Introduced LRU caching for reading and parsing YAML files to enhance performance by avoiding multiple reads of the same file. Added READ_YAML_FILE_CACHE_MAXSIZE environment variable with a default value of 256 to configure the cache size.
Add validations for flow ID to allow only alphanumeric characters, underscores, and hyphens except for the first character.
The LLMCommandGenerator component has been renamed to SingleStepLLMCommandGenerator. There is no change to the functionality.Using the LLMCommandGenerator as the name of the component results in a deprecation warning as it will be permanently renamed to SingleStepLLMCommandGenerator in 4.0.0. Please modify the assistant’s configuration to use the SingleStepLLMCommandGenerator instead of the LLMCommandGenerator to avoid seeing the deprecation warning.
Make improvements to rasa data validate that check if the usage of slot mappings in a CALM assistant is valid:
- a slot cannot have both a from_llm mapping and either a nlu-predefined mapping or a custom slot mapping
- a slot collected in a flow by a custom action has an associated action_ask_ defined in the domain
- a CALM assistant with slots that have nlu-based predefined mappings include NLUCommandAdapter in the config pipeline
- a NLU-based assistant cannot have slots that have a from_llm mapping
Modify post processing of commands - Clarify command with single option is converted into a StartFlow command.
Improve debug logging for predicate evaluation.

Bugfixes

Properly handle projects where rasa studio download is run in a project with no NLU data.
Tracing is supported for actions called over gRPC protocol.
Fix the hash function of ClarifyCommand to return a hashed list of options.
Raise an error if action_reset_routing is used without the defined ROUTE_TO_CALM_SLOT / router.
Add a few bugfixes to the CALM slot mappings feature:
- Coexistence bot should ignore NoOpCommand when checking if the processed message contains commands.
- Update condition under which FlowPolicy triggers pattern_internal_error for slots with custom slot mappings.
Remove invalid warnings during collect step.
- Fixed issue where messages with invalid intent triggers ('/<intent>') were not handled correctly. Now triggering the pattern_cannot_handle.
- Introduced a new reason cannot_handle_invalid_intent for use in the pattern_cannot_handle switch mechanism to improve error handling.
Validates that a collect step in a flow either has an action or an utterance defined in the domain to avoid the bot being silent.
Slots that are set via response buttons should not trigger pattern_cannot_handle regardless of the slots' mapping type.
Coerce "None", "null" or "undefined" slot values set via response buttons to be of type NoneType instead of str.
Avoid raising a UserWarning during validation of response buttons which contain double curly braces.
Do not run NLUCommandAdapter during message parsing when receiving a /SetSlots button payload. This is because the NLUCommandAdapter run during message parsing (when the graph is skipped) is meant to handle intent button payloads only.
Exclude slots that are not collected in any flow from being set by the NLUCommandAdapter in a coexistence assistant.
Default action action_extract_slots should not run custom actions specified in custom slot mappings for slots that are set by custom actions in the flows/CALM system of a coexistence assistant.
Fix pattern flows being unavailable during input preparation and template rendering in MultiStepLLMCommandGenerator.
Skip command cleaning when no commands are present in NLUCommandAdapter. Fix get active flows to return the correct active flows, including all the nested parent flows if present.
If FlowPolicy tries to collect a slot with a custom slot mapping without the action key or action_ask specified in the domain, it will trigger pattern_cancel_flow first, then pattern_internal_error.
Cancel user flow in progress and invoke pattern_internal_error if the flow reached a collect step which does not have an associated utter_ask response or action_ask action defined in the domain.
IntentlessPolicy abstains from making a prediction during coexistence when it's the turn of the NLU-based system.

Miscellaneous internal changes

Miscellaneous internal changes.

[3.8.17] - 2024-10-18

Rasa Pro 3.8.17 (2024-10-18)

Improvements

Change default response of utter_free_chitchat_response from "placeholder_this_utterance_needs_the_rephraser" to "Sorry, I'm not able to answer that right now.".

[3.8.16] - 2024-10-02

Rasa Pro 3.8.16 (2024-10-02)

No significant changes.

[3.8.15] - 2024-10-01

Rasa Pro 3.8.15 (2024-10-01)

Bugfixes

Fixes an issue where the CountVectorsFeaturizer and LogisticRegressionClassifier would throw error during inference when no NLU training data is provided.

[3.8.14] - 2024-09-20

Rasa Pro 3.8.14 (2024-09-20)

Deprecations and Removals

Dropped support for Python 3.8 ahead of Python 3.8 End of Life in October 2024. In Rasa Pro versions 3.10.0, 3.9.11 and 3.8.13, we needed to pin the TensorFlow library version to 2.13.0rc1 in order to remove critical vulnerabilities; this resulted in poor user experience when installing these versions of Rasa Pro with uv pip. Removing support for Python 3.8 will make it possible to upgrade to a stabler version of TensorFlow.

Improvements

Update Keras and Tensorflow to version 2.14. This will eliminate the need to use the --prerelease allow flag when installing Rasa Pro using uv pip tool.

Bugfixes

Fixed UnexpecTEDIntentlessPolicy training errors that resulted from a change to batching behavior. Changed the batching behavior back to the original for all components. Made the changed batching behavior accessible in DietClassifier using drop_small_last_batch: True.

Miscellaneous internal changes

Miscellaneous internal changes.

[3.8.13] - 2024-09-12

Rasa Pro 3.8.13 (2024-09-12)

Bugfixes

Fixed telemetry collection for the components Rephraser, LLM Intent Classifier, Intentless Policy and Enterprise Search Policy to ensure that the telemetry data is only collected when it is enabled
Update Keras to 2.13.1 and Tensorflow to 2.13.0rc0 to fix critical vulnerability (CVE-2024-3660).

[3.8.12] - 2024-08-12

Rasa Pro 3.8.12 (2024-08-12)

Bugfixes

Fix TypeError: InformationRetrieval.search() got an unexpected keyword argument when tracing is enabled with EnterpriseSearchPolicy.
Fix extraction of tracing context from the request headers and injection into the Rasa server tracing context.
Update the usage of time.process_time_ns with time.perf_counter_ns to fix the inconsistencies between duration metrics and trace spans duration.
YamlValidationException will correctly return line number of the element where the error occurred when line number of that element is not returned by ruamel.yaml (for elements of primitive types, e.g. str, int, etc.), instead of returning the line number of the parent element.
Updated setuptools to fix security vulnerability.

Miscellaneous internal changes

Miscellaneous internal changes.

[3.8.11] - 2024-07-04

Rasa Pro 3.8.11 (2024-07-04)

Improvements

Improve debug logging for predicate evaluation.

Bugfixes

Raise an error if action_reset_routing is used without the defined ROUTE_TO_CALM_SLOT / router.
Remove invalid warnings during collect step.
- Fixed issue where messages with invalid intent triggers ("/intent_name") were not handled correctly. Now triggering the pattern_cannot_handle.
- Introduced a new reason cannot_handle_invalid_intent for use in the pattern_cannot_handle switch mechanism to improve error handling.
Validates that a collect step in a flow either has an action or an utterance defined in the domain to avoid the bot being silent.
Skip command cleaning when no commands are present in NLUCommandAdapter. Fix get active flows to return the correct active flows, including all the nested parent flows if present.
Update the handling of incorrect use of slash syntax. Messages with undefined intents do not automatically trigger pattern_cannot_handle; instead, they are sanitized (prepended slash(es) are removed) and passed through the graph.
Modify the validation to throw an error for a missing associated action/utterance in a collect step only if the slot does not have a defined initial value.

[3.8.10] - 2024-06-19

Rasa Pro 3.8.10 (2024-06-19)

Improvements

Added NLG validation to the rasa model training process.

Bugfixes

Fixes Clarify command being dropped by command processor due to presence of coexistence slot - route_session_to_calm
Fix validation for LLMBasedRouter to check only for calm_entry.sticky

[3.8.9] - 2024-06-14

Rasa Pro 3.8.9 (2024-06-14)

Improvements

Add validations for flow ID to allow only alphanumeric characters, underscores, and hyphens except for the first character.

[3.8.8] - 2024-06-07

Rasa Pro 3.8.8 (2024-06-07)

Bugfixes

Add wrappers around openai clients that can set the self-signed certs via REQUESTS_CA_BUNDLE env variable.

[3.8.7] - 2024-05-29

Rasa Pro 3.8.7 (2024-05-29)

Bugfixes

Add support for domain entities in CALM import
Download NLU data when running rasa studio download for a modern assistant with NLU triggers. Previously, this data was not downloaded, leading to a partial assistant.

[3.8.6] - 2024-05-27

Rasa Pro 3.8.6 (2024-05-27)

Improvements

Adds tracker_state attribute to OutputChannel. It simplifies the access of tracker state for custom channel connector with CollectingOutputChannel.tracker_state.

Bugfixes

If a button in a response does not have a payload, socketio channel will use the title as payload by default rather than throwing an exception.

[3.8.5] - 2024-05-03

Rasa Pro 3.8.5 (2024-05-03)

Bugfixes

Trigger pattern_internal_error if collection does not exist in a Qdrant vector store.

[3.8.4] - 2024-04-30

Rasa Pro 3.8.4 (2024-04-30)

Improvements

Added support for NLU Triggers by supporting uploading the NLU files for CALM Assistants

[3.8.3] - 2024-04-26

Rasa Pro 3.8.3 (2024-04-26)

Improvements

- Throw validation error and exit when duplicate responses are found across domains. This is a breaking change, as it will cause training to fail if duplicate responses are found. If you have duplicate responses in your training data, you will need to remove them before training.
- Update domain importing to ignore the warnings about duplicates when merging with the default flow domain

Bugfixes

Use AzureChatOpenAI class instead of AzureOpenAI class to instantiate openai models deployed in Azure. This fixes the usage of gpt-3.5-turbo model in Azure.
Fixes validation to catch empty placeholders in response that dumps entire context.
Fix security vulnerabilities by updating poetry environment: fonttools, CVE-2023-45139, from 4.40.0 to 4.43.0 aiohttp, CVE-2024-27306, from 3.9.3 to 3.9.4 dnspython, CVE-2023-29483, from 2.3.0 to 2.6.1 pymongo, CVE-2024-21506, from 4.3.3 to 4.6.3
Numbers that are part of the body of the LLM answer in EnterpriseSearch should not be matched as citation references in the postprocessing method.
Errors from the Flow Retrieval API are now both logged and thrown. When such errors occur, an ErrorCommand is emitted by the Command Generator.

[3.8.2] - 2024-04-25

Rasa Pro 3.8.2 (2024-04-25)

Bugfixes

Add the currently active flow as well as the called flow (if present) to the list of available flows for the LLMCommandGenerator.
Fix custom prompt not read from the model resource path for LLMCommandGenerator.

[3.8.1] - 2024-04-17

Rasa Pro 3.8.1 (2024-04-17)

Improvements

Adjusted chat widget behavior to remain open when clicking outside the chat box area.
Improve debug logs to include information about evaluation of if-else conditions in flows at runtime.
Remove the ContextualResponseRephraser from the tutorial template to keep it simple as it is not needed anymore.
Update poetry package manager version to 1.8.2. Check the migration guide for instructions on how to update your environment.

Bugfixes

Introduced support for numbered Markdown lists.
Added support for uploading assistants with default domain directory.

Miscellaneous internal changes

Miscellaneous internal changes.

[3.8.0] - 2024-04-03

Rasa Pro 3.8.0 (2024-04-03)

Features

Introduces semantic retrieval of flows at runtime to reduce the size of the prompt sent to the LLM by utilizing similarity between vector embeddings. It enables the assistant to scale to a large number of flows.
Flow retrieval is enabled by default. To configure it, you can modify the settings under the flow_retrieval property of LLMCommandGenerator component. For detailed configuration options, refer to our documentation.
Introduces always_include_in_prompt field to the flow definition. If field is set to true and the flow guard defined in the if field evaluates to true, the flow will be included in the prompt.
Introduction of coexistence between CALM and NLU-based assistants. Coexistence allows you to use policies from both CALM and NLU-based assistants in a single assistant. This allows migrating from NLU-based paradigm to CALM in an iterative fashion.
Introduction of call step. You can use a call step to embed another flow. When the execution reaches a call step, Rasa starts the called flow. Once the called flow is complete, the execution continues with the calling flow.

Improvements

Instrument the command_processor module, in particular the following functions:
- execute_commands
- clean_up_commands
- validate_state_of_commands
- remove_duplicated_set_slots
Improve the instrumentation of LLMCommandGenerator:
- extract more LLM configuration parameters, e.g. type, temperature, request-timeout, engine and deployment (the latter 2 being only for the Azure OpenAI service).
- instrument the private method _check_commands_against_startable_flows to track the commands with which the LLM responded, as well as the startable flow ids.
Instrument flow_executor.py module, in particular these functions:
- advance_flows(): extract available_actions tracing tag
- advance_flows_until_next_action(): extract action name and score, metadata and prediction events as tracing tags from the returned prediction value
- run_step(): extract step custom id, description and current flow id.
Instrument Policy._prediction() method for each of the policy subclasses.
Instrument IntentlessPolicy methods such as:
- find_closest_response: extract the response and score from the returned tuple;
- select_response_examples: extract the ai_response_examples from returned value;
- select_few_shot_conversations: extract the conversation_samples from returned value;
- extract_ai_responses: extract the ai_responses from returned value;
- generate_answer: extract the llm_response from returned value.
1. Instrument InformationRetrieval.search method for supported vector stores: extract query and document metadata tracing attributes.
2. Instrument EnterpriseSearchPolicy._generate_llm_answer method: extract LLM config tracing attributes.
3. Extract dialogue stack current context in the following functions:
- rasa.dialogue_understanding.processor.command_processor.clean_up_commands
- rasa.core.policies.flows.flow_executor.advance_flows
- rasa.core.policies.flows.flow_executor.run_step
1. Instrument NLUCommandAdapter.predict_commands method and extract the commands from the returned value, as well as the user message intent.
2. Improve LLM config tracing attribute extraction for ContextualResponseRephraser.
Add new config boolean property trace_prompt_tokens that would enable the tracing of the length of the prompt tokens for the following components:
- LLMCommandGenerator
- EnterpriseSearchPolicy
- IntentlessPolicy
- ContextualResponseRephraser
Enable execution of single E2E tests by including the test case name in the path to test cases, like so: path/to/test_cases.yml::test_case_name or path/to/folder_containing_test_cases::test_case_name.
Implement MetricInstrumentProvider interface whose role is to:
- register instruments during metrics configuration
- retrieve the appropriate instrument to record measurements in the relevant instrumentation code section
Enabled the setting of a minimum similarity score threshold for retrieved documents in Enterprise Search's vector_store with the addition of the threshold property. If no documents are retrieved, it triggers Pattern Cannot Handle. This feature is supported in Milvus and Qdrant vector stores.
Record measurements for the following metrics in the instrumentation code:
- CPU usage of the LLMCommandGenerator
- memory usage of LLMCommandGenerator
- prompt token usage of LLMCommandGenerator
- method call duration for LLM specific calls (in LLMCommandGenerator, EnterpriseSearchPolicy, IntentlessPolicy, ContextualResponseRephraser)
- rasa client request duration
- rasa client request body size
Instrument EndpointConfig.request() method call in order to measure the client request metrics.
Improvements around default behaviour of ChitChatAnswerCommand():
- The command processor will issue CannotHandleCommand() instead of the ChitChatCommand() when pattern_chitchat uses an action step action_trigger_chitchat without the IntentlessPolicy being configured. During training a warning is raised.
- Changed the default pattern_chitchat to:
pattern_chitchat:
description: handle interactions with the user that are not task-oriented
name: pattern chitchat
steps:
- action: action_trigger_chitchat
- Default rasa init template for CALM comes with IntentlessPolicy added to pipeline.
Add support for OTLP Collector as metrics receiver which can forward metrics to the chosen metrics backend, e.g. Prometheus.
Enable document source citation for Enterprise Search knowledge answers by setting the boolean citation_enabled: true property in the config.yml file:
policies:
- name: EnterpriseSearchPolicy
citation_enabled: true
Add telemetry events for flow retrieval and call step
Tighten python dependency constraints in pyproject.toml, hence reducing the installation time to around 20 minutes with pip (and no caching enabled).
Improved tracing clarity of the Contextual Response Rephraser by adding the _create_history method span, including its LLM configuration attributes.
Users now have enhanced control over the debugging process of LLM-driven components. This update introduces a fine-grained, customizable logging that can be controlled through specific environment variables.
For example, set the LOG_LEVEL_LLM environment variable to enable detailed logging at the desired level for all the LLM components or specify the component you are debugging:
Example configuration
export LOG_LEVEL_LLM=DEBUG
export LOG_LEVEL_LLM_COMMAND_GENERATOR=INFO
export LOG_LEVEL_LLM_ENTERPRISE_SEARCH=INFO
export LOG_LEVEL_LLM_INTENTLESS_POLICY=DEBUG
export LOG_LEVEL_LLM_REPHRASER=DEBUG
If the user wants to chat with the assistant at the end of rasa init, we are now calling rasa inspect instead of rasa shell.
A slot can now be collected via an action action_ask_<slot-name> instead of the utterance utter_ask_<slot-name> in a collect step. You can either define an utterance or an action for the collect step in your flow. Make sure to add your custom action action_ask_<slot-name> to the domain file.
Validate the configuration of the coexistence router before the actual training starts.
Improved error handling in Enterprise Search Policy, changed the prompt to improve formatting of documents and ensured empty slots are not added to the prompt.
Implement asynchronous graph execution. CALM assistants rely on a lot of I/O calls (e.g. to a LLM service), which impaired performances. With this change, we've improved the response time performance by 10x. All policies and components now support async calling.
Merge rasa and rasa-plus packages into one. As a result, we renamed the Python package to rasa-pro and the Docker image to rasa-pro. Please head over to the migration guide here for installation, and here for the necessary configuration updates.

Bugfixes

Updated pillow and jinja2 packages to address security vulnerabilities.
Fix OpenTelemetry Invalid type NoneType for attribute value warning.
Add support for metadata_payload_key for Qdrant Vector Store with an error message if content_payload_key or metadata_payload_key are incorrect
Changed the ordering of returned events to order by ID (previously timestamp) in SQL Tracker Store
Improved the end-to-end test comparison mechanism to accurately handle and strip trailing newline characters from expected bot responses, preventing false negatives due to formatting mismatches.
Fixed a bug that caused inaccurate search results in Enterprise Search when a bot message appeared before the last user message.
Fixes flow guards pypredicate evaluatation bug: pypredicate was evaluated with Slot instances instead of slot values
Post-process source citations in Enterprise Search Policy responses so that they are enumerated in the correct order.
Resolves issue causing the FlowRetrieval.populate to always use default embeddings.
Fix the bug with the validation of routing setup crashing when the pipeline is not specified (null)
Remove conversation turns prior to a restart when creating a conversation transcript for an LLM call.
This helps in cases where the prior conversation is not relevant for the current session. Information which should be carried to the next session should explicitly be stored in slots.
Add tracker back to the LLMCommandGenerator.parse_command to ensure compatibility with custom command generator built with 3.7.
Move coexistence routing setup validation from rasa.validator.Validator to rasa.engine.validation. This gave access to graph schema which allowed for validation checks of subclassed routers.
Fixes a bug in determining the name of the model based on provided parameters.
LogisticRegressionClassifier checks if training examples are present during training and logs a warning in case no training examples are provided.
Fixes the bug that resulted in an infinite loop on a collect step in a flow with a flow guard set to if: False.
Fix training the enterprise search policy multiple times with a different source folder name than the default name "docs".
Log message llm_command_generator.predict_commands.finished is set to debug log by default. To enable logging of the LLMCommandGenerator set LOG_LEVEL_LLM_COMMAND_GENERATOR to INFO.
Improvements and fixes to cleaning up commands:
- Clean up predicted StartFlow commands from the LLMCommandGenerator if the flow, that should be started, is already active.
- Clean up predicted SetSlot commands from the LLMCommandGenerator if the value of the slot is already set on the tracker.
- Use string comparison for slot values to make sure to capture cases when the LLMCommandGenerator predicted a string value but the value set on the tracker is, for example, an integer value.
Remove context from list of restricted slots
Improved handling of categorical slots with text values when using CALM.
Slot values extracted by the command generator (LLM) will be stored in the same casing as the casing used to define the categorical slot values in the domain. E.g. A categorical slot defined to store the values ["A", "B"] will store "A" if the LLM predicts the slot to be filled with "a". Previously, this would have stored "a".

Miscellaneous internal changes

Miscellaneous internal changes.

[3.7.9] - 2024-03-26

Rasa Pro 3.7.9 (2024-03-26)

Improvements

Add validations for flow ID to allow only alphanumeric characters, underscores, and hyphens except for the first character.

Bugfixes

Changed the ordering of returned events to order by ID (previously timestamp) in SQL Tracker Store
Fixes flow guards pypredicate evaluatation bug: pypredicate was evaluated with Slot instances instead of slot values
Improved handling of categorical slots with text values when using CALM.
Slot values extracted by the command generator (LLM) will be stored in the same casing as the casing used to define the categorical slot values in the domain. E.g. A categorical slot defined to store the values ["A", "B"] will store "A" if the LLM predicts the slot to be filled with "a". Previously, this would have stored "a".
Log message llm_command_generator.predict_commands.finished is set to debug log by default. To enable logging of the LLMCommandGenerator set LOG_LEVEL_LLM_COMMAND_GENERATOR to INFO.
Improvements and fixes to cleaning up commands:
- Clean up predicted StartFlow commands from the LLMCommandGenerator if the flow, that should be started, is already active.
- Clean up predicted SetSlot commands from the LLMCommandGenerator if the value of the slot is already set on the tracker.
- Use string comparison for slot values to make sure to capture cases when the LLMCommandGenerator predicted a string value but the value set on the tracker is, for example, an integer value.

[3.7.8] - 2024-02-28

Rasa Pro 3.7.8 (2024-02-28)

Improvements

Improved UX around ClarifyCommand by checking options for existence and ordering them. Also, now dropping Clarify commands if there are any other commands to prevent two questions or statements to be uttered at the same time.
LLMCommandGenerator returns CannotHandle() command when is encountered with scenarios where it is unable to predict a valid command.

Bugfixes

Replace categorical slot values in a predicate with lower case replacements. This fixes the case sensitive slot comparisons in flow guards, branches in flows and slot rejections.
Modify flows YAML schema to make next step mandatory to noop step.
Flush messages when Kafka producer is closed. This is to ensure that all messages in the producer's internal queue are sent to the broker. Ensure to import all pattern stack frame subclasses of DialogueStackFrame when retrieving tracker from the tracker store, a required step during rasa export.
Add support for metadata_payload_key for Qdrant Vector Store with an error message if content_payload_key or metadata_payload_key are incorrect

[3.7.7] - 2024-02-06

Rasa Pro 3.7.7 (2024-02-06)

Bugfixes

Updated pillow and jinja2 packages to address security vulnerabilities.

[3.7.6] - 2024-02-01

Rasa Pro 3.7.6 (2024-02-01)

Bugfixes

Fix reported issue, e.g. https://github.com/RasaHQ/rasa/issues/5461 in Rasa Pro: Do not unpack json payload if data key is not present in the response custom output payloads when using socketio channel. This allows assistants which use custom output payloads to work with the Rasa Inspector debugging tool.
Make flow description a required property in the flow json schema.
Fix training the enterprise search policy multiple times with a different source folder name than the default name "docs".

Miscellaneous internal changes

Miscellaneous internal changes.

[3.7.5] - 2024-01-24

Rasa Pro 3.7.5 (2024-01-24)

Improvements

Add new embedding types: huggingface and huggingface_bge. These new types import the HuggingFaceEmbeddings and HuggingFaceBgeEmbeddings embedding classes from Langchain.

Bugfixes

Fixes a bug that caused the full_retrieval_intent_name key to be missing in the published event. Rasa Analytics makes use of this key to get the Retrieval Intent Name
Pin grpcio indirect dependency to 1.56.2 to address CVE-2023-33953 Pin aiohttp to version 3.9.0 to address CVE-2023-49081
Fixes the bug that resulted in an infinite loop on a collect step in a flow with a flow guard set to if: False.
Changed the parameters request timeout to 10 seconds and maximum number of retries to 1 for the default LLM used by Enterprise Search Policy. Any error during vector search or LLM API calls should now trigger the pattern pattern_internal_error. Updated the default enterprise search policy prompt to respond more succinctly to queries.

[3.7.4] - 2024-01-03

Rasa Pro 3.7.4 (2024-01-03)

Improvements

Add embeddings type azure to simplify azure configurations, particularly when using Enterprise Search Policy

Bugfixes

Add a validation in rasa data validate to check the LinkFlowStep refers to a valid flow ID

[3.7.3] - 2023-12-21

Rasa Pro 3.7.3 (2023-12-21)

Improvements

Persist prompt as part of the model and reread prompt from the model storage instead of original file path during loading. Impacts LLMCommandGenerator.
Replaced soon to be depracted text-davinci-003 model with gpt-3.5-turbo. Affects components - LLM Intent Classifier and Contextual Response Rephraser.

Bugfixes

Fix stale cache of local knowledge base used by EnterpriseSearchPolicy by implementing the fingerprint_addon class method.

Miscellaneous internal changes

Miscellaneous internal changes.

[3.7.2] - 2023-12-07

Rasa Pro 3.7.2 (2023-12-07)

Bugfixes

Fix propagation of context across rasa spans when running rasa run --enable-api in the case when no additional tracing context is passed to rasa.
Fixed a bug in policy invocation that made Enterprise Search Policy and action_trigger_search behaved strangely when used with rules and stories
Updated aiohttp, cryptography and langchain to address security vulnerabilities.

Miscellaneous internal changes

Miscellaneous internal changes.

[3.7.1] - 2023-12-01

Rasa Pro 3.7.1 (2023-12-01)

Improvements

Improved error handling in Enterprise Search Policy, changed the prompt to improve formatting of documents and ensured empty slots are not added to the prompt

[3.7.0] - 2023-11-22

Rasa Pro 3.7.0 (2023-11-22)

Features

Added Enterprise Search Policy that uses an LLM with conversation context and relevant knowledge base documents to generate rephrased responses. The LLM is prompted to answer the user questions given the chat transcript, documents retrived from a document search and the slot values so far. This policy supports an in-memory Faiss vector store and connecting to instances of Milvus or Qdrant vector store.

Improvements

Skip executing the pipeline when the user message is of the form /intent or /intent + entities.
Remove tensorflow-addons from dependencies as it is now deprecated.
Add building multi-platform Docker image (amd64/arm64)
Switch struct log to FilteringBoundLogger in order to retain log level set in the config.
Added metadata as an additional argument as an additional parameter to an Actions run method.
Added an additional default action called action_send_text which allows a policy to respond with a text. The text is passed to the action using the metadata, e.g. metadata={"message": {"text": "Hello"}}.
Added LLM utility functions.
Passed request headers from REST channel.
Added additional method fingerprint_addon to the GraphComponent interface to allow inclusion of external data into the fingerprint calculation of a component
Added Schema file and schema validation for flows.
Added environment variables to configure JWT and auth token. For JWT the following environment variables are available:
- JWT_SECRET
- JWT_METHOD
- JWT_PRIVATE_KEY
For auth token the following environment variable is available:
- AUTH_TOKEN
Add skip question command
Update the CALM starter template by:
- adding the following flows from the financial chatbot:
  - add_contact.yml
  - remove_contact.yml
  - list_contacts.yml
- using multiple modules (in the form of yml files) to segregate the flows (a good model to be followed)
- adding e2e tests:
  - happy paths
  - cancelations
  - corrections
- Enhanced the Rasa error pattern for accommodating various error types.
- Upgraded the LLMCommandGenerator for processing the new 'user_input' configuration section. This update includes handling of messages that surpass the defined character limit.
Configuration Update:
The LLMCommandGenerator now supports a user-defined character limit via the 'user_input' configuration:
- name: LLMCommandGenerator
llm:
...
user_input:
max_characters: 500
Default Behavior:
In the absence of a specified limit, it defaults to a 420-character cap. To bypass the limit entirely, set the 'max_characters' value to -1.
- Bot now returns a default message in response to an empty user message. This improves user experience by providing feedback even when no input is detected.
- LLMCommandGenerator behavior updated. It now returns an ErrorCommand for empty user messages.
- Updated default error pattern and added the default utterance in default_flows_for_patterns.yml
Add support for Vault namespaces. To use namespace set either:
- VAULT_NAMESPACE environment variable
- namespace property in secrets_manager section at endpoints.yaml
Added Rasa Labs LLM components. Added components are:
- LLMIntentClassifier
- IntentlessPolicy
- ContextualResponseRephraser
Made it possible for the Rasa REST channel to accept OpenTelemetry tracing context.
Improved the naming of trace spans and added more trace tags.
Add slot_was_not_set to E2E testing for asserting that a slot was not set and that a slot was not set with a specific value.
Introduced the rasa studio download command, enabling data retrieval from the studio. Implemented the option to refresh the Keycloak token. Expanded the functionality of RasaPrimitiveStorageMapper with the addition of flows. Added flows support to rasa studio train.
Instrument LLMCommandGenerator._generate_action_list_using_llm and Command.run_command_on_tracker methods.
Added the default values for the number of tokens generated by the LLM (max_tokens)
Make the instrumentation of Command.run_command_on_tracker method applicable to all subclasses of the Command class`
Instrument ContextualResponseRephraser._generate_llm_response and ContextualResponseRephraser.generate methods.
Extract commands as tracing attributes from message input when previous node was the LLMCommandGenerator.
Rename rasa chat command to rasa inspect and rename channel name to inspector.
Extract events and optional_events when GraphNode is FlowPolicy.

Bugfixes

uvloop is disabled by default on apple silicon machines
Add rasa_events to the list of anonymizable structlog keys and rename structlog keys.
Introduce a validation step in rasa data validate and rasa train commands to identify non-existent paths and empty domains.
Rich responses containing buttons with parentheses characters are now correctly parsed. Previously any characters found between the first identified pair of () in response button took precedence.
Resolve dependency incompatibility: Pin version of dnspython to ==2.3.0.
Fixed KeyError which resulted when domain_responses doesn't exist as a keyword argument while using a custom action dispatcher with nlg server.
Fixed incompatibility with latest python-socketio release.
The python-socketio released a backwards incompatible change on their minor release. This fix addresses this and makes the code compatible with prior and the new python-socketio version.
https://github.com/miguelgrinberg/python-socketio/blob/main/CHANGES.md
Fixed the 404 Not Found Github actions error while removing packages.
Corrected E2E diff behavior to prevent it from going out of sync when more than one turn difference exists between actual and expected events. Fixed E2E tests from propagating errors when events and test steps did not have the same length. Fixed the issue where E2E tests couldn't locate slot events that were not arranged chronologically. Resolved the problem where E2E tests were incorrectly diffing user utter events when they were not in the correct order.
Fixed E2E runner wrongly selecting the first available bot utterance when generating the test fail diff.
Updated werkzeug and urllib3 to address security vulnerabilities.
Fix cases when E2E test runner crashes when there is no response from the bot.

Improved Documentation

Update wording in Rasa Pro installation page.
Updated docs on sending Conversation Events to Multiple DBs.
Corrected action server api sample in docs.
Document support for Vault namespaces.
Updated tracing documentation to include tracing in the action server and the REST Channel.

Miscellaneous internal changes

Miscellaneous internal changes.

[3.6.13] - 2023-10-23

Rasa Pro 3.6.13 (2023-10-23)

Bugfixes

Fix wrong conflicts that occur when rasa validate stories is run with slots that have active_loop set to null in mapping conditions.

[3.6.12] - 2023-10-10

Rasa Pro 3.6.12 (2023-10-10)

Improvements

Added username to the connection parameters for ConcurrentRedisLockStore.

Bugfixes

Refresh headers used in requests (e.g. action server requests) made by EndpointConfig using its headers attribute.
Upgrade pillow to 10.0.1 to address security vulnerability CVE-2023-4863 found in 10.0.0 version.
Fix setuptools security vulnerability CVE-2022-40897 in Docker build by updating setuptools in poetry's environment.

[3.6.11] - 2023-10-05

Rasa Pro 3.6.11 (2023-10-05)

Bugfixes

Intent names will not be falsely abbreviated in interactive training (fixes OSS-413).
This will also fix a bug where forced user utterances (using the regex matcher) will be reverted even though they are present in the domain.
Cache EndpointConfig session object using cached_property decorator instead of recreating this object on every request. Initialize these connection pools for action server and model server endpoints as part of the Sanic after_server_start listener. Also close connection pools during Sanic after_server_stop listener.

[3.6.10] - 2023-09-26

Rasa Pro 3.6.10 (2023-09-26)

Improvements

Improved handling of last batch during DIET and TED training. The last batch is discarded if it contains less than half a batch size of data.
Added username to the connection parameters for RedisLockStore and RedisTrackerStore
Telemetry data is only send for licensed users.

Improved Documentation

Remove the Playground from docs.

[3.6.9] - 2023-09-15

Rasa Pro 3.6.9 (2023-09-15)

Improvements

Added additional method fingerprint_addon to the GraphComponent interface to allow inclusion of external data into the fingerprint calculation of a component

Bugfixes

Fixed KeyError which resulted when domain_responses doesn't exist as a keyword argument while using a custom action dispatcher with nlg server.

[3.6.8] - 2023-08-30

Rasa Pro 3.6.8 (2023-08-30)

Bugfixes

Fix E2E testing diff algorithm to support the following use cases:
- asserting a slot was not set under a slot_was_set block
- asserting multiple slot names and/or values under a slot_was_set block Additionally, the diff algorithm has been improved to show a higher fidelity result.

[3.6.7] - 2023-08-29

Rasa Pro 3.6.7 (2023-08-29)

Bugfixes

Updated certifi, cryptography, and scipy packages to address security vulnerabilities.
Updated setuptools and wheel to address security vulnerabilities.

[3.6.6] - 2023-08-23

Rasa Pro 3.6.6 (2023-08-23)

Bugfixes

Updated setuptools and wheel to address security vulnerabilities.

[3.6.5] - 2023-08-17

Rasa Pro 3.6.5 (2023-08-17)

Improvements

Use the same session across requests in RasaNLUHttpInterpreter

Bugfixes

Resolve dependency incompatibility: Pin version of dnspython to ==2.3.0.
Fix the issue in rasa test e2e where test diff inaccurately displayed actual event transcripts, leading to the duplication of BotUtter`` or UserUtter events. This occurred specifically when `SetSlot events took place that were not explicitly defined in the Test Cases.

Improved Documentation

Updated PII docs with new section on how to use Rasa X/Enterprise with PII management solution, and a new note on debug logs being displayed after the bot message with rasa shell.

[3.6.4] - 2023-07-21

Rasa Pro 3.6.4 (2023-07-21)

Bugfixes

Extract conditional response variation and channel variation filtering logic into a separate component. Enable usage of this component in the NaturalLanguageGenerator subclasses (e.g. CallbackNaturalLanguageGenerator, TemplatedNaturalLanguageGenerator). Amend nlg_request_format to include a single response ID string field, instead of a list of IDs.
Added details to the logs of successful and failed cases of running the markers upload command.

Improved Documentation

Updated commands with square brackets e.g (pip install rasa[spacy]) to use quotes (pip install 'rasa[spacy]') for compatibility with zsh in docs.

[3.6.3] - 2023-07-20

Rasa Pro 3.6.3 (2023-07-20)

Improvements

Added a human readable component to structlog using the event_info key and made it the default rendered key if present.

Bugfixes

Fix the issue with the most recent model not being selected if the owner or permissions where modified on the model file.
Fixed BlockingIOError which occured as a result of too large data passed to strulogs.
Fixed the error handling mechanism in rasa test e2e to quickly detect and communicate errors when the action server, defined in endpoints.yaml, is not available.
Allow hyphens - to be present in e2e test slot names.
Resolved issues in rasa test e2e where errors occurred when the bot concluded the conversation with SetSlot events while there were remaining steps in the test case. Corrected the misleading error message '- No slot set' to '- Slot types do not match' in rasa test e2e when a type mismatch occurred during testing.

Improved Documentation

Update action server documentation with new capability to extend Sanic features by using plugins. Update rasa-sdk dependency to version 3.6.1.
Updated commands with square brackets e.g (pip install rasa[spacy]) to use quotes (pip install 'rasa[spacy]') for compatibility with zsh in docs.

[3.6.2] - 2023-07-06

Rasa Pro 3.6.2 (2023-07-06)

Improvements

Add building Docker container for arm64 (e.g. to allow running Rasa inside docker on M1/M2).
Bumped the version of OpenTelemetry to meet the requirement of protobuf 4.x.

Bugfixes

Resolves the issue of importing TensorFlow on Docker for ARM64 architecture.

[3.6.1] - 2023-07-03

Rasa Pro 3.6.1 (2023-07-03)

Improvements

Add building multi-platform Docker image (amd64/arm64)
Switch struct log to FilteringBoundLogger in order to retain log level set in the config.
Add new anonymizable structlog keys.

Bugfixes

Add rasa_events to the list of anonymizable structlog keys and rename structlog keys.
Introduce a validation step in rasa data validate and rasa train commands to identify non-existent paths and empty domains.
Rich responses containing buttons with parentheses characters are now correctly parsed. Previously any characters found between the first identified pair of () in response button took precedence.
Add PII bugfixes (e.g. handling None values and casting data types to string before being passed to the anonymizer) after testing manually with Audiocodes channel.

Improved Documentation

Update wording in Rasa Pro installation page.
Document new PII Management section.
Added Documentation for Realtime Markers Section.
Add "Rasa Pro Change Log" to documentation.
Document new Load Testing Guidelines section.
Changes the formatting of realtime markers documentation page

[3.6.0] - 2023-06-14

Rasa Pro 3.6.0 (2023-06-14)

Deprecations and Removals

Removed Python 3.7 support as it reaches its end of life in June 2023

Features

Implemented PII (Personally Idenfiable Information) management using Microsoft Presidio as the entity analyzer and anonymization engine. The feature covers the following:
- anonymization of Rasa events (UserUttered, BotUttered, SlotSet, EntitiesAdded) before they are streamed to Kafka event broker anonymization topics specified in endpoints.yml.
- anonymization of Rasa logs that expose PII data
The main components of the feature are:
- anonymization rules that define in endpoints.yml the PII entities to be anonymized and the anonymization method to be used
- anonymization executor that executes the anonymization rules on a given text
- anonymization orchestrator that orchestrates the execution of the anonymization rules and publishes the anonymized event to the matched Kafka topic.
- anonymization pipeline that contains a list of orchestrators and is registered to a singleton provider component, which gets invoked in hook calls in Rasa Pro when the pipeline must be retrieved for anonymizing events and logs.
Please read through the PII Management section in the official documentation to learn how to get started.
Implemented support for real time evaluation of Markers with the Analytics Data Pipeline, enabling you to gain valuable insights and enhance the performance of your Rasa Assistant.
For this feature, we've added support for rasa markers upload command. Running this command validates the marker configuration file against the domain file and uploads the configuration to Analytics Data Pipeline.

Improvements

Add optional property ids to the nlg server request body. IDs will be transmitted to the NLG server and can be used to identify the response variation that should be used.
Add building Docker container for arm64 (e.g. to allow running Rasa inside docker on M1/M2).
Add support for Location data from Whatsapp on Twilio Channel
Add validation to rasa train to align validation expectations with rasa data validate. Add --skip-validation flag to disable validation and --fail-on-validation-warnings, --validation-max-history to rasa train to have the same options as rasa data validate.
Updated tensorflow to version 2.11.1 for all platforms except Apple Silicon which stays on 2.11.0 as 2.11.1 is not available yet
Slot mapping conditions accept active_loop specified as null in those cases when slots with this mapping condition should be filled only outside form contexts.
Add an optional description key to the Markers Configuration format. This can be used to add documentation and context about marker's usage. For example, a markers.yml can look like
marker_name_provided:
description: “Name slot has been set”
slot_was_set: name

marker_mood_expressed:
description: “Unhappy or Great Mood was expressed”
or:
- intent: mood_unhappy
- intent: mood_great
Add rasa marker upload command to upload markers to the Rasa Pro Services. Usage: rasa marker upload --config=<path-to-config-file> -d=<path-to-domain-file> -rasa-pro-services-url=<url>.
Enhance the validation of the anonymization key in endpoints.yaml by introducing checks for required fields and duplicate IDs.

Bugfixes

Fix running custom form validation to update required slots at form activation when prefilled slots consist only of slots that are not requested by the form.
Anonymize rasa_events structlog key.
Fixes issue with uploading locally trained model to a cloud rasa-plus instance where the conversation does not go as expected because slots don't get set correctly, e.g. an error is logged Tried to set non existent slot 'placeholder_slot_name'. Make sure you added all your slots to your domain file.. This is because the updated domain during the cloud upload did not get passed to the wrapped tracker store of the AuthRetryTrackerStore rasa-plus component. The fix was to add domain property and setter methods to the AuthRetryTrackerStore component.
When using rasa studio upload, if no specific intents or entities are specified by the user, the update will now include all available intents or entities.

Improved Documentation

Explicitly set Node.js version to 12.x in order to run Docusaurus.
Update obselete commands in docs README.
Correct docker image name for deploy-rasa-pro-services in docs.
Update Compatibility Matrix.
Implement rasa data split stories to split stories data into train/test parts.
Updated knowledge base action docs to reflect the improvements made in knowledge base actions in Rasa Pro 3.6 version. This enhancement now allows users to query for the object attribute without the need for users to request a list of objects of a particular object type beforehand. The docs update mentions this under :::info New in 3.6 section.
Fix dead link in Analytics documentation.

Miscellaneous internal changes

Miscellaneous internal changes.

[3.5.12] - 2023-06-23

Rasa Pro 3.5.12 (2023-06-23)

Bugfixes

Rich responses containing buttons with parentheses characters are now correctly parsed. Previously any characters found between the first identified pair of () in response button took precedence.

Miscellaneous internal changes

Miscellaneous internal changes.

[3.5.11] - 2023-06-08

Rasa Pro 3.5.11 (2023-06-08)

Bugfixes

Fix running custom form validation to update required slots at form activation when prefilled slots consist only of slots that are not requested by the form.

[3.5.10] - 2023-05-23

Rasa Pro 3.5.10 (2023-05-23)

Improved Documentation

Added documentation for spaces alpha

[3.5.9] - 2023-05-19

Rasa Pro 3.5.9 (2023-05-19)

No significant changes.

[3.5.8] - 2023-05-12

Rasa Pro 3.5.8 (2023-05-12)

Bugfixes

Explicitly handled BufferError exception - Local: Queue full in Kafka producer.

[3.5.7] - 2023-05-09

Rasa Pro 3.5.7 (2023-05-09)

Bugfixes

SlotSet events will be emitted when the value set by the custom action is the same as the existing value of the slot. This was fixed for AugmentedMemoizationPolicy to work properly with truncated trackers.
To restore the previous behaviour, the custom action can return a SlotSet only if the slot value has changed. For example,
class CustomAction(Action):
def name(self) -> Text:
return "custom_action"

def run(self, dispatcher: CollectingDispatcher,
tracker: Tracker,
domain: Dict[Text, Any]) -> List[Dict[Text, Any]]:
# current value of the slot
slot_value = tracker.get_slot('my_slot')

# value of the entity
# this is parsed from the user utterance
entity_value = next(tracker.get_latest_entity_values("entity_name"), None)

if slot_value != entity_value:
return[SlotSet("my_slot", entity_value)]

[3.5.6] - 2023-04-28

Rasa Pro 3.5.6 (2023-04-28)

Bugfixes

Addresses Regular Expression Denial of Service vulnerability in slack connector (https://owasp.org/www-community/attacks/Regular_expression_Denial_of_Service_-_ReDoS)
Fix parsing of RabbitMQ URL provided in endpoints.yml file to include vhost path and query parameters. Re-allows inclusion of credentials in the URL as a regression fix (this was supported in 2.x).

[3.5.5] - 2023-04-20

Rasa Pro 3.5.5 (2023-04-20)

Bugfixes

Allow slot mapping parameter intent to accept a list of intent names (as strings), in addition to accepting an intent name as a single string.
Fix BlockingIOError when running rasa shell on utterances with more than 5KB of text.
Use ruamel.yaml round-trip loader in order to preserve all comments after appending assistant_id to config.yml.
Fix AttributeError: 'NoneType' object has no attribute 'send_response' caused by retrieving tracker via GET /conversations/{conversation_id}/tracker endpoint when action_session_start is customized in a custom action. This was addressed by passing an instance of CollectingOutputChannel to the method retrieving the tracker from the MessageProcessor.

Improved Documentation

Updated AWS model loading documentation to indicate what should AWS_ENDPOINT_URL environment variable be set to. Added integration test for AWS model loading.
Updated Rasa Pro Services documentation to add KAFKA_SSL_CA_LOCATION environment variable. Allows connections over SSL to Kafka
Added note to CLI documentation to address encoding and color issues on certain Windows terminals

Miscellaneous internal changes

Miscellaneous internal changes.

[3.5.4] - 2023-04-05

Rasa Pro 3.5.4 (2023-04-05)

Bugfixes

Fix issue with failures while publishing events to RabbitMQ after a RabbitMQ restart. The fix consists of pinning aio-pika dependency to 8.2.3, since this issue was introduced in aio-pika v8.2.4.
Patch redis Race Conditiion vulnerability.

Miscellaneous internal changes

Miscellaneous internal changes.

[3.5.3] - 2023-03-30

Rasa Pro 3.5.3 (2023-03-30)

Improved Documentation

Add new Rasa Pro page in docs, together with minimal content changes.

[3.5.2] - 2023-03-30

Rasa Pro 3.5.2 (2023-03-30)

Improvements

Add a self-reference of the synonym in the EntitySynonymMapper to handle entities extracted in a casing different to synonym case. (For example if a synonym austria is added, entities extracted with any alternate casing of the synonym will also be mapped to austria). It addresses ATO-616

Bugfixes

Make custom actions inheriting from rasa-sdk FormValidationAction parent class an exception of the selective_domain rule and always send them domain.
Fix 2 issues detected with the HTTP API:
- The GET /conversations/{conversation_id}/tracker endpoint was not returning the tracker with all sessions when include_events query parameter was set to ALL. The fix constituted in using TrackerStore.retrieve_full_tracker method instead of TrackerStore.retrieve method in the function handling the GET /conversations/{conversation_id}/tracker endpoint. Implemented or updated this method across all tracker store subclasses.
- The GET /conversations/{conversation_id}/story endpoint was not returning all the stories for all sessions when all_sessions query parameter was set to true. The fix constituted in using all events of the tracker to be converted in stories instead of only the applied_events.

Improved Documentation

Add documentation for secrets managers.

[3.5.1] - 2023-03-24

Rasa Pro 3.5.1 (2023-03-24)

Bugfixes

Fixes training DIETCLassifier on the GPU.
A deterministic GPU implementation of SparseTensorDenseMatmulOp is not currently available

Improved Documentation

Updated Test your assistant section to describe the new end-to-end testing feature. Also updated CLI and telemetry reference docs.
Update Compatibility Matrix.

[3.5.0] - 2023-03-21

Rasa Pro 3.5.0 (2023-03-21)

Features

Add a new required key (assistant_id) to config.yml to uniquely identify assistants in deployment. The assistant identifier is extracted from the model metadata and added to the metadata of all dialogue events. Re-training will be required to include the assistant id in the event metadata.
If the assistant identifier is missing from the config.yml or the default identifier value is not replaced, a random identifier is generated during each training.
An assistant running without an identifier will issue a warning that dialogue events without identifier metadata will be streamed to the event broker.
End-to-end testing is an enhanced and comprehensive CLI-based testing tool that allows you to test conversation scenarios with different pre-configured contexts, execute custom actions, verify response texts or names, and assert when slots are filled. It is available ysing the new rasa test e2e command.
You can now store your assistant's secrets in an external credentials manager. In this release, Rasa Pro currently supports credentials manager for the Tracker Store with HashiCorp Vault.

Improvements

Add capability to send compressed body in HTTP request to action server. Use COMPRESS_ACTION_SERVER_REQUEST=True to turn the feature on.

Bugfixes

Address potentially missing events with Pika consumer due to weak references on asynchronous tasks, as specifcied in Python official documentation.
Sets a global seed for numpy, TensorFlow, keras, Python and CuDNN, to ensure consistent random number generation.

Improved Documentation

Clarify in the docs, how rules are designed and how to use this behaviour to abort a rule

Miscellaneous internal changes

Miscellaneous internal changes.

[3.4.14] - 2023-06-08

Rasa Pro 3.4.14 (2023-06-08)

Bugfixes

Fix running custom form validation to update required slots at form activation when prefilled slots consist only of slots that are not requested by the form.

[3.4.13] - 2023-05-19

Rasa Pro 3.4.13 (2023-05-19)

No significant changes.

[3.4.12] - 2023-05-12

Rasa Pro 3.4.12 (2023-05-12)

Bugfixes

Explicitly handled BufferError exception - Local: Queue full in Kafka producer.

[3.4.11] - 2023-05-09

Rasa Pro 3.4.11 (2023-05-09)

Bugfixes

Fix parsing of RabbitMQ URL provided in endpoints.yml file to include vhost path and query parameters. Re-allows inclusion of credentials in the URL as a regression fix (this was supported in 2.x).
SlotSet events will be emitted when the value set by the custom action is the same as the existing value of the slot. This was fixed for AugmentedMemoizationPolicy to work properly with truncated trackers.
To restore the previous behaviour, the custom action can return a SlotSet only if the slot value has changed. For example,
class CustomAction(Action):
def name(self) -> Text:
return "custom_action"

def run(self, dispatcher: CollectingDispatcher,
tracker: Tracker,
domain: Dict[Text, Any]) -> List[Dict[Text, Any]]:
# current value of the slot
slot_value = tracker.get_slot('my_slot')

# value of the entity
# this is parsed from the user utterance
entity_value = next(tracker.get_latest_entity_values("entity_name"), None)

if slot_value != entity_value:
return[SlotSet("my_slot", entity_value)]

Miscellaneous internal changes

Miscellaneous internal changes.

[3.4.10] - 2023-04-17

Rasa Pro 3.4.10 (2023-04-17)

Miscellaneous internal changes

Miscellaneous internal changes.

[3.4.9] - 2023-04-05

Miscellaneous internal changes

Miscellaneous internal changes.

[3.4.8] - 2023-04-03

Rasa Pro 3.4.8 (2023-04-03)

Bugfixes

Fix issue with failures while publishing events to RabbitMQ after a RabbitMQ restart. The fix consists of pinning aio-pika dependency to 8.2.3, since this issue was introduced in aio-pika v8.2.4.

[3.4.7] - 2023-03-30

Rasa Pro 3.4.7 (2023-03-30)

Improvements

Add a self-reference of the synonym in the EntitySynonymMapper to handle entities extracted in a casing different to synonym case. (For example if a synonym austria is added, entities extracted with any alternate casing of the synonym will also be mapped to austria). It addresses ATO-616

Bugfixes

Fix 2 issues detected with the HTTP API:
- The GET /conversations/{conversation_id}/tracker endpoint was not returning the tracker with all sessions when include_events query parameter was set to ALL. The fix constituted in using TrackerStore.retrieve_full_tracker method instead of TrackerStore.retrieve method in the function handling the GET /conversations/{conversation_id}/tracker endpoint. Implemented or updated this method across all tracker store subclasses.
- The GET /conversations/{conversation_id}/story endpoint was not returning all the stories for all sessions when all_sessions query parameter was set to true. The fix constituted in using all events of the tracker to be converted in stories instead of only the applied_events.
Make custom actions inheriting from rasa-sdk FormValidationAction parent class an exception of the selective_domain rule and always send them domain.

[3.4.6] - 2023-03-16

Rasa Pro 3.4.6 (2023-03-16)

Bugfixes

Fixes CountVectorFeaturizer to train when min_df != 1.

[3.4.5] - 2023-03-09

Rasa Pro 3.4.5 (2023-03-09)

Bugfixes

Check unresolved slots before initiating model training.
Fixes the bug when a slot (with from_intent mapping which contains no input for intent parameter) will no longer fill for any intent that is not under the not_intent parameter.
Fix validation metrics calculation when batch_size is dynamic.

Improved Documentation

Add a link to an existing docs section on how to test the audio channel on localhost.

[3.4.4] - 2023-02-17

Rasa Pro 3.4.4 (2023-02-17)

Improvements

Add capability to send compressed body in HTTP request to action server. Use COMPRESS_ACTION_SERVER_REQUEST=True to turn the feature on.

Bugfixes

Fix the error which resulted during merging multiple domain files where at least one of them contains custom actions that explicitly need send_domain set as True in the domain.

[3.4.3] - 2023-02-14

Rasa Pro 3.4.3 (2023-02-14)

Improvements

Add support for custom RulePolicy.
Add capability to select which custom actions should receive domain when they are invoked.

Bugfixes

Fix calling the form validation action twice for the same user message triggering a form.
Fix conditional response does not check other conditions if first condition matches.

Improved Documentation

Add section in tracker store docs to document the fallback tracker store mechanism.

[3.4.2] - 2023-01-27

Rasa Pro 3.4.2 (2023-01-27)

Bugfixes

Decision to publish docs should not consider next major and minor alpha release versions.
Exit training/running Rasa model when SpaCy runtime version is not compatible with the specified SpaCy model version.
The new custom logging feature was not working due to small syntax issue in the argparse level.
Previously, the file name passed using the argument --logging-config-file was never retrieved so it never creates the new custom config file with the desired formatting.

Miscellaneous internal changes

Miscellaneous internal changes.

[3.4.1] - 2023-01-19

Rasa Pro 3.4.1 (2023-01-19)

Bugfixes

Changed categorical slot comparison to be case insensitive.
Exit training when transformer_size is not divisible by the number_of_attention_heads parameter and update the transformer documentations.

Improved Documentation

Update compatibility matrix between Rasa-plus and Rasa Pro services.

[3.4.0] - 2022-12-14

Rasa Pro 3.4.0 (2022-12-14)

Features

Add metadata to Websocket channel. Messages can now include a metadata object which will be included as metadata to Rasa. The metadata can be supplied on a user configurable key with the metadata_key setting in the socketio section of the credentials.yml.
Use a new IVR Channel to connect your assistant to AudioCodes VoiceAI Connect.

Improvements

Added ./docker/Dockerfile_pretrained_embeddings_spacy_it to include Spacy's Italian pre-trained model it_core_news_md.
Replace kafka-python dependency with confluent-kafka async Producer API.
Add support for Python 3.10 version.
Added CLI option --logging-config-file to enable configuration of custom logs formatting.

Bugfixes

Implements a new CLI option --jwt-private-key required to have complete support for asymmetric algorithms as specified originally in the docs.

Improved Documentation

Clarify in the documentation how to write testing stories if a user presses a button with payload.
Clarify prioritisation of used slot asking option in forms in documentation.

Miscellaneous internal changes

Miscellaneous internal changes.

[3.3.8] - 2023-04-06

Miscellaneous internal changes

Miscellaneous internal changes.

[3.3.7] - 2023-03-31

Improvements

Add a self-reference of the synonym in the EntitySynonymMapper to handle entities extracted in a casing different to synonym case. (For example if a synonym austria is added, entities extracted with any alternate casing of the synonym will also be mapped to austria). It addresses ATO-616

Bugfixes

Fix issue with failures while publishing events to RabbitMQ after a RabbitMQ restart. The fix consists of pinning aio-pika dependency to 8.2.3, since this issue was introduced in aio-pika v8.2.4.
Fix 2 issues detected with the HTTP API:
- The GET /conversations/{conversation_id}/tracker endpoint was not returning the tracker with all sessions when include_events query parameter was set to ALL. The fix constituted in using TrackerStore.retrieve_full_tracker method instead of TrackerStore.retrieve method in the function handling the GET /conversations/{conversation_id}/tracker endpoint. Implemented or updated this method across all tracker store subclasses.
- The GET /conversations/{conversation_id}/story endpoint was not returning all the stories for all sessions when all_sessions query parameter was set to true. The fix constituted in using all events of the tracker to be converted in stories instead of only the applied_events.

[3.3.6] - 2023-03-09

Rasa Pro 3.3.6 (2023-03-09)

Bugfixes

Fixes the bug when a slot (with from_intent mapping which contains no input for intent parameter) will no longer fill for any intent that is not under the not_intent parameter.
Fix validation metrics calculation when batch_size is dynamic.

[3.3.5] - 2023-02-21

No significant changes.

[3.3.4] - 2023-02-14

Rasa Pro 3.3.4 (2023-02-14)

Improvements

Add capability to send compressed body in HTTP request to action server. Use COMPRESS_ACTION_SERVER_REQUEST=True to turn the feature on.
Add support for custom RulePolicy.

[3.3.3] - 2022-12-01

Bugfixes

Bypass Windows path length restrictions upon saving and reading a model archive in rasa.engine.storage.LocalModelStorage.

Improvements

Updated tensorflow to 2.8.4.

[3.3.2] - 2022-11-30

Improvements

Added support for camembert french bert model

Bugfixes

Fixes RuntimeWarning: coroutine 'Bot.set_webhook' was never awaited issue encountered when starting the rasa server, which caused the Telegram bot to be unresponsive.

Improved Documentation

The documentation was updated for Buttons using messages that start with '/'. Previously, it wrongly stated that messages with '/' bypass NLU, which is not the case.
Add documentation for Rasa Pro Services upgrades.

[3.3.1] - 2022-11-09

Improved Documentation

Add docs on how to set up additional data lakes for Rasa Pro analytics pipeline.
Update migration guide and form docs with prescriptive recommendation on how to implement dynamic forms with custom slot mappings.

Improvements

Updated numpy and scikit learn version to fix vulnerabilities of those dependencies

[3.3.0] - 2022-10-24

Features

Tracing capabilities for your Rasa Pro assistant. Distributed tracing tracks requests as they flow through a distributed system (in this case: a Rasa assistant), sending data about the requests to a tracing backend which collects all trace data and enables inspecting it. With this version of the Tracing feature, Rasa Pro supports OpenTelemetry.
ConcurrentRedisLockStore is a new lock store that uses Redis as a persistence layer and is safe for use with multiple Rasa server replicas.

Improvements

Added option --offset-timestamps-by-seconds to offset the timestamp of events when using rasa export
Rasa supports native installations on Apple Silicon (M1 / M2). Please follow the installation instructions and take a look at the limitations.
Caching Message and Features fingerprints unless they are altered, saving up to 2/3 of fingerprinting time in our tests.
Added package versions of component dependencies as an additional part of fingerprinting calculation. Upgrading an dependency will thus lead to a retraining of the component in the future. Also, by changing fingerprint calculation, the next training after this change will be a complete retraining.
Export events continuously rather than loading all events in memory first when using rasa export. Events will be streamed right from the start rather than loading all events first and pushing them to the broker afterwards.
Adds new dependency pluggy, with which it was possible to implement new plugin functionality. This plugin manager enables the extension and/or enhancement of the Rasa command line interface with functionality made available in the rasa-plus package.

Bugfixes

Fix BlockingIOError when running rasa interactive, after the upgrade of prompt-toolkit dependency.
Fixes a bug that lead to initial slot values being incorporated into all rules by default, thus breaking most rules when the slot value changed from its initial value
Made logistic regression classifier output a proper intent ranking and made ranking length configurable

Deprecations and Removals

Remove code related to Rasa X local mode as it is deprecated and scheduled for removal.

Miscellaneous internal changes

Miscellaneous internal changes.

[3.2.13] - 2023-03-09

Rasa 3.2.13 (2023-03-09)

Bugfixes

Fix validation metrics calculation when batch_size is dynamic.
Fixes the bug when a slot (with from_intent mapping which contains no input for intent parameter) will no longer fill for any intent that is not under the not_intent parameter.

[3.2.12] - 2023-02-21

Features

Add metadata to Websocket channel. Messages can now include a metadata object which will be included as metadata to Rasa. The metadata can be supplied on a user configurable key with the metadata_key setting in the socketio section of the credentials.yml.

Improvements

Add capability to send compressed body in HTTP request to action server. Use COMPRESS_ACTION_SERVER_REQUEST=True to turn the feature on.

Bugfixes

Decision to publish docs should not consider next major and minor alpha release versions.

[3.2.11] - 2022-12-05

Improvements

Caching Message and Features fingerprints unless they are altered, saving up to 2/3 of fingerprinting time in our tests.

Bugfixes

Implements a new CLI option --jwt-private-key required to have complete support for asymmetric algorithms as specified originally in the docs.

[3.2.10] - 2022-09-29

Bugfixes

Fixes scenarios in which a slot with from_trigger_intent mapping that specifies an active_loop condition was being filled despite that active_loop not being activated. In addition, fixes scenario in which a slot with from_trigger_intent mapping without a specified active_loop mapping condition is only filled if the form gets activated. Removes unnecessary validation warning that a slot with from_trigger_intent and a mapping condition should be included in the form's required_slots.
Fixed a bug where DIETClassier crashed during training when both masked language modelling and evaluation during training were used.

Improved Documentation

Rasa SDK documentation lives now in Rasa Open Source documentation under the Rasa SDK category.

Miscellaneous internal changes

Miscellaneous internal changes.

[3.2.9] - 2022-09-09

Yanked.

[3.2.8] - 2022-09-08

Bugfixes

Fix bug where KeywordIntentClassifier overrides preceding intent classifiers' predictions although the KeyWordIntentClassifier was not matching any keywords.

[3.2.7] - 2022-08-31

Improvements

Improve rasa data validate command so that it uses custom importers when they are defined in config file.

Bugfixes

Re-instates the REST channel metadata feature. Metadata can be provided on the metadata key.

[3.2.6] - 2022-08-12

Bugfixes

This fix makes sure that when a Domain object is loaded from multiple files where one file specifies a custom session config and the rest do not, the default session configuration does not override the custom session config.

Miscellaneous internal changes

Miscellaneous internal changes.

[3.2.5] - 2022-08-05

Bugfixes

Fix KeyError which resulted when action_two_stage_fallback got executed in a project whose domain contained slot mappings.
Fixes regression in which slot mappings were prioritized according to reverse order as they were listed in the domain, instead of in order from first to last, as was implicitly expected in 2.x. Clarifies this implicit priority order in the docs.
Enables the dispatching of bot messages returned as events by slot validation actions.

[3.2.4] - 2022-07-21

Bugfixes

Added session_config key as valid domain key during domain loading from directory containing a separate domain file with session configuration.
Run default action action_extract_slots after a custom action returns a UserUttered event to fill any applicable slots.
Handle the case when an EndpointConfig object is given as parameter to the AwaitableTrackerStore.create() method.

[3.2.3] - 2022-07-18

Bugfixes

- Fixed error in creating response when slack sends retry messages. Assigning None to response.text caused TypeError: Bad body type. Expected str, got NoneType.
- Fixed Slack triggering timeout after 3 seconds if the action execution is too slow. Running on_new_message as an asyncio background task instead of a blocking await fixes this by immediately returning a response with code 200.
Revert change in #10295 that removed running the form validation action on activation of the form before the loop is active.
SlotSet events will be emitted when the value set by the current user turn is the same as the existing value.
Previously, ActionExtractSlots would not emit any SlotSet events if the new value was the same as the existing one. This caused the augmented memoization policy to lose these slot values when truncating the tracker.

[3.2.2] - 2022-07-05

Improved Documentation

Update documentation for customizable classes such as tracker stores, event brokers and lock stores.

Miscellaneous internal changes

Miscellaneous internal changes.

[3.2.1] - 2022-06-17

Bugfixes

Fix failed check in rasa data validate that verifies forms in rules or stories are consistent with the domain when the rule or story contains a default action as active_loop step.

[3.2.0] - 2022-06-14

Deprecations and Removals

NLU training data in JSON format is deprecated and will be removed in Rasa Open Source 4.0. Please use rasa data convert nlu -f yaml --data <path to NLU data> to convert your NLU JSON data to YAML format before support for NLU JSON data is removed.

Improvements

Make TrackerStore interface methods asynchronous and supply an AwaitableTrackerstore wrapper for custom tracker stores which do not implement the methods as asynchronous.
Added flag use_gpu to TEDPolicy and UnexpecTEDIntentPolicy that can be used to enable training on CPU even when a GPU is available.
Add --endpoints command line parameter to rasa train parser.

Bugfixes

The azure botframework channel now validates the incoming JSON Web Tokens (including signature).
Previously, JWTs were not validated at all.
rasa shell now outputs custom json unicode characters instead of \uxxxx codes

Improved Documentation

Clarify aspects of the API spec GET /status endpoint: Correct response schema for model_id - a string, not an object.
GET /conversations/{conversation_id}/tracker: Describe each of the enum options for include_events query parameter
POST & PUT /conversations/{conversation_id}/tracker/eventss: Events schema added for each event type
GET /conversations/{conversation_id}/story: Clarified the all_sessions query parameter and default behaviour.
POST /model/test/intents : Remove JSON payload option since it is not supported
POST /model/parse: Explain what emulation_mode is and how it affects response results

Miscellaneous internal changes

Miscellaneous internal changes.

[3.1.7] - 2022-08-30

Miscellaneous internal changes

Miscellaneous internal changes.

[3.1.6] - 2022-07-20

Bugfixes

Run default action action_extract_slots after a custom action returns a UserUttered event to fill any applicable slots.

[3.1.5] - 2022-07-15

Bugfixes

SlotSet events will be emitted when the value set by the current user turn is the same as the existing value.
Previously, ActionExtractSlots would not emit any SlotSet events if the new value was the same as the existing one. This caused the augmented memoization policy to lose these slot values when truncating the tracker.

[3.1.4] - 2022-06-21 No significant changes.

Upgrade dependent libraries with security vulnerabilities (Pillow, TensorFlow, ujson).

[3.1.3] - 2022-06-17

Bugfixes

The azure botframework channel now validates the incoming JSON Web Tokens (including signature).
Previously, JWTs were not validated at all.
Backports fix for failed check in rasa data validate that verifies forms in rules or stories are consistent with the domain when the rule or story contains a default action as active_loop step.

[3.1.2] - 2022-06-08

Miscellaneous internal changes

Miscellaneous internal changes.

[3.1.1] - 2022-06-03

Bugfixes

Remove warning for Rasa X localmode not being supported when the --production flag is present.
Pin requirement for scipy<1.8.0 since scipy>=1.8.0 is not backward compatible with scipy<1.8.0 and additionally requires Python>=3.8, while Rasa supports Python 3.7 as well.
Fix the extraction of values for slots with mapping conditions from trigger intents that activate a form, which was possible in 2.x.

[3.1.0] - 2022-03-25

Features

Add configuration options (via env variables) for library logging.
Support other recipe types.
This pull request also adds support for graph recipes, see details at https://rasa.com/docs/rasa/model-configuration and check Graph Recipe page.
Graph recipe is a raw format for specifying executed graph directly. This is useful if you need a more powerful way to specify your model creation.
Added optional ssl_keyfile, ssl_certfile, and ssl_ca_certs parameters to the Redis tracker store.
Added LogisticRegressionClassifier to the NLU classifiers.
This model is lightweight and might help in early prototyping. The training times typically decrease substantially, but the accuracy might be a bit lower too.
Added support for Python 3.9.

Improvements

Bump TensorFlow version to 2.7.
caution
We can't guarantee the exact same output and hence model performance if your configuration uses LanguageModelFeaturizer. This applies to the case where the model is re-trained with the new rasa open source version without changing the configuration, random seeds, and data as well as to the case where a model trained with a previous version of rasa open source is loaded with this new version for inference.
We suggest training a new model if you are upgrading to this version of Rasa Open Source.
Make rasa data validate check for duplicated intents, forms, responses and slots when using domains split between multiple files.
Add an influence_conversation flag to entites to provide a shorthand for ignoring an entity for all intents.
Add --request-timeout command line argument to rasa shell, allowing users to configure the time a request can take before it's terminated.

Bugfixes

Validate regular expressions in nlu training data configuration.
Unset the default values for num_threads and finetuning_epoch_fraction to None in order to fix cases when CLI defaults override the data from config.
Update rasa data validate to not fail when active_loop is null
Fixes Domain loading when domain config uses multiple yml files.
Previously not all configures attributes were necessarily known when merging Domains, and in the case of entities were not being properly assigned to intents.
Fix max_history truncation in AugmentedMemoizationPolicy to preserve the most recent UserUttered event. Previously, AugmentedMemoizationPolicy failed to predict next action after long sequences of actions (longer than max_history) because the policy did not have access to the most recent user message.
Add RASA_ENVIRONMENT header in Kafka only if the environmental variable is set.
Merge domain entities as lists of dicts, not lists of lists to support entity roles and groups across multiple domains.
Add an option to specify --domain for rasa test nlu CLI command.

Improved Documentation

Fixed an over-indent in the Tokenizers section of the Components page of the docs.

Miscellaneous internal changes

Miscellaneous internal changes.

[3.0.10] - 2022-03-15## [3.0.10] - 2022-03-15

Bugfixes

Fix broken conversion from Rasa JSON NLU data to Rasa YAML NLU data.

[3.0.9] - 2022-03-11

Bugfixes

Fix Socket IO connection issues by upgrading sanic to v21.12.
The bug is caused by an invalid function signature and is fixed in v21.12.
This update brings some deprecations in sanic:
- Sanic and Blueprint may no longer have arbitrary properties attached to them
  - Fixed this by moving user defined properties to the instance.ctx object
- Sanic and Blueprint forced to have compliant names
  - Fixed this by using string literal names instead of the module's name via __name__
- sanic.exceptions.abort is Deprecated
  - Fixed by replacing it with sanic.exceptions.SanicException
- sanic.response.StreamingHTTPResponse is deprecated
  - Fixed by replacing it with sanic.response.ResponseStream
Update rasa data validate to not fail when active_loop is null

Improved Documentation

Updated the model_confidence parameter in TEDPolicy and DIETClassifier. The linear_norm is removed as it is no longer supported.
Added an additional step to Receiving Messages section in slack.mdx documentation. After a slack update this additional step is needed to allow direct messages to the bot.
Backport the updated deployment docs to 3.0.x.

[3.0.8] - 2022-02-11

Improvements

Allow single tokens in rasa end-to-end test files to be annotated with multiple entities.
Some entity extractors (s.a. RegexEntityExtractor) can generate multiple entities from a single expression. Before this change, rasa test would fail in this case, because test stories could not be annotated correctly. New annotation option is
stories:
- story: Some story
steps:
- user: |
cancel my [iphone][{"entity":"iphone", "value":"iphone"},{"entity":"smartphone", "value":"true"}{"entity":"mobile_service", "value":"true"}]
intent: cancel_contract

Bugfixes

Fixed a bug where the POST /conversations/<conversation_id>/tracker/events endpoint repeated session start events when appending events to a new tracker.

[3.0.7] - 2022-02-09

Bugfixes

Checkpoint weights were never loaded before. Implements overwriting checkpoint weights to the final model weights after training of DIETClassifier, ResponseSelector and TEDPolicy.
Allow arbitrary keys under each slot in the domain to allow for custom slot types.
Fix issue with missing running event loop in MainThread when starting Rasa Open Source for Rasa X with JWT secrets.

[3.0.6] - 2022-01-28

Deprecations and Removals

Removed CompositionView.

Bugfixes

Fixes a bug which was caused by DIETClassifier (ResponseSelector, SklearnIntentClassifier and CRFEntityExtractor have the same issue) trying to process message which didn't have required features. Implements removing unfeaturized messages for the above-mentioned components before training and prediction.
Enable slots with from_entity mapping that are not part of a form's required slots to be set during active loop.
Catch ValueError for any port values that cannot be cast to integer and re-raise as RasaException during the initialisation of SQLTrackerStore.
Use tf.function for model prediction to improve inference speed.
Tie prompt-toolkit to ^2.0 to fix rasa-shell.

Improved Documentation

Update dynamic form behaviour docs section with an example on how to override required_slots in case of removal of a form required slot.

[3.0.5] - 2022-01-19

Bugfixes

Corrects transformer_size parameter value (None by default) with a default size during loading in case ResponseSelector contains transformer layers.

Miscellaneous internal changes

Miscellaneous internal changes.

[3.0.4] - 2021-12-22

Miscellaneous internal changes

Miscellaneous internal changes.

[3.0.3] - 2021-12-16

Bugfixes

Copy lookup tables to train and test folds in cross validation. Before, the generated folds did not have a copy of the lookup tables from the original NLU data, so that RegexEntityExtractor could not recognize any entities during the evaluation.
Do not print warning when subintent actions have response.

Miscellaneous internal changes

Miscellaneous internal changes.

[3.0.2] - 2021-12-09

Bugfixes

Update SQLAlchemy version to a compatible one in case other dependencies force a lower version.
Fix overriding of default config with custom config containing nested dictionaries. Before, the keys of a nested dictionary in the default config that were not specified in the custom config got lost.
Add UserWarning to alert users trying to run rasa x CLI command with rasa version 3.0 or higher that rasa-x currently doesn't support rasa 3.x.

Improved Documentation

Added note to the slot mappings section of the migration guide to recommend checking dynamic form behavior on migrated assistants.

[3.0.1] - 2021-12-02

Bugfixes

Fix previous slots getting filled after a restart. Previously events were searched from oldest to newest which meant we would find first occurrence of a message and use slots from thereafter. Now we use the last utterance or the restart event.

Miscellaneous internal changes

Miscellaneous internal changes.

[3.0.0] - 2021-11-23

Deprecations and Removals

Remove backwards compatibility code with Rasa Open Source 1.x, Rasa Enterprise 0.35, and other outdated backwards compatibility code in rasa.cli.x, rasa.core.utils, rasa.model_testing, rasa.model_training and rasa.shared.core.events.
Removed Python 3.6 support as it reaches its end of life in December 2021.
Follow through on removing deprecation warnings for synchronous EventBroker methods.
Follow through on deprecation warnings for policies and policy ensembles.
Follow through on deprecation warnings for rasa.shared.data.
Follow through on deprecation warnings for the Domain. Most importantly this will enforce the schema of the forms section in the domain file. This further includes the removal of the UnfeaturizedSlot type.
Remove deprecated change_form_to and set_form_validation methods from DialogueStateTracker.
Remove the support of Markdown training data format. This includes:
- reading and writing of story files in Markdown format
- reading and writing of NLU data in Markdown format
- reading and writing of retrieval intent data in Markdown format
- all the Markdown examples and tests that use Markdown
Removed automatic renaming of deprecated action action_deactivate_form to action_deactivate_loop. action_deactivate_form will just be treated like other non-existing actions from now on.
Remove deprecated sorted_intent_examples method from TrainingData.
Raising RasaException instead of deprecation warning when using class_from_module_path for loading types other than classes.
Specifying the retrieve_events_from_previous_conversation_sessions kwarg for the any TrackerStore was deprecated and has now been removed. Please use the retrieve_full_tracker() method instead.
Deserialization of pickled trackers was deprecated and has now been removed. Rasa will perform any future save operations of trackers using json serialisation.
Removed catch for missing (deprecated) session_date when saving trackers in DynamoTrackerStore.
Removed the deprecated dialogue policy state featurizers: BinarySingleStateFeature and LabelTokenizerSingleStateFeaturizer.
Removed the deprecated method encode_all_actions of SingleStateFeaturizer. Use encode_all_labels instead.
Follow through with removing deprecated policies: FormPolicy, MappingPolicy, FallbackPolicy, TwoStageFallbackPolicy, and SklearnPolicy.
Remove warning about default value of max_history in MemoizationPolicy. The default value is now None.
Follow through on deprecation warnings and remove code, tests, and docs for ConveRTTokenizer, LanguageModelTokenizer and HFTransformersNLP.
rasa.shared.nlu.training_data.message.Message method get_combined_intent_response_key has been removed. get_full_intent should now be used in its place.
Intent IDs sent with events (to kafka and elsewhere) have been removed, intent names can be used instead (or if numerical values are needed for backwards compatibility, one can also hash the names to get previous ID values, ie. hash(intent_name) is the old ID values). Intent IDs have been removed because they were providing no extra value and integers that large were problematic for some event broker implementations.
Remove loop argument from train method in rasa. This argument became redundant when Python 3.6 support was dropped as asyncio.run became available in Python 3.7.
Remove template_variables and e2e arguments from get_stories method of TrainingDataImporter. This argument was used in Markdown data format and became redundant once Markdown was removed.
weight_sparsity has been removed. Developers should replace it with connection_density in the following way: connection_density = 1-weight_sparsity.
softmax is not available as a loss_type anymore.
The linear_norm option has been removed as possible value for model_confidence. Please, use softmax instead.
minibatch has been removed as a value for tensorboard_log_level, use batch instead.
Removed deprecation warnings related to the removed component config values.
Follow through on removing deprecation warnings raised in these modules:
- rasa/server.py
- rasa/core/agent.py
- rasa/core/actions/action.py
- rasa/core/channels/mattermost.py
- rasa/core/nlg/generator.py
- rasa/nlu/registry.py
Remove deprecation warnings associated with the "number_additional_patterns" parameter of rasa.nlu.featurizers.sparse_featurizer.regex_featurizer.RegexFeaturizer. This parameter is no longer needed for incremental training.
Remove deprecation warnings associated with the "additional_vocabulary_size" parameter of rasa.nlu.featurizers.sparse_featurizer.count_vectors_featurizer.CountVectorsFeaturizer. This parameter is no longer needed for incremental training.
Remove deprecated functions training_states_actions_and_entities and training_states_and_actions from rasa.core.featurizers.tracker_featurizers.TrackerFeaturizer. Use training_states_labels_and_entities and training_states_and_labels instead.
Follow through on deprecation warning for NGramFeaturizer
The CLI commands rasa data convert config and rasa data convert responses which converted from the Rasa Open Source 1 to the Rasa Open Source 2 formats were removed. Please use a Rasa Open Source 2 installation to convert your training data before moving to Rasa Open Source 3.
rasa.core.agent.Agent.visualize was removed. Please use rasa visualize or rasa.core.visualize.visualize instead.
Removed slot auto-fill functionality, making the key invalid to use in the domain file. The auto_fill parameter was also removed from the constructor of the Slot class. In order to continue filling slots with entities of the same name, you now have to define a from_entity mapping in the slots section of the domain. To learn more about how to migrate your 2.0 assistant, please read the migration guide.

Features

Training data version upgraded from 2.0 to 3.0 due to breaking changes to format in Rasa Open Source 3.0
A new experimental feature called Markers has been added. Markers allow you to define points of interest in conversations as a set of conditions that need to be met. A new command rasa evaluate markers allows you to apply these conditions to your existing tracker stores and outputs the points at which the conditions were satisfied.
Rasa Open Source now uses the model configuration to build a
directed acyclic graph. This graph describes the dependencies between the items in your model configuration and how data flows between them. This has two major benefits:
- Rasa Open Source can use the computational graph to optimize the execution of your model. Examples for this are efficient caching of training steps or executing independent steps in parallel.
- Rasa Open Source can represent different model architectures flexibly. As long as the graph remains acyclic Rasa Open Source can in theory pass any data to any graph component based on the model configuration without having to tie the underlying software architecture to the used model architecture.
This change required changes to custom policies and custom NLU components. See the documentation for a detailed migration guide.
Added explicit mechanism for slot filling that allows slots to be set and/or updated throughout the conversation. This mechanism is enabled by defining global slot mappings in the slots section of the domain file.
In order to support this new functionality, implemented a new default action: action_extract_slots. This new action runs after each user turn and checks if any slots can be filled with information extracted from the last user message based on defined slot mappings.
Since slot mappings were moved away from the forms section of the domain file, converted the form's required_slots to a list of slot names. In order to restrict certain mappings to a form, you can now use the conditions key in the mapping to define the applicable active_loop, like so:
slots:
location:
type: text
influence_conversation: false
mappings:
- type: from_entity
entity: city
conditions:
- active_loop: booking_form
To learn more about how to migrate your 2.0 assistant, please read the migration guide.

Improvements

Updated the /status endpoint response payload, and relevant documentation, to return/reflect the updated 3.0 keys/values.
Bump TensorFlow version to 2.6.
This update brings some security benefits (see TensorFlow release notes for details). However, internal experiments suggest that it is also associated with increased train and inference time, as well as increased memory usage.
You can read more about why we decided to update TensorFlow, and what the expected impact is here.
If you experience a significant increase in train time, inference time, and/or memory usage, please let us know in the forum.
Users can no longer set TF_DETERMINISTIC_OPS=1 if they are using GPU(s) because a tf.errors.UnimplementedError will be thrown by TensorFlow (read more here).
:::caution This breaks backward compatibility of previously trained models. It is not possible to load models trained with previous versions of Rasa Open Source. Please re-train your assistant before trying to use this version.
Added authentication support for connecting to external RabbitMQ servers. Currently user has to hardcode a username and a password in a URL in order to connect to an external RabbitMQ server.
1) Failed test stories will display full retrieval intents.
2) Retrieval intents will be extracted during action prediction in test stories so that we won't have unnecessary mismatches anymore.
Let's take this example story:
- story: test story
steps:
- user: |
what is your name?
intent: chitchat/ask_name
- action: utter_chitchat/ask_name
- intent: bye
- action: utter_bye
Before:
steps:
- intent: chitchat # 1) intent is not displayed in it's original form
- action: utter_chitchat/ask_name # predicted: utter_chitchat
# 2) retrieval intent is not extracted during action prediction and we have a mismatch

- intent: bye # some other fail
- action: utter_bye # some other fail
Both 1) and 2) problems are solved.
Now:
steps:
- intent: chitchat/ask_name
- action: utter_chitchat/ask_name
- intent: bye # some other fail
- action: utter_bye # some other fail
Added -i command line option to make RASA listen on a specific ip-address instead of any network interface
rasa data validate now checks that forms referenced in active_loop directives are defined in the domain
Every conversation event now includes in its metadata the ID of the model which loaded at the time it was created.
Send indices of user message tokens along with the UserUttered event through the event broker to Rasa X.
Added optional flag to convert intent ID hashes from integer to string in the KafkaEventBroker.
Make it possible to use or functionality for slot_was_set events.
Upgraded the spaCy dependency from version 3.0 to 3.1.
Implemented fingerprint methods in these classes:
- Event
- Slot
- DialogueStateTracker
Added debug message that logs when a response condition is used.
The naming scheme for trained models was changed. Unless you provide a --fixed-model-name to rasa train, Rasa Open Source will now generate a new model name using the schema <timestamp>-<random name>.tar.gz, e.g.
- 20211018-094821-composite-pita.tar.gz (for a model containing a trained NLU and dialogue model)
- nlu-20211018-094821-composite-pita.tar.gz (for a model containing only a trained NLU model but not a dialogue model)
- core-20211018-094821-composite-pita.tar.gz (for a model containing only a trained dialogue model but no NLU model)
Due to changes in the model architecture the behavior of rasa train --dry-run changed. The exit codes now have the following meaning:
- 0 means that the model does not require an expensive retraining. However, the responses might still require updating by running rasa train
- 1 means that one or multiple components require to be retrained.
- 8 means that the --force flag was used and hence any cached results are ignored and the entire model is retrained.
Machine learning components like DIETClassifier, ResponseSelector and TEDPolicy using a ranking_length parameter will no longer report renormalised confidences for the top predictions by default.
A new parameter renormalize_confidences is added to these components which if set to True, renormalizes the confidences of top ranking_length number of predictions to sum up to 1. The default value is False, which means no renormalization will be applied by default. It is advised to leave it to False but if you are trying to reproduce the results from previous versions of Rasa Open Source, you can set it to True.
Renormalization will only be applied if model_confidence=softmax is used.

Bugfixes

Fixed validation behavior and logging output around unused intents and utterances.
rasa test nlu --cross-validation uses autoconfiguration when no pipeline is defined instead of failing
Update DynamoDb tracker store to correctly retrieve all sender_ids from a DynamoDb table.
Fix for failed_test_stories.yml not printing the correct message when the extracted entity specified in a test story is incorrect.
Fix CVE-2021-41127

Improved Documentation

Added new docs for Markers.
Update pip in same command which installs rasa and clarify supported version in docs.
Update pika consumer code in Event Brokers documentation.
Adds documentation on how to use CRFEntityExtractor with features from a dense featurizer (e.g. LanguageModelFeaturizer).
Updated docs (Domain, Forms, Default Actions, Migration Guide, CLI) to provide more detail over the new slot mappings changes.
Updated documentation publishing mechanisms to build one version of the documentation for each major version of Rasa Open Source, starting from 2.x upwards. Previously, we were building one version of the documentation for each minor version of Rasa Open Source, resulting in a poor user experience and high maintenance costs.

Miscellaneous internal changes

Miscellaneous internal changes.

[2.8.16] - 2021-12-09

Improvements

The value of the RASA_ENVIRONMENT environmental variable is sent as a header in messages logged by KafkaEventBroker. This value was previously only made available by PikaEventConsumer.

Bugfixes

Make action_metadata json serializable and make it available on the tracker. This is a backport of a fix in 3.0.0.

[2.8.15] - 2021-11-25

Bugfixes

Validate regular expressions in nlu training data configuration.

[2.8.14] - 2021-11-18

Bugfixes

Bump TensorFlow version to 2.6.2. We have plans to port this change to 3.x (see this issue).
Downgrade google-auth to <2.

[2.8.13] - 2021-11-11

Bugfixes

Fixed new intent creation in rasa interactive command. Previously, this failed with 500 from the server due to UnexpecTEDIntentPolicy trying to predict with the new intent not in domain.
Install mitie library when preparing test runs. This step was missing before and tests were thus failing as we have many tests which rely on mitie library. Previously, make install-full was required.

Miscellaneous internal changes

Miscellaneous internal changes.

[2.8.12] - 2021-10-21

Bugfixes

Fixed a bug where rasa test --fail-on-prediction-errors would raise a WrongPredictionException for entities which were actually predicted correctly.
This happened in two ways:
1. if for a user message some entities were extracted multiple times (by multiple entity extractors) but listed only once in the test story,
2. if the order in which entities from a message were extracted didn't match the order in which they were listed in the test story.

Improved Documentation

Improve the documentation for training TEDPolicy with data augmentation.

[2.8.11] - 2021-10-20

Bugfixes

Updates dependency on sanic-jwt (1.5.0 -> ">=1.6.0, <1.7.0")
This removes the need to pin the version of pyjwt as the newer version of sanic-jwt manages this properly.

[2.8.10] - 2021-10-14

Bugfixes

Add List handling in the send_custom_json method on channels/facebook.py. Bellow are some examples that could cause en error before.
Example 1: when the whole json is a List
[
{
"blocks": {
"type": "progression_bar",
"text": {"text": "progression 1", "level": "1"},
}
},
{"sender": {"id": "example_id"}},
]
Example 2: instead of being a Dict, blocks is a List when there are 2 type keys under it
{
"blocks": [
{"type": "title", "text": {"text": "Conversation progress"}},
{
"type": "progression_bar",
"text": {"text": "Look how far we are...", "level": "1"},
},
]
}
Fixed bug when using wit.ai training data to train. Training failed with an error similarly to this:
File "./venv/lib/python3.8/site-packages/rasa/nlu/classifiers/diet_classifier.py", line 803, in train
self.check_correct_entity_annotations(training_data)
File "./venv/lib/python3.8/site-packages/rasa/nlu/extractors/extractor.py", line 418, in check_correct_entity_annotations
entities_repr = [
File "./venv/lib/python3.8/site-packages/rasa/nlu/extractors/extractor.py", line 422, in <listcomp>
entity[ENTITY_ATTRIBUTE_VALUE],
KeyError: 'value'
Fix CVE-2021-41127

[2.8.9] - 2021-10-08

Improvements

Bump TensorFlow version to 2.6.
This update brings some security benefits (see TensorFlow release notes for details). However, internal experiments suggest that it is also associated with increased train and inference time, as well as increased memory usage.
You can read more about why we decided to update TensorFlow, and what the expected impact is here.
If you experience a significant increase in train time, inference time, and/or memory usage, please let us know in the forum.
Users can no longer set TF_DETERMINISTIC_OPS=1 if they are using GPU(s) because a tf.errors.UnimplementedError will be thrown by TensorFlow (read more here).
caution
This breaks backward compatibility of previously trained models. It is not possible to load models trained with previous versions of Rasa Open Source. Please re-train your assistant before trying to use this version.

[2.8.8] - 2021-10-06

Improvements

Added a function to display the actual text of a Token when inspecting a Message in a pipeline, making it easier to debug.

Improved Documentation

Removing the experimental feature warning for conditional response variations from the Rasa docs. The behaviour of the feature remains unchanged.
Updates quick install documentation with optional venv step, better pip install instructions, & M1 warning

[2.8.7] - 2021-09-20

Bugfixes

Explicitly set the upper limit for currently compatible TensorFlow versions.

[2.8.6] - 2021-09-09

Bugfixes

Fix rules not being applied when a featurised categorical slot has as one of its allowed values none, NoNe, None or a similar value.

[2.8.5] - 2021-09-06

Bugfixes

AugmentedMemoizationPolicy is accelerated for large trackers
Bump tensorflow to 2.3.4 to address security vulnerabilities

[2.8.4] - 2021-09-02

Improvements

Increase speed of augmented lookup for AugmentedMemoizationPolicy

Bugfixes

Fix --data being treated as if non-optional on sub-commands of rasa data convert
Fixes bug where hide_rule_turn was defaulting to None when ActionExecuted was deserialised.

Miscellaneous internal changes

Miscellaneous internal changes.

[2.8.3] - 2021-08-19

Bugfixes

Ignore checking that intent is in domain for E2E story utterances when running rasa data validate. Previously data validation would fail on E2E stories.

[2.8.2] - 2021-08-04

Bugfixes

Fixes a bug which caused training of UnexpecTEDIntentPolicy to crash when end-to-end training stories were included in the training data.
Stories with end-to-end training data will now be skipped for the training of UnexpecTEDIntentPolicy.

Improved Documentation

Removing the experimental feature warning for the story validation tool from the rasa docs. The behaviour of the feature remains unchanged.
Removing the experimental feature warning for entity roles and groups from the rasa docs, and from the code where it previously appeared as a print statement. The behaviour of the feature remains otherwise unchanged.

[2.8.1] - 2021-07-22

Improvements

Add support for cafile parameter in endpoints.yaml. This will load a custom local certificate file and use it when making requests to that endpoint.
For example:
action_endpoint:
url: https://localhost:5055/webhook
cafile: ./cert.pem
This means that requests to the action server localhost:5055 will use the certificate cert.pem located in the current working directory.

Bugfixes

Fixes wrong overriding of epochs parameter when TEDPolicy or UnexpecTEDIntentPolicy is not loaded in finetune mode.

[2.8.0] - 2021-07-12

Deprecations and Removals

The option model_confidence=linear_norm is deprecated and will be removed in Rasa Open Source 3.0.0.
Rasa Open Source 2.3.0 introduced linear_norm as a possible value for model_confidence parameter in machine learning components such as DIETClassifier, ResponseSelector and TEDPolicy. Based on user feedback, we have identified multiple problems with this option. Therefore, model_confidence=linear_norm is now deprecated and will be removed in Rasa Open Source 3.0.0. If you were using model_confidence=linear_norm for any of the mentioned components, we recommend to revert it back to model_confidence=softmax and re-train the assistant. After re-training, we also recommend to re-tune the thresholds for fallback components.
The fallback mechanism for spaCy models has now been removed in Rasa 3.0.0.
Rasa Open Source 2.5.0 introduced support for spaCy 3.0. This introduced a breaking feature because models would no longer be manually linked. To make the transition smooth Rasa would rely on the language parameter in the config.yml to fallback to a medium spaCy model if no model was configured for the SpacyNLP component. In Rasa Open Source 3.0.0 and onwards the SpacyNLP component will require the model name (like "en_core_web_md") to be passed explicitly.

Features

Added sasl_mechanism as an optional configurable parameters for the Kafka Producer.
Introduces a new policy called UnexpecTEDIntentPolicy.
UnexpecTEDIntentPolicy helps you review conversations and also allows your bot to react to unexpected user turns in conversations. It is an auxiliary policy that should only be used in conjunction with at least one other policy, as the only action that it can trigger is the special and newly introduced action_unlikely_intent action.
The auto-configuration will include UnexpecTEDIntentPolicy in your configuration automatically, but you can also include it yourself in the policies section of the configuration:
policies:
- name: UnexpecTEDIntentPolicy
epochs: 200
max_history: 5
As part of the feature, it also introduces:
- IntentMaxHistoryTrackerFeaturizer to featurize the trackers for UnexpecTEDIntentPolicy.
- MultiLabelDotProductLoss to support UnexpecTEDIntentPolicy's multi-label training objective.
- A new default action called action_unlikely_intent.

rasa test command has also been adapted to UnexpecTEDIntentPolicy:

If a test story contains action_unlikely_intent and the policy ensemble does not trigger it, this leads to a test error (wrongly predicted action) and the corresponding story will be logged in failed_test_stories.yml.
If the story does not contain action_unlikely_intent and Rasa Open Source does predict it then the prediction of action_unlikely_intent will be ignored for the evaluation (and hence not lead to a prediction error) but the story will be logged in a file called stories_with_warnings.yml.

The rasa data validate command will warn if action_unlikely_intent is included in the training stories. Accordingly, YAMLStoryWriter and MarkdownStoryWriter have been updated to not dump action_unlikely_intent when writing stories to a file.

:::caution The introduction of a new default action breaks backward compatibility of previously trained models. It is not possible to load models trained with previous versions of Rasa Open Source. Please re-train your assistant before trying to use this version.

:::

Improvements

Added detailed json schema validation for UserUttered, SlotSet, ActionExecuted and EntitiesAdded events both sent and received from the action server, as well as covered at high-level the validation of the rest of the 20 events. In case the events are invalid, a ValidationError will be raised.
Users don't need to specify an additional buffer size for sparse featurizers anymore during incremental training.
Space for new sparse features are created dynamically inside the downstream machine learning models - DIETClassifier, ResponseSelector. In other words, no extra buffer is created in advance for additional vocabulary items and space will be dynamically allocated for them inside the model.
This means there's no need to specify additional_vocabulary_size for CountVectorsFeaturizer or number_additional_patterns for RegexFeaturizer. These parameters are now deprecated.
Before
pipeline:
- name: "WhitespaceTokenizer"
- name: "RegexFeaturizer"
number_additional_patterns: 100
- name: "CountVectorsFeaturizer"
additional_vocabulary_size: {text: 100, response: 20}
Now
pipeline:
- name: "WhitespaceTokenizer"
- name: "RegexFeaturizer"
- name: "CountVectorsFeaturizer"
Also, all custom layers specifically built for machine learning models - RasaSequenceLayer, RasaFeatureCombiningLayer and ConcatenateSparseDenseFeatures now inherit from RasaCustomLayer so that they support flexible incremental training out of the box.
Speed up the contradiction check of the RulePolicy by a factor of 3.
Change the confidence score assigned by FallbackClassifier to fallback intent to be the same as the fallback threshold.
Issue a UserWarning if a specified domain folder contains files that look like YML files but cannot be parsed successfully. Only invoked if user specifies a folder path in --domain paramater. Previously those invalid files in the specified folder were silently ignored. Does not apply to individually specified domain YAML files, e.g. --domain /some/path/domain.yml, those being invalid will still raise an exception.

Bugfixes

Fix for unnecessary retrain and duplication of folders in the model

Miscellaneous internal changes

Miscellaneous internal changes.

[2.7.2] - 2021-08-09

Bugfixes

Ignore checking that intent is in domain for E2E story utterances when running rasa data validate. Previously data validation would fail on E2E stories.
Fix for unnecessary retrain and duplication of folders in the model

[2.7.1] - 2021-06-16

Bugfixes

Best model checkpoint allows for metrics to be equal to previous best if at least one metric improves, rather than strict improvement for each metric.
Fixes a bug where multiple plots overlap each other and are rendered incorrectly when comparing performance across multiple NLU pipelines.
Don't evaluate entities if no entities present in test data.
Also, catch exception in plot_paired_histogram when data is empty.

[2.7.0] - 2021-06-03

Improvements

Changed the default config to train the RulePolicy before the TEDPolicy. This means that conflicting rule/stories will be identified before a potentially slow training of the TEDPolicy.
Updated validator used by rasa data validate to verify that actions used in stories and rules are present in the domain and that form slots match domain slots.
Rename plot_histogram to plot_paired_histogram and fix missing bars in the plot.
Changed --data option type in the ``rasa data validate``` command to allow more than one path to be passed.

Bugfixes

The file failed_test_stories.yml (generated by rasa test) now also includes the wrongly predicted entity as a comment next to the entity of a user utterance. Additionally, the comment printed next to the intent of a user utterance is printed only if the intent was wrongly predicted (irrelevantly if there was a wrongly predicted entity or not in the specific user utterance).
Added check in PikaEventBroker constructor: if port cannot be cast to integer, raise RasaException
Fixed bug where missing intent warnings appear when running rasa test
Update should_retrain function to return the correct fingerprint comparison result even when there is a problem with model unpacking.
Handle correctly Telegram edited message.

Miscellaneous internal changes

Miscellaneous internal changes.

[2.6.3] - 2021-05-28

Bugfixes

ResponseSelector can now be trained with the transformer enabled (i.e. when a positive number_of_transformer_layers is provided) even if one doesn't specify the transformer's size. Previously, not specifying transformer_size led to an error.
Return EntityEvaluationResult during evaluation of test stories only if parsed_message is not None.
Ignore OSError in Sentry reporting.
Replaced ValueError with RasaException in TED model _check_data method.
Changed import to fix agent creation in Jupyter.

Miscellaneous internal changes

Miscellaneous internal changes.

[2.6.2] - 2021-05-18

Bugfixes

Fixed a bug where ListSlots were filled with single items in case only one matching entity was extracted for this slot.
Values applied to ListSlots will be converted to a List in case they aren't one.
Fix bug with false rule conflicts
This essentially reverts PR 8446, except for the tests. The PR is redundant due to PR 8646.
Handle AttributeError thrown by empty slot mappings in domain form through refactoring.
Fixed incorrect The action 'utter_<response selector intent>' is used in the stories, but is not a valid utterance action error when running rasa data validate with response selector responses in the domain file.

Improved Documentation

Added a note to clarify best practice for resetting all slots after form deactivation.

Miscellaneous internal changes

Miscellaneous internal changes.

[2.6.1] - 2021-05-11

Bugfixes

Made SchemaError message available to validator so that the reason why reason schema validation fails during rasa data validate is displayed when response text value is null. Added warning message when deprecated MappingPolicy format is used in the domain.
When there are multiple entities in a user message, they will get sorted when creating a representation of the current dialogue state.
Previously, the ordering was random, leading to inconsistent state representations. This would sometimes lead to memoization policies failing to recall a memorised action.

[2.6.0] - 2021-05-06

Deprecations and Removals

In forms, the keyword required_slots should always precede the definition of slot mappings and the lack of it is deprecated. Please see the migration guide for more information.
rasa.data.get_test_directory, rasa.data.get_core_nlu_directories, and rasa.shared.nlu.training_data.training_data.TrainingData::get_core_nlu_directories are deprecated and will be removed in Rasa Open Source 3.0.0.
Update the minimum compatible model version to "2.6.0". This means all models trained with an earlier version will have to be retrained.

Features

Feature enhancement enabling JWT authentication for the Socket.IO channel. Users can define jwt_key and jwt_method as parameters in their credentials file for authentication.
Allows a Rasa bot to be connected to a Twilio Voice channel. More details in the Twilio Voice docs
Conditional response variations are supported in the domain.yml without requiring users to write custom actions code.
A condition can be a list of slot-value mapping constraints.

Improvements

Added an optional ignored_intents parameter in forms.
- To use it, add the ignored_intents parameter in your domain.yml file after the forms name and provide a list of intents to ignore. Please see Forms for more information.
- This can be used in case the user never wants to fill any slots of a form with the specified intent, e.g. chitchat.
Add function to carry max_history to featurizer
Improved the machine learning models' codebase by factoring out shared feature-processing logic into three custom layer classes:
- ConcatenateSparseDenseFeatures combines multiple sparse and dense feature tensors into one.
- RasaFeatureCombiningLayer additionally combines sequence-level and sentence-level features.
- RasaSequenceLayer is used for attributes with sequence-level features; it additionally embeds the combined features with a transformer and facilitates masked language modeling.
Added the following usability improvements with respect to entities getting extracted multiple times:
- Added warnings for competing entity extractors at training time and for overlapping entities at inference time
- Improved docs to help users handle overlapping entity problems.
Replace weight_sparsity with connection_density in all transformer-based models and add guarantees about internal layers.
We rename DenseWithSparseWeights into RandomlyConnectedDense, and guarantee that even at density zero the output is dense and every input is connected to at least one output. The former weight_sparsity parameter of DIET, TED, and the ResponseSelector, is now roughly equivalent to 1 - connection_density, except at very low densities (high sparsities).
All layers and components that used to have a sparsity argument (Ffnn, TransformerRasaModel, MultiHeadAttention, TransformerEncoderLayer, TransformerEncoder) now have a density argument instead.
Rasa test now prints a warning if the test stories contain bot utterances that are not part of the domain.
Updated asyncio.Task.all_tasks to asyncio.all_tasks, with a fallback for python 3.6, which raises an AttributeError for asyncio.all_tasks. This removes the deprecation warning for the Task.all_tasks usage.
Change variable name from i to array_2D
Implement a new interface run_inference inside RasaModel which performs batch inferencing through tensorflow models.
rasa_predict inside RasaModel has been made a private method now by changing it to _rasa_predict.

Bugfixes

Fixed a bug for plotting trackers with non-ascii texts during interactive training by enforcing utf-8 encoding
Fix masked language modeling in DIET to only apply masking to token-level (sequence-level) features. Previously, masking was applied to both token-level and sentence-level features.
Make it possible to use null entities in stories.
Introduce a skip_validation flag in order to speed up reading YAML files that were already validated.
Fixed a bug in interactive training that lead to crashes for long Chinese, Japanese, or Korean user or bot utterances.

[2.5.2] - 2021-06-16

Features

Added sasl_mechanism as an optional configurable parameters for the Kafka Producer.

[2.5.1] - 2021-04-28

Bugfixes

Fixed prediction for rules with multiple entities.
Mitigated Matplotlib backend issue using lazy configuration and added a more explicit error message to guide users.

[2.5.0] - 2021-04-12

Deprecations and Removals

The following import abbreviations were removed:
- rasa.core.train: Please use rasa.core.train.train instead.
- rasa.core.visualize: Please use rasa.core.visualize.visualize instead.
- rasa.nlu.train: Please use rasa.nlu.train.train instead.
- rasa.nlu.test: Please use rasa.nlu.test.run_evaluation instead.
- rasa.nlu.cross_validate: Please use rasa.nlu.test.cross_validate instead.

Features

Upgraded Rasa to be compatible with spaCy 3.0.
This means that we can support more features for more languages but there are also a few changes.
SpaCy 3.0 deprecated the spacy link <language model> command so that means that from now on the full model name needs to be used in the config.yml file.
Before
Before you could run spacy link en en_core_web_md and then we would be able to pick up the correct model from the language parameter.
language: en

pipeline:
- name: SpacyNLP
Now
This behavior will be deprecated and instead you'll want to be explicit in config.yml.
language: en

pipeline:
- name: SpacyNLP
model: en_core_web_md
Fallback
To make the transition easier, Rasa will try to fall back to a medium spaCy model when-ever a compatible language is configured for the entire pipeline in config.yml even if you don't specify a model. This fallback behavior is temporary and will be deprecated in Rasa 3.0.0.
We've updated our docs to reflect these changes. All examples now show a direct link to the correct spaCy model. We've also added a warning to the SpaCyNLP docs that explains the fallback behavior.

Improvements

Improved CLI startup time.
Add augmentation and num_threads arguments to API POST /model/train
Fix boolean casting issue for force_training and save_to_default_model_directory arguments
Add minimum compatible version to --version command
Updated warning for unexpected slot events during prediction time to Rasa Open Source 2.0 YAML training data format.
Hide dialogue turns predicted by RulePolicy in the tracker states for ML-only policies like TEDPolicy if those dialogue turns only appear as rules in the training data and do not appear in stories.
Add set_shared_policy_states(...) method to all policies. This method sets _rule_only_data dict with keys:
- rule_only_slots: Slot names, which only occur in rules but not in stories.
- rule_only_loops: Loop names, which only occur in rules but not in stories.
This information is needed for correct featurization to hide dialogue turns that appear only in rules.
Faster reading of YAML NLU training data files.
Added partition_by_sender flag to Kafka Producer to optionally associate events with Kafka partition based on sender_id.

Bugfixes

Fixed the 'loading model' message which was logged twice when using rasa run.
Change training data validation to only count nlu training examples.
Rule tracker states no longer include the initial value of slots. Rules now only require slot values when explicitly stated in the rule.
rasa test, rasa test core and rasa test nlu no longer show temporary paths in case there are issues in the test files.
Resolved memory problems with dense features and CRFEntityExtractor
Handle empty intent and entity mapping in the domain.
There is now an InvalidDomain exception raised if in the domain.yml file there are empty intent or entity mappings. An example of empty intent and entity mappings is the following :
intents:
- greet:
- goodbye:

entities:
- cuisine:
- number:
Fixed a bug in a form where slot mapping doesn't work if the predicted intent name is substring for another intent name.
Fixes bug where stories could not be retrieved if entities had no start or end.
Catch ChannelNotFoundEntity exception coming from the pika broker and raise as ConnectionException.
Fix bug with NoReturn throwing an exception in Python 3.7.0 when running rasa train
Throw RasaException instead of ValueError in situations when environment variables specified in YAML cannot be expanded.
Updated python-engineio version for compatibility with python-socketio

Miscellaneous internal changes

Miscellaneous internal changes.

[2.4.3] - 2021-03-26

Bugfixes

Fixes bug where stories could not be retrieved if entities had no start or end.

[2.4.2] - 2021-03-25

Bugfixes

Fix UnicodeException in is_key_in_yaml.
Fixed the bug that events from previous conversation sessions would be re-saved in the SQLTrackerStore or MongoTrackerStore when retrieve_events_from_previous_conversation_sessions was true.

[2.4.1] - 2021-03-23

Bugfixes

Fix TEDPolicy training e2e entities when no entities are present in the stories but there are entities in the domain.
Fixed missing model configuration file validation.
In Rasa 2.4.0, support for using template in utter_message when handling a custom action was wrongly deprecated. Both template and response are now supported, though note that template will be deprecated at Rasa 3.0.0.

[2.4.0] - 2021-03-11

Deprecations and Removals

NLG Server
- Changed request format to send response as well as template as a field. The template field will be removed in Rasa Open Source 3.0.0.
rasa.core.agent
- The terminology template is deprecated and replaced by response. Support for template from the NLG response will be removed in Rasa Open Source 3.0.0. Please see here for more details.
rasa.core.nlg.generator
- generate() now takes in utter_action as a parameter.
- The terminology template is deprecated and replaced by response. Support for template in the NaturalLanguageGenerator will be removed in Rasa Open Source 3.0.0.
rasa.shared.core.domain
- The property templates is deprecated. Use responses instead. It will be removed in Rasa Open Source 3.0.0.
- retrieval_intent_templates will be removed in Rasa Open Source 3.0.0. Please use retrieval_intent_responses instead.
- is_retrieval_intent_template will be removed in Rasa Open Source 3.0.0. Please use is_retrieval_intent_response instead.
- check_missing_templates will be removed in Rasa Open Source 3.0.0. Please use check_missing_responses instead.
Response Selector
- The field template_name will be deprecated in Rasa Open Source 3.0.0. Please use utter_action instead. Please see here for more details.
- The field response_templates will be deprecated in Rasa Open Source 3.0.0. Please use responses instead. Please see here for more details.

Improvements

The following endpoints now require the existence of the conversation for the specified conversation ID, raising an exception and returning a 404 status code.
- GET /conversations/<conversation_id:path>/story
- POST /conversations/<conversation_id:path>/execute
- POST /conversations/<conversation_id:path>/predict
Simplify our training by overwriting train_step instead of fit for our custom models.
This allows us to use the build-in callbacks from Keras, such as the Tensorboard Callback, which offers more functionality compared to what we had before.
:::warning If you want to use Tensorboard for DIETClassifier, ResponseSelector, or TEDPolicy and log metrics after every (mini)batch, please use 'batch' instead of 'minibatch' as 'tensorboard_log_level'.
When TED is configured to extract entities rasa test now evaluates them against the labels in the test stories. Results are saved in /results along with the results for the NLU components that extract entities.
We're now running integration tests for Rasa Open Source, with initial coverage for SQLTrackerStore (with PostgreSQL), RedisLockStore (with Redis) and PikaEventBroker (with RabbitMQ). The integration tests are now part of our CI, and can also be ran locally using make test-integration (see Rasa Open Source README for more information).
Allow tests to be located anywhere, not just in tests directory.
Model configuration files are now validated whether they match the expected schema.
Speed up YAMLStoryReader.is_key_in_yaml function by making it to check if key is in YAML without actually parsing the text file.
Speed up YAML parsing by reusing parsers, making the process of environment variable interpolation optional, and by not adding duplicating implicit resolvers and YAML constructors to ruamel.yaml
Drastically improved finger printing time for large story graphs
Remove console logging of conversation level F1-score and precision since these calculations were not meaningful.
Add conversation level accuracy to core policy results logged to file in story_report.json after running rasa test core or rasa test.
Improved the lock store debug log message when the process has to queue because other messages have to be processed before this item.

Bugfixes

Fixed the bug that OR statements in stories would break the check whether a model needs to be retrained
Update the spec of POST /model/test/intents and add tests for cases when JSON is provided.
Fix the incorrect temporary file extension for the data that gets extracted from the payload provided in the body of POST /model/test/intents request.
Fix for the cli command rasa data convert config when migrating Mapping Policy and no rules.
Making rasa data convert config migrate correctly the Mapping Policy when no rules are available. It updates the config.yml file by removing the MappingPolicy and adding the RulePolicy instead. Also, it creates the data/rules.yml file even if empty in the case of no available rules.
Allow to have slots with values that result to a dictionary under the key slot_was_set (in stories.yml file).
An example would be to have the following story step in stories.yml:
- slot_was_set:
- some_slot:
some_key: 'some_value'
other_key: 'other_value'
This would be allowed if the some_slot is also set accordingly in the domain.yml with type any.
Update the fingerprinting function to recognize changes in lookup files.
Fixed a bug when interpolating environment variables in YAML files which included $ in their value. This led to the following stack trace:
ValueError: Error when trying to expand the environment variables in '${PASSWORD}'. Please make sure to also set these environment variables: '['$qwerty']'.
(13 additional frame(s) were not displayed)
...
File "rasa/utils/endpoints.py", line 26, in read_endpoint_config
content = rasa.shared.utils.io.read_config_file(filename)
File "rasa/shared/utils/io.py", line 527, in read_config_file
content = read_yaml_file(filename)
File "rasa/shared/utils/io.py", line 368, in read_yaml_file
return read_yaml(read_file(filename, DEFAULT_ENCODING))
File "rasa/shared/utils/io.py", line 349, in read_yaml
return yaml_parser.load(content) or {}
File "rasa/shared/utils/io.py", line 314, in env_var_constructor
" variables: '{}'.".format(value, not_expanded)
The REQUESTED_SLOT always belongs to the currently active form.
Previously it was possible that after form switching, the REQUESTED_SLOT was for the previous form.
Update the LanguageModelFeaturizer tests to reflect new default model weights for bert, and skip all bert tests with default model weights on CI, run bert tests with bert-base-uncased on CI instead.

Improved Documentation

Update links to Sanic docs in the documentation.
Update Rasa Playground to correctly use tracking_id when calling API methods.

Miscellaneous internal changes

Miscellaneous internal changes.

[2.3.5] - 2021-06-16

Features

Added sasl_mechanism as an optional configurable parameters for the Kafka Producer.

Improvements

Drastically improved finger printing time for large story graphs
Improved the lock store debug log message when the process has to queue because other messages have to be processed before this item.

Bugfixes

Fixed the bug that OR statements in stories would break the check whether a model needs to be retrained
Updated python-engineio dependency version for compatibility with python-socketio.

Improved Documentation

Update links to Sanic docs in the documentation.

[2.3.4] - 2021-02-26

Bugfixes

Setting model_confidence=cosine in DIETClassifier, ResponseSelector and TEDPolicy is deprecated and will no longer be available. This was introduced in Rasa Open Source version 2.3.0 but post-release experiments suggest that using cosine similarity as model's confidences can change the ranking of predicted labels which is wrong.
model_confidence=inner is deprecated and is replaced by model_confidence=linear_norm as the former produced an unbounded range of confidences which broke the logic of assistants in various other places.
We encourage you to try model_confidence=linear_norm which will produce a linearly normalized version of dot product similarities with each value in the range [0,1]. This can be done with the following config:
- name: DIETClassifier
model_confidence: linear_norm
constrain_similarities: True
This should ease up tuning fallback thresholds as confidences for wrong predictions are better distributed across the range [0, 1].
If you trained a model with model_confidence=cosine or model_confidence=inner setting using previous versions of Rasa Open Source, please re-train by either removing the model_confidence option from the configuration or setting it to linear_norm.
model_confidence=cosine is removed from the configuration generated by auto-configuration.

[2.3.3] - 2021-02-25

Bugfixes

Fixed bug where the conversation does not lock before handling a reminder event.

[2.3.2] - 2021-02-22

Bugfixes

Fix a bug where, if a user injects an intent using the HTTP API, slot auto-filling is not performed on the entities provided.

[2.3.1] - 2021-02-17

Bugfixes

Fixed a YAML validation error which happened when executing multiple validations concurrently. This could e.g. happen when sending concurrent requests to server endpoints which process YAML training data.

[2.3.0] - 2021-02-11

Improvements

Expose diagnostic data for action and NLU predictions.
Add diagnostic_data field to the Message and Prediction objects, which contain information about attention weights and other intermediate results of the inference computation. This information can be used for debugging and fine-tuning, e.g. with RasaLit.
For examples of how to access the diagnostic data, see here.
Using the TrainingDataImporter interface to load the data in rasa test core.
Failed test stories are now referenced by their absolute path instead of the relative path.
Improve error handling and Sentry tracking:
- Raise MarkdownException when training data in Markdown format cannot be read.
- Raise InvalidEntityFormatException error instead of json.JSONDecodeError when entity format is in valid in training data.
- Gracefully handle empty sections in endpoint config files.
- Introduce ConnectionException error and raise it when TrackerStore and EventBroker cannot connect to 3rd party services, instead of raising exceptions from 3rd party libraries.
- Improve rasa.shared.utils.common.class_from_module_path function by making sure it always returns a class. The function currently raises a deprecation warning if it detects an anomaly.
- Ignore MemoryError and asyncio.CancelledError in Sentry.
- rasa.shared.utils.validation.validate_training_data now raises a SchemaValidationError when validation fails (this error inherits jsonschema.ValidationError, ensuring backwards compatibility).
Allow PolicyEnsemble in cases where calling individual policy's load method returns None.
User message metadata can now be accessed via the default slot session_started_metadata during the execution of a custom action_session_start.
from typing import Any, Text, Dict, List
from rasa_sdk import Action, Tracker
from rasa_sdk.events import SlotSet, SessionStarted, ActionExecuted, EventType

class ActionSessionStart(Action):
def name(self) -> Text:
return "action_session_start"

async def run(
self, dispatcher, tracker: Tracker, domain: Dict[Text, Any]
) -> List[Dict[Text, Any]]:
metadata = tracker.get_slot("session_started_metadata")

# Do something with the metadata
print(metadata)

# the session should begin with a `session_started` event and an `action_listen`
# as a user message follows
return [SessionStarted(), ActionExecuted("action_listen")]
Add BILOU tagging schema for entity extraction in end-to-end TEDPolicy.
Added two new parameters constrain_similarities and model_confidence to machine learning (ML) components - DIETClassifier, ResponseSelector and TEDPolicy.
Setting constrain_similarities=True adds a sigmoid cross-entropy loss on all similarity values to restrict them to an approximate range in DotProductLoss. This should help the models to perform better on real world test sets. By default, the parameter is set to False to preserve the old behaviour, but users are encouraged to set it to True and re-train their assistants as it will be set to True by default from Rasa Open Source 3.0.0 onwards.
Parameter model_confidence affects how model's confidence for each label is computed during inference. It can take three values:
1. softmax - Similarities between input and label embeddings are post-processed with a softmax function, as a result of which confidence for all labels sum up to 1.
2. cosine - Cosine similarity between input label embeddings. Confidence for each label will be in the range [-1,1].
3. inner - Dot product similarity between input and label embeddings. Confidence for each label will be in an unbounded range.
Setting model_confidence=cosine should help users tune the fallback thresholds of their assistant better. The default value is softmax to preserve the old behaviour, but we recommend using cosine as that will be the new default value from Rasa Open Source 3.0.0 onwards. The value of this option does not affect how confidences are computed for entity predictions in DIETClassifier and TEDPolicy.
With both the above recommendations, users should configure their ML component, e.g. DIETClassifier, as
- name: DIETClassifier
model_confidence: cosine
constrain_similarities: True
...
Once the assistant is re-trained with the above configuration, users should also tune fallback confidence thresholds.
Configuration option loss_type=softmax is now deprecated and will be removed in Rasa Open Source 3.0.0 . Use loss_type=cross_entropy instead.
The default auto-configuration is changed to use constrain_similarities=True and model_confidence=cosine in ML components so that new users start with the recommended configuration.
EDIT: Some post-release experiments revealed that using model_confidence=cosine is wrong as it can change the order of predicted labels. That's why this option was removed in Rasa Open Source version 2.3.3. model_confidence=inner is deprecated as it produces an unbounded range of confidences which can break the logic of assistants in various other places. Please use model_confidence=linear_norm which will produce a linearly normalized version of dot product similarities with each value in the range [0,1]. Please read more about this change under the notes for release 2.3.4.
Use simple random uniform distribution of integers in negative sampling, because negative sampling with tf.while_loop and random shuffle inside creates a memory leak.
Added support to configure exchange_name for pika event broker.
If MaxHistoryTrackerFeaturizer is used, invert the dialogue sequence before passing it to the transformer so that the last dialogue input becomes the first one and therefore always have the same positional encoding.

Bugfixes

Fixed an error when using the endpoint GET /conversations/<conversation_id:path>/story with a tracker which contained slots.
Add the option to configure whether extracted entities should be split by comma (",") or not to TEDPolicy. Fixes crash when this parameter is accessed during extraction.
When switching forms, the next form will always correctly ask for the first required slot.
Before, the next form did not ask for the slot if it was the same slot as the requested slot of the previous form.
Fix the bug when RulePolicy handling loop predictions are overwritten by e2e TEDPolicy.
When switching forms, the next form is cleanly activated.
Before, the next form was correctly activated, but the previous form had wrongly uttered the response that asked for the requested slot when slot validation for that slot had failed.
Fix a bug in incremental training when passing a specific model path with the --finetune argument.
Fix the role of unidirectional_encoder in TED. This parameter is only applied to transformers for text, action_text and label_action_text.

Miscellaneous internal changes

Miscellaneous internal changes.

[2.2.10] - 2021-02-08

Improvements

Updated error message when using incompatible model versions.

Bugfixes

Limit numpy version to < 1.2 as tensorflow is not compatible with numpy versions >= 1.2. pip versions <= 20.2 don't resolve dependencies conflicts correctly which could result in an incompatible numpy version and the following error:
NotImplementedError: Cannot convert a symbolic Tensor (strided_slice_6:0) to a numpy array. This error may indicate that you're trying to pass a Tensor to a NumPy call, which is not supported

[2.2.9] - 2021-02-02

Bugfixes

Correctly include the confused_with field in the test report for the POST /model/test/intents endpoint.

[2.2.8] - 2021-01-28

Bugfixes

Fixes a bug in forms where the next slot asked was not consistent after returning to a form from an unhappy path.

[2.2.7] - 2021-01-25

Improvements

Add support for in RasaYAMLWriter for writing intent and example metadata back into NLU YAML files.

Bugfixes

Fixed a bug with Domain.is_domain_file() that could raise an Exception in case the potential domain file is not a valid YAML.

[2.2.6] - 2021-01-21

Bugfixes

Fix wrong warning The method 'EventBroker.close' was changed to be asynchronous when the EventBroker.close was actually asynchronous.
Fix incremental training for cases when training data does not contain entities but DIETClassifier is configured to perform entity recognition also.
Now, the instance of RasaModelData inside DIETClassifier does not contain entities as a feature for training if there is no training data present for entity recognition.

[2.2.5] - 2021-01-12

Bugfixes

Fixed key-error bug on rasa data validate stories.

Miscellaneous internal changes

Miscellaneous internal changes.

[2.2.4] - 2021-01-08

Improvements

Improve the warning in case the RulePolicy or the deprecated MappingPolicy are missing from the model's policies configuration. Changed the info log to a warning as one of this policies should be added to the model configuration.

Bugfixes

Explicitly specify the crypto extra dependency of pyjwt to ensure that the cryptography dependency is installed. cryptography is strictly required to be able to be able to verify JWT tokens.

[2.2.3] - 2021-01-06

Bugfixes

Correctly retrieve intent ranking from UserUttered even during default affirmation action implementation.
Fixed a problem when using the POST /model/test/intents endpoint together with a model server. The error looked as follows:
ERROR rasa.core.agent:agent.py:327 Could not load model due to Detected inconsistent loop usage. Trying to schedule a task on a new event loop, but scheduler was created with a different event loop. Make sure there is only one event loop in use and that the scheduler is running on that one.
This also fixes a problem where testing a model from a model server would change the production model.

[2.2.2] - 2020-12-21

Bugfixes

Fixed incompatibility between Rasa Open Source 2.2.x and Rasa X < 0.35.

[2.2.1] - 2020-12-17

Bugfixes

Fixed a problem where a form wouldn't reject when the FormValidationAction re-implemented required_slots.
Fixed an error when using the SQLTrackerStore with a Postgres database and the parameter login_db specified.
The error was:
psycopg2.errors.SyntaxError: syntax error at end of input
rasa-production_1 | LINE 1: SELECT 1 FROM pg_catalog.pg_database WHERE datname = ?

[2.2.0] - 2020-12-16

Deprecations and Removals

Domain.random_template_for is deprecated and will be removed in Rasa Open Source 3.0.0. You can alternatively use the TemplatedNaturalLanguageGenerator.
Domain.action_names is deprecated and will be removed in Rasa Open Source 3.0.0. Please use Domain.action_names_or_texts instead.
Interfaces for Policy.__init__ and Policy.load have changed. See migration guide for details.
Deprecate training and test data in Markdown format. This includes:
- reading and writing of story files in Markdown format
- reading and writing of NLU data in Markdown format
- reading and writing of retrieval intent data in Markdown format
Support for Markdown data will be removed entirely in Rasa Open Source 3.0.0.
Please convert your existing Markdown data by using the commands from the migration guide:
rasa data convert nlu -f yaml --data={SOURCE_DIR} --out={TARGET_DIR}
rasa data convert nlg -f yaml --data={SOURCE_DIR} --out={TARGET_DIR}
rasa data convert core -f yaml --data={SOURCE_DIR} --out={TARGET_DIR}
Domain.add_categorical_slot_default_value, Domain.add_requested_slot and Domain.add_knowledge_base_slots are deprecated and will be removed in Rasa Open Source 3.0.0. Their internal versions are now called during the Domain creation. Calling them manually is no longer required.

Features

Incremental training of models in a pipeline is now supported.
If you have added new NLU training examples or new stories/rules for dialogue manager, you don't need to train the pipeline from scratch. Instead, you can initialize the pipeline with a previously trained model and continue finetuning the model on the complete dataset consisting of new training examples. To do so, use rasa train --finetune. For more detailed explanation of the command, check out the docs on incremental training.
Added a configuration parameter additional_vocabulary_size to CountVectorsFeaturizer and number_additional_patterns to RegexFeaturizer. These parameters are useful to configure when using incremental training for your pipelines.
Add the option to use cross-validation to the POST /model/test/intents endpoint. To use cross-validation specify the query parameter cross_validation_folds in addition to the training data in YAML format.
Add option to run NLU evaluation (POST /model/test/intents) and model training (POST /model/train) asynchronously. To trigger asynchronous processing specify a callback URL in the query parameter callback_url which Rasa Open Source should send the results to. This URL will also be called in case of errors.

Make TED Policy an end-to-end policy. Namely, make it possible to train TED on stories that contain intent and entities or user text and bot actions or bot text. If you don't have text in your stories, TED will behave the same way as before. Add possibility to predict entities using TED.

Here's an example of a dialogue in the Rasa story format:

stories:
- story: collect restaurant booking info  # name of the story - just for debugging
  steps:
  - intent: greet                          # user message with no entities
  - action: utter_ask_howcanhelp           # action that the bot should execute
  - intent: inform                         # user message with entities
    entities:
    - location: "rome"
    - price: "cheap"
  - bot: On it                             # actual text that bot can output
  - action: utter_ask_cuisine
  - user: I would like [spanish](cuisine). # actual text that user input
  - action: utter_ask_num_people

Some model options for TEDPolicy got renamed. Please update your configuration files using the following mapping:

Old model option	New model option
transformer_size	dictionary “transformer_size” with keys
	“text”, “action_text”, “label_action_text”, “dialogue”
number_of_transformer_layers	dictionary “number_of_transformer_layers” with keys
	“text”, “action_text”, “label_action_text”, “dialogue”
dense_dimension	dictionary “dense_dimension” with keys
	“text”, “action_text”, “label_action_text”, “intent”,
	“action_name”, “label_action_name”, “entities”, “slots”,
	“active_loop”

Improvements

Added a message showing the location where the failed stories file was saved.
Add support for the top-level response keys quick_replies, attachment and elements refered to in rasa.core.channels.OutputChannel.send_reponse, as well as metadata.
Changed the format of the histogram of confidence values for both correct and incorrect predictions produced by running rasa test.
Run bandit checks on pull requests. Introduce make static-checks command to run all static checks locally.
Add rasa train --dry-run command that allows to check if training needs to be performed and what exactly needs to be retrained.
POST /model/test/intents now returns the report field for intent_evaluation, entity_evaluation and response_selection_evaluation as machine-readable JSON payload instead of string.
Make rasa data validate stories work for end-to-end.
The rasa data validate stories function now considers the tokenized user text instead of the plain text that is part of a state. This is closer to what Rasa Core actually uses to distinguish states and thus captures more story structure problems.

Bugfixes

Rename language_list to supported_language_list for JiebaTokenizer.
A float slot returns unambiguous values - [1.0, <value>] if successfully converted, [0.0, 0.0] if not. This makes it possible to distinguish an empty float slot from a slot set to 0.0.
caution
This change is model-breaking. Please retrain your models.
Fix an erroneous attribute for Redis key prefix in rasa.core.tracker_store.RedisTrackerStore: 'RedisTrackerStore' object has no attribute 'prefix'.
Remove token when its text (for example, whitespace) can't be tokenized by LM tokenizer (from LanguageModelFeaturizer).
Temporary directories which were created during requests to the HTTP API are now cleaned up correctly once the request was processed.
Add option use_word_boundaries for RegexFeaturizer and RegexEntityExtractor. To correctly process languages such as Chinese that don't use whitespace for word separation, the user needs to add the use_word_boundaries: False option to those two components.
Correctly fingerprint the default domain slots. Previously this led to the issue that rasa train core would always retrain the model even if the training data hasn't changed.

Improved Documentation

Return the "Migrate from" entry to the docs sidebar.

Miscellaneous internal changes

Miscellaneous internal changes.

[2.1.3] - 2020-12-04

Improvements

Removed multidict from the project dependencies. multidict continues to be a second order dependency of Rasa Open Source but will be determined by the dependencies which use it instead of by Rasa Open Source directly.
This resolves issues like the following:
sanic 20.9.1 has requirement multidict==5.0.0, but you'll have multidict 4.6.0 which is incompatible.

Bugfixes

SingleStateFeaturizer checks whether it was trained with RegexInterpreter as nlu interpreter. If that is the case, RegexInterpreter is used during prediction.
Make sure the responses are synced between NLU training data and the Domain even if there're no retrieval intents in the NLU training data.
Categorical slots will have a default value set when just updating nlg data in the domain.
Previously this resulted in InvalidDomain being thrown.
- Preserve domain slot ordering while dumping it back to the file.
- Preserve multiline text examples of responses defined in domain and NLU training data.

[2.1.2] - 2020-11-27

Bugfixes

Slots that use initial_value won't cause rule contradiction errors when conversation_start: true is used. Previously, two rules that differed only in their use of conversation_start would be flagged as contradicting when a slot used initial_value.
In checking for incomplete rules, an action will be required to have set only those slots that the same action has set in another rule. Previously, an action was expected to have set also slots which, despite being present after this action in another rule, were not actually set by this action.
Fixed Rasa Open Source not being able to fetch models from certain URLs.

[2.1.1] - 2020-11-23

Bugfixes

Sender ID is correctly set when copying the tracker and sending it to the action server (instead of sending the default value). This fixes a problem where the action server would only retrieve trackers with a sender_id default.

[2.1.0] - 2020-11-17

Deprecations and Removals

The Policy interface was changed to return a PolicyPrediction object when predict_action_probabilities is called. Returning a list of probabilities directly is deprecated and support for this will be removed in Rasa Open Source 3.0.
You can adapt your custom policy by wrapping your probabilities in a PolicyPrediction object:
from rasa.core.policies.policy import Policy, PolicyPrediction
# ... other imports

def predict_action_probabilities(
self,
tracker: DialogueStateTracker,
domain: Domain,
interpreter: NaturalLanguageInterpreter,
**kwargs: Any,
) -> PolicyPrediction:
probabilities = ... # an action prediction of your policy
return PolicyPrediction(probabilities, "policy_name", policy_priority=self.priority)
The same change was applied to the PolicyEnsemble interface. Instead of returning a tuple of action probabilities and policy name, it is now returning a PolicyPrediction object. Support for the old PolicyEnsemble interface will be removed in Rasa Open Source 3.0.
caution
This change is model-breaking. Please retrain your models.
The Pika Event Broker no longer supports the environment variables RABBITMQ_SSL_CA_FILE and RABBITMQ_SSL_KEY_PASSWORD. You can alternatively specify RABBITMQ_SSL_CA_FILE in the RabbitMQ connection URL as described in the RabbitMQ documentation.
event_broker:
type: pika
url: "amqps://user:password@host?cacertfile=path_to_ca_cert&password=private_key_password"
queues:
- my_queue

Support for RABBITMQ_SSL_KEY_PASSWORD was removed entirely.
The method Event Broker.close was changed to be asynchronous. Support for synchronous implementations will be removed in Rasa Open Source 3.0.0. To adapt your implementation add the async keyword:
from rasa.core.brokers.broker import EventBroker

class MyEventBroker(EventBroker):

async def close(self) -> None:
# clean up event broker resources

Features

Policies can now return obligatory and optional events as part of their prediction. Obligatory events are always applied to the current conversation tracker. Optional events are only applied to the conversation tracker in case the policy wins.

Improvements

Changed Agent.load method to support pathlib paths.
If you are using the feature Entity Roles and Groups, you should now also list the roles and groups in your domain file if you want roles and groups to influence your conversations. For example:
entities:
- city:
roles:
- from
- to
- name
- topping:
groups:
- 1
- 2
- size:
groups:
- 1
- 2
Entity roles and groups can now influence dialogue predictions. For more information see the section Entity Roles and Groups influencing dialogue predictions.
Predictions of the FallbackClassifier are ignored when evaluating the NLU model Note that the FallbackClassifier predictions still apply to test stories.
Adapt the training data reader and emulator for wit.ai to their latest format. Update the instructions in the migrate from wit.ai documentation to run Rasa Open Source in wit.ai emulation mode.
Adding configurable prefixes to Redis Tracker and Lock Stores so that a single Redis instance (and logical DB) can support multiple conversation trackers and locks. By default, conversations will be prefixed with tracker:... and all locks prefixed with lock:.... Additionally, you can add an alphanumeric-only prefix: value in endpoints.yml such that keys in redis will take the form value:tracker:... and value:lock:... respectively.
Log the model's relative path when using CLI commands.
Adds the option to configure whether extracted entities should be split by comma (",") or not. The default behaviour is True - i.e. split any list of extracted entities by comma. This makes sense for a list of ingredients in a recipie, for example "avocado, tofu, cauliflower", however doesn't make sense for an address such as "Schönhauser Allee 175, 10119 Berlin, Germany".
In the latter case, add a new option to your config, e.g. if you are using the DIETClassifier this becomes:
...
- name: DIETClassifier
split_entities_by_comma: False
...
in which case, none of the extracted entities will be split by comma. To switch it on/off for specific entity types you can use:
...
- name: DIETClassifier
split_entities_by_comma:
address: True
ingredient: False
...
where both address and ingredient are two entity types.
This feature is also available for CRFEntityExtractor.
Fetching test stories from the HTTP API endpoint GET /conversations/<conversation_id>/story no longer triggers an update of the conversation session.
Added a new boolean query parameter all_sessions (default: false) to the HTTP API endpoint for fetching test stories (GET /conversations/<conversation_id>/story).
When setting ?all_sessions=true, the endpoint returns test stories for all conversation sessions for conversation_id. When setting ?all_sessions=all_sessions, or when omitting the all_sessions parameter, a single test story is returned for conversation_id. In cases where multiple conversation sessions exist, only the last story is returned.
Specifying the retrieve_events_from_previous_conversation_sessions kwarg for the Tracker Store class is deprecated and will be removed in Rasa Open Source 3.0. Please use the retrieve_full_tracker() method instead.
Improve the rasa data convert nlg command and introduce the rasa data convert responses command to simplify the migration from pre-2.0 response selector format to the new format.
Added warning for when an option is provided for a component that is not listed as a key in the defaults for that component.
Forms no longer reject their execution before a potential custom action for validating / extracting slots was executed. Forms continue to reject in two cases automatically:
- A slot was requested to be filled, but no slot mapping applied to the latest user message and there was no custom action for potentially extracting other slots.
- A slot was requested to be filled, but the custom action for validating / extracting slots didn't return any slot event.
Additionally you can also reject the form execution manually by returning a ActionExecutionRejected event within your custom action for validating / extracting slots.
Remove dependency between ConveRTTokenizer and ConveRTFeaturizer. The ConveRTTokenizer is now deprecated, and the ConveRTFeaturizer can be used with any other Tokenizer.
Remove dependency between HFTransformersNLP, LanguageModelTokenizer, and LanguageModelFeaturizer. Both HFTransformersNLP and LanguageModelTokenizer are now deprecated. LanguageModelFeaturizer implements the behavior of the stack and can be used with any other Tokenizer.
Gray out "Download" button in Rasa Playground when the project is not yet ready to be downloaded.
Slot mappings for Forms in the domain are now optional. If you do not provide any slot mappings as part of the domain, you need to provide custom slot mappings through a custom action. A form without slot mappings is specified as follows:
forms:
my_form:
# no mappings
The action for forms can now be overridden by defining a custom action with the same name as the form. This can be used to keep using the deprecated Rasa Open Source FormAction which is implemented within the Rasa SDK. Note that it is not recommended to override the form action for anything else than using the deprecated Rasa SDK FormAction.
Changed the default model weights loaded for HFTransformersNLP component.
Use a language agnostic sentence embedding model as the default model. These model weights should help improve performance on intent classification and response selection.
Add validations for slot mappings. If a slot mapping is not valid, an InvalidDomain error is raised.
Adapt the training data reader and emulator for LUIS to their v3 format and add support for roles. Update the instructions in the "Migrate from LUIS" documentation page to reflect the recent changes made to the UI of LUIS.
Adapt the training data reader and emulator for DialogFlow to their latest format and add support for regex entities.
The Pika Event Broker was reimplemented with the [aio-pika library[(https://docs.aio-pika.com/). Messages will now be published to RabbitMQ asynchronously which improves the prediction performance.
The confidence of the FallbackClassifier predictions is set to 1 - top intent confidence.

Bugfixes

ActionRestart will now trigger ActionSessionStart as a followup action.
Fixed a bug with rasa data split nlu which caused the resulting train / test ratio to sometimes differ from the ratio specified by the user or by default.
The splitting algorithm ensures that every intent and response class appears in both the training and the test set. This means that each split must contain at least as many examples as there are classes, which for small datasets can contradict the requested training fraction. When this happens, the command issues a warning to the user that the requested training fraction can't be satisfied.
Fixed bug where slots with influence_conversation=false affected the action prediction if they were set manually using the POST /conversations/<conversation_id/tracker/events endpoint in the HTTP API.
Update Pika event broker to be a separate process and make it use a multiprocessing.Queue to send and process messages. This change should help avoid situations when events stop being sent after a while.
Ignore rules when validating stories
- Updated Slack Connector for new Slack Events API
Update Rasa Playground "Download" button to work correctly depending on the current chat state.
Test stories can now contain both: normal intents and retrieval intents. The failed_test_stories.yml, generated by rasa test, also specifies the full retrieval intent now. Previously rasa test would fail on test stories that specified retrieval intents.
The converter tool is now able to convert test stories that contain a number as entity type.
The converter tool now converts test stories and stories that contain full retrieval intents correctly. Previously the response keys were deleted during conversion to YAML.
The slack connector requires a configuration for slack_signing_secret to make the connector more secure. The configuration value needs to be added to your credentials.yml if you are using the slack connector.
Fixed model fingerprinting - it should avoid some more unecessary retrainings now.
Fixed a problem when slots of type text or list were referenced by name only in the training data and this was treated as an empty value. This means that the two following stories are equivalent in case the slot type is text:
stories:
- story: Story referencing slot by name
steps:
- intent: greet
- slot_was_set:
- name

- story: Story referencing slot with name and value
steps:
- intent: greet
- slot_was_set:
- name: "some name"

Note that you still need to specify values for all other slot types as only text and list slots are featurized in a binary fashion.

Improved Documentation

Correct data validation docs

Miscellaneous internal changes

Miscellaneous internal changes.

[2.0.8] - 2020-11-26

Bugfixes

Slots that use initial_value won't cause rule contradiction errors when conversation_start: true is used. Previously, two rules that differed only in their use of conversation_start would be flagged as contradicting when a slot used initial_value.
In checking for incomplete rules, an action will be required to have set only those slots that the same action has set in another rule. Previously, an action was expected to have set also slots which, despite being present after this action in another rule, were not actually set by this action.

[2.0.7] - 2020-11-24

Bugfixes

ActionRestart will now trigger ActionSessionStart as a followup action.
Fixed Rasa Open Source not being able to fetch models from certain URLs.
This addresses an issue introduced in 2.0.3 where rasa-production could not use the models from rasa-x in Rasa X server mode.
SingleStateFeaturizer checks whether it was trained with RegexInterpreter as NLU interpreter. If that is the case, RegexInterpreter is used during prediction.

[2.0.6] - 2020-11-10

Bugfixes

Fixed a bug that occurred when setting multiple Sanic workers in combination with a custom Lock Store. Previously, if the number was set higher than 1 and you were using a custom lock store, it would reject because of a strict check to use a Redis Lock Store.
Fixed a bug in the TwoStageFallback action which reverted too many events after the user successfully rephrased.

[2.0.5] - 2020-11-10

Bugfixes

Fix a bug because of which only one retrieval intent was present in all_retrieval_intent key of the output of ResponseSelector even if there were multiple retrieval intents present in the training data.

[2.0.4] - 2020-11-08

Bugfixes

Fixed error when starting Rasa X locally without a proper git setup.
Properly validate incoming webhook requests for the Slack connector to be authentic.

[2.0.3] - 2020-10-29

Bugfixes

Fix ConveRTTokenizer failing because of wrong model URL by making the model_url parameter of ConveRTTokenizer mandatory.
Since the ConveRT model was taken offline, we can no longer use the earlier public URL of the model. Additionally, since the licence for the model is unknown, we cannot host it ourselves. Users can still use the component by setting model_url to a community/self-hosted model URL or path to a local directory containing model files. For example:
pipeline:
- name: ConveRTTokenizer
model_url: <remote/local path to model>
Update example formbot to use FormValidationAction for slot validation

[2.0.2] - 2020-10-22

Bugfixes

Fix description of previous event in output of rasa data validate stories
Fixed command line coloring for windows command lines running an encoding other than utf-8.

Miscellaneous internal changes

Miscellaneous internal changes.

[2.0.1] - 2020-10-20

Bugfixes

Create correct KafkaProducer for PLAINTEXT and SASL_SSL security protocols.
- Fix YAMLStoryReader not being able to represent OR statements in conversion mode.
- Fix MarkdownStoryWriter not being able to write stories with OR statements (when loaded in conversion mode).

[2.0.0] - 2020-10-07

Deprecations and Removals

Removed previously deprecated packages rasa_nlu and rasa_core.
Use imports from rasa.core and rasa.nlu instead.
Removed previously deprecated classes:
- event brokers (EventChannel and FileProducer, KafkaProducer, PikaProducer, SQLProducer)
- intent classifier EmbeddingIntentClassifier
- policy KerasPolicy
Removed previously deprecated methods:
- Agent.handle_channels
- TrackerStore.create_tracker_store
Removed support for pipeline templates in config.yml
Removed deprecated training data keys entity_examples and intent_examples from json training data format.
Removed restaurantbot example as it was confusing and not a great way to build a bot.
LabelTokenizerSingleStateFeaturizer is deprecated. To replicate LabelTokenizerSingleStateFeaturizer functionality, add a Tokenizer with intent_tokenization_flag: True and CountVectorsFeaturizer to the NLU pipeline. An example of elements to be added to the pipeline is shown in the improvement changelog 6296`.
BinarySingleStateFeaturizer is deprecated and will be removed in the future. We recommend to switch to SingleStateFeaturizer.
Specifying the parameters force and save_to_default_model_directory as part of the JSON payload when training a model using POST /model/train is now deprecated. Please use the query parameters force_training and save_to_default_model_directory instead. See the API documentation for more information.
The conversation event form was renamed to active_loop. Rasa Open Source will continue to be able to read and process old form events. Note that serialized trackers will no longer have the active_form field. Instead the active_loop field will contain the same information. Story representations in Markdown and YAML will use active_loop instead of form to represent the event.
Removed support for queue argument in PikaEventBroker (use queues instead).
Domain file:
- Removed support for templates key (use responses instead).
- Removed support for string responses (use dictionaries instead).
NLU Component:
- Removed support for provides attribute, it's not needed anymore.
- Removed support for requires attribute (use required_components() instead).
Removed _guess_format() utils method from rasa.nlu.training_data.loading (use guess_format instead).
Removed several config options for TED Policy, DIETClassifier and ResponseSelector:
- hidden_layers_sizes_pre_dial
- hidden_layers_sizes_bot
- droprate
- droprate_a
- droprate_b
- hidden_layers_sizes_a
- hidden_layers_sizes_b
- num_transformer_layers
- num_heads
- dense_dim
- embed_dim
- num_neg
- mu_pos
- mu_neg
- use_max_sim_neg
- C2
- C_emb
- evaluate_every_num_epochs
- evaluate_on_num_examples
Please check the documentation for more information.
The conversation event form_validation was renamed to loop_interrupted. Rasa Open Source will continue to be able to read and process old form_validation events.
SklearnPolicy was deprecated. TEDPolicy is the preferred machine-learning policy for dialogue models.
Slots of type unfeaturized are now deprecated and will be removed in Rasa Open Source 3.0. Instead you should use the property influence_conversation: false for every slot type as described in the migration guide.
Conversation sessions are now enabled by default if your Domain does not contain a session configuration. Previously a missing session configuration was treated as if conversation sessions were disabled. You can explicitly disable conversation sessions using the following snippet:
domain.yml
session_config:
# A session expiration time of `0`
# disables conversation sessions
session_expiration_time: 0
Using the default action action_deactivate_form to deactivate the currently active loop / Form is deprecated. Please use action_deactivate_loop instead.

Features

Added template name to the metadata of bot utterance events.
BotUttered event contains a template_name property in its metadata for any new bot message.
Added a --num-threads CLI argument that can be passed to rasa train and will be used to train NLU components.
You can now define what kind of features should be used by what component (see Choosing a Pipeline).
You can set an alias via the option alias for every featurizer in your pipeline. The alias can be anything, by default it is set to the full featurizer class name. You can then specify, for example, on the DIETClassifier what features from which featurizers should be used. If you don't set the option featurizers all available features will be used. This is also the default behavior. Check components to see what components have the option featurizers available.
Here is an example pipeline that shows the new option. We define an alias for all featurizers in the pipeline. All features will be used in the DIETClassifier. However, the ResponseSelector only takes the features from the ConveRTFeaturizer and the CountVectorsFeaturizer (word level).
pipeline:
- name: ConveRTTokenizer
- name: ConveRTFeaturizer
alias: "convert"
- name: CountVectorsFeaturizer
alias: "cvf_word"
- name: CountVectorsFeaturizer
alias: "cvf_char"
analyzer: char_wb
min_ngram: 1
max_ngram: 4
- name: RegexFeaturizer
alias: "regex"
- name: LexicalSyntacticFeaturizer
alias: "lsf"
- name: DIETClassifier:
- name: ResponseSelector
epochs: 50
featurizers: ["convert", "cvf_word"]
- name: EntitySynonymMapper
caution
This change is model-breaking. Please retrain your models.
Added --port commandline argument to the interactive learning mode to allow changing the port for the Rasa server running in the background.
Add new entity extractor RegexEntityExtractor. The entity extractor extracts entities using the lookup tables and regexes defined in the training data. For more information see RegexEntityExtractor.
Introduced a new YAML format for Core training data and implemented a parser for it. Rasa Open Source can now read stories in both Markdown and YAML format.
You can now enable threaded message responses from Rasa through the Slack connector. This option is enabled using an optional configuration in the credentials.yml file
slack:
slack_token:
slack_channel:
use_threads: True
Button support has also been added in the Slack connector.
Add support for rules data and forms in YAML format.
The NLU interpreter is now passed to the Policies during training and inference time. Note that this requires an additional parameter interpreter in the method predict_action_probabilities of the Policy interface. In case a custom Policy implementation doesn't provide this parameter Rasa Open Source will print a warning and omit passing the interpreter.
Added the new dialogue policy RulePolicy which will replace the old “rule-like” policies Mapping Policy, Fallback Policy, Two-Stage Fallback Policy, and Form Policy. These policies are now deprecated and will be removed in the future. Please see the rules documentation for more information.
Added new NLU component FallbackClassifier which predicts an intent nlu_fallback in case the confidence was below a given threshold. The intent nlu_fallback may then be used to write stories / rules to handle the fallback in case of low NLU confidence.
pipeline:
- # Other NLU components ...
- name: FallbackClassifier
# If the highest ranked intent has a confidence lower than the threshold then
# the NLU pipeline predicts an intent `nlu_fallback` which you can then be used in
# stories / rules to implement an appropriate fallback.
threshold: 0.5
Added possibility to split the domain into separate files. All YAML files under the path specified with --domain will be scanned for domain information (e.g. intents, actions, etc) and then combined into a single domain.
The default value for --domain is still domain.yml.
Add optional metadata argument to NaturalLanguageInterpreter's parse method.
The Rasa Open Source API endpoint POST /model/train now supports training data in YAML format. Please specify the header Content-Type: application/yaml when training a model using YAML training data. See the API documentation for more information.
Added a YAML schema and a writer for 2.0 Training Core data.
Users can now use the rasa data convert {nlu|core} -f yaml command to convert training data from Markdown format to YAML format.
Add option use_lemma to CountVectorsFeaturizer. By default it is set to True.
use_lemma indicates whether the featurizer should use the lemma of a word for counting (if available) or not. If this option is set to False it will use the word as it is.

Improvements

Add support for Python 3.8.
Changed the project structure for Rasa projects initialized with the CLI (using the rasa init command): actions.py -> actions/actions.py. actions is now a Python package (it contains a file actions/__init__.py). In addition, the __init__.py at the root of the project has been removed.
DIETClassifier now also assigns a confidence value to entity predictions.
Added behavior to the rasa --version command. It will now also list information about the operating system, python version and rasa-sdk. This will make it easier for users to file bug reports.
Support for additional training metadata.
Training data messages now to support kwargs and the Rasa JSON data reader includes all fields when instantiating a training data instance.
Standardize testing output. The following test output can be produced for intents, responses, entities and stories:
- report: a detailed report with testing metrics per label (e.g. precision, recall, accuracy, etc.)
- errors: a file that contains incorrect predictions
- successes: a file that contains correct predictions
- confusion matrix: plot of confusion matrix
- histogram: plot of confidence distribution (not available for stories)
To avoid the problem of our entity extractors predicting entity labels for just a part of the words, we introduced a cleaning method after the prediction was done. We should avoid the incorrect prediction in the first place. To achieve this we will not tokenize words into sub-words anymore. We take the mean feature vectors of the sub-words as the feature vector of the word.
caution
This change is model breaking. Please, retrain your models.
Move option case_sensitive from the tokenizers to the featurizers.
- Remove the option from the WhitespaceTokenizer and ConveRTTokenizer.
- Add option case_sensitive to the RegexFeaturizer.
If a user sends a voice message to the bot using Facebook, users messages was set to the attachments URL. The same is now also done for the rest of attachment types (image, video, and file).
Creating a Domain using Domain.fromDict can no longer alter the input dictionary. Previously, there could be problems when the input dictionary was re-used for other things after creating the Domain from it.
The debug-level logs when instantiating an SQLTrackerStore no longer show the password in plain text. Now, the URL is displayed with the password hidden, e.g. postgresql://username:***@localhost:5432.
Shorten the information in tqdm during training ML algorithms based on the log level. If you train your model in debug mode, all available metrics will be shown during training, otherwise, the information is shorten.
Ignore conversation test directory tests/ when importing a project using MultiProjectImporter and use_e2e is False. Previously, any story data found in a project subdirectory would be imported as training data.
Implemented model checkpointing for DIET (including the response selector) and TED. The best model during training will be stored instead of just the last model. The model is evaluated on the basis of evaluate_every_number_of_epochs and evaluate_on_number_of_examples.
Checkpointing is enabled iff the following is set for the models in the config.yml file:
- checkpoint_model: True
- evaluate_on_number_of_examples > 0
The model is stored to whatever location has been specified with the --out parameter when calling rasa train nlu/core ....
rasa data split nlu now makes sure that there is at least one example per intent and response in the test data.
The method ensure_consistent_bilou_tagging now also considers the confidence values of the predicted tags when updating the BILOU tags.
We updated the way how we save and use features in our NLU pipeline.
The message object now has a dedicated field, called features, to store the features that are generated in the NLU pipeline. We adapted all our featurizers in a way that sequence and sentence features are stored independently. This allows us to keep different kind of features for the sequence and the sentence. For example, the LexicalSyntacticFeaturizer does not produce any sentence features anymore as our experiments showed that those did not bring any performance gain just quite a lot of additional values to store.
We also modified the DIET architecture to process the sequence and sentence features independently at first. The features are concatenated just before the transformer.
We also removed the __CLS__ token again. Our Tokenizers will not add this token anymore.
caution
This change is model-breaking. Please retrain your models.
Add endpoint kwarg to rasa.jupyter.chat to enable using a custom action server while chatting with a model in a jupyter notebook.
Support for rasa conversation id with special characters on the server side - necessary for some channels (e.g. Viber)
Add support for proxy use in slack input channel.
Log the number of examples per intent during training. Logging can be enabled using rasa train --debug.
Support for other remote storages can be achieved by using an external library.
Add output_channel query param to /conversations/<conversation_id>/tracker/events route, along with boolean execute_side_effects to optionally schedule/cancel reminders, and forward bot messages to output channel.
Allow Rasa to boot when model loading exception occurs. Forward HTTP Error responses to standard log output.
Rename DucklingHTTPExtractor to DucklingEntityExtractor.
- Modified functionality of SingleStateFeaturizer.
  SingleStateFeaturizer uses trained NLU Interpreter to featurize intents and action names. This modified SingleStateFeaturizer can replicate LabelTokenizerSingleStateFeaturizer functionality. This component is deprecated from now on. To replicate LabelTokenizerSingleStateFeaturizer functionality, add a Tokenizer with intent_tokenization_flag: True and CountVectorsFeaturizer to the NLU pipeline. Please update your configuration file.
  For example:
  language: en
  pipeline:
  - name: WhitespaceTokenizer
  intent_tokenization_flag: True
  - name: CountVectorsFeaturizer
  Please train both NLU and Core (using rasa train) to use a trained tokenizer and featurizer for core featurization.
  The new SingleStateFeaturizer stores slots, entities and forms in sparse features for more lightweight storage.
  BinarySingleStateFeaturizer is deprecated and will be removed in the future. We recommend to switch to SingleStateFeaturizer.
- Modified TEDPolicy to handle sparse features. As a result, TEDPolicy may require more epochs than before to converge.
- Default TEDPolicy featurizer changed to MaxHistoryTrackerFeaturizer with infinite max history (takes all dialogue turns into account).
- Default batch size for TED increased from [8,32] to [64, 256]
Response selector templates now support all features that domain utterances do. They use the yaml format instead of markdown now. This means you can now use buttons, images, ... in your FAQ or chitchat responses (assuming they are using the response selector).
As a consequence, training data form in markdown has to have the file suffix .md from now on to allow proper file type detection-
Support for test stories written in yaml format.
Response Selectors are now trained on retrieval intent labels by default instead of the actual response text. For most models, this should improve training time and accuracy of the ResponseSelector.
If you want to revert to the pre-2.0 default behavior, add the use_text_as_label=true parameter to your ResponseSelector component.
You can now also have multiple response templates for a single sub-intent of a retrieval intent. The first response template containing the text attribute is picked for training(if use_text_as_label=True) and a random template is picked for bot's utterance just as how other utter_ templates are picked.
All response selector related evaluation artifacts - report.json, successes.json, errors.json, confusion_matrix.png now use the sub-intent of the retrieval intent as the target and predicted labels instead of the actual response text.
The output schema of ResponseSelector has changed - full_retrieval_intent and name have been deprecated in favour of intent_response_key and response_templates respectively. Additionally a key all_retrieval_intents is added to the response selector output which will hold a list of all retrieval intents(faq,chitchat, etc.) that are present in the training data.An example output looks like this -
"response_selector": {
"all_retrieval_intents": ["faq"],
"default": {
"response": {
"id": 1388783286124361986, "confidence": 1.0, "intent_response_key": "faq/is_legit",
"response_templates": [
{
"text": "absolutely",
"image": "https://i.imgur.com/nGF1K8f.jpg"
},
{
"text": "I think so."
}
],
},
"ranking": [
{
"id": 1388783286124361986,
"confidence": 1.0,
"intent_response_key": "faq/is_legit"
},
]
An example bot demonstrating how to use the ResponseSelector is added to the examples folder.
Do not modify conversation tracker's latest_input_channel property when using POST /trigger_intent or ReminderScheduled.
Do not set the output dimension of the sparse-to-dense layers to the same dimension as the dense features.
Update default value of dense_dimension and concat_dimension for text in DIETClassifier to 128.
Retrieval actions with respond_ prefix are now replaced with usual utterance actions with utter_ prefix.
If you were using retrieval actions before, rename all of them to start with utter_ prefix. For example, respond_chitchat becomes utter_chitchat. Also, in order to keep the response templates more consistent, you should now add the utter_ prefix to all response templates defined for retrieval intents. For example, a response template chitchat/ask_name becomes utter_chitchat/ask_name. Note that the NLU examples for this will still be under chitchat/ask_name intent. The example responseselectorbot should help clarify these changes further.
Added telemetry reporting. Rasa uses telemetry to report anonymous usage information. This information is essential to help improve Rasa Open Source for all users. Reporting will be opt-out. More information can be found in our telemetry documentation.
Update extract_other_slots method inside FormAction to fill a slot from an entity with a different name if corresponding slot mapping of from_entity type is unique.
Slots of any type can now be ignored during a conversation. To do so, specify the property influence_conversation: false for the slot.
slot:
a_slot:
type: text
influence_conversation: false
The property influence_conversation is set to true by default. See the documentation for slots for more information.
A new slot type any was added. Slots of this type can store any value. Slots of type any are always ignored during conversations.
Improved exception handling within Rasa Open Source.
All exceptions that are somewhat expected (e.g. errors in file formats like configurations or training data) will share a common base class RasaException.
::warning Backwards Incompatibility Base class for the exception raised when an action can not be found has been changed from a NameError to a ValueError. ::
Some other exceptions have also slightly changed:
- raise YamlSyntaxException instead of YAMLError (from ruamel) when failing to load a yaml file with information about the line where loading failed
- introduced MissingDependencyException as an exception raised if packages need to be installed
Debug logs from matplotlib libraries are now hidden by default and are configurable with the LOG_LEVEL_LIBRARIES environment variable.
Update KafkaEventBroker to support SASL_SSL and PLAINTEXT protocols.

Bugfixes

Fixed issue where temporary model directories were not removed after pulling from a model server.
If the model pulled from the server was invalid, this could lead to large amounts of local storage usage.
Fixed a bug in the CountVectorsFeaturizer which resulted in the very first message after loading a model to be processed incorrectly due to the vocabulary not being loaded yet.
Fixed Rasa shell skipping button messages if buttons are attached to a message previous to the latest.
Stack level for FutureWarning updated to level 2.
If custom utter message contains no value or integer value, then it fails returning custom utter message. Fixed by converting the template to type string.
Don't create TensorBoard log files during prediction.
Fixed DIET breaking with empty spaCy model.
Pinned the library version for the Azure Cloud Storage to 2.1.0 since the persistor is currently not compatible with later versions of the azure-storage-blob library.
Remove clean_up_entities from extractors that extract pre-defined entities. Just keep the clean up method for entity extractors that extract custom entities.
Fixed issue where the DucklingHTTPExtractor component would not work if its url contained a trailing slash.
Changed to variable CERT_URI in hangouts.py to a string type
Slots will be correctly interpolated for button responses.
Previously this resulted in no interpolation due to a bug.
Remove option token_pattern from CountVectorsFeaturizer. Instead all tokenizers now have the option token_pattern. If a regular expression is set, the tokenizer will apply the token pattern.
Allow user to retry failed file exports in interactive training.
Fixed a bug when custom metadata passed with the utterance always restarted the session.
WhitespaceTokenizer does not remove vowel signs in Hindi anymore.
Convert entity values coming from DucklingHTTPExtractor to string during evaluation to avoid mismatches due to different types.
Update FeatureSignature to store just the feature dimension instead of the complete shape. This change fixes the usage of the option share_hidden_layers in the DIETClassifier.
Unescape the \n, \t, \r, \f, \b tokens on reading nlu data from markdown files.
On converting json files into markdown, the tokens mentioned above are espaced. These tokens need to be unescaped on loading the data from markdown to ensure that the data is treated in the same way.
Fix the way training data is generated in rasa test nlu when using the -P flag. Each percentage of the training dataset used to be formed as a part of the last sampled training dataset and not as a sample from the original training dataset.
Prevent WhitespaceTokenizer from outputting empty list of tokens.
Add EntityExtractor as a required component for EntitySynonymMapper in a pipeline.
Better handling of input sequences longer than the maximum sequence length that the HFTransformersNLP models can handle.
During training, messages with longer sequence length should result in an error, whereas during inference they are gracefully handled but a debug message is logged. Ideally, passing messages longer than the acceptable maximum sequence lengths of each model should be avoided.
When using the DynamoTrackerStore, if there are more than 100 DynamoDB tables, the tracker could attempt to re-create an existing table if that table was not among the first 100 listed by the dynamo API.
Fixed a deprication warning that pops up due to changes in numpy
Update rasabaster to fix an issue with syntax highlighting on "Prototype an Assistant" page.
Update default stories and rules on "Prototype an Assistant" page.
Fixed a bug in the serialise method of the EvaluationStore class which resulted in a wrong end-to-end evaluation of the predicted entities.
Forms with slot mappings defined in domain.yml must now be a dictionary (with form names as keys). The previous syntax where forms was simply a list of form names is still supported.
Remove BILOU tag prefix from role and group labels when creating entities.
Fixed a bug in the featurization of the boolean slot type. Previously, to set a slot value to "true", you had to set it to "1", which is in conflict with the documentation. In older versions true (without quotes) was also possible, but now raised an error during yaml validation.
Fixed a bug in rasa interactive. Now it exports the stories and nlu training data as yml file.
Fixed slots not being featurized before first user utterance.
Fixed AugmentedMemoizationPolicy to forget the first action on the first going back
Fixed the remote URL of ConveRT model as it was recently updated by its authors.
Treat the length of OOV token as 1 to fix token align issue when OOV occurred.
Fixed the bug when entity was extracted even if it had a role or group but roles or groups were not expected.
Fixed the bug that caused supported_language_list of Component to not work correctly.
To avoid confusion, only one of supported_language_list and not_supported_language_list can be set to not None now
Fixed issue where responses including text: "" and no custom key would incorrectly fail domain validation.
Fixed issue where extra keys other than title and payload inside of buttons made a response fail domain validation.
Do not filter training data in model.py but on component side.
Check if a model was provided when executing rasa test core. If not, print a useful error message and stop.
Transfer only response templates for retrieval intents from domain to NLU Training Data.
This avoids retraining the NLU model if one of the non retrieval intent response templates are edited.

Improved Documentation

Added documentation on ambiguity_threshold parameter in Fallback Actions page.
Remove outdated whitespace tokenizer warning in Testing Your Assistant documentation.
Updated Facebook Messenger channel docs with supported attachment information
Update rasa shell documentation to explain how to recreate external channel session behavior.
Event brokers documentation should say url instead of host.
Update rasa init documentation to include tests/conversation_tests.md in the resulting directory tree.
Update "Validating Form Input" section to include details about how FormValidationAction class makes it easier to validate form slots in custom actions and how to use it.
Update the examples in the API docs to use YAML instead of Markdown

Miscellaneous internal changes

Miscellaneous internal changes.

[1.10.26] - 2021-06-17

Features

Added sasl_mechanism as an optional configurable parameter for the Kafka Producer.

[1.10.25] - 2021-04-14

Features

Added partition_by_sender flag to Kafka Producer to optionally associate events with Kafka partition based on sender_id.

Improvements

Improved the lock store debug log message when the process has to queue because other messages have to be processed before this item.

[1.10.24] - 2021-03-29

Bugfixes

Added group_id parameter back to KafkaEventBroker to fix error when instantiating event broker with a config containing the group_id parameter which is only relevant to the event consumer

[1.10.23] - 2021-02-22

Bugfixes

Fixed bug where the conversation does not lock before handling a reminder event.

[1.10.22] - 2021-02-05

Bugfixes

Backported the Rasa Open Source 2 PikaEventBroker implementation to address problems when using it with multiple Sanic workers.

[1.10.21] - 2021-02-01

Improvements

The url option now supports a list of servers url: ['10.0.0.158:32803','10.0.0.158:32804']. Removed group_id because it is not a valid Kafka producer parameter.

Bugfixes

Fixed a bug that occurred when setting multiple Sanic workers in combination with a custom Lock Store. Previously, if the number was set higher than 1 and you were using a custom lock store, it would reject because of a strict check to use a Redis Lock Store.
Fix a bug where, if a user injects an intent using the HTTP API, slot auto-filling is not performed on the entities provided.

[1.10.20] - 2020-12-18

Bugfixes

Fix scikit-learn crashing during evaluation of ResponseSelector predictions.

[1.10.19] - 2020-12-17

Improvements

Kafka Producer connection now remains active across sends. Added support for group and client id. The Kafka producer also adds support for the PLAINTEXT and SASL_SSL protocols.
DynamoDB table exists check fixed bug when more than 100 tables exist.
Replace use of python-telegram-bot package with pyTelegramBotAPI
Use response selector keys (sub-intents) as labels for plotting the confusion matrix during NLU evaluation to improve readability.

[1.10.18] - 2020-11-26

Bugfixes

#7340: Fixed an issues with the DynamoDB TrackerStore creating a new table entry/object for each TrackerStore update. The column session_date has been deprecated and should be removed manually in existing DynamoDB tables.

[1.10.17] - 2020-11-12

Bugfixes

Prevent the message handling process in PikaEventBroker from being terminated.

[1.10.16] - 2020-10-15

Bugfixes

Update Pika event broker to be a separate process and make it use a multiprocessing.Queue to send and process messages. This change should help avoid situations when events stop being sent after a while.

[1.10.15] - 2020-10-09

Bugfixes

Fixed issue where temporary model directories were not removed after pulling from a model server. If the model pulled from the server was invalid, this could lead to large amounts of local storage usage.
Treat the length of OOV token as 1 to fix token align issue when OOV occurred.
Fixed MappingPolicy not predicting action_listen after the mapped action while running rasa test.

Improvements

Debug logs from matplotlib libraries are now hidden by default and are configurable with the LOG_LEVEL_LIBRARIES environment variable.

[1.10.14] - 2020-09-23

Bugfixes

Fixed the remote URL of ConveRT model as it was recently updated by its authors. Also made the remote URL configurable at runtime in the corresponding tokenizer's and featurizer's configuration.

[1.10.13] - 2020-09-22

Bugfixes

Remove BILOU tag prefix from role and group labels when creating entities.

[1.10.12] - 2020-09-03

Bugfixes

Fix slow training of CRFEntityExtractor when using Entity Roles and Groups.

[1.10.11] - 2020-08-21

Improvements

Do not deepcopy slots when instantiating trackers. This leads to a significant speedup when training on domains with a large number of slots.
Added more debugging logs to the Lock Stores to simplify debugging in case of
connection problems.
Added a new parameter socket_timeout to the RedisLockStore. If Redis doesn't answer within socket_timeout seconds to requests from Rasa Open Source, an error is raised. This avoids seemingly infinitely blocking connections and exposes connection problems early.

Bugfixes

Fixed a bug where domain fields such as store_entities_as_slots were overridden with defaults and therefore ignored.
If two entities are separated by a comma (or any other symbol), extract them as two separate entities.
If two entities are separated by a single space and uses BILOU tagging, extract them as two separate entities based on their BILOU tags.

[1.10.10] - 2020-08-04

Bugfixes

Fixed TypeError: expected string or bytes-like object issue caused by integer, boolean, and null values in templates.

[1.10.9] - 2020-07-29

Improvements

Rasa Open Source will no longer add responses to the actions section of the domain when persisting the domain as a file. This addresses related problems in Rasa X when Integrated Version Control introduced big diffs due to the added utterances in the actions section.

Bugfixes

Consider entity roles/groups during interactive learning.

[1.10.8] - 2020-07-15

Bugfixes

Add 'Access-Control-Expose-Headers' for 'filename' header
Fixed a bug where an invalid language variable prevents rasa from finding training examples when importing Dialogflow data.

[1.10.7] - 2020-07-07

Features

Add not_supported_language_list to component to be able to define languages that a component can NOT handle.
WhitespaceTokenizer is not able to process languages which are not separated by whitespace. WhitespaceTokenizer will throw an error if it is used with Chinese, Japanese, and Thai.

Bugfixes

WhitespaceTokenizer only removes emoji if complete token matches emoji regex.

[1.10.6] - 2020-07-06

Bugfixes

Prevent WhitespaceTokenizer from outputting empty list of tokens.

[1.10.5] - 2020-07-02

Bugfixes

Explicitly remove all emojis which appear as unicode characters from the output of regex.sub inside WhitespaceTokenizer.

[1.10.4] - 2020-07-01

Bugfixes

WhitespaceTokenizer does not remove vowel signs in Hindi anymore.
Previously, specifying a lock store in the endpoint configuration with a type other than redis or in_memory would lead to an AttributeError: 'str' object has no attribute 'type'. This bug is fixed now.
Fix Interpreter parsed an intent ... warning when using the /model/parse endpoint with an NLU-only model.
Convert entity values coming from any entity extractor to string during evaluation to avoid mismatches due to different types.
The assistant will respond through the webex channel to any user (room) communicating to it. Before the bot responded only to a fixed roomId set in the credentials.yml config file.

[1.10.3] - 2020-06-12

Improvements

Reduced duplicate logs and warnings when running rasa train.

Bugfixes

Remove the clean_up_entities method from the DIETClassifier and CRFEntityExtractor as it let to incorrect entity predictions.
Fix server crashes that occurred when Rasa Open Source pulls a model from a model server and an exception was thrown during model loading (such as a domain with invalid YAML).

[1.10.2] - 2020-06-03

Bugfixes

Responses used in ResponseSelector now support new lines with explicitly adding \\n between them.
Fixed a bug in rasa export) which caused Rasa Open Source to only migrate conversation events from the last Session configuration.

[1.10.1] - 2020-05-15

Improvements

Creating a Domain using Domain.fromDict can no longer alter the input dictionary. Previously, there could be problems when the input dictionary was re-used for other things after creating the Domain from it.

Bugfixes

Don't create TensorBoard log files during prediction.
Fix: DIET breaks with empty spaCy model
Remove clean_up_entities from extractors that extract pre-defined entities. Just keep the clean up method for entity extractors that extract custom entities.
Fixed issue where the DucklingHTTPExtractor component would not work if its url contained a trailing slash.
Fix list index out of range error in ensure_consistent_bilou_tagging.

Miscellaneous internal changes

Miscellaneous internal changes.

[1.10.0] - 2020-04-28

Features

Add support for entities with roles and grouping of entities in Rasa NLU.
You can now define a role and/or group label in addition to the entity type for entities. Use the role label if an entity can play different roles in your assistant. For example, a city can be a destination or a departure city. The group label can be used to group multiple entities together. For example, you could group different pizza orders, so that you know what toppings goes with which pizza and what size which pizza has. For more details see Entities Roles and Groups.
To fill slots from entities with a specific role/group, you need to either use forms or use a custom action. We updated the tracker method get_latest_entity_values to take an optional role/group label. If you want to use a form, you can add the specific role/group label of interest to the slot mapping function from_entity (see Forms).
note
Composite entities are currently just supported by the DIETClassifier and CRFEntityExtractor.
Update training data format for NLU to support entities with a role or group label.
You can now specify synonyms, roles, and groups of entities using the following data format: Markdown:
[LA]{"entity": "location", "role": "city", "group": "CA", "value": "Los Angeles"}
JSON:
"entities": [
{
"start": 10,
"end": 12,
"value": "Los Angeles",
"entity": "location",
"role": "city",
"group": "CA",
}
]
The markdown format [LA](location:Los Angeles) is deprecated. To update your training data file just execute the following command on the terminal of your choice: sed -i -E 's/\\[([^)]+)\\]\$([^)]+):([^)]+)\$/[\\1]{"entity": "\\2", "value": "\\3"}/g' nlu.md
For more information about the new data format see Training Data Format.

Improvements

Suppressed pika logs when establishing the connection. These log messages mostly happened when Rasa X and RabbitMQ were started at the same time. Since RabbitMQ can take a few seconds to initialize, Rasa X has to re-try until the connection is established. In case you suspect a different problem (such as failing authentication) you can re-enable the pika logs by setting the log level to DEBUG. To run Rasa Open Source in debug mode, use the --debug flag. To run Rasa X in debug mode, set the environment variable DEBUG_MODE to true.
Include the source filename of a story in the failed stories
Include the source filename of a story in the failed stories to make it easier to identify the file which contains the failed story.
Add confusion matrix and “confused_with” to response selection evaluation
If you are using ResponseSelectors, they now produce similiar outputs during NLU evaluation. Misclassfied responses are listed in a “confused_with” attribute in the evaluation report. Similiarily, a confusion matrix of all responses is plotted.
Added socketio to the compatible channels for Reminders and External Events.
Update POST /model/train endpoint to accept retrieval action responses at the responses key of the JSON payload.
All Rasa Open Source images are now using Python 3.7 instead of Python 3.6.
Update dependencies based on the dependabot check.
Add dropout between FFNN and DenseForSparse layers in DIETClassifier, ResponseSelector and EmbeddingIntentClassifier controlled by use_dense_input_dropout config parameter.
DIETClassifier only counts as extractor in rasa test if it was actually trained for entity recognition.
Remove regularization gradient for variables that don't have prediction gradient.
Raise a warning in CRFEntityExtractor and DIETClassifier if entities are not correctly annotated in the training data, e.g. their start and end values do not match any start and end values of tokens.
Add full_retrieval_intent property to ResponseSelector rankings
Change default values for hyper-parameters in EmbeddingIntentClassifier and DIETClassifier
Use scale_loss=False in DIETClassifier. Reduce the number of dense dimensions for sparse features of text from 512 to 256 in EmbeddingIntentClassifier.

Bugfixes

Fixed issue where posting to certain callback channel URLs would return a 500 error on successful posts due to invalid response format.
One word can just have one entity label.
If you are using, for example, ConveRTTokenizer words can be split into multiple tokens. Our entity extractors assign entity labels per token. So, it might happen, that a word, that was split into two tokens, got assigned two different entity labels. This is now fixed. One word can just have one entity label at a time.
An entity label should always cover a complete word.
If you are using, for example, ConveRTTokenizer words can be split into multiple tokens. Our entity extractors assign entity labels per token. So, it might happen, that just a part of a word has an entity label. This is now fixed. An entity label always covers a complete word.
Fixed an issue that happened when metadata is passed in a new session.
Now the metadata is correctly passed to the ActionSessionStart.
Updated Python dependency ruamel.yaml to >=0.16. We recommend to use at least 0.16.10 due to the security issue CVE-2019-20478 which is present in in prior versions.

Miscellaneous internal changes

Miscellaneous internal changes.

[1.9.7] - 2020-04-23

Improvements

The stream reading timeout for rasa shell\ is now configurable by using the environment variable ``RASA_SHELL_STREAM_READING_TIMEOUT_IN_SECONDS. This can help to fix problems when usingrasa shell` with custom actions which run 10 seconds or longer.

Bugfixes

Reverted changes in 1.9.6 that led to model incompatibility. Upgrade to 1.9.7 to fix self.sequence_lengths_for(tf_batch_data[TEXT_SEQ_LENGTH][0]) IndexError: list index out of range error without needing to retrain earlier 1.9 models.
Therefore, all 1.9 models except for 1.9.6 will be compatible; a model trained on 1.9.6 will need to be retrained on 1.9.7.

[1.9.6] - 2020-04-15

Bugfixes

Fix rasa test nlu plotting when using multiple runs.
Fixed issue where max_number_of_predictions was not considered when running end-to-end testing.

Miscellaneous internal changes

Miscellaneous internal changes.

[1.9.5] - 2020-04-01

Improvements

Support for PostgreSQL schemas in SQLTrackerStore. The SQLTrackerStore accesses schemas defined by the POSTGRESQL_SCHEMA environment variable if connected to a PostgreSQL database.
The schema is added to the connection string option's -csearch_path key, e.g. -options=-csearch_path=<SCHEMA_NAME> (see the PostgreSQL docs for more details). As before, if no POSTGRESQL_SCHEMA is defined, Rasa uses the database's default schema (public).
The schema has to exist in the database before connecting, i.e. it needs to have been created with
CREATE SCHEMA schema_name;

Bugfixes

Fixed ambiguous logging in DIETClassifier by adding the name of the calling class to the log message.

[1.9.4] - 2020-03-30

Bugfixes

Fix memory leak problem on increasing number of calls to /model/parse endpoint.

[1.9.3] - 2020-03-27

Bugfixes

Set default value for weight_sparsity in ResponseSelector to 0. This fixes a bug in the default behavior of ResponseSelector which was accidentally introduced in rasa==1.8.0. Users should update to this version and re-train their models if ResponseSelector was used in their pipeline.

[1.9.2] - 2020-03-26

Improved Documentation

Fix documentation to bring back Sara.

[1.9.1] - 2020-03-25

Bugfixes

Fix an issue where the deprecated queue parameter for the Pika Event Broker was ignored and Rasa Open Source published the events to the rasa_core_events queue instead. Note that this does not change the fact that the queue argument is deprecated in favor of the queues argument.

[1.9.0] - 2020-03-24

Features

Channel hangouts for Rasa integration with Google Hangouts Chat is now supported out-of-the-box.
Add an optional path to a specific directory to download and cache the pre-trained model weights for HFTransformersNLP.
Add options tensorboard_log_directory and tensorboard_log_level to EmbeddingIntentClassifier, DIETClasifier, ResponseSelector, EmbeddingPolicy and TEDPolicy.
By default tensorboard_log_directory is None. If a valid directory is provided, metrics are written during training. After the model is trained you can take a look at the training metrics in tensorboard. Execute tensorboard --logdir <path-to-given-directory>.
Metrics can either be written after every epoch (default) or for every training step. You can specify when to write metrics using the variable tensorboard_log_level. Valid values are 'epoch' and 'minibatch'.
We also write down a model summary, i.e. layers with inputs and types, to the given directory.

Improvements

Make response timeout configurable. rasa run, rasa shell and rasa x can now be started with --response-timeout <int> to configure a response timeout of <int> seconds.
Add full retrieval intent name to message data ResponseSelector will now add the full retrieval intent name e.g. faq/which_version to the prediction, making it accessible from the tracker.
Added PikaEventBroker (Pika Event Broker) support for publishing to multiple queues. Messages are now published to a fanout exchange with name rasa-exchange (see exchange-fanout for more information on fanout exchanges).
The former queue key is deprecated. Queues should now be specified as a list in the endpoints.yml event broker config under a new key queues. Example config:
event_broker:
type: pika
url: localhost
username: username
password: password
queues:
- queue-1
- queue-2
- queue-3
Change rasa init to include tests/conversation_tests.md file by default.
The endpoint PUT /conversations/<conversation_id>/tracker/events no longer adds session start events (to learn more about conversation sessions, please see Session configuration) in addition to the events which were sent in the request payload. To achieve the old behavior send a GET /conversations/<conversation_id>/tracker request before appending events.
Make scale_loss for intents behave the same way as in versions below 1.8, but only scale if some of the examples in a batch has probability of the golden label more than 0.5. Introduce scale_loss for entities in DIETClassifier.

Bugfixes

Fixed the bug when FormPolicy was overwriting MappingPolicy prediction (e.g. /restart). Priorities for Mapping Policy and Form Policy are no longer linear: FormPolicy priority is 5, but its prediction is ignored if MappingPolicy is used for prediction.
Fixed issue related to storing Python float values as decimal.Decimal objects in DynamoDB tracker stores. All decimal.Decimal objects are now converted to float on tracker retrieval.
Added a new docs section on DynamoTrackerStore.
Fixed bug where FallbackPolicy would always fall back if the fallback action is action_listen.
Fixed bug where starting or ending a response with \\n\\n led to one of the responses returned being empty.
Fixes issue where model always gets retrained if multiple NLU/story files are in a directory, by sorting the list of files.
Fixed ambiguous logging in DIETClassifier by adding the name of the calling class to the log message.

Improved Documentation

Restructure the “Evaluating models” documentation page and rename this page to Testing Your Assistant.
Improved documentation on how to build and deploy an action server image for use on other servers such as Rasa X deployments.

Miscellaneous internal changes

Miscellaneous internal changes.

[1.8.3] - 2020-03-27

Bugfixes

Fixes issue where model always gets retrained if multiple NLU/story files are in a directory, by sorting the list of files.
Fixed ambiguous logging in DIETClassifier by adding the name of the calling class to the log message.
Set default value for weight_sparsity in ResponseSelector to 0. This fixes a bug in the default behavior of ResponseSelector which was accidentally introduced in rasa==1.8.0. Users should update to this version or rasa>=1.9.3 and re-train their models if ResponseSelector was used in their pipeline.

Improved Documentation

Improved documentation on how to build and deploy an action server image for use on other servers such as Rasa X deployments.

[1.8.2] - 2020-03-19

Bugfixes

Fixed bug when installing rasa with poetry.
Fixed bug with EmbeddingIntentClassifier, where results weren't the same as in 1.7.x. Fixed by setting weight sparsity to 0.

Improved Documentation

Explain how to run commands as root user in Rasa SDK Docker images since version 1.8.0. Since version 1.8.0 the Rasa SDK Docker images does not longer run as root user by default. For commands which require root user usage, you have to switch back to the root user in your Docker image as described in Building an Action Server Image.
Made improvements to Building Assistants tutorial

[1.8.1] - 2020-03-06

Bugfixes

Fixed issue with using language models like xlnet along with entity_recognition set to True inside DIETClassifier.

Miscellaneous internal changes

Miscellaneous internal changes.

[1.8.0] - 2020-02-26

Deprecations and Removals

Removed Agent.continue_training and the dump_flattened_stories parameter from Agent.persist.
Properties Component.provides and Component.requires are deprecated. Use Component.required_components() instead.

Features

Add default value __other__ to values of a CategoricalSlot.
All values not mentioned in the list of values of a CategoricalSlot will be mapped to __other__ for featurization.
Add story structure validation functionality (e.g. rasa data validate stories –max-history 5).
Add LexicalSyntacticFeaturizer to sparse featurizers.
LexicalSyntacticFeaturizer does the same featurization as the CRFEntityExtractor. We extracted the featurization into a separate component so that the features can be reused and featurization is independent from the entity extraction.
Integrate language models from HuggingFace's Transformers Library.
Add a new NLP component HFTransformersNLP which tokenizes and featurizes incoming messages using a specified pre-trained model with the Transformers library as the backend. Add LanguageModelTokenizer and LanguageModelFeaturizer which use the information from HFTransformersNLP and sets them correctly for message object. Language models currently supported: BERT, OpenAIGPT, GPT-2, XLNet, DistilBert, RoBERTa.
Added a new CLI command rasa export to publish tracker events from a persistent tracker store using an event broker. See Export Conversations to an Event Broker, Tracker Stores and Event Brokers for more details.
Refactor how GPU and CPU environments are configured for TensorFlow 2.0.
Please refer to the documentation on Configuring TensorFlow to understand which environment variables to set in what scenarios. A couple of examples are shown below as well:
# This specifies to use 1024 MB of memory from GPU with logical ID 0 and 2048 MB of memory from GPU with logical ID 1
TF_GPU_MEMORY_ALLOC="0:1024, 1:2048"

# Specifies that at most 3 CPU threads can be used to parallelize multiple non-blocking operations
TF_INTER_OP_PARALLELISM_THREADS="3"

# Specifies that at most 2 CPU threads can be used to parallelize a particular operation.
TF_INTRA_OP_PARALLELISM_THREADS="2"
Added a new NLU component DIETClassifier and a new policy TEDPolicy.
DIET (Dual Intent and Entity Transformer) is a multi-task architecture for intent classification and entity recognition. You can read more about this component in the DIETClassifier documentation. The new component will replace the EmbeddingIntentClassifier and the CRFEntityExtractor in the future. Those two components are deprecated from now on. See migration guide for details on how to switch to the new component.
TEDPolicy is the new name for EmbeddingPolicy. EmbeddingPolicy is deprecated from now on. The functionality of TEDPolicy and EmbeddingPolicy is the same. Please update your configuration file to use the new name for the policy.
The sentence vector of the SpacyFeaturizer and MitieFeaturizer can be calculated using max or mean pooling.
To specify the pooling operation, set the option pooling for the SpacyFeaturizer or the MitieFeaturizer in your configuration file. The default pooling operation is mean. The mean pooling operation also does not take into account words, that do not have a word vector.

Improvements

Added command line argument --conversation-id to rasa interactive. If the argument is not given, conversation_id defaults to a random uuid.
Added a new command-line argument --init-dir to command rasa init to specify the directory in which the project is initialised.
Added support to send images with the twilio output channel.
Part of Slack sanitization: Multiple garbled URL's in a string coming from slack will be converted into actual strings. Example: health check of <http://eemdb.net|eemdb.net> and <http://eemdb1.net|eemdb1.net> to health check of eemdb.net and eemdb1.net
New command-line argument –conversation-id will be added and wiil give the ability to set specific conversation ID for each shell session, if not passed will be random.
Messages sent to the Pika Event Broker are now persisted. This guarantees the RabbitMQ will re-send previously received messages after a crash. Note that this does not help for the case where messages are sent to an unavailable RabbitMQ instance.
Added support for mattermost connector to use bot accounts.
We updated our code to TensorFlow 2.
Events exported using rasa export receive a message header if published through a PikaEventBroker. The header is added to the message's BasicProperties.headers under the rasa-export-process-id key (rasa.core.constants.RASA_EXPORT_PROCESS_ID_HEADER_NAME). The value is a UUID4 generated at each call of rasa export. The resulting header is a key-value pair that looks as follows:
'rasa-export-process-id': 'd3b3d3ffe2bd4f379ccf21214ccfb261'
Added followlinks=True to os.walk calls, to allow the use of symlinks in training, NLU and domain data.
Support invoking a SlackBot by direct messaging or @<app name> mentions.

Bugfixes

Fixed timestamp parsing warning when using DucklingHTTPExtractor
Fixed issue with action_restart getting overridden by action_listen when the Mapping Policy and the Two-Stage Fallback Policy are used together.
Fixed incorrectly raised Error encountered in pipelines with a ResponseSelector and NLG.
When NLU training data is split before NLU pipeline comparison, NLG responses were not also persisted and therefore training for a pipeline including the ResponseSelector would fail.
NLG responses are now persisted along with NLU data to a /train directory in the run_x/xx%_exclusion folder.
Fixed sending custom json with Twilio channel

Improved Documentation

Updated the documentation to properly suggest not to explicitly add utterance actions to the domain.
Added user guide for reminders and external events, including reminderbot demo.

Miscellaneous internal changes

Miscellaneous internal changes.

[1.7.4] - 2020-02-24

Bugfixes

Tracker stores supporting conversation sessions (SQLTrackerStore and MongoTrackerStore) do not save the tracker state to database immediately after starting a new conversation session. This leads to the number of events being saved in addition to the already-existing ones to be calculated correctly.
This fixes action_listen events being saved twice at the beginning of conversation sessions.

[1.7.3] - 2020-02-21

Bugfixes

Fix segmentation fault when running rasa train or rasa shell.

Improved Documentation

Fix doc links on “Deploying your Assistant” page

[1.7.2] - 2020-02-13

Bugfixes

Fixed incompatibility of Oracle with the SQLTrackerStore, by using a Sequence for the primary key columns. This does not change anything for SQL databases other than Oracle. If you are using Oracle, please create a sequence with the instructions in the SQLTrackerStore docs.

Improved Documentation

Added section on setting up the SQLTrackerStore with Oracle
Renamed “Running the Server” page to “Configuring the HTTP API”

[1.7.1] - 2020-02-11

Bugfixes

Fixed file loading of non proper UTF-8 story files, failing properly when checking for story files.
Fix problem with multi-intents. Training with multi-intents using the CountVectorsFeaturizer together with EmbeddingIntentClassifier is working again.
Fix bug ValueError: Cannot concatenate sparse features as sequence dimension does not match.
When training a Rasa model that contains responses for just some of the intents, training was failing. Fixed the featurizers to return a consistent feature vector in case no response was given for a specific message.
If no text features are present in EmbeddingIntentClassifier return the intent None.
Resolve version conflicts: Pin version of cloudpickle to ~=1.2.0.

[1.7.0] - 2020-01-29

Deprecations and Removals

The endpoint /conversations/<conversation_id>/execute is now deprecated. Instead, users should use the /conversations/<conversation_id>/trigger_intent endpoint and thus trigger intents instead of actions.
Remove option use_cls_token from tokenizers and option return_sequence from featurizers.
By default all tokenizer add a special token (__CLS__) to the end of the list of tokens. This token will be used to capture the features of the whole utterance.
The featurizers will return a matrix of size (number-of-tokens x feature-dimension) by default. This allows to train sequence models. However, the feature vector of the __CLS__ token can be used to train non-sequence models. The corresponding classifier can decide what kind of features to use.

Features

Rename templates key in domain to responses.
templates key will still work for backwards compatibility but will raise a future warning.
Added a new configuration parameter, ranking_length to the EmbeddingPolicy, EmbeddingIntentClassifier, and ResponseSelector classes.
External events and reminders now trigger intents (and entities) instead of actions.
Add new endpoint /conversations/<conversation_id>/trigger_intent, which lets the user specify an intent and a list of entities that is injected into the conversation in place of a user message. The bot then predicts and executes a response action.
Add ConveRTTokenizer.
The tokenizer should be used whenever the ConveRTFeaturizer is used.
Every tokenizer now supports the following configuration options: intent_tokenization_flag: Flag to check whether to split intents (default False). intent_split_symbol: Symbol on which intent should be split (default _)

Improvements

Remove the need of specifying utter actions in the actions section explicitly if these actions are already listed in the templates section.
Entity examples that have been extracted using an external extractor are excluded from Markdown dumping in MarkdownWriter.dumps(). The excluded external extractors are DucklingHTTPExtractor and SpacyEntityExtractor.
The EmbeddingPolicy, EmbeddingIntentClassifier, and ResponseSelector now by default normalize confidence levels over the top 10 results. See Rasa 1.6 to Rasa 1.7 for more details.
ReminderCancelled can now cancel multiple reminders if no name is given. It still cancels a single reminder if the reminder's name is specified.

Bugfixes

Requests to /model/train do not longer block other requests to the Rasa server.
Fixed default behavior of rasa test core --evaluate-model-directory when called without --model. Previously, the latest model file was used as --model. Now the default model directory is used instead.
New behavior of rasa test core --evaluate-model-directory when given an existing file as argument for --model: Previously, this led to an error. Now a warning is displayed and the directory containing the given file is used as --model.
Updated the dependency networkx from 2.3.0 to 2.4.0. The old version created incompatibilities when using pip.
There is an imcompatibility between Rasa dependecy requests 2.22.0 and the own depedency from Rasa for networkx raising errors upon pip install. There is also a bug corrected in requirements.txt which used ~= instead of ==. All of these are fixed using networkx 2.4.0.
Fixed compatibility issue with Microsoft Bot Framework Emulator if service_url lacked a trailing /.
DynamoDB tracker store decimal values will now be rounded on save. Previously values exceeding 38 digits caused an unhandled error.

Miscellaneous internal changes

Miscellaneous internal changes.

[1.6.2] - 2020-01-28

Improvements

Switching back to a TensorFlow release which only includes CPU support to reduce the size of the dependencies. If you want to use the TensorFlow package with GPU support, please run pip install tensorflow-gpu==1.15.0.

Bugfixes

Fixes Exception 'Loop' object has no attribute '_ready' error when running rasa init.
Updated the end-to-end ValueError you recieve when you have a invalid story format to point to the updated doc link.

[1.6.1] - 2020-01-07

Bugfixes

Use an empty domain in case a model is loaded which has no domain (avoids errors when accessing agent.doman.<some attribute>).
Replace error message with warning in tokenizers and featurizers if default parameter not set.
Pin sanic patch version instead of minor version. Fixes sanic _run_request_middleware() error.
Fix wrong calculation of additional conversation events when saving the conversation. This led to conversation events not being saved.
Fix wrong order of conversation events when pushing events to conversations via POST /conversations/<conversation_id>/tracker/events.

[1.6.0] - 2019-12-18

Deprecations and Removals

Removed ner_features as a feature name from CRFEntityExtractor, use text_dense_features instead.
The following settings match the previous NGramFeaturizer:
pipeline:
- name: 'CountVectorsFeaturizer'
analyzer: 'char_wb'
min_ngram: 3
max_ngram: 17
max_features: 10
min_df: 5
To use custom features in the CRFEntityExtractor use text_dense_features instead of ner_features. If text_dense_features are present in the feature set, the CRFEntityExtractor will automatically make use of them. Just make sure to add a dense featurizer in front of the CRFEntityExtractor in your pipeline and set the flag return_sequence to True for that featurizer.
Deprecated Agent.continue_training. Instead, a model should be retrained.
Specifying lookup tables directly in the NLU file is now deprecated. Please specify them in an external file.

Features

Replaced the warnings about missing templates, intents etc. in validator.py by debug messages.
Added conversation sessions to trackers.
A conversation session represents the dialog between the assistant and a user. Conversation sessions can begin in three ways: 1. the user begins the conversation with the assistant, 2. the user sends their first message after a configurable period of inactivity, or 3. a manual session start is triggered with the /session_start intent message. The period of inactivity after which a new conversation session is triggered is defined in the domain using the session_expiration_time key in the session_config section. The introduction of conversation sessions comprises the following changes:
- Added a new event SessionStarted that marks the beginning of a new conversation session.
- Added a new default action ActionSessionStart. This action takes all SlotSet events from the previous session and applies it to the next session.
- Added a new default intent session_start which triggers the start of a new conversation session.
- SQLTrackerStore and MongoTrackerStore only retrieve events from the last session from the database.
note
The session behavior is disabled for existing projects, i.e. existing domains without session config section.
Preparation for an upcoming change in the EmbeddingIntentClassifier:
Add option use_cls_token to all tokenizers. If it is set to True, the token __CLS__ will be added to the end of the list of tokens. Default is set to False. No need to change the default value for now.
Add option return_sequence to all featurizers. By default all featurizers return a matrix of size (1 x feature-dimension). If the option return_sequence is set to True, the corresponding featurizer will return a matrix of size (token-length x feature-dimension). See Text Featurizers. Default value is set to False. However, you might want to set it to True if you want to use custom features in the CRFEntityExtractor. See passing custom features to the CRFEntityExtractor
Changed some featurizers to use sparse features, which should reduce memory usage with large amounts of training data significantly. Read more: Text Featurizers .
caution
These changes break model compatibility. You will need to retrain your old models!

Improvements

Added --no-plot option for rasa test command, which disables rendering of confusion matrix and histogram. By default plots will be rendered.
If matplotlib couldn't set up a default backend, it will be set automatically to TkAgg/Agg one
Add the option \random_seed`to the`rasa data split nlu`` command to generate reproducible train/test splits.
Changed url __init__() arguments for custom tracker stores to host to reflect the __init__ arguments of currently supported tracker stores. Note that in endpoints.yml, these are still declared as url.
The kafka-python dependency has become as an “extra” dependency. To use the KafkaEventConsumer, rasa has to be installed with the [kafka] option, i.e.
$ pip install rasa[kafka]
Allow creation of natural language interpreter and generator by classname reference in endpoints.yml.
Made it explicit that interactive learning does not work with NLU-only models.
Interactive learning no longer trains NLU-only models if no model is provided and no core data is provided.
The intent_report.json created by rasa test now creates an extra field confused_with for each intent. This is a dictionary containing the names of the most common false positives when this intent should be predicted, and the number of such false positives.
rasa test nlu --cross-validation now also includes an evaluation of the response selector. As a result, the train and test F1-score, accuracy and precision is logged for the response selector. A report is also generated in the results folder by the name response_selection_report.json

Bugfixes

If a wait_time_between_pulls is configured for the model server in endpoints.yml, this will be used instead of the default one when running Rasa X.
Training Luis data with luis_schema_version higher than 4.x.x will show a warning instead of throwing an exception.
Running rasa interactive with no NLU data now works, with the functionality of rasa interactive core.
When loading models from S3, namespaces (folders within a bucket) are now respected. Previously, this would result in an error upon loading the model.
“rasa init” will ask if user wants to train a model
Pin multidict dependency to 4.6.1 to prevent sanic from breaking, see the Sanic GitHub issue for more info.
Fix errors during training and testing of ResponseSelector.

[1.5.3] - 2019-12-11

Improvements

Improved error message that appears when an incorrect parameter is passed to a policy.

Bugfixes

Added rasa/nlu/schemas/config.yml to wheel package
Pin multidict dependency to 4.6.1 to prevent sanic from breaking, see the Sanic GitHub issue

[1.5.2] - 2019-12-09

Improvements

rasa interactive will skip the story visualization of training stories in case there are more than 200 stories. Stories created during interactive learning will be visualized as before.
The log level for SocketIO loggers, including websockets.protocol, engineio.server, and socketio.server, is now handled by the LOG_LEVEL_LIBRARIES environment variable, where the default log level is ERROR.
Updated all example bots and documentation to use the updated dispatcher.utter_message() method from rasa-sdk==1.5.0.

Bugfixes

rasa interactive will not load training stories in case the visualization is skipped.
Fixed error where spacy models where not found in the docker images.
Fixed unnecessary kwargs unpacking in rasa.test.test_core call in rasa.test.test function.
Training data files now get loaded in the same order (especially relevant to subdirectories) each time to ensure training consistency when using a random seed.
Locks for tickets in LockStore are immediately issued without a redundant check for their availability.

Improved Documentation

Added towncrier to automatically collect changelog entries.
Document the pipeline for pretrained_embeddings_convert in the pre-configured pipelines section.
Proactively Reaching Out to the User Using Actions now correctly links to the endpoint specification.

[1.5.1] - 2019-11-27

Improvements

When NLU training data is dumped as Markdown file the intents are not longer ordered alphabetically, but in the original order of given training data

Bugfixes

End to end stories now support literal payloads which specify entities, e.g. greet: /greet{"name": "John"}
Slots will be correctly interpolated if there are lists in custom response templates.
Fixed compatibility issues with rasa-sdk 1.5
Updated /status endpoint to show correct path to model archive

[1.5.0] - 2019-11-26

Features

Added data validator that checks if domain object returned is empty. If so, exit early from the command rasa data validate.
Added the KeywordIntentClassifier.
Added documentation for AugmentedMemoizationPolicy.
Fall back to InMemoryTrackerStore in case there is any problem with the current tracker store.
Arbitrary metadata can now be attached to any Event subclass. The data must be stored under the metadata key when reading the event from a JSON object or dictionary.
Add command line argument rasa x --config CONFIG, to specify path to the policy and NLU pipeline configuration of your bot (default: config.yml).
Added a new NLU featurizer - ConveRTFeaturizer based on ConveRT model released by PolyAI.
Added a new preconfigured pipeline - pretrained_embeddings_convert.

Improvements

Do not retrain the entire Core model if only the templates section of the domain is changed.
Upgraded jsonschema version.

Deprecations and Removals

Remove duplicate messages when creating training data (issues/1446).

Bugfixes

MultiProjectImporter now imports files in the order of the import statements
Fixed server hanging forever on leaving rasa shell before first message
Fixed rasa init showing traceback error when user does Keyboard Interrupt before choosing a project path
CountVectorsFeaturizer featurizes intents only if its analyzer is set to word
Fixed bug where facebooks generic template was not rendered when buttons were None
Fixed default intents unnecessarily raising undefined parsing error

[1.4.6] - 2019-11-22

Bugfixes

Fixed Rasa X not working when any tracker store was configured for Rasa.
Use the matplotlib backend agg in case the tkinter package is not installed.

[1.4.5] - 2019-11-14

Bugfixes

NLU-only models no longer throw warnings about parsing features not defined in the domain
Fixed bug that stopped Dockerfiles from building version 1.4.4.
Fixed format guessing for e2e stories with intent restated as /intent

[1.4.4] - 2019-11-13

Features

PikaEventProducer adds the RabbitMQ App ID message property to published messages with the value of the RASA_ENVIRONMENT environment variable. The message property will not be assigned if this environment variable isn't set.

Improvements

Updated Mattermost connector documentation to be more clear.
Updated format strings to f-strings where appropriate.
Updated tensorflow requirement to 1.15.0
Dump domain using UTF-8 (to avoid \\UXXXX sequences in the dumped files)

Bugfixes

Fixed exporting NLU training data in json format from rasa interactive
Fixed numpy deprecation warnings

[1.4.3] - 2019-10-29

Bugfixes

Fixed Connection reset by peer errors and bot response delays when using the RabbitMQ event broker.

[1.4.2] - 2019-10-28

Deprecations and Removals

TensorFlow deprecation warnings are no longer shown when running rasa x

Bugfixes

Fixed 'Namespace' object has no attribute 'persist_nlu_data' error during interactive learning
Pinned networkx~=2.3.0 to fix visualization in rasa interactive and Rasa X
Fixed No model found error when using rasa run actions with “actions” as a directory.

[1.4.1] - 2019-10-22

Regression: changes from 1.2.12 were missing from 1.4.0, readded them

[1.4.0] - 2019-10-19

Features

add flag to CLI to persist NLU training data if needed
log a warning if the Interpreter picks up an intent or an entity that does not exist in the domain file.
added DynamoTrackerStore to support persistence of agents running on AWS
added docstrings for TrackerStore classes
added buttons and images to mattermost.
CRFEntityExtractor updated to accept arbitrary token-level features like word vectors (issues/4214)
SpacyFeaturizer updated to add ner_features for CRFEntityExtractor
Sanitizing incoming messages from slack to remove slack formatting like <mailto:xyz@rasa.com|xyz@rasa.com> or <http://url.com|url.com> and substitute it with original content
Added the ability to configure the number of Sanic worker processes in the HTTP server (rasa.server) and input channel server (rasa.core.agent.handle_channels()). The number of workers can be set using the environment variable SANIC_WORKERS (default: 1). A value of >1 is allowed only in combination with RedisLockStore as the lock store.
Botframework channel can handle uploaded files in UserMessage metadata.
Added data validator that checks there is no duplicated example data across multiples intents

Improvements

Unknown sections in markdown format (NLU data) are not ignored anymore, but instead an error is raised.
It is now easier to add metadata to a UserMessage in existing channels. You can do so by overwriting the method get_metadata. The return value of this method will be passed to the UserMessage object.
Tests can now be run in parallel
Serialise DialogueStateTracker as json instead of pickle. DEPRECATION warning: Deserialisation of pickled trackers will be deprecated in version 2.0. For now, trackers are still loaded from pickle but will be dumped as json in any subsequent save operations.
Event brokers are now also passed to custom tracker stores (using the event_broker parameter)
Don't run the Rasa Docker image as root.
Use multi-stage builds to reduce the size of the Rasa Docker image.
Updated the /status api route to use the actual model file location instead of the tmp location.

Deprecations and Removals

Removed Python 3.5 support

Bugfixes

fixed missing tkinter dependency for running tests on Ubuntu
fixed issue with conversation JSON serialization
fixed the hanging HTTP call with ner_duckling_http pipeline
fixed Interactive Learning intent payload messages saving in nlu files
fixed DucklingHTTPExtractor dimensions by actually applying to the request

[1.3.10] - 2019-10-18

Features

Can now pass a package as an argument to the --actions parameter of the rasa run actions command.

Bugfixes

Fixed visualization of stories with entities which led to a failing visualization in Rasa X

[1.3.9] - 2019-10-10

Features

Port of 1.2.10 (support for RabbitMQ TLS authentication and port key in event broker endpoint config).
Port of 1.2.11 (support for passing a CA file for SSL certificate verification via the –ssl-ca-file flag).

Bugfixes

Fixed the hanging HTTP call with ner_duckling_http pipeline.
Fixed text processing of intent attribute inside CountVectorFeaturizer.
Fixed argument of type 'NoneType' is not iterable when using rasa shell, rasa interactive / rasa run

[1.3.8] - 2019-10-08

Improvements

Policies now only get imported if they are actually used. This removes TensorFlow warnings when starting Rasa X

Bugfixes

Fixed error Object of type 'MaxHistoryTrackerFeaturizer' is not JSON serializable when running rasa train core
Default channel send_ methods no longer support kwargs as they caused issues in incompatible channels

[1.3.7] - 2019-09-27

Bugfixes

re-added TLS, SRV dependencies for PyMongo
socketio can now be run without turning on the --enable-api flag
MappingPolicy no longer fails when the latest action doesn't have a policy

[1.3.6] - 2019-09-21

Features

Added the ability for users to specify a conversation id to send a message to when using the RasaChat input channel.

[1.3.5] - 2019-09-20

Bugfixes

Fixed issue where rasa init would fail without spaCy being installed

[1.3.4] - 2019-09-20

Features

Added the ability to set the backlog parameter in Sanics run() method using the SANIC_BACKLOG environment variable. This parameter sets the number of unaccepted connections the server allows before refusing new connections. A default value of 100 is used if the variable is not set.
Status endpoint (/status) now also returns the number of training processes currently running

Bugfixes

Added the ability to properly deal with spaCy Doc-objects created on empty strings as discussed in issue #4445. Only training samples that actually bear content are sent to self.nlp.pipe for every given attribute. Non-content-bearing samples are converted to empty Doc-objects. The resulting lists are merged with their preserved order and properly returned.
asyncio warnings are now only printed if the callback takes more than 100ms (up from 1ms).
agent.load_model_from_server no longer affects logging.

Improvements

The endpoint POST /model/train no longer supports specifying an output directory for the trained model using the field out. Instead you can choose whether you want to save the trained model in the default model directory (models) (default behavior) or in a temporary directory by specifying the save_to_default_model_directory field in the training request.

[1.3.3] - 2019-09-13

Bugfixes

Added a check to avoid training CountVectorizer for a particular attribute of a message if no text is provided for that attribute across the training data.
Default one-hot representation for label featurization inside EmbeddingIntentClassifier if label features don't exist.
Policy ensemble no longer incorrectly wrings “missing mapping policy” when mapping policy is present.
“text” from utter_custom_json now correctly saved to tracker when using telegram channel

Deprecations and Removals

Removed computation of intent_spacy_doc. As a result, none of the spacy components process intents now.

[1.3.2] - 2019-09-10

Bugfixes

SQL tracker events are retrieved ordered by timestamps. This fixes interactive learning events being shown in the wrong order.

[1.3.1] - 2019-09-09

Improvements

Pin gast to == 0.2.2

[1.3.0] - 2019-09-05

Features

Added option to persist nlu training data (default: False)
option to save stories in e2e format for interactive learning
bot messages contain the timestamp of the BotUttered event, which can be used in channels
FallbackPolicy can now be configured to trigger when the difference between confidences of two predicted intents is too narrow
experimental training data importer which supports training with data of multiple sub bots. Please see the docs for more information.
throw error during training when triggers are defined in the domain without MappingPolicy being present in the policy ensemble
The tracker is now available within the interpreter's parse method, giving the ability to create interpreter classes that use the tracker state (eg. slot values) during the parsing of the message. More details on motivation of this change see issues/3015.
add example bot knowledgebasebot to showcase the usage of ActionQueryKnowledgeBase
softmax starspace loss for both EmbeddingPolicy and EmbeddingIntentClassifier
balanced batching strategy for both EmbeddingPolicy and EmbeddingIntentClassifier
max_history parameter for EmbeddingPolicy
Successful predictions of the NER are written to a file if --successes is set when running rasa test nlu
Incorrect predictions of the NER are written to a file by default. You can disable it via --no-errors.
New NLU component ResponseSelector added for the task of response selection
Message data attribute can contain two more keys - response_key, response depending on the training data
New action type implemented by ActionRetrieveResponse class and identified with response_ prefix
Vocabulary sharing inside CountVectorsFeaturizer with use_shared_vocab flag. If set to True, vocabulary of corpus is shared between text, intent and response attributes of message
Added an option to share the hidden layer weights of text input and label input inside EmbeddingIntentClassifier using the flag share_hidden_layers
New type of training data file in NLU which stores response phrases for response selection task.
Add flag intent_split_symbol and intent_tokenization_flag to all WhitespaceTokenizer, JiebaTokenizer and SpacyTokenizer
Added evaluation for response selector. Creates a report response_selection_report.json inside --out directory.
argument --config-endpoint to specify the URL from which rasa x pulls the runtime configuration (endpoints and credentials)
LockStore class storing instances of TicketLock for every conversation_id
environment variables SQL_POOL_SIZE (default: 50) and SQL_MAX_OVERFLOW (default: 100) can be set to control the pool size and maximum pool overflow for SQLTrackerStore when used with the postgresql dialect
Add a bot_challenge intent and a utter_iamabot action to all example projects and the rasa init bot.
Allow sending attachments when using the socketio channel
rasa data validate will fail with a non-zero exit code if validation fails

Improvements

added character-level CountVectorsFeaturizer with empirically found parameters into the supervised_embeddings NLU pipeline template
NLU evaluations now also stores its output in the output directory like the core evaluation
show warning in case a default path is used instead of a provided, invalid path
compare mode of rasa train core allows the whole core config comparison, naming style of models trained for comparison is changed (this is a breaking change)
pika keeps a single connection open, instead of open and closing on each incoming event
RasaChatInput fetches the public key from the Rasa X API. The key is used to decode the bearer token containing the conversation ID. This requires rasa-x>=0.20.2.
more specific exception message when loading custom components depending on whether component's path or class name is invalid or can't be found in the global namespace
change priorities so that the MemoizationPolicy has higher priority than the MappingPolicy
substitute LSTM with Transformer in EmbeddingPolicy
EmbeddingPolicy can now use MaxHistoryTrackerFeaturizer
non zero evaluate_on_num_examples in EmbeddingPolicy and EmbeddingIntentClassifier is the size of hold out validation set that is excluded from training data
defaults parameters and architectures for both EmbeddingPolicy and EmbeddingIntentClassifier are changed (this is a breaking change)
evaluation of NER does not include 'no-entity' anymore
--successes for rasa test nlu is now boolean values. If set incorrect/successful predictions are saved in a file.
--errors is renamed to --no-errors and is now a boolean value. By default incorrect predictions are saved in a file. If --no-errors is set predictions are not written to a file.
Remove label_tokenization_flag and label_split_symbol from EmbeddingIntentClassifier. Instead move these parameters to Tokenizers.
Process features of all attributes of a message, i.e. - text, intent and response inside the respective component itself. For e.g. - intent of a message is now tokenized inside the tokenizer itself.
Deprecate as_markdown and as_json in favour of nlu_as_markdown and nlu_as_json respectively.
pin python-engineio >= 3.9.3
update python-socketio req to >= 4.3.1

Bugfixes

rasa test nlu with a folder of configuration files
MappingPolicy standard featurizer is set to None
Removed text parameter from send_attachment function in slack.py to avoid duplication of text output to slackbot
server /status endpoint reports status when an NLU-only model is loaded

Deprecations and Removals

Removed --report argument from rasa test nlu. All output files are stored in the --out directory.

[1.2.12] - 2019-10-16

Features

Support for transit encryption with Redis via use_ssl: True in the tracker store config in endpoints.yml

[1.2.11] - 2019-10-09

Features

Support for passing a CA file for SSL certificate verification via the –ssl-ca-file flag

[1.2.10] - 2019-10-08

Features

Added support for RabbitMQ TLS authentication. The following environment variables need to be set: RABBITMQ_SSL_CLIENT_CERTIFICATE - path to the SSL client certificate (required) RABBITMQ_SSL_CLIENT_KEY - path to the SSL client key (required) RABBITMQ_SSL_CA_FILE - path to the SSL CA file (optional, for certificate verification) RABBITMQ_SSL_KEY_PASSWORD - SSL private key password (optional)
Added ability to define the RabbitMQ port using the port key in the event_broker endpoint config.

[1.2.9] - 2019-09-17

Bugfixes

Correctly pass SSL flag values to x CLI command (backport of

[1.2.8] - 2019-09-10

Bugfixes

SQL tracker events are retrieved ordered by timestamps. This fixes interactive learning events being shown in the wrong order. Backport of 1.3.2 patch (PR #4427).

[1.2.7] - 2019-09-02

Bugfixes

Added query dictionary argument to SQLTrackerStore which will be appended to the SQL connection URL as query parameters.

[1.2.6] - 2019-09-02

Bugfixes

fixed bug that occurred when sending template elements through a channel that doesn't support them

[1.2.5] - 2019-08-26

Features

SSL support for rasa run command. Certificate can be specified using --ssl-certificate and --ssl-keyfile.

Bugfixes

made default augmentation value consistent across repo
'/restart' will now also restart the bot if the tracker is paused

[1.2.4] - 2019-08-23

Bugfixes

the SocketIO input channel now allows accesses from other origins (fixes SocketIO channel on Rasa X)

[1.2.3] - 2019-08-15

Improvements

messages with multiple entities are now handled properly with e2e evaluation
data/test_evaluations/end_to_end_story.md was re-written in the restaurantbot domain

[1.2.3] - 2019-08-15

Improvements

messages with multiple entities are now handled properly with e2e evaluation
data/test_evaluations/end_to_end_story.md was re-written in the restaurantbot domain

Bugfixes

Free text input was not allowed in the Rasa shell when the response template contained buttons, which has now been fixed.

[1.2.2] - 2019-08-07

Bugfixes

UserUttered events always got the same timestamp

[1.2.1] - 2019-08-06

Features

Docs now have an EDIT THIS PAGE button

Bugfixes

Flood control exceeded error in Telegram connector which happened because the webhook was set twice

[1.2.0] - 2019-08-01

Features

add root route to server started without --enable-api parameter
add --evaluate-model-directory to rasa test core to evaluate models from rasa train core -c <config-1> <config-2>
option to send messages to the user by calling POST /conversations/{conversation_id}/execute

Improvements

Agent.update_model() and Agent.handle_message() now work without needing to set a domain or a policy ensemble
Update pytype to 2019.7.11
new event broker class: SQLProducer. This event broker is now used when running locally with Rasa X
API requests are not longer logged to rasa_core.log by default in order to avoid problems when running on OpenShift (use --log-file rasa_core.log to retain the old behavior)
metadata attribute added to UserMessage

Bugfixes

rasa test core can handle compressed model files
rasa can handle story files containing multi line comments
template will retain { if escaped with {. e.g. {{“foo”: {bar}}} will result in {“foo”: “replaced value”}

[1.1.8] - 2019-07-25

Features

TrainingFileImporter interface to support customizing the process of loading training data
fill slots for custom templates

Improvements

Agent.update_model() and Agent.handle_message() now work without needing to set a domain or a policy ensemble
update pytype to 2019.7.11

Bugfixes

interactive learning bug where reverted user utterances were dumped to training data
added timeout to terminal input channel to avoid freezing input in case of server errors
fill slots for image, buttons, quick_replies and attachments in templates
rasa train core in comparison mode stores the model files compressed (tar.gz files)
slot setting in interactive learning with the TwoStageFallbackPolicy

[1.1.7] - 2019-07-18

Features

added optional pymongo dependencies [tls, srv] to requirements.txt for better mongodb support
case_sensitive option added to WhiteSpaceTokenizer with true as default.

Bugfixes

validation no longer throws an error during interactive learning
fixed wrong cleaning of use_entities in case it was a list and not True
updated the server endpoint /model/parse to handle also messages with the intent prefix
fixed bug where “No model found” message appeared after successfully running the bot
debug logs now print to rasa_core.log when running rasa x -vv or rasa run -vv

[1.1.6] - 2019-07-12

Features

rest channel supports setting a message's input_channel through a field input_channel in the request body

Improvements

recommended syntax for empty use_entities and ignore_entities in the domain file has been updated from False or None to an empty list ([])

Bugfixes

rasa run without --enable-api does not require a local model anymore
using rasa run with --enable-api to run a server now prints “running Rasa server” instead of “running Rasa Core server”
actions, intents, and utterances created in rasa interactive can no longer be empty

[1.1.5] - 2019-07-10

Features

debug logging now tells you which tracker store is connected
the response of /model/train now includes a response header for the trained model filename
Validator class to help developing by checking if the files have any errors
project's code is now linted using flake8
info log when credentials were provided for multiple channels and channel in --connector argument was specified at the same time
validate export paths in interactive learning

Improvements

deprecate rasa.core.agent.handle_channels(...)\. Please use ``rasa.run(...)orrasa.core.run.configure_app` instead.
Agent.load() also accepts tar.gz model file

Deprecations and Removals

revert the stripping of trailing slashes in endpoint URLs since this can lead to problems in case the trailing slash is actually wanted
starter packs were removed from Github and are therefore no longer tested by Travis script

Bugfixes

all temporal model files are now deleted after stopping the Rasa server
rasa shell nlu now outputs unicode characters instead of \\uxxxx codes
fixed PUT /model with model_server by deserializing the model_server to EndpointConfig.
x in AnySlotDict is now True for any x, which fixes empty slot warnings in interactive learning
rasa train now also includes NLU files in other formats than the Rasa format
rasa train core no longer crashes without a --domain arg
rasa interactive now looks for endpoints in endpoints.yml if no --endpoints arg is passed
custom files, e.g. custom components and channels, load correctly when using the command line interface
MappingPolicy now works correctly when used as part of a PolicyEnsemble

[1.1.4] - 2019-06-18

Features

unfeaturize single entities
added agent readiness check to the /status resource

Improvements

removed leading underscore from name of '_create_initial_project' function.

Bugfixes

fixed bug where facebook quick replies were not rendering
take FB quick reply payload rather than text as input
fixed bug where training_data path in metadata.json was an absolute path

[1.1.3] - 2019-06-14

Bugfixes

fixed any inconsistent type annotations in code and some bugs revealed by type checker

[1.1.2] - 2019-06-13

Bugfixes

fixed duplicate events appearing in tracker when using a PostgreSQL tracker store

[1.1.1] - 2019-06-13

Bugfixes

fixed compatibility with Rasa SDK
bot responses can contain custom messages besides other message types

[1.1.0] - 2019-06-13

Features

nlu configs can now be directly compared for performance on a dataset in rasa test nlu

Improvements

update the tracker in interactive learning through reverting and appending events instead of replacing the tracker
POST /conversations/{conversation_id}/tracker/events supports a list of events

Bugfixes

fixed creation of RasaNLUHttpInterpreter
form actions are included in domain warnings
default actions, which are overriden by custom actions and are listed in the domain are excluded from domain warnings
SQL data column type to Text for compatibility with MySQL
non-featurizer training parameters don't break SklearnPolicy anymore

[1.0.9] - 2019-06-10

Improvements

revert PR #3739 (as this is a breaking change): set PikaProducer and KafkaProducer default queues back to rasa_core_events

[1.0.8] - 2019-06-10

Features

support for specifying full database urls in the SQLTrackerStore configuration
maximum number of predictions can be set via the environment variable MAX_NUMBER_OF_PREDICTIONS (default is 10)

Improvements

default PikaProducer and KafkaProducer queues to rasa_production_events
exclude unfeaturized slots from domain warnings

Bugfixes

loading of additional training data with the SkillSelector
strip trailing slashes in endpoint URLs

[1.0.7] - 2019-06-06

Features

added argument --rasa-x-port to specify the port of Rasa X when running Rasa X locally via rasa x

Bugfixes

slack notifications from bots correctly render text
fixed usage of --log-file argument for rasa run and rasa shell
check if correct tracker store is configured in local mode

[1.0.6] - 2019-06-03

Bugfixes

fixed backwards incompatible utils changes

[1.0.5] - 2019-06-03

Bugfixes

fixed spacy being a required dependency (regression)

[1.0.4] - 2019-06-03

Features

automatic creation of index on the sender_id column when using an SQL tracker store. If you have an existing data and you are running into performance issues, please make sure to add an index manually using CREATE INDEX event_idx_sender_id ON events (sender_id);.

Improvements

NLU evaluation in cross-validation mode now also provides intent/entity reports, confusion matrix, etc.

[1.0.3] - 2019-05-30

Bugfixes

non-ascii characters render correctly in stories generated from interactive learning
validate domain file before usage, e.g. print proper error messages if domain file is invalid instead of raising errors

[1.0.2] - 2019-05-29

Features

added domain_warnings() method to Domain which returns a dict containing the diff between supplied {actions, intents, entities, slots} and what's contained in the domain

Bugfixes

fix lookup table files failed to load issues/3622
buttons can now be properly selected during cmdline chat or when in interactive learning
set slots correctly when events are added through the API
mapping policy no longer ignores NLU threshold
mapping policy priority is correctly persisted

[1.0.1] - 2019-05-21

Bugfixes

updated installation command in docs for Rasa X

[1.0.0] - 2019-05-21

Features

added arguments to set the file paths for interactive training
added quick reply representation for command-line output
added option to specify custom button type for Facebook buttons
added tracker store persisting trackers into a SQL database (SQLTrackerStore)
added rasa command line interface and API
Rasa HTTP training endpoint at POST /jobs. This endpoint will train a combined Rasa Core and NLU model
ReminderCancelled(action_name) event to cancel given action_name reminder for current user
Rasa HTTP intent evaluation endpoint at POST /intentEvaluation. This endpoints performs an intent evaluation of a Rasa model
option to create template for new utterance action in interactive learning
you can now choose actions previously created in the same session in interactive learning
add formatter 'black'
channel-specific utterances via the - "channel": key in utterance templates
arbitrary json messages via the - "custom": key in utterance templates and via utter_custom_json() method in custom actions
support to load sub skills (domain, stories, nlu data)
support to select which sub skills to load through import section in config.yml
support for spaCy 2.1
a model for an agent can now also be loaded from a remote storage
log level can be set via environment variable LOG_LEVEL
add --store-uncompressed to train command to not compress Rasa model
log level of libraries, such as tensorflow, can be set via environment variable LOG_LEVEL_LIBRARIES
if no spaCy model is linked upon building a spaCy pipeline, an appropriate error message is now raised with instructions for linking one

Improvements

renamed all CLI parameters containing any _ to use dashes - instead (GNU standard)
renamed rasa_core package to rasa.core
for interactive learning only include manually annotated and ner_crf entities in nlu export
made message_id an additional argument to interpreter.parse
changed removing punctuation logic in WhitespaceTokenizer
training_processes in the Rasa NLU data router have been renamed to worker_processes
created a common utils package rasa.utils for nlu and core, common methods like read_yaml moved there
removed --num_threads from run command (server will be asynchronous but running in a single thread)
the _check_token() method in RasaChat now authenticates against /auth/verify instead of /user
removed --pre_load from run command (Rasa NLU server will just have a maximum of one model and that model will be loaded by default)
changed file format of a stored trained model from the Rasa NLU server to tar.gz
train command uses fallback config if an invalid config is given
test command now compares multiple models if a list of model files is provided for the argument --model
Merged rasa.core and rasa.nlu server into a single server. See swagger file in docs/_static/spec/server.yaml for available endpoints.
utter_custom_message() method in rasa_core_sdk has been renamed to utter_elements()
updated dependencies. as part of this, models for spacy need to be reinstalled for 2.1 (from 2.0)
make sure all command line arguments for rasa test and rasa interactive are actually used, removed arguments that were not used at all (e.g. --core for rasa test)

Deprecations and Removals

removed possibility to execute python -m rasa_core.train etc. (e.g. scripts in rasa.core and rasa.nlu). Use the CLI for rasa instead, e.g. rasa train core.
removed _sklearn_numpy_warning_fix from the SklearnIntentClassifier
removed Dispatcher class from core
removed projects: the Rasa NLU server now has a maximum of one model at a time loaded.

Bugfixes

evaluating core stories with two stage fallback gave an error, trying to handle None for a policy
the /evaluate route for the Rasa NLU server now runs evaluation in a parallel process, which prevents the currently loaded model unloading
added missing implementation of the keys() function for the Redis Tracker Store
in interactive learning: only updates entity values if user changes annotation
log options from the command line interface are applied (they overwrite the environment variable)
all message arguments (kwargs in dispatcher.utter methods, as well as template args) are now sent through to output channels
utterance templates defined in actions are checked for existence upon training a new agent, and a warning is thrown before training if one is missing

[3.11.1] - 2024-12-13#

Bugfixes#

[3.11.0] - 2024-12-11#

Deprecations and Removals#

Features#

Improvements#

Bugfixes#

Miscellaneous internal changes#

[3.10.14] - 2024-12-04#

Bugfixes#

[3.10.13] - 2024-11-29#

Bugfixes#

[3.10.12] - 2024-11-25#

Bugfixes#

[3.10.11] - 2024-11-20#

Bugfixes#

Miscellaneous internal changes#

[3.10.10] - 2024-11-14#

Bugfixes#

[3.10.9] - 2024-11-13#

Bugfixes#

[3.10.8] - 2024-10-24#

Bugfixes#

Miscellaneous internal changes#

[3.10.7] - 2024-10-17#

Improvements#

Bugfixes#

[3.10.6] - 2024-10-04#

Bugfixes#

Miscellaneous internal changes#

[3.10.5] - 2024-10-01#

Bugfixes#

[3.10.4] - 2024-09-25#

Bugfixes#

[3.10.3] - 2024-09-20#

[3.10.2] - 2024-09-19#

Deprecations and Removals#

Improvements#

Bugfixes#

Miscellaneous internal changes#

[3.10.1] - 2024-09-11#

Bugfixes#

[3.10.0] - 2024-09-04#

Deprecations and Removals#

Features#

Improvements#

Bugfixes#

Miscellaneous internal changes#

[3.9.17] - 2024-12-05#

Bugfixes#

[3.9.16] - 2024-11-26#

Bugfixes#

[3.9.15] - 2024-10-18#

Improvements#

Bugfixes#

Miscellaneous internal changes#

[3.9.14] - 2024-10-02#

[3.9.13] - 2024-10-01#

Bugfixes#

[3.9.12] - 2024-09-20#

Deprecations and Removals#

Improvements#

Bugfixes#

Miscellaneous internal changes#

[3.9.11] - 2024-09-13#

Bugfixes#

[3.9.10] - 2024-09-12#

Bugfixes#

[3.9.9] - 2024-08-23#

Bugfixes#

[3.9.8] - 2024-08-21#

Bugfixes#

[3.9.7] - 2024-08-15#

Bugfixes#

Miscellaneous internal changes#

[3.9.6] - 2024-08-07#

Miscellaneous internal changes#

[3.9.5] - 2024-08-01#

Improvements#

Bugfixes#

[3.11.1] - 2024-12-13

Bugfixes

[3.11.0] - 2024-12-11

Deprecations and Removals

Features

Improvements

Bugfixes

Miscellaneous internal changes

[3.10.14] - 2024-12-04

Bugfixes

[3.10.13] - 2024-11-29

Bugfixes

[3.10.12] - 2024-11-25

Bugfixes

[3.10.11] - 2024-11-20

Bugfixes

Miscellaneous internal changes

[3.10.10] - 2024-11-14

Bugfixes

[3.10.9] - 2024-11-13

Bugfixes

[3.10.8] - 2024-10-24

Bugfixes

Miscellaneous internal changes

[3.10.7] - 2024-10-17

Improvements

Bugfixes

[3.10.6] - 2024-10-04

Bugfixes

Miscellaneous internal changes

[3.10.5] - 2024-10-01

Bugfixes

[3.10.4] - 2024-09-25

Bugfixes

[3.10.3] - 2024-09-20

[3.10.2] - 2024-09-19

Deprecations and Removals

Improvements

Bugfixes

Miscellaneous internal changes

[3.10.1] - 2024-09-11

Bugfixes

[3.10.0] - 2024-09-04

Deprecations and Removals

Features

Improvements

Bugfixes

Miscellaneous internal changes

[3.9.17] - 2024-12-05

Bugfixes

[3.9.16] - 2024-11-26

Bugfixes

[3.9.15] - 2024-10-18

Improvements

Bugfixes

Miscellaneous internal changes

[3.9.14] - 2024-10-02

[3.9.13] - 2024-10-01

Bugfixes

[3.9.12] - 2024-09-20

Deprecations and Removals

Improvements

Bugfixes

Miscellaneous internal changes

[3.9.11] - 2024-09-13

Bugfixes

[3.9.10] - 2024-09-12

Bugfixes

[3.9.9] - 2024-08-23

Bugfixes

[3.9.8] - 2024-08-21

Bugfixes

[3.9.7] - 2024-08-15

Bugfixes

Miscellaneous internal changes

[3.9.6] - 2024-08-07

Miscellaneous internal changes

[3.9.5] - 2024-08-01

Improvements

Bugfixes