Telemetry Event Reference

Telemetry events are only reported if telemetry is enabled. A detailed explanation on the reasoning behind collecting optional telemetry events can be found in our telemetry documentation.

Model Training#

Training Started#

backend A training of a Rasa machine learning model got started. The event provides information on aggregated training data statistics. Event properties:

• language (string): Language model is trained with, e.g. 'en'.
• training_id (string): Generated unique identifier for this training.
• type (string): Type of model trained, either 'nlu', 'core' or 'rasa'.
• pipeline (undefined): List of the pipeline configurations used for training.
• policies (undefined): List of the policy configurations used for training.
• train_schema (undefined): Training graph schema for graph recipe
• predict_schema (undefined): Predict graph schema for graph recipe
• num_intent_examples (integer): Number of NLU examples.
• num_entity_examples (integer): Number of entity examples.
• num_actions (integer): Number of actions defined in the domain.
• num_templates (integer): Number of templates or responses defined in the domain.
• num_conditional_response_variations (integer): Number of conditional response variations defined in the domain.
• num_slot_mappings (integer): Number of total slot mappings defined in the domain.
• num_custom_slot_mappings (integer): Number of custom slot mappings defined in the domain.
• num_conditional_slot_mappings (integer): Number of slot mappings with conditions attached defined in the domain.
• num_slots (integer): Number of slots defined in the domain.
• num_forms (integer): Number of forms defined in the domain.
• num_intents (integer): Number of intents defined in the domain.
• num_entities (integer): Number of entities defined in the domain.
• num_story_steps (integer): Number of story steps available.
• num_lookup_tables (integer): Number of different lookup tables.
• num_synonyms (integer): Total number of entity synonyms defined.
• num_regexes (integer): Total number of regexes defined.
• is_finetuning (boolean): True if a model is trained by finetuning an existing model.
• recipe (string): Recipe used in training the model, either 'default.v1' or 'graph.v1'.

Training Completed#

backend The training of a Rasa machine learning model finished. The event provides information about the resulting model. Event properties:

• training_id (string): Generated unique identifier for this training. Can be used to join with 'Training Started'.
• type (string): Type of model trained, either 'nlu', 'core' or 'rasa'.
• runtime (integer): The time in seconds it took to train the model.

Model Testing#

Model Core Tested#

backend Triggered when a Core model is getting tested. Event properties:

• project (string,null): Fingerprint of the project the tested model got trained in.
• num_story_steps (integer): Number of story steps used for testing
• end_to_end (boolean): Indicates if tests are running in end-to-end mode, testing message handling and dialogue handling at the same time

Model NLU Tested#

backend Triggered when an NLU model is getting tested. Event properties:

• num_intent_examples (integer): Number of NLU examples.
• num_entity_examples (integer): Number of entity examples.
• num_lookup_tables (integer): Number of different lookup tables.
• num_synonyms (integer): Total number of entity synonyms defined.
• num_regexes (integer): Total number of regexes defined.

End-to-End Testing#

End-to-End Testing Started#

backend Triggered when end-to-end testing has been started. Event properties:

• number_of_test_cases (integer): Number of test cases to be run.
• number_of_fixtures (integer): Number of fixtures defined globally.
• uses_fixtures (boolean): Indicates if any fixtures have been defined globally.

Model Serving#

Interactive Learning Started#

backend Triggered when an interactive learning session got started. Event properties:

• skip_visualization (boolean): Whether the visualization of stories should be shown during the interactive learning session
• save_in_e2e (boolean): Whether the data should be stored in end-to-end format

Server Started#

backend Triggered when a Rasa Open Source server gets started. Event properties:

• input_channels (array): Names of the used input channels
• api_enabled (boolean): Indicator if the API is enabled or if only the input channel is running
• number_of_workers (integer): Amount of Sanic workers started as part of the server
• endpoints_nlg (string,null): Type of the used NLG endpoint
• endpoints_nlu (string,null): Type of the used NLU endpoint
• endpoints_action_server (string,null): Type of the used action server
• endpoints_model_server (string,null): Type of the used model server
• endpoints_tracker_store (string,null): Type of the used tracker store
• endpoints_lock_store (string,null): Type of the used lock store
• endpoints_event_broker (string,null): Type of the used event broker
• project (string,null): Hash of the deployed model the server is started with

Shell Started#

backend Triggered when a shell session is started to talk to a trained bot. Event properties:

• type (string): Type of the model, either 'nlu', 'core' or 'rasa'.

Rasa X Local Started#

backend Triggered when a Rasa X is started in local mode.

Markers Extraction#

Markers Extraction Initiated#

backend Triggered when marker extraction has been initiated. Event properties:

• strategy (string): Strategy to use when selecting trackers to extract from.
• only_extract (boolean): Indicates if path to write out statistics hasn't been specified.
• seed (boolean): The seed to initialise the random number generator for use with the 'sample' strategy.
• count (integer,null): Number of trackers to extract from (for any strategy except 'all').

Markers Extracted#

backend Triggered when markers have been extracted. Event properties:

• trackers_count (integer): Number of processed trackers.

Markers Parsed#

backend Triggered when markers have been successfully parsed. Event properties:

• marker_count (integer): Number of parsed markers.
• max_depth (integer): Maximum depth of the parsed markers.
• branching_factor (integer): Maximum number of children of any of the parsed markers.

Markers Statistics Computed#

backend Triggered when marker statistics have been computed. Event properties:

• trackers_count (integer): Number of processed trackers.

Data Handling#

Training Data Split#

backend Triggered when training data gets split. Event properties:

• fraction (number): Percentage of the data which goes into training data (the rest goes into the test set).
• type (string): Type of data, either 'nlu', 'core' or 'rasa'.

Training Data Validated#

backend Triggered when training data gets validated. Event properties:

• validation_success (boolean): whether the validation was successful

Training Data Converted#

backend Triggered when training data gets converted. Event properties:

• output_format (string): target format of the converter
• type (string): Type of data, either 'nlu', 'core', 'config' or 'nlg'.

Tracker Exported#

backend Triggered when conversations get exported from a tracker store through an event broker. Event properties:

• event_broker (string): Name of the used event broker
• tracker_store (string): Name of the used tracker store
• number_of_exported_events (integer): Number of events exported through the event broker

Story Visualization Started#

backend Triggered when stories are getting visualized.

Miscellaneous#

Telemetry Disabled#

backend Triggered when telemetry reporting gets disabled. Last event sent before disabling telemetry. This event is not sent, if the user never enabled telemetry reporting before deactivating it.

Project Created#

backend Triggered when a project is created using rasa init. Event properties:

• init_directory (string): Hash of the directory path the project is created in