This is unreleased documentation for Rasa Open Source Documentation Master/Unreleased version.
For the latest released documentation, see the latest version (2.6.x).

Version: Master/Unreleased


ConveRTTokenizer Objects

class ConveRTTokenizer(WhitespaceTokenizer)

This tokenizer is deprecated and will be removed in the future.

The ConveRTFeaturizer component now sets the sub-token information for dense featurizable attributes of each message object.


| __init__(component_config: Dict[Text, Any] = None) -> None

Initializes ConveRTTokenizer with the ConveRT model.


  • component_config - Configuration for the component.