Utility Component for interfacing between Transformers library and Rasa OS.
The transformers(https://github.com/huggingface/transformers) library is used to load pre-trained language models like BERT, GPT-2, etc. The component also tokenizes and featurizes dense featurizable attributes of each message.
Compute tokens and dense features for each message in training data.
training_data- NLU training data to be tokenized and featurized
config- NLU pipeline config consisting of all components.
Process an incoming message by computing its tokens and dense features.
message- Incoming message object