Introduction

In this blog we plan to cover:

1. using conditional response variations to reduce the number of training stories,
2. understanding the variation selection criteria for complex responses with conditional response variations, channels, and default responses, and
3. guidance for responsible and ethical use of conditional response variations.

Turning multiple stories into one with conditional response variations

Rasa developers often need to implement a single story with slight variations on responses depending on relevant details, for example, user account data or external world state (like the availability of human customer service agents). To achieve this, developers would often represent this information as one or more slots, and then write variations of the same story structure and condition the final response or responses on the values of the slots. This means the developer has to write many structurally identical training stories. Conditional response variations eliminate the need to do this.

As a concrete example, let's compare two implementations. First, we'll demonstrate how multiple stories would be used to create variations, and then, we'll show how conditional response variations can be used to consolidate these stories. In this example, our user type is represented as a combination of a boolean slot, is_new_user, and a categorical slot, account_type, which can be either "primary" or "secondary". This means we have four user types in total:

For each user type, we would like to give slight variations on responses. Perhaps a new user gets more explanatory responses and help while a return user gets more concise responses; primary account holders are shown more options than secondary account holders.

To handle a scenario where a user greets the bot and asks a question whose answer depends on user type, we would have to provided four different training stories:

- story: New Primary User
  steps:
  - intent: LOGIN
  - slot_was_set:
    - is_new_user: true
    - account_type: "primary"
  - action: utter_greet_new_user
  - action: utter_options_new_user_primary
  - intent: how_change_plan
  - action: utter_change_plan_primary

- story: Return Primary User
  steps:
  - intent: LOGIN
  - slot_was_set:
    - is_new_user: false
    - account_type: "primary"
  - action: utter_greet_return_user
  - action: utter_options_return_user
  - intent: how_change_plan
  - action: utter_change_plan_primary

- story: New Secondary User
  steps:
  - intent: LOGIN
  - slot_was_set:
    - is_new_user: true
    - account_type: "secondary"
  - action: utter_greet_new_user
  - action: utter_options_new_user_secondary
  - intent: how_change_plan
  - action: utter_change_plan_secondary

- story: Return Secondary User
  steps:
  - intent: LOGIN
  - slot_was_set:
    - is_new_user: false
    - account_type: "secondary"
  - action: utter_greet_return_user
  - action: utter_options_return_user
  - intent: how_change_plan
  - action: utter_change_plan_secondary

Each story relies on user type-specific responses (e.g. utter_change_plan_primary vs utter_change_plan_secondary) which would be defined in the domain.yml responses field:

responses:
  utter_greet_new_user:
    - text: "Thank you for signing up! What do you need help with?"

  utter_greet_return_user:
    - text: "Welcome back! What do you need help with?"

  utter_options_new_user_primary:
    - text: "Would you like to file a claim, add secondary accounts, or something else?"

  utter_options_new_user_secondary:
    - text: "Would you like to file a claim, explain benefits, or something else?"

  utter_options_return_user:
    - text: "Would you like to check on the status of a claim, file a  new claim, or something else?"

  utter_change_plan_primary:
    - text: "Click [here](url/to/benefits) to make any changes to your current plan."

  utter_change_plan_secondary:
    - text: "Only the primary account holder can make changes to the current plan."

Alternatively, with conditional response variations you would only need to write this story once:

- story: login and explain benefits
  steps:
  - intent: LOGIN
  - action: utter_greet
  - action: utter_options
  - intent: how_change_plan
  - action: utter_change_plan

The logic of the response variation selection would then be defined directly in the domain.yml responses section:

responses:
  utter_greet:
    - condition:
        - type: slot
          name: is_new_user
          value: true
      text: "Thank you for signing up! What do you need help with?"
    - condition:
        - type: slot
          name: is_new_user
          value: false
      text: "Welcome back! What do you need help with?"

  utter_options:
    - condition:  # A conditional response variation with multiple slot-value constraints.
        - type: slot
          name: is_new_user
          value: true
        - type: slot
          name: account_type
          value: "primary"
      text: "Would you like to file a claim, add secondary accounts, or something else?"
    - condition:
        - type: slot
          name: is_new_user
          value: true
        - type: slot
          name: account_type
          value: "secondary"
      text: "Would you like to file a claim, explain benefits, or something else?"
    - condition:
        - type: slot
          name: is_new_user
          value: false
      text: "Would you like to check on the status of a claim, file a  new claim, or something else?"

  utter_change_plan:
    - condition:
        - type: slot
          name: account_type
          value: "primary"
      text: "Click [here](url/to/benefits) to make any changes to your current plan."
    - condition:
        - type: slot
          name: account_type
          value: "secondary"
      text: "Only the primary account holder can make changes to the current plan."

At run time, when an action with conditional response variations is executed, the dialogue manager checks that each slot-value constraint in the condition field is equal to that of the corresponding slot-value in the current dialogue state (the implementation uses the python __eq__ comparator between slot values). For a conditional response variation with multiple slot-value constraints defined in the condition field to be eligible, all its slot-value constraints must be satisfied (effectively an AND-ing of each individual constraint check). See the first two conditional response variations for the utter_options action above for examples with more than one slot-value constraint.

In summary, using conditional response variations we can represent multiple stories with only one training story. This also makes it easier on the dialogue policy since the complexity of the story is offloaded to response selection.

Why not use custom actions?

Experienced Rasa developers might also achieve something similar using custom actions. In this case you could still use one story to represent many:

- story: login and explain benefits
  steps:
  - intent: LOGIN
  - action: action_greet
  - action: action_options
  - intent: how_change_plan
  - action: action_change_plan

but you would put the variation selection logic into the custom actions like so:

class Options(Action):

  def name(self) -> Text:
    return "action_options"

  def run(
    self,
    dispatcher: CollectingDispatcher,
    tracker: Tracker,
    domain: DomainDict
  ) -> List:

    if tracker.slots.get("is_new_user", False):
      if tracker.slots.get("account_type", None) == "primary":
        dispatcher.utter_message(
          response="utter_options_new_user_primary"
        )
      elif tracker.slots.get("account_type", None) == "secondary":
        dispatcher.utter_message(
          response="utter_options_new_user_secondary"
        )
    else:
      dispatcher.utter_message(
        response="utter_options_return_user"
      )

    return []

The disadvantage to this approach, however, is that non-technical teammates won't be able to change the text of the response variation. Conditional response variations keep the eligibility logic and text of all response variations centralized in the domain.yml, where non-technical teammates can easily read and edit them. In the future, we hope to make this feature accessible in Rasa X to the domain experts and copywriters on the team without needing to look at a YAML file.

When is a conditional response variation eligible? Conditional Response Variations, Defaults, and Channels

In the last section, we outlined the benefits that come with conditional response variations: less repetition in stories and greater access to editing responses for non-technical users. How does Rasa achieve this?

On the backend, conditional response variations are part of response generation, which is handled after NLU and action prediction. Conditional response variations can be specified alongside channel-specific variations, where channel is the platform through which the user connects to the assistant, e.g. Slack. Let's take a look under the hood to understand how the assistant identifies the relevant response. We'll be using this diagram as a guide:

The starting point is all response variations available for the predicted action. Then Rasa divides these variations into four non-overlapping pools of eligible responses:

I. Conditional response variations whose conditions are satisfied by the current dialogue state and whose channel field matches the user's channel.

II. Default responses whose channel field matches the user's channel.

III. Conditional response variations whose conditions are satisfied by the current dialogue state and whose channel field is not set.

IV. Default responses whose channel field is not set.

These pools are searched in order (I, II, III, IV), stopping at the first non-empty pool. Rasa then picks at random one of the responses in this pool.

Let's consider the following example:

slots:
  is_new_user:
    type: bool
    influence_conversation: false
  time_of_day:
    type: categorical
    values:
        - morning
        - afternoon
        - evening
    influence_conversation: false

responses:
  utter_welcome:
    # Pool I
    - text: "Good morning and welcome on Slack! This is your first time here so I'll help you get set-up."
      condition:
        - type: slot
          name: time_of_day
          value: morning
        - type: slot
          name: is_new_user
          value: true
      channel: "slack"
    - text: "Good morning and welcome on Slack!"
      condition:
        - type: slot
          name: time_of_day
          value: morning
      channel: "slack"
    # Pool II
    - text: "Welcome on Slack!"
      channel: "slack"
    # Pool III
    - text: "Welcome! We hope you have a fine morning."
      condition:
        - type: slot
          name: time_of_day
          value: morning
    # Pool IV
    - text: "Welcome!"

If the user is on Slack and utter_welcome is executed, Rasa will check the current state of the slot values against the conditional response variations in pool I. If time_of_day is set to "morning" and is_new_user is False, "Good morning and welcome on Slack!" will be chosen. If time_of_day is equal to "morning" and is_new_user is True, both conditional response variations in pool I would be eligible and Rasa would pick one at random.

If the user is on Slack but time_of_day is not equal to "morning", Rasa will choose from pool II, defaults with channel match, e.g. "Welcome on Slack".

If the user is not on Slack, but the time_of_day slot is set as "morning", pools I & II will be empty and therefore Rasa will choose from pool III: conditional response variations without a channel field, in this case, "Welcome! We hope you have a fine morning."

Finally, for users not connected via Slack with time_of_day either unset or different from "morning", the first three pools will be empty, in which case Rasa will select from pool IV, e.g. defaults without channel field, which contains the most generic response, "Welcome!"

Responsible Usage

The power of conditional response variations is that it makes it easier to personalize or contextualize bot responses.

However, there are some pitfalls every chatbot developer should be aware of. Let's walk through an example of a customer service bot for a clothing retailer. The design team wants the bot responses to display more empathy when the user is frustrated. They sketch out the behavior with some example dialogues.

Neutral or Positive Customer → Prompt but Neutral Response.
User: I'd like to return the shirt I ordered.
Bot: Sure! I can help you with that. What was the order number?

Upset Customer → Apologetic Response
User: I WANT A REFUND. THE SHIRT ARRIVED WITH A LARGE HOLE IN IT!
Bot: Of course! I'm very sorry to hear that. What was the order number?

The developer team decides to implement this using a text classifier to predict the sentiment of the user's last message inside a custom action which fills a user_sentiment slot. Conditional response variations then deliver increasingly apologetic responses based on the value of the slot (positive, neutral, or negative sentiment).

Ideally, this would work as outlined above by the designers. Unfortunately, sentiment classifiers are imperfect and you risk that your bot will choose incorrect responses.

User: take your shirt and give me back my $$$
Sentiment Classifier: positive (incorrect)
Bot: Sure! I can help you with that. What was the order number?
(The user should probably receive the apologetic response.)

Research has shown that text classifiers will typically have a higher error rate for speakers from minority communities. For example, while sentiment analysis might work in this case:

Bot: And what's the reason for returning the item?
User: the shirt was excellent but it didn't fit
Sentiment Prediction: positive (correct)
Bot: Thank you. Sending an email confirmation of the return order now.
(User gets the neutral response the design team envisioned)

If we replace "excellent" with "lit," a term from the African American English dialect which might convey a similar meaning to "excellent" in this context, it can change the sentiment prediction completely if the creators of the sentiment classifier did not properly account for the African American English dialect.

Bot: And what's the reason for returning the item?
User: the shirt was lit but it didn't fit
Sentiment Prediction: negative (incorrect)
Bot: I'm very sorry to hear that! Sending an email confirmation of the return order now. (The user receives an overtly apologetic response which was not the design team's intent.)

In general you should only condition responses on information relevant to your specific task like account status or offer updates, or information the user has told you directly. We do not recommend making predictions about aspects of a user's identity directly from the text.

As an alternative that still accomplishes the designers goals, you could wait to get more information directly from the user and then use conditional response variations to deliver the appropriate response with more confidence. In the current scenario, this might mean waiting until knowing the reason for the return.

Return is not the fault of the retailer → Prompt but Neutral Response.
User: I'd like to return the shirt I ordered.
Bot: I can help you with that. What was the order number?
User: LV-426
Bot: And can you confirm the reason for returning?

Didn't Fit Wrong Item Damaged Item

User: Didn't Fit
Bot: Thank you. Sending an email confirmation of the return order now.

Return is the fault of the retailer → Apologetic Response
User: I WANT A REFUND. THE SHIRT ARRIVED WITH A LARGE HOLE IN IT!
Bot: I can help you with that. What was the order number?
User: LV-426
Bot: And can you confirm the reason for returning?

Didn't Fit Wrong Item Damaged Item

User: Damaged Item
Bot: I'm very sorry to hear that! Sending an email confirmation of the return order now.

The above scenario could be implemented with the following story and responses:

stories:
- story: Ask for return
  steps:
  - intent: return_item
  - action: utter_what_order_number
  - intent: order_number
  - action: utter_ask_return_reason
  - intent: inform
  - action: utter_return_confirmation

responses:
  utter_return_confirmation:
    # Apologetic when return is because of retailer error
    - condition:
        - type: slot
          name: return_reason
          value: damaged_item
      text: I'm very sorry to hear that! Sending an email confirmation of the return order now.

    # Neutral when return is not because of retailer error
    - text: Thank you. Sending an email confirmation of the return order now.

Try it out

To get started, install the latest version of Rasa Open Source and try out conditional response variations with your conversational assistant. Our technical documentation for the feature can be found here. This is currently an experimental feature and we are actively looking for ways to improve its functionality. If you have questions or feedback please post on the Rasa user forums