Conversational AI — Rasa Framework basics

In this blog:

• What is Conversational AI?
• What is Rasa NLU?
• Official Rasa Developer Certification Topics

What is Conversational AI?

Conversational AI is a set of technologies that enable computers to understand, process, and respond to voice or text inputs in natural ways, and is typically used in conjunction with bots or intelligent virtual agents (IVAs). Done well, it helps people interact with complex systems in faster and easier ways, and helps businesses deliver personalized engagements and support at scale. — kore.ai

What is Rasa NLU?

AI assistants have to fulfil two tasks, first assistants needs to understand the input message i.e. Intent classification and entity extraction, then by giving correct responses based on the intent and entity.

The Rasa stack tackles these tasks with the natural language understanding component Rasa NLU and the dialogue management component Rasa Core. What makes NLP for contextual AI assistants. Rasa NLU provides this full customisability by processing user messages in a so called pipeline.

A pipeline defines different components which process a user message sequentially and ultimately lead to the classification of user messages into intents and the extraction of entities.

Official Rasa Development Certification

Pre-workshop Setup

Installing latest version of rasa on you respective operating system sometimes become challenging specially you are not using virtual environment and you breaking up the dependencies for your all projects, so i advise you to use anaconda virtual environment or virtualenv via pip.

Getting the latest rasa version and installing steps all are included in link.

This is the link provided as resource in Rasa Development Workshop.

Introduction to Rasa Open Source

Rasa is open source machine learning framework or automated text and voice-based conversations. Understand messages, hold conversations, and connect to messaging channels and APIs.

I am mentioning the official rasa youtube playlist link, it is one of the best conceptual explanation you can found on youtube.

Now let’s understand the project file structure.

1. Updating the configuration:

For Frequent questions and some casual conversation, you always want your assistant to respond the same way every time type of question is asked.

Rules allow you to do exactly that. To se rules, the you need to add the Rule Policy to your policies in your configuration file.

So there are two parts you have to configure in your config.yml file, one is the policy who makes the what is the next best action for the latest input from user and another is the pipeline in order to classify and extract the intent and entity from the sentences (message).

It Include the ResponseSelector in your NLU pipeline in your configuration file. The ResponseSelector requires a featurizer and intent classifier to work, so it should come after these components in your pipeline.

2. Creating rules:

You need to write only one rule for each retrieval intent. All intents grouped under that retrieval intent will then be handled the same way. The action name starts with utter_ and ends with the retrieval intent’s name. Write rule for responding to inputs.

3. NLU Training Data:

NLU training examples for the ResponseSelector look the same as regular training examples, except that their names must refer to the retrieval intent they are grouped under.

4. Defining static responses:

You have to define some static responses for the ResponseSelector the same naming convention as retrieval intents.

5. Custom Actions:

You have to define some customer actions for the custom tasks and by custom actions you can call API or database query or run some calculations.

Build your custom assistant

NLU (Natural Language Understanding) is the part of Rasa Open Source that performs intent classification, entity extraction, and response retrieval.

NLU will take in a sentence such as “I am looking for some quality dinner night restaurant in down town” and return structured data like:

Rasa Open Source uses YAML as a unified and extendable way to manage all training data, including NLU data, stories and rules.

You can split the training data over any number of YAML files, and each file can contain any combination of NLU data, stories and rules. The training data parser determines the training data type using top level keys.

The domain uses the same YAML format as the training data and can also be split across multiple files or combined in one file.

NLU training data consists of example user utterances categorised by intent. Training examples can also include entities. Entities are structured pieces of information that can be extracted from a user’s message. You can also add extra information such as regular expressions and lookup tables to your training data to help the model identify intents and entities correctly.

Intent

• Training examples grouped by user intent e.g. optionally with annotated entities.

nlu:
- intent: check_balance
  examples: |
    - What's my [credit](account) balance?
    - What's the balance on my [credit card account {"entity":"account","value":"credit"}

• Synonyms

nlu:
- synonym: credit
  examples: |
    - credit card account
    - credit account

• Regular expressions

nlu:
- regex: account_number
  examples: |
    - \d{10,12}

• Lookup tables

nlu:
- lookup: banks
  examples: |
    - JPMC
    - Comerica
    - Bank of America

Entities

Entities are structured pieces of information that can be extracted from a user’s message.

Entities are annotated in training examples with the entity’s name. In addition to the entity name, you can annotate an entity with synonyms, roles, or groups.

The full possible syntax for annotating an entity is:

[<entity-text>]{"entity": "<entity name>", "role": "<role name>", "group": "<group name>", "value": "<entity synonym>"}

Stories

Stories are composed of:

• Story: The story’s name. The name is arbitrary and not used in training; you can use it as a human-readable reference for the story.
• Metadata: Arbitrary and optional, not used in training, you can use it to store relevant information about the story like e.g. the author.
• A list of steps: The user messages and actions that make up the story.

stories:
- story: Greet the user
  metadata:
    author: Somebody
    key: value
  steps:
  # list of steps
  - intent: greet
  - action: utter_greet

Each step can be one of the following:

• A user message, represented by intent and entities.
• An or statement, which includes two or more user messages under it.
• A bot action.
• A form.
• A slot was set event.
• A checkpoint, which connects the story to another story.

Rules

Rules are listed under the rules key and look similar to stories. A rule also has a steps key, which contains a list of the same steps as stories do. Rules can additionally contains the conversation_started and conditions keys.

These are used to specify conditions under which the rule should apply.

rules:
- rule: Only say `hey` when the user provided a name
  condition:
  - slot_was_set:
    - user_provided_name: true
  steps:
  - intent: greet
  - action: utter_greet

Domain

The domain defines the universe in which your assistant operates. It specifies the intents, entities, slots, responses, forms, and actions your bot should know about. It also defines a configuration for conversation sessions.

version: "2.0"

intents:
  - affirm
  - deny
  - greet
  - thankyou
  - goodbye
  - search_concerts
  - search_venues
  - compare_reviews
  - bot_challenge
  - nlu_fallback
  - how_to_get_started

entities:
  - name

slots:
  concerts:
    type: list
    influence_conversation: false
  venues:
    type: list
    influence_conversation: false
  likes_music:
    type: bool
    influence_conversation: true

responses:
  utter_greet:
    - text: "Hey there!"
  utter_goodbye:
    - text: "Goodbye :("
  utter_default:
    - text: "Sorry, I didn't get that, can you rephrase?"
  utter_youarewelcome:
    - text: "You're very welcome."
  utter_iamabot:
    - text: "I am a bot, powered by Rasa."
  utter_get_started:
    - text: "I can help you find concerts and venues. Do you like music?"
  utter_awesome:
    - text: "Awesome! You can ask me things like \"Find me some concerts\" or \"What's a good venue\""

actions:
  - action_search_concerts
  - action_search_venues
  - action_show_concert_reviews
  - action_show_venue_reviews
  - action_set_music_preference

session_config:
  session_expiration_time: 60  # value in minutes
  carry_over_slots_to_new_session: true

Slots

The one things we haven’t been gone though is the slots.

Slots are your bot’s memory. They act as key-value store which can be used to store information the user provided as well as information gather about the outside would.

Most of the time, you want slots to influence how the dialogue processes. There are different slot types for different behaviours.

For examples, if your user has provided their home city, you might have a text slot called home_city. If the user asks for the weather, and you don’t know their home city, you will have to ask them for it. A text slot only tells Rasa Core whether the slot has a value. The specific value of a text doesn’t make any difference.

If the value itself is important, use a categorical or a bool slot. There are also float, and list slots. If you just want to store some data, but don’t want it to affect the flow of conversation, use an unfeaturised slot.

slots:
  name:
    type: text
    initial_value: "human"

• Text
• Boolean
• Categorical
• Float
• List
• Any
• Custom Slot type

Model Configuration

The configuration file defines the components and policies that your model will use to make predictions based on user input.

The language and pipeline keys specify the components used by the model to make NLU predictions. The policies key defines the policies used by the model to predict the next action.

Policies

your assistant uses policies to decide which action to take at each step in a conversation. There are machine-learning and rule-based policies that your assistant can use in tandem.

language:  # your language
pipeline:
  # - <pipeline components>

policies:
  - name: MemoizationPolicy
  - name: TEDPolicy
    max_history: 5
    epochs: 200
  - name: RulePolicy

You can customise the policies your assistant uses by specifying the policies key in your project’s config.yml There are different policies to choose from, and you can include multiple policies in a single configuration.

Policy Priority

In the case that two policies predict with equal confidence. For example, the Memorisation and Rule Policies might both predict with confidence 1. The priority fo the policies is considered.

Rasa Open Source policies have default priorities that are set to ensure the expected outcome in the case of a tie. They look like this, where higher numbers have higher priority:

• 6 — RulePolicy
• 3 — MemoizationPolicy or AugmentedMemoizationPolicy
• 1 — TEDPolicy

Pipeline

You can use Rasa to build assistants in any language you want.

Your Rasa assistant can be used on training data in any language. If there are no word embeddings for your language, you can train your featurizers fro scratch with the data you provide.

language: "fr"  # your two-letter language code

pipeline:
  - name: WhitespaceTokenizer
  - name: RegexFeaturizer
  - name: LexicalSyntacticFeaturizer
  - name: CountVectorsFeaturizer
  - name: CountVectorsFeaturizer
    analyzer: "char_wb"
    min_ngram: 1
    max_ngram: 4
  - name: DIETClassifier
    epochs: 100
  - name: EntitySynonymMapper
  - name: ResponseSelector
    epochs: 100

Actions

After each user message, the model will predict an action that the assistant should perform next.

Responses

A response is a message the assistant will send back to the user. This is the action you will use most often, when you want the assistant to send text, images, buttons or similar to the users.

Custom Actions

A custom action is an action that can run any code you want. This can be used to make an API call, or to query a database for example.

Forms

forms are a special type of custom action, designed to handle business logic. If you have any conversation designs where you expect the assistant to ask for a specific set of information, you should use forms.

Default Actions

Default actions are actions that are built into the dialogue manager by default. Most of these are automatically predicted based on certain conversation situations. You may want to customise these to personalise your assistant.

Custom Actions

A custom action can run any code you want, including API calls, database queries etc.

Any custom action that you want to use in your stories should be added into the actions section of your domain.

When the dialogue engine predicts a custom actions to be executed, it will call the action server.

from rasa_sdk import Action
from rasa_sdk.events import SlotSet

class ActionCheckWeather(Action):
   def name(self) -> Text:
      return "action_check_weather"

   def run(self,
           dispatcher: CollectingDispatcher,
           tracker: Tracker,
           domain: Dict[Text, Any]) -> List[Dict[Text, Any]]:

      dispatcher.utter_message("Hello World! from custom action")      return []

Important Links:

Note: mentioned likes used to write this article and writing the basic knowledge that i learned form Official Rasa Development Certification.

• https://rasa.com/showcase/
• https://rasa.com/docs/rasa/
• https://blog.rasa.com/rasa-education-and-certification/

Twiter Account:

Linkedin:

Vishvdeep Dasadiya — Associate Software Engineer — Accenture | LinkedIn

GitHub:

https://github.com/vdnew/