EasyChair Preprint

№ 8338

Effective Analysis of Chatbot Frameworks: RASA

and Dialogflow

Shalini Singh and Satwinder Singh

EasyChair preprints are intended for rapid

dissemination of research results and are
integrated with the rest of EasyChair.

June 21, 2022

 Effective Analysis of Chatbot Frameworks: RASA and Dialogflow

Shalini Singh1[0000-1111-2222-3333] and Dr. Satwinder Singh2[1111-2222-3333-4444]

1
PG Student, Department of Computer Science and Technology, Central University of Punjab, Bathinda, India 151401
2 Head of Department, Computer Science and Technology, Central University of Punjab, Bathinda, India 151401

[email protected]

Abstract: In recent times, use of AI based chatbots have increased tremendously. Chatbots have turned to be very helpful in
the field of education, marketing, environmental etc. In this study the focus is maintained towards creating a chatbot for the
educational organisation namely, Central University of Punjab, Bathinda. The chatbot is created with Rasa and Dialogflow.
The queries and response were self-created for dataset. DIET classifier was used for Rasa for intent classification and entity
extraction. BERT and RoBERTa for Rasa configuration and LSTM for predicting actions are used .Similarly, same dataset is
used for creation of Dialogflow chatbot .Lastly, an analysis is performed on both to check the efficiency. This research
depicts the path to create chatbots with Rasa and Dialogflow and also the effectiveness amongst them.

Keywords: Chatbot, Rasa, Conversational Agent, Dialogflow, BERT, DIET

1 Introduction

While exploring more natural ways to integrate automation into daily life, conversational systems are becoming
more common as a basis for human-computer interaction. The rapid development of artificial intelligence
[Pantic, et al., 2003] and natural language processing are two technologies strongly related to chatbots on the
rise currently. It increases the demand for technological advances in AI and NLP. Chatbots are also called
conversational agents. It is designed with the help of AI software. A chatbot carries many of the mobile
application's features and websites within a messaging service. Increasing popularity of the chatbots in different
fields .Conversational agents use Natural Language Processing (NLP) and other AI techniques to engage in
contextual discussion. To begin, an agent must comprehend the context of the end user [Singh and Beniwal,
2021]. Any user goal can be gracefully handled by a chatbot and it can assist in completing tasks as efficiently
as possible. This does not always imply that the bot gives the answer every query but it will be capable of
handling the discussion.
The popularity of chatbots has prompted the creation of a variety of chatbot development platforms and
frameworks. There are several chatbot development platforms (e.g. Pandorabots, Manychat, and Chatfue) and
frameworks (e.g. Rasa, chatterbot, Dialogflow) out there. It can also be defined as a software with a specific
amount of artificial intelligence that can communicate with a person or another chatbot to give the idea that the
conversation is taking place with a real person rather than a computer programme (ZEMČÍK 2019) A chatbot
objective is to facilitate the exchange of meaningful information without the use of a human agent, yet the bot
itself mimics a living entity. The MIT Technology Review named chatbot technology as one of 2016's
breakthrough technologies. (Zhi and Metoyer 2020) . Chatbots, on the other hand, were invented far earlier.
Alan Turing was the inspiration for creating a chatbot. “Turing Test” developed in 1950 (Belfin et al. 2019). As
per Krol (1999), the test works as follows: an interrogator asks two subjects the same questions, and both
respond in the same way. A computer will be one of the participants, while a person will be the other. The
machine behaves intelligently just like a human.

1.1 Types of Chatbots

Based on work, the chatbots are divided into Rule-based chatbots and AI-based chatbots. Rule-based chatbot
work on specific instructions given by the user. The commands could be given through typing. The Rule-based
system is a straightforward but it is difficult to manage. Developers use AIML (Artificial intelligence Mark-up
Language) to write rules for that chatbot system. AIML is XML based language. Machine learning and natural
language processing are used by AI-based chatbots to give more interactive experience. Chatbots powered by AI
learn interactions through a mechanism based on human thinking. Machine learning is used by chatbots.
Machines can forecast and act in previous situations and interact with the humans (Hasan, 2019).
 Chatbots

Rule Based Chatbots AI Based Chatbots

Fig.1.Types of Chatbots

1.2 Natural Language Understanding (NLU)

Natural language understanding NLU [Kang, et al., 2020] is a branch of natural language processing (NLP) that
requires converting human language into a machine-readable format. Computers can automatically examine data
using natural language understanding (NLU) and machine learning. Rasa NLU and Dialogflow NLU can be
used in the Chatbot.

1.3 Rasa

Rasa stack (Rasa 2020)1 aims to bridge the gap among the research and implementation by making recent
developments in machine learning accessible to non-experts interested in implementing conversational AI
systems. Introduce the Rasa stack, Rasa NLU, and Core as simple building blocks for conversational systems.
Rasa stack is a open-source machine learning technologies for building contextual AI assistants and chatbots.
Rasa NLU is a natural language understanding library that includes entity extraction and purpose categorization.
Rasa Core is a chatbot framework that uses machine learning to manage interaction and anticipates the optimal
next steps based on NLU input, conversation history, and training data. (Tom Bocklish et.al.2017)

1.4 Dialogflow

Dialogflow (Dialogflow 2020)2 is a Google service that allows anyone to create interesting speech and text-
based conversational agents. Using the Google cloud platform a natural language processing (NLP) platform for
creating conversational applications. API.AI was the previous name of the company, which was acquired by
Google in 2016. It was renamed Dialogflow after that. Virtual assistant services for chatbots and contact centres
are provided by CX and ES.

2 Related Work

Pavel, (2021) provided the research community with the comparative analysis of two chatbot frameworks i.e.
Botkit and Rasa. For the comparison, each framework and a set of criteria were used to create two identical
chatbots were can be taken. The comparison was made utilising those types of characteristics, as well as the
implemented chatbots and available documentation for various frameworks. When those criteria are considered,
the Rasa Framework is more versatile and provides assistance than the Botkit Framework, according to the
findings of this study.

Flores, et al., (2020) focussed on Performance Comparison of Natural Language Understanding Engines in the
Educational Domain. Compare the most popular natural language understanding (NLU) engines to see which
one will perform better in the educational setting. In the educational area, a Rasa service to identify which
chatbot solution performs the best. With the right settings, Rasa's performance can be significantly enhanced. It's
important to remember that it's an open-source chatbot platform with a powerful natural language understanding
engine.

1
https://rasa.com/
2
https://cloud.google.com/dialogflow/
 Tseng, et al., (2020) anticipated the topic An Intelligent Disease Query System Based on Rasa NLU The
method uses data from Taiwan's disease control centre. The research was succeeded in making the health
service's question and response intelligent. In the actual world, this system was providing a win-win situation for
both individuals and health call centres.

Ngugen, et al., (2021) focused on the research NEU-chatbot: Chatbot for admission of national Economics
University. Most of the communication of students and university performed manually it consumed more time.
Researchers was developed an AI-based chatbot that allows students to receive daily curriculum updates and
admission information for new students. Proposed the Vietnamese chatbot .NEU – Chatbot achieved the best
97.1% accuracy on test set. Using whitespace tokenizer customizes a tokenizer for only Vietnamese words. In
this research. After three months the chatbot was adding to the Facebook fan page 50,000 questions in total.

Shin, et al., (2020) proposed the topic Designing Everyday Conventional Agents for Managing Health and
Wellness provided the conventional conversational agents have been developed for supported the wide array of
areas, including, decision making, autonomous vehicles and health behavior change design heuristics and
recommendations for conversational agents that support health and wellness and beyond. The design heuristic of
conversational agents formalises 11 heuristics for conversational agents. Researchers was improved the
conversational agents.

Shah and Shah, (2019) provided the comparison of various chatbot frameworks. The research gives an overview
of each and also states its features. Compared these frameworks on a certain features and identify the Google
best work. The steps on how to select a framework for the business were also discussed and which would be
suitable based on what the user needs the features in the chatbot.

Roman, et al., (2020) provided the research on “Hey assistant, how I can become a donor?” In the case of the
blood donation virtual agent it can be used the Dialogflow framework to develop the chatbot. 50 participants
who was interacted with the virtual agent. It is capable to respond to the utterances exactly related to the 30
common questions and concerns about the blood donation. A cognitive agent is a creative way for spreading
knowledge and engaging the people regarding the blood donation.

Yan, (2018) provided the research chitty-chitty Chatbot: Deep Learning for Conversational AI. This research
focused on the surveyed of non-task-oriented chabots assistant. It is open domain research work.

Meshram, et al., (2021) provide a review on conversational AI: chatbots. None of the chatbots were bi- or multi-
lingual, according to reports. The chatbot's capacity to talk in a variety of languages gives better results and
reaches different communities in the world. The chatbot has a lot of advantages that can be used everywhere, 24
x 7 accessibility.

Stoica, et al., (2021) provided the research is to offer an empirical investigation of the influence of Romanian
diacritics using the performance of two well-known frameworks for intent recognition and slot filling: Rasa and
wit.ai API. The slot filling task appears to be the most affected by this factor, with performance drops of up to
30% for Rasa and more than 40% for wit.ai.

Gamage, et al., (2021) providing the traditional interfaces to it services make it easier for more individuals to
use them. Such interfaces must be available in the community's native language. The paper improves the
employing Sinhala word embeddings in an implementation based on the frequently used Rasa Stack, accuracy
can be improved.

Christopherjames, et al., (2021) focuses on the researcher are of the people are not medically qualified for
studying or understanding the extremity of disease or symptoms.AI powered healthcare chatbots are beneficial
for assisting patients. The health based assistant system we developed the chatbot using Dialogflow Application
Program Interface (API). This Chatbot system related to the health system.

Vartimiadis, et al., 2021 focus on the research of for museums, a graph-based conversational approach to a
distributed collaborative multi-chatbot technique is being developed. Museums are creating chatbots to assist
tourists and give a more enjoyable visit. They define the predefined proper dialog routes. The paper proposed a
representative set of developed museum chatbot and the systematic evaluation approach for evaluating both
chatbot and platform. Utilizes KGs has strongly conversational with NLU and ML techniques. It is critical to
emphasise that the proposed approach's success is contingent on the quality of the KGs used as the underlying
knowledge representation technology.
Rahman, et al., 2019 focused on the study of some healthcare chatbot available in English and their languages
but not in Bangla. Machine Learning based closed domain chatbot on Bangla Language is known as ‘Disha’.
The user can converse the chatbot in Bangla Language and learn from interaction with the user. Decision tree,
Random Forest, Multinomial NB, SVM, Adaboost, and KNN are six supervised machine learning methods
used. The best accuracy is shown by SVM.

Bharti, et al, (2020) anticipated the topic Medbot: Conversational Artificial Intelligence Powered Chatbot for
Delivering Tele-Health after COVID-19 it focused on telemedicine can be used by the practitioner to cure the
patients during the coronavirus outbreak. Without having the physical visit for their treatment in a hospital
employing a conversational artificial intelligence-based application. Developing the bot can use a multilingual
conversational bot based on Natural Language Processing (NLP) will provide free primary therapy. On the
Google Cloud Platform, researchers suggest a conversational Bot called "apka chikitsak" for conveying
information. Even after the lockdown, our programme will make it easier for doctors to relieve their workload
without having to visit hospitals. Patients can consult and get information in the comfort of their homes.

Khadija, et al., (2021) proposed AI-powered health chatbots: toward a general architecture A chatbot is a
conversational or cognitive agent that invigorates human conversation, through text or voice messages. This
research provided us with the general architecture four AI-powered health chatbots available. Natural language
understanding (NLU) and natural language production were used by the four chatbots (NLG) for
communicating with the user. The main part used deep learning for providing solutions to the requests raised by
the user. The main four components mentioned in the study are the client platform, the APIs, the NLP Engine,
and the core engine.

Mnasri, (2019) focused on Recent advances in conversational NLP: Towards Chatbot Standardization compares
the regarding the present state of the art of conversation systems, their categories, and alternative ways to
constructing them. Sequence-to-sequence, reinforcement learning, and hybrid approaches. Developing the
problem of the natural conversational agent is too complex to be handled in a single model. As was the situation
with a lot of issues. When many systems are combined, the results are frequently better than when they are used
alone.

Liu, (2021) anticipated the benchmarking of natural language understanding services for building conversational
agents. The analysis mentioned about wide coverage evaluation and precision of on a huge, a multi-domain
dataset of around 25k user utterances was gathered and evaluated with intent and entity type. NLU services were
used. the performance of Watson outperformed. Dialogflow, LUIS, and Rasa were also used and all of the
frameworks performed well. This study focused on the evaluation of Watson NLU, Dialogflow, LUIS, and
Rasa. The result showed that out of all the given frameworks Rasa gave the best output.

Adellatif, et al. (2021) proposed the comparison of Natural Language Understanding Platforms for Chatbots in
Software Engineering. The performance of NLUs in identifying intents, calculating confidence ratings,
maintaining stability, and extracting entities was investigated. IBM Watson, Google Dialogflow, Rasa, and
Microsoft LUIS were used to evaluate the performance of four major NLUs. The NLUs' performance in intents
classification, confidence score, and entity extraction was examined in the study utilising two independent tasks
generated from a Repository and Stack Overflow contexts.

3. Methodology
We used Rasa platforms to create a chatbot for Central University of Punjab in this project. The Rasa platform is
made up of two primary components: Rasa Core and Rasa NLU. The conversation flow is managed by Rasa
Core. The purpose of Rasa NLU is to interpret and categorise intents, as well as extract entities from text inputs.
The Rasa Core is a solid performer. This demonstrates how to build a Rasa and Dialogflow chatbot for Central
University of Punjab,
Fig.2. Proposed Work
3.1. Dataset
In Rasa and Dialogflow can make a Knowledge Base chatbot so have to define the data in the rasa template and
Dialogflow template. In the Rasa chatbot, manually constructed a dataset with 84 distinct intents, 1324 intent
examples 1580 sentence patterns of intents, 371 entities, and 650 stories. Also, constructed the corpus of the
conversations for the Dialogflow and RASA. In the Dialogflow constructed he knowledge base chatbot.
Development of Artificial Intelligence (AI) based chatbot was developed for university inquiry for handling the
FAQs in Central University of Punjab, Bathinda. the implementation of the chatbot using Dialogflow and Rasa,
a Natural Language Processing NLP [Kaur and Singh, 2021] module is used to translate students’ queries
during conversations to structure the data.

In Rasa framework having the dataset in three files nlu.yml, rule.yml, and stories.yml. We have customized the
dataset in a suitable format in the Rasa Framework. The main component of the Rasa dataset is nlu.yml. Giving
the intents and stories in the yml file. Giving the dataset in the Rasa Framework in the Rasa template and CSV
form. In the Dialogflow the dataset is given in the CSV form with a CSV file. Given the dataset in the
Dialogflow template like intents and entities.

3.2 Workflow of Rasa

The action server can handle the call to the action, which may acquire the use of various external services. The
chat tracker is saved in the tracker storage. External database is used to save tracker objects. Conversation locks
are saved in a lock store. To keep the dialogue synchronised, Rasa employs a ticket lock system. Unless
otherwise provided, conversation locks, like tracker stores, are saved in memory. Trained models can be
accessible via a file system, such as a local hard drive or an HTTP server, or from the cloud (such as AWS).

Fig.3. Workflow of Rasa [From Rasa Documentation https://rasa.com/docs/rasa/arch-overview/]

3.3 Workflow of Dialogflow

Agent: Dialogflow converts end-user text or audio into structured data that your apps and services. A
Dialogflow agent resembles a human call centre representative.
Intent: Dialogflow matches end-user expressions to the best intent in your agent when they are written or
spoken by an end-user. Intent classification is the process of matching intent.
Training phrases: Dialogflow matches the intent when an end-user expression resembles one of these phrases.
Because Dialogflow built-in machine learning expands on your list with other, comparable phrases, you don't
have to define every possible example.
Action: For each intent, you can provide an action. When an intent is matched, Dialogflow sends an action to
your system, which you may use to trigger specific activities in your system.
Parameters: Dialogflow offers the extracted values from the end- user expression as parameters when intent is
matched at runtime.
Responses: To return to the end-user, you define text, speech, or graphic answers. These can deliver replies to
the end-user, ask for more information, or stop the interaction.

Entities: Entities can be extracted the data from the end user expression.

Contexts: Contexts is similar to natural language contexts. Dialogflow handle an end user expression. It
requires providing with context in order to correctly match intent.

Fig. 4. Workflow of Dialogflow [Sabharwal and Agrawal (2020)]

3.4 DIET Classifier

The Dual Intent and Entity Transformer (DIET) [Astuti, et al., 2021] is a transformer that can handle both intent
categorization and entity recognition at the same time, as its name suggests. With Rasa 1.8. DIET's greatest
advantage is its adaptability. It allows to such pre-trained embeddings such as BERT, Glove, Convert, and
others. As a result, depending on your information and the quantity of training samples, the overall loss is
calculated by adding entity, mask, and intent losses together. This is called DIET may be trained to perform all
three NLP functions at the same time. By default, mask loss is disabled.

Pipeline

Messages are processed by a series of components in Rasa Open Source. These components are executed in a
processing pipeline described in config.yml, one after the other. Customise the model and fine-tune it on your
dataset by using an NLU pipeline. There was the word embeddings that have been pre-trained. Pre-trained word
embeddings are advantageous since they already contain some linguistic information.

3.5 LSTM
Long short-term memory (LSTM) networks [Egan, et al., 2017] are a type of recurrent neural network that can
learn the order in which dependencies in prediction tasks should be learnt. This is a need in a broad range of
tasks, including machine translation and speech recognition, to name a few. The LSTMs of deep learning are a
difficult issue. LSTM may be used for intent classification.

3.6 BERT

BERT is called as the machine learning framework for the most natural language processing that is open source
(NLP). BERT is a program that given the environment to give the meaning words in a text. The BERT
framework was trained using Wikipedia text and may be fine-tuned using question and answering datasets.
BERT (Bidirectional Encoder Representations from Transformers [Devlin, et al., 2016] s a deep learning model
in which every output element may be linked to every input element, and the weightings between them are
dynamically interrogated and decided depending on their relationship. The basic model of BERT (uncased). A
masked language model (MLM) objective was used to train a model on the English language. I am using BERT
in the Rasa configuration.

3.7 RoBERTa

RoBERTa introduced on Facebook, Robustly optimized BERT approach. RoBERTa computation power is
more. To improve the training procedure, RoBERTa [Liu, et al., 2019] removes the next sentence prediction
Roberta uses 160 GB of text for pretraining. RoBERTa base Using a masked language modelling (MLM) aim, a
pre-trained model was applied to the English language. RoBERTa can be used in the Rasa configuration.

4. Results and Discussions:-

4.1 Experimental setup of RASA

In this work, experimented with the dataset. In Rasa dataset includes 84 intents, 1324 intent examples, and 650
stories. As see in the table 1 want to ask about the courses at the Central University of Punjab, Bathinda, send
the message what about the courses in the central university of Punjab. The Rasa NLU extracts the necessary
data and determines the user's intent "courses." Because dataset had a large number of observations, constructed
nlu.yml global train/test split is 80% train, 20% test. To perform k fold cross-validation, the training data was
divided into five groups of equal size. Although this cross-validation method is computationally intensive, it
produces a model when the dataset is substantial.

Table 1. Intent Example

Intent Example
Courses What different courses are offered?
Ask_apply_online Can I apply online?
Fee Schedule What are the admission fees?
CUPB Why is CUPB best?

Once finished requiring the model: how much good the model to assess the effectiveness of each response. Need
to categorize comments based on whether they are rational or intelligible, as determined by an independent
person. Precision, F1-Score, and accuracy measures will be used to assess the chatbot. When anticipate that an
observation belongs to a class when it doesn't, get a false positive (FP). A false negative (FN) is a result in
which the model forecasts the negative class inaccurately.

Precision = TP/TP+FP

Recall = TP/TP+FN

F1 =2* (Precision*Recall/Precision+Recall)

Accuracy = TP+TN/TP+TN+FP+FN

Firstly, using LSTM in the Rasa model the metrics can be defined and predicted the actions. The intent
classification is evaluated by F1-Score, accuracy, and precision.
 Table.2. Intent Classification

Metric Score
F1-Score 85.2

Accuracy 85.2

Precision 86.3

Use the DIET classifier for the Intent Classification and the Entity recognition. The Intent Classification is
evaluated by F1-Score, accuracy, and precision using cross-validation using DIET Classifier.

Table.3. Intent Classification

Metric Score
F1-Score 97.1

Accuracy 97.4

Precision 97.2

Compared three different pipeline 4 configurations: an original configuration, a light configuration, and a heavy
configuration using the Bert model and RoBERTa model.

Table.4. Configuration
Configuration Accuracy Precision F1-Score
Original Configuration 94.9 94.3 94.4
Diet-Light Configuration 95.3 94.7 94.7
Diet-Heavy Configuration using Bert 97.4 97.1 97.2
Diet-Heavy Configuration Using RoBERTa 95.6 95.2 95.6
 Configuration
94.9 94.3 94.4 95.3 94.7 94.7 97.4 97.1 97.2 95.6 95.2 95.6
100
90
80
70
60
50
40
30
20
10
Original Diet-Light Diet-Heavy Diet-Heavy
Configuration Configuration Configuration Using Configuration Using
BERT RoBERTa

Accuracy Precision F1-Score

Fig.5. Configuration

4.2 Some screenshots of the Rasa Framework Chatbot

Fig.6. Chatbot of Rasa

Fig .7. Chatbot of Rasa

4.4 Experimental Setup of Dialogflow

In this work experimented the same dataset on Dialogflow with 84 intents 1324 intent examples. This procedure
was repeated for the second time. In the Dialogflow we have to put the intent and entities. In the Dialogflow
constructed the knowledge base chatbot. AI based chatbot was developed under dialogflow framework. The
chatbot was designed for handling CUPB university inquiries and fAQs. For the implementation of the chatbot
using agent trained in the DialogFlow NLU. The Agent Training has completed then the agent was ready to talk.
When working with Dialogflow, after providing it with data, it gives us the results in the form of Intent
Detection Confidence and Match Confidence. In experimental phase we provide Central University of Punjab
dataset. Here are some queries results in table.5

Table.5 Dialogflow Results

4.5 Some Screenshots of Dialogflow Chatbot.

Fig. 8.Chatbot of Dialogflow

Fig.9. Chatbot of Dialogflow

4.6 Analysis of Rasa and Dialogflow

Analysis through human survey
The integration of the chatbot was done on social network application called telegram. Both the chatbots built
under rasa and dialogflow framework were integrated on telegram for the efficiency analysis. Compared the
Rasa and Dialogflow. One of the objectives of this research was to analyse the effectiveness of both the
chatbots. Integrate the chatbot in the telegram and surveyed it, around 100 students of the university used the
chatbots, 63% of people found the RASA to be an effective chatbot and 37% of people found Dialogflow
effective. The Chatbot has brought about several benefits without any human intervention. Inquiry chatbot to the
university, especially the Central University of Punjab, Bathinda.

100
90
80
70
60
RASA
50
40 Dialogflow
30
20
10
0
Analysis between the two Chatbots

Fig.10. Effectiveness of two chatbots

Analysis through frontend work

Fig. 11. Comparison of Chatbot

In this Fig.11the two chatbots screenshots are there one is Dialogflow and another one is Dialogflow. See the
Screenshot and analyse the query. In RASA the query is “what about the courses “the bot will reply correctly
with greet. In Dialogflow try the same query and see the response. The Dialogflow bot will never gave the
response properly.
Fig.12. Comparison of Chatbot

In this fig.12 the two chatbots screenshots are there one is Dialogflow and another one is Dialogflow. See the
Screenshot and analyse the query. In RASA the query is “okk “the bot will reply correctly with response bye. In
Dialogflow try the same query and see the response. The Dialogflow bot will never gave the response properly
and correctly.

5. Conclusion
Effective Analysis of Chatbot Framework: Rasa and Dialogflow. This Framework can develop the chatbot. The
Knowledge domain of our chatbot is about the Central University of Punjab, Bathinda. Analysis of both the
chatbot frameworks which one is more customizable. Using the right setup the dialogue management may very
effectively accomplish the work of intent extraction and user interaction in the discussion. Using this chatbot
will not only benefit the student to get the knowledge about the Central University of Punjab, Bathinda. Use
another huge dataset in the Rasa and Dialogflow give the better accuracy.

References
1. Abdellatif, A., Badran, K., Costa, D., & Shihab, E. (2021). A Comparison of Natural Language Understanding
Platforms for Chatbots in Software Engineering. IEEE Transactions on Software Engineering.
2. Astuti, W., Putri, D. P. I., Wibawa, A. P., Salim, Y., & Ghosh, A. (2021, April). Predicting Frequently Asked Questions
(FAQs) on the COVID-19 Chatbot using the DIET Classifier. In 2021 3rd East Indonesia Conference on Computer and
Information Technology (EIConCIT) (pp. 25-29). IEEE.
3. Belfin, R. V., Shobana, A. J., Manilal, M., Mathew, A. A., & Babu, B. (2019, March). A graph based chatbot for cancer
patients. In 2019 5th International Conference on Advanced Computing & Communication Systems (ICACCS) (pp. 717-
721). IEEE
4. Bocklisch, T., Faulkner, J., Pawlowski, N., & Nichol, A. (2017). Rasa: Open source language understanding and
dialogue management.
5. Bharti, U., Bajaj, D., Batra, H., Lalit, S., Lalit, S., & Gangwani, A. (2020, June). Medbot: Conversational artificial
intelligence powered chatbot for delivering tele-health after covid-19. In 2020 5th International Conference on
Communication and Electronics Systems (ICCES) (pp. 870-875). IEEE.
6. Christopherjames, J. E., Saravanan, M., Thiyam, D. B., Sahib, M. Y. B., Ganapathi, M. V., & Milton, A. (2021,
August). Natural Language Processing based Human Assistive Health Conversational Agent for Multi-Users. In 2021
 Second International Conference on Electronics and Sustainable Communication Systems (ICESC) (pp. 1414-1420).
IEEE.
7. Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2016). Bert: Bidirectional Encoder Representations from
Transformers.
8. Egan, S., Fedorko, W., Lister, A., Pearkes, J., & Gay, C. (2017). Long Short-Term Memory (LSTM) networks with jet
constituents for boosted top tagging at the LHC. arXiv preprint arXiv:1711.09059.
9. Flores, V. J. J., Flores, O. J. J., Flores, J. C. J., & Castilla, J. U. J. (2020). Performance Comparison of Natural Language
Understanding Engines in the Educational Domain. International Journal of Advanced Computer Science and
Applications (IJACSA), 11(8).
10. Gamage, B., Pushpananda, R., & Weerasinghe, R. (2020, November). The impact of using pre-trained word
embeddings in Sinhala chatbots. In 2020 20th International Conference on Advances in ICT for Emerging Regions
(ICTer) (pp. 161-165). IEEE.
11. Hassan, A. (2021). The Usage of Artificial Intelligence in Digital Marketing: A Review. Applications of Artificial
Intelligence in Business, Education and Healthcare, 357-383.
12. Khadija, A., Zahra, F. F., & Naceur, A. (2021). AI-Powered Health Chatbots: Toward a general architecture. Procedia
Computer Science, 191, 355-360.
13. Kang, Y., Cai, Z., Tan, C. W., Huang, Q., & Liu, H. (2020). Natural language processing (NLP) in management
research: A literature review. Journal of Management Analytics, 7(2), 139-172.
14. Liu, X., Eshghi, A., Swietojanski, P., & Rieser, V. (2021). Benchmarking natural language understanding services
for building conversational agents. In Increasing Naturalness and Flexibility in Spoken Dialogue Interaction (pp. 165-
183). Springer, Singapore
15. Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., ... & Stoyanov, V. (2019). Roberta: A robustly optimized bert
pretraining approach. arXiv preprint arXiv:1907.11692.
16. Livingstone, D. (2006). Turing's test and believable AI in games. Computers in Entertainment (CIE), 4(1), 6-es.
17. Mnasri, M. (2019). Recent advances in conversational NLP: Towards the standardization of Chatbot building. arXiv
preprint arXiv:1903.09025.
18. Meshram, S., Naik, N., Megha, V. R., More, T., & Kharche, S. (2021, June). Conversational AI: Chatbots. In 2021
International Conference on Intelligent Technologies (CONIT) (pp. 1-6). IEEE.
19. Pavel, M. I. (2021). Comparing chatbot frameworks: a study of rasa and botkit.
20. Patel, N. P., Parikh, D. R., Patel, D. A., & Patel, R. R. (2019, June). AI and web-based human-like interactive university
chatbot (UNIBOT). In 2019 3rd International conference on Electronics, Communication and Aerospace Technology
(ICECA), 148-150. IEEE.
21. Pantic, M., & Rothkrantz, L. J. (2003). Toward an affect-sensitive multimodal human-computer
interaction. Proceedings of the IEEE, 91(9), 1370-1390.
22. Roman, M. K., Bellei, E. A., Biduski, D., Pasqualotti, A., De Araujo, C. D. S. R., & De Marchi, A. C. B. (2020). “Hey
assistant, how can I become a donor?” The case of a conversational agent designed to engage people in blood
donation. Journal of Biomedical Informatics, 107, 103461.
23. Rahman, M. M., Amin, R., Liton, M. N. K., & Hossain, N. (2019, December). Disha: An implementation of machine
learning based Bangla healthcare Chatbot. In 2019 22nd International Conference on Computer and Information
Technology (ICCIT) (pp. 1-6). IEEE.
24. Rasa Documentation https://rasa.com/docs/rasa/arch-overview/
25. Shah, S., & Shah, S. (2019). A comparison of various chatbot frameworks. J. Multi-Criteria Decis. Anal, 6, 375-383.
26. Shin, J. Y., & Huh-Yoo, J. (2020, May). Designing Everyday Conversational Agents for Managing Health and
Wellness: A Study of Alexa Skills Reviews. In Proceedings of the 14th EAI International Conference on Pervasive
Computing Technologies for Healthcare (pp. 50-61).
27. Singh, S., & Beniwal, H. (2021). A survey on near-human conversational agents. Journal of King Saud University -
Computer and Information Sciences,
28. Sabharwal, N., & Agrawal, A. (2020). Introduction to Google dialogflow. In Cognitive virtual assistants using Google
Dialogflow (pp. 13-54). Apress, Berkeley, CA.
29. Stoica, A. D., Rad, A. C., Muntean, I. H., Daian, G., Lemnaru, C., Potolea, R., & Dinsoreanu, M. (2020, May). The
impact of Romanian diacritics on intent detection and slot filling. In 2020 IEEE International Conference on
Automation, Quality and Testing, Robotics (AQTR) (pp. 1-6). IEEE.
30. Tseng, H. T., Hsieh, C. C., & Lin, Y. W. (2020, November). An Intelligent Disease Query System Based on Rasa NLU.
In 2020 International Symposium on Computer, Consumer and Control (IS3C) (pp. 458-459). IEEE.
31. Varitimiadis, S., Kotis, K., Pittou, D., & Konstantakis, G. (2021). Graph-Based Conversational AI: Towards a
Distributed and Collaborative Multi-Chatbot Approach for Museums. Applied Sciences, 11(19), 9160.
32. Yan, R. (2018, July). “Chitty-Chitty-Chat Bot": Deep Learning for Conversational AI. In IJCAI (Vol. 18, pp. 5520-
5526).
33. ZEMČÍK, M. T. (2019). A brief history of chatbots. DEStech Transactions on Computer Science and Engineering,
(aicae).
34. Zhi, Q., & Metoyer, R. (2020, April). Gamebot: a visualization-augmented chatbot for sports game. In Extended
Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (pp. 1-7).