International Education Studies; Vol. 14, No.

2; 2021
ISSN 1913-9020 E-ISSN 1913-9039
Published by Canadian Center of Science and Education

Chatbot Development as a Digital Learning Tool to Increase Students’

Research Knowledge
Patchara Vanichvasin1
1
Faculty of Education, Kasetsart University, Bangkok, Thailand
Correspondence: Patchara Vanichvasin, Faculty of Education, Kasetsart University, Bangkok, Thailand. E-mail:
[email protected]

Received: August 4, 2020 Accepted: October 8, 2020 Online Published: January 27, 2021
doi:10.5539/ies.v14n2p44 URL: https://doi.org/10.5539/ies.v14n2p44

Abstract
The research aimed to: 1) develop the chatbot; 2) evaluate its effectiveness; and 3) investigate its effects on
students’ research knowledge. The sample consisted of 36 Thai university students. The research instruments
consisted of: 1) the chatbot; 2) an evaluation form; 3) an effectiveness questionnaire; and 4) research tests. Data
analysis used was mean, standard deviation, content analysis and a t-test. The findings indicated that: 1) the
chatbot was evaluated by experts with the applicability at a very high level ( = 4.67, S.D. = 0.08) with
recommendation to add more research content and interactive learning. The pilot test was done with 14 non-target
group of students. Students perceived the chatbot’s effectiveness at a high level ( = 4.43, S.D. = 0.35) with
comments to add more examples and graphics to make the chatbot more interesting; 2) the 36 target group of Thai
university students perceived the chatbot as an effective technology to use as a digital learning tool at a high level
( = 4.37, S.D. = 0.48). They thought that chatbot technology was easy to use, easy to understand, innovative and
fun for learning. They could get answers instantly and be able to seek specific information without waiting for
responses. However, in response to questions not matched keywords specified, further details of finding proper
answers such as links should be provided instead of leaving those questions unanswered. Also, the chatbot only
provided responses when typing correctly so there should be an option to choose from a list of questions or
keywords; and 3) the post-test scores were significantly higher than the pre-test scores at the 0.05 level of
significance. In conclusion, using chatbot technology in education settings to increase students’ research
knowledge gave positive results as it led to positive learning outcomes and helped provide better personalized
learning experience for students.
Keywords: chatbot, digital learning tool, research knowledge
1. Introduction
In educational settings, the role of research has greatly increased in modern times (Kothari, 2004) because research
is studied in some subjects and sometimes required by educational institutions as graduation criteria. However, the
lack of basic research knowledge on research skills may cause students to feel inadequate or result in serious
academic difficulties and not success in study (Sutasuwan, Sumalee, & Supsombat, 2016). Therefore, it is
important to understand basic knowledge of research so that the translation of that research knowledge into
research skills will be easily facilitated.
There are many possible means with technology support that can be integrated as a digital learning tool to help
enhance student’s research knowledge through personalized learning support due to the fact that technology for
education nowadays is changing how students learn (Singh, 2018) and it can help students learn faster with a wider
perspective (Havenstein, 2008), increase teaching’s effectiveness (Ellison, Steinfield, & Lampe, 2007), offer
guidance and use as information retrieval tool to give a positive and sometimes fun digital learning experience to
students who have good adoption of technology-based learning.
Chatbots are considered as one of the latest and trendy learning technologies with artificial intelligence. A chatbot
is programmed with text messages or speeches to simulate human conversation in a scripted way (Rouse, 2017;
Hettige & Karunananda, 2015; Imran, 2015; Heller & Procter, 2010; Dryden, 2008; Wang, 2008; Bayan & Atwell,
2007; Heller, Procter, Mah, Jewell, & Cheung, 2005; Weizenbaum, 1975), engages with a computer, often, over the
internet through text messages or speeches (Xu, Liu, Guo, Sinha, & Akkiraju, 2017), and can be trained with any
language or some jokes to create fun learning. Only typing questions or keywords are needed to receive answers or

44
ies.ccsenet.org International Education Studies Vol. 14, No. 2; 2021

explanation instantly (Shawar, 2005). Two types of chatbots are popular. Firstly, it is a rule-based or retrieval-based
chatbot that uses preprogrammed responses to match with the text messages or speeches inputted. Secondly, it is a
generative chatbot that generates as suitable a response as possible from the input through Natural Language
Processing (NLP) and deep machine learning (Liu et al., 2016; Nieves, 2018).
With integrated artificial intelligence, chatbots can be used as a digital learning tool to ask questions, give answers
to questions, retrieve information (Shawar & Atwell, 2007), visualize the contents (Bayan, 2005), explore online
content (James, 2016), provide useful information (Brandtzaeg & Følstad, 2017), establish the knowledge that the
students has to learn (Wenger, 1987), and propose possible solutions to the students individually (Singh, 2018)
because several studies have shown that poor personalized support can lead to weak student learning outcomes
while good personalized support can improve student learning outcomes (Pane et al., 2017; Hone & El Said, 2016;
Brinton et al., 2015; Eom, Wen, & Ashill, 2006).
Therefore, educational institutions should utilize chatbots to provide interactive e-learning environment for
students (Kowalski, Hoffman, Jain, & Mumtaz, 2011; Bii, 2013) and sometimes fun learning. Responses to
frequently asked questions can be programmed and instant responses can be provided (Cameron et al., 2017) while
making it easier and more comfortable for students to ask questions and obtain immediate answers to questions. As
a result, chatbots are able to support the learning process to be more engaging, interesting and entertaining for the
students while facilitating the teaching process (Singh, 2018) to free up teachers from responding to repetitive
questions and act as a virtual assistant or tutors (Garcia-Brustenga, Fuertes-Alpiste, & Molas-Castells, 2018) for
students.
Using chatbot technology has many potential benefits. The advantages of using chatbots include low cost, less
time in obtaining answers, better interaction, creative learning and improved efficiency when using in instruction
(Llic & Markovic, 2016; Bii, 2013) because users find chatbots safe and easy to chat online (Cameron et al., 2017)
with the ability to operate as a 24/7 support service, provide responses to repetitive or frequently asked questions
that can easily resolved, and give access to learning contents (Garcia-Brustenga et al., 2018; Winkler & Söllner,
2018) when required. In addition, students can refresh their memory by using chatbots to help recall, revise and
remember the knowledge studied. With chatbots, timely and efficient assistance or information can be obtained
with reported motivation and curiosity in their entertaining, social and relational factors. Students also view
chatbots as a novel phenomenon.
Furthermore, chatbots can play the role of teaching guide (Silvervarg, Kirkegaard, Nirme, Haake, & Gulz, 2014)
and assistant throughout the learning process with wide range of functions of obtaining information for students,
giving knowledge and enhancing its understanding with uninterrupted availability if learning through chatbot
technology is properly designed. The teachers can also use questions asked to collect data, modify a knowledge
base, and expand more knowledge by using chatbot technology to look for questions and add additional answers to
those questions asked in its knowledge base. Most students prefer using chatbot technology because chatbots are
able to give direct answers instead of links for further searching like using search and sort-based tools (Shawar &
Atwell, 2007; Shawar, 2005).
Nowadays, chatbots can work on Facebook’s instant messaging platform that is installed on most smart phones
(Stalista, 2017) and becomes the main network where students and teachers have their own account users with no
need to download any application and more comfort to use the application that they are already familiar with
(Lachs, 2017; Singh, 2018; Preedasutti, 2010). Apart from that, it is not necessary to have programming
knowledge to use chatbot technology as chatbots can recognize the language inputted by users, and recognize their
intent to provide relevant responses according to the input (Cameron et al., 2017).
Chatbots have significant educational potential and positive impact on student learning and satisfaction through
their personalized learning support (Winkler & Söllner, 2018). Although there were numerous studies related to
successful implementation of chatbots (Dutta, 2017; Huang, Lee, Kwon, & Kim, 2017; Kerly, Hall, & Bull, 2007),
only few of them have been used for educational purposes (Kowalski et al., 2011). At present, the use of chatbots in
education was scarce as not much research has been done on the utilization of chatbot technology in classrooms
(Baker, 2016; Goos et al., 1998; Bayan & Atwel, 2007; Gimeno, 2008; Wang, 2008; Torma, 2011; Govindasamy,
2014; Osodo, Indoshi, & Ongati, 2010). In Thailand context, there were some researches done on the use of
chatbots to provide customer service (Santirattanaphakdi, 2018), to guide a system (Bungodchai, 2017), to act as
performer agents (Lerdsahapan, 2015) and to diagnose a disease (Mokarat, Unchai, & Marpae, 2016) but there is
little research being done in education although chatbot technology has high potentials as a digital learning tool for
providing personalized learning support. Therefore, more researches are necessary to widen findings related to
chatbot technology.

45
ies.ccsenet.org International Education Studies Vol. 14, No. 2; 2021

Due to the possible and potential of chatbot technology as a digital learning tool with many functions to use for
education and the limited research on chatbot technology in educational settings, especially in Thailand, the
current study tended to use chatbot technology as a digital learning tool on Facebook’s instant messaging platform
to increase research knowledge of Thai students. It is anticipated that the chatbot developed can be used as a digital
learning tool in class to increase students’ research knowledge and create positive personalized learning support
through this digital platform.
2. Objective
1) To develop the chatbot as a digital learning tool with personalized learning support to increase students’
research knowledge
2) To evaluate the effectiveness of the chatbot from student perceptions
3) To investigate the effects of the chatbot on student’s research knowledge
3. Method
The experimental research was conducted with a mix method design. The quantitative research was related to the
chatbot applicability, its effectiveness, and students’ research knowledge before and after the chatbot use. The
qualitative research was related to recommendations from experts and feedback on the chatbot technology from
students. This research consisted of 2 main stages: 1) the development of the chatbot, questionnaires, and research
tests and 2) the implementation of the developed chatbot to investigate the effectiveness and effects of the chatbot
with related questionnaires and research tests.
Stage 1 involved the literature review of chatbot technology, the study of research content, and the development of
the chatbot, an evaluation form, an effectiveness questionnaire, and research tests.
Stage 2 involved the evaluation of the chatbot and related research instruments by 3 experts and the pilot test with
non-target group of 14 students. The results of the experts’ opinions and the pilot study were utilized to improve the
chatbot. Next, the chatbot was implemented with 36 Thai university students for one semester to investigate the
effectiveness and the effects on students’ research knowledge after the use of chatbot.
3.1 Population and Sampling Procedures
There were 60 Thai university students under the curriculum of Business and Computer Education at the Faculty of
Education, Kasetsart University, who were required by the curriculum to conduct research as graduation criteria.
36 Thai university students were selected using purposive sampling method as they were entitled to conduct
researches under the selected curriculum so they must have research knowledge in order to conduct researches
effectively and successfully, were Facebook messenger users as Facebook messenger was the instant messaging
platform where the developed chatbot was integrated, enrolled in classes which the researcher was the instructor,
and were willing to participate in this study. Although most of participants did not have online learning experience
with chatbot technology before, they were able to keep up with chatbot technology after the brief introduction of
how to use chatbot technology was provided as they were already accustomed to the Facebook’s instant messaging
platform.
3.2 Instruments
The chatbot was designed and developed to explore the effectiveness of chatbot technology and its effects with
three questionnaires, that is, an evaluation form, an effectiveness questionnaire and research tests. The details were
as following.
The chatbot was developed from the literature review of chatbot technology and content of research knowledge
with strong focus on the content, organization of the content and the application of chatbot. 3 experts evaluated it
with an evaluation from and the pilot test was done with 14 non-target group of students, who needed to conduct
researches as required by the selected curriculum and were Facebook messenger members with strong focus on the
chatbot effectiveness in 9 areas, that is, quickness, flexibility, convenience, interaction, providing information,
searching information, familiarity, learning and satisfaction.
The first questionnaire was an evaluation form based on the key components of chatbot technology in 3 areas, that
is, content of knowledge with 4 items, organization of content with 3 items, and application of chatbot with 5
items. Therefore, there were 12 items on a 5 Likert scale basis (1=very low to 5= very high) with an open-ended
question at the end of the form. The evaluation criteria of average scores () was established at a level of 4.50-5.00
as the highest, 3.50-4.49 as higher, 2.50-3.49 as average, 1.50-2.49 as lower, and 1.00-1.49 as lowest. The level of
4.50-5.00 was accepted applicability of the chatbot. This questionnaire was then utilized to obtain opinions related
to the chatbot applicability from experts before its use. The results of scores and recommendations were reported

46
ies.ccsenet.org International Education Studies Vol. 14, No. 2; 2021

and utilized to improve the chatbot before its use.

The second questionnaire was an effectiveness questionnaire based on the literature review of chatbot technology
in 9 areas, that is, quickness, flexibility, convenience, interaction, providing information, searching information,
familiarity, learning, and satisfaction. Each area consisted of 2 items. Therefore, there were 18 items on a 5 Likert
scale basis (1 = very low to 5 = very high) with an open-ended question at the end of the questionnaire for further
recommendation. The evaluation criteria of average scores () was established at a level of 4.50-5.00 as the
highest, 3.50-4.49 as higher, 2.50-3.49 as average, 1.50-2.49 as lower, and 1.00-1.49 as lowest. This questionnaire
was then utilized to obtain students’ perceived effectiveness of the chatbot including feedback and comments on its
use. The values of the index of congruence (IOC) ranged from 0.67-1.00 according to opinions of experts while the
reliability coefficient was established at 0.89 according to the pilot study with 14 non-target group of students.
The third questionnaire was research tests. They were developed from the content of research using a
pretest-posttest design with the same item order to investigate the effects of the chatbot on students’ research
knowledge before and after the implementation. There were two tests. Each test consisted of 15 multiple choice
questions, each with 4 possible answers but only one correct answer to investigate the understanding of research
knowledge. Each correct answer was worth one score. The total scores in each test ranged from 0 to 15 indicating
the understanding of research knowledge. The maximum total score of each test was 15. Total scores of two tests
were 30. The validity of each test was verified by 3 experts with the values of the index of congruence (IOC)
ranging from 0.67-1.00. The pilot study was done with 14 non-target group of students with the reliability values of
0.83. The difficulty and discrimination of the two tests were analyzed and reported at a level of difficulty (P)
ranging from 0.20-0.80 and discrimination (r) ranging from 0.20-1.00, respectively.
3.3 Data Collection
The target group of students took the research knowledge pre-tests in the first session before the use of chatbot.
After completing the pre-tests, they used the chatbot as a digital learning tool to provide personalized learning
support in order to recall, revise and remember research knowledge studied. At the end of semester, the post-tests
were distributed to investigate the chatbot’s effects on students’ research knowledge and the effectiveness
questionnaire to obtain student perceptions towards the chatbot’s effectiveness including feedback and comments
on its use. Finally, the completeness and accuracy of all instruments were checked before proceeding to data
analysis.
3.4 Data Analysis
Data obtained from an evaluation form, an effectiveness questionnaire, research tests were analyzed. Quantitative
data from opinions of experts and participants on the applicability and the effectiveness of chatbot technology were
used. A 5 Likert scale basis (1 = very low to 5 = very high) was utilized to evaluate the chatbot applicability
according to opinions of experts and the perceived effectiveness of chatbot technology from students. The t-test
was used to determine if there was a significant difference between the means of students’ research knowledge
before and after the use of chatbot. Qualitative data from open ended questions was reported to summarize
recommendations from experts and feedback on the chatbot technology from students. The data collected were
analyzed using mean, standard deviation, content analysis, and a t-test with the p-value set at the 0.05 level of
significance.
4. Results and Discussion
The research findings were as following.
4.1 Development of the Chatbot
The chatbot was designed and developed as a digital learning tool to provide personalized learning support
according to the literature review of chatbot technology. It was a rule-based or retrieval-based chatbot to match text
messages inputted with appropriate predefined responses to help increase research knowledge of students through
chat conversation so that students can make understanding of the research knowledge when using this digital
learning tool as their personalized learning support to recall, revise and remember the knowledge studied.
Facebook messenger, which students were familiar with, was used as an instant messaging platform over the
internet and Chatfuel was the chatbot for Facebook messenger with no need to install as it was already installed on
most smart phones and computers. Students asked the chatbot a series of simple questions through text messages
or typed any keyword to interact with content of research knowledge via smart phones or computers. Predefined
responses according to questions asked or keywords typed were provided in short time. The research knowledge
covered research methodologies, research process, research instruments, population, sample group, variables, data
collection, data analysis and research report.

47
ies.ccsenet.org International Education Studies Vol. 14, No. 2; 2021

The chatbot was then verified by 3 experts with strong expertise and experiences in education and technology to
check for its applicability and recommendation on its improvement so that it could be further improved to increase
students’ research knowledge and create a better personalized learning support. The chatbot applicability for use
was demonstrated in Table 1.

Table 1. Mean and standard deviation of the chatbot applicability

Chatbot Applicability  S.D. Interpretation
Content of Knowledge
1. Content is related to research knowledge 5.00 0.00 Very high
2. Content covers necessary research knowledge 4.33 0.58 High
3. Content of research knowledge is suitable for students 4.33 0.58 High
4. Content of research knowledge is easy to understand 4.33 0.58 High
Organization of Content
5. Organization of content is typing through text messages 4.67 0.58 Very high
6. Organization of content is interesting 5.00 0.00 Very high
7. Organization of content is interactive 5.00 0.00 Very high
Application of Chatbot
8. Chatbot is easy to use 5.00 0.00 Very high
9. Chatbot can be used at anytime and anywhere 4.67 0.58 Very high
10. Chatbot provides research knowledge students need 4.33 0.58 High
11. Chatbot provides research knowledge in short time 5.00 0.00 Very high
12. Chatbot provides correct answers 4.33 0.58 High
Total 4.67 0.08 Very high

Table 1 demonstrated that the overall mean was 4.67 and standard deviation (S.D.) was 0.08, which was in very
high level. In particular, the mean and standard deviation of content of knowledge was high ( = 4.50, S.D. = 0.25)
while the mean and standard deviation of organization of content and application of chatbot was very high ( =
4.89, S.D. = 0.19,  = 4.67, S.D. = 0.12, respectively). The experts recommended an addition of research
knowledge and interaction between users and the chatbot for more interactive learning. The necessary changes
were made according to experts’ opinions. The chatbot was then tested with 14 non-target group of students with
the high level of perceived effectiveness ( = 4.43, S.D. = 0.35). The students recommended an addition of more
graphics and more examples to make the chatbot more interesting. The necessary changes were made according to
the students to improve the chatbot prior to its use with the target group of students.
4.2 Students’ Perceived Effectiveness of the Chatbot
The chatbot was introduced at the beginning of the selected semester and implemented over one semester with
target group of students who were encouraged to use the chatbot as a self-help digital learning tool in inputting
their research queries and/or searching for information needed to get replies instantly without waiting for long
time. Students conversed and interacted with chatbots via smart phones or computers at any time, either in class or
outside class for personalized learning support as many times as needed. They were given an effectiveness
questionnaire at the end of semester to investigate their perceptions of the chatbot effectiveness. Student
perceptions towards the chatbot effectiveness were demonstrated in Table 2.

48
ies.ccsenet.org International Education Studies Vol. 14, No. 2; 2021

Table 2. Mean and standard deviation of students’ perceived effectiveness

Student’s Perceived Effectiveness of the Chatbot  S.D. Interpretation
Quickness
1. The chatbot provides instant responses after typing texts 4.25 0.69 High
2. The chatbot shortens time in waiting for responses 4.39 0.60 High
Flexibility
3. The chatbot can be used at anytime anywhere 4.69 0.52 Very high
4. The chatbot can be used via any device 4.39 0.64 High
Convenience
5. There is no need to download any application 4.61 0.49 Very high
6. The chatbot is easily accessible whenever needed 4.50 0.61 Very High
Interaction
7. The chatbot interacts through text messages 4.17 0.81 High
8. The chatbot interacts in the same context typed 3.94 0.75 High
Providing information
9. The chatbot presents research knowledge 4.44 0.56 High
10. The chatbot provides research knowledge needed 4.39 0.60 High
Searching information
11. The chatbot shortens time in searching 4.47 0.65 High
12. The chatbot gives right answers searched 4.03 0.81 High
Familiarity
13. The chatbot can work on Facebook, which students are already accustomed to this instant messaging
4.22 0.76 High
platform
14. There is no need for technical skills or proficiency to use chatbot technology as it is very easy to use 4.42 0.77 High
Learning
15. The chatbot stimulates interest in learning 4.42 0.69 High
16. The chatbot stimulates engagement in learning 4.39 0.84 High
Satisfaction
17. The chatbot provides research knowledge in a fun, interesting and innovative way 4.39 0.69 High
18. The chatbot is effective when used as a digital learning tool to provide personalised learning support 4.50 0.61 Very High
Total 4.37 0.48 High

Table 2 demonstrated that students’ perceived effectiveness of the chatbot was in a high level ( = 4.37, S.D. =
0.48). In particular, students perceived the chatbot effectiveness in terms of quickness, interaction, providing
information, searching information, familiarity, learning and satisfaction at high levels ( = 4.32, S.D. = 0.58,  =
4.06, S.D. = 0.69,  = 4.42, S.D. = 0.50,  = 4.25, S.D. = 0.66,  = 4.32, S.D. = 0.70,  = 4.40, S.D. = 0.74 and 
= 4.44, S.D. = 0.62, respectively) while perceived the chatbot in terms of flexibility and convenience at very high
levels ( = 4.54, S.D. = 0.47 and  = 4.56, S.D. = 0.50, respectively). Students reflected that chatbot technology
was easy to use, easy to understand, innovative and fun for learning. They could get answers instantly and able to
seek specific information related to research knowledge without waiting for responses. However, in response to
questions that did not match the keywords specified, further details of finding proper answers such as links for
further action should be provided instead of inputting the new questions and left those questions unanswered. Also,
the chatbots only provided responses when typing correctly so there should be an option for the students to choose
from a list of questions or keywords without typing any texts to get the answers required.
4.3 Effects of the Chatbot on Students’ Research Knowledge
Two research tests were assigned to students. Before starting the chatbot, students were given pre-tests. At the end
of semester, post-tests were distributed to determine if there were changes in student’s research knowledge. The
paired sample t-test was then used to determine if there was a significant difference between the two means of
students’ research knowledge at the different times (before and after use of the chatbot). The researcher checked
assumptions of normality before starting statistical procedures to ensure that there were no missing values and
outliers in the differences between the two means for the same group of students. The dependent variables were
normally distributed. Assumptions of normality were met for the two means and their testing showed no violation.
Table 3 demonstrated students’ research knowledge before and after implementing the chatbot.

49
ies.ccsenet.org International Education Studies Vol. 14, No. 2; 2021

Table 3. Students’ research knowledge before and after implementing the chatbot
Student’s research knowledge Total Scores  S.D. t-test Sig*
Pre-test 30 21.83 1.54
9.81* 0.00
Post-test 30 25.75 1.76

Table 3 demonstrated that the mean of the pre-test scores was 21.83 (S.D. = 1.54). After the chatbot use, the mean
of post-test scores increased to 25.75 (S.D. = 1.76). The post-test scores were higher than the pre-test scores at the
0.05 level of significance. The findings indicated that the chatbot increased students’ research knowledge because
students with higher scores reflected that the chatbot made it easier to get answers needed and make understanding
of those answers. At the end, students’ research knowledge was significantly increased when students understood
the knowledge clearly through using the chatbot as a digital tool to help recall, revise and remember the knowledge
studied with personalized learning support.
The main findings indicated that the chatbot, when used as an effective digital learning tool to provide personalized
learning support, was effective according to students’ perceived effectiveness and increased research knowledge
after the chatbot use. The findings were in line with Winkler and Söllner (2018) who reported that chatbots had
positive impact on successful learning and student satisfaction when using as personalized learning support. The
more chatbots learn from questions and keywords, the better chatbots can be further developed to provide better
content knowledge, which was in accordance with Knill, Carlsson, Chi, and Lezama (2004) who found that
chatbots helped the teacher see through asked questions where student had problems, gauge student learning and
weaknesses so that personalized support can be provided. Also, the findings can help instructors better integrate
chatbot technology to monitor student’s knowledge and create better personalized learning support in the future.
Therefore, using chatbot technology was effective in enhancing students’ knowledge, as the research findings
illustrated that there was significant difference in students’ scores before and after the chatbot implementation. The
current study can provide guidelines of using chatbot technology as a useful digital learning tool with personalized
learning support in an educational context to monitor students’ understanding and increase content knowledge.
With possible and potential use of chatbot technology, chatbots can present content knowledge with texts, images,
graphs, audio, videos or a combination of these to increase knowledge and motivate student learning through its
personalized learning support.
5. Conclusion and Recommendation
In conclusion, using chatbot technology in education settings to give personalized learning support in order to
increase students’ research knowledge gave positive results as it led to positive learning outcomes and helped
provide better personalized learning support through this digital platform. However, the sample size in this
research was quite small. Therefore, it was suggested that the future research may use a larger sample size with a
control and experimental group to extend results to a broader extent in using chatbot technology to create an
optimistic personalized learning support for students. Furthermore, the version of chatbot used in this study
consisted of some inflexibility and limitations as it was unable to handle unseen texts or keywords. Therefore, a
more flexible chatbot platform, which can generate as suitable responses as possible, may be considered for next
research.
Acknowledgements
This research would not be accomplished if the researcher did not receive kind assistance and great advice from the
experts during the development of chatbot and investigation of its use. Thank you for all Business and Computer
Education students from the Faculty of Education, Kasetsart University who participated very well in this research
study.
References
Baker, R. S. (2016). Stupid Tutoring Systems, Intelligent Humans. International Journal of Artificial Intelligence
in Education, 26(2), 600-614. https://doi.org/10.1007/s40593-016-0105-0
Bayan, A. S. (2005). A corpus based approach to generalising a chatbot system. Retrieved from
https://www.comp.leeds.ac.uk/research/pubs/theses/abushawar.pdf
Bii, P. (2013). Chatbot Technology: A Possible Means of Unlocking Student Potential to Learn How to Learn.
Educational Research, 4(2), 218-221.
Brandtzaeg, P. B., & Følstad, A. (2017). Why people use chatbots. Proceedings of the 4th International
Conference on Internet Science, 22-24 November, 2017, Thessaloniki, Greece.

50
ies.ccsenet.org International Education Studies Vol. 14, No. 2; 2021

https://doi.org/10.1007/978-3-319-70284-1_30
Brinton, C. G., Rill, R., Ha, S., Chiang, M., Smith, R., & Ju, W. (2015). Individualization for Education at Scale:
MIIC Design and Preliminary Evaluation. IEEE Transactions on Learning Technologies, 8(1), 136-148.
https://doi.org/10.1109/TLT.2014.2370635
Bungodchai, J. (2017). The development of chatbot prototype for guidance on a research government budget
system. The 9th NPRU National Academic Conference. 28-29 September 2017. Nakhon Pathom Rajabhat
University, Nakhon Pathom.
Cameron, G., Cameron, D., Megaw, G., Bond, R., Mulvenna, M., O’Neil, S., … McTear, M. (2017). Towards a
chatbot for digital counselling. Proceedings of the 31st British Computer Society Human Computer
Interaction Conference (pp. 1-7). https://doi.org/10.14236/ewic/HCI2017.24
Clark, D. (2018). The fallacy of “Robot” teachers. Donald Clark Plan B. Retrieved from
https://donaldclarkplanb.blogspot.com/search?q=10+uses+for+Chatbots+in+learning+(with+examples)
Dryden, G. (2008). MakeAiml: An AIML creation tool. Retrieved from
http://makeaiml.aihub.org/tutorials/aiml_template.php
Dutta, D. (2017) Developing an intelligent chat-bot tool to assist high school students for learning general
knowledge subjects. Georgia Institute of Technology. Atlanta.
Ellison, N., Steinfield, C., & Lampe, C. (2007). The Benefits of Facebook ‘Friends’: Exploring the Relationship
between College Students. Use of Online Social Networks and Social Capital. Journal of Computer
Mediated Communication, 12(4), 1143-1168. https://doi.org/10.1111/j.1083-6101.2007.00367.x
Eom, S. B., Wen, H. J., & Ashill, N. (2006). The Determinants of Students’ Perceived Learning Outcomes and
Satisfaction in University Online Education: An Empirical Investigation. Decision Sciences Journal of
Innovative Education, 4(2), 215-235. https://doi.org/10.1111/j.1540-4609.2006.00114.x
Garcia-Brustenga, G., Fuertes-Alpiste, M., & Molas-Castells, N. (2018). Briefing paper: Chatbots in education.
Barcelona: eLearn Center. Universitat Oberta de Catalunya. https://doi.org/10.7238/elc.chatbots.2018
Gimeno, A. (2008). The EUROCALL review. An evaluation of chatbots as aids to learning English as a second
language. Retrieved from http://www.eurocall-languages.org/review/index.html
Goos, G., Hartmanis, J., van Leeuwen, J., Goettl, B. P., Halff, H. M., Redfield, C. L., & Shute, V. J. (Eds.) (1998).
Intelligent tutoring systems. Berlin, Heidelberg: Springer Berlin Heidelberg.
Govindasamy, M. K. (2014). Animated Pedagogical Agents: A Review of Agent Technology Software in
Electronic Learning Environments. Journal of Educational Multimedia and Hypermedia, 23(2), 163-188.
Havenstein, H. (2008). Companies are Looking for New Ways to Measure Web 2.0. Computerworld, 42(45),
14-15.
Heller, B. & Procter, M. (2010). Conversational agents and learning outcomes: An experimental investigation.
Retrieved from http://auspace.athabascau.ca/handle/2149/2537
Heller, B., Procter, M., Mah, D., Jewell, L., & Cheung, B. (2005). Freudbot: An investigation of chatbot
technology in distance education. Retrieved from
http://psych.athabascau.ca/html/chatterbot/ChatAgentcontent/EdMediaFreudbotFinal.pdf
Hettige, B., & Karunananda, A. (2015). Octopus: A multi agent chatbot. Proceedings of 8th International
Research Conference, KDU (November, pp. 41-47).
Hone, K. S., & El Said, G. R. (2016). Exploring the Factors Affecting MOOC Retention: A Survey. Computers &
Education, 98, 157-168. https://doi.org/10.1016/j.compedu.2016.03.016
Huang, J. X., Lee, K. S., Kwon, O. W., & Kim, Y. K. (2017). A chatbot for a dialogue-based second language
learning system. CALL in a climate of change: Adapting to turbulent global conditions: 151.
https://doi.org/10.14705/rpnet.2017.eurocall2017.705
Imran A., & Shikha, S. (2015). AIML based Voice Enabled Artificial Intelligent Chatterbot. International
Journal of an e-Service, Science and Technology, 8(2), 375-384.
https://doi.org/10.14257/ijunesst.2015.8.2.36
James, G. (2016). A complete guide to chatbots. Retrieved from
http://www.garethjames.net/completeguide-chatbots/

51
ies.ccsenet.org International Education Studies Vol. 14, No. 2; 2021

Kerly, A., Hall, P., & Bull, S. (2007). Bringing Chatbots into Education: Towards Natural Language Negotiation
of Open Learner Models. Knowledge-Based Systems, 20(2), 177-185.
https://doi.org/10.1016/j.knosys.2006.11.014
Knill, O., Carlsson, J., Chi, A., & Lezama, M. (2004). An artificial intelligence experiment in college Math
education. Retrieved from http://www.math.harvard.edu/∼knill/preprints/sofia.pdf
Kothari, C. R. (1998). Research methodology: Method & technique. WishwaPrakashan, New Delhi.
Kowalski, S., Hoffman, R., Jain, R., & Mumtaz, M. (2011). Using conversational agents to help teach
information security risk analysis. SOTICS 2011: The First International Conference on Social
Eco-Informatics. Retrieved from https://www.thinkmind.org/download.php?
Lachs, J. (2017). Will chatbots revolutionise education? Retrieved from
https://www.opencolleges.edu.au/informed/features/will-chatbots-revolutionise-education/
Lerdsahapan, P. (2015). Role and communication of bot performer agents on Twitter (Master of Arts
(Communication Arts) Program, Faculty of Communication Arts, Chulalongkorn University).
Liu, C.-W., Lowe, R., Serban, I. V., Noseworthy, M., Charlin, L., & Pineau, J. (2016). How NOT to evaluate your
dialogue system: An empirical study of unsupervised evaluation metrics for dialogue response generation.
arXiv preprint arXiv:1603.08023. https://doi.org/10.18653/v1/D16-1230
Llic, D. J., & Markovic, B. (2016). Possibilties, Limitations and Economic Aspects of Artificial Intelligence
Applications in Healthcare. Ecoforum Journal, 5(1), 1-8.
Mokarat, C., Unchai, W., & Marpae, S. (2016). An ontology based chatbot application for diabetes diagnosis.
Proceeding of 2016 International Computer Science and Engineering Conference (ICSEC 2016).
Nieves, B. (2018). IA conversacional: Definición y conceptos básicos. Planeta Chatbot. Retrieved from
https://planetachatbot.com/ia-conversacional-conceptos-basicos-y-ladefinicion-107529e213c1
Osodo, J., Indoshi, F. C., & Ongati, O. (2010). Attitudes of Students and Teachers towards Use of Computer
Technology in Geography Education. Educational Research, 1(5), 145-149.
Pane, J. F., Steiner, E. D., Baird, M. D., Hamilton, L. S., & Pane, J. D. (2017). How Does Personalized Learning
Affect Student Achievement? https://doi.org/10.7249/RB9994
Preedasutti, S. (2010). Secondary school students’ internet consuming behavior in daily life: A study of
Traim-Udom Patanakarn school (Master Degree of Arts (Social Development), Faculty of Social
Development, Institute of Development Administration (NIDA)).
Rouse, M. (2017). What is chatbot? Retrieved from http://searchcrm.techtarget.com/definition/chatbot
Santirattanaphakdi, C. (2018). Online Marketing and Customer Service by Chatbot Case Study: Chatfuel in
Customer Interactive on Messenger. Sripatum Review of Science and Technology, 10, 71-87.
Sharwar, B. A., & Atwell, E. S. (2007). Chatbots: Are They Really Useful? Journal for Language Technology
and Computational Linguistics, 22(1), 29-49.
Shawar, B. A. (2005). A corpus based approach to generalising a chatbot system. School of Computing,
University of Leeds, Leeds.
Silvervarg, A., Kirkegaard, C., Nirme, J., Haake, M., & Gulz, A. (2014). Steps towards a challenging teachable
agent. In A. T. Bickmore, S. Marsella, & C. Sidner (Eds.), Intelligent Virtual Agents. 14th International
conference, IVA, Boston, MA, USA, August 27-29. https://doi.org/10.1007/978-3-319-09767-1_52
Singh, R. (2018). AI and chatbots in education: What does the future hold? Retrieved from
https://chatbotsmagazine.com/ai-and-chatbots-in-education-what-does-the-futurehold-9772f5c13960
Stalista. (2017). Facebook messenger monthly active users 2017. Retrieved from
https://stalista.com/statistics/417295/facebookmessenger-monthly-active-users
Sutasuwan, S., Sumalee, S., & Supsombat, W. (2016). Study of the Essential Research Skills for Graduate
Education. Kasetsart Education Review, 31(3), 102-113.
Torma, N. (2011). Artificial intelligence: Overview on question answering and chatbots. Retrieved from
http://www.logic.at/lvas/185054/Torma.pdf
Wang, Y. (2008). Designing chatbot interfaces for language learning: Ethnographic research into affect and
user’s experiences. The University of British Columbia, Vancouver. Retrieved from

52
ies.ccsenet.org International Education Studies Vol. 14, No. 2; 2021

https://circle.ubc.ca/handle/2429/2742
Weizenbaum, J. (1975). Computer power and human reason (1st ed.). San Francisco, W, H: Freeman.
Winkler, R., & Söllner, M. (2018). Unleashing the potential of chatbots in education: A state-of-the-art analysis.
https://doi.org/10.5465/AMBPP.2018.15903abstract
Xu, A., Liu, Z., Guo, Y., Sinha, V., & Akkiraju, R. (2017). A new chatbot for customer service on social media.
Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (pp. 3506-3510). ACM,
New York. https://doi.org/10.1145/3025453.3025496

Copyrights
Copyright for this article is retained by the author(s), with first publication rights granted to the journal.
This is an open-access article distributed under the terms and conditions of the Creative Commons Attribution
license (http://creativecommons.org/licenses/by/4.0/).

53