Expert chat-bots for cultural heritage

Expert Chat-Bots for Cultural Heritage Giovanni Pilato2 , Giorgio Vassallo1 , Agnese Augello1 , Maria Vasile1 , and Salvatore Gaglio1,2 1 DINFO - Dipartimento di ingegneria INFOrmatica University of Palermo Viale delle Scienze - 90128 Palermo - Italy {augello,vasile}@csai.unipa.it, {gvassallo,gaglio}@unipa.it 2 ICAR - Istituto di CAlcolo e Reti ad alte prestazioni Italian National Research Council Viale delle Scienze - 90128 Palermo - Italy [email protected] Abstract. A key role in the human reasoning capability is given by intuition. The aim of the work is to design a framework for an human-like information supplier, which combines “intuition” with the traditional, more rigid, rules-based knowledge. It is presented a preliminary system that represents the first step toward the aforementioned framework. The system allows a user to interact with a community of chat-bots having specific competences in order to navigate in a concepts space automatically generated with the Latent Semantic Analysis paradigm. The chat-bots knowledge bases have been created to deal with the Cultural Heritage field. Then they have been sub-symbolically coded in the semantic space created with LSA. Thanks to this approach, chat-bots are capable to estimate their own competence regarding questions formulated by the user, which are mapped in the same semantic space. 1 Introduction Conversational agents represent an original approach to the Human-Computer interaction problems. The simplest way to implement a dialogue system is to use software agents called chat-bots that interact with the user by using patternmatching based rules: these systems look for a lexical matching between the user query and the query-answer modules written in their knowledge base. Chat-bots systems allow to realize simply a dialogue system based on natural language, for this reason they can be used as interfaces to a vastness of applications which sweep from entertaining applications to educational applications and e-learning platforms, research engines, e-commerce web-site navigation and other applications. Many approaches try to integrate natural language interfaces, or the simple technology of chat-bots systems, with more sophisticated techniques[1][2][3][4]. It is well known that the human being left brain deals with logic and rationality; while the right brain deals with imagination and creativity. A key role in the human reasoning capability is given by intuition, which can be defined as the ability to see meaning, relationships and possibilities beyond the reach of senses. Intuition is closer to knowledge than to unfounded ideas and, thus, there is real value to be captured from leveraging it more often. Besides, there are no established methods to separate ideas based on psychological factors from those founded on undocumented but reality-based concepts. What is lacking in AI, and, as a consequence, in conversational system is the “right brain” capability of humans: intuition and genuine creation. An attempt to better the human characteristics of chat-bots has been realized with the CyN project[5]: it is an experiment on linking the pattern matching interpreter of ALICE, a prominent conversational system, to OpenCyc[6] the largest commonsense knowledge base available today. This system provides a pattern-based parser for use with OpenCyc suitable for chat-bots / virtual human experimentation and basic knowledge entry of logical relationships, create a set of example stimulus-response couples to answer users questions using OpenCyc and provide a natural language front-end ahead of the current OpenCyc schedule. It should be pointed out that often, during a conversation, the dialogue between human beings can sweep from a topic to another one, thanks to the associative capabilities of people. The aim of the work is to design a framework for an human-like information supplier, which combines “intuition” with the traditional, more rigid, rules-based knowledge. We believe that this “intuitive”/associative capability can be obtained using the LSA (Latent Semantic Analysis) methodology [7]: this paradigm infers the latent relations between the words belonging to a large collection of documents; words are represented as vector in a large-dimensional semantic space and to the similarity between words can be calculated using the distance between the vectors codifying them. Studies on LSA show that vectors related to words appearing in similar contexts are near in the obtained semantic space [8]. Therefore, tanks to the properties of LSA, it can be created a geometric knowledge representation model based on the automatic creation of conceptual axes. The goal is to build a framework where it is possible to integrate the associative intelligence with the more traditional rule-based one. In particular, in this paper, it is presented a preliminary system that represents the first step toward the aforementioned framework. The system allows to interact with a community of chat-bots having specific competences in order to realize an human-computer communication system with “intuitive” cleverness that leads to enhance the fluency of the dialogue, with particular regards to the Cultural Heritage field. The chat-bots have been realized with the ALICE technology [9] and they are capable to estimate their own competence about the questions formulated by the user; this capability is obtained with the LSA (Latent Semantic Analysis) methodology [7], that allows to overcome the limits of traditional, rules-based, chat-bots knowledge-base representations. The scenario reflects what happens in the real world: when, in a community of people, there is a conversation and one person asks for a question, only those people who recognize the topic or associate the heard sentence with their own knowledge answer him. The system is composed of a community of n chat-bots, including n − 1 specialized chat-bots and one generic chat-bot whose task is to answer about generic queries. The user will ask a question, that will be forwarded to all chat-bots, and an autonomous, introspective process, based on the sub-symbolic knowledge representation given by the LSA paradigm, will start inside each chat-bot. Then, only those chat-bots that recognize to have the necessary knowledge to cope with the question given by the user, will answer him. If in the knowledge base of the chat-bot it is not present the right pattern which matches the user query, it will be given the answer present in the knowledge base of the chat-bot that has activated the chat-bot. If none of the expert chat-bot can adequately answer the question, a generic chat-bot will continue the conversation, until the dialogue is redirected to a specific subject manageable by, at least, one of the expert chat-bots. Preliminary experimental results, regarding a number of 300 queries submitted to the system, show an effective selection of the right specialized chat-bot, according to the current topic of conversation. For completeness, it has also been estimated the Information Retrieval capabilities of the system: 62 queries have been used; for each query the pertaining categories have been selected and the related Precision and Recall [10] measures have been calculated comparing the results with a traditional IR system, obtaining satisfactory results. 2 2.1 Background Latent Semantic Analysis Latent Semantic Analysis (LSA) [7] is a paradigm to extract and represent the meaning of words by statistical computations applied to a large corpus of texts. LSA is based on the vector space method : given a text corpus of n documents and m words, it is represented as a matrix A where rows are associated to words, while columns are associated to documents. The content of the (i,j )-th cell of A is a function of the i-th word frequency in the j -th text; then the matrix A is replaced with a low-rank approximation generated by the truncated singularvalue decomposition (SVD) technique [8]. The SVD decomposes A into a set of r = rank(A) triplets of left ui and right vi singular column vectors and scalar singular valued σi : A= r X σi ui vi T (1) i=1 The ui and vi vectors form orthonormal sets, and the positive scalars σi are sorted in decreasing order. The truncated SVD can be used to generate an approximation of rank k of A by only using the first k << r triplets., i.e. P A ≈ Ak = ki=1 σi ui vi T . The LSA paradigm defines a mapping between the m words and the n documents and a continuous vector space S, where each word wi is associated to a vector ui in S, and each document dj is associated a vector vj in S[11]. 2.2 ALICE A.L.I.C.E. (Artificial Linguistic Internet Computer Entity) is an artificial intelligence natural language chat robot based on an experiment specified by Alan M. Turing in 1950. The A.L.I.C.E. software utilizes AIML, an XML-like language designed for creating stimulus-response chat robots. The ALICE chatbot’s knowledge base is composed of question-answer modules, called categories and structured with AIML (Artificial Intelligence Mark-up Language)[12]. The model of learning in A.L.I.C.E. is called supervised learning because a person, called botmaster, plays a crucial role. The botmaster monitors the robots conversations and creates new AIML content to make the responses more appropriate, accurate, believable, or human, or whatever the botmaster intends[12]. The basic knowledge unit in AIML consists of an input question, an output answer, and an optional context. This basic knowledge unit is called “category”. The question, or stimulus, is called the “pattern”. The answer, or response, is called the “template”. The dialogue is based on algorithms for automatic detection of patterns in the dialogue data. ALICE has won twice (2000, 2001) the annual Loebner Prize [13] which declares to “advance AI and serve as a tool to measure the state of the art”, which makes it a good candidate tool for human-like natural-language based dialogue systems. 3 System Design The associative mechanism is typical in human dialogues. Often, during a conversation, the dialogue can sweep from a topic to another one thanks to the associative capabilities of people. The scenario reflects what happens in the real world: when, in a community of people, there is a conversation and one person asks for a question, the answer is usually given by only those people who recognize the subject or associate the heard sentence with some topic residing in their own “knowledge base”. The aim of the work is the design of an human-like information supplier, which combines the “intuitive” knowledge, modeled as an associative mechanism, with the traditional, more rigid, rules-based knowledge. The presented solution is a first step toward a flexible human-computer conversational system that combines the “rational” capabilities of the pattern-matching rules with the “intuitive” capabilities given by a semantic space where words, and their associated meaning, can be mapped. To reach this goal, a set of specialized chat-bots have been designed with their own related knowledge base, interacting with the user exploiting an association mechanism primed by LSA. The conversational system is made of a community of n chat-bots, including n − 1 expert chat-bots and one generic chat-bot whose task is to answer about generic questions. The chat-bots are based on the ALICE chat engine which implements the Artificial Intelligence Markup Language (AIML)[9]. Each chat-bot takes part in conversation when its own competence is pertaining about the user questions. The system is based on a geometric representation of knowledge: to get the association capability, the AIML knowledge base of each chat-bot is sub-symbolically coded in a semantic space generated by LSA. Then the association is made mapping the question of the user in the same semantic space and comparing it with the sub-symbolically coded categories of the chat-bots. This technique primes an “intuitive” association-based mechanism. The user will ask the system a question, that will be forwarded to all chat-bots, and an autonomous, introspective, process, based on the LSA paradigm, will start inside each chat-bot. Then, only those chat-bots that recognize to have the necessary knowledge to cope with the question given by the user, will answer him. If in the AIML knowledge base of the chat-bot it is not present the right pattern which matches the user query, it will be given the answer present in the knowledge base of the chat-bot that has activated the chat-bot. If none of the expert chat-bot can adequately answer the question, a generic chat-bot will continue the conversation, until the dialogue is redirected to a specific subject manageable by, at least, one of the expert chat-bots. Both the chat-bots competence evaluation and the dialogue with the user are based on a LSA approach [7]. The procedure involves the following steps: – – – – Expert chat-bot creation and training; Generation of a semantic space by using the LSA technique; Sub-symbolic coding of the AIML categories of each chat-bot; Dialogue session with automatic selection of the best chat-bot capable to deal with the current subject. Each one of the previous steps is described in the following paragraphs. 3.1 Chat-bot creation and training The training of the experts chat-bots is got by a set of AIML files writing. Each chat-bot will have its own knowledge built as a list of natural language question/answer couples, coded in AIML categories, related about the specific competence argument of the chat-bots, according to the ALICE specifications. 3.2 Building of the Semantic Space by Using the Latent Semantic Analysis Technique It is generated a semantic space according to the LSA paradigm[8]. The LSA application begins with the selection of a set of documents, preferably inherent to the specific expertise of the chat-bots, from an huge repository, like Internet. Let m be the number of chosen documents, and n be the number of words of the whole corpus: from the analysis and processing of the documents it is obtained a matrix A(m × n),in which each row stands for a unique word and each column stands for a document. Each cell a(i, j) is a function of local and global weights to estimate the relevance of terms within or among documents[14][15][16]. Therefore, the weight a(i, j) will be: a(i, j) = Lij · Gi · Nj = SQRTij · IGF Fi · COSNj where: 1. Lij is the local weight of the i-th word in the j-th document; 2. Gi is the global weight of the i-th word in the collection of documents; 3. Nj is a normalization factor. and, if fij is the number of occurrences of the i-th term in the j-th document, ni is the number of documents where the i-th term appears, Fi is the frequency of the i-th term in the documents corpus: with p fij − 0.5 + 1 if fij > 0 SQT Rij = 0 if fij = 0 IGF Fi = COSNj = pPm Fi ni 1 i=0 (IGF Fi · SQT Rij )2 After the creation of the A matrix, the truncated SVD[17] allows to derive the closest, in a least square sense, rank-k approximation of A, given by: Ak = U ΣV T where U (m × k) and V (n × k) are matrices and Σ(k × k) is a diagonal matrix. The matrix Ak captures the major associational structure in the matrix A and removes the noise[18]. Each word present in the document corpus, used to generate the k -dimensional semantic space S, is therefore represented by a vector ri , given by the i-th row of the U matrix. Being the generated space a semantic space, the orthogonal vectors, given by the columns of the matrix U , can be seen as “conceptual axises”. The knowledge base of each chat-bot and the user queries will then be mapped in this created conceptual space. 3.3 Sub-Symbolic coding of the knowledge base of each chat-bot The chat-bots knowledge base is made by categories; each AIML category is composed by words, and each word is coded as a vector in the generated k dimensional semantic space S. Each AIML category can be considered as just a document, which can be represented as a vector in the k -dimensional space as the sum of its component term vectors. Each AIML file will therefore be mapped in a set of vectors that will constitute the sub-symbolic coding of the chat-bot knowledge base in the previously generated semantic space S. 3.4 Dialogue with the Chat-bots After that all the chat-bots have their own KB mapped in the semantic space S, the conversation between the user and the chat-bots can take place. The query of the user is coded in the same manner of the AIML categories, in the same semantic space S, as a vector sum of the term vectors composing the query. The similarity between this vector and the vectors representing the categories of all chat-bots is then calculated using the inner-product between each category vector and the user query vector. Let q be the user query and q its associated vector, let cij be the i-th category of the j-th chat-bot and cij its corresponding vector; their similarity is computed as: ci,j · q sim(ci,j , q) = (2) kci,j k kqk Besides, for a given user query it is calculated a chat-bot Competence Factor f cj defined as: f cj = max {sim(ci,j , q), i = 1 . . . nc|sim(ci,j , q) > T } where j is referred to the j-th chat-bot, nc is the number of chat-bot categories, ci,j the i-th chat-bot category, q is the query and T is a threshold, experimentally determined, above which the category ci,j can be considered pertaining the query. Each chat-bot compares its own competence factor with the competence factor of all others chat-bots. The k -th chat-bot with the greatest Competence Factor f cj takes part in the conversation with the user and tries to answer activating the ALICE engine answer module. This module is based on a pattern matching algorithm that searches for a lexical matching between the query and the chatbot’s categories. It can happen that the pattern matching module fails in finding the right answer, although the chat-bot has been considered competent about the query’s domain; in such a case, the chat-bot will answer with the category that has determined his activation in the conversation. At last, if none of the expert chat-bots is capable to deal with the current query of the user, the generic chat-bot will take part in conversation. The above procedure is repeated for each question of the user, until a goodbye message is written. 4 Results To evaluate the results of the proposed approach it has been realized a community of four chat-bots, one of which is a generic chat-bot with a knowledge base provided from the ALICE free-software community; the other chat-bots have been trained in the following domains: 1. Classic Greek Architecture; 2. Renaissance and Mannerism Painting; 3. Renaissance and Mannerism Sculpture; To generate the semantic space according to the LSA paradigm, 850 documents have been selected from the web; to create a space coherent with the topics managed by the chat-bots, the downloaded documents are related to the competence sectors of the chat-bots. In particular, the documents are HTML paragraphs or, in the case of particular art works, the title of the work followed by a short description. From this document corpus it has been obtained a (7965×850) terms-documents matrix A, the truncated SVD technique has been applied and the semantic space S has been therefore generated, giving a sub-symbolic coding of all terms present in the document corpus. To generate the semantic space S, a number of k = 100 dimensions has been chosen. Afterwards, the knowledge bases of the expert chat-bots have been created, except the knowledge base of the generic chat-bot that has been trained with 76 AIML files selected by the ALICE web-site. Then all AIML categories of the KBs have been then sub-symbolically coded as vectors in the previously generated semantic space S. Table 1 indicates the number of files and AIML categories that have been created for each chat-bot. Let N be the number of queries about a specific argument a, let Na be the number of times that has answered the chat-bot expert in the subject and let Ns the number of times that has answered a chat-bot not expert about the current topic. 300 of queries have been submitted to the system: a number of N = 100 queries for each competence domain of each chat-bot. For every group of queries the Argument Precision P (a) and the Argument Rejection S(a), defined as: Na Ns P (a) = , S(a) = (3) N N have then been evaluated. The threshold for determining the competence factor f ci of the i-th expert chat-bot has been experimentally fixed to 0.5. In the columns of table 2 for each argument are indicated respectively: 1. the number of times that has answered the Greek Art expert; Greek Art Expert Painting Expert Sculpture Expert N. AIML Files N. AIML Categories 14 237 10 137 7 158 Table 1. Number of File and AIML categories created for each expert 2. 3. 4. 5. 6. the the the the the number of times that has answered the Painting expert; number of times that has answered the Sculpture expert; number of times that has answered the Generic chat-bot; Precision Argument P(a); Rejection Argument S(a). To evaluate, for completeness, the Information Retrieval capabilities of the whole system, 63 queries, equally distributed among Greek Art, Painting and Sculpture. For each query the pertaining categories have been selected and the related Precision and Recall [10] measures have been calculated. The obtained results have been compared with the results obtained on the same queries set with Lucene[19], a traditional information retrieval system, based on the lexical matching between the boolean expression of the query and the documents. For both systems (IR based on LSA and Lucene) the Precision-Recall curve has been evaluated on the standard levels of Recall [0%, 10%, . . . , 100%]. In figure 1 it is shown the average Precision-Recall curve. In the following paragraph, a real conversation sample between the user and the expert chat-bots is given. 4.1 Dialogue Example User: Who was Leonardo da Vinci? Painting Expert: Leonardo da Vinci was a Florentine artist. User: Who was his teacher? Painting Expert: In Verrocchio’s workshop Leonardo was introduced to many activities. User: Could you tell me about the exponents of mannerism? Painting Expert: Leading Mannerists include Pontormo, Bronzino, Parmigianino, El Greco ,Tintoretto, Bernini, Brunelleschi, Benvenuto Cellini, Caravaggio and Tiziano. Sculpture Expert: Leading Mannerists include Pontormo, Bronzino, Parmigianino, El Greco, Tintoretto, Bernini, Brunelleschi, Benvenuto Cellini, Caravaggio and Tiziano. Greek Art Painting Sculpture Generic Chat-bot P(A) S(A) Greek Art 84 2 7 7 0.84 0.16 Painting 2 76 16 6 0.76 0.24 Sculpture 1 16 75 8 0.75 0.25 Table 2. Number of answers obtained from each chat-bot and Precision and Rejection Argument for each queries group Fig. 1. Precision-Recall comparative diagrams between the system and a traditional IR system (Lucene) User: I’m interested in David of Michelangelo Sculpture Expert: David / Apollo 1530. Marble, height: 146 cm. Museo Nazionale del Bargello, Florence. The statue was commissioned by the hated Papal Governor of Florence, Baccio Valori... User: Tell me about greek colonization in Sicily. Greek Art Expert: The Greek colonies in Sicily preserve the clearest traces of the pre-citystate... User: Please, speak about Agrigento’s valley. Greek Art Expert: The principal temples at the Acragas’s valley are Concordia, Castor and Pollux, Hera, Hercules, Vulcan or Hephaistos,Zeus Jupiter. User: Which is the oldest of these temples? Greek Art Expert: The temple of Hercules is Agrigento’s oldest temple and was constructed with 38 columns, of which only nine are still standing. User: I want now to speak about generic arguments. Generic Chat-Bot: Interesting. I want to find out more about you. 5 Conclusions and Future Work Thanks to the properties of Latent Semantic Analysis, it can be created a geometric knowledge representation model based on the automatic creation of conceptual axsises. The goal is to build a framework where it is possible to integrate the associative intelligence with the more traditional rule-based one. In particular, in this paper, it is presented a preliminary system that represents the first step toward the aforementioned framework. The system allows to interact with a community of chat-bots having specific competences in order to realize an human-computer communication system with “intuitive” cleverness that leads to enhance the fluency of the dialogue, with particular regards to the Cultural Heritage field. The chat-bots have been realized with the ALICE technology [9] and they are capable to estimate their own competence about the questions formulated by the user; this capability is obtained with the LSA (Latent Semantic Analysis) methodology [7], that allows to overcome the limits of traditional, rules-based, chat-bots knowledge-base representations. Preliminary experimental results, regarding a number of 300 queries submitted to the system, show an effective selection of the right specialized chat-bot, according to the current topic of conversation. For completeness, it has also been estimated the Information Retrieval capabilities of the system: 62 queries have been used; for each query the pertaining categories have been selected and the related Precision and Recall [10] measures have been calculated comparing the results with a traditional IR system, obtaining satisfactory results. Future work will regard a better integration of the chatbots rules-based knowledge representation with the associative capabilities given by the geometric knowledge representation generated by the LSA paradigm. Acknowledgments Authors would thank Engineering Ingegneria Informatica S.p.A. for its contribution to this research. References 1. O.Stock. Language-based interfaces and their application for cultural tourism. AI Magazine, March 22, pp 85-97, 2001. 2. K.Mori; A.Jatowt and M.Ishizuka. Enhancing conversational flexibility in multimodal interactions with embodied lifelike agents. Proc. of the 8th international conference on Intelligent user interfaces, Miami, Florida, USA, pp.270-272, 2003. 3. A.Koutamanis. Azile: an intelligent verbal interface for the retrieval of pictorial architectural information. proc. of CIB W78 conference, Distributing Knowledge in Building., 2002. 4. E.Mathews; G. Tanner Jackson; A.Graesser; N.Person and the Tutoring Research Group. Discourse patterns in why/autotutor. proc. of Natural Language Generation in Spoken and Written Dialogue, AAAI Spring Symposium 2003,March 24-26., 2003. 5. K.Coursey. Living in cyn:mating aiml and cyc together with program n. Resources avalaible at: http://www.daxtron.com, 2004. 6. OpenCyc. Resources avalaible at : http://www.opencyc.org/. 7. P.W. Foltz T.K.Landauer and D. Laham. An introduction to latent semantic analysis. Discourse Processes, 25:259–284, 1998. 8. S.T. Dumais T.K. Landauer. A solution to plato’s problem: the latent semantic analysis theory of acquisition, induction and representation of knowledge. Psychological Review, 1997. 9. Artificial linguistic computer entity (alice). Resources avalaible at: http://alice.sunlitsurf.com/alice/about.html. 10. B.Ribeiro-Neto R.Baeza-Yates. Modern Information Retrieval. Addison Wesley, 1999. 11. B. Dorow. D. Widdows, S. Cederberg. Visualisation techniques for analysing meaning. Fifth International Conference on Text, Speech and Dialogue, Brno, Czech Republic, September 2002, 2002. 12. Artificial intelligence markup language (aiml). Resources avalaible at: http://alice.sunlitsurf.com/alice/aiml.html. 13. Loebner Price. Resources avalaible at : http://www.loebner.net/prizef/loebnerprize.html. 14. G. Salton and C. Buckley. Improving retrieval performance by relevance feedback. Readings in information retrieval, pp.355-364, 1997. 15. T.G. Kolda E.Chisholm. New term weighting formulas for the vector space method in information retrieval, 1999. 16. S. Dumais. Improving the retrieval of information from external sources. Behavior Research Method, Instruments and Computers, No.23, pp:229-236, 1991. 17. M.W.Berry. Large scale singular value computations. International Journal of Supercomputer Application, 1992. 18. G.W. Furnas; S.Deerwester; S.T.Dumais; T.K.Landauer and R.A.Harshman. Indexing by latent semantic analysis. The American Society for Information Science, 1990. 19. Jakarta Lucene. Resources avalaible at : http://jakarta.apache.org/lucene/. View publication stats