Workflow based just-in-time training

Workflow Based Just-in-time Training J. Davis, J. Kay, K. Lin, J. Poon, A. Quigley, G. Saunders, K. Yacef School of Information Technologies University of Sydney Australia {jdavis,judy,kenl,josiah,aquigley,gsaunder,kalina}@it.usyd.edu.au Abstract This paper focuses on the problem of information overload for newcomers in an organisation. We propose to address it by constructing a smart personal training assistant based upon workflow tools to drive temporal management of a just-intime workplace training system which will deliver a personalised and structured presentation of organisational documents. Keywords Document Workflow, Management, Information Retrieval. 1 Document Introduction Improving newcomers’ learning and integration into an organisation, institution, board or project, is a crucial problem encountered in almost any domain. Newcomers to a project team or organisation often face information overload that impedes their learning [4] and their resultant performance. If the newcomer holds an executive position, the situation becomes critical for the entire organisation. As with any training, structuring the exposure of newcomers to this knowledge helps them to learn more quickly and effectively, thus improving their efficiency and motivation, and also reducing financial costs for the organisation. We propose to combine the benefits of research in personalised training, personalisation based on a user modelling framework for representing a user’s knowledge and preferences and on knowledge management and the use of workflow technology to tackle this problem. The combination of workflows, personalisation and tutoring is novel. However, there has been some work exploiting workflow technology to supThe work reported in this paper is funded in part by the Cooperative Research Centres Program through the Department of the Prime Minister and Cabinet of the Commonwealth Government of Australia. Proceedings of the 7th Australasian Document Computing Symposium, Sydney, Australia, December 16, 2002. port teaching. The Flex-eL project [2, 1, 3] aims to support students as they work their way through a University level course. It provides “flexible learning pathways” based on workflow models of the study processes for the course. This approach relaxes constraints on the timing of learning processes, enabling the student to proceed at their own pace. It is also capable of providing the student with multiple pathways through the course, enabling them to tailor their learning to their own individual style. 2 Overview of Architecture Figure 1 illustrates the architecture of our JITT system. The training assistant makes use of workflow tools and draws upon user modelling and a document repository with links to the workflow. Workflow tools. The workflow tools enable the training assistant to deliver information in a timely fashion. A workflow model of each organisational process is generated which defines the temporal ordering of the various stages of the process. In addition, the workflow contains information about the time intervals between the various stages. When the user starts an organisational process, a workflow instance is created enabling the training assistant to track the progress of the user as they go about their various tasks. The assistant is then able to determine, based on the current stage in the workflow, what information the user requires in order to complete the current stage and prepare for imminent stages in the workflow. User modelling. The system maintains a model of each user of the system. This model contains information about the user’s present knowledge and past experience and is used by the training assistant to personalise the presentation of information to the user. The personalisation enables the user to easily find the information they need without having to sift through irrelevant data or data which they already know. In addition, the user may modify their user model to incorporate their personal preferences about the frequency and form of information delivery. Document Repository Personalised Document push Complete Workflow Collection instances for this user "Tell me what to read" Training Assistant Workflows pull Personalised Document skeletons "Tell me about X" User Model Figure 1: Just-in-time Training architecture Document repository. The system maintains a repository of training documents which form the basis for the content provided by the training assistant. These training documents are linked with various stages in the workflow models of organisational processes. In addition, the documents are formatted in such a way as to enable the training assistant to extract the most relevant information and format it in a consistent manner. It is also possible to link a trainer, that is, a human being with expertise on a given topic, to various stages in the workflow. Essentially, this generalises the notion of a ‘document’ so that the system can deliver the details of the expertise and contact details of a potential human advisor. This is particularly important for enabling JITT to take advantage of tacit knowledge, which is available as in a formal document. In summary, the training assistant examines the current stage(s) in the various workflows and generates a list of documents from the document repository that are associated with these workflow stages. This list of documents is then processed to remove irrelevant or previously known information based on the user model. Finally, the assistant formats the documents according to the user’s preferences and makes suggestions to the user about the documents they may be interested in or ought to read. In addition, the user may request information on a certain topic and the training assistant is able to answer this request with personalised documents on the relevant subject matter. 3 Workflows for JITT Figure 2 is a mockup of our JITT system with an intelligent conversational tutor interface called Justin.1 At the top right an example workflow can be seen describing the process of claiming pay. In this workflow, a casual lecturer can elect to be paid 1 Virtual Charter Image, kindly reproduced with the permission of the Centre for Speech Technology, Sweden. fortnightly, in a single lump sum or by issuing an invoice from a company. A number of documents may be associated with the “Claiming Pay” workflow. For example, the casual lecturer contract; a bank account declaration form; examples of completed versions of this form; tax details declaration form; University policy documents for casual academics etc. In Figure 2, Justin has suggested that the next logical step for the user is to fill in a personalised casual (pay) claim form which can be seen at bottom right. 4 Personalisation for JITT Underpinning the personalisation in our JITT system is the model of each user. This needs to represent relevant aspects of their knowledge and learning preferences. In general, the design of a user model involves a tradeoff between the desirable levels of detail that might be useful for personalisation against the practicality of capturing all relevant information about the user. The workflow foundation of the JITT system provides a systematic basis for designing the user model. Essentially, each workflow in the system can be regarded as one context for the modelling. Within that single workflow context, the elements of the workflow constitute elements of the user model. Once the elements of the workflow have been defined, it is possible to automatically generate the skeletal elements of the associated user model. To make user modelling feasible, we need to ensure there are mechanisms for determining whether the user knows each of these elements. Natural sources of user modelling information follow from the JITT architecture. The most obvious is available if the system is able to determine that the user has accessed a document. This is one form of evidence that the user knows something about each workflow element that has a link to that document. Another core source of evidence follows when Figure 2: Mockup of “Justin”, Our Intelligent Just-in-time Trainer. the user succeeds in performing one cycle of the workflow. For example, if the system has access to information that the user has acquired keys, it can use this as strong evidence that the user knows all the critical components of the workflow. It can also serve as weaker evidence about other components. We collect evidence from sources such as these to reason about the user’s knowledge and manage it in an evidence-based reasoning system. The architecture has been designed with the expectation that each workflow within the system will be quite simple. This reflects that each individual process within this learning domain is quite small. The problem for the user is that there are many different processes in which they engage. We now illustrate this with an example scenario. Sandra is a Professor of Computer Engineering and has just joined the University. She has several roles, including those of researcher in computer engineering, teacher of postgraduate and honours thesis students and teacher of an undergraduate course. These roles each need to be translated into a collection of workflows, one for each of the processes relevant to it. For example, her research activities mean that she needs to know about several processes, such as those for applying for university research grants, gaining internal funds for equipment, interacting with support staff when she has problems with equipment, managing university accounts for her existing grants and a diverse range of other processes such as those for organising conference travel. Each process involves a quite small workflow and its small collection of associated documents. At any one point in time, Sandra has successfully moved to a point in each workflow, perhaps not yet having started some of workflows at all, or perhaps having partially completed others and fully completed yet others. JITT can select information that might be suitable for pushing to Sandra by making a list of all those documents associated with points just ahead of her current position on each of the workflows that are active for her. It is the task of the training assistant to select from these and decide on the delivery of documents. For the pull process, the currently active workflows constitute a form of context for Sandra’s understanding of organisational processes. This means that requests she makes to JITT can exploit this context in searching the document base. It can rate documents as more likely to be relevant to a query if they are just ahead of her current position on each of the workflows that are active for her. It can also rate documents as more likely to be relevant to a query if they are on workflows she has not started to date. The training assistant combines this assessment with other document retrieval techniques based on term matches. 5 Teaching Strategies Teaching strategies are concerned with what to teach, how to teach and when. What In traditional intelligent teaching systems, the ordering of learning topics, i.e. the curriculum, is either explicit or relies on the structure of the knowledge being taught. However, in our approach the ordering is provided by the workflow. Each organisational process is represented by a workflow, with appropriate organisational documents attached to relevant points in each workflow. The advantage of workflows is that by their nature they sequence the steps through a particular process, making them naturally suited for suggesting the delivery of just-in-time documentation. When Documents are suggested to the user in two ways: pull and push. • The user may query the system to retrieve specific documents. By doing so, they may activate a new workflow process. • The system identifies documents that are becoming relevant to the user according to the currently active workflows and pushes them to the user on a needs basis. At any time several organisational processes are active for an employee, e.g. a teaching process, the process of writing a grant proposal, and so on. This means that several documents are relevant to the user at any given time. Our JITT system, behaving as an assistant, presents a list of documents to the user, sorted by several dynamic criteria (such as priority, associated workflow and due date). We plan to explore a range of interfaces and modalities to support interaction with the user. The style of interface shown in Figure 2 enables us to explore the use of the workflow in helping the user appreciate the context. We also intend to explore a more conventional style of interface which is a combination of email and search-engine interfaces, with a browser search-engine interface for the pull mechanism. These have the advantage of being familiar to the user and offer space for additional text about each document, allowing the user to click through to the actual document. Notification mechanisms such as email can also be added. How The level of recommendation varies according to the necessity of the document. Some documents are crucial, such as the pay claim form in our “Claiming Pay” example, while others are merely suggested, such as when the system cannot determine whether the knowledge contained in a document is already known to the user. Suppose the user returns to the “Claiming Pay” workflow a year later, would they want to download the claim form again? Explanations help learners to understand. Workflows contribute in two different ways. First, they provide causal links between a sequence of actions and documents, in a way that users can easily understand. Users do not have the time to read a large number of documents, so knowing why they need to read a specific document, and how this document will help them do a certain task, is crucial. Second, they can be simulated: the user could decide to simulate the actions of a part of a workflow to have a clearer picture of the actions to come, documents to be read and can also identify potential pitfalls or overloads before they arise. 6 Conclusion In this paper we have described an architecture for a just-in-time training system which exploits the structure afforded by workflow technology. In an organisation which has tools to manage its document collection and to capture workplace processes, the JITT architecture offers the possibility of additional leverage from the work invested in the workflow system so that this can be extended to support employee training. The workflow also serves as a foundation for generating a user model systematically, at the precise level of granularity of the workflow itself. This means that effort invested in modelling the workflow would be reused to generate the structure of the user model for JITT. The teaching strategy also benefits from the presence of workflow technology, allowing for the structured presentation of information and for the easy identification of relevant information. Additionally, the workflow allows for the simulation of a process, indicating documents that should be read and helping to identify problems before they occur. References [1] J. Lin, C. Ho, W. Sadiq and M. Orlowska. On workflow enabled e-learning services. In Proceedings of the IEEE International Conference on Advanced Learning Technologies, ICELT’2001, Madison, USA, August 2001. [2] O. Marjanovic and M. Orlowska. Making flexible learning more flexible. In IEEE International Workshop on Advanced Learning Technologies, New Zealand, December 2000. [3] S. Sadiq, W. Sadiq and M. Orlowska. Workflow driven e-learning: Beyond collaborative environments. In Proceedings of the World Congress on Networked Learning in a global environment, Challenges and Solutions for Virtual Education, Technical University of Berlin, Germany, May 2002. [4] J. Sweller. Some cognitive processes and their consequences for the organisation and presentation of information. Australian Journal of Psychology, Volume 45, Number 1, pages 1–8, 1993.