Socially Relevant Knowledge Based Telemedicine

AD AWARD NUMBER: TITLE: W81XWH-08-2-0186 Socially Relevant Knowledge Based Telemedicine PRINCIPAL INVESTIGATOR: Marshall Smith, MD CONTRACTING ORGANIZATION: REPORT DATE: TYPE OF REPORT: PREPARED FOR: Banner Health Phoenix, Arizona 85006 October 2012 Final U.S. Army Medical Research and Materiel Command Fort Detrick, Maryland 21702-5012 DISTRIBUTION STATEMENT: Approved for public release; distribution unlimited The views, opinions and/or findings contained in this report are those of the author(s) and should not be construed as an official Department of the Army position, policy or decision unless so designated by other documentation. Form Approved OMB No. 0704-0188 REPORT DOCUMENTATION PAGE Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing this collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to Department of Defense, Washington Headquarters Services, Directorate for Information Operations and Reports (0704-0188), 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 222024302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to any penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number. PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ADDRESS. 1. REPORT DATE 2. REPORT TYPE 3. DATES COVERED (From - To) October 2012 Final 1 5Sep 4. TITLE AND SUBTITLE 2011 – 14 Sept 2012 5a. CONTRACT NUMBER Socially Relevant Knowledge Based Telemedicine 5b. GRANT NUMBER W81XWH-08-2-0186 5c. PROGRAM ELEMENT NUMBER 6. AUTHOR(S) 5d. PROJECT NUMBER Marshall Smith, MD 5e. TASK NUMBER [email protected] 5f. WORK UNIT NUMBER 7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 8. PERFORMING ORGANIZATION REPORT NUMBER Banner Health Phoenix, AZ 85006 9. SPONSORING / MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR’S ACRONYM(S) U.S. Army Medical Research & Materiel Command Fort Detrick, MD 21702-5014 11. SPONSOR/MONITOR’S REPORT NUMBER(S) 12. DISTRIBUTION / AVAILABILITY STATEMENT Approved for public release; distribution unlimited. 13. SUPPLEMENTARY NOTES 14. ABSTRACT The advent of the Internet followed by the diffusion of Web 2.0 has the potential to revolutionize the delivery of clinical training in healthcare in both remote and urban clinical environments [1-4]. This is of significant relevance to the military given the shortage of healthcare providers and the remote locations in which the military has to operate. The objective of this proposal was to design, develop and evaluate a socially relevant knowledge driven collaborative training network. The scope of the project included non-collocated distributed clinical teams solving medical decision making problems with the help of Web 3.0 tools. We developed the collaborative virtual environments and defined clinical team activities for which the virtual worlds were used. Three different training modalities were utilized; traditional training, virtual training with persuasive techniques, and virtual training without persuasive techniques. Focusing on Advanced Cardiac Life Support training, we developed a virtual world platform to enable training of disparate teams on ACLS training, and then were tested on developed clinical scenarios. Then by coupling haptic devices with the virtual world, we enabled a multisensorial platform for team training. The preliminary results shows the validity of the collaborative virtual environment with persuasive techniques to address needs of clinical training as well as current methodologies and sets the stage for further evaluation of this approach. 15. SUBJECT TERMS Training for Collaborative Decision Making, Virtual Worlds, Web 3.0 16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF ABSTRACT a. REPORT b. ABSTRACT c. THIS PAGE UU U U U 18. 19a. NAME OF RESPONSIBLE PERSON NUMBER USAMRMC OF PAGES 19b. TELEPHONE NUMBER 29 Standard Form 298 (Rev. 8-98) Prescribed by ANSI Std. Z39.18 TABLE OF CONTENTS: Page Introduction ..................................................................................................................... 2 Body ............................................................................................................................... 2 Key Research Accomplishments ................................................................................... 20 Reportable Outcomes ................................................................................................... 21 Conclusion .................................................................................................................... 23 References ................................................................................................................... 24 Appendices ................................................................................................................... 27 INTRODUCTION The grant began in 2008 and slightly realigned in 2009 with a different research team under TATRC’s guidance and formal approval. The primary goal of this project has been to design and develop a virtual interactive collaborative team training environment that persuades users to perform a sequence of cognitive as well as psychomotor actions in timeconstrained environment. The events take place using a virtual learning environment which includes collaborative interactions by users who are located at different sites (noncollocated) integrated with virtual environments called Collaborative Virtual Environments (CVE.) These provide immersive virtual environments where users can perform various actions and but also can communicate and collaborate with others in the environment. Training in the CVE with Advanced Cardiac Life Support (ACLS) was evaluated for both initial learning as well as retention and degradation of skills over time, and these results are compared to the learning and retention in the traditional classroom collocated methodologies employed today. Virtual training not only has the ability to deploy more persuasive technologies with the potential of having a greater impact on changing behaviors, but also has significant economies of resources and time over more traditional methods of training. BODY: After developing the initial framework for virtual worlds and developing the underlying architecture in the first two years, this year our experiment focused on improving the virtual experience of the user in the clinical area and on the development of algorithms in cardiopulmonary resuscitation. Cardiopulmonary arrest (more commonly known as cardiac arrest) is the absence of mechanical activity of heart or, abrupt loss of functionality of heart. According to the American Heart Association, almost 80 percent of cardiac arrests, which occur out of hospital, are witnessed at home by a family member [33]. Approximately 6.4 percent of the patients who have a cardiac arrest ultimately survive [33]. This shows the importance of Advanced Cardiac Life Support (ACLS) skills, which requires a team to perform various tasks within a few minutes of patients’ arrival in the emergency room. It is a time-constrained, sequential procedure and a complex team event that requires fluid communication and coordination between the team members in order to save a patient’s life. The ACLS team has only five minutes to perform the sequence of actions, both cognitive (eg. decision-making such as which medicine to give, diagnosis of treatment scenario) and psychomotor (eg. CPR using proper techniques of frequency and depth of compressions), in order to save a patient. In most fields in which time is the most important factor and which require expertise in both cognitive and psychomotor skills for better decision-making, novices require an expert to disseminate knowledge and skills to them. Theoretical knowledge can be learned 2 in classroom environments whereas procedural skills and communication skills require more hands on practice to perfect. This approach of master-apprenticeship (or apprenticeship in common) model of education has been in existence for many years, where an expert performs a procedure and trainees carefully observe the procedures and practice them. In the case of learning psychomotor and communication skills, these are important because initially most of the trainees initially have limited skills and knowledge of the procedure. However, there exists a limitation to the number of trainees a trainer can train using the traditional collocated classroom methodologies employed today. [1]. Hamman, Beaubien, and Seiler [34] present the fact that errors in health care are directly related to the failures in the structure and function of the systems. The authors also mention that team training is given less preference than training an individual, although most of the care delivery is performed by teams of people. As mentioned earlier, ACLS is a team-based time critical event, so in order to deliver better care to the patients, it is important to understand the significance of team training as well as to consider more effective ways to provide training for these teams. ACLS: current training approach Almost all patient-care organizations provide regular ACLS training to facilitate emergency care providers to enhance their ACLS skills. In a typical training session, training team members have to take off work to attend a required class. When they congregate and start the class they initiate the process by assigning roles at first, then divide the tasks according to the roles, and follow the tasks. The team’s performance is monitored and evaluated (subjectively) by an evaluator throughout the period. After the session, the evaluator gives a final score based on the team’s performance, and later s/he debriefs what happened and what should have been done in the practice room. There might be a brief didactic session on ACLS too. After the debriefing session (and the didactic session when present), the team will perform another test, and the team is expected to perform better than the previous session. The same evaluator will evaluate the second session as well for improvements. The problem(s) Although the current training methodology looks comprehensive, there are various issues that are sub-optimal. The cost associated with overall setup falls on a higher range, and the time taken for training takes about 2 to 3 hours to complete. Much of this time is due to the large amount of orientation needed for training. In the context of learning, the training participants are not guided during the practice session. So, they have to recall what they had learned previously in the didactic session. There are rarely adequate trainers to provide training to the trainees, and frequently the trainees have limited time to practice the procedures properly. Apart from these, the ACLS training sessions are usually available on a limited basis, which is insufficient training and practice considering the criticality of the ACLS skills. Learning in virtual worlds With rapid development of computer storage, memory, processors, and high speed network infrastructure, it is now possible to create a virtual reality based simulations in a networked (distributed) environment that helps users to learn team coordination skills. 3 Computer Supported Cooperative Work (CSCW), in general terms, is considered to be a collaborative work done by users who are located at different sites. Telemedicine, telehealth, tele-conferencing all are examples of CSCW. When CSCW is integrated with the term Virtual Reality (VR), the environment is called as Collaborative Virtual Environments (CVE), or simply “Virtual Worlds”, which provide immersive virtual environment where users can perform various actions, and can also communicate and collaborate with others in the environment. CVEs have been used in various fields like gaming [4], online community building or socializing [4, 5], educational or working environments [6, 7]. CVEs are able to convey the social dynamics like turn taking, cooperation, appraisal, communication to users in a proper manner. In addition to that, users can be assigned different roles like doctor, patient, trainer, trainee etc. Current CVEs also support different media required for communication (text, audio, video), which are very important for group discussions. How virtual worlds can persuade users to change their behavior and attitude Because of the features that virtual worlds provide, they have potential to change behavior and/or attitude at different situations and different circumstances. Fogg mentions that there are many reasons that computers can be better persuaders than humans [8]. Some of the important reasons are: computers are more persistent; they provide greater anonymity; they can offer various modalities; computer programs can be re-scaled as per users’ need; and the most important one – “computer can be ubiquitous”. Virtual worlds provide all these features. They are more persistent; they are able to hide users’ information; various input output methods can be integrated with the virtual worlds; and can be modified as per the requirements. In presence of internet, virtual worlds can be accessed from any part of the world. Hence, we can say that CVEs are an integral part of persuasive framework in various fields like gaming (eg World of Warcraft), communications (eg. virtual shops: Amazon.com, eBay.com etc.), training systems for physical exercise (eg. virtual trainers: TripleBeat, Wii Fit, etc). With these abilities, computerized virtual reality based interactive systems have potential to persuade human users in the field of education as well. Advantage(s) of training in virtual worlds The most important advantage of use of computer based simulation in the field of education is that it can motivate students to learn and practice in a safe environment [9]. Simulation also enables students to practice different procedures in different contexts and different situations. Chodos et. al. suggest that virtual world simulations consume less resources and are capable of providing safe and realistic environment to practice [1]. The added persuasion in the computer simulation allows trainees to learn what the causes are and the effects caused. This persuades students to enhance their skills on role-playing, and changing their attitudes towards different perspectives [8]. Contribution and hypothesis In this study, we attempt to address the issue of team training in time critical events (ACLS in our case) and also the learning behavior of participants in different scenarios: when the participants are provided with persuasive elements and when they are not. We then discuss whether the participants can transfer the learned skills to the training room at a 4 hospital. Finally, we also see whether the participants can retain the skills in the virtual world. We also discuss the novel approach of integration of haptic device to the virtual world for time critical activities that requires psychomotor skills. After the study, we predict that: Hypothesis 1#: Virtual worlds are at least as effective in delivering team training. Hypothesis 2#: Virtually trained participants will retain the skills as long as or longer than traditional methods Objectives CVEs have a huge potential to provide training to many users in a virtual environment simultaneously. Our primary goal of this study is to design and develop an interactive collaborative team training simulator that persuades users to perform a sequence of cognitive as well as psychomotor actions in time-constrained environment. The study also focuses on the following important issues:  Evaluate the validity of virtual worlds in delivering team training and retention over a long period of time.  Monitor and record activities (and hence performance) of users while performing a collaborative task.  Create an online result sheet, which can be accessed from anywhere to view performances. (The security feature of the performance sheet can be customized: teams can view only their results; whereas a supervisor can view all results). Background The original project commenced in October 2008. From a financial perspective, many original quotes for equipment were no longer valid due to significant price increases of the equipment since the original proposal was submitted. This limited the ability to complete the proposed project for developing physical telemedicine connections across the western region of Banner. More importantly, the project did not have a clinical champion as the Principle Investigator and that would have been a major roadblock in accomplishing the goals of collaborative telemedicine. These factors were recognized within the first three months of the project, and at which stage TATRC was informed about the difficulties that had arisen. Arizona State University (ASU) continued to develop the web 2.0 backbone for the project, but the project was halted at that point. At this stage TATRC was contacted and engaged to better define a new project within the lines of military relevance and of importance to our organization. Banner Health presented a new plan to TATRC and it was approved on June 12, 2009. The actual project started in July 2009, and this is work presented herein. To lay the foundations of our work, we will present the related work and then highlight our conceptual framework Related Work We sub categorize this section into three parts: Team Training, Training in Virtual Worlds, and Persuasive Technologies. 5 Team Training Any coordinated effort, performed by a number of people in a group is termed as team work. Communication, coordination, cohesion etc. are typical characteristics of a team. All the team members should possess these skills in order to carry out assigned task. Team training is very crucial if well-coordinated team work is required. Today, almost every single case of care delivery in hospitals or outside hospitals involves a team of healthcare professionals, yet individual training is given more importance in real life [27]. There are various reasons behind this fact such as it is often hard to set up training sessions according to each individual’s schedule, health care professional trainees are from disparate locations etc. These healthcare training programs need to increase training experience of working in interdisciplinary teams for every individual caregiver. Hamman W et al demonstrated that identifying and focusing on team critical tasks and events prior to and during the training respectively, actually lead to significant performance improvement in teamwork skills [27]. Implicit coordination is one of the characteristics of high performance teams, where communication overhead is much less because the participants have access to the information without asking explicitly [28]. Communication overhead is typically the cost of communication and/or interaction measured in time, internet bandwidth, etc. [29]. Another aspect that vitally affects an individual’s ability to work in a team is shared mental models. As team members engage in a group activity, they tend to have similar thoughts/ideas in order to accomplish the task which ultimately results in less communication across the team [30]. These aspects are essentially a part of team dynamics which is important to be considered in a design phase of any experimental groupware activity. A competitive score is an important factor in motivating participation. Toups Z et al observed that if points are given based on team efforts, participants try harder to work as a team and accomplish the task in a well-coordinated and organized team effort [31]. Advanced Cardiac Life Support (ACLS) is a time-critical activity carried out by a dedicated high performance team. Training for such high performance teams is in real life scenarios is neither possible nor advisable since it is a life or death result to the patient, and simulation training is one of the best possible solutions available. According to Wayne et al, simulator training has shown significant performance improvement in a team of physicians while performing ACLS [32]. Training in Virtual Worlds Based on their purpose, Collaborative Virtual Environments (CVEs) or virtual worlds can be categorized into one of the following types: gaming, socializing or online community building, and educational or working environments [19]. [19, 20] outline the various factors that need to be present in a virtual world to be suitable for educational purpose. The authors compare various CVEs and come to the conclusion that selection of a particular CVE depends on the purpose of the training system. Below, we will briefly explain the research on CVEs that focus on healthcare and emergency training. 6 Wiecha et al explored the potential of a virtual world, Second Life (SL), as a delivering tool for continuing medical education (CME) [10]. In their study, participants had to select and adjust insulin level for patients with type 2 diabetes. For that purpose, participants had to listen to an instructional 40-minute insulin therapy talk. Two mock patients are also included in the study so that the participants can interact with the patients, and discuss within themselves. A questionnaire was provided to the participants before and after the talk session. The study shows that virtual world is very helpful for CME education by showing significant increase in the score after the talk than prior to it. Losh [15] lists several research work done by the Interactive Media Laboratory at Dartmouth Medical School based on virtual environments. Virtual Clinic is one of such work where a virtual clinic is designed by following the master floor plan. The main objective of this work is to allow learners to learn about social behavior and various procedures in clinical environments. The Virtual Terrorism Response Academy (VRTA) is a simulation based game to train users on how to act during crisis. The simulation focuses on providing rescue efforts when hazardous materials are involved. Before starting the game, users have to choose and assign themselves a ‘role’. Based on the role, which can be a fireman, emergency medical technician, etc., training is provided in didactic learning space. Quizzes and interactive videos are also included in order to engage the users. In an experimental session, a scenario is provided to the users and the main objective of the users is to practice with radiation meters and see how the exposure levels change when nearing hazardous objects. Similar to VRTA is Play2Train [18]. It is a virtual hospital and town environment which is created by Idaho Bioterrorism Awareness and Preparedness Program (IDAPP). The realistic virtual environment of Play2Train provides various kinds of emergency preparedness videos in virtual classrooms, and also supplements several training exercises to prepare users in case of emergency situations. After the practice sessions, the procedure followed by the students can be debriefed by the instructor to clarify the experiences; an essential part of simulation-based training. Callaghan et al use Second Life to create a virtual learning environment for engineering education. They demonstrate various interactive simulations that are part of engineering education [12]. Apart from the simulations, a virtual lecture theater is also present in the virtual world which contains interactive mini/main lecture slideshow viewer, media center for streaming video content and message centers for feedback. As Second Life does not provide SDK, the authors use open source e-learning software SLOODLE that links Second Life with a course management tool named Moodle. After demonstration of the simulations, the participants are asked questions: if they answer it incorrectly, they have to run the simulation again and answer the questions correctly. However, this study lacks the assessment and the evaluation of the participants and they mention that these shortcomings will be their main focus in the future. Boulos, Hetherington, and Wheeler [16] describe the potential use of Second Life in medical and health education. The authors provide two scenarios – ‘Virtual Neurological Education 7 Centre’ (VNEC,(http://www.vnec.co.uk) and ‘HealthInfo Island’ (http://infoisland.org/health_info). The former demonstrates a scenario where users are exposed to most common neurological disability symptoms. Apart from the symptoms, they are also provided with related information, events, and facilities in the Second Life. The latter involves providing training programs for virtual communities. It also intends to provide support to Second Life residents by providing them opportunities to participate in different medical groups dealing with stroke support, cerebral palsy etc. The research study performed by Chodos et al [1] focuses on the development of a research based virtual environment to enhance communication skills for health science education. They provide two case studies. The first one is the development of EMT/ER training simulation, which delivers an environment to train EMT/ER personnel on taking care of accident victim before taking him to a hospital. This case also focuses on exchange of patient information between EMT and ER personnel. The second case is designed to teach various competencies to students like rehabilitation medicine, nutrition, physical education etc. For the second case, the authors design a simulation in order to increase communication between the students to develop a home-care plan for elderly patient. Based on the case studies, they discuss the expectations of students towards virtual world based learning and the quality of learning. There are several other projects that focus on virtual healthcare system. Second Health is one of such projects where users can learn about how to use medical devices in hospital settings [12]. An interactive clinical scenario is provided to learn medical device training in simulated clinical environment. The participants are provided with both formative and summative feedback during the training session. However, the system does not provide clinical-skills training component in a collaborative environment where multiple users make a team and perform a collaborative task. Similarly, the Ann Myers Medical Centre [13] and the nursing training program from Duke University [14] provide meeting places for medical educators and students, where instructors can present lectures and present educational materials, and students can interact with each other. Persuasive Technologies Various researchers have worked on finding appropriate way to persuade users to perform various activities. Fogg [8] defines persuasive technologies as “interactive computing systems designed to change people’s attitudes and behaviors”. He lists various persuasive technology tools (terminologies) that can be an integral part of any system in order to either encourage or discourage users to perform some actions within the system and change their attitude and/or behavior while doing so. In medical training/education, persuasion is one of the most important factors that can affect the performance of trainees/students. Use of meaningful persuasive components (rewards, realism, social presence etc) enhances the learning where as bad design of persuasive components hinders it. In this section, we will mention some of the research work that has been done to encourage users to perform activities within a given system. Conradi et. al. [17] propose an idea of collaborative learning through problem-based learning (PBL) in Second Life, which they call PREVIEW. Researchers prepared five virtual 8 patient scenarios for learners, which were later delivered to the learners through Second Life platform. The main objective of the study was to find whether computerized simulation based PBL can be more effective than classroom based PBL. To engage students effectively in training the environment provided greater realism, active decision making, and suitable collaboration environment where the participants can interact with each other. The study shows that realism, and suitable interaction environment provided by Second Life engages students effectively in learning. Consolvo et al look at the design requirements for technology to encourage physical activity in [21]. For this study, they come up with a mobile phone application to encourage users to perform physical activity. The application has three different versions: baseline, personal, and sharing. The sharing version was the most advanced where users not only can see their activity, but also can share their performance to others and view others performance. Based on their study, they describe various factors that motivate users to perform physical activity. Giving proper credit on completion of each task, and providing personal awareness on users’ past performance, and current performance are the basic elements of the system that persuaded users. Another important factor is social interaction. According to the authors, social influence creates social pressure, which motivates users to be the best (or at least not the worst) in the society. TripleBeat [22] is also a similar kind of mobile phone based system that motivate runners to achieve predefined exercise goals using musical feedback as well as competition based persuasion, and real-time personal awareness. The experiment results conclude that the system is “significantly more effective” in helping runners to achieve the goals. How blogs and podcasts can be helpful tools to provide more sense of community in a group is explained by Firpo et al [23]. The major objective of their study is to change attitude and behavior of a community at School of Information Systems and Technology (SISAT) in order to foster a sense of community amongst its members. Based on the functional triad explained in [8], the authors conclude that social presence and credibility as the key factors to persuade the members in the community. Several virtual reality based games have already evolved to motivate users to maintain good health. The following simulation based applications have proved the fact that simulated environments are very effective to change one’s attitude and behavior. The Tetrix VR Bike [24] is an environmental simulation that motivates users to work out on this device by exploring the virtual environment. The faster users pedal, the faster will be the exploration. Another simulated environment is Bronkie the Bronchiasaurus [25], which is designed to help kids with asthma to manage their condition. The study showed that the asthmatic children who played the game for at least 30 minutes report increased selfefficacy to take care of their chronic condition. Similarly, HIV Roulette [26] is another simulation to provide immediate insights into sexual behavior. Users can view and select hypothetical character along with gender and behavior. Based on the selection criteria, the system reports whether the specified behavior is likely to cause HIV or any other sexually transmitted diseases. 9 Research Design 1. Usability testing and experimental setup Usability testing and experimental setup became even more critical because of the change of the platform, and the addition and integration of added assessments and the electronic form. Several “dress rehearsals” were run (without any data being utilized and/or saved for the experiment.) Any procedural and experimental errors were mitigated and resolved. Data collection began in April as planned and both phases completed as reported The “Help” menu/interface was created and implemented into the exercise before the data collection was begun, and has performed very well. The “Help” interface is different for each role and it is intended to help the participants in performing various tasks in the virtual environment. Figure 1. Help interface for “Compressor” role. Various instructions, alerts, and messages were developed and integrated into the game, and were displayed to the participants in the “Persuasive” group only. These instructions and alerts were displayed in regular intervals for important tasks to guide the participants in saving the virtual patient throughout the simulation, and also functioned very well during the sessions and data collection. A valid scoring system had previously been generated and implemented into the exercise. The scoring system was developed by following the ACLS protocol developed by American Heart Association. After consulting with professional ACLS trainers, severity levels were assigned to all tasks in the simulation, which were used in designing the scoring system. Several hundred sets of data were collected and analysis was performed. The tutorials provided detail instructions on interacting with the simulation thereby allowing the participants to understand the functionalities provided by the various icons used in the simulation. 10 Various usability test sessions were conducted to identify and mitigate any errors that might occur during VR training. Mock training sessions were organized for participating nurses to identify possible flaws, but these all functioned well during the data collection. System Design Figure 2: System Design 11 The virtual training environment has been designed using the server-client architecture. The simulated environment is hosted on a server and six clients can be connected from remote location. Figure 1 depicts the design of the simulator from the perspective of a single local user. Each client connected to the VR simulation consists of a unique User Interface (UI). Each UI consists of a graphical user interface (GUI) consisting of the role specific HUD and the feedback system, a method for inter user communication via headsets using the TeamSpeak VOIP API (TeamSpeak, 2002), and a modified Novint Falcon haptic joystick (Novint, 2000) required to perform CPR. Each user’s UI also comes with a performance evaluation module represented in the form of a patient outcome meter. This meter reflects the expected outcome of the patient that is assessed according to an adherence to the ACLS protocol. The patient outcome meter is common to all users along with the rendered environment. The outcome meter reflects the result of evaluation of all user performance in the scenario. Figure 3: Novint Falcon with CPR adaption The communication is hands-free i.e. voice activated. Each user can speak to all other users in the VR environment. The current speaker is identified according to their ACLS role and this information is displayed on HUD of all users. The Novint Falcon was modified to mimic the tactile pressures associated with compressing the chest of a patient. This was accomplished by the integration of a Laerdal spring used in CPR manikins into the Novint Falcon. All data from simulation is stored in an online database by integrating MySQL into the development environment API. Platform The ACLS simulator was implemented using the UnrealEngine3 via the Unreal Development Toolkit. (UDK, 1998) UDK is a free for non-commercial use game development kit that provides a means to create, edit, and deploy high fidelity 3D environments with sounds, animations, feedback via HUDs and menus, and allows for the integration of custom third party software libraries using C++ dynamic linked libraries. The custom libraries being used in for the ACLS scenario are TeamSpeak API for voice communication, Novint SDK for CPR feedback using the haptic joystick and finally MySQL 12 integration using cSQL libraries to provide database functionality. A major advantage of UDK over virtual world software is that UDK is free to use and allows the developers to create dedicated servers that may hold the database and run the simulation centrally. This results in the developers having complete control over all information collected in the simulation and also gives users the ability to rapidly customize the scenario if required. Furthermore, UDK allows for the creation of scenarios at a much higher level of fidelity than any existing virtual world software. Finally, UDK is a mainstream game development toolkit and therefore contains extensive documentation and support, features that are missing from virtual world development software and this helps in quick deployment of new simulations or modifying existing simulations. The environments are created using the UDK editor which allows for real time content creation and the backend scripting is done using native UDK script called UnrealScript. The C++ dynamic linked libraries required for third party APIs are called within UnrealScript classes. ACLS Scenarios Two types of ACLS code case scenarios were created-, 1) Shockable Rhythm : Ventricular Fibrillation (VFib) or Ventricular Tachycardia (VTach); and 2)Non-Shockable Rhythm: Pulseless Electrical Activity (PEA). Each rhythm type has a set of steps in common within the ACLS protocol and steps that vary depending on the type. Upon logging into the simulation the users are given one of two cases based on a random selection and they are required to identify the case and proceed accordingly. The scenario consists of six roles (Leader, Airway Manager, Respirator, Medicator, Compressor, Defibrillator) as seen in Figure 3. The ACLS code is a time critical scenario therefore all events within are time dependent. Figure 4: Virtual Reality Environment 13 Study Design The study comprised of three different study groups- Control, Persuasive, and Minimally Persuasive. The “Control” group was provided with conventional method of ACLS training whereas the other two groups were provided with virtual reality based training. The difference between the training in Persuasive and Minimally Persuasive groups was the use of persuasive messages/objects during training. Persuasive group was provided with various persuasive components whereas Minimally Persuasive group could only see a minimal set of persuasive components during training. The study was divided into two phases. The first phase (phase – I) was the comparative study between the performances of ACLS teams that were trained using conventional method of ACLS training and that of the teams trained in virtual reality training simulations, both with and without persuasive components. The data collection for the first phase started in the last week of March and was completed in June, 2012. Phase – I The first phase of the study consisted of 148 participants divided into 24, 3-6 member teams depending on availability. The distribution of the teams and the number of members is shown below: Number of team members Total number of teams 6 21 5 3 4 1 3 1 Chart 1: Distribution and team sizes for treatment groups Data for the teams with fewer than 5 members was eventually discarded. There was, therefore, data from 24 teams for the first phase of study with each treatment group consisting of 8 teams. All the participants were randomly selected to form the teams and the teams were randomly categorized into the groups. Experiment session for phase-I Each experiment session lasted for 3 hours. At first, the participants were informed about the study. In this session, participants had to fill up a demographics questionnaire and sign the consent form. Then the team was provided with two ACLS cases (Ventricular Fibrilation/Tachycardia, VF/VT; Pulseless Electric Activity, PEA), where the team had to follow ACLS protocol in order to save a simulated patient (manikin). This pre-test, gave us the baseline performance of each team before the training that would help in determining the effectiveness of the training afterwards. Each case lasted for approximately 5 minutes. The pre14 test was followed by a 20 minute video-based didactic training, where a video tutorial on ACLS protocol was provided to the team in a closed room. After the didactic training session, the team was provided with the “mode of training” based on the treatment group of that team. For the control group, the mode of training was the conventional skills training on a low-fidelity manikin; for the other two groups virtual reality based training was provided with full or partial persuasive components. A post-test followed the training session, in which the team was provided with two cases as in the pre-test session. After the completion of the post-test session, the participants were given a feedback questionnaire. Pre-test and Post-test sessions were evaluated by two expert evaluators who were blinded to the group and team formation i.e., the evaluators did not know which team was assigned to which group. Results Phase – I The preliminary analysis of the phase I data shows that there was not a significant difference in the majority of learning and performance metrics of the teams in the Control Group and the teams in the Persuasive Virtual Groups. This suggests that Virtual Training Environments, as long as persuasive elements are integrated, perform at least as well as traditional learning techniques in teaching in collaborative team environments. However, the performance and learning of the teams in the Minimally Persuasive group were significantly less than that of the other two groups, suggesting the necessity for the integration of persuasive elements into Virtual environments in order to achieve full potential for learning and training. Some of the groups had outliers with wide dispersions of values, and further analysis with statistical approaches is being conducted to see if these data points can be better superimposed for comparative results. However, most showed insignificant differences between the control and virtual with persuasive groups. The entire data set is being reviewed and will be completed within 30 days and will be submitted in its entirety in the final report. Following are some examples of the datasets showing minimal differences between the traditional and virtual with persuasion methodologies. 15 Figure 5: First, time to initiation of medication in Pulseless Electrical Activity patients Figure 6: Second, time to initiation of medication in Pulseless Electrical Activity patients 16 Figure 7: Time to initiation of medicine in Shockable patients Figure 8: Time to initiation of cardiac compression in Shockable patients 17 Phase – II Phase 2 of this research experiment was conducted from August 3rd to September 28th of 2012 to measure retention of ACLS skills. Eighty six of the original participants returned three to five months (100 to 150 days) after their initial training that occurred during phase 1. Due to the logistical barriers of maintaining team composition by bringing back all members from each original team, the participants were assembled into new teams from members of the same treatment group and assigned to the same roles that they had in phase I. Ensuring the formation of full six member teams was quite difficult due to last minute participant cancellations and no-shows. A total of twenty teams returned; seven from the control (conventional training) group, six from the minimally persuasive virtual reality group, and seven from the persuasive virtual reality group. Of the twenty there were 2 teams of 6, 4 teams of 5, 12 teams of 4, and 2 teams of 3. Number of team members Total number of teams 6 2 5 4 4 12 3 2 Chart 2: Team numbers The following table shows the distribution of team size for each treatment group. Treatment Number of Team Members 3 4 5 6 Control 1 4 1 1 Minimally Persuasive 0 3 2 1 Persuasive 1 5 1 0 Total 6 48 20 12 Chart 3: Distribution of team size for treatment groups All participants completed a short questionnaire to identify if they have been involved in a further ACLS training, mock code training, or real code situations since their initial training in part 1 and to determine in which roles they feel the most and least confident. Each team performed three consecutive mock codes. The basic scenarios were still Vfib and PEA, but the patient history was been changed from the Post-test so the participants would not recognize the scenario from the previous exercises. The three mock codes for testing retention were followed by three more mock codes with the team’s roles reorganized along each member’s declared weakest and strongest roles. Finally, to ensure participants returned to active patient care without any inadvertent misconceptions, the 18 expert evaluators debriefed the teams on unresolved errors and observed deviations from American Heart Association ACLS guidelines. Results Phase – II This dataset has just been completed at the time of this report and is presently being analyzed by the statistics team and is scheduled to be completed within 30 days. In addition to analysis of the comparison groups, the retention of skills and learning from the Phase I will be studied as well and analyzed for metrics of retention of learning and skills. Organizing teams along strong and weak formations may assist us in determining the influence of team leaders throughout the training and learning process. It also could help us assess the influence of a key difference between the virtual and conventional training methods in this study. The conventional method had everyone switch roles. It is difficult for all team members receive adequate time practicing in every role. In theory this should not be a problem since they should all be equally qualified to assume all roles, but in reality we know this is an issue among ACLS certified populations. A distributed training platform like the one we created may eventually enable users to spend more time training in every role. 19 KEY RESEARCH ACCOMPLISHMENTS 1. Development of virtual world which is based on actual code cart contents, code cart medications at Banner Health, and EKG monitor. 2. Development of Shockable Rhythm (Ventricular Fibrillation (VFib) or Ventricular Tachycardia (VTach)) and Non-Shockable Rhythm: Pulseless Electrical Activity (PEA) clinical scenarios for ACLS in the virtual world 3. Linking of actions/skills on a haptic device to an avatar in a virtual world for CPR training 4. Validation of the CPR training module 5. Development of the persuasive framework in virtual worlds 6. Validation of Virtual World based training simulation for ACLS 7. Validation of persuasive framework in virtual worlds. 8. Development and proof of concept research on an electronic, weighted checklist specific for monitoring competency and skills in ACLS. 9. Determination that the majority of practicing providers who perform CPR in training scenarios do not compress chest a full 2 inches as described as critical in the Guidelines for ACLS by the American Heart Association. 10. Development of a Virtual Reality training program (Collaborative Virtual Environments) for training participants in ACLS in a non-collocated environment that preliminarily appears to produce equal learning and skills as traditional and conventional training requiring collocated participants with higher costs. Further study will be required, but preliminary results are promising. 20 REPORTABLE OUTCOMES: Peer Reviewed Journal Articles P Khanal, S Parab, K Kahol, Mark Smith; Collaborative, Time-Critical, Multi-Sensory Training in Virtual Worlds with Persuasive Elements; Computer Human Interaction (CHI), 2011 K Kahol, M Vankipuram, V Patel, M Smith, “Deviations from Protocol in a complex Trauma environment: Errors or innovations?”, Journal of Biomedical Informatics, vol 44, 425-431, 2011 P. Khanal, A. Gupta, M. Smith, R. A. Greenes, Virtual Worlds in Healthcare: Systematic Review and Research Opportunities, Annals of Information Systems special issue in Healthcare Informatics, conditionally accepted, 2012. Peer Reviewed Major Presentations Marshall Smith, “Socially Relevant Knowledge Based Telemedicine”, TATRC’s Symposium Continuing Clinical Competence and Skills Deterioration, 18th Annual Medicine Meets Virtual Reality, February, 2011 Marshall Smith, “Virtual Reality in Team Based Training”, Military Health System Research Symposium, Fort Lauderdale, August, 2012 Peer Reviewed Poster/Abstract/Demo: A. Vankipuram, P. Khanal, A. Ashby, K. Josey, M. Smith, “Development of Virtual Reality based Advanced Cardiac Life Support Training Simulator in Unreal Development Kit®”, accepted, Theatrestyle demonstration, AMIA 2012, Nov 3-7, Chicago P. Khanal, S. Parab, K. Kahol, K. Josey, K. Zittergruen, M. Smith, “Virtual Reality based Advanced Cardiac Life Support Training Simulator using Active Worlds”, accepted, Poster, AMIA 2012, Nov 3-7, Chicago. P. Khanal, A. Vankipuram, A. Ashby, K. Josey, A. Gupta, M. Smith, “Virtual World for Advanced Cardiac Life Support Training”, submitted, 22nd Annual Workshop on Information Technologies and Systems (WITS), Dec 15-16, Orlando. Thesis/Dissertations Supported Sainath Parab, “Time Critical Team Training in Virtual Worlds”, Masters of Science Thesis, Arizona State University, November, 2010, Graduation December 2010 Akshay Vankipuram, "Design and Development of an Immersive Virtual Reality Team Trainer for Advance Cardiac Life Support", Masters of Science Thesis, Arizona State University, 2012 Graduation date: July, 2012 21 Prabal Khanal, "Design, Development, and Evaluation of Collaborative Training Method in Virtual Worlds for Time-critical Medical Procedure - Case Study on Advanced Cardiac Life Support”, PhD dissertation, Arizona State University Expected graduation date: March, 2013 Aaron Ashby, “Analysis of Clinical Team Communication during Cardiac Resuscitations: A Network Perspective”, PhD dissertation, Arizona State University Expected graduation date: Aug, 2013 22 CONCLUSION: Given the impending changes in healthcare with increased cost restrictions, concurrent with the demand for lower error rates and improved quality of patient care, newer ways have to be developed to provide equal if not improved training in team based events for non-collocated providers. New and innovative learning technologies must be integrated into learning processes for team training, both to improve the initial learning event as well as to increase the retention of these learned skills. Innovative technologies must be developed to remove the inefficiencies of having team training in collocated environments, and there is increasing needs to be able to train medical teams more efficiently and effectively in non-collocated environments. This study is of increased significance for not only the private healthcare sector, but especially in the military with their healthcare units becoming more and more disparate and isolated in today’s global deployments. If further work fully substantiates these preliminary results and conclusions, then this not only will revolutionize current methods of collocated provider skill training, but also open the door for training of health provider skills anywhere in the world with virtual reality training. Perhaps someday healthcare educators in Education Centers in the United States can train soldiers and medics in remote global deployments, midwives in rural India with high maternal mortality rates, and AIDS providers in Nigeria…all in one day and without leaving the U.S. 23 REFERENCES: [1] Chodos, D., Stroulia, E., Boechler, P., King, S., Kuras, P., Carbonaro, M., Jong, E., d., Healthcare Education with Virtual-World Simulations, SEHC’10, ACM, May 3-4, 2010, Cape Town, South Africa. [2] Hook, J.E.V., Building a National Agenda for Simulation-based Medical Education, AIMS, 2004. [3] Ziv, A., Simulation-based medical education: an ethical imperative, Journal of Academic Medicine, vol 78, no. 8, pp 783, 2003. [4] Brown, B., Bell, M., CSCW at play: ‘There’ as a collaborative virtual environment, CSCW’04, ACM, Nov 6-10, 2004, Chicago, Illinois, USA. [5] Ducheneaut, N., Moore, R. J., The Social Side of Gaming: A Study of Interaction Patterns in a Massively Multiplayer Online Game, CSCW’04, ACM, Nov 6-10, Chicago, Illinois, USA. [6] Toups, Z. O., Kerne, A., Hamilton, W., Blevins, A., Emergent Team Coordination: From Fire Emergency Response Practice to a Non-Mimetic Simulation Game, GROUP’09, ACM, May 1013, 2009, Sanibel Island, Florida, USA. [7] Callaghan, M.J., McCusker, K., Losada, J. L., Harkin, J.G., Wilson, S., Integrating Virtual Worlds & Virtual Learning Environments for Online Education, IEEE Consumer Electronics Society Conference Games Innovation, ICE-GIC, 2009, pp 54-63. [8] Fogg, B. J., Persuasive Technology: Using computers to change what we think and do, 2003, Morgan Kaufman. [9] Pedersen, P., Simulations: A safe place to take risks in discussing cultural differences, Simulation & Gaming, 26 (2): 201-206, 1995. [10] Wiecha J., Heyden, R., Sternthal, E., Merialdi, M., Learning in a Virtual World: Experience with Using Second Life for Medical Education, Journal of Medical Internet Research, JMIR, Jan – Mar, vol 12, no. 1, 2010. [11] Callaghan, M. J., McCusker, K., Losada, J. L., Harkin, J.G., Wilson, S., Integrating Virtual Worlds & Virtual Learning Environments for Online Education, IEEE Consumer Electronics Society Conference Games Innovation, 2010. [12] Second Health, website, retrieved August 29, 2010, http://secondhealth.wordpress.com, , 2009. [13] Ann Myers Medical Centre, website, retrieved September 1, 2010, http://ammc.wordpress.com/, 2009. [14] Johnson, C., Vorderstrasse, A., Shaw, R., Virtual Worlds in Health Care Higher Education, Journal of Virtual Worlds Research, vol 2, no. 2, 2009. [15] Losh, E., The Birth of Virtual Clinic: The Virtual Terrorism Response Academy as Serious Game and Epistemological Space, Sandbox Symposium, ACM, August 04-05, 2007, San Diego, California, USA. [16] Boulos, M. N. K., Hetherington, L., Wheeler, S., Second Life: an overview of the potential of 3-D virtual worlds in medical and health education, Health Information and Libraries Journal, 2007, vol 24, pp. 233-245. 24 [17] Conradi, E., Kavia, S., Burden, D., Rice, A., Woodham, L., Beaumont, C., Savin-Baden, M., & Poulton, T. (2009). Virtual patients in a virtual world: Training paramedic students for practice. Medical Teacher, 31(8), 713. [18] Boulos, M. N. K., Ramloll, R., Jones, R., Toth-Cohen, S., Web 3D for Public, Environmental and Occupational Health: Early Examples from Second Life, International Journal of Environmental Research and Public Health, 2008. [19] Tsiatsos, T., Andreas, K., Andreas, P., Collaborative Educational Virtual Environments Evaluation:The case of Croquet, Workshop on Intelligent and Innovative Support for Collaborative Learning Activities. [20] Tsiatsos, T., Konstantinidis, A., Ioannidis, L., & Tseloudi, C. (2009). Implementing collaborative e-learning techniques in collaborative virtual environments: The case of second life. Informatics, 2009. BCI '09. Fourth Balkan Conference in Title={Implementing Collaborative e-Learning Techniques in Collaborative Virtual Environments: The Case of Second Life, 181. [21] Consolvo, S., Everitt, K., Smith, I., Landay, J. A., Design Requirements for Technologies that Encourage Physical Activity, Computer-Human Interaction, April 22-27, 2006, Montreal, Quebec, Canada. [22] Oliveira, R. de, TripleBeat: Enhancing Exercise Performance with Persuasion, MobileHCI 2008, ACM, September 2-5, 2008, Amsterdam, Netherlands. [23] Firpo, D., Kasemvilas, S., Ractham, P., Zhang, X., Generating a Sense of Community in a Graduate Educational Setting through Persuasive Technology, Persuasive ’09, ACM, April 26-29, 2009, Claremont, California, USA. [24] source: Persuasive Technologies: Using Computers to Change What We Think and Do, pp 70-71, Morgan Kauffman [25] Lieberman, D. A., Three studies of an asthma education video game, in Report to NIH: National Institute of Allergy and Infectious Diseases, April, 1995. [26] Exploratorium Exhibit Services: HIV Roulette, website, retrieved September 25, 2010, http://exs.exploratorium.edu/exhibits/hiv-roulette/, 2006. [27] Hamman W. R., Beaubein J. M., Beaudin-Seiler B. M. Simulation for the Training of Human Performance and Technical Skills: The Intersection of How We Will Train Healthcare Professionals in the Future, Journal of Graduate Medical Education, December 2009. [28] Entin, E. E., Serfaty, D. Adaptive Team Coordination. Human Factors 41, 2 (1999), 312 – 325. [29] MacMillan, J., Entin E. E., Serfaty, D. Team Cognition: Understanding the factors that drive Process and Performance, 1st ed. APA, 2004, ch. Communication overhead: The hidden cost of team training [30] Cannon-Bowers, J. A., Salas, E., Converse, S. Individual and Group Decision Making: Current Issues, Erlbaum Associates, Shared mental models in expert team decision making, 221 – 246. [31] Toups Z. O., Kerne A., Hamilton W., Blevins A., Emergent Team Coordition : From Fire Emergency Response Practice to a Non-Mimetic Simulation Game. ACM GROUP, May 2009. 25 [32] Wayne, D. B. Butter, J. Siddall, V. J. Fudala, M. J., Mastery Learning of Advanced Cardiac Life Support Skills by Internal Medicine Residents Using Simulation Technology and Deliberate Practice. SGIM, September 30, 2005, Chicago, Illinois, USA. [33]Cardiopulmonary Resuscitation (CPR), website, retrieved May, 2010, http://www.americanheart.org/presenter.jhtml?identifier=4479, American Hearth Association. [34] Hamman, W. R., Beaubien, J. M., Beudin-Seiler, B. M., Simulation for the Training of Human Performance and Technical Skills: The Intersection of How We Will Train Health Care Professionals in the Future,J Grad Med Educ. 2009 December; 1(2): 245–252 26 APPENDICES: No Appendices. 27