See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/316476424

Usability Evaluation of a Virtual Museum Environment: A Case Study in

Terengganu State Museum, Malaysia

Article  in  Journal of Computational and Theoretical Nanoscience · October 2016

DOI: 10.1166/asl.2016.7109

CITATIONS READS

5 322

5 authors, including:

Normala Rahim Tengku Siti Meriam Tengku Wook Tengku Wook

Universiti Sultan Zainal Abidin Universiti Kebangsaan Malaysia
14 PUBLICATIONS   49 CITATIONS    99 PUBLICATIONS   628 CITATIONS   

SEE PROFILE SEE PROFILE

Nor Azan Mat Zin Norkhairani abdul rawi

Universiti Kebangsaan Malaysia Universiti Sultan Zainal Abidin | UniSZA
139 PUBLICATIONS   1,028 CITATIONS    19 PUBLICATIONS   32 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Sketch Technique View project

Virtual Museum View project

All content following this page was uploaded by Tengku Siti Meriam Tengku Wook Tengku Wook on 02 April 2018.

The user has requested enhancement of the downloaded file.

 RESEARCH ARTICLE Adv. Sci. Lett. 22, 2780–2784, 2016

Copyright © 2016 American Scientific PublishersAdvanced Science Letters

All rights reserved Vol. 22
Printed in the United States of America

Usability Evaluation of Virtual Museum

Environment: A Case Study in Terengganu State
Museum, Malaysia
Normala Rahim1,TengkuSitiMeriamTengku Wook2Nor Azan MatZin2NorkhairaniAbd. Rawi1Roslinda Muda1
1
Faculty of Informatics & Computing, Universiti Sultan ZainalAbidin, Terengganu, Malaysia
2
Faculty of Information Science & Technology, The National University of Malaysia, Selangor, Malaysia

This paper conducted the usability evaluation of Virtual Museum Environment (VME) for Terengganu State Museum. The
model that has been applied in this research was based on User-Centered Design (UCD) involving many users in every stage.
User testing was performed online using questionnaire with a total of 52 respondents. The usability evaluation measured
three factors that include navigation, manipulation and system control. Three usability elements tested in this study were:
ease of use, efficiency and users’ satisfaction. Overall, the mean score obtained for each element that is over the scale of 4.00
indicates high usability of the VME prototype.

Keywords: Virtual Environment, Usability evaluation, User Centered Design, Virtual Museum .

interactions that occur in the virtual museum still at a

1. INTRODUCTION low level3,4. The main problem in user interaction is
virtual environment control and limited object
A study was conducted in the field of virtual manipulation in a 360 degree technique. On top of that,
reality that focused on the Image-Based Modeling the interface design of a virtual museum application
approach. The field of Virtual Reality (VR) has been does not have the elements and principles of design.
chosen due to the lack of implementation of VR Most museums having problems in introducing their
application in museum institutions compared to other visitors to the actual wealth of their exhibits, where most
domains1. Currently, only two VR applications for of the artifacts are not accessible by the visitors5. Zara
museum in Malaysia have been introduced. However, (2003) stated that museum exhibitions provided by the
the applications did not integrate the whole functional museum institutions still need to be strengthened due to
values of Virtual Museum. Besides, usability evaluation low acceptance by most visitors6. Thus, the institution of
is still limited and not seriously considered2. Therefore, the museum should have new initiatives to present a
there is a call to design and develop a comprehensive creative and innovative exhibition to attract more users
virtual museum prototype and Terengganu State to explore the treasures left by the ancestors. Norazimah
Museum has been selected in this study. User interaction et al. (2009) found that 74.1% of museum visitors would
issues in designing Virtual Museum Environment like to view an interactive based exhibition but there are
(VME) and usability issues were the main focus in this many museums that still use traditional techniques in
study. presenting their exhibitions7. Virtual museum also can
Based on an initial investigation, a number of attract people to visit the physical museum8.
problems in VME have been identified especially from In order to increase the usability of the VME,
the aspects of user interaction. Nowadays, user this study has been conducted by involving the potential
users during the evaluation of the application via the
*
Email Address: [email protected] usability study. The involvement of potential users is a

1
 Adv. Sci. Lett. 22, 2780–2784, 2016 RESEARCH ARTICLE

crucial part to acquire the exact requirements for the Terengganu of which take into consideration the user
development of the prototype. Usability is an important interactions. The two phases involved are:
aspect to ensure that the VME prototype can be fully
utilized by the users9. Thus, an evaluation towards VME a) Implementation of user
has been conducted to identify the usability level. b) Data collection
The structure of this paper is as follows: Section
1 gives a brief background of the study. Section 2
highlights the methodology used in the study. Results USER
and findings of the study were elaborated in Section 3 IMPLEMENTATION
followed by the conclusions in Section 4.
Online Invitation
2. USABILITY EVALUATION METHOD

The objective of the usability evaluation was to get DATA

necessary feedback about VME prototype based on COLLECTION
respondents’ experience and feedback. Therefore, a
preliminary analysis was conducted on existing virtual Questionnaire
museum applications namely Virtual Museum Sarawak
and Virtual Museum Malacca to identify research
problems. Usability evaluation is very important to USABILITY LEVEL
identify user requirements and propose suitable solutions. OF VIRTUAL
However, only Virtual Museum Sarawak has been MUSEUM
chosen by authors to be compared because of the ENVIRONMENT
similarity of the system with this research. The result
from this study is shown in Figure 1. Fig.3. Method of Usability Evaluation of Virtual
Museum Environment

In this research, questionnaires technique has been

used to collect data. Usability questionnaires are good
for quickly collecting large sets of data either in a
laboratory setting or “on the fields”10. The respondents
for this survey were recruited randomly online through
social media such as Facebook, email and messenger.
They were given a link of the VME system for them to
explore it about 15 to 20 minutes. After that, they need
to answer the questionnaire based on their experience.
Two divisions of phases were carried out to obtain the
Fig.1. Usability Factor of Virtual Museum Sarawak usability results of VME.

Figure 2 shows the variables of VME which A. Implementation of User

incorporated elements of user interaction that includes
navigation, manipulation, and system control which are User testing conducted through online facilities
the main focus in this study. The usability factors for where the VME prototype was uploaded into server at
each element of the user interaction are considered. Faculty of Information Science Technology (FTSM)
mars.ftsm.ukm.my/virtualmuseumtrg/pmm.html.

Implementation of user testing procedure was as

follows:
1) Online Invitation
Send the URL via email, yahoo messenger and
Facebook to 52 respondents randomly. The instruction
given was that they should use VME prototype within
20 minutes to 30 minutes.
Fig.2. Independent variables of VME
2) User Feedback
Figure 3 show the method employed in the Respondents have read the instructions and answer
evaluation of a Virtual Museum Environment
2
 RESEARCH ARTICLE Adv. Sci. Lett. 22, 2780–2784, 2016

the questionnaire by selecting a scale of 1 to 5, which when the result exceed 0.704,12. Table 4 shows the
has been declared. Cronbach’sAlpha reliability test for the questionnaire’s
items in VME prototype usability study. The results
3) Submission were analyzed using SPSS where the mean score
Respondents only need to click on the submit obtained was used to measure the level of each
button. independent variable of the test.

B. Data Collection via Questionnaire Table.4. Cronbach Alpha reliability test results of the
questionnaire in VME prototype usability study
52 respondents participated in this study that Questionnaires Part Alpha
consists of novice and expert level users. They were Cronbach Value
given a certain period of time to surf the VME. Easy to use factor 0.860
Respondents were free to undertake the task of search
Efficiency factor 0.702
and navigation of bibliographic information according to
their needs. Satisfaction factor 0.725
For this study, scale based has been chosen as the
evaluation approach due to establishments of this type of B. Results of the Usability Evaluation of VME
questionnaires in evaluating usability aspects. Likert
scale with the range from 1 to 5 was adapted in the Results of the usability evaluation of VME
questions. According to Nielson (1993), criteria for the prototype is shown in Table 5. Mean scores for the easy
subject to be measured must show the average value of to use, efficiency and satisfaction were 4.08, 4.27 and
at least 4 on a Likert scale of 1 to 511. This means at least 4.12 respectively. The results indicate that the mean
50% of users chose a scale of four or five and not more score of VME prototypes usability were over the scale
than 5% who chose a scale of one. of four. This is aligned with Nielson (1993)that the
Questionnaire was used to collect data from criteria for the subject to be measured must show the
respondents regarding the usability of the VME average value of at least 4 on a Likert scale of 1 to 511.
prototype. The questionnaire was constructed using Based on the Table 5, the difference in mean scores
google doc and consists of two parts, demographic for each of the three factors of usability in VME
information of the users and usability prototype. prototypes can be compared. Since the mean score is
Demography Division consists of 5 questions that above the scale of four, it is also proven that usability
include age, occupation, level of education, experience level of VME prototypes is high. The next part will
and frequency of computer use on the Internet. The discuss the result for each usability factors in detail.
second part comprises 30 questions that cover the
usability factors. Table.5. VME Prototype Usability Scores
A pilot study was conducted to test the reliability of USABILITY OF VME PROTOTYPE
the instrument, which was built on the three dependent
variables consist of the ease of use, efficiency and Easy
Efficiency Satisfaction
satisfaction. A pilot study can help researchers detect to use
Min Min Min
problems that may arise before the item actual survey ITEM
Score
ITEM
Score
ITEM
Score
conducted. A pilot study was conducted with 52 Q1 4.35 Q11 4.29 Q21 4.00
respondents to determine the reliability of the instrument Q2 4.04 Q12 4.12 Q22 4.12
based on the Cronbach’s Alpha coefficient.
Q3 4.02 Q13 4.23 Q23 4.17
Q4 4.02 Q14 4.17 Q24 4.04

3.RESULTS AND FINDINGS Q5 4.21 Q15 4.19 Q25 4.08

Q6 4.00 Q16 4.38 Q26 4.27
This section illustrates the result of two which are Q7 4.10 Q17 4.29 Q27 4.21
pilot test and final testing of usability evaluation. Q8 4.02 Q18 4.23 Q28 4.13
Q9 4.02 Q19 4.42 Q29 4.02
A. A pilot test of reliability using Cronbach’s Alpha
Q10 4.00 Q20 4.37 Q30 4.08
coefficient
MIN 4.08 MIN 4.27 MIN 4.12

Analysis of the reliability of the questionnaire was 1) Ease to Use

measured using Cronbach's Alpha coefficient shows that Easy to use means that the every function in system
all the items of the questionnaire can be trusted. These should be easy to learn and easy to use for the users.
results indicate that all the items in the questionnaire The ease of use aspect was correlated to functions
were consistent and reliable. According to Pallant (2001) provided in the application, where in in this case, the
and De Vellis (2005), reliability coefficient is achieved
3
 Adv. Sci. Lett. 22, 2780–2784, 2016 RESEARCH ARTICLE

navigation, manipulation and system control. Q1, Q3

and Q10 focused on the navigation functions. Each of
this question scores 4.02, 4.00 and 4.35 respectively.
The results indicate that the navigation function of VME
prototypes is easy to use due to the results score that
were above 4.0013.
While Q4, Q5 and Q9 were purposely used to
acquire feedback in the ease of use aspects in terms of
manipulation where 4.02, 4.21 and 4.02 were the scores
for each of them. Again, it supports the easy to use
factors in term of manipulation. Meanwhile, Q2, Q6, Q7
and Q8 targeted on system control aspects. It was shown
that system control of VME prototype is easy to use. It is
proven through the results of the score that are 4.04,
4.00, 4.10 and 4.02 respectively. Summary of the results
Fig 5. Mean Score for Efficiency
is represented in Figure 4 where the mean score for each
item from the perspective of easy to use were indicated.
3) Satisfaction
Satisfaction is another point on usability evaluation.
For this point, 10 questions were used to investigate the
satisfaction level from user perspectives towards
navigation, manipulation and system control of the VME
prototype. Q23, Q29 and Q30 were dedicated to evaluate
user satisfaction on navigation function. The scores
obtained were 4.17, 4.02 and 4.08 thus confirming that
users were satisfied with the navigation provided.
Satisfaction on manipulation in VME prototype
was derived from Q21, Q24, Q22 and Q28. For this
group of questions, results of 4.00, 4.04, 4.12 and 4.13
were obtained indicating that the satisfaction in terms of
manipulation in the VME was fulfilled. Finally, Q25,
Q26 and Q27 were used to identify users’ satisfaction on
system control in the VME prototype. The result of 4.08,
Fig 4. Mean Score for Easy to Use 4.27 and 4.21 respectively for each of the question
verified that the users were satisfied with system control
in the VME prototype. Summary of the results is shown
2) Efficiency in Figure 6.
Figure 5 summarized the results for efficiency
factors. Efficiency refers to how the system should work
efficiently when used so the user does not have to use
the system in a long time and they need not re-learn the
system if they want to use it again. The efficiency
element from the aspects of navigation was investigated
through Q13, Q14, Q15 and Q19. Efficient navigation of
the VME was demonstrated by the scores of 4.23, 4.17,
4.19 and 4.42 respectively.
For Q11, Q16 and Q20 the score for each of them
were 4.29, 4.38 and 4.37. This group of question was
used to reveal efficiency level of manipulation in VME.
Results extracted from the test confirmed that
manipulation of VME is efficient. Similarly, the score
of 4.29, 4.12 and 4.23 for Q17, Q12, and Q18 revealed
that the system control elements were also efficient. In
conclusion, the overall results is from the questionnaire Fig 6. Mean Score for Satisfaction
is promising where it validated the efficiency of system
control in the VME prototype.

4
 RESEARCH ARTICLE Adv. Sci. Lett. 22, 2780–2784, 2016

4) Overall Usability Performance of VME

The overall results of the usability evaluation on
VME prototype is shown in Figure 7. The overall scores REFERENCES
of 4.08, 4.27 and 4.12 respectively for usability factors
that comprise ease of use, efficiency and satisfaction. [1] Lee, S. H. (1999). Usability testing for developing effective
From these results clearly demonstrated that the interactive multimedia software:Concepts, dimensions, and
procedures. Educational Technology & Society, 2 (2).
usability of the VME prototype has been fulfilled where [2] Pallant, J. 2001. SPSS survival Manual. A Step By step Guide
all scores are above the scale of four. to Data Analysis Using SPSS for Windows (Version 10). NSW:
Allen&Unwin.
[3] Kaur, K., SutcIiffe, A. & Maiden, N. 1998. Improving
Overall Mean Score on Usability Evalua7on Interaction with Virtual Environments.
[4] Nielsen, J. 2003. Usability 101:Introduction to usability.
5 http://www.useit.com/alertbox/20030825.html [26Mei2011].
4.08 4.27 4.12 [5] Fevgas, A.; Fraggogiannis, N.; Tsompanopoulou, P.; Bozanis,
4.5 P., "The iMuse Virtual Museum: towards a cultural education
4 platform," in Information, Intelligence, Systems and
Likert Scale

3.5 Applications, IISA 2014, The 5th International Conference on ,

Easy to use vol., no., pp.171-175, 7-9 July 2014
3
[6] Zara, J. 2004. Virtual Reality and Cultural Heritage on the Web.
2.5 Efficiency Proceedings of the 7th International Conference on Computer
2 Graphics and Artificial Intelligence (3IA 2004) Limoges,
Sa=sfac=on
1.5 France.101-112.
1 [7] NorazimahAwang, Abdul RazakYaakub&Zakirah Othman.
2009. Assessing User Acceptance towards Virtual Museum: The
Usability factors Case in Kedah State Museum, Malaysia. Sixth International
Conference on Computer Graphics, Imaging and Visualization.
Fig 7. Overall Mean Score on Usability of the VME [8] Rizvic, Selma; Pletinckx, Daniel; Okanovic, Vensada,
"Enhancing museum exhibitions with interactive digital content:
Prototype Sarajevo city model interactive," in Information,
Communication and Automation Technologies (ICAT), 2015
XXV International Conference on , vol., no., pp.1-5, 29-31 Oct.
4. CONCLUSIONS 2015.
[9] Petridis, P., White, M., Mourkousis, N., Liarokapis, F.,
Sifniotis, M., Basu, A. &Gatzidis, C. 2005. Exploring and
ICT growth has contributed to the expansion of its Interacting with Virtual Museums
utilization in most area. Virtual applications that exploit [10] Pescarin, S.; Wallergird, M.; Hupperetz, W.; Pagano, A.; Ray,
the ICT have been recognized as an interesting and C., "Archeovirtual 2011: An evaluation approach to virtual
convenient information delivery method. Museum has museums," in Virtual Systems and Multimedia (VSMM), 2012
been chosen as domain to get the advantages of virtual 18th International Conference on , vol., no., pp.25-32, 2-5 Sept.
environment. The findings of user requirements through 2012.
questionnaires technique enhanced the need of user [11] Nielsen, J. 1993. Usability engineering. Mountain View,
interactions and feedback in developing virtual museum. California.Nielsen, J. (1993). Usability Engineering. Academic
Press, Boston, ISBN 0-12-518405-0 (hardcover), 0-12-518406-9
Therefore, a VME prototype using image-based model (softcover). Japanese translation ISBN 4-8101-9009-9.
has been developed to fulfill user requirements. In order [12] DeVellis, R.F. 2003. Scale development: Theory and
to assess the usefulness of the VME, a usability applications. 2nd edition. Thousand Oaks, CA: Sage.
evaluation has been conducted to collect data about the [13] Normala Rahim, TengkuSitiMeriamTengkuWook, and Nor
usability of the VME design by randomly selected Azan Mat Zin. 2011. Developing conceptual model of virtual
respondents. Results obtained from the usability test in museum environment based on user interaction issues. In
Proceedings of the Second international conference on Visual
the aspects of navigation, manipulation and system
informatics: sustaining research and innovations - Volume Part
control depicted that the VME prototype has fulfilled the II (IVIC'11), HalimahBadiozeZaman, Peter Robinson, Maria
three main elements of usability that are: easy to use, Petrou, Patrick Olivier, and Timothy K. Shih (Eds.), Vol. Part
efficiency and satisfaction. In such, the prototype could II. Springer-Verlag, Berlin, Heidelberg, 253-260.
further be enhanced by incorporating other aspects such
as collaborative features, embedded with the augmented
technology and etc.

ACKNOWLEDGMENTS

This research has been supported by the Ministry of

Education of Malaysia under SLAB/SLAI scheme and
Universiti Sultan ZainalAbidin.

5
 Adv. Sci. Lett. 22, 2780–2784, 2016 RESEARCH ARTICLE

View publication stats