See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/261298591

A Comparison of Usability Testing Methods for an E-Commerce Website: A

Case Study on a Malaysia Online Gift Shop

Conference Paper · April 2013

DOI: 10.1109/ITNG.2013.129

CITATIONS READS

39 1,316

6 authors, including:

Fong-Woon Lai A. Sivaji

Universiti Teknologi PETRONAS, Bandar Seri Iskandar, Perak, Malaysia Malaysian Institute of Microelectronic Systems
31 PUBLICATIONS   269 CITATIONS    44 PUBLICATIONS   350 CITATIONS   

SEE PROFILE SEE PROFILE

Shi-Tzuaan Soo
Malaysian Institute of Microelectronic Systems
17 PUBLICATIONS   175 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

behavioral culture product aesthetic View project

Cultural UX/Usability View project

All content following this page was uploaded by A. Sivaji on 05 June 2015.

The user has requested enhancement of the downloaded file.

 A COMPARISON OF MULTIPLE USABILITY TESTING METHODS TO
EVALUATE AND ANALYSE AN E-COMMERCE WEBSITE: A MALAYSIAN
CASE STUDY ON AN ONLINE GIFT SHOP

ABSTRACT computer applications affects how people interact with the

Customer’s perceived trust towards an e-commerce website website and also their perception towards the website. The
is crucial for the success of online business. Effective design overall goal of usability from a user perspective is to
of web interfaces increases perceived trust of customers. measure and improve effectiveness, efficiency and
Given many associated usability issues when performing satisfaction [2]. The usability of an e-commerce website is to
tasks on a website, it is important for technopreneurs provide users with satisfactory transaction effectively and
embarking on online business to understand issues related to efficiently. It helps to obtain a complete understanding of
usability problems of an e-commerce website and the
techniques to identify these issues. In this study, usability user’s needs and to improve product development in order to
evaluation was performed on an online gift shop with a provide a better user experience. Indeed, the usability of an
group of potential consumers with age range of 18-22. Four e-commerce website is of utmost importance as it will affect
different evaluation methods were used: Feedback Capture consumer’s trust towards the website and in turn their
after Task (FCAT), Retrospective Think Aloud (RTA), purchase intention [11]. Hence, the usability of an e-
Retrospective Think Aloud with Eye Movement (RTE) and commerce website will determine the success of the e-
observation. This study reveals that the major defects found commerce website. The purpose of this study is to outline
were language and content, user guidance and support,
flexibility and control, and visual clarity. Therefore, this the user interface defects in an online gift shop that
study suggest that applying good user interface design could contribute to trust and purchase intention on consumers in
provide better user experience and thus increase perceived Malaysia context. The study focuses on Malaysian age
trust and user satisfaction towards the website. between 18-22, the group that has the most experience to
KEYWORDS shop online.

Evaluation method, retrospective think aloud, observation, 2. Related Works

usability testing, eye tracking

Trust and satisfaction are the main determinants for

1. Introduction successful business relationships in business to consumer
electronic commerce [3]. Trust plays a vital role in any form
of business that requires monetary transaction. Customer’s
According to PayPal’s “Online and Mobile Shopping
satisfaction has a direct relationship to customer’s purchase
Insights” study [1], the size of Malaysia online shopping was
intention, repurchase intention and word of mouth marketing.
RM1.8 billion in 2010, estimated to be RM5 billion by 2014.
Consumer satisfaction is a measure of how well the product
Out of the RM14.8 billion, about RM825 million (45%) was
or services provided by companies meeting its consumer’s
spent on local business websites. This shows strong
expectations. A satisfied customer can expect higher
development and opportunity of e-commerce market in
purchase intention, repurchase intention and in effect
Malaysia. Majority of the Malaysians who purchased online
promote the company to other potential consumers. Hence,
falls into the age group between 21-40. In terms of products
there is clearly a need to study how an e-commerce website
bought online, travel products such as flight tickets has the
will influence consumer’s trust and satisfaction.
highest percentage which constitutes 24% of the total
product and services bought online, followed by bill
Usability of a website is critical in determining the
payment (18%), entertainment (14%) and IT & Electronics
success or failure of a company [4]. However, many e-
(12%). The less favorable products to buy online were gifts
commerce applications still do not meet customers’ usability
and collectibles. Hence our study involves evaluating and
requirements [5, 6]. Customers judge the credibility of a
proposing some guidelines for designing an online gift shop.
website within the first 50 milliseconds. Therefore, the
website needs to be design with customer’s behavior in mind.
The importance of usability testing to evaluate e-
In this line, usability is an essential component of e-
commerce has been well defined [5,11]. User interface of
commerce strategy [7, 8]. Considering the importance of users are observed and the entire conversation between the
usability on e-commerce website, there were many studies users and the moderator were recorded. Collected
conducted to understand how every aspect of an e-commerce observations and recorded audio files were used as inputs to
website can influence customer’s behavior. For example, further enhance the online gift shop website interface to
Papadopoulou and Pelet [9] studied how colors in an e- make it more user-friendly.
commerce website can affect consumer’s mood and
purchase intention. Work by Kamoun and Halaweh [10] 4. Results and Discussion
investigated HCI factors that contributes to customer’s
Table 1 list the task that needs to be completed by each user.
security perception towards an e-commerce website. Their
work has also re-emphasized on the importance of a good
Table 1
user interface design as an effective technique for increasing
Tasks and its description
customer’s trust and purchase intention. Sivaji et al. [11]
showed the importance of applying Gestalt Principle, Fitts’
Law and affordance across e-commerce websites. Since Tasks Description
these elements are frequently present in a conventional 1 Next Friday is your partner’s birthday; you wish to
shopping mall, it has been recommended to be virtualized buy him/her a small gift. You have a budget of RM30
for online transactions. The study also showed that once to buy a gift for your partner. What are the 2 gifts that
fundamental usability principles have been applied to the you think is most suitable for your partner?
website, other principles such as trust, social presence, 2 Read the description of each product.
online and offline communication elements needs to be 3 Find the product comparison tool and compare the
incorporated. price of ‘Love Letter Keychain’ and ‘3D Character
Keychain’.
Most previous study uses existing HCI evaluation method 4 You are a new user of to this website. Create a new
such as user testing, heuristic evaluation, inquiry methods account for this website.
and simulation methods to evaluate web applications. 5 You do not wish to buy anything today. You want to
Among all, user testing and heuristic evaluation method are log off the account.
most commonly used for e-commerce website [11,12]. To
From the usability testing conducted, four sets of
perform a more complete evaluation, most previous studies
qualitative data was obtained, namely textual feedback or
uses more than one evaluation method in evaluating their
also known as feedback capture after task (FCAT),
web application. For example, a combination of user testing
(think aloud protocol) and inquiry methods (interview, verbalization or also known as retrospective think aloud
questionnaire). These methods help in generating a list of (RTA), retrospective think aloud with eye movement (RTE)
usability problems; however it does not provide guidelines and observation. In the traditional think aloud method, also
on how to resolve these problems. Therefore, usability known as concurrent think aloud (CTA) [13, 14], a
evaluation must take into account discovering the usability participant would verbalize each action that they are
problems and repairing them [12]. As with regards to the performing at the point of time. However, it was found that
sample size for performing lab based usability testing, [24, cultural barriers make CTA method less suitable in countries
25] found that based on a binomial probability, six users will with high power distances per the Hofstede’s model [15]
be needed to discover between 85% - 90% of the problems, such as Malaysia. Sivaji and Ahmad [15] found that
given that the occurrence of the problem is 30%. Although although the subjects have been encouraged to think aloud,
this sample size does not represent the Malaysian they are reluctant as they are afraid that a failure of
demography, it is sufficient to detect significant usability completion of a given task would reflect poorly on their
problems. Another study by [26], also suggested 6 users if performance. This is despite the moderator briefing the user
the research method involves qualitative eye tracking such at the beginning of the task that the purpose of the usability
as analyzing the gaze replays as opposed to solely depending testing is to assess the web interface and not the user
on heat maps, which would require larger sample size. themselves. A detailed thorough study in regards to culture
and think aloud method was done by Clemmensen et. al.
3. Methodology [21]. Since the website was in English, language barrier also
exist [15].Thus, the traditional think aloud method will not
Six users were purposefully sampled from among the provide us with the necessary feedback, which is useful for
targeted web visitors to participate in the first user-based us to capture usability issues. Hence, [15] has proposed to
web usability testing at MIMOS Berhad UX Lab using the complement retrospective think aloud (RTA) methods with
Tobii T60 Eye Tracker, Tobii Studio and URANUS [16].All eye tracking analysis as it is able to reveal key biometric
of them were male; with age range from 18-22.They were information regardless of the cultural and language barriers.
chosen as users as the website we are testing is mainly In this situation, a combination of 3 methods has been
targeted for male users who are keen on purchasing gifts applied to capture defects or difficulties faced by the
online. The details of the procedure have been published in subjects to complete the tasks mentioned.
previous studies [11, 14]. The six users took turns to enter
the UX Lab, guided by a moderator, to complete a list of 4.1 Feedback Capture after Task (FCAT)
five outlined test cases (tasks). Each session lasted for
duration of about one hour. The entire tests for all the six This method is suitable, as it has been found that
Malaysian users prefer to type their thoughts instead of
verbalizing them due to the language and cultural barrier of FCAT RTA RTE
the users [15]. The URANUS system [16] used in this study 6 10 18
facilitates the end-to-end usability testing and FCAT. This is
achieved by prompting the user after the completion of each Now, we describe each defect in detail. Table 3 shows the
task to provide feedback on their experiences. FCAT defect detection capability of each method. A ‘Yes’ indicates
however does not involve playback of any videos but instead the method was able to detect the defects while a ‘No’
rely purely on their short-term memory of the experience. means otherwise.
There are high chances of users forgetting some of the issues
they have faced. In this circumstance, the role of the 5.2 Defects Detection by RTE Analysis
moderator is to remind them. However, the advantage of this Figures 1 to 12 shows the various visual cues obtained
method is that users will remember the issues that impacted using the eye tracker such as a gaze plot (Fig
them the most, hence it is expected that most of the 1,2,4,5,7,9a,9b,10,11) , gaze overlay (Fig 8a, 8b,12) and heat
important issues will be fed back. Since no video playback is map (Fig 3, 6) . RTE reveals difficulty in information
done, this method is also the fastest method to capture extraction by users as shown in Fig 9a, fixation 22 which
defects. The details of the defects captured using FCAT is lasted for a duration of 1.366 seconds. This is consistent
further described in Section 5.2. with previous studies [19] whereby long fixation duration
4.2 Retrospective Think Aloud (RTA) last more than 320ms. In this study, the long fixation is due
to the usage of highly saturated colors that increase cognitive
The second method, known as RTA, includes video load. RTE also reveals user interface that has less efficient
playback of the task performed by the user. Using this search as shown by the high number of saccades and fixation
method, users are prompted by the moderator to talk about pairs in Fig 1, 2, 4, 5 and 7, which is consistent with studies
what he or she has just done to complete the tasks, which [19, 22, 23]. The major defects experienced by users are
will include any additional comments the users might have. described in detailed in Table 3 (defect details column) and
This method is particularly useful for us as it has proven to correspondingly the RTE column make reference to visual
detect usability defects [14, 15] and it enables the users and cues in Fig 1 to 12.
moderator to observe and verbalize the activities that they
have performed. Similar to FCAT, during RTA users have a Table 3
tendency to forget their train of thought during performing Defect Detection Capability of FCAT, RTA and RTE
the task, which may lead to fewer, incomplete, or
reconstructed verbalizations of their thoughts [17]. User Defect Details FCAT RTA RTE
4.3 Retrospective Think Aloud with Eye Movements 6 It takes a user a No No Yes
(RTE) significant amount
The third method is similar to RTA but additionally of time (23 seconds)
during playback, the various eye tracking features such as to spot the “Gift”
gaze overlay, gaze plot and heat map is used to provide more label
insights visually. It is a combination of RTA with eye 6 During the 23 No No Yes
tracking. Also known as RTE or retrospective think aloud seconds of searching (Fig. 1)
with eye movements, a term coined by Elling et al. [17], it is for the “Gift” label,
expected that there are a considerable increase in visual cues
that may help the user’s memory, hence on the number and there was a
types of observation and verbalization. RTE involves significant amount
watching what each individual user does as they perform of cognitive
task and also listening to what they are saying as they think workload resulting
aloud. RTE provides the ability to follow user's gaze (gaze in 108 saccades
plot) around the screen and to review the recording in slow - 6 User unable to find No Yes Yes
motion to watch exactly where the user was looking [26].
pricing information (Fig. 2)
5. Analysis and Discussion from the “Gift” link.
After searching for
5.1Comparison of defect count by method
45 seconds, user
A comparison on the number (Table 2) and types of defect gave up
detected (Table 3) for each method was carried out. From 6 User misses gazing No No Yes
Table 2, it could be seen that using the RTE, 18 defects were
on pricing (Fig. 3,
observed and/or verbalized as compared to 10 and 6 defects
for the RTA and FCAT defects respectively. The findings information while 4)
from this study are similar to Ball et al. [18] whereby RTE zooming into the
offers promising results and more insightful information. image - Pricing
This however is in contrary to Elling et al. [17] findings information was not
whereby they found no additional value of showing users prominent as there
eye movements. Our findings however shed some light over was no fixation
Elling et al.’s [17] expectation of RTE whereby it improves
the interaction between subjects and moderator in terms of recorded both from
difficulties. heat map and gaze
plot
Table 2
Defect count for each method
6 When the zoom No Yes Yes 4 While searching for No No Yes
window is open, the (Fig. 3) product, fixation 8 (Fig. 10,
price information is shows that user was 11,12)
clipped off trying to perform
6 Insufficient product Yes Yes Yes some action on the
choices (Fig. 1) image for about 3
1 User was scanning No Yes Yes seconds. (2:17:894 –
2:25:309). The user
for more products
tried to click on the
1 User was scanning No No Yes View “Love Letter
within the same page (Fig. 5, Keychain” but there
for a period of 1 6) was no response.
minute and 21
Two reasons for this:
seconds with 250
1) The target size of
fixation and
the “View” label
saccades. User was
is too small
attempting to look
(Fitt’s Law)
for more product Yes
2) The performance
options, gave up at No No (Fig. 10,
of the system
the end and settled 11,12)
was slow
for 2 products within
Hence, the user
the budget as shown
starts to right click
in the main page
instead to try
2 Insufficient product Yes Yes Yes performing some
choices action. When this
2 User hopes to see all Yes Yes Yes did not work, the
products at once user went away to
2 User states inability Yes Yes Yes fixation 9, 10 and in
to sort product by fixation 11, the user
price range came back to the
2 User states that there Yes Yes Yes target (“View”) to
is no advance search try again for
option for user to 1.016seconds and
type budget and only at 2:28.551, the
display products system responded, so
according to budget there was a delay of
2 User states that there Yes Yes Yes about 7 seconds and
is no information the user had moved
that list the most on to fixation 38
purchased items 4 User is unsure the No Yes Yes
within the users difference between
budgets selecting the “View”
2 User faces difficulty No No Yes hyperlink and
performing task as it (Fig. 7) clicking on the
requires scanning for product label. There
1 minutes and 42 is poor visual clarity
seconds with over on what is the
336 gazes purpose of each
2 User’s cognitive No No Yes function. The visual
load increases when (Fig. 8a, for the product label
trying to understand 8b, 9a, is black in color as
the items in the sub- 9b) compared to the
menu. The high red view that is yellow
saturation used for in color. There exist
the menus slows consistency issues.
down reading as the
white fonts used are
not visually clear.
Figure 1. Cognitive workload – 108 fixation-saccades pair Figure 4. Gaze plot of user misses gazing on pricing

Figure 2. Missing pricing information in gift link Figure 5. Searching task producing 250 fixation within a
period of 1 minute and 21 seconds

Figure 3. Heat map of user misses gazing on pricing Figure 6. Searching task within a period of 1 minute and
information 21 seconds. The areas of high interest in the heat map could
have been filled up with more product choices
Figure 7.User faced difficulty performing task as requires Figure 9b. Fixation 25 at 1.748 second duration
scanning for 1 minute and 42 seconds with over 336 gazes

Figure 10. Fixation 8 and 11 with 3 mouse clicks

Figure 8a. Large gaze overlay on ‘Gift” menu

Figure 8b. Larger gaze overlay on ‘Gift” menu Figure 11.Fixation 8 duration 3.265 seconds

Figure 12. Gaze overlay with and right hand click action
Figure 9a.Fixation 22 at 1.366 second duration
5.3 Defect Categorization for FCAT found to compliment the retrospective think aloud conducted.
However, prompting by the moderator was necessary to
Based on FCAT method, We extracted the problems encourage them to comment in detail. Reminders of what
mentioned by users into categories. The qualitative feedback they said during the think aloud session helped them to
gathered from all six users broadly falls into eight categories: remember the problems encountered earlier and that will
1. Language and content, 2. User guidance and support, 3. trigger them to explain more.
Flexibility and control, 4. Visual clarity, 5. Consistency and
standard, 6. Navigation, 7. Functionality, 8. Informative
feedback. This paper would only highlight major problems
faced in some of these categories.
1) Language and content: 33% of the users raised the
issue that there were insufficient product choices. As the
items sold on this website were handmade and personalized,
mass production and variation of products is not feasible in
such a small-scale business. However, more products were
eventually added into the website to allow variations to
potential buyers.
2) User guidance and support: It was raised by 33% of the
participants that they found it confusing trying to sign up for
the website. In task 4, users were asked to create a new
account. Users did not expect this feature to be found in Figure 15. Overlapping problem faced by user
login (Figure 13). This comes to show that separate term
must be explicitly mentioned to show separate process to 5.3 Observations
sign up and login as shown in Figure 14. 1) Task 1: Most of the users did not realize or took some
time to realize the existence of the “Gift” menu in the top
navigation bar to start their product browsing. Four out of
six users suggested to have an additional product category
that holds all the products sold in order to allow users to
Figure 13. Login choose gift from all the products that are available, instead
of having to browse product category after product category
to see what gifts are available. Other suggestions included
enabling product filtering function and eliminating
malfunctioning links.

2) Task 2: Almost all users requested the fonts used to be

larger. Two users commented that the product descriptions
Figure 14. Login and Sign up could come with more practical details, like how long would
it take for the customers to receive the ordered products
3) Flexibility and control: In task 3, users were asked to
find the tool to compare the price between two products. 50% from the time the order is placed, as well practicality of
of the users mentioned that they were not able to compare some of the products that might be new and unfamiliar to
more than two products at a time. And another 50% were not users. Contrarily, other four users commented that the
able to find the comparison tool, thus were unable to existing product descriptions were sufficient to help them
complete the task. This comparison feature was taken off make purchase decision. One user mentioned that he would
because users mentioned the function is not too crucial and initiate to contact the web owner to clarify any doubt before
is more useful in comparing products like mobile phones.
placing order.
4) Visual clarity: Each product displayed on the website
has a description of the product and its features. 33% 3) Task 3: Three users skipped this task case due to the
mentioned that the font size used for its product description
difficulty in locating the product compare button, while the
and features were not legible enough to read. Font size was
increased to ensure legible reading for users. As mentioned, last user did not participate in accomplishing test cases 3
it was also noted by 50% of the users that it was very through 5 due to the unexpected downtime of the web
difficult to find the comparison tool, thus they were not able hosting server. For the two users who managed to
to complete task 3. 50% of the users also mentioned that accomplish this task, they both commented that the
some products were not visible due to overlapping product overlapping interface, which occurred when comparing three
description while doing task 3 as seen in Figure 15. This was products, distracted their browsing experience. Another
a technical issue that was not expected prior to the test. For
other categories not mentioned specifically, problems raised suggestion was to allow users to add the product into
by the users were minor and only affects a small percentage comparison basket right from the product page itself instead
(16.67% or one user). Some examples would be no of only enabling the comparison button in product category
welcoming note when the user signs into the website or was page.
not able to zoom into the product picture. Minor issues were
easily fixed based on participants’ comments and 4) Task 4: Among the five users, all of them were able to
recommendation. After users completed each task, they were
prompted to type in some comments. Comments typed were locate the log in button very quickly but two were reluctant
to create an account as requested by the task due to privacy
concern of personal information. However, all five users [9] Papadopoulou, P. and Pelet, J. (2009).The effect of colors of e-
commerce websites on consumer mood, memorization and buying intention,
created their respective accounts in order to accomplish this MCIS 2009 Proceedings. Paper 116.
task. Suggestions received included adding tooltips on [10] Kamoun, F. and Halaweh, M. (2012). User Interface Design and E-
required fields on the account registration form which are Commerce Security Perception: An Empirical Study, E-business Research,
8(2).
unfamiliar to users, eliminating unnecessary fields required [11] Sivaji, A., Mazlan, M. F., Shi–Tzuaan, S., Abdullah, A., & Downe, A.
in order to make purchase, displaying welcoming message to G. (2011).Importance of Incorporating Fundamental Usability with Social,
explicitly inform users that they have successfully logged in Trust & Persuasive Elements for E-Commerce Website, IEEEXplore
and allowing registered users to personalize the website Business, Engineering and Industrial Applications (ICBEIA), pp. 221-226,
Kuala Lumpur, Malaysia
theme color according to their preference. Besides, the [12] Fernandez, A., Insfran, E. and Abrahao, S. (2011).Usability evaluation
position of the displayed user’s name after logging in methods for the web: A systematic mapping study, Information and
overlapped other web content due to long user name. Software Technology, 53, 789-81.
[13] Benjamin Yu and Kimberly Voll. 2011. Probing student problem
solving skills in mathematical induction using a scenario based think aloud
5) Task 5: All five users who did this test case managed to protocol. In Proceedings of the 16th Western Canadian Conference on
locate the log out button very quickly and they could log out Computing Education (WCCCE '11). ACM, New York, NY, USA, 33-37.
[14] M.J. van den Haak, M. D. T. DeJong and P J Schellens, “Retrospective
successfully. One user suggested adding a confirmation vs. concurrent think-aloud protocols: testing the usability of an online
dialogue box to make sure that the user really wants to log library catalogue”. Behaviour & Information Technology, vol22, no. 5,339-
out every time after the log out button is clicked. 351, 8 Nov 2010
[15] Sivaji, A.,Ahmad,W. Benefits of complementing Eye Tracking
analysis with Think Aloud protocol in a multilingual country with high
From RTA and observational method, it was found that power distance (Hofstede’s model) for website user experience testing. In
the there were no significant differences between the two press Current Themes in Eye Tracking Research: Method and Application.
Eye Track Australia 2012 conference
method used, even as reported by [20]. [16] Sivaji, A. S.Shi-Tzuaan, “Website User Experience (UX) Testing Tool
Development using Open Source Software (OSS)” in press IEEEXplore,
2nd South East Asian Network of Ergonomics Societies (SEANES),
6. Conclusion Malaysia, 2012
From this study, it was found that by using FCAT, RTE and [17]Sanne Elling, Leo Lentz, and Menno de Jong. 2011. Retrospective
RTA method, a significant number of defects were detected think-aloud method: using eye movements as an extra cue for participants'
verbalizations. In Proceedings of the 2011 annual conference on Human
through this test. The methods used complemented each factors in computing systems (CHI '11). ACM, New York, NY, USA, 1161-
other in finding defects, while some methods were easier to 1170.DOI=10.1145/1978942.1979116
identify than others or revealed more. However, for RTE [18]Ball, L.J., Eger, N., Stevens, R., Dodd, J. Applying the PEEP method
in usability testing. Interfaces, 67 (2006), 15-19
method, we manage to find 3 times more defects as [19] Claudia Ehmke and Stephanie Wilson. 2007. Identifying web usability
compared to RTA. RTE reveals more data than we expected problems from eye-tracking data. In Proceedings of the 21st British HCI
with the usage of eye-tracking device. Since data through Group Annual Conference on People and Computers: HCI...but not as we
know it - Volume 1 (BCS-HCI '07), Vol. 1. British Computer Society,
this method were abundant, proper analysis is necessary to Swinton, UK, UK, 119-128.
identify and extract only relevant data. Eye tracking [20] Ester Baauw and Panos Markopoulous. 2004. A comparison of think
(cognitive load) method reveals lots of information, thus aloud and post-task interview for usability testing with children. In
Proceedings of the 2004 conference on Interaction design and children:
further understanding is necessary to comprehend cognitive building a community (IDC '04). ACM, New York, NY, USA, 115-116.
processing and how this method could be better exploited to [21] Torkil Clemmensen, Morten Hertzum, Kasper Hornbaek, Qingzin Shi,
reveal identification of more defects and its accuracy. Pradeep Yammiyavar. 2008. Cultural Cognition in the Thinking-Aloud
Method for Usability Evaluation.Journal Interacting with Computers, 21 (3)
212-22
References [22] Goldberg, J. H., and Kotval, X. P. 1999. Computer interface evaluation
using eye movements: Methods and constructs, International Journal of
Industrial Ergonomics, 24, 6,631-645
[1] PayPal: Online Shopping Hits 1.8 Billion. (2011). Retrieved from [23] Poole, A., and Ball, L. J. 2005 Eye tracking in human-computer
http://marketing-interactive.com/news/25946 interaction and usability research. In C. Ghaoui (ed.), Encyclopedia of
[2] Bevan, N. (2008). Classifying and selecting UX and usability measures, human computer interaction. Idea Group, Pennsylvania, 211-219
Proceedings of Meaningful Measures: Valid Useful User Experience [24] Sauro, J. Why you only need to test with five users (explained) (2010)
Measurement (VUUM), 5th COST294-MAUSE Open Workshop, 18th June Retrieved from http://www.measuringusability.com/five-users.php
2008, Reykjavik, Iceland [25] Sivaji, A., Mohd Nor, Z., & Downe, A. G. (2011). Comparison of the
[3] Kim, D.J., Ferrin, D. L. and Rao, H. R. (2009).Trust and Satisfaction, Lab Based Usability Testing and Exploratory Heuristic Evaluation: A Case
Two Stepping Stones for Successful E-Commerce Relationships: A Study on Human Factor Conf. Website Usability. Malaysian J. of
Longitudinal Exploration, Journal Information Systems Research, 20(2), pp. Ergonomics , 57-71.
237-257. [26] Pernice, K, Nielsem,J, (2009). How to conduct eye tracking studies,
[4] Tezza, R., Bornia, A.C. and Andrade, D. F. (2011). Measuring web 20,51 Retrieved from
usability using item response theory: Principles, features and opportunities, http://nngroup.com.s3.amazonaws.com/media/reports/free/How_to_Conduc
Journal Interacting with Computers, 23(2), pp. 167-175. t_Eyetracking_Studies.pdf
[5] Li, F. and Li, Y. (2011). Usability evaluation of e-commerce on B2C
websites in China, Procedia Engineering, 15, 5299-5304.
[6] J. Offutt, Quality attributes of Web software applications, in: IEEE
Software:Special Issue on Software Engineering of Internet Software, 2002,
pp. 25–32.
[7] Oneupweb. (2010). Revolutionizing Website Design. Retrieved from:
http://www.oneupweb.com
[8] Shi, M. (2011). Website Characteristics And Their Influences: A
Review On Web Design, Association of Business Information Systems
(ABIS) Refereed Proceedings. Retrieved from http://www.abis-fbd.org/wp-
content/uploads/2011/03/ABIS_2011_Proceedings.pdf#page=147

View publication stats