(c

University of Sindh Journal of Information and Communication Technology

(USJICT)
Volume 5, Issue 4, December 2021
ISSN-E: 2523-1235, ISSN-P: 2521-5582 © Published by University of Sindh, Jamshoro
Website: http://sujo.usindh.edu.pk/index.php/USJICT/

A Usability Evaluation of M-commerce Apps

Sehrish Aqeel
University of Malaysia Sarawak
[email protected]

Abstract: In today’s era, Internet has become an increasingly popular technology globally. Huge numbers of
websites and applications have been developed and majority of them are selling products over the internet. Even
with the availability of a large number of M-commerce applications, still the target audiences of these M-commerce
applications such as students, office employees, housewives, who need a mobile application to do their online
shopping, are not using them efficiently due to usability issues encountered at user interfaces of m-commerce
applications. This is because of poor interface design along with neglected usability guidelines that are provided by
various researchers and essentially followed by standardize companies like Apple and Google. Therefore, it is
highly recommended that the interface of such a system should be explored through different stakeholders by
considering numerous parameters and criteria. To tackle UX/UI problems well, this study is conducted to aim the
usability improvement of m-commerce applications and for the same, two popular applications, Daraz and Elo, is
selected by considering current download rate and positive reviews. They are considered as top ranked m-
commerce applications, working successfully in Pakistan for the year 2020. However, some usability problems
were noticed for these Apps and were reported in the reviews of the application too. So, these problems are
addressed and targeted to filter out the most significant issues in usability. The methodology used for evaluation
and analysis of the currently available m-commerce applications was heuristic evaluation (from expert users) and
user testing (from novice users) based upon the integrated usability guidelines proposed by different usability
experts. The research finding concluded the three significant criteria which are error prevention, aesthetic and
minimalist design, and help & documentation. Further, different decision making techniques will be applied to
attain user satisfied m-commerce platforms through prototype design in near future.

Keywords: Usability; UX/UI; User Testing; Novice Users;

Usability relies on multiple attributes to make the system

I. INTRODUCTION more useful. There are different ways to measure these
Online Shopping is significantly popular phenomenon usability attributes. HCI provides numerous methods to
globally where retailers sell their products on the internet. evaluate usability issues in the user interface of the system
Due to enhancement in information technology and such as user-based, Experts based, and tool base methods[4].
economic globalization, e-shopping has become an emerging Mobile users are increasing very rapidly and statistics of
trend in business development in local as well as foreign mobile usage indicated that in 2020 mobile users will reach
trade. Electronic commerce (e-commerce) is rapidly 2.87 billion users annually [4]. Mobile commerce transactions
flourishing day by day [1]. In daily life, we mostly depend on will also reach 1 billion in the end of year 2020. End-users use
recommendations from friends for reading books, novels, and mobile devices for different purposes like marketing, banking,
watching movies. People like to buy products online through ticketing, social activities, and especially for online shopping
friend suggestions by using various websites and applications [5].
[2]. Such platforms uses standardize system as an additional M-commerce is rapidly becoming a new form of trade.
tool to increase sales and also satisfy the customers as per Mostly, customers view the product price, quality features,
their needs. A recommender system is a web technology and specifications on m-commerce platforms before buying
which helps the customers for online shopping with the good a product. E-commerce transactions have reached 3.53
decision in m-commerce platforms. Many authors defined e- trillion U.S dollars worldwide. E-commerce sales revenues
commerce in different ways. Electronic commerce is an are estimated to grow by up to 6.54 trillion U.S dollars in
emerging concept in which buying and selling products is 2022 [6]. According to a global digital report, there are 4.39
done online. There are many types of e-commerce such as billion internet users in 2019. E-commerce usability is much
business to business (B2B), business to customers(B2C), important because unusable websites and apps will lead
customer to customer(C2C) [3]. Human-Computer buyers to leave these websites and apps and eventually loss
interaction (HCI) is a field that enhances the interaction of the e-business. Nielsen, who is the founder of usability,
between human and computer through usability guidelines. stated that 40% of repeat visits of customers don’t visit it with
HCI makes the system easy to use, learn and understand. please who initially had bad experience with the sites and
 University of Sindh Journal of Information and Communication Technology (USJICT) Vol.5(4), pg.: 184-207

apps. Research estimates that bad web design will result in an popular all over the world nowadays. A lot of websites and
approximate loss of 50 percent of potential sales from apps have been built and used for buying and selling products
customers who cannot find exactly for what they want on the over the internet like amazon.cm, alibaba.com, and many
sites or apps [7]. more. M-commerce is a quickly emerging field as compare to
From the customer’s point of view, an M-commerce e-commerce [10].
platform also depends on usability factors which are beyond Many authors have defined usability in many ways.
the accuracy of the algorithm or the software. It also depends Usability is the name of ease of use and learnability of
on how the users interact with it effectively. The finding of products. Usability means how quickly users use a product
this study will contribute towards e buying and selling for and learn it the first time. International organization for
both e-commerce retailers and customers. For customers, the standardization (ISO) defined usability as: “The degree to
benefits include higher efficiency in finding wanted items, which any software product can be used for achieving
more confidence in making a purchase decision, and the specific goals by specific users in the specific context of use
potential to discover something new. For e-commerce with efficiency, effectiveness, and satisfaction. ISO gives
retailers, this technology can significantly enhance the three attributes for usability [11].
overall revenue.
The research and design community related to these m- • Efficiency: How quickly the user can perform your
commerce apps must take into consideration some guidelines task using the product.
in an array to craft technology accessible to the all ages, • Effectiveness: How quickly the user completes
despite of any gender and race. By captivating this perspective your tasks with less error action.
in mind, this study is conducted to discover that whether m- • Satisfaction: How many users enjoy/ pleasant with
commerce apps are accessible to the general public, office
the product when it is used.
workers and IT specialists to fulfill their needs or not. M-
commerce applications are used by customers for buying and
selling products over the internet. They faced UX/UI issues Condos proposed usability factors such as navigation,
when used on personal cell phones. Cell phones have inherited Contents, information Architecture, Error prevention, and
limitations like small screen size, limited input mechanism, Menu Visualization. Quim Model also proposed usability
display resolution, etc. Also, they have huge variations in factors as Trustfulness, Accessibility, and Usefulness.
different brands and all have provided different usability Nielsen is called the father of usability. Nielsen describes the
guidelines to their developers for making cell phone five usability attributes [12] which are:
applications. According to the end-users point of view, the
design of the interface of the mobile application is one of the • Learnability: How easily users learn product tasks
most essential components. It is a place that the main when it’s used for the first time to perform any task.
interaction with the application will occur. In a broad sense, • Memorability: How much difficult for users is to
the problem can be described that how to increase usability for again perform the same task after a period of time
an m-commerce applications interface for cell phone users after leaving the system.
based on usability guidelines [8]. A large number of m- • Efficiency: How quickly the user can perform a task
commerce applications are lacking usability guidelines as
while using the system.
provided by ISO and other usability experts like Jacob Neilson
and others. There is a need to redesign these application’s • Error rate: What is the error rate ratio of the
interface to propose a new design which is based upon product? What is the error condition?
integrated heuristic guidelines provided by different mobile • Satisfaction: How much user is pleasant when he is
brands and usability experts [9]. To achieve this objective, it using the design?
is important to assess the usability problems of the m-
commerce application's user interface by selecting any Some of the usability models are mentioned in Table 1 that
prioritizing technique which is previously used efficiently and are used to make the product more usable. ISO 9241-11
proved its successfulness in similar domain of research described usability in terms of achieving the specific goals in
studies. a specific context of an environment with efficiency,
effectiveness, and satisfaction [13, 16]. Similarly, Jacob
II. LITERATURE REVIEW Nielsen proposed five usability attributes for achieving
This part covers up the literature review about enhancing usability goals. Jacob Nielsen's attributes described usability
the usability of m-commerce applications by using usability in terms of overall system acceptability [14, 15, 17].
evaluation techniques. Moreover, Condos also proposed usability factors such as
navigation, Contents, information Architecture, Error
M-commerce is the extension of the e-commerce field. M-
prevention, and Menu Visualization. Quim Model also
commerce can be defined as a monetary transaction that takes
proposed usability factors Trustfulness, Accessibility, and
place to buy or sell products using wireless mobile devices. A Usefulness.
mobile device has wireless fidelity technology which provides
mobility to end-users. The Internet has become increasingly TABLE I. USABILITY EVALUATION MODELS

185
 University of Sindh Journal of Information and Communication Technology (USJICT) Vol.5(4), pg.: 184-207

Usability Usability Factors performed set of tasks in the cognitive walkthrough and asked
Guidelines four questions during performing tasks. User testing is a
ISO 9241-11 [16] Efficiency, effectiveness, Satisfaction method in which real users of the system performs tasks to
Nielsen [17] Learnability, Memorability, Satisfaction, Error measure usability in terms of efficiency, effectiveness, and
Rate, Efficiency satisfaction [23].
Condos [18] Navigation, Contents, information Architecture, Heuristic Evaluation is a widely used usability technique to
Error Prevention, Presentation, Menu visualization, find out usability problems in the m-commerce application
Input rate, and Menu visualization
interface and for the same, it is used actively in previous
Baharuddin [19] Effectiveness, Efficiency, Satisfaction, Usefulness, research studies. Heuristic Evaluation uses three to five
Aesthetic, Learnability, Simplicity, Intuitiveness,
Understandable and Attractiveness experts to assess the user interface of mobile commerce
QUIM Model Efficiency, Effectiveness, Productivity, applications. There is no involvement of real users in
[20] Satisfaction, Learnability, Safety, Trustfulness, heuristic evaluation. Expert users know usability guidelines
Accessibility, Universality, and Usefulness which are proposed by different researchers in their studies.
Nielsen’s Heuristic Evaluation is a technique to test the
Usability evaluation means measuring the degree of product usability in which one or more than one usability specialists
or software functionality and interface in terms of ease of use examine the user interface of a website concerning a set of
for end-users. By using usability evaluation methods, we can Heuristics. A heuristic evaluation is a fast and less expensive
identify usability issues in the product interface. Sunday, way to evaluate the interface of your website [24]. Ten
Ariyo, and Ajibola proposed different usability evaluation heuristics originated by Nielsen are the famous heuristics:
methods for mobile applications in their research studies.
Usability evaluation can be executed at any respect levels 1) Visibility of system status
inside the software development life cycle. Usability 2) Match between system and the real world
evaluation is divided into two parts in the software 3) User control and freedom
development life cycle. The formative evaluation takes place 4) Consistency and standards
from the requirement stage to the implementation stage of the 5) Error prevention
product. The summative assessment happens after the 6) Recognition rather than recall
implementation stage of the system. A summative evaluation 7) Flexibility and efficiency of use
is performed on the final product after released for 8) Aesthetic and minimalist design
commercial purposes [21].
9) Help users recognize, diagnose, and recover from
Usability evaluation methods are classified into three major
errors
categories based on users, models, and experts. The user-
10) Help and documentation
based evaluation method is the one in which real users of the
User testing is also called performance measurement. In
system are involved to assess the usability of the system. On
this technique, end users are involved to evaluate the usability
the other hand, expert-based evaluation methods are those in
metrics' effectiveness, efficiency, and satisfaction. Different
which expert users of the system are used to identify the
tasks are given to end-users. End users can be novice,
usability issues in product interface based on usability
intermediate, and experts. We can use think aloud and
guidelines. Expert users know usability guidelines which are
questionnaire methods to identify the most important
proposed by different researchers in their studies. Similarly,
usability issues. Using this method, we can calculate the
Model-based evaluation methods are based on the
success rate of completion tasks done by the end-users.
psychological prediction of a user’s performance on given
User experience (UX) is the overall users experience with
tasks in the product interface. These methods are less used to
the system interface. UX improves the quality of the user’s
evaluate the usability of the system interface. GOMS is such
interaction with the system. UX designers have skills in
a type of model in which calculates user’s time on specific
research strategies [27]. The three attributes can better clarify
tasks [22].
the UX; firstly, the primary one is the holistic individual of
Reza khajouei, Misagh, and Yunes mentioned different
UX like What is supposed through holistic nature is that UX
usability evaluation methods in detail methodology
envelops an expansive scope of characteristics and includes
framework with pros and cons in their research studies.
not just the visual, cloth, sound-related components of the
Heuristic evaluation involves experts to assess the usability
framework but additionally how the framework functions
of the product interface. Experts should have experience with
beneath a suitable use condition or placing. The second
usability guidelines which are given by different authors in
characteristic is that the UX middle is vigorously tilted
their research studies [23].
toward the purchaser's point of view. UX is regularly
Experts validate each interface element according to
misunderstood for UI, as their abbreviations are comparable.
usability guidelines. Experts write down violations of
UI tends to tilt in the direction of nine computer facet, and UI
usability guidelines during the assessment of the product
critiques are regularly subjected to quantitative measurement
interface. In the Cognitive walkthrough method, experts are
or usability checking out. UX, in comparison, issues how
involving the assessment of the product interface. They have

186
 University of Sindh Journal of Information and Communication Technology (USJICT) Vol.5(4), pg.: 184-207

users reflect, revel in, and act. The third attribute is that UX and two of the m-commerce applications with the highest
has premeditated well worth inside the development of an installations, reviews, and ratings for Pakistani users were
intended product. UX has currently emerged as a vital selected for study.
situation for the pinnacle executives of the arena due to its The objective of this study is the evaluation of the
tremendous significance [25, 26]. usability of m-commerce applications to improve the m-
The user interface (UI) is a system interface that is used commerce application design for enhancement of usability.
for interaction with the system. UI is a visual representation Heuristic Evaluation is a widely used usability technique to
of menus, buttons, icons, and contents. It is likewise the path find out usability problems in the m-commerce application
by which a user/consumer interfaces with an application or a interface in previous research studies. Heuristic Evaluation
site. Despite the application, the intention of a good quality uses three to five experts to assess the user interface of m-
UI should be on usability that is user-friendly for the users. If commerce applications. There is no involvement of real users
all things considered, we apprehend that how baffling it may in heuristic evaluation. Expert users know usability
be to utilize a gadget that doesn't work the way in which we guidelines which are proposed by different researchers in
need it to respond [27]. their studies.
It was necessary to define exclusion and inclusion criteria Nielsen’s Heuristic Evaluation Technique is a technique
for conducting most relevant literatures. An inclusion and to test the usability in which one or more than one usability
exclusion criterion is based on our key research questions. specialist examines the user interface of a website through a
The evaluation of selected studies is based on the criteria set of Heuristics. A heuristic evaluation is a fast and less
mentioned in Table 2. expensive way to evaluate the interface of your website.
TABLE II. INCLUSION AND EXCLUSION CRITERIA FOR RELEVANT A. Nielsen’s Heuristic Guidelines
LITERARURE
Nielsen’s Heuristic Evaluation Technique is a technique to test
the usability in which one or more than one usability specialist
Inclusion Criteria Exclusion Criteria examines the user interface of a website through a set of Heuristics.
A heuristic evaluation is a fast and less expensive way to evaluate
the interface of your website [24]. Ten famous heuristics originated
by Nielsen are:
Directly related to Usability evaluation Irrelevant to Usability
of M-commerce applications. evaluation of M-commerce 1) Visibility of system status
applications. 2) Match between system and the real world
3) User control and freedom
4) Consistency and standards
5) Error prevention
6) Recognition rather than recall
7) Flexibility and efficiency of use
III. METHODOLOGY
8) Aesthetic and minimalist design
It includes the details of the selection of two m-commerce 9) Help users recognize, diagnose, and recover from
applications from the play store and app store, procedure, and errors
evaluation through an integrated heuristic approach and user 10) Help and documentation
testing. Previous studies indicate that heuristic and user
testing approaches are most commonly used methods in B. Apple Usability Guidelines
evaluating the usability of m-commerce applications. Apple Company has also proposed usability guidelines
Moreover, usability is an approach to figure out for mobile applications. They guided application developers
effectiveness, efficiency and satisfaction. Heuristic to follow usability heuristics of user control and consistency,
evaluation is quick technique to identify the UX/UI issues in in particular, while developing IOS Applications.
user interface of m-commerce applications; based on
established heuristics. Many authors have defined usability in C. Google or Android Usability Guidelines
many ways. Usability is the name of ease of use and Google Company also developed usability guidelines for
learnability of products. Usability means how quickly users mobile applications. There are three main reasons behind
use a product and learn to use it for the first time. these usability guidelines. Firstly, all mobile devices have
International organization for standardization (ISO) defined inherited limitations. Secondly, there is a lot of variability in
usability as: “The degree to which any software product can different mobile brands. Lastly, each company follow its own
be used for achieving specific goals by specific users in the standards and policies. Most common usability heuristics for
specific context of use with efficiency, effectiveness, and Android users are information hierarchy, structure, dynamic
satisfaction [14]. engagement, and visibility.
The first step is to select m-commerce applications for
usability evaluation. For this purpose, top ten m-commerce
applications were reviewed on the app store and play store,

187
 University of Sindh Journal of Information and Communication Technology (USJICT) Vol.5(4), pg.: 184-207

D. Integrated Heuristic Approach Phase 1-Step 2

To evaluate the usability of m-commerce applications, we ➢ Usability Evaluation of M-commerce apps by Expert and
will use an integrated usability heuristics approach. There are Novice Users.
three main reasons for the adoption of an integrated heuristics
approach to evaluate mobile applications. First, there is a To collect the feedback from experts on heuristic
huge variability of cell phones of different brands. Second, evaluation, a questionnaire was distributed and asked the
different cell phone brands have inherited limitations. Lastly, experts to rank each usability violation based on the result
the most important reason is that every cell phone company statement according to Nielson’s severity ranking scale as
offers usability guidelines for mobile applications. These mentioned in Table 4. The questionnaire that researcher have
integrated usability heuristic approaches identify usability made was based on 13 usability heuristics guidelines for
issues on the top two m-commerce applications[28]. novice user testing and heuristic evaluation adopted from
User testing is also called performance measurement. In previously research studies [30, 32]. These Integrated
this technique, end users are involved to evaluate the usability Heuristics are mentioned in Table 5.
metrics' effectiveness, efficiency, and satisfaction. Different TABLE IV. NIELSEN’S SEVERITY RATING SCALE
tasks are given to end-users. End users can be novice,
intermediate, and experts. We can use think aloud and Rank [0-4] Definition
questionnaire methods to identify the most important 0 Not any Usability Problem
usability issues. Using this method, we can calculate the
1 Only Cosmetic Usability Problem
success rate of completion tasks by end-users [29].
Success rate = completion tasks + half tasks * 0.5 / Total 2 Minor Usability Issues
Number of tasks.
3 Major Usability Issues
In this evaluation method, direct end-users are involved in
testing the system. It is the best way to detect usability issues 4 Usability catastrophe
in the system because both experts and novice users are
involved in this testing.
If actual users’ suggestions, opinions, and feedback are TABLE V. SELECTIED INTEGRATED USABILITY HEURISTICS
involved in building a new interface/design for a product then
it is called Participatory design. As in this study, both experts Heuristic Usability Rule Definition
and novice users are the participants to identify issues in m- NH1: Visibility of system The system should always inform users
commerce apps, so, the end product interface would be a status and content about what is going on, within reasonable
time, using proper feedback.
participatory design based on the analysis of the
questionnaire filled by 26 novice users and 4 expert users. For NH2: Match between The system should use the user’s
expert, different IT offices were visited. We selected 4 system and real world language with words, phrases that are
much familiar by end-users not by
experts that had good experience of these applications, HCI system-oriented words, and phrases
theories, research and industrial experience. Moreover, a language.
training session was also given to the experts on usability
NH3: User control and The system should provide the user’s
guidelines. After this process, 26 novice users from different freedom facilities to make their strategies for undo
field of life were selected for the novice user’s evaluation of and redo activities rather than the systems
m-commerce applications. Research aim was explained to all do for them.
of them. All these 26 novice users were from IT background NH4: Consistency and The system should follow platform
and they were having experience of using different mobile Standards convention and users should not have to
applications. The idea of choosing both experienced and worry about different words, actions, and
novice users was to measure the difference in performance in styles.
these two groups. We concentrated our research to evaluate NH5: Error Prevention The system should alert the user when he
usability issues by the users. Table 3 below depicts the steps is doing some mistakes through pop up
involved in the research. and prevent errors.
NH6: Recognition rather The system should provide options,
than recall objects, and actions visible to users. A
user does not need to remember again
TABLE III. WORK METHODOLOGY PHASE WISE STEPS these things. The system should reduce
the memory load on users by providing
Enhancing the Usability of M-commerce Apps by using Heuristic suggestions.
and User Testing based on Integrated Heuristic Guidelines
NH7: Flexibility and The system should provide extra
Phase 1-Step 1 efficiency of use advanced options to expert users
according to their demands.
➢ Selection of Two M-commerce Apps; Draz.pk, Elo.com
(Based on Reviews, Installation, and Rating).

188
 University of Sindh Journal of Information and Communication Technology (USJICT) Vol.5(4), pg.: 184-207

NH8: Aesthetic and The system should always provide f) Add payment method: Check out the products and
minimalist design relevant information, according to user adds payment methods to successfully place an order.
needs, on the interface rather than
irrelevant information.
In Heuristic Evaluation, each usability experts assess the
NH10: Help users The system should always show errors in
recognize, diagnose and human-readable language rather than in interface of the system alone and find out the usability issues
recover from error machine code like 404 errors. in the system. Usability experts give the rate of each usability
issue according to the Nielsen severity rating scale. Each
GH1: Dynamic The app should provide facility users to
Engagement engage with other apps, services, and expert user evaluates the user interface of the m-commerce
users in two-way communications. applications by using the above mentioned usability
GH2: Information and The mobile app should display visual
heuristics and eventually problems were noted by each
visual hierarchy objects and information in a hierarchy expert. In the post-evaluation meeting, expert evaluators
order based on the user’s function of meet with each other and rate the severity of usability issues
sorting, searching, and swiping, etc. applying the Nielsen severity ranking scale, which is
AH1: Natural-Interaction The mobile app should utilize the mentioned below in Table 6. In this meeting, each expert
hardware and software capabilities for unanimously agrees on the usability problems, category and
application gestures like a pinch. its severity rank. The severity of the problem depends on the
occurrence of the problems. Moreover, issues were
To conduct a heuristic evaluation, two heuristic evaluation categorized by the expert evaluators on the basis of this
approaches were selected; first is Task-based Analysis and severity and occurrence basis on occurrence scale.
the second one is Free-Flow.
TABLE VI. NIELSON’S SEVERITY RATING SCALE

1) Tasks based Approach:

In this approach, each expert performs predefined Rating Description
tasks to evaluate the user interface of the system according to
heuristic guidelines checklists. 0 This doesn’t seem a usability issue.

2) Free-Flow Approach: 1 Cosmetic issue. Fixed this problem when you have
buffer time in your project timeline
In this approach, each expert has already inspected the
user interfaces of the system several times by using the
2 Minor issue. Fixed this issue on low priority.
heuristic checklist which is provided by Jacob Nielsen,
Google Android Company, and Apple Company. So, each 3 Major issue. Fixed this issue on high priority.
expert is free to hand to evaluate every part of the system by
using heuristic guidelines and find out issues in it for further 4 Catastrophic issue. Fixed this issue before product
improvement [35]. release.
Usability of m-commerce apps depends upon the sign up,
login page, search product page and checkout process flow
pages. In this study, task based approach is used to assess the
usability of specific pages of m-commerce apps[30-32].
There are five tasks that we used to evaluate the Usability
problems in m-commerce applications.
IV. RESULTS
a) Assert the registration process to create an account
and sign in to the application. Two expert evaluators, who are working as Software
Quality Assurance Engineer and Team Lead Developer,
b) Update your profile information.
highlighted the problems experienced in draz.pk m-
c) Adds funds: Add money into your wallet by any commerce app based on severity and provided solutions to
method. resolve the problems in the current interface of m-commerce
d) Search watch and add to cart: Search out the applications. Table 7 below mentions the usability issues
product, add the product to cart, and continue shopping. founded by the experts collectively for the draz.pk m-
commerce application. Table 7 highlights the detailed
e) Find a technical solution to the problem: Find some
qualitative results after an assessment of the user interface of
technical help on any topic.
draz.pk by using above mentioned usability heuristics.
TABLE VII. SEVERITY RATE OF ERROR ACCORDING TO INTEGRATED HEURISTIC EVALUATION

Integrated Usability Heuristics Draz.pk M-Commerce Application

189
 University of Sindh Journal of Information and Communication Technology (USJICT) Vol.5(4), pg.: 184-207

0 1 2 3 4 Total

1. Visibility of system status 0 0 2 2 0 4

2. Match between system and real 0 0 1 1 0 2

world

3. User control and freedom 1 0 1 1 0 3

4. Consistency and standard 0 1 1 0 0 2

5. Error prevention 0 0 0 2 0 2

6. Recognition rather than recall 0 1 0 0 0 1

7. Flexibility and efficiency to use 0 0 1 2 0 3

8. Aesthetic and minimalist design 0 1 0 3 0 4

9. Helps user recognize, diagnose 0 0 1 0 0 1

and recover from errors

10. Help and Documentation 1 1 1 0 0 3

11. Dynamic Engagement 0 1 0 0 0 1

12. Information and visual hierarchy 0 0 2 0 0 2

13. Natural interaction 1 0 0 0 0 1

3 5 10 11 0 29

10.34% 17.24% 34.48% 37.93% 0%

This Detailed expert evaluation shows the collective important to redesign the user interface of draz.pk
result of two expert evaluators on draz.pk. They have found application. Table 8 highlights some major usability issues in
a total of 29 issues in draz.pk m-commerce application. In draz.pk with recommendations and screen shorts.
these 29 issues, only 12 were major issues which make
37.93% of the total found issues while 10 were minor issues
that make 34.48% of the total issues founded by the experts.
Major issues percentage 37.93% makes it

TABLE VIII. HEURISTIC EVALUATION FIDING WITH RECOMMENDATIONS

190
 University of Sindh Journal of Information and Communication Technology (USJICT) Vol.5(4), pg.: 184-207

Sr. Issue/Recommendation Screen

1 Visibility of the system status

Issue:
At the top bar, there is no header.

Recommendation:
Add the title on the top bar.

2 Visibility of the system status

Issue:
No Confirmation Massage dialog is shown on the screen after mobile
number verification.

Recommendation:
Show Properly Dialog.

3 Error Prevention

Issue:
Confusion between the Text label and the button on the main screen.

Recommendation:
There should be a proper login button with proper color font size.

4 User Control and freedom

Issue:
There is no option available for removing the picture.

Recommendation:
There should be a proper button.

191
 University of Sindh Journal of Information and Communication Technology (USJICT) Vol.5(4), pg.: 184-207

5 Flexibility and efficiency of use

Issue:
Keywords or search tags are not available for quick search. Sorting and
filtering option are less available like average rating review, A to Z alphabets.

Recommendation:
There should add some keywords for quick search and more advanced options
for searching.

6 Help users recognize, diagnose, recover from errors

Issue:
Users click on Top-up Transactions. The app does not give a pop-up dialog for
any kind of suggestion.

Recommendation:
The system should provide Suggestion Dialog.

7 Help and documentation

Issue:
There is no Video Teaching Tutorial for a user to learn application tasks.

Recommendation:
There should be a Video-based FAQ section where users can search out their
common problem’s solution.

192
 University of Sindh Journal of Information and Communication Technology (USJICT) Vol.5(4), pg.: 184-207

8 Aesthetic and minimalist design

Issue:
Irrelevant information is displayed on the home button that creates a visibility
issue.

Recommendation:
Provide only essential info on the screen.

9 Error prevention

Issue:
No confirmation dialog before payment of the product using easy paisa without
the email box.

Recommendation:
There should be a confirmation dialog before pressing the pay securely button.

10 Match between the system and the real world

Issue:
The picture uploading icon is not matched with the system and the real world.
It is a confusion icon.

Recommendation:
There should be a proper icon for a picture with text.

193
 University of Sindh Journal of Information and Communication Technology (USJICT) Vol.5(4), pg.: 184-207

11 User Control and Freedom

Issue:
The system goes to an unwanted stage during adding a payment method for
purchasing products.

Recommendation:
The System Should Provide a Home button to back the main Page.

12 Visibility of System Status

Issue:
Top up button in balance detail is not working or giving response after clicking
on it.

Recommendation:
The system should give proper feedback.

13 User Control and Freedom

Issue:
The keyboard is not hidden on slay out click.

Recommendation:
Hide Keyboard on Screen Touch.

14 Aesthetic and Minimalist Design

Issue:
There is an Extra Search Bar for account information.

Recommendation:
There is no need for any search bar here.

194
 University of Sindh Journal of Information and Communication Technology (USJICT) Vol.5(4), pg.: 184-207

15 Aesthetic and Minimalist Design

Issue:
Store option is an extra option as compared to Buy Now and Add to Cart here.
Violation of the hick's law here.

Recommendation:
There is a need to remove it.

16 Flexibility and Efficiency to Use

Issue:
This app has not supported both orientations (Horizontal and vertical).

Recommendation:
Landscape orientation should be present in it.

• There is no message or notice provided by the app

Different other issues explained by expert’s evaluator on which informs that when the rider will arrive.
draz.pk m-commerce applications are mentioned below.
• There is no progress bar to give better visual hints to
• Sometimes video advertisements are shown at the users to place an order from shipping to payment
draz.pk interface and there is no option available to steps.
remove it quickly.
• The app uses very small font sizes for contents in the Similarly, two expert users evaluated the Elo.com m-
interface. commerce application. One of the expert users was a Graphic
• There is no speech to text searching facility Designer in a software house. He has been using the Elo.com
available on draz.pk. application on his android phone from a long time while other
• There is no option available to rate the draz.pk app. expert user was working as an android developer. He has
• The draz.pk app takes too much space for mobile been working with other team members to finalize the User
storage. interface of the applications after finding issues and hence
• The speed of the draz.pk app is slow. There is a need delivering an end application to end-users. He was also the
to make better performance in terms of speed. user of Elo.com application for online purchasing products
• Search results are not matching with the search from a significant time period. Table 9 below highlights the
query. collective details of qualitative results after usability testing
• GPS is not working properly in the draz.pk app. of the user interface of Elo.com application by using above
mentioned usability Guidelines/Heuristics.
• When you place your order, draz.pk app informs too
late that the product is out of stock.
• The app closes when we add a picture to the review.
• Order delivery is very late in the draz.pk app.

195
 University of Sindh Journal of Information and Communication Technology (USJICT) Vol.5(4), pg.: 184-207

TABLE IX. SEVERITY RATE OF ERROR ACCORDING TO INTEGRATED HEURISTIC EVALUATION

Integrated Usability Heuristics Elo.com M-Commerce Application

0 1 2 3 4 Total

1. Visibility of system status 0 0 0 1 0 1

2. Match between system and real world 0 0 2 0 0 2

3. User control and freedom 0 1 0 1 0 2

4. Consistency and standard 1 0 1 0 0 2

5. Error prevention 0 1 2 1 0 4

6. Recognition rather than recall 0 0 2 1 0 3

7. Flexibility and efficiency to use 0 1 2 0 0 3

8. Aesthetic and minimalist design 0 1 1 1 0 3

9. Helps user recognize, diagnose and recover from errors 0 0 1 0 0 1

10. Help and Documentation 0 0 2 3 0 5

11. Dynamic Engagement 0 0 1 0 0 1

12. Information and visual hierarchy 0 1 1 0 0 2

13. Natural interaction 0 1 0 0 0 1

1 6 15 8 0 30

3.33% 20% 50% 26.66% 0%

percentage was 26.66% that makes it important to redesign

This Detailed expert evaluation shows the collective result of the user interface of the Elo.com application. Table
two expert evaluators on elo.com. They have found a total of
30 issues in the Elo.com m-commerce application. Of these
35 issues, only 8 were major issues that make 26.66% of the 10 highlights some major usability issues in Elo.com with
total issues found while 15 were minor issues that make 50% recommendations and screen shorts.
of the total issues founded by experts. Major issues

196
University of Sindh Journal of Information and Communication Technology (USJICT) Vol.5(4), pg.: 184-207

197
 University of Sindh Journal of Information and Communication Technology (USJICT) Vol.5(4), pg.: 184-207

TABLE X. HEURISTIC EVALUATION FIDING WITH RECOMMENDATIONS

Sr. Issue/Recommendation Screen
1 Visibility of System Status

Issue:
The logo is not Sufficiently Visible.

Recommendation:
The logo design should be clear and visible.

2 Error Prevention

Issue:
There is no Sign-Up link available on the main menu.

Recommendation:
There should be an option in the main menu bar.

3 Match Between System and real world

Issue:
There is no menu choice order in the most logical way.

Recommendation:
There should be an order on the menu.

4 Error Prevention

Issue:
There is no visual difference between the interaction
object and the information object. The “forgot” button
confuses the user.

Recommendation:
There should be a proper visual forgot button.

5 Error Prevention

Issue:
There is no visual difference between the interaction
object and the information object. The registration button
confuses the user. The back group and button color are
the same. It is text info or link?

Recommendation:
There should be a proper visual Register button.

198
 University of Sindh Journal of Information and Communication Technology (USJICT) Vol.5(4), pg.: 184-207

6 User Control and Freedom

Issue:
There is no option to view the entered password by the
users.

Recommendation:
There should be an option for viewing password.

7 Dynamic Engagement

Issue:
There is only one option for login into the app using an
existing social media account.

Recommendation:
There should be more options like gmail.com,
yahoo.com, etc. to create an account on elo.com.

8 Recognition rather than recall

Issue:
No text available with icons at the main menu bar.

Recommendation:
There should be the mentioned text content of used icons.

9 Flexibility and efficiency to used

Issues:
GPS is not working during profile updating and order
placing.

Recommendation:
The system should use the geo location of users when
needed.

199
 University of Sindh Journal of Information and Communication Technology (USJICT) Vol.5(4), pg.: 184-207

10 User control and freedom

Issues:
There is no digital wallet option available at the Elo.com
app.

Recommendation:
There should be a digital wallet option in-app.

11 Aesthetic and minimalist design

Issues:
The app shows unnecessary information at specific
product description.

Recommendation:
The app should provide only essential information for
decision making to users.

12 Help and documentation

Issues:
There is no help and documentation module available at
the Elo.com application.

Recommendation:
There must be present help and documentation section in
the application.

200
 University of Sindh Journal of Information and Communication Technology (USJICT) Vol.5(4), pg.: 184-207

13 Information and visual hierarchy

Issues:
App does not save and display recent search history to
users.

Recommendation:
The app should provide recent search history to users.

14 Aesthetic and minimalist design

Issues:
App provides unnecessary category in the Categories list.

Recommendation:
The interface of the app should be clear from
unnecessary advertisements and information. The app
should provide only essential information to users.

• The app and website are not connected properly with

Different other issues faced by expert’s evaluator on Elo.com each other and are two distant platforms.
m-commerce applications are mentioned below along with
explanation.
• Favorite items in the app are not available on the
• There is no margin in between the item screen. website.
• The color scheme is poorly selected. • There is no setting available to mute unnecessary
• The splash screen indicates the error. notifications.
• The app and website are not connected properly with • The app is not redirected to my home page after
each other and are two distant platforms. Favorite login successfully on it.
items in the app are not available on the website. • The app is not providing an option to update profiles
• There is no setting available to mute unnecessary like the picture.
notifications. • The Elo.com app is providing fewer options for
• The app is not redirected to my home page after payment delivery.
login successfully on it. • GPS is not working on the Elo.com app.
• The app is not providing an option to update profiles • We cannot see “my old purchased products list” in
like the picture. it.
• The Elo.com app is providing fewer options for • We are receiving notifications of an update but there
payment delivery. is no update available.
• There is no margin in between the item screen. • We are not seeing the privacy policy and term of
• The color scheme is poorly selected. service section.
• The splash screen indicates the error. • We do not see our reviews given to products.

201
 University of Sindh Journal of Information and Communication Technology (USJICT) Vol.5(4), pg.: 184-207

• The app is horrible in terms of searching. There is

no appropriate search result. Users need to visit
many pages for searching. No one has much time to
see all pages.

202
 University of Sindh Journal of Information and Communication Technology (USJICT) Vol.5(4), pg.: 184-207

The objective of user testing from novice users was to A. Specific Tasks Performed on Draz.pk Application
identify usability problems related to the user interface of 1) Create the account on Draz.pk Application.
both m-commerce apps draz.pk and elo.com. After 2) Update your address profile information.
performing user testing with novice user, researcher can 3) Find any help from the Help and Documentation
easily compare the novice and expert user’s performance on Module.
both m-commerce apps. Think aloud is a famous method that
4) Search out the KN 95 Mask and add it to the cart.
is used by researchers to record the user’s suggestions,
5) Find out the term of the return policy of Application.
behaviour, and opinions about the system during user testing.
6) Search out the surgical mask and add its favorite list.
The researcher asked the users to speak loudly during
performing tasks in this method. This method also was TABLE XII. SPECIFIC TASK PERFORMANCE ON DRAZ.PK
chosen for usability testing on both m-commerce apps for
No. of Successful Partial Fail Rating Average
novice users in this research study. Different novice users Tasks Users Successful Users Time
from different fields like students, office employees, and Users Taken
housewives were asked to perform specific tasks on these
both m-commerce apps in 1 to 1 meeting. After completing 1 13 0 0 100% 57sec
the specific tasks, they were required to fill out the survey
questionnaire which was based on different usability expert’s 2 10 1 2 80% 78 sec
guidelines. Mobile screen recorder software (Vidma
3 13 0 0 100% 27 sec
Recorder) was used to record the actual time that is used to
carry out tasks by each novice users during usability testing. 4 13 0 0 100% 17 sec
By user testing, we obtained the efficiency and effectiveness 5 8 3 2 73% 30 sec
of the system. ISO organization proposed the usability
6 13 0 0 100% 19 sec
metrics. These usability metrics are efficiency, effectiveness,
and satisfaction. Efficiency means how much time is
required by the system to perform each task. Effectiveness The Table above 12 shows the expert tasks performance on
means how many tasks are successfully carried by the user draz.pk app. 2 Novice users were failed to perform task 2
[33-35]. We can calculate the effectiveness of the system by regarding update personal address and 2 novice users were
using the below formula: also not able to perform tasks 5 regarding finding return
policy of product on draz.pk app.
Successful Rate = (successful tasks + (partially successful B. Specific Tasks Performed on Elo.com Application
tasks) * 0.5) / Total Number of tasks
1) Create the account on Elo.com Application.
These novice users were divided into two groups Draz.pk 2) Update your address profile information.
and Elo.com.13 novice users were assigned to perform 3) Find any help from the Help and Documentation
specific tasks on draz.pk and 13 novice users were assigned Module.
to perform specific tasks on elo.com. All novice users were 4) Search out the KN 95 Mask and add it to Cart.
having much experience with mobile applications. However, 5) Find out the term of the return policy of Application.
not all novice users have already used these m-commerce 6) Search out the surgical mask and add its favorite list.
apps. A brief demo was given to each novice users about
specific tasks on both m-commerce apps and questionnaires.
TABLE XIII. SPECIFIC TASK PERFORMANCE ON ELO.COM
Below Table 11 is giving the complete demographic of
novice users. No. of Successful Partial Fail Rating Average
Tasks Users Successful Users Time
Users Taken
TABLE XI. DEMOGRAPHIC INFORMATION OF NOVICE USERS
Sr. Gender Age Age Age Experience 1 11 2 0 92% 63 sec
No Between Between Above with Mobile
(18-25) (25-30) (30) Applications 2 13 0 0 100% 87 sec

3 0 0 13 0% 0 sec
1 Male 14 9 1 Yes
4 13 0 0 100% 41 sec

2 Female 1 1 0 Yes 5 0 0 13 0% 0 sec

6 7 2 4 61% 18 sec

203
 University of Sindh Journal of Information and Communication Technology (USJICT) Vol.5(4), pg.: 184-207

The Table above 13 shows the expert tasks performance on Question 6 – 8 – Learnability, Flexibility, and efficiency to
Elo.com app. 13 Novice users failed to perform task 3 use
regarding find any help from the help and documentation Question 9– 10 – Error Prevention
module and 13 Novice users were also not able to perform
tasks 5 regarding finding return policy of product on Elo.com Question 11 – 12 – Help and Documentation
app. Question 13-14 – Dynamic Engagement
In user testing, the last step was to fill out a questionnaire Question 15 – 16 – Effectiveness and Satisfaction
survey with novice users to identify the UI/UX issues in both
QUESTION 17 – INFORMATION AND VISUAL HIERARCH
of these m-commerce apps. Each survey question was
categorized according to apple usability guidelines, android
usability guidelines, and Jacob nelson ten heuristics.

The questions below refer to the following integrated

heuristics Guidelines terms:

Question 1 - 5 – Visibility of system status, Aesthetic and

Minimalist Design

Figure 1. Draz.pk Questions 1-5 Figure 2. Elo.com Questions 1-5 Figure 3. Draz.pk Questions 6-8

In Questionnaire, one to five questions Fig. 2 represents the highest percentage Questionnaire 6 to 8 were asked from the
were asked from the novice user that of participants who disagreed with novice users that come under the
comes under the Visibility to system questions 4 and 5. Learnability, Flexibility and efficiency to
status, Aesthetic and Minimalist use. Fig. 3 represents that the half
Design. Fig. 1 represents the highest percentage of participants disagreed with
percentage of participants who question 6 that comes under learnability.
disagreed with these points.

204
 University of Sindh Journal of Information and Communication Technology (USJICT) Vol.5(4), pg.: 184-207

Figure 4. Elo.com Questions 6-8 Figure 5. Draz.pk Questions 9-10 Figure 6. Elo.com Questions 9-10

Questions 6 to 8 were asked from the In questionnaires, 9 to 10 were asked In questionnaires, 9 to 10 were asked from
novice users that come under the from the novice users that come under the novice users that come under the Error
Learnability, Flexibility and efficiency the Error Prevention. Fig. 5 represents a Prevention. Fig. 6 represents a high
to use. Fig. 4 represents that half high percentage of participants who percentage of participants who disagreed
percentage of participants disagreed agreed with it. with it.
with questions 7 and 8 that come under
Flexibility and efficiency to use.

Figure 7. Draz.pk Questions 11-12 Figure 8. Elo.com Questions 11-12 Figure 9. Draz.pk Questions 13-14

In Questionnaire 11 to 12 were asked Questionnaire 11 to 12 were asked from Questionnaire 13 to 14 were asked from
from the novice users that come under the novice users that come under the the novice users that come under the
the Help and Documentation. Fig. 7 Help and Documentation. Fig. 8 Dynamic Engagement. Fig. 9 represents a
represents a high percentage of represents a high percentage of high percentage of participants strongly
participants agreed with it. There is a participants who strongly disagreed with agreed with it.
need to improve this module because it. There is no present help and
some users have disagreed with its documentation module available in it.
functionality and feedback.

Figure 10. Elo.com Questions 13-14 Figure 11. Draz.pk Questions 15-16 Figure 12. Elo.com Questions 15-16

Questions 13 to 14 were asked from the In Questionnaire, questions 15 to 16 were In Questionnaire, questions 15 to 16 were
novice users that come under the asked from the novice users that come asked from the novice users that come
Dynamic Engagement. Fig. 10 under the Effectiveness and Satisfaction. under the Effectiveness and Satisfaction.
represents a high percentage of Fig. 11 represents a high percentage of Fig. 12 represents that more than half the
participants strongly agreed and agreed participants who gave neutral remarks. percentage of participants have shown
with it. There is a need for more agreeness while others gave neutral
improvement in this term because some remarks.
users have disagreed with this point.

205
 University of Sindh Journal of Information and Communication Technology (USJICT) Vol.5(4), pg.: 184-207

Figure 13. Elo.com Question 17 Figure 14. Draz.pk Question 17

In Questionnaire, question 17 was asked from the novice users Question 17 was asked from the novice users that come under
that come under the Information and Visual Hierarchy. Fig. 13 the Information and Visual Hierarchy. Fig. 14 represents that
represents that less than half the percentage of participants has more than half the percentage of participants has given agree
given disagree remarks. response.

In future, we will continue with results of this research and

Below Table 14 highlights the factors that were identified will apply MCDA and AHP techniques to prioritize the
after user testing on both these m-commerce apps from
novice users. usability issues and target them well in prototype design to
ultimately reach towards the user satisfied m-commerce
TABLE XIV. FACTORS FROM USER TESTING OF BOTH APPS
platforms.
Draz.pk Elo.com
Visibility of System status Aesthetic and Minimalist Design
Issues Issues
REFERENCES
Aesthetic and Minimalist Help and Documentation Issues
Design Issues [1] H. A. Aljifri, A. Pons, and D. Collins, “Global e-commerce: A
Learnability Issues Flexibility and Efficiency Issues framework for understanding and overcoming the trust barrier,”
Inf. Manag. Comput. Secur., vol. 11, no. 2–3, pp. 130–138, 2003,
Error Prevention Issues Dynamic Engagement Issues doi: 10.1108/09685220310480417.
Information and Visual Error Prevention Issues [2] P. Resnick, H. R. Varian, and G. Editors, “Recomender
Hierarchy Issues Systems,” Commun. ACM, vol. 40, no. 3, pp. 56–58, 1997.
[3] L. Hasan, “Institutional Repository Usability evaluation
framework for e-commerce websites in developing countries This
item was submitted to Loughborough University ’ s Institutional
Repository c Layla Hasan,” no. October, 2015.
V. CONCLUSION AND FUTURE WORK [4] P. Zhang, I. Benbasat, J. Carey, F. Davis, D. F. Galletta, and D.
Strong, “Human-Computer Interaction Research in the MIS
The main aim of which is ‘Enhancing usability of m- Discipline,” Commun. Assoc. Inf. Syst., vol. 9, no. 20, pp. 334–
commerce applications’ is targeted and usability problems in 355, 2002.
the currently available applications are evaluated to resolve [5] P. Lorca, J. de Andres, and J. Garcia-Diez, “Impact of e-
currently faced issues experienced by the users. It was commerce sales on profitability and revenue. The case of the
manufacturing industry,” Eng. Econ., vol. 30, no. 5, pp. 544–555,
concluded based on the integrated heuristic guidelines 2019, doi: 10.5755/j01.ee.30.5.21254.
followed with heuristic and user testing, that we can evaluate [6] L. Rajabion, “Critical factors for adoption of mobile commerce
the user interface of m-commerce applications by HCI services,” Int. J. Bus. Soc. Sci., vol. 6, no. 12, pp. 26–35, 2015.
Guidelines. Heuristics evaluation and user testing are the [7] A. Tonacci et al., “An innovative, unobtrusive approach to
investigate smartphone interaction in nonaddicted subjects based
most popular methods to evaluate the user interface of m- on wearable sensors: A pilot study,” Med., vol. 55, no. 2, pp. 1–
commerce applications. Jacob Nielsen, famous as the father 11, 2019, doi: 10.3390/medicina55020037.
of usability, proposed the ten usability guidelines to evaluate [8] M. Išoraitė and N. Miniotienė, “Electronic Commerce: Theory
the user interface by using different usability evaluation and Practice,” IJBE (Integrated J. Bus. Econ., vol. 2, no. 2, p. 73,
2018, doi: 10.33019/ijbe.v2i2.78.
methods. Apple and Android companies also provided [9] S. I. Swaid and T. Z. Suid, Usability of Mobile Apps : An
UX/UI design guidelines to evaluate the user interface of Integrated Approach Usability Heuristics for M-Commerce Apps,
mobile applications. So, enhanced usability by m-commerce vol. 1, no. November. Springer International Publishing, 2018.
applications is achieved for novice as well as expert users that [10] S. A. Ali, A. Ejaz, H. A. Khan, and S. Siddiqui, “User Interface
Evaluation by Two Factor Model and Heuristic Evaluation : A
belong to various domains of life. Perspective from Ecommerce Industry of Pakistan,” vol. 12, no.

206
 University of Sindh Journal of Information and Communication Technology (USJICT) Vol.5(4), pg.: 184-207

June, pp. 1–11, 2019, doi: 10.17485/ijst/2019/v12i22/142830. J. Adv. Res. Dyn. Control Syst., vol. 10, no. 10, pp. 1212–1217,
[11] Iris A. Junglas, “U-COMMERCE: A CONCEPTUAL 2018.
EXTENSION OF E- AND M-COMMERCE,” 2001, p. 23. [26] A. Hussain, E. O. C. Mkpojiogu, N. H. Jamaludin, and S. T. L.
[12] W. Aziz and Y. Hashmi, “MASTER ’ S THESIS Usability Moh, “A usability evaluation of Lazada mobile application,” AIP
Principles for Mobile Commerce,” Luleå Univ. Technol., p. 105, Conf. Proc., vol. 1891, no. October, 2017, doi:
2009. 10.1063/1.5005392.
[13] F. Paz and J. A. Pow-Sang, “Usability Evaluation Methods for [27] A. Hussain, E. O. C. Mkpojiogu, and K. Suleiman, “A usability
Software Development: A Systematic Mapping Review,” Proc. - testing of a mobile print shop booking and design application,” J.
8th Int. Conf. Adv. Softw. Eng. Its Appl. ASEA 2015, vol. 10, no. Adv. Res. Dyn. Control Syst., vol. 10, no. 10 Special Issue, pp.
1, pp. 1–4, 2016, doi: 10.1109/ASEA.2015.8. 1359–1365, 2018.
[14] A. Seffah, M. Donyaee, R. B. Kline, and H. K. Padda, “Usability [28] A. Hussain, E. O. C. Mkpojiogu, J. Musa, and S. Mortada, “A
measurement and metrics: A consolidated model,” Softw. Qual. user experience evaluation of Amazon Kindle mobile
J., vol. 14, no. 2, pp. 159–178, 2006, doi: 10.1007/s11219-006- application,” AIP Conf. Proc., vol. 1891, no. October, 2017, doi:
7600-8. 10.1063/1.5005393.
[15] I. Standard, “Iso 9241-11,” vol. 1998, 1998. [29] A. Hussain, E. O. C. Mkpojiogu, A. Jama Isse, and R. A.
[16] P. Tsai, “a Survey of Empirical Usability Evaluation Methods,” A Mohammed, “Grab mobile application: A usability evaluation,”
Surv. Empir. usability Eval. methods, pp. 1–18, 1996. AIP Conf. Proc., vol. 2016, no. October, pp. 1–6, 2018, doi:
[17] H. B. L. Duh, G. C. B. Tan, and V. H. H. Chen, “Usability 10.1063/1.5055456.
evaluation for mobile device: A comparison of laboratory and [30] A. Hussain, E. O. C. Mkpojiogu, H. Abubakar, and H. M.
field tests,” ACM Int. Conf. Proceeding Ser., vol. 159, no. Hassan, “The usability evaluation of Mudah.my on mobile
January, pp. 181–186, 2006, doi: 10.1145/1152215.1152254. device,” AIP Conf. Proc., vol. 1891, no. October, 2017, doi:
[18] R. Baharuddin, D. Singh, and R. Razali, “Usability dimensions 10.1063/1.5005391.
for mobile applications-a review,” Res. J. Appl. Sci. Eng. [31] Ö. Öztürk and K. Rizvanoǧlu, “M-commerce usability: An
Technol., vol. 5, no. 6, pp. 2225–2231, 2013, doi: explorative study on Turkish private shopping apps and mobile
10.19026/rjaset.5.4776. sites,” Lect. Notes Comput. Sci. (including Subser. Lect. Notes
[19] B. A. Seffah and E. Metzker, “O m u s e,” 2004. Artif. Intell. Lect. Notes Bioinformatics), vol. 8015 LNCS, no.
[20] S. A. Ajibola and L. Goosen, “Development of Landscape of PART 4, pp. 623–630, 2013, doi: 10.1007/978-3-642-39253-
Usability Evaluation Methods for Mobile Applications,” vol. 8, 5_69.
no. 02, pp. 89–102, 2019. [32] S.-P. Shu, C. Perng, W.-C. Chiou, and T.-Y. Ou, “Usability
[21] Y. Guo, R. W. Proctor, and G. Salvendy, “A conceptual model of Evaluation of Mobile Commerce Website on Internet – An
the axiomatic usability evaluation method,” Lect. Notes Comput. empirical study,” Wuhan Int. Conf. E-bus., pp. 317–323, 2014.
Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes [33] A. Bin Hussain, A. T. Mahmood, and R. K. Naser, “Investigating
Bioinformatics), vol. 6771 LNCS, no. PART 1, pp. 93–102, the effect of m-commerce design usability on customers’ trust,”
2011, doi: 10.1007/978-3-642-21793-7_11. AIP Conf. Proc., vol. 1891, no. October, 2017, doi:
[22] A. Hussain and E. O. C. Mkpojiogu, “Usability evaluation 10.1063/1.5005410.
techniques in mobile commerce applications: A systematic [34] A. Balcita, R. J. Ventayen, and T. Palaoag, “Usability of
review,” AIP Conf. Proc., vol. 1761, no. October 2017, 2016, KITIKIT: A mobile-based app for implementing customizable
doi: 10.1063/1.4960889. online product ordering,” IOP Conf. Ser. Mater. Sci. Eng., vol.
[23] J. Lazar, J. H. Feng, and H. Hochheiser, Research Methods in 482, no. 1, 2019, doi: 10.1088/1757-899X/482/1/012020.
Human-Computer Interaction. 2017. [35] A. Hussain, E. O. C. Mkpojiogu, and K. Suleiman, “A heuristic
[24] E. D. Mekler and K. Hornbæk, “A framework for the experience evaluation of Achik.Biz mobile shopping app,” Int. J. Recent
of meaning in human-computer interaction,” Conf. Hum. Factors Technol. Eng., vol. 8, no. 2 Special issue 2, pp. 123–126, 2019,
Comput. Syst. - Proc., pp. 1–15, 2019, doi: doi: 10.35940/ijrte.B1023.0782S219.
10.1145/3290605.3300455.
[25] A. Hussain, E. O. C. Mkpojiogu, N. B. Yahaya, and N. Z. B. A.
Bakar, “A mobile usability assessment of an M-Shopping app,”

207