HUMAN COMPUTER INTERACTION

B.V. Raju Institute of Technology

(Autonomous)
Vishnupur, Narsapur, Medak (District) – 502313
Department of Computer Science and
Engineering

SYLLABUS

Unit I: (12 Periods)

Interactive system design: Concept of usability, HCI and software engineering, GUI Design
and aesthetics. Prototyping techniques.
Model-based design and Evaluation: Basic idea, introduction to different types of models,
GOMS family of models (KLM and CMN-GOMS), Fitt’s law and and Hick-hyman’s
law,Model-based design.
Unit II: (10 Periods)
Guidelines in HCI: Shneiderman’s eight golden rules, Norman’s seven principles, Norman’s
model of interaction, Nielsen’s ten heuristics with example of its use , Heuristic evaluation,
contextual inquiry, Cognitive walkthrough.
Unit III: (10 Periods)
Empirical research methods in HCI: Introduction (motivation, issues, research question
formulation techniques), experiment design and data analysis.
Task modeling and analysis: Hierarchical task analysis (HTA), Engineering task models
and Concur Task Tree (CTT).
Unit IV: (6 Periods)
Dialog Design: Introduction to formalism in dialog design design using FSM (Finite State
Machines), State charts and (Classical) Petri Nets in dialog design.
Unit V: (8 Periods)
Cognitive Architecture: Introduction to CA, CA types, relevance of CA in IS design, Model
Human Processor (MHP).

Textbooks:
1. Dix A., Finlay J.,Abowd G. D. and Beale R. Human Computer Interaction, 3 rd edition,
Pearson Education, 2005
References:
1. Yvonne Rogers, Helen Sharp, Jennifer Preece; Interaction Design 3 rd Edition Wiley 2011.
2. Preece J., Rogers Y.,Sharp H.,Baniyon D., Holland S. and Carey T. Human Computer
Interaction,Addison-Wesley, 1994.
3. B.Shneiderman; Designing the User Interface,Addison Wesley 2000 (Indian Reprint).
NPTEL Course https://nptel.ac.in/content/syllabus_pdf/106103115.pdf

1 | Page
 HUMAN COMPUTER INTERACTION
Unit I
Introduction
Human-computer interaction (commonly referred to as HCI) researches the design
and use of computer technology, focused on the interfaces between people (users) and
computers. Researchers in the field of HCI both observe the ways in which humans interact
with computers and design technologies that let humans interact with computers in novel
ways.
User
By "user", we may mean an individual user, a group of users working together. An
appreciation of the way people's sensory systems (sight, hearing, touch) relay information is
vital. Also, different users form different conceptions or mental models about their
interactions and have different ways of learning and keeping knowledge and. In addition,
cultural and national differences play a part.
Computer
When we talk about the computer, we're referring to any technology ranging from desktop
computers, to large scale computer systems. For example, if we were discussing the design of
a Website, then the Website itself would be referred to as "the computer". Devices such as
mobile phones or VCRs can also be considered to be ―computers.
Interaction
There are obvious differences between humans and machines. In spite of these, HCI attempts
to ensure that they both get on with each other and interact successfully. In order to achieve a
usable system, you need to apply what you know about humans and computers, and consult
with likely users throughout the design process. In real systems, the schedule and the budget
are important, and it is vital to find a balance between what would be ideal for the users and
what is feasible in reality.
*********************************************************************
Interactive system design: Concept of usability
Usability Engineering is a method in the progress of software and systems, which
includes user contribution from the inception of the process and assures the effectiveness of
the product through the use of a usability requirement and metrics.
It thus refers to the Usability Function features of the entire process of abstracting,
implementing & testing hardware and software products. Requirements gathering stage to
installation, marketing and testing of products, all fall in this process.
Goals of Usability Engineering
Usability is one of the key concepts in HCI. It is concerned with making systems easy
to learn and use. A usable system is:
 Easy to learn
 Easy to remember how to use
 Effective to use
 Efficient to use
 Safe to use
 Enjoyable to use

2 | Page
 HUMAN COMPUTER INTERACTION
Usability
Usability has 3 components - effectiveness, efficiency & satisfaction, using which,
users accomplish their goals in particular environments. Let us look in brief about these
components.
 Effectiveness − The completeness with which users achieve their goals.
 Efficiency − The competence used in using the resources to effectively achieve the goals.
 Satisfaction − The ease of the work system to its users.
Usability Study
The methodical study on the interaction between people, products, and environment
based on experimental assessment. Example: Psychology, Behavioral Science, etc.
Usability Testing
The scientific evaluation of the stated usability parameters as per the user’s
requirements, competences, prospects, safety and satisfaction is known as usability testing.
Acceptance Testing
Acceptance testing also known as User Acceptance Testing (UAT), is a testing
procedure that is performed by the users as a final checkpoint before signing off from a
vendor. Let us take an example of the handheld barcode scanner. Let us assume that a
supermarket has bought barcode scanners from a vendor. The supermarket gathers a team of
counter employees and make them test the device in a mock store setting. By this procedure,
the users would determine if the product is acceptable for their needs. It is required that the
user acceptance testing "pass" before they receive the final product from the vendor.
Software Tools
A software tool is a programmatic software used to create, maintain, or otherwise
support other programs & applications. Some of the commonly used software tools in HCI
are as follows −
 Specification Methods − The methods used to specify the GUI. Even though these are
lengthy and ambiguous methods, they are easy to understand.
 Grammars − Written Instructions or Expressions that a program would understand.
They provide confirmations for completeness and correctness.
 Transition Diagram − Set of nodes and links that can be displayed in text, link
frequency, state diagram, etc. They are difficult in evaluating usability, visibility,
modularity and synchronization.
 Statecharts − Chart methods developed for simultaneous user activities and external
actions. They provide link-specification with interface building tools.
 Interface Building Tools − Design methods that help in designing command languages,
data-entry structures, and widgets.
 Interface Mockup Tools − Tools to develop a quick sketch of GUI. E.g., Microsoft
Visio, Visual Studio .Net, etc.
 Software Engineering Tools − Extensive programming tools to provide user interface
management system.
 Evaluation Tools − Tools to evaluate the correctness and completeness of programs.
*********************************************************************
3 | Page
 HUMAN COMPUTER INTERACTION
HCI and software engineering
 Software engineering provides a means of understanding the structure of the design
process, and that process can be assessed for its effectiveness in interactive system
design.
 Usability engineering promotes the use of explicit criteria to judge the success of a
product in terms of its usability.
 Iterative design practices work to incorporate crucial customer feedback early in the
design process to inform critical decisions which affect usability.
 Design involves making many decisions among numerous alternatives. Design rationale
provides an explicit means of recording those design decisions and the context in which
the decisions were made.
Software Life cycle models
In the development of a software product, we consider two main parties: the customer
who requires the use of the product and the designer who must provide the product.
Typically, the customer and the designer are groups of people and some people can be both
customer and designer. It is often important to distinguish between the customer who is the
client of the designing company and the customer who is the eventual user of the system.
These two roles of customer can be played by different people. The group of people who
negotiate the features of the intended system with the designer may never be actual users of
the system. This is often particularly true of web applications. In this chapter, we will use the
term ‗customer‗ to refer to the group of people who interact with the design team and we
will refer to those who will interact with the designed system as the user or end-user.
The Waterfall Method

4 | Page
 HUMAN COMPUTER INTERACTION
Phases of software life cycle:
 Requirement specification
 Architectural Design
 Detailed Design
 Coding and Unit testing
 Integration and testing
 Operation and Maintenance
Requirements specification:
Designer and customer try capture what the system is expected to provide can be
expressed in natural language or more precise languages, such as a task analysis would
provide.
Architectural design:
High-level description of how the system will provide the services required factor system
into major components of the system and how they are interrelated needs to satisfy both
functional and non-functional requirements
 Present functionality through a familiar metaphor.
 Provide similar execution style of analogous operations in different applications.
 Organize the functionality of a system to support common user tasks.
 Make invisible parts and processes visible to the user.
Detailed design:
Refinement of architectural components and interrelations to identify modules to be
implemented separately the refinement is governed by the non-functional requirements
Coding and Unit Testing:
The detailed design for a component of the system should be in such a form that it is
possible to implement it in some executable programming language. After coding, the
component can be tested to verify that it performs correctly, according to some test criteria
that were determined in earlier activities.
Integration and testing:
Testing is done to ensure correct behavior and acceptable use of any shared resources.
Maintenance:
After product release, all work on the system is considered under the category of
maintenance, until such time as a new version of the product demands a total redesign or the
product is phased out entirely.
Validation and Verification:
Verification - Designing the product right
Validation - Designing the right product

5 | Page
 HUMAN COMPUTER INTERACTION

The formality gap between the real world and structured design

Feedback from maintenance activity to other design activities

The uni-directional movement of the waterfall model of Software Engineering shows

that every phase depends on the preceding phase and not vice-versa. However, this model is
not suitable for the interactive system design.
The interactive system design shows that every phase depends on each other to serve
the purpose of designing and product creation. It is a continuous process as there is so much
to know and users keep changing all the time. An interactive system designer should
recognize this diversity.

*********************************************************************

6 | Page
 HUMAN COMPUTER INTERACTION

GUI Design and Aesthetics

Graphic User Interface (GUI) is the interface from where a user can operate programs,
applications or devices in a computer system. This is where the icons, menus, widgets, labels
exist for the users to access.
It is significant that everything in the GUI is arranged in a way that is recognizable
and pleasing to the eye, which shows the aesthetic sense of the GUI designer. GUI aesthetics
provides a character and identity to any product.

*********************************************************************

Prototyping Techniques
 Iterative design overcomes inherent problems of incomplete requirements •
 Prototypes
o simulate or animate some features of intended system
o different types of prototypes
 throw-away
 incremental
 evolutionary
 Management issues
o time
o planning
o non-functional features
o contracts
Throw-way:
The prototype is built and tested. The design knowledge gained from this exercise is
used to build the final product, but the actual prototype is discarded.

Throw-away prototyping within requirements specification

Incremental prototype:
The final product is built as separate components, one at a time. There is one overall
design for the final system, but it is partitioned into independent and smaller components.
The final product is then released as a series of products, each subsequent release including
one more component

7 | Page
 HUMAN COMPUTER INTERACTION

Incremental prototyping within the life cycle

Evolutionary prototype: Here the prototype is not discarded and serves as the basis for the
next iteration of design.

Evolutionary prototyping throughout the life cycle

Techniques for prototyping (prototyping in practice):
Storyboards
The simplest notion of a prototype is the storyboard, which is a graphical depiction of the
outward appearance of the intended system
Limited functionality simulations
Some part of system functionality provided by designers tools like HyperCard .
High-level programming support
Hyper Talk was an example of a special-purpose high-level programming language which
makes it easy for the designer to program certain features of an interactive system
Warning about iterative design

*********************************************************************

8 | Page
 HUMAN COMPUTER INTERACTION
Model-based design and Evaluation
Evaluation: Tests the usability, functionality and acceptability of an interactive system.
 Occurs in laboratory, field/or in collaboration with users.
 Evaluate both design and implementation.
 Should be considered at all stage in the design life cycle.
Evaluation may be done by:
1. The designer for assessing early design and prototypes.
2. Usability expert of the working prototype or implementation
GOALS OF EVALUATION
1. To assess the extent and accessibility of the system’s functionality.
2. To assess users’ experience of the interaction.
3. To identify any specific problems with the system.
Evaluation through expert analysis
4 approaches to expert analysis considered:
1. Cognitive Walkthrough
2. Heuristic evaluation
3. The use of model
4. Use of previous work
*********************************************************************

Introduction to different types of models

Types of Models
Two broad categorization of the models used in HCI
 Descriptive/prescriptive models: some models in HCI are used to explain/describe user
behavior during interaction in qualitative terms. An example is the Norman’s model of
interaction (to be discussed in a later lecture). These models help in formulating
(prescribing) guidelines for interface design
 Predictive engineering models: these models can “Predict” behavior of a user in
quantitative terms. An example is the GOMS model (to be discussed later in this
module), which can predict the task completion time of an average user for a given
system.
Predictive Engineering Models
The predictive engineering models used in HCI are of three types
1. Formal (system) model
2. Cognitive (user) models
3. Syndetic (hybrid) model
1. Formal (System) Model
In these models, the interactive system (interface and interaction) is represented using ‘formal
specification’ techniques.
For example, the interaction modeling using state transition networks.

9 | Page
 HUMAN COMPUTER INTERACTION
Essentially models of the ‘external aspects’ of interactive system (what is seen from outside)
Interaction is assumed to be a transition between states in a ‘system state space’
 A ‘system state’ is characterized by the state of the interface (what the user sees) It is
assumed that certain state transitions increase usability while the others do not.
 The models try to predict if the proposed design allows the users to make usability-
enhancing transitions.
 By applying ‘reasoning’ (manually or using tools) on the formal specification

2. Cognitive (User) Models

These models capture the user’s thought (cognitive) process during interaction
For example, a GOMS model tells us the series of cognitive steps involved in typing a word
Essentially models are the ‘internal aspects’ of interaction (what goes on inside user’s mind)
Usability is assumed to depend on the ‘complexity’ of the thought process (cognitive
activities)
 Higher complexity implies less usability Cognitive activities involved in interacting
with a system is assumed to be composed of a series of steps (serial or parallel).
 More the number of steps (or more the amount of parallelism involved), the more
complex the cognitive activities are.
 The models try to predict the number of cognitive steps involved in executing
‘representative’ tasks with the proposed designs.
 Which leads to an estimation of usability of the proposed design.

3. Syndetic (Hybrid) Model

HCI literature mentions one more type of model, called ‘Syndetic’ model. In this model, both
the system (external aspect) and the cognitive activities (internal aspect) are combined and
represented using formal specification

Cognitive Models in HCI

 Although we said before that cognitive models are models of human thinking process,
they are not exactly treated as the same in HCI.
 Since interaction is involved, cognitive models in HCI not only model human
cognition (thinking) alone, but the perception and motor actions also (as interaction
requires ‘perceiving what is in front’ and ‘acting’ after decision making).
Thus cognitive models in HCI should be considered as the models of human perception
(perceiving the surrounding), cognition (thinking in the ‘mind’) and motor action (result of
thinking such as hand movement, eye movement etc.

In HCI, broadly three different approaches are used to model cognition

1. Simple models of human information processing
2. Individual models of human factors
3. Integrated cognitive architectures

10 | Page
 HUMAN COMPUTER INTERACTION
1. Simple Models of Human Information Processing
These are the earliest cognitive models used in HCI. These model complex cognition as a
series of simple (primitive/atomic) cognitive steps:
 Most well-known and widely used models based on this approach is the GOMS
family of models.
 Due to its nature, application of such models to identify usability issues is also known
as the “cognitive task analysis (CTA)”

2. Individual Models of Human Factors

In this approach, individual human factors such as manual (motor) movement, eye
movement, decision time in the presence of visual stimuli etc. are modeled. '
 The models are basically analytical expressions to compute task execution times in
terms of interface and cognitive parameters.
 Examples are the Hick-Hyman law, the Fitts’ law

3. Integrated Cognitive Architectures

Here, the whole human cognition process (including perception and motor actions) is
modeled.
 Models capture the complex interaction between different components of the
cognitive mechanism unlike the first approach.
 Combines all human factors in a single model unlike the second approach.
 Examples are MHP, ACT-R/PM, Soar

*********************************************************************

GOMS family of models (KLM and CMN-GOMS),

GOMS stands for Goals, Operators, Methods and Selection Rules

The family consists of FOUR models:
 Keystroke Level Model or KLM
 Original GOMS proposed by Card, Moran and Newell, popularly known as CMN-
GOMS.
 Natural GOMS Language or NGOMSL.
 Cognitive Perceptual Motor or (CPM) GOMS [also known as Critical Path Method
GOMS]

Keystroke Level Model (KLM)

The model was proposed way back in 1980 by Card, Moran and Newell; retains its popularity
even today This is the earliest model to be proposed in the GOMS family (and one of the first
predictive models in HCI)

11 | Page
 HUMAN COMPUTER INTERACTION
KLM - Purpose
The model provides a quantitative tool (like other predictive engineering models)
 The model allows a designer to ‘predict’ the time it takes for an average user to
execute a task using an interface and interaction method.
For example, the model can predict how long it takes to close this PPT using the “close” menu
option.

How KLM Works

 In KLM, it is assumed that any decision-making task is composed of a series of
‘elementary’ cognitive (mental) steps that are executed in sequence. These
‘elementary’ steps essentially represent low-level cognitive activities, which cannot
be decomposed any further.
 The method of breaking down a higher-level cognitive activity into a sequence of
elementary steps is simple to understand, provides a good level of accuracy and
enough flexibility to apply in practical design situations.
The Idea of Operators
To understand how the model works, we first have to understand this concept of ‘elementary’
cognitive steps:
 These elementary cognitive steps are known as operators.
 For example, a key press, mouse button press and release etc.
 Each operator takes a pre-determined amount of time to perform.
 The operator times are determined from empirical data (i.e., data collected from
several. users over a period of time under different experimental conditions).
 That means, operator times represent average user behavior (not the exact behavior of
an individual).
 The empirical nature of operator values indicate that, we can predict the behavior of
average user with KLM.
 The model cannot predict individual traits.

There are seven operator defined, belonging to three broad groups

1. Physical (motor) operators
2. Mental operator
3. System response operator

1. Physical (Motor) Operators

There are five operators that represent five elementary motor actions with respect to an
interaction
K The motor operator representing a key-press
B The motor operator representing a mouse-button press or release
P The task of pointing (moving some pointer to a target)
H Homing or the task of switching hand between mouse and keyboard
12 | Page
 HUMAN COMPUTER INTERACTION
D Drawing a line using mouse (not used much nowadays)
2. Mental Operator
Unlike physical operators, the core thinking process is represented by a single operator M,
known as the “mental operator” Any decision-making (thinking) process is modeled by M.
3. System Response Operator
KLM originally defined an operator R, to model the system response time (e.g., the time
between a key press and appearance of the corresponding character on the screen). When the
model was first proposed (1980), R was significant. However, it is no longer used since we are
accustomed to almost instantaneous system response, unless we are dealing with some
networked system where network delay may be an issue.

(CMN)GOMS
CMN stands for Card, Moran and Newell –the surname of the three researchers who proposed it.
KLM vs (CMN) GOMS
 In (CMN) GOMS, a hierarchical cognitive (thought) process is assumed, as opposed
to the linear thought process of KLM
 Both assumes error-free and ‘logical’ behavior
 A logical behavior implies that we think logically, rather than driven by emotions

(CMN) GOMS –Basic Idea:

(CMN) GOMS allows us to model the task and user actions in terms of four constructs (goals,
operators, methods, selection rules).
Goals: represents what the user wants to achieve, at a higher cognitive level. This is a
way to structure a task from cognitive point of view. The notion of Goal allows us to
model a cognitive process hierarchically.
Operators: elementary acts that change user’s mental (cognitive) state or task
environment. This is similar to the operators we have encountered in KLM, but here the
concept is more general.
Methods: these are sets of goal-operator sequences to accomplish a sub-goal.
Selection rules: sometimes there can be more than one method to accomplice a goal.
Selection rules provide a mechanism to decide among the methods in a particular context
of interaction.
Operator in (CMN) GOMS:
 As mentioned before, operators in (CMN) GOMS are conceptually similar to
operators in KLM.
 The major difference is that in KLM, only seven operators are defined. In (CMN)
GOMS, the notion of operators is not restricted to those seven.
 The modeler has the freedom to define any “elementary” cognitive operation and use
that as operator.
The operator can be defined
 At the keystroke level (as in KLM).

13 | Page
 HUMAN COMPUTER INTERACTION
 At higher levels (for example, the entire cognitive process involved in “closing a file
by selecting the close menu option” can be defined as operator).
 (CMN) GOMS gives the flexibility of defining operators at any level of cognition and
different parts of the model can have operators defined at various levels.
*********************************************************************

Fitt’s law and and Hick-hyman’s law

Individual Models of Human Factors –I
In the GOMS family, KLM and (CMN) GOMS, are simple models of human information
processing. They are one of three cognitive modeling approaches used in HCI. A second type of
cognitive models used in HCI is the individual models of human factors
Two well-known models belonging to this category:
 The Fitts’ law: a law governing the manual (motor) movement.
 The Hick-Hyman law: a law governing the decision making process in the presence of
choice.
Fitts’ Law:
 It is one of the earliest predictive models used in HCI (and among the most well-known
models in HCI also).
 First proposed by PM Fitts (hence the name) in 1954.
 The Fitts’ law is a model of human motor performance.
 It mainly models the way we move our hand and fingers.
 A very important thing to note is that the law is not general; it models motor performance
under certain constraints.
Task Difficulty:
 The key concern in the law is to measure “task difficulty” (i.e., how difficult it is for a
person to acquire, with his hand/finger, a target at a distance D from the hand/finger’s
current position)
 Note that the movement is assumed to be rapid, aimed and error-free

Fitts, in his experiments, noted that the difficulty of a target acquisition task is related to
two factors:
 Distance (D): the distance by which the person needs to move his hand/finger. This is
also called amplitude (A) of the movement. The larger the D is, the harder the task
becomes
 Width (W): the difficulty also depends on the width of the target to be acquired by the
person As the width increase, the task becomes easier.

Measuring Task Difficulty:

 The qualitative description of the relationships between the task difficulty and the target
distance (D) and width (W) cannot help in “measuring” how difficult a task is.
 Fitts’ proposed a ‘concrete’ measure of task difficulty, called the “index of difficulty”

14 | Page
 HUMAN COMPUTER INTERACTION
(ID)
 From the analysis of empirical data, Fitts’ proposed the following relationship between
ID, D and W.
ID = log2 (D/W+1) [unit is bits]

Throughput
 Fitts’ also proposed a measure called the index of performance (IP), now called
throughput (TP)
 Computed as the difficulty of a task (ID, in bits) divided by the movement time to
complete the task (MT, in seconds)
TP = ID/MT bits/S

The Hick-Hyman Law

While Fitts’ law relates task performance to motor behavior, there is another law popularly
used in HCI, which tell us the “reaction time” (i.e., the time to react to a stimulus) of a person
in the presence of “choices”. The law is called the Hick-Hyman law, named after its
inventors.

Hick’s Law (or the Hick-Hyman Law) is named after a British and an American psychologist
team of William Edmund Hick and Ray Hyman. In 1952, this pair set out to examine the
relationship between the number of stimuli present and an individual’s reaction time to any
given stimulus. As you would expect, the more stimuli to choose from, the longer it takes the
user to make a decision on which one to interact with. Users bombarded with choices have to
take time to interpret and decide, giving them work they don’t want.

The formula for Hick’s Law is defined as follows:

*********************************************************************

15 | Page