HCI Midterm Masterlist

Produced with ♡ by Sarah/Xifan

October 2022

1
 Table of Contents

1 Introduction 3

2 Understanding the User and their Needs 4

2.1 Contextual Inquiry . . . . . . . . . . . . . . . . . . . . . . . . 5
2.2 Task Analysis and other Needfinding Techniques . . . . . . . . 7
2.3 Interviews . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.4 FAQ on Needfinding Techniques . . . . . . . . . . . . . . . . . 9
2.5 Principles of Survey Design . . . . . . . . . . . . . . . . . . . 10

3 Prototyping 12
3.3 More on Prototyping . . . . . . . . . . . . . . . . . . . . . . . 13

4 Experiments and How to Evaluate Them 14

5 Stats/Quantitative Methods 17
5.1 Helpful Stats Tools . . . . . . . . . . . . . . . . . . . . . . . . 19

6 General Overview Practice 21

2
 Introduction

Why did I make this?: I realized that given it is fall break and we have a homework
assignment due on the 14th, it would be great to write up a review guide for the exam that
includes all the topics that could show up on it, so not the facial recognition content (as I
don’t think it will be on the exam). This guide condenses potentially hundreds of slides into
a 21-page document.

How do I quickly find a topic?: Use the Table of Contents, which has links to each
topic/subtopic, or you can try to search by keyword using your browser or document reader
application.

How can I use this to study?: This doc is meant to be used in tandem with (not
replace) the presentations from class and the Sample Midterm (both linked on the class site).
An idea could be to read through this guide and then test yourself on the sample midterm.
See if you can answer all the questions without looking and then verify your answers. This
document is not an answer key for the sample, but I think you can figure out all the answers
if you read this. I include practice questions throughout the doc but there is no answer key
as they are meant to get you thinking about the ideas/topics more generally.

Where can I contact the author?: Email [email protected] for any spelling,
grammar, and factual errors, or if you just want to give me a suggestion for something to
add to the doc.

Important Note! DandyHacks is the University of Rochester’s annual

42-hour long hackathon! It will be happening October 14-16. Partici-
pants, colloquially called hackers, develop a project over the course of the
event and compete for prizes in three distinct tracks/themes, this year being
Video Game Development, Community, and Productivity. During the event,
there are many amenities that hackers can take advantage of, such as free
food, free swag, workshops, games of all sorts, and so much more! Learn
more about this year’s annual Dandyhacks at https://dandyhacks.net/
and follow the Instagram @dandyhacks! The Computer Interest Floor
(CIF) will be hosting a workshop on Git while the Google Student
Developer Club will be hosting a workshop on frontend webdev. If
you participate and present in the hackathon, you will be given extra credit
for the class. Professor Hoque says it will stand in for/replace a pop quiz
grade. Links for CIF: https://cif.rochester.edu/ and Google Student
Developer Club: https://ccc.rochester.edu/gdsc/home/

3
 Understanding the User and their Needs

Developers are not the typical user...

Why does this matter?

Even though we all come from different backgrounds, what are some shared
frames of reference we have? To start, we all have experience to some signifi-
cant extent with the intimate parts of software and how to use the machines
that run them. Through practicing programming, we become acclimated
to thinking through how a computer executes a program and dealing with
the frustrations of software that does not function how one wants it to. We
use specialized terminology when interacting with peers, and we judge the
success or failure of a project by certain metrics and values. This can even
affect the way we approach or think about non-technical problems or topics.
Sometimes this is taken at face value as it is easy to overlook. When there
is a non-technical end user among us, how do we then find and address their
needs?

Where do we start?
Consider how users WORK on the task or problem at hand. Tools
for this include task analysis, contextual inquiry, interviews, self-reporting,
and observation. Consider how users THINK using tools from cognitive
science/psychology and observation. Consider how users INTERACT with
prototypes, beta tests, etc., through rigorous observation.

Think about this

This is written using LaTeX. If you have used LaTeX, think about the UIs
and UX you have encountered while text editing with LaTeX. What do you
like and not like about them? Do you think your UX is good overall? Why
do you think more people aren’t using it compared to the large population of
people who do text-editing? If you don’t know about LaTeX, look up LaTeX
and Overleaf. Are you interested in learning how to use it?

4
 The 4 Components of Contextual Inquiry and their
Principles

Context: Way of understanding users’ needs and work

practices

Partnership: Investigator is a humble observer, the

participant is the knowledgeable informant. The master
does the work and talks about it while working–investigator
interrupts to ask questions as they go.

Interpretation: Convert raw information into inter-

pretation

Focus: You can’t focus on every detail, so you need

a focus to filter out irrelevant data and show you the
relevant data...

Think ‘Where’, ‘How’, and ‘What’ to get to the

‘Why’

5
 Component Principles

Context Principles: go to the workplace/location and see the work as it

unfolds, people will summarize, but the investigator wants details → keep it
concrete if people start abstracting (example is asking for an email if someone
mentions the reports are distributed through email)

Interpretation Principles: facts are only the starting point, design

based on interpretation

Two important aspects: Validate and Rephrase

• Share interpretations to check your reasoning
• People will be uncomfortable until the phrasing is right

Focus Principles: interviewer needs data about specific kind of work,

and should respect triggers (flags to change focus)
• “Steer” conversation to stay on useful topic
• Pay attention to environment and interlocutor(s)

Real-world scenario
The focus here is not to find the right or wrong answer but instead to
think through the concept: What type(s) of contextual inquiry would you
use in your investigation if you were part of a team aiming to develop a
data analytics application for the public health department? Why would you
select this approach? Suppose that instead you are aiming to develop it for
computer scientists/programmers. Would this change your response? Why
or why not?

6
 Task Analysis and other Needfinding Techniques

Task Analysis: Task Analysis is used to find out what the potential
userbase is, the tasks they perform, their existing workflow and practices,
and to develop realistic use case scenarios. By doing this, you empower
yourself to test out new ideas without having to start to code.

Benefits of Task Analysis

• Helps match system to user needs
• Helps match system with user tasks
• Enables you to think of the context of the use cases
• Just using guidelines are too vague to build a system on

Observation: Done by spending time with the potential user-base to

discover their values, goals, behaviors, their current activities, and similarities
and differences between group members. Some important things to note
are that observation requires the participants to be informed and want to
participate, and that investigators stay mindful of the Hawthorne effect,
which is when study participants modify their behavior when they know they
are being observed.

Process mapping: This is a tool for developing a map model of the

experience as a whole, and asks you to start with the experience you are
considering and then identify the first stage in the process. Once you do this,
you step through the whole process until the end. Then, you can analyze the
map and identify relationships between the steps.

Camera Studies: In this kind of study, cameras are used to document

how a user interacts with whatever you are studying and to better understand
their experience by literally seeing through their own physical point of view.

7
 Interviews

Lead User Interviews: This kind of interview focuses on early adopters

or ’fanatics’ who enjoy keeping on top of technology trends. It is helpful to
conduct this to explore the future of use cases and it also allows you to ask
open-ended and blunt questions. By engaging with this type of user, you also
expose potential early adopters to your technology and engagement with the
developers/designers.

Expert Interviews: This kind of interview is used for when domain

experts are needed or helpful to better understand the users, use cases, and
general topic(s) a technology is centered around. Some benefits include: helps
you quickly understand the context your system will be used in, allows you to
educate yourself on the topic, gather ideas for needs, and better understand
potential decision-makers in your system’s field.

Laddering/5 why Interviews: This kind of interview is used to really

dig into a specific topic or issue found by earlier needfinding tools. The
participants are asked for their reasoning behind the opinions they hold, and
are asked why they agree with those lines of reasoning. This can be annoying,
but if participants agree, then it can be very fruitful.

Culture Context Interview: This kind of interview can be used at the

end of another type of interview, after building rapport with participants. It
is meant to be general in scope and enable you to understand the participant’s
background and culture. Intent is important here (try to connect it to the
use case/technology).

How would you begin to fix Dr. Chatbot?

Look over slides 13-14 on the Needfinding presentation (link on the class
site) and consider how focusing on these questions could have benefited the
team developing Dr. Chatbot. Which of these do you think the project
designers did not strongly consider? Pick two kinds of interviews and consider
how you could incorporate these questions into an interview setting.

8
 FAQ on Needfinding Techniques

What are aspects of good needfinding tools?

Elements of good needfinding techniques include the presence of both struc-
ture and flexibility. Investigators should have a plan, but be willing to incor-
porate new ideas into the plan. They should help you develop stories about
users and use cases through assets such as quotes and images, and potentially
audio/video if you believe the benefits outweigh the drawbacks.

What are examples of ah-ha moments?

Signals that your needfinding strategies are going well include: (1) de-
veloping implicit insights, (2) finding a surprising or missing element, (3)
understanding why people do unusual things, (4) understanding the context
behind contradictions, (5) predicting what participants will say to a ques-
tion, (6) developing rich stories, and (7) developing excitement about what
you have learned.

What are some general drawbacks?

Users are not always right about technology or even their own wants and
needs. We aim to build tools that help them do what they want even if
they can’t say it. However, we have to be careful to not speak over them.
Needfinding without prototyping will hinder one’s progress.

What are the six stages of an interview?

1. Intro
2. Kickoff the process
3. Build rapport
4. Main experiment
5. Reflection
6. Wrap-up

9
 Principles of Survey Design

Designing surveys is hard-perhaps so much so that governments and corpo-

rations frequently get it wrong. Surveying is a great tool but learning about
potential drawbacks and strategies to avoid biased data is important.

Keeping it simple: Core principles include (1) make the survey interest-
ing enough to be worth engaging, (2) use sentences that are clear and concise,
and (3) make the survey short, even if you find yourself wanting to add more.

Avoiding biased responses: People may respond to a survey in a man-

ner that is inaccurate because they are afraid of appearing bad, incompetent,
etc., or out of a desire to respond in the way they think the analysts want
to hear (or not hear, if they have strong negative feelings). Have you ever
been in a class where a teacher asks the class if everyone is familiar with xyz
technology? Did you notice the number of hands or yeses increasing over
time despite the question not being complex?

Wording questions correctly: Ideally, you should avoid making ques-

tions that have two or more parts to them. If the parts in question are
very important and related, then you can place them in separate questions
near each other. Avoid using questions with multiple negations (ex. Do
you think the menu should not be un-movable, and why not?). Don’t make
it sound like a MATH 150 word problem-even if you enjoy them, users most
likely don’t. On a similar note, avoid jargon unless it is domain knowledge
all users understand, and don’t use needlessly complex vocabulary. Another
aspect of wording questions well is avoiding terms that are clearly sub-
jective. Examples include the term frequently, often, probably, perhaps, and
maybe.

Ordering Questions: General questions should come before specific ones,

and personal questions at the end. The ordering within questions also matters
as the order can prime the respondents into thinking from a certain frame of
reference which can bias the responses.

10
 Additional Principles: Redundancies can be introduced to help gauge
the accuracy and consistency of responses, but should not be too redundant
to the point the survey becomes boring. Providing explanations to why you
are doing the survey and what kind of impact it may have on the technology
would be helpful and engaging. Providing a progress bar on the survey can
also help the participant visualize their completion. Having at least one
open-ended question (short response) can provide more detailed feedback.
The survey designers should also test the survey before releasing it and fix
any mistakes.

Limitations of Surveys: Understanding the limitations or drawbacks of

surveys will enable you to develop stronger surveys and interpret the data
more accurately. A major limitation of surveys is that you are receiving
self-reported data and explanations. This can be subject to biases as stated
above or may just unintentionally leave out potentially important informa-
tion. Some people might not read all the questions and still answer each
one, which can be because of survey fatigue. You may only capture people’s
feelings at the end, and still struggle to interpret what exactly they indicate.

Apply your knowledge

Think of a survey you have filled out, whether you found it from some
organization or from another student (such as the Dr. Chatbot survey).
Name one strength of their survey techniques and one weakness. How would
you suggest mitigating the weakness? Now that you understand the principles
of survey design, will you answer surveys any differently than you previously
have? The school has us do course evaluations for our classes at the end
of the semester. If you have done them and are familiar with the question
design, to what extent do you think the feedback they receive is useful for
judging a professor’s performance?

11
 Prototyping
What is a prototype?: Prototypes are representations of a design made
before final artifacts exist.

What are the three main steps of iterative design, and why are
they important?: Prototype, Review, and Refine. These are not strictly
sequential as multiple cycles of the process should occur before the first com-
plete release. They are important because iteration should be occurring
throughout the entire process.

Why is prototyping important?: You can get feedback earlier and at

lower cost than if you waited until you had a working example. You can
experiment with different and alternative designs, and they are relatively
easier to change if needed. They are also easier to abandon or trash than if
you had to invest coding time into an example which can build an emotional
connection between you and the test program.

What are some aspects of prototypes?: Prototypes can try to model

to varying extents: Role, Look and Feel, and Implementation.

What does fidelity mean?: A low fidelity prototype omits significant

details while a high fidelity prototype is closer to a finished product. Fidelity
has multiple dimensions, including breath and depth. Breadth is the % of
features covered overall, while depth addresses the degree of the functionality
of the features. Look is the prototype’s appearance/design, and Feel is the
input method. For example, a paper prototype of a website has a different
Feel than an online interactive wireframe.

12
 More on Prototyping
Storyboarding: This can be performed prior to prototyping and by uti-
lizing data from needfinding techniques. Storyboarding gives you an idea of
the tasks and scenarios that you will need to support. Basically, it makes
you think about your use cases in detail.

Paper Prototypes: The advantages of paper prototyping include speed,

ability to change, greater participation from non-technical team members,
and that they focus attention toward the big picture instead of overthinking
design details or eliciting nitpick feedback from users. Paper prototyping is
not simply sketching, as it is an interactive model that involves simulat-
ing a more natural interaction between the users and the prototype team.
Although it is low fidelity in look and feel, it is high fidelity in depth. The
team should designate members to simulate the computer, facilitate the
interaction by guiding them through the test, and the observer should
be silent and take notes. You can test out a lot of things (conceptual
model/user understanding, functionality, navigation/task flow, ter-
minology, and screen contents) through paper prototyping, but you can’t
test everything. This includes look, feel, response time, whether small
changes will be noticed, and to what extent users explore.

Other Kinds of Prototyping: Form prototypes are focused on appear-

ance and visual design. They are meant to look good and accurate, but
functionality is limited. A function prototype is the reverse. The design is
minimal but the functionality is meant to be strong and closer to the final
product. A Wizard of Oz prototype aims to simulate the software by means
of using humans. This can be used to simulate technology that has not been
developed yet. This can be difficult as the wizard has to only operate as if a
computer would, and you have two UIs to deal with (user, wizard).

An analog Dr. Chatbot?

Let’s say you are part of a team iterating on Dr. Chatbot. Would you
want to paper prototype, and why or why not?

13
 Experiments and How to Evaluate Them
How do we go beyond surveying our users about our systems? How do
we know for sure our system is better than another or how exactly it needs
to be improved? This section covers experiment design and analysis of both
qualitative and quantitative data.

Experiment Design: When an experiment uses a between-subjects de-

sign, half of the participants will use one interface/system, and the other half
uses another. In terms of advantages, this provides a way to prevent carryover
effects such as getting better at learning or performing a task due to practice,
or getting fatigued from doing so many experiment tasks over a short period.
However, some disadvantages are that you need more participants involved in
the study to match the same statistical capability as within-subjects experi-
ments, and it is even more crucial to make sure that the different groups do
not differ from each other in a significant way to avoid biasing results. While
within-subjects designs may avoid these problems, they add a new problem
of their own. This is that every participant uses everything, so you have to
worry about the order in which participants complete tasks. You can use
counterbalancing to mitigate the weaknesses of study designs. An exam-
ple talked about in class is the Latin Square, which ensures that the order in
which the participants experience the different conditions is varied and also
that each condition is only used once in each order.

Making test plans:

• Purpose of test
• Problem statement/test objectives
• Participant profile (inclusion/exclusion criteria)
• Method/technique to be used
• List of tasks to be done
• Test environment and materials/resources
• Experimenter’s role(s)
• Evaluation measures to be used

14
 Types of Studies: Studies can be categorized according to the type
of data they are collecting and analyzing: qualitative, quantitative, and
both. Studies can differ in their scale: pilot (small number of participants),
exploratory (small to moderate), and large-scale (many participants). They
can also be categorized by the time frame used during the study. A cross-
sectional study is one that takes place at a single time. This can mean the
data is affected by fluctuations or chance. A longitudinal study takes place
over a longer scale of time. This mitigates the previous effects but can have
its own weaknesses (people dropping out, differences in memory retention,
keeping a constant experiment setup, and maturation threat (as participants
grow, these changes can affect the test results).

Variables: A variable is like the data type we use all the time: its an entity
that can take on different values (it varies). An attribute is a specific value of
a variable. Independent variables are chosen ahead of time for the experiment
and are reflected through the variance in the test (for example the light/dark
mode UI), and a dependent variable’s values depend on the other variables
and conditions. It is what the designer is interested in measuring/observing.
Controls are things you want to stay consistent (virtually everything else in
the experiment, to avoid biasing results).

Qualitative Analysis: When you do this, you have to keep in mind

that the data doesn’t really exist to be analyzed until it is written down.
It involves the process of transcription and interpretation. Content analysis
breaks down writing into units by applying a ruleset. There are many things
to consider when doing this: themes, what gets counted, units of analysis
(by word, paragraph, or theme), and what you should even be analyzing
(frequency, direction, intensity, and space). Manifest content is overt and
visible, while latent content is analyzed symbolically and semantically.

Sampling: Sampling techniques are discussed in the context of the popu-

lation in the study. The theoretical population is a generalized concept (the
group(s) you wish to have the study results be valid for), and the accessible
population is the subset of the population that is actually available for an
experiment. A sampling frame is the list of the accessible population you can
draw your sample from, and the sample is the group that will be participat-

15
ing in your study. You can sample through RNG (simple random sampling),
through doing RNG on homogenous subgroups (stratified random sampling),
through selecting every nth individual (systematic random sampling), or by
randomly selecting clusters to sample (clsuter random). You can also use a
combination of them.

Testing Dr. Chatbot 2.0

Have you ever participated in a study? What did you have to do to get
selected? What kind of study design did the experimenters use, and do
you think the way the data was collected had a method that utilized proper
techniques? If you had to set up an experiment to test your new Dr. Chatbot
prototype after having completed needfinding, how would you try to show
that it is systematically better than the old one?

16
 Stats/Quantitative Methods

Why is data important?

Data helps us evaluate the validity/strength of claims by using quantitative
methods. In addition, the more data we add, the more accurate our results
can become, as the confidence interval might get narrower (and therefore
become more confident).

What is the problem with just taking the mean?

The mean leaves out a lot of context that is necessary to interpret the data
(ex. median, mode), but most importantly, does not inform you about the
variance between individual data points. For that, the standard deviation
helps you understand the spread of a data set.

How do we connect the sample mean to the population

mean?
When we calculate the mean and SD for our data set, what we are actu-
ally calculating is the sample mean and SD. We are trying to generalize our
analysis to the population which our results should be valid for. A confidence
interval calculation is used to determine the range of the unknown population
mean in relation to the sample mean, given a certain level of confidence. 95%
is used as a benchmark in part because it represents 1.96 SDs above/below
the mean on a standard bell curve. A narrower margin of error indicates there
is a narrow confidence interval, which means we have a relatively better sense
of the population mean.

What is a correlation coefficient?

With correlation, we are measuring the degree of the linear relationship

17
between the variables. A perfect correlation is a perfect linear relationship
(graph would be like the Big O of an exactly linear algorithm). We are
looking for positive. negative, or no correlation.

How do you perform a chi-squared test?

The null hypothesis is that the data does not deviate from the expected
frequency (no bias, random fluctuations, etc.), while the alternative hy-
pothesis is that there is some kind of systematic deviation (due to some
cause other than randomness). The steps of a chi-squared test are to: (1)
determine the expected frequencies, (2) calculate the test statistic,
and (3) compare the test statistic to the table. In the class example,
absences from work would be expected to be equally distributed throughout
the days of the week. Then, the chi-squared formula is used to calculate the
test statistic (3), and finally compared to the chart of values based on the
number of degrees of freedom (5 work days in a week - 1 = 4) and the
significance level (a = .05 for the 95% confidence).

What is the Method of Contradiction used for designing

hypotheses?
It assists in building hypothesis statements that can be proven false (falsifi-
able) and enables us to use statistical methods to attempt to do so. Statistics
can’t ever prove a hypothesis to be true, but sometimes it can prove one to
be false. A falsifiable hypothesis allows us to draw solid conclusions.

How do we use the null and alternative hypothesis to

make a conclusion?
In the class example, the null hypothesis was that there is no difference
in usage duration between light and dark mode UIs, while the alternative
hypothesis is that there is. The skeptical position is the default until the
data shows evidence otherwise. If a significant difference is shown, the null

18
hypothesis is rejected, and if there is no significance then the results are
inconclusive. The null hypothesis is NOT accepted. This is for the same
reason the null hypothesis has an = sign, because we can’t prove two things
to be the same (in an empirical test).

How do we choose the right statistical test?

To choose the right statistical test, we look at the number of Factors and
Levels, and whether the experiment was within-subject or between-subject. A
factor is an independent variable that you are testing in the study by varying
the Levels (if UI is the Factor, Dark and Light modes are the Levels). Then,
we consult the chart below to determine the statistical test we should use.

Factors Levels Within/Between Test name

1 2 Between Independent sample t-test
1 2 Within Paired sample t-test
1 >2 Between One-way ANOVA
1 >2 Within One-way Repeated Measures ANOVA

Helpful Stats Tools

Confidence interval = sample mean ± margin of error

Margin of error = z ∗ × √σ
n

n = sample size
σ = standard deviation
z ∗ = level of confidence (1.96 for 95%)

Chi-Square test statistic

n (oi − ei )2
χ2 =
X

i=1 ei

19
Chi-Square lookup table

20
 Combination of Topics

Putting it all together

Let’s suppose you wanted to develop a site for University of Rochester stu-
dents to post documents such as this, discuss the class and topics, and ask
general class questions to other students and past students. What existing
applications cover these types of functions? What needfinding techniques
would you use to decide on what to include on the site, and why would you
pick them? What would the drawbacks be and how would you try to mitigate
them? Then, select two kinds of prototypes you would do for the site. How
would you design a test to show that this site helps students better than cur-
rent tools such as Blackboard or Chegg? What kinds of populations would
you include in your study, and what kind of design would you use? Name one
weakness of that design and how you can make it better. What statistical
test would you perform on the data for this experiment? As you iterate on
the site, what kinds of populations would you first want to test the site with,
and what would be the next groups to expand to?

Stats Practice
Have you and a friend each go on a dice-rolling site and have the RNG do 50
rolls. Perform statistical calculations on the results or make a chart of your
results. Who has the results most closest to the expected values? Try it with
100 rolls. Did your answer change?

21