Industrial Marketing Management 98 (2021) 193–206

Contents lists available at ScienceDirect

Industrial Marketing Management

journal homepage: www.elsevier.com/locate/indmarman

How to design good experiments in marketing: Types, examples,

and methods
Giampaolo Viglia a, b, Ghasem Zaefarian c, Aulona Ulqinaku c, *
a
University of Portsmouth, Marketing Department, Richmond Building, PO13DE Portmouth, United Kingdom
b
University of Aosta Valley, Department of Economics and Political Science, Street Cappuccini 2, 11100 Aosta, Italy
c
University of Leeds, Leeds University Business School, Maurice Keyworth Building, Leeds LS2 9JT, United Kingdom

A R T I C L E I N F O A B S T R A C T

Keywords: In this article, we present key tenets of good experimental design and provide some practical considerations for
Experimental design industrial marketing researchers. We first discuss how experiments have the ability to assess causal claims. Next,
Causation we provide an experimental taxonomy table, which brings out the value and limitations of different types of
Randomization
experiments and maps the various goals of business marketing research within each category. Here, we pay
Field experiment
Sample size
particular attention to field experiments since they provide experimental realism by measuring respondents’
Manipulation checks actual behavior. We also provide a thorough discussion on important practical issues such as questions on
Attention checks experimental design, sample size, and how to involve business organizations in the implementing steps. The
Analyze experimental data in marketing paper concludes by stressing the importance of combining data types (e.g., field plus laboratory experiments) and
by offering methodological advice on how to analyze experimental data in marketing.

1. Introduction While experimental research remains one of the main methodolog­

ical approaches in marketing and their related disciplines such as con­
Experiments enable researchers to determine causal relationships sumer behavior research (see Simester, 2017; Viglia & Dolnicar, 2020),
between an independent variable and a dependent variable, by manip­ its application in business marketing is scant. Our assessment of studies
ulating the independent variable with a high degree of control over the published in Industrial Marketing Management over the last two decades
rest of the environment (Kirk, 2013). Assessing cause-effect relation­ suggests that a growing number of studies published in recent years are
ships is a key motivation for experimental research compared to cross- benefitting from experimental research as their core methodology (e.g.,
sectional surveys. Although this latter approach is widely used in Saab & Botelho, 2020; Zimmer, Salonen, & Wangenheim, 2020). How­
quantitative business marketing research, it can be problematic with ever, most quantitative studies in business marketing research still often
regards to endogeneity (Ullah, Akhtar, & Zaefarian, 2018; Zaefarian, employ not experimental methods. Specifically, running a systematic
Kadile, Henneberg, & Leischnig, 2017). In general, failing to establish search in Scopus of all the papers published in Industrial Marketing
causality is a serious limitation in any research study (Zellmer-Bruhn, Management in the last decade (i.e., since 2010), we identified only 39
Caligiuri, & Thomas, 2016). papers using this methodology, with a flat trend, except for a slightly
Experiments allow researchers to assess the effect of a predictor, i.e., increase in 2020 (see Fig. S1 in Online Appendix).
the independent variable, on a specific outcome, i.e., the dependent Different justifications can be offered for the paucity of experimental
variable, while controlling for other factors. As such, a key tenet of good studies in business marketing research. For example, business research is
experimental design is the accuracy of manipulation. Manipulation in an often multidisciplinary and looks at macro-level, long-term phenomena
experiment refers to the procedure through which the researcher (Zellmer-Bruhn et al., 2016). Sometimes, random assignment is simply
changes or alters the predicted cause (i.e., the independent variable) in a not a possibility because there is lack of sufficient samples to assign some
treatment group and a control group. Randomizing participants into firms to the treatment group i.e., to be exposed to the treatment, and
these groups, it is possible to investigate how this change may affect the other firms to the control group (Cuervo-Cazurra, Andersson, Brannen,
outcome (i.e., the dependent variable; Allen, 2017). Nielsen, & Reuber, 2016).

* Corresponding author.
E-mail address: [email protected] (A. Ulqinaku).

https://doi.org/10.1016/j.indmarman.2021.08.007
Received 15 February 2021; Received in revised form 13 August 2021; Accepted 18 August 2021
Available online 1 September 2021
0019-8501/© 2021 Elsevier Inc. All rights reserved.
G. Viglia et al. Industrial Marketing Management 98 (2021) 193–206

The difficulty in randomly assigning firms or senior or top-level complicated and difficult to randomly allocate firms or buyers to the
managers to experimental groups may discourage researchers in in­ treatment and control conditions in B2B research, a solution is that of
dustrial marketing from carrying experimental research. Relatedly, the randomly allocating managers and employees to these conditions
nature of the sample used in business marketing (e.g., senior or top-level instead (Hada, 2021). This way, the researcher can study the differences
managers) collides with the use does not make the use of laboratory in the outcomes across conditions (e.g., in decision-making).
experiments with a convenience sample (. However, we suggest that There are different types of experiments, from the most conventional
most of the experiments in business research can be conducted in the ones, such as field, online, and laboratory experiments, to quasi-
field with real managers. Having high experimental realism and experiments, and conjoint analysis. We discuss more in detail these
measuring actual behavior, this type of experiment provides direct in­ types of experiments in the section below.
dustry recommendations. Specifically, compared to surveys or hypo­
thetical laboratory experiments based on self-reported intentions, field 3. Different types of experiments
experiments are particularly important because people lack perfect ra­
tionality, struggle to accurately predict their own preference and Internal and external validity are often understood to be opposing
behavior, tend to misreport even unconsciously, and sometimes even forces or competing with each other in experimental designs (Schram,
construct ex-post explanations for behavior that makes sense to them 2005). Internal validity is the extent to which we can reliably conclude
(Kahneman & Egan, 2011). By measuring real behavior, field experi­ that the independent variable is the main responsible variable for the
ments close the so-called ‘attitude-behavior gap’ (see Blake, 1999; Car­ changes in the dependent variable(s) (Kirk, 2013). External validity
rington, Neville, & Whitwell, 2014). A/B testing (see Anderson & refers to the extent to which the results can be generalized across pop­
Simester, 2011) is an actionable way to run field experiments with ulations (Kirk, 2013). Table 1 presents a taxonomy table of different
businesses. Practically, companies can run field experiments through A/ types of experiments, with the advantages and disadvantages of appli­
B testing when evaluating if one technique (e.g., using one logo vs. cation of each of the categories in Business to Business (B2B), but that
another, a specific font in the contract vs. another, high vs. low levels of apply to any other marketing areas too. The types of experiments are
supplier monitoring or contract specificity, etc.) produces more favor­ presented in a continuum that goes from the maximum level of the in­
able outcomes compared to another one. ternal validity to the maximum level of external validity. Compared to
Against this backdrop, the aim of this article is to provide an conventional laboratory experiments and field experiments, experi­
explanation of the value and importance of experiment methodology in ments with increased behavioral realism are an intermediate category
business research and discuss different types and key tenets of good where there is a lower level of internal validity and a higher level of
experimental design using multiple examples. More specifically, we external validity. The last two categories presented in the table include
explain the importance of the experimental method, how to decide be­ types of studies that are sometimes referred to as experiments. The first
tween different types of experiments, how to design an experiment, the type – quasi experiments/natural data – encompasses situations where
role of manipulation and attention checks, how to determine the sample data are organic, it is not possible to randomly allocate to treatment and
size and where to possibly recruit respondents, and how to analyze and control conditions, and there is no intervention by the experimenter (e.
interpret the results of experimental data. Finally, we offer a checklist g., Garrett & Gopalakrishna, 2019; Laursen & Andersen, 2016; Ruiz &
for authors and reviewers running or evaluating experimental studies in Kowalkowski, 2014). Longitudinal experiments are a type of quasi-
marketing. experiments where the same participant (be this an organization or
single respondent) is repeatedly examined over time, investigating
2. Basics of experiments possible changes in the dependent variable at any point in time or
detecting trends (Zellmer-Bruhn et al., 2016). The second category –
Experiments are defined as “a plan for assigning experimental units conjoint analysis – includes studies where participants have to express
to treatment levels and the statistical analysis associated with the plan” their evaluation and/or ranking order for a number of carefully designed
(Kirk, 1995, p. 1). To accurately capture causality, it is important to i) attributes (e.g., Bendixen, Bukasa, & Abratt, 2004). (See Table 2.)
manipulate the independent variable by having at least one manipulated
group exposed to the treatment and one control group not exposed to the 3.1. Why conventional laboratory experiments?
treatment, ii) have a randomized design where participants are assigned
randomly to the conditions, iii) make sure that the independent variable Laboratory experiments may be feasible when field experiments
is manipulated before the measurement of the dependent variable, and appear difficult to achieve when the researcher cannot plausibly acquire
iv) test for differences in one (or more) dependent variables among the necessary control. When focusing specifically on the mechanism
conditions (Kirk, 2013). Randomization avoids respondents’ “selection” behind the effect (i.e., theory application), convenience samples, such as
into treatment so that the only difference between groups is the inter­ students, are often used to investigate a vast array of matters (Calder,
vention. The control tells what would have happened without inter­ Phillips, & Tybout, 1981; Summers, 2019), also in business research
vention (counterfactual). Since respondents are randomly assigned to (Bello, Leung, Radebaugh, Tung, & Van Witteloostuijn, 2009). While it
the two groups (or more in case of several treatment groups or several is not wrong to use student samples in many cases (e.g., when testing
control groups) and respondents’ characteristics are assumed to be general principles, consumer preferences and behavior, or personality
normally distributed, there is no reason to expect that one group would traits), it is wrong to use them as surrogates of the general population for
be different than the other. Therefore, we can expect that the effect of other matters that they are not representative of (e.g., market in­
the treatment is causal on the dependent variable. Moreover, individual vestments, delinquency, or decision-making in the B2B sphere; Flere &
characteristics that make one respondent different from the other are Lavrič, 2008). B2B studies frequently rely on executive MBA students as
spread across the groups, not allowing for these aspects to affect the their sampling frame, arguing that these students have relevant back­
whole treatment or control group producing biased differences. A ground and experience (e.g., Jap, Robertson, Rindfleisch, & Hamilton,
further beneficial aspect of experimental design is that of controlling for 2013), to overcome the student sample limitations, many studies with
– and ruling out – alternative explanations that may cause the effect of the characteristics of laboratory experiments are now conducted with a
one variable onto the other. If there are other factors which could be non-student sample (i.e., experiments with increased behavioral realism
responsible for changes in the dependent variable, we cannot be confi­ in Table 1).
dent that the presumed cause-effect relationship is correct. Laboratory studies might be used to test the mechanisms behind an
It is important to note here that there is a difference between random effect with more control, increasing the extent of internal validity of a
allocation of subjects in B2C and B2B research. Since it is fairly study and enhancing the theoretical contribution (Calder et al., 1981).

194
G. Viglia et al. Industrial Marketing Management 98 (2021) 193–206

Table 1
Types of experiments.

(2007) extend previous research by investigating the effects of person­

Table 2
alization and envelope color on response rate, speed, and quality, by
Advantages of different experimental designs.
using real behavioral data with members of the Public Relations of the
Experimental Advantages Institute’s Membership Handbook in Northern Ireland.
design
Compared to laboratory experiments, field experiments are often
Between-subject - Easier experimental setup weak when the goal is understanding mechanisms behind the effect, and
design - Simpler experimental data analysis they often involve a significant loss of control over the experimental
- Lower risk of participants understanding the purpose of the
experiment and providing biased responses
procedure. For instance, Gneezy (2017) and Putnam-Farr and Riis
- Shorter experimental sessions required (2016) underline risks to the perfect randomization of participants and
Within-subject - Smaller sample size required slight changes in incentives as possible hiccups that may challenge the
design - Greater probability of grasping true differences among smooth ongoing of the field experiment.
conditions (less noise)
- Greater statistical power to the study
- Greater alignment with most marketing theoretical mindsets 3.3. Benefits of combining field and lab experiments
Mixed design - Greater statistical power
- Less learning effects When the aim of the field experiment is to test previously established
- Less order effects
theories with greater behavioral realism, one way to approach it may be
to first conduct the experiment in a controlled setting like the labora­
Laboratory experiments have been used in B2B research (e.g., An, tory, before investing resources into the field experiment. In this case,
Kreutzer, & Heidenreich, 2020; Liang, Kale, & Cherian, 2014; Oh, Pe­ researchers can start with a laboratory experiment to first support the
ters, & Johnston, 2014). As mentioned in the taxonomy table, however, theoretical evidence and then check the external validity by going into
laboratory experiments present generalizability issues that should be the field. Alternatively, the logical approach could be to first conduct a
carefully discussed by researchers. A possible way to address these field experiment to investigate the main effects of the independent
concerns is to run a complementary field study to see whether there is variable on the dependent variable, and then follow up with a laboratory
consistency of the findings when measuring actual behavior with the experiment to investigate the potential mechanisms underpinning the
target population (e.g., McCoy & Hargie, 2007). effect or other factors that can make the relationship between variables
stronger or weaker, or even reverse (e.g., from a positive effect of the
independent variable on the dependent variable to a negative one or the
3.2. Why field experiments? opposite).
In some cases, the nature of one’s research question directly dictates
Field experiments are experiments that study the actual population the experimental context. For example, when trying to seed information
in the actual context, integrating into what is already taking place. Being about their brands and products, marketers may need to rely on social
conducted in natural settings, they are representative of the target media influential endorsers. Hence, marketers may be faced with the
population, and they allow for measuring actual behavior. Some of the challenge of identifying the most appropriate set of influencers to
findings coming from the use of field experiments in marketing question collaborate with. When considering this category of collaborators (e.g.,
previous established relationships. For instance, McCoy and Hargie Valsesia, Proserpio, & Nunes, 2020), real data from social media in

195
G. Viglia et al. Industrial Marketing Management 98 (2021) 193–206

combination with a controlled experiment may provide results that are

more valid and generalizable.

4. Improving realism and measuring actual respondents’

behavior

Experimental realism refers to the extent to which an experimental

study reproduces a real behavioral situation. The experimental realism
of a study ranges from very artificial to very realistic (Levitt & List,
2007), depending on the context used and on the choice of the inde­
pendent variable. There is a growing emphasis on collecting data that
are close to the real world and to the field of marketing (van Heerde,
Moorman, Moreau, & Palmatier, 2021). Previous research suggests that
the greater the inclusion of realism in the employed variables in an
experiment, the greater the external validity and the generalizability of Fig. 1. Experimental realism of independent and behavioral measure of the
results (Morales, Amir, & Lee, 2017). This is consistent with Galizzi and dependent variable.
Navarro-Martínez (2019) who show that, to have results that are
externally valid, there is the need to test phenomena in natural settings.
Barasch, Levine, Berman, & Small, 2014; Bruine de Bruin & Ulqinaku,
When the interaction between researcher and participant is mostly
2021; Liang et al., 2014; Sarial-Abi, Vohs, Hamilton, & Ulqinaku, 2017;
remote and anonymous, such as in online experiments, it is challenging
Ulqinaku, Sarial-Abi, & Kinsella, 2020).
to employ a realistic scenario, and it is usually easier to use a hypo­
There are some papers in B2B research where one aspect is optimal,
thetical one. However, this can be resolved by investing more efforts in
and the other aspect has some room for improvement. For instance, on
the application of an experimental realism also online, having partici­
the one hand, Taylor, Hajmohammad, and Vachon (2021), respectively,
pants invest actual effort in making choices, searching, or acquiring
have high experimental realism (IV is a vignette that presents a situation
extra information, for instance by using scenario-based manipulations
where respondents are “asked to assume a specific role and react to the
and vignettes with managers and business decision-makers (e.g., Jap
information presented as if they themselves were in that situation;”
et al., 2013; Saab & Botelho, 2020; Tangpong, Li, & Hung, 2016). For
Taylor et al., 2021, p. 4), but low on behavioral measures (DV just
instance, Saab and Botelho (2020) manipulated purchase decision
measuring intentions and recommendations and not actual behavior).
importance by creating vignettes that were randomly shown either
On the other hand, Liang et al. (2014) make use of actual behavioral
portraying a high-importance purchase decision (i.e., the purpose of the
measures for the DV, introducing an element of reality into the experi­
good/service was administrative) or a low-importance purchase deci­
ment – making decisions to launch or not launch a new product devel­
sion (i.e., the purpose of the good/service was customer use). Vignettes
opment project – but presented a low experimental realism because it
were used as manipulations of the independent variables and after being
was conducted with MBA students and in a laboratory. Generalizing
exposed to those, respondents could be assessed on the dependent var­
from these measures to strategy or policy implications still requires a bit
iables (e.g., assessing functional risks of the decision).
of a leap of faith.
Another important way to increase experimental realism is the use of
Given the resources needed to conduct field experiments, it is
technological tracking in field experiments. One example here is the
important to assure collaborations with partners in the industry for data
study conducted by Ferguson and Mohan (2020). The authors employ
collection, keeping in mind that smaller organizations usually provide
eye-tracking monitoring to investigate the effects of influencers in B2B
faster and easier collaboration agreements (Gneezy, 2017). Choi, Sun,
ads on managers’ attention, ad recall, and ad attitudes, by featuring B2B
Liu, and Chen (2020), for instance, managed to collaborate with a small
advertisements to managers and tracking their eye movements. Actions
jewelry store in a local shopping mall to conduct a field experiment on
could range from choosing to purchase an item to moving ones’ eyes to a
how being promotion (vs. prevention) oriented can affect consumers’
particular location (Morales et al., 2017). Behavior carries some con­
price choices. Another example can be found in Tsiros and Irmak (2020)
sequences (e.g., social, financial, effort, time, self-efficacy) that extends
where the authors got to collaborate with a lunch stand at a Farmers
beyond indicating one’s thoughts about a given matter. It is easier and
Market to investigate donations for a local elementary school with the
safer to draw more information on real behavior of respondents when
purchase of every tuna bowl. Garrett and Gopalakrishna (2019)
researchers use dependent variables that are behavioral and conse­
collaborated with an insurance company to conduct a field quasi-
quential. Responding to a scale, or even declaring behavioral intentions
experiment for a period of 4 weeks, using life insurance as a reward
in a hypothetical manner, does not fit this criterion. This is because
for participants, in line with the collaborating company’s products.
reporting one’s theories about behavior, emotions, or intended actions
does not directly translate to actual behavior. This attitude-behavior gap
5. Methodological considerations
phenomenon is defined as “the differences between what people say and
what people do” (Blake, 1999, p.275). One possible explanation for this
Experiments present at least four methodological considerations to
phenomenon is the tendency for survey respondents to answer questions
be addressed carefully, namely (1) the needed sample size; (2) the
in a manner that will be viewed favorably by others, the so-called social
appropriate design to fit the research question (i.e., within, between or
desirability bias (Grimm, 2010).
mixed); (3) manipulation checks; and (4) the moderation and mediation
Fig. 1 summarizes graphically the discussion on the two identified
analyses to investigate mitigating factors and processes behind the ef­
dimensions, experimental realism of the independent variable (IV) and
fect, respectively.
behavioral measure of the dependent variable (DV), clarifying the con­
cerns (or benefits) of each quadrant of the matrix.
For instance, one way to increase experimental realism online could 5.1. Sample arrangements
be by increasing the interaction of participants with the stimuli (e.g.,
product, brand, company, different investment opportunities, strategic The natural question that a marketing researcher would ask when
decisions they would make for the company, etc.), using a natural designing an experimental study is: “How many participants do I need for
context, and incentivizing greater information seek and engagement. an experiment?” The answer is not a magic number but depends on how
Examples of such techniques have been used in previous research (e.g., large the treatment effect (i.e., how much the treatment moves the

196
G. Viglia et al. Industrial Marketing Management 98 (2021) 193–206

outcome) and on the standard deviation of the dependent variable. 5.2. Between, within or mixed design?
The treatment effect is generally presented as an effect size, i.e., the
extent of the difference of the variable object of investigation between There are two main typologies of experimental designs: between-
groups. This measure is often presented also showing the outcome’s subjects design and within-subjects design. In the case of the between-
standard deviation. The larger the treatment effect is, the fewer people subject design (e.g., Seggie, Griffith, & Jap, 2013, Study 2), partici­
are needed in the experiment. The way to know the treatment effect is to pants are exposed to only one of the conditions (treatment or control). In
run a pilot study before running the actual experiment and comparing the case of within-subject design, participants are exposed to all con­
the mean outcome in the treatment group versus the control group ditions. A between-subjects design compares differences between sub­
(Viglia & Dolnicar, 2020). jects who were exposed to different stimuli, while a within-subjects
Before presenting a tool to calculate the ideal sample size for a study, design focuses on changes among the same set of respondents from
there is the need to summarize the probability of making errors in hy­ before and after the exposure to stimuli. Sometimes, these designs are
potheses testing. There are two main types of errors: Type 1, which re­ combined, resulting in mixed designs (e.g., Dean, Griffith, & Calantone,
fers to the risk of incorrectly rejecting the null hypothesis and claiming 2016; Seggie et al., 2013, Study 1), where participants may be exposed
that the means of the treatment and control conditions are significantly to only one of the conditions of one factor (the between-subjects
different (when they are not), and Type 2, which refers to the risk of element), but all conditions of the other factor (the within-subjects
incorrectly not rejecting the null hypothesis and claiming that the means element).
of the treatment and control conditions are not significantly different There are pros and cons behind these designs. While for within-
(when they are). The probability of making Type 1 error is referred to as subjects design the sample size required is smaller, and the probability
α and probability of making Type 2 error is referred to as β. 1- β is the of grasping true differences among conditions is greater, the setup and
statistical power of the experiment. Previous research has defined sta­ the analyses are easier for between-subjects designs instead. Moreover,
tistical power as “the probability that its null hypothesis (H0) will be employing a within-subjects design would give more statistical power to
rejected given that it is in fact false” (Faul, Erdfelder, Lang, & Buchner, the study because we are not interested in the portion of error attrib­
2007, p. 175). utable to individual differences among participants in this type of
Given that lack of statistical power decreases the possibility to reli­ design, but the risk is that exposing participants to all levels of the same
ably discriminate between the null hypothesis and the alternative hy­ factor may result in them understanding the purpose of the experiment
pothesis, ensuring that a study is sufficiently powered is of great and providing biased responses. A solution to this is the randomization
importance (Faul et al., 2007). Here we present an empirical tool of the order of the treatment conditions so that different participants are
(among others available) used to calculate the necessary sample size for exposed to the conditions in a different order. For instance, An et al.
running an experiment. G*Power is a power analysis program and a (2020) have randomized the order of the scenarios that were shown
standalone software developed by Erdfelder, Faul, and Buchner (1996) within respondents. In employing a within-subjects design, the authors
for statistical tests commonly used also in marketing behavioral exposed every participating team to all three conditions (control con­
research. The utility of G*Power is that it calculates the necessary sta­ dition, treatment 1, treatment 2) in a random order. This part of the
tistical power based on a series of frequently used tests such as t, F, z, χ 2 experiment, given its characteristics, was a within-subject design.
or binomial reference distributions. G*Power1 offers a free design-based However, in this study, the authors combined a within-design experi­
approach where the researcher (1) selects the category of statistical test ment with a between-subject one. Specifically, in their research, the
that one is interested in (e.g., correlations, means, proportions, vari­ authors use a two factors design, with one of the factors being within-
ances, regressions), and (2) specifies the design characteristics (e.g., subject and the other one between. Specifically, the authors first
number of groups, whether the samples are independent or dependent, randomly assign each of the teams to different simulations (team
number of controls, expected level of power, etc.). At the specification of cooperation, inter-team competition, intra-team competition), which is
the design of the study, G*Power estimates a sample size that would the within-subject factor, and then randomly assign the teams to either
provide the requested statistical power. high or low organizational identification scenario, which is the between-
Hence, the necessary sample size depends on i) the suggested sig­ subject factor. Employing mixed-designs (e.g., Seggie et al., 2013, Study
nificance level and power, ii) the variance of outcomes and iii) the effect 1) provides benefits related both to within-subjects – greater statistical
size between conditions. With the same treatment effect, a higher or power – and to between-subjects – less learning and order effects.
lower sample dimension will make a null hypothesis rejected or not.
When specifying the sample size, the researcher needs to take into 5.3. Manipulation checks
consideration also common issues with sample arrangements: (1) over-
coverage in the sampling frame (which occurs when the sample includes Manipulation checks are questions that are used to make sure that
units that should not be part of the population, e.g., requesting only US the treatment was perceived as intended. They are usually operated as
participants on an online platform and users with VPN from non-US quantitative questions asking for self-reported answers after the expo­
countries participating), (2) under-coverage in the sampling frame sure of participants to the manipulation (Ejelöv & Luke, 2020). These
(which occurs when the sample does not include units that should be answers (i.e., the manipulation checks) provide confidence that the ef­
part of the population, e.g., failing to capture a representation of a fects on the dependent variables are due to the manipulation (e.g., Dean
portion of the population), and (3) non-response bias (which occurs when et al., 2016; Seggie et al., 2013; Taylor et al., 2021). If the majority of the
units that are supposed to be included, are not included for some reason, respondents does not perceive the manipulation as intended, then the
e.g., failing to get a portion of the sample because the invitation to take design of the study is flawed.
part in the study ends up in their spam; Mittal, 2019). These issues can Ideally, researchers might look at manipulation checks previously
be solved by including more demographic screening, different types of validated in the literature, but it can also be easily possible to present a
incentives, or by acknowledging these limitations in the study (Mittal, completely new manipulation check, especially when the variable to be
2019). manipulated is an objective variable.
While previous researchers have raised concerns regarding the
effectiveness of manipulation checks from a quantitative point of view
and regarding the possibility of creating demand effects among partic­
ipants (Fiedler, McCaughey, & Prager, 2021), manipulation checks, if
1
(https://www.psychologie.hhu.de/arbeitsgruppen/allgemeine-psychologie applied correctly, are highly beneficial to experimental researchers in
-und-arbeitspsychologie/gpower.html) understanding the effect of the independent variable on the dependent

197
G. Viglia et al. Industrial Marketing Management 98 (2021) 193–206

variable (i.e., presence of causality; Ejelöv & Luke, 2020; Fiedler et al., that of comparing groups for differences. This allows them to suggest
2021). that a difference in the mean of the dependent variable between
Another aspect to consider related to manipulation checks is their different groups can be a sign in the direction of causation claims.
positioning in the study, regarding (1) their inclusion in the main study Hence, among the first analyses that researchers apply when investi­
(with the risk of overcomplicating the study) or in a pilot study, and (2) gating experimental data would be a comparison of means or fre­
their inclusion before or after the dependent variable assessment (with quencies between the experimental groups. For instance, a typical
the risk of creating demand effects; Kidd, 1976; Mills, 1969; Parro & analysis is that of variance, also known as ANOVA, and defined as “a
Hertel, 1999). One solution to the concerns regarding where to position statistical technique utilizing an F ratio to determine if an independent
manipulation checks may be to run separate pre-tests before conducting variable has a statistically significant effect on a dependent variable”
the main study (Hauser, Ellsworth, & Gonzalez, 2018; Kidd, 1976). (Picardi & Masick, 2014, p. 105).2
However, the inclusion of manipulation checks in the main study re­ There are two additional common aims of experiments, besides
mains crucial especially when there is the concern of confounds, as there testing for main effects of the independent variable on the dependent
is no guarantee that an experimental treatment will only manipulate the variable: i) testing for moderation (also referred to as boundary condi­
focal independent variable, without varying other variables too (Fiedler tions) and ii) testing for mediation. A moderator is a variable that
et al., 2021). strengthens or weakens an existing established relationship while a
It is at the consideration of the researcher whether these questions mediator is a variable that clarifies the mechanism behind that rela­
are asked before or after the dependent variable, based on the theoret­ tionship. While the researcher may find empirically evidence for a
ical expectation on the effects of manipulation checks over the study and mediator explaining why a relationship between two variable holds, this
on the dependent variable. Importantly, manipulation checks used in cannot rule out that there are no other possible explanations for why this
each study should always be reported, possibly be drawn from previous relationship holds. One way to ensure that the effects have not been
research – and if impossible, adapted – and should possibly be oper­ driven by confounds is the use of manipulation checks, as we have
ationalized using multi-item measures (Pechmann, 2019). explained in the prior sections of this manuscript. Another solution is the
measurement of those constructs that the researcher expects to possibly
5.4. Attention checks explain the relationship between the independent and the dependent
variable. Measuring the constructs that present alternative explanations
While manipulation checks serve the purpose of understanding for why a relationship may hold will allow the researcher to test and
whether the manipulation worked as intended, instructional attention eventually rule out their role as mechanisms in the predicted causality
checks – often called attention checks (Paas, Dolnicar, & Karlsson, 2018) between the variables.
– help to see whether participants are paying attention to the provided For instance, a study might want to measure if the effect of the
instructions. Attention checks vary from reverse scaled questions, to manipulated independent variable is stronger or weaker depending on
more content-related questions, to timing response times in online- individual differences between respondents (e.g., personality traits, age,
operated studies (Abbey & Meloy, 2017). Abbey and Meloy (2017) and gender) or some contextual factors (e.g., time of the day, brand
mainly classify attention checks into logical statements (e.g., preferring positioning, etc.). This variable takes the name of moderator. Taylor
to eat fruit vs. paper), directed queries (e.g., For this query, mark X et al. (2021) investigate the role of target decision legitimacy as a
[insert X] and move on.), open-ended queries (e.g., Please summarize moderator in the relationship between target decision and activists’
what was written in the scenario you just read), infrequency (e.g., asking recommendation legitimacy, and in the relationship between activists’
several times the date of birth and checking for consistency), response recommendation legitimacy and observer adoption. Dean et al. (2016)
time (e.g., evaluating efficiency and accuracy based on how long it took investigate the role of situation-specific factors relevant to new product
to the participant to answer), memory recall (e.g., please write down the introductions as moderator instead. In other cases, the moderator can
name of the brand you were shown in the scenario before), and reverse also be investigated as affecting another interaction effect, as it happens
scaling (e.g., changing the direction of the agreement from I agree to I do for instance in the study from Griffith, Hoppner, Lee, and Schoenherr
not agree with the statement). Researchers should consider using mul­ (2017).
tiple attention checks especially if the study is lengthy (Abbey & Meloy, To test for this statistically, we expect an interaction between the
2017). independent variable and the proposed moderator. For this reason, a
For instance, Saab and Botelho (2020) used two questions to assess moderator is often called an interaction effect (Das & Dirienzo, 2010;
the attention of their respondents. First, to assess their attention in Imbens & Wooldridge, 2009). The moderator can affect i) the main
general, they asked their participants to indicate the product they had relationship between the independent and the dependent variable, ii)
been informed about in the manipulation text. Moreover, as a further the relationship between the independent variable and the mediator, iii)
attention check, the authors asked participants to calculate the price the relationship between the mediator and the dependent variable, or iv)
difference of the two competing offerings for the products they read multiple relationships. The position of the moderator on the conceptual
about in the manipulation text. model should be guided by previous theoretical knowledge.
Again, it is of absolute importance to disclose the specific attention When both the independent variable and the moderator are cate­
checks that were used in a study and if exclusion of participants based on gorical, the researcher can use ANOVAs to analyze the differences in
attention checks significantly changes the results of the study (Abbey & means between conditions and the overall interaction effect. However,
Meloy, 2017). It is ideal to decide on the criteria for participants’ there are two additional analyses often applied when one of the vari­
exclusion before the study has been conducted, in order to diminish the ables in the interaction is continuous and the other is categorical:
risk of false positives (e.g., by preregistering a study, see Van’t Veer & spotlight analysis and floodlight analysis. The spotlight analysis pro­
Giner-Sorolla, 2016). vides an estimate of the effect of one of the variables in the interaction –
To summarize, manipulation checks are to be treated differently the categorical one – at specific values of the other variable in the
from the attention check; the former one aims to assess the validity of interaction – the continuous one (Spiller, Fitzsimons, Lynch Jr, &
the manipulation and the latter the attention that is being put into the McClelland, 2013). This is particularly relevant when there are specific
study (Ejelöv & Luke, 2020; Sigall & Mills, 1998). points in the continuous variable that are of interest to the researcher.

5.5. Analyzing experimental data in marketing

2
For an overview of analyses of experiments with one or two factors (i.e.,
The first aim of researchers employing an experimental approach is one-way and two-way ANOVAs), please refer to Appendix 2.

198
G. Viglia et al. Industrial Marketing Management 98 (2021) 193–206

The floodlight analysis provides an estimate of the effect of the cate­ enrich the internal validity of findings. That is because some type of
gorical variable in the interaction at all values of the continuous variable questions – such as short-term and long-term implications of findings,
(Spiller et al., 2013). This is relevant when the aim is to spot the areas boundary conditions, what are the effects of the treatment on the
where the interaction is significant and where it is not (for more details outcome and which players are the most affected – would have been
on spotlight and floodlight analyses, see Krishna, 2016; Spiller et al., hard to test in a hypothetical scenario with possible questions on the
2013). In Table S1 in Appendix 1 and in Appendix 3, we provide guid­ veracity of the findings also because of possible social desirability bias
ance on how to run these analyses with STATA or SPSS. concerns.
A mediator is a concept that is different from that of the moderator. In this article, we provide a short guide on how to conduct experi­
The mediator does not affect the strength of the relationship between ments in marketing research, borrowing from the procedures and
two variables, it tries to explain why a change in the treatment may guidelines provided from the other marketing areas. Specifically, we
affect the outcome. Seggie et al. (2013) investigate the role of trans­ guide the researcher from the reasons behind choosing an experimental
action cost as one potential mediator between opportunism form and approach to investigate the conceptual model, to the types of available
satisfaction with the performance of the relationship. Pieters (2017), experiments and when and why each of them should be chosen. More­
Rucker, Preacher, Tormala, and Petty (2011), and Zhao, Lynch Jr, and over, we provide some guidelines on how to design experiments,
Chen (2010) offer important insights and guidelines into meaningful explaining the main necessary criteria that should be respected when
mediation testing in experiments. The PROCESS macro for SPSS pro­ conducting experiments, and we suggest how the sample size should be
vides an efficient means to test for mediation effects using a boot­ determined. Finally, we provide some short guidelines on how to
strapping approach (Hayes, 2017). In Appendix 4, we present guidelines analyze different types of data resulting from experiments.
on how to conduct mediation testing using SPSS. It is worth mentioning We see several benefits in applying experiments in the business
that mediation can take other forms, such as parallel mediation (when marketing field. Combining experimental designs and approaches in a
multiple mechanisms are tested contemporarily) or serial mediation way to enhance behavioral realism will lead to greater robustness of
(when one mechanism leads to another and so on). findings and greater external validity of them too. Especially now, in the
An additional important aspect of the combination of mediator and digital world, randomized experiments can be cheap and fast. It is
moderator in a model is what Hayes (2013) refers to as conditional possible to write a line of code to randomly assign participants to one
process analysis. This analysis is used when the researcher is investi­ group and it is not needed for users to fill surveys or even tell users that
gating conditions under which the relation between the independent they are part of an experiment.
variable affects the dependent variable via a mediator. Hence, it links Combining data types for converging evidence and carefully
together into a single integrated analytical model both the mediator and considering the needed sample size also showing the effect size is a good
the moderator. The author provides explanations with examples from way to proceed. Moreover, combining different experimental ap­
past literature where the conditional effect (i.e., moderation) happens in proaches in the empirical package can help to increase the extent of
the a-path of the model (i.e., between the independent variable and the behavioral realism, documenting the phenomenon and some potential
mediator), in the b-path (i.e., between the mediator and the dependent mechanism behind the effect in a laboratory or online experiment, and
variable), or both in the a-path and the b-path. The conditional effect can then enriching the external validity by running a field study (e.g.,
also occur in the direct effect between the independent and the depen­ Steward, Narus, & Roehm, 2018).
dent variable (i.e., c-path). In this case, the indirect effect is uncondi­ Important empirical questions would ideally be addressed using at
tional because, being it a product of the a-path and b-path least a couple of toolboxes (e.g., a lab plus a field study; some natural
unconditioned effects, it remains unconditional when the moderation data plus a lab study; a qualitative approach plus an experiment). Some
occurs in the c-path. In the other cases (i.e., moderation occurring at possible examples in B2B can be found in Seggie et al. (2013) or in
least in the a-path, b-path, both a-path and b-path), we are dealing with Steward et al. (2018), where the authors complement field experiments
a conditional indirect effect. If it occurs both in the c-path and any of the with other methods, such as in-depth interviews in Steward et al. (2018)
indirect links between the independent variable and the dependent one, or with longitudinal data in Seggie et al. (2013). In general, in the era of
the effect is considered to be conditional both at the indirect and direct an enormous amount of available data, experiments are useful to draw
effect (Hayes, 2013). causal relationships and understand “what is causing what”. Table 3
When analyzing moderation and mediation models, presenting the provides a short checklist for authors and reviewers.
conceptual model helps the reader in understanding the relationships To conclude, controlled experiments that are well-planned and
predicted and to be tested. This usually allows the researcher to also tell
apart the moderator from the mediator, given that a moderator can still Table 3
be sometimes confused with a mediator (Hayes, 2009). For more A checklist for authors and reviewers running or evaluating experimental studies
mediation models and guidelines, refer to Kirk (2013) and Hayes (2017). in marketing.
Question Check
6. General discussion and conclusion ✓

Are authors addressing a real marketing

Experiments remain one of the main methodological approaches in RELEVANCE problem?
marketing and social sciences for testing causality between an inde­ Does the chosen sample allow for internal/
pendent and a (or many) dependent variable(s). To conclude that the VALIDITY external validity?
treatment causes a specific variation in the dependent variable when Have authors come up with a clear design that
rules out possible alternative explanations for the
applying an experimental approach, the researcher has to ensure that RIGOR effects?
the requirements of the experiment are being carefully respected and Is the design clearly explained so that it can be
attentively executed. While experiments have been greatly used in REPLICABILITY replicated?
consumer-related research in marketing, their use in B2B remains Is the dependent variable measuring actual
behavior or – at the very least – showing some
behind and sometimes falls short of the applicability of this method.
IMPACT behavioral realism?
Most papers discuss the external validity and generalizability of their Have authors included detailed analyses (going
results, predicting real behavior, but they do so by basing their discus­ INFORMATIVENESS beyond p-value by showing effect sizes)?
sion on findings of surveys. However, both editors and reviewers of the Have authors included an adequate visual
B2B and Industrial Marketing Management (IMM) field, have been ILLUSTRATIVENESS representation of the results?

increasingly suggesting and requesting experimental approaches to

199
G. Viglia et al. Industrial Marketing Management 98 (2021) 193–206

executed have a practical bent. They help us learn interventions that to business marketing researchers and practitioners.
work and that do not under different conditions, offering clear insights

Appendix 1

Table S1
Generic STATA codes for experimental testing

Tests STATA codes

One-way ANOVA oneway DV IV, tabulate

One-way ANOVA (post hoc testing)1 mean comparison pwmean DV, over[IV], mcompare(LSD) effects
One-way ANOVA (post hoc testing) contrast comparison pwcompare DV, over[IV], mcompare(LSD) effects
manova DV1 DV2 DV3 DV4 = IV
One-way MANOVA mvreg
Two-way between-subjects ANOVA anova DV IV1##IV2
anova DV IV1##IV2
contrast ar.IV1@IV2
Two-way between-subjects ANOVA (contrasts)2 contrast ar.IV2@IV1
Within-subjects ANOVA3 anova DV IV1 IV2, repeated(IV2)
reg DV IV1##IV2
Spotlight analysis4 margins, dydx(IV1) at(IV2 = (“value 1 of IV2” “value 2 of IV2”)) atmeans
reg DV IV1##IV2
Floodlight analysis5 margins, dydx(IV1) at(IV2 = (“min value of IV2” (“unit change”) “max value of IV2”)) atmeans
1
We have assumed LSD approach, but other approaches can be specified here, e.g., tukey, bonferroni, etc.
2
Here we are assuming IV1 is between-subjects and IV2 is repeated within-subjects
3
First line: to compare differences in marginal means of all levels of IV1 at IV2
second line: to compare differences in marginal means of all levels of IV2 at IV1
4
E.g., investigating the effect for moderator values 0 and 1, the 2nd part of the code would be:
margins, dydx(IV1) at(IV2 = (01)) atmeans
5
E.g., investigating the effect for moderator that takes value 1 to 20, with 1 unit increase, the 2nd part of the code would be:
margins, dydx(IV1) at(IV2 = (1 (1) 20)) atmeans

Appendix 2

How to analyze experimental data with one-way and two-way ANOVA.

• One-way ANOVA

To test for mean comparison using a One-way ANOVA in SPSS the steps to follow are these:
Analyze – Compare Means – One-way ANOVA.

• Two-way ANOVA

To test a linear model for independent between-subjects factorial design in SPSS the steps to follow are these:
Analyze – General Linear Model – Univariate.

200
G. Viglia et al. Industrial Marketing Management 98 (2021) 193–206

To test a model with a repeated model design in SPSS the steps to follow are these:
Analyze – General Linear Model – Repeated.
We assume we have a repeated measure that we call Measure_pre and Measure_post, so it was measure before and after, for each participant. This
variable has, hence, 2 levels (pre and post). Below is how we would set this up on SPSS:

201
G. Viglia et al. Industrial Marketing Management 98 (2021) 193–206

Appendix 3

• Moderation: floodlight using PROCESS

The PROCESS macro at SPSS allows for moderation testing both when variables are categorial, and for continuous and categorical variables
combinations. For example, here, we’ve tested the Model 1 of the PROCESS macro, which predicts a simple moderation.

Moreover, the PROCESS macro at SPSS allows for the floodlight analysis when testing for any model that includes a moderation. For example,
we’ve again tested Model 1 of the PROCESS macro, but it can be applied to any model including moderation (e.g., Model 7, 8, 14 of the PROCESS
macro, etc.).

Here’s an example of the presentation of the floodlight analysis results using the macro for excel provided by Carden, Holtzman, and Strube (2017):

202
G. Viglia et al. Industrial Marketing Management 98 (2021) 193–206

Appendix 4

• Mediation: using PROCESS

Instructions on how to install the PROCESS macro into SPSS and on the different model numbers can be found at: https://www.processmacro.
org/uploads/3/4/8/5/34858384/dialoginstall.pdf

• Moderated mediation: using PROCESS

Theme a: When moderation is predicted in the c-path:

203
G. Viglia et al. Industrial Marketing Management 98 (2021) 193–206

Theme b: When moderation is predicted in the a-path:

Theme c: When moderation is predicted in the b-path:

204
G. Viglia et al. Industrial Marketing Management 98 (2021) 193–206

More combinations and more models, e.g., when the moderation is predicted on multiple paths, or when there are multiple mediators predicted,
are available on Hayes (2012, 2013).

Appendix A. Supplementary data

Supplementary data to this article can be found online at https://doi.org/10.1016/j.indmarman.2021.08.007.

References Ejelöv, E., & Luke, T. J. (2020). “Rarely safe to assume”: Evaluating the use and
interpretation of manipulation checks in experimental social psychology. Journal of
Experimental Social Psychology, 87, 103937.
Abbey, J. D., & Meloy, M. G. (2017). Attention by design: Using attention checks to
Erdfelder, E., Faul, F., & Buchner, A. (1996). GPOWER: A general power analysis
detect inattentive respondents and improve data quality. Journal of Operations
program. Behavior Research Methods, Instruments, & Computers, 28(1), 1–11.
Management, 53, 63–70.
Faul, F., Erdfelder, E., Lang, A. G., & Buchner, A. (2007). G* Power 3: A flexible statistical
Allen, M. (2017). The sage encyclopedia of communication research methods (Vols. 1–4).
power analysis program for the social, behavioral, and biomedical sciences. Behavior
SAGE Publications, Inc: Thousand Oaks, CA. https://doi.org/10.4135/
Research Methods, 39(2), 175–191.
9781483381411.
Ferguson, J. L., & Mohan, M. (2020). Use of celebrity and non-celebrity persons in B2B
An, D., Kreutzer, M., & Heidenreich, S. (2020). Always play against par? The effect of
advertisements: Effects on attention, recall, and hedonic and utilitarian attitudes.
inter-team coopetition on individual team productivity. Industrial Marketing
Industrial Marketing Management, 89, 594–604.
Management, 90, 155–169.
Fiedler, K., McCaughey, L., & Prager, J. (2021). Quo vadis, methodology? The key role of
Anderson, E. T., & Simester, D. (2011). A step-by-step guide to smart business
manipulation checks for validity control and quality of science. Perspectives on
experiments. Harvard Business Review, 89(3), 98–105.
Psychological Science, 16(4), 816–826. https://doi.org/10.1177/
Barasch, A., Levine, E. E., Berman, J. Z., & Small, D. A. (2014). Selfish or selfless? On the
1745691620970602.
signal value of emotion in altruistic behavior. Journal of Personality and Social
Flere, S., & Lavrič, M. (2008). On the validity of cross-cultural social studies using
Psychology, 107(3), 393–413.
student samples. Field Methods, 20(4), 399–412.
Bello, D., Leung, K., Radebaugh, L., Tung, R. L., & Van Witteloostuijn, A. (2009). From
Galizzi, M. M., & Navarro-Martínez, D. (2019). On the external validity of social
the editors: Student samples in international business research. Journal of
preference games: A systematic lab-field study. Management Science, 65(3),
International Business Studies, 40(3), 361–364.
976–1002.
Bendixen, M., Bukasa, K. A., & Abratt, R. (2004). Brand equity in the business-to-business
Garrett, J., & Gopalakrishna, S. (2019). Sales team formation: The right team member
market. Industrial Marketing Management, 33(5), 371–380.
helps performance. Industrial Marketing Management, 77, 13–22.
Blake, J. (1999). Overcoming the “value-action gap”in environmental policy: Tensions
Gneezy, A. (2017). Field experimentation in marketing research. Journal of Marketing
between national policy and local experience. Local Environment, 4(3), 257–278.
Research, 54(1), 140–143.
Bruine de Bruin, W., & Ulqinaku, A. (2021). Effect of mortality salience on charitable
Griffith, D. A., Hoppner, J. J., Lee, H. S., & Schoenherr, T. (2017). The influence of the
donations: Evidence from a national sample. Psychology and Aging, 36(4), 415–420.
structure of interdependence on the response to inequity in buyer–supplier
https://doi.org/10.1037/pag0000478.
relationships. Journal of Marketing Research, 54(1), 124–137.
Calder, B. J., Phillips, L. W., & Tybout, A. M. (1981). Designing research for application.
Grimm, P. (2010). Social desirability bias. Wiley International Encyclopedia of Marketing.
Journal of Consumer Research, 8(2), 197–207.
Hada, M. (2021). Two steps: A primer on B2B experiments. Impact at JMR (January),
Carden, S. W., Holtzman, N. S., & Strube, M. J. (2017). CAHOST: An excel workbook for
Available at: https://www.ama.org/2021/01/26/two-steps-a-primer-on-b2b-exp
facilitating the Johnson-Neyman technique for two-way interactions in multiple
eriments/.
regression. Frontiers in Psychology, 8, 1293.
Hauser, D. J., Ellsworth, P. C., & Gonzalez, R. (2018). Are manipulation checks
Carrington, M. J., Neville, B. A., & Whitwell, G. J. (2014). Lost in translation: Exploring
necessary? Frontiers in Psychology, 9, 998.
the ethical consumer intention–behavior gap. Journal of Business Research, 67(1),
Hayes, A. F. (2009). Beyond Baron and Kenny: Statistical mediation analysis in the new
2759–2767.
millennium. Communication Monographs, 76(4), 408–420.
Choi, W. J., Sun, H., Liu, Y., & Chen, H. (2020). Guess who buys cheap? The effect of
Hayes, A. F. (2012). PROCESS: A versatile computational tool for observed variable
consumers’ goal orientation on product preference. Journal of Consumer Psychology,
mediation, moderation, and conditional process modeling.
30(3), 506–514.
Hayes, A. F. (2013). Introduction to mediation, moderation, and conditional process analysis:
Cuervo-Cazurra, A., Andersson, U., Brannen, M. Y., Nielsen, B. B., & Reuber, A. R. (2016).
A regression-based approach. Guilford Publications.
From the editors: Can I trust your findings? Ruling out alternative explanations in
Hayes, A. F. (2017). Introduction to mediation, moderation, and conditional process analysis
international business research. Journal of International Business Studies, 47, 881–897.
second edition: A regression-based approach.
Das, J., & Dirienzo, C. (2010). Tourism competitiveness and corruption: A cross-country
Imbens, G. W., & Wooldridge, J. M. (2009). Recent developments in the econometrics of
analysis. Tourism Economics, 16(3), 477–492.
program evaluation. Journal of Economic Literature, 47(1), 5–86.
Dean, T., Griffith, D. A., & Calantone, R. J. (2016). New product creativity:
Jap, S. D., Robertson, D. C., Rindfleisch, A., & Hamilton, R. (2013). Low-stakes
Understanding contract specificity in new product introductions. Journal of
opportunism. Journal of Marketing Research, 50(2), 216–227.
Marketing, 80(2), 39–58.

205
G. Viglia et al. Industrial Marketing Management 98 (2021) 193–206

Kahneman, D., & Egan, P. (2011). Thinking, fast and slow (Vol. 1). New York: Farrar, Sarial-Abi, G., Vohs, K. D., Hamilton, R., & Ulqinaku, A. (2017). Stitching time: Vintage
Straus and Giroux. consumption connects the past, present, and future. Journal of Consumer Psychology,
Kidd, R. F. (1976). Manipulation checks: Advantage or disadvantage? Representative 27(2), 182–194.
Research in Social Psychology, 7, 160–165. Schram, A. (2005). Artificiality: The tension between internal and external validity in
Kirk, R. E. (1995). Experimental design: Procedures for the behavioral sciences (3rd ed.). economic experiments. Journal of Economic Methodology, 12(2), 225–237.
Pacific Grove, CA: Brooks/Cole. Seggie, S. H., Griffith, D. A., & Jap, S. D. (2013). Passive and active opportunism in
Kirk, R. E. (2013). Experimental design: Procedures for the behavioral sciences. Thousand interorganizational exchange. Journal of Marketing, 77(6), 73–90.
Oaks, CA: SAGE Publications, Inc.. https://doi.org/10.4135/9781483384733 Sigall, H., & Mills, J. (1998). Measures of independent variables and mediators are useful
Krishna, A. (2016). A clearer spotlight on spotlight: Understanding, conducting and in social psychology experiments: But are they necessary? Personality and Social
reporting. Journal of Consumer Psychology, 26(3), 315–324. Psychology Review, 2, 218–226.
Laursen, L. N., & Andersen, P. H. (2016). Supplier involvement in NPD: A quasi- Simester, D. (2017). Field experiments in marketing. In , Vol. 1. Handbook of economic
experiment at Unilever. Industrial Marketing Management, 58, 162–171. field experiments (pp. 465–497). North-Holland.
Levitt, S. D., & List, J. A. (2007). What do laboratory experiments measuring social Spiller, S. A., Fitzsimons, G. J., Lynch, J. G., Jr., & McClelland, G. H. (2013). Spotlights,
preferences reveal about the real world? Journal of Economic Perspectives, 21(2), floodlights, and the magic number zero: Simple effects tests in moderated regression.
153–174. Journal of Marketing Research, 50(2), 277–288.
Liang, B., Kale, S. H., & Cherian, J. (2014). Is the future static or dynamic? The role of Steward, M. D., Narus, J. A., & Roehm, M. L. (2018). An exploratory study of business-to-
culture on escalation of commitment in new product development. Industrial business online customer reviews: External online professional communities and
Marketing Management, 43(1), 155–163. internal vendor scorecards. Journal of the Academy of Marketing Science, 46(2),
McCoy, M., & Hargie, O. (2007). Effects of personalization and envelope color on 173–189.
response rate, speed and quality among a business population. Industrial Marketing Summers, J. O. (2019). Guidelines for conducting research and publishing in marketing:
Management, 36(6), 799–809. From conceptualization through the review process. In How to get published in the best
Mills, J. (1969). Experimental social psychology. Toronto: MacMillian. marketing journals. Edward Elgar Publishing.
Mittal, V. (2019). How to get published in the best marketing journals. Edward Elgar Tangpong, C., Li, J., & Hung, K. T. (2016). Dark side of reciprocity norm: Ethical
Publishing. compromise in business exchanges. Industrial Marketing Management, 55, 83–96.
Morales, A. C., Amir, O., & Lee, L. (2017). Keeping it real in experimental Taylor, K. M., Hajmohammad, S., & Vachon, S. (2021). Activist engagement and
research—Understanding when, where, and how to enhance realism and measure industry-level change: Adoption of new practices by observing firms. Industrial
consumer behavior. Journal of Consumer Research, 44(2), 465–476. Marketing Management, 92, 295–306.
Oh, J. H., Peters, L. D., & Johnston, W. J. (2014). Who’s acquiring whom?— Tsiros, M., & Irmak, C. (2020). Lowering the minimum donation amount increases
Experimental evidence of firm size effect on B2B mergers and marketing/sales tasks. consumer purchase likelihood of products associated with cause-related marketing
Industrial Marketing Management, 43(6), 1035–1044. campaigns. Journal of Marketing Research, 57(4), 755–770.
Paas, L. J., Dolnicar, S., & Karlsson, L. (2018). Instructional manipulation checks: A Ullah, S., Akhtar, P., & Zaefarian, G. (2018). Dealing with endogeneity bias: The
longitudinal analysis with implications for MTurk. International Journal of Research in generalized method of moments (GMM) for panel data. Industrial Marketing
Marketing, 35(2), 258–269. Management, 71, 69–78.
Parro, W. G., & Hertel, P. (1999). Research methods in cognition and emotion. In Ulqinaku, A., Sarial-Abi, G., & Kinsella, E. L. (2020). Benefits of heroes to coping with
T. Dalgleish, & M. J. Power (Eds.), Handbook of cognition and emotion (pp. 61–81). mortality threats by providing perceptions of personal power and reducing
Chichester, England: Wiley. unhealthy compensatory consumption. Psychology & Marketing, 37(10), 1433–1445.
Pechmann, C. (2019). How to get published in the best marketing journals. Edward Elgar Valsesia, F., Proserpio, D., & Nunes, J. C. (2020). The positive effect of not following
Publishing. others on social media. Journal of Marketing Research, 57(6), 1152–1168.
Picardi, C. A., & Masick, K. D. (2014). Research methods. In Designing and conducting. van Heerde, H. J., Moorman, C., Moreau, C. P., & Palmatier, R. W. (2021). Reality check:
SAGE Publications. Infusing ecological value into academic marketing research. Journal of Marketing, 85
Pieters, R. (2017). Meaningful mediation analysis: Plausible causal inference and (2), 1–13. https://doi.org/10.1177/0022242921992383.
informative communication. Journal of Consumer Research, 44(3), 692–716. Van’t Veer, A. E., & Giner-Sorolla, R. (2016). Pre-registration in social psychology—A
Putnam-Farr, E., & Riis, J. (2016). “Yes/no/not right now”: Yes/no response formats can discussion and suggested template. Journal of Experimental Social Psychology, 67,
increase response rates even in non-forced-choice settings. Journal of Marketing 2–12.
Research, 53(3), 424–432. Viglia, G., & Dolnicar, S. (2020). A review of experiments in tourism and hospitality.
Rucker, D. D., Preacher, K. J., Tormala, Z. L., & Petty, R. E. (2011). Mediation analysis in Annals of Tourism Research, 80, 102858.
social psychology: Current practices and new recommendations. Social and Zaefarian, G., Kadile, V., Henneberg, S. C., & Leischnig, A. (2017). Endogeneity bias in
Personality Psychology Compass, 5(6), 359–371. marketing research: Problem, causes and remedies. Industrial Marketing Management,
Ruiz, C. A. D., & Kowalkowski, C. (2014). Market representations in industrial marketing: 65, 39–46.
Could representations influence strategy? Industrial Marketing Management, 43(6), Zellmer-Bruhn, M., Caligiuri, P., & Thomas, D. C. (2016). From the editors: Experimental
1026–1034. designs in international business research. Journal of International Business Studies,
Saab, A. B., & Botelho, D. (2020). Are organizational buyers rational? Using price 47, 399–407.
heuristics in functional risk judgment. Industrial Marketing Management, 85, Zhao, X., Lynch, J. G., Jr., & Chen, Q. (2010). Reconsidering Baron and Kenny: Myths and
141–151. truths about mediation analysis. Journal of Consumer Research, 37, 197–206.
Zimmer, M., Salonen, A., & Wangenheim, F. V. (2020). Business solutions as market
signals that facilitate product sales. Industrial Marketing Management, 91, 30–40.

206