Papers by Gabriele Paolacci
Many argue that there is a reproducibility crisis in psychology. We investigated nine well-known ... more Many argue that there is a reproducibility crisis in psychology. We investigated nine well-known effects from the cognitive psychology literature—three each from the domains of perception/action, memory, and language, respectively—and found that they are highly reproducible. Not only can they be reproduced in online environments, but they also can be reproduced with nonnaïve participants with no reduction of effect size. Apparently, some cognitive tasks are so constraining that they encapsulate behavior from external influences, such as testing situation and prior recent experience with the experiment to yield highly robust effects.
Using capture-recapture analysis we estimate the effective size of the active Amazon Mechanical T... more Using capture-recapture analysis we estimate the effective size of the active Amazon Mechanical Turk (MTurk) population
that a typical laboratory can access to be about 7,300 workers. We also estimate that the time taken for half of the workers to
leave the MTurk pool and be replaced is about 7 months. Each laboratory has its own population pool which overlaps, often
extensively, with the hundreds of other laboratories using MTurk. Our estimate is based on a sample of 114,460 completed sessions from 33,408 unique participants and 689 sessions across seven laboratories in the US, Europe, and Australia from January 2012 to March 2015.
Many natural systems involve thresholds that, once triggered, imply irreversible damages for the ... more Many natural systems involve thresholds that, once triggered, imply irreversible damages for the users. Although the existence of such thresholds is undisputed, their location is highly uncertain. We explore experimentally how threshold uncertainty affects collective action in a series of threshold public goods games. Whereas the public good is always provided when the exact value of the threshold is known, threshold uncertainty is generally detrimental for the public good provision as contributions become more erratic. The negative effect of threshold uncertainty is particularly severe when it takes the form of ambiguity, i.e. when players are not only unaware of the value of the threshold, but also of its probability distribution. Early and credible commitment helps groups to cope with uncertainty.
Using capture-recapture analysis we estimate the effective size of the active Amazon Mechanical T... more Using capture-recapture analysis we estimate the effective size of the active Amazon Mechanical Turk (MTurk) population
that a typical laboratory can access to be about 7,300 workers. We also estimate that the time taken for half of the workers to
leave the MTurk pool and be replaced is about 7 months. Each laboratory has its own population pool which overlaps, often
extensively, with the hundreds of other laboratories using MTurk. Our estimate is based on a sample of 114,460 completed sessions from 33,408 unique participants and 689 sessions across seven laboratories in the US, Europe, and Australia from January 2012 to March 2015.
Duplicate respondents across related experiments are a substantial problem for conducting program... more Duplicate respondents across related experiments are a substantial problem for conducting programmatic research on AMT (Chandler, Mueller, & Paolacci). In this tutorial, we provide a
Although researchers often assume their participants are naive to experimental materials, this is... more Although researchers often assume their participants are naive to experimental materials, this is not always the case.
We investigated how prior exposure to a task affects subsequent experimental results. Participants in this study
completed the same set of 12 experimental tasks at two points in time, first as a part of the Many Labs replication
project and again a few days, a week, or a month later. Effect sizes were markedly lower in the second wave than in
the first. The reduction was most pronounced when participants were assigned to a different condition in the second
wave. We discuss the methodological implications of these findings.
Uploads
Papers by Gabriele Paolacci
that a typical laboratory can access to be about 7,300 workers. We also estimate that the time taken for half of the workers to
leave the MTurk pool and be replaced is about 7 months. Each laboratory has its own population pool which overlaps, often
extensively, with the hundreds of other laboratories using MTurk. Our estimate is based on a sample of 114,460 completed sessions from 33,408 unique participants and 689 sessions across seven laboratories in the US, Europe, and Australia from January 2012 to March 2015.
that a typical laboratory can access to be about 7,300 workers. We also estimate that the time taken for half of the workers to
leave the MTurk pool and be replaced is about 7 months. Each laboratory has its own population pool which overlaps, often
extensively, with the hundreds of other laboratories using MTurk. Our estimate is based on a sample of 114,460 completed sessions from 33,408 unique participants and 689 sessions across seven laboratories in the US, Europe, and Australia from January 2012 to March 2015.
We investigated how prior exposure to a task affects subsequent experimental results. Participants in this study
completed the same set of 12 experimental tasks at two points in time, first as a part of the Many Labs replication
project and again a few days, a week, or a month later. Effect sizes were markedly lower in the second wave than in
the first. The reduction was most pronounced when participants were assigned to a different condition in the second
wave. We discuss the methodological implications of these findings.
that a typical laboratory can access to be about 7,300 workers. We also estimate that the time taken for half of the workers to
leave the MTurk pool and be replaced is about 7 months. Each laboratory has its own population pool which overlaps, often
extensively, with the hundreds of other laboratories using MTurk. Our estimate is based on a sample of 114,460 completed sessions from 33,408 unique participants and 689 sessions across seven laboratories in the US, Europe, and Australia from January 2012 to March 2015.
that a typical laboratory can access to be about 7,300 workers. We also estimate that the time taken for half of the workers to
leave the MTurk pool and be replaced is about 7 months. Each laboratory has its own population pool which overlaps, often
extensively, with the hundreds of other laboratories using MTurk. Our estimate is based on a sample of 114,460 completed sessions from 33,408 unique participants and 689 sessions across seven laboratories in the US, Europe, and Australia from January 2012 to March 2015.
We investigated how prior exposure to a task affects subsequent experimental results. Participants in this study
completed the same set of 12 experimental tasks at two points in time, first as a part of the Many Labs replication
project and again a few days, a week, or a month later. Effect sizes were markedly lower in the second wave than in
the first. The reduction was most pronounced when participants were assigned to a different condition in the second
wave. We discuss the methodological implications of these findings.