Open Access
(Predicting) replications and variation in new outcomes : which results generalize?
avs request failed
License
Cadmus Permanent Link
Full-text via DOI
ISBN
ISSN
Issue Date
Type of Publication
Keyword(s)
LC Subject Heading
Other Topic(s)
EUI Research Cluster(s)
Initial version
Published version
Succeeding version
Preceding version
Published version part
Earlier different version
Initial format
Author(s)
Citation
MWP; Video Lecture; 2025/01
Cite
DREBER ALMENBERG, Anna, (Predicting) replications and variation in new outcomes : which results generalize?, MWP, Video Lecture, 2025/01 - https://hdl.handle.net/1814/77830
Abstract
What share of results replicate in different literatures in some of the experimental social sciences? I will discuss several large replication projects in mainly psychology and economics, where my coauthors and I have redone experiments published in high impact journals with new and larger samples to see whether the main result replicates. I will also cover our studies on forecasting like prediction markets and forecasting surveys, where we ask researchers to predict scientific results, and discuss our recent work on the possibility of using decision markets to select which studies to replicate. In our prediction markets, researchers are given monetary endowments to bet on whether results picked for replication will replicate with the definition of successful replication being a statistically significant effect in the same direction as the original study. In decision markets, we use market prices to decide which studies to replicate. I will also discuss work on analytical heterogeneity and design heterogeneity. These are projects where researchers are asked to test the same hypotheses on the same experimental or non-experimental data, or where researchers are asked to design experiments to test the same hypothesis. The results suggest that a large share of results do not replicate and that this is to some extent predictable. The results also suggest that there is substantial analytical and design heterogeneity, limiting the generalizability of individual results.
Table of Contents
Additional Information
MWP Lecture delivered at the European University Institute in Florence on 08 January 2025.
Prof. Anna Dreber Almenberg (Stockholm School of Economics) was interviewed by MWP Fellows Vasundhara Mallick (RSC) and Alexandra Jabbour (SPS) on 09 January 2025. Interview link: https://youtu.be/2r714DoL234?si=tcYj526iYy_aDd3F
Prof. Anna Dreber Almenberg (Stockholm School of Economics) was interviewed by MWP Fellows Vasundhara Mallick (RSC) and Alexandra Jabbour (SPS) on 09 January 2025. Interview link: https://youtu.be/2r714DoL234?si=tcYj526iYy_aDd3F