The replication crisis in the social and biomedical sciences has led to an increase in the conduct of replication studies. However, from a statistical perspective, it is often unclear how to design and analyse such studies. We aim to provide researchers with a toolkit of principled methods for doing so. In addition to methodological contributions, our research also aims to provide implementations in free and open source software so that the developed methods can be easily applied by researchers.
Project Website
Project Lead
Leonhard Held
CRS Collaborators
Charlotte Micheloud, Rachel Heyard, Samuel Pawel, Simon Schwab
External Collaborators
Guido Consonni, Robert Matthews, Francesca Freuli, Manuela Ott, Fadoua Balabdaoui, Frederik Aust, Eric-Jan Wagenmakers
Publications
- Held, L. (2023). Beyond the two-trials rule. Preprint. https://doi.org/10.48550/ARXIV.2307.04548
- Freuli, F., Held, L., and Heyard, R. (2023). Replication success under questionable research practices - a simulation study. Preprint. https://doi.org/10.31222/osf.io/s4b65
- Pawel, S., Heyard, R., Micheloud, C., and Held, L. (2023). Replication of "null results" - Absence of evidence or evidence of absence? eLife (to appear). https://doi.org/10.48550/arXiv.2305.04587
- Pawel, S., Aust, F., Held, L., and Wagenmakers, E.-J. (2023). Power priors for replication studies. TEST (to appear). https://doi.org/10.48550/ARXIV.2207.14720
- Pawel, S., Consonni, G., and Held, L. (2023). Bayesian approaches to designing replication studies. Psychological Methods. https://doi.org/10.1037/met0000604
- Micheloud, C., Balabdaoui, F., and Held, L. (2023). Assessing replicability with the sceptical p-value: Type-I error control and sample size planning. Statistica Neerlandica. https://doi.org/10.1111/stan.12312
- Held, L., Matthews, R., Ott, M., and Pawel, S. (2022). Reverse-Bayes methods for evidence assessment and research synthesis. Research Synthesis Methods, 13(3):295-314. https://doi.org/10.1002/jrsm.1538
- Held, L., Micheloud, C., and Pawel, S. (2022). The assessment of replication success based on relative effect size. The Annals of Applied Statistics, 16(2):706-720. https://doi.org/10.1214/21-aoas1502
- Micheloud, C. and Held, L. (2022). Power calculations for replication studies. Statistical Science, 37(3):369-379. https://doi.org/10.1214/21-sts828
- Pawel, S. and Held, L. (2022). The sceptical Bayes factor for the assessment of replication success. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 84(3):879-911. https://doi.org/10.1111/rssb.12491
- Held, L., Pawel, S. and Schwab, S. (2020). Replication power and regression to the mean. Significance, 17(6):10-11. https://doi.org/10.1111/1740-9713.01462
- Held, L. and Pawel, S. (2020). Comment on "The role of p-values in judging the strength of Statistics in Biopharmaceutical Research, 13(1):46-48. evidence and realistic replication expectations". https://doi.org/10.1080/19466315.2020.1828161
- Held, L. (2020). The harmonic mean X2-test to substantiate scientific findings. Journal of the Royal Statistical Society, Series C, 69(3):697-708. https://doi.org/10.1111/rssc.12410
- Pawel, S. and Held, L. (2020). Probabilistic forecasting of replication studies. PLOS ONE, 15(4):e0231416. https://doi.org/10.1371/journal.pone.0231416
- Held, L. (2020). Discussion on the meeting on 'Signs and sizes: understanding and replicating statistical findings'. Journal of the Royal Statistical Society, Series A, 183(2):449-469. https://doi.org/10.1111/rssa.12544
- Held, L. (2020). A new standard for the analysis and design of replication studies (with discussion). Journal of the Royal Statistical Society, Series A, 183(2):431-469. https://doi.org/10.1111/rssa.12493
R packages
