8. Module: QUANTITATIVE ANALYSIS: META-ANALYSIS

Meta-analysis is a fundamental tool of modern evidence-based practice that enables researchers to systematically and statistically combine data from different studies to derive broader insights. Meta-analysis is the synthesis of many studies from an integrated perspective, and this Module seeks to explore the principles, designs, and applications of meta-analysis from its theoretical and practical applications in different fields.

With this Module we expect to accustom participants with the systematic and statistical principles that lead to high-quality meta-analyses and to be competent with different disciplines where this method applies.

It is carefully designed with content that provides an overview of the essentials of systematic reviews and meta-analysis, as well as procedural steps on how to conduct it, such as:

1. Ability To Critically Assess Research: Learn to evaluate the quality and reliability of primary studies, a crucial step to build an evidence base.
2. Combine Complex Data: Learn statistical methods to integrate and analyse different types of data, uncovering meaningful and generalizable insights.
3. Address Variability: Tackle the challenges of heterogeneity inherent in individual study designs and outcomes by employing advanced models, such as fixed-effect and random-effects models.
4. Bias Mitigation: Utilize methods to recognize and reduce publication bias, preserving the rigor of combined outcomes.
5. Enhance Decision Making: Derive actionable recommendations from meta-analytic results for policy, practice, and future research agendas.

This Module aims to equip learners or experienced researchers with the principles and instruments needed to perform meta-analyses of the highest quality, from systematic reviews and effect size estimation to model choice, and to cultivate a culture of evidence-based inquiry and innovation in answering complex research questions.

Participants in this Module should be able to:

1. Differentiate between Systematic Review and Meta-Analysis;

2. Understand and apply the intricate methodology, its components, and its metrics;

3. Assess its quality, factors, and potential biases;

4. Carry out your Meta-Analysis Projects.

This module is designed to equip you with the essential skills and knowledge needed to conduct and interpret meta-analysis, a powerful statistical tool for synthesizing research results across multiple studies.

Throughout the module, you will learn about key concepts and methodologies that can be applied in various fields, including medicine, psychology, education and social sciences.

By the end of this course, you will be able to critically evaluate meta-analytic literature, perform your own meta-analyses, and apply these techniques to increase the rigour and reliability of research.

Whether you are a novice or an experienced researcher, this course will provide you with information and tools to improve your understanding and practice of meta-analysis.

José Manuel Carvalho Vieira (PhD)

Maia University (Portugal)

AJE Team. (2023). Assessing and Avoiding Publication Bias in Meta-analyses | AJE. Springer Nature. https://www.aje.com/arc/assessing-and-avoiding-publication-bias-in-meta-analyses/

Blackhall, K., & Ker, K. (2007). Finding studies for inclusion in systematic reviews of interventions for injury prevention – the importance of grey and unpublished literature. Injury Prevention, 13(5), 359. https://doi.org/10.1136/ip.2007.017020

Cheung, M. W.-L. (2015). Meta-Analysis: A Structural Equation Modeling Approach. Wiley.

Cheung, M. W.-L., & Vijayakumar, R. (2016). A Guide to Conducting a Meta-Analysis. NEUROPSYCHOLOGY REVIEW, 26(2), 121–128. https://doi.org/10.1007/s11065-016-9319-z

Côté, I. M., & Jennions, M. D. (2013). 2. The Procedure of Meta-analysis in a Nutshell. In Handbook of Meta-analysis in Ecology and Evolution (pp. 14–24). Princeton University Press. https://doi.org/10.1515/9781400846184-004

Davis, D. W., Carrier, B., Barrios, B., Cruz, K., & Navalta, J. W. (2021). A protocol and novel tool for systematically reviewing the effects of mindful walking on mental and cardiovascular health. PLOS ONE, 16(10), 1–11. https://doi.org/10.1371/journal.pone.0258424

Deeks, J., Higgins, J., & Altman, D. (2023). Chapter 10: Analysing data and undertaking meta-analyses. In Cochrane Handbook for Systematic Reviews of Interventions version 6.4. https://training.cochrane.org/handbook/current

DerSimonian, R., & Kacker, R. (2007). Random-effects model for meta-analysis of clinical trials: An update. Contemporary Clinical Trials, 28(2), 105–114. https://doi.org/10.1016/j.cct.2006.04.004

Field, A. P., & Gillett, R. (2010). How to do a meta‐analysis. British Journal of Mathematical and Statistical Psychology, 63(3), 665–694. https://doi.org/10.1348/000711010X502733

Freelon, D. (2010). ReCal: Intercoder Reliability Calculation as a Web Service. International Journal of Internet Science, 5, 20–33.

Freelon, D. (2013). ReCal OIR: Ordinal, Interval, and Ratio Intercoder Reliability as a Web Service. Int. J. Internet Sci., 8, 10–16.

Glass, G. V. (1976). Primary, Secondary, and Meta-Analysis of Research. Educational Researcher, 5(10), 3–8. https://doi.org/10.3102/0013189X005010003

Gurevitch, J., Koricheva, J., Nakagawa, S., & Stewart, G. (2018). Meta-analysis and the science of research synthesis. Nature, 555(7695), 175–182. https://doi.org/10.1038/nature25753

Haddaway, N. R., Page, M. J., Pritchard, C. C., & McGuinness, L. A. (2022). PRISMA2020: An R package and Shiny app for producing PRISMA 2020-compliant flow diagrams, with interactivity for optimised digital transparency and Open Synthesis. Campbell Systematic Reviews, 18(2), 1–12. https://doi.org/10.1002/cl2.1230

Hansen, C., Steinmetz, H., & Block, J. (2022). How to conduct a meta-analysis in eight steps: A practical guide. Management Review Quarterly, 72(1), 1–19. https://doi.org/10.1007/s11301-021-00247-4

Harrer, M., Cuijpers, P., Furukawa, T. A., & Ebert, D. D. (2021). Pooling Effect Sizes | Doing Meta-Analysis in R. In Doing Meta-Analysis with R: A Hands-On Guide (1st ed.). Chapman & Hall/CRC. https://bookdown.org/MathiasHarrer/Doing_Meta_Analysis_in_R/pooling-es.html

Havranek, T., & Irsova, Z. (2016). Do Borders Really Slash Trade? A Meta-Analysis. IMF Economic Review, 65(2), 365–396. https://doi.org/10.1057/s41308-016-0001-5

Higgins, J., Thomas, J., Cumpston, C., & Welch, V. (2023). Cochrane Handbook for Systematic Reviews of Interventions version 6.4. Cochrane. https://training.cochrane.org/handbook/current

Ioannidis, J. (2017). Next-generation systematic reviews: Prospective meta-analysis, individual-level data, networks and umbrella reviews. British Journal of Sports Medicine, 51(20), 1456–1458. https://doi.org/10.1136/bjsports-2017-097621

Irsova, Z., Doucouliagos, H., Havranek, T., & Stanley, T. D. (2023). Meta-analysis of social science research: A practitioner’s guide. Journal of Economic Surveys, n/a(n/a). https://doi.org/10.1111/joes.12595

Jak, S. (2015). Meta-Analytic Structural Equation Modelling. Springer International Publishing. https://doi.org/10.1007/978-3-319-27174-3

Kaufmann, E., & Reips, U.-D. (2024). Meta-analysis in a digitalized world: A step-by-step primer. Behavior Research Methods. https://doi.org/10.3758/s13428-024-02374-8

Kepes, S., Wang, W., & Cortina, J. M. (2023). Heterogeneity in Meta-Analytic Effect Sizes: An Assessment of the Current State of the Literature. Organizational Research Methods, 10944281231169942. https://doi.org/10.1177/10944281231169942

Koricheva, J., Gurevitch, J., & Mengersen, K. (Eds.). (2013). Handbook of meta-analysis in ecology and evolution. Princeton University Press.

Mathur, M. B. (2024). Sensitivity analysis for the interactive effects of internal bias and publication bias in meta-analyses. Research Synthesis Methods, 15(1), 21–43. https://doi.org/10.1002/jrsm.1667

O’Rourke, K. (2007). An historical perspective on meta-analysis: Dealing quantitatively with varying study results. Journal of the Royal Society of Medicine, 100(12), 579–582. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2121629/

Page, M. J., Sterne, J. A. C., Higgins, J. P. T., & Egger, M. (2021). Investigating and dealing with publication bias and other reporting biases in meta-analyses of health research: A review. Research Synthesis Methods, 12(2), 248–259. https://doi.org/10.1002/jrsm.1468

Papakostidis, C., & Giannoudis, P. V. (2023). Meta-analysis. What have we learned? Injury, 54(3), S30–S34. https://doi.org/10.1016/j.injury.2022.06.012

Schmid, C. H., Stijnen, T., & White, I. R. (2020). Handbook of Meta-Analysis (C. H. Schmid, T. Stijnen, & I. White, Eds.; 1st ed.). Taylor and Francis. https://doi.org/10.1201/9781315119403

Seidler, A. L., Hunter, K. E., Cheyne, S., Ghersi, D., Berlin, J. A., & Askie, L. (2019). A guide to prospective meta-analysis. BMJ, l5342. https://doi.org/10.1136/bmj.l5342

Sen, S., & Yildirim, I. (2022). A Tutorial on How to Conduct Meta-Analysis with IBM SPSS Statistics. Psych, 4(4), Article 4. https://doi.org/10.3390/psych4040049

Sterne, J. A. C., Savović, J., Page, M. J., Elbers, R. G., Blencowe, N. S., Boutron, I., Cates, C. J., Cheng, H.-Y., Corbett, M. S., Eldridge, S. M., Emberson, J. R., Hernán, M. A., Hopewell, S., Hróbjartsson, A., Junqueira, D. R., Jüni, P., Kirkham, J. J., Lasserson, T., Li, T., … Higgins, J. P. T. (2019). RoB 2: A revised tool for assessing risk of bias in randomised trials. BMJ (Clinical Research Ed.), 366, l4898. https://doi.org/10.1136/bmj.l4898

Tawfik, G. M., Dila, K. A. S., Mohamed, M. Y. F., Tam, D. N. H., Kien, N. D., Ahmed, A. M., & Huy, N. T. (2019). A step by step guide for conducting a systematic review and meta-analysis with simulation data. Tropical Medicine and Health, 47(1), 46. https://doi.org/10.1186/s41182-019-0165-6

Wallace, B. C., Dahabreh, I. J., Schmid, C. H., Lau, J., & Trikalinos, T. A. (2013). Modernizing the systematic review process to inform comparative effectiveness: Tools and methods. Journal of Comparative Effectiveness Research, 2(3), 273–282. https://doi.org/10.2217/cer.13.17

Yusuff, H. (2023). Systematic review and meta-analysis. Journal of Global Medicine, 3(S1), e133. https://doi.org/10.51496/jogm.v3.S1.133

Zigraiova, D., Havranek, T., & Novak, J. (2020). How puzzling is the forward premium puzzle? A meta-analysis (46; Working Paper Series). European Stability Mechanism. https://www.esm.europa.eu/system/files/document/wp46.pdf