Information
In conducting meta-analyses, it is imperative to possess a comprehensive comprehension of the subject matter. This may entail engaging in primary research, authoring an exhaustive narrative literature review, or demonstrating extensive pedagogical experience. Should the need for a co-author from the sub-field arise, it is essential to engage a collaborator with similar expertise. If a meta-analysis on the topic is extant, it is incumbent upon the researcher to substantiate the added value of their meta-analysis. This may hinge on factors such as the absence of accommodation for publication bias or heterogeneity in the original meta-analysis. The mere proliferation of new primary studies does not suffice as a justification (Irsova et al., 2023).
Furthermore, it is imperative to exhibit a substantial methodological advancement vis-à-vis the original meta-analysis. Superficial updates are best left as pedagogical exercises or the purview of artificial intelligence. Notwithstanding, exceptions to these guidelines may be warranted when significant advancements in research approaches and methodologies have cast doubt on the robustness of prior meta-analytic findings. Additionally, structural shifts within societies may have rendered previous effect sizes non-representative.
Based on your knowledge of the topic, assemble a list of five primary studies you must include in the meta-analysis. You may enlist a large language model to ensure you have selected the five most important studies. But be careful about relying too much on artificial intelligence since current models often provide factually incorrect results; always double-check and prioritise your expertise. Then, design your main search query using Google Scholar. We prefer Google Scholar to other databases because it includes all papers that have appeared online and allows you to go through the full text of papers, not just the title, abstract, and keywords. This flexibility in search query design empowers you to tailor your search to your specific needs. Using a single main query for a universal database makes it easier for other researchers to replicate your meta-analysis. Remember that Google Scholar's algorithms are subject to change, so depending on your topic, it might be beneficial to use an additional database to strengthen your approach. Use different combinations of the keywords employed in primary studies.
If the five most critical primary studies identified above are among the first hits, you will know that your query is reasonably well prepared. Spend several days fine-tuning the query (improving the percentage of highly relevant primary studies returned among the first 50 hits) and paying attention to the correct search syntax.
Fig. 4 describes the PRISMA standard to guide your search and selection and report your results (Haddaway et al., 2022; Kaufmann & Reips, 2024).
