How To Do A Meta-Analysis In A Systematic Review | Stepwise Transparent Reproducible

What A Meta-Analysis Adds To A Systematic Review

Systematic reviewing maps the field; meta-analysis answers the “how big” question. By turning study outcomes into common effect sizes and pooling them, you move past anecdotes and see a clearer signal. Done well, this work sets expectations for patients, policy, and product teams. The key is discipline: predefine the question, lock methods, and keep a full audit trail. The reward is a transparent estimate with uncertainty, paired with plain-language takeaways that readers can trust.

Doing A Meta-Analysis Inside A Systematic Review: Scope And Plan

Start with a tight question. PICO (Population, Intervention, Comparator, Outcome) keeps scope in bounds and turns wish lists into testable aims. Write a protocol that names outcomes, study designs, time frames, and analysis plans. Register it so the record is public. Many teams use PROSPERO for that step. For reporting, line your sections up with the PRISMA 2020 checklist. For methods details, keep the Cochrane Handbook at hand. Name your primary outcome, set a small set of prespecified subgroups, and write exact rules for tie-breaks before you see any results.

Core Planning Milestones And Outputs

Search And Select Studies Without Bias

Work with a librarian or an experienced searcher. Build strings from controlled terms plus free text. Search at least two large databases for biomedicine or the top sources for your field. Add trial registers and preprints if they matter for your topic. Export everything with full fields, remove exact duplicates, then screen titles and abstracts in pairs with masked, independent votes. Resolve conflicts with a third voter using your written rules. Track counts at every step and draw a PRISMA flow chart. Keep a log of outreach for missing data and of any recoding choices.

Eligibility Rules That Stay Consistent

Inclusion and exclusion rules should match your PICO and be testable. Define study designs up front. Set language and date limits only when they are part of the question. Keep reasons for exclusion short and standardized so the PRISMA flow is easy to read. Do a pilot round on a small set to tune the rules, then lock them. If you must change a rule midstream, write the change, the reason, and the date.

Extract Data With Reproducible Fields

Build a form that captures study ID, design, setting, arm sizes, outcome measures, time points, summary stats, and risk-of-bias judgments. Include fields for unit-of-analysis quirks such as cluster designs, crossovers, or multi-arm trials. Capture both raw counts and the transformed effect size so you can retrace steps. Two people extract independently; a third resolves differences. Keep original numbers and units in one place and your derived numbers in another. Link each number to a page, table, or figure in the source so back-checks are quick.

Assess Study Quality And Risk Of Bias

Use tools that match design. For randomized trials, RoB 2 fits many cases. For nonrandomized studies of interventions, ROBINS-I maps common pitfalls. Score domains, not just totals. Keep judgments and quotes linked to specific text in the paper. Plan sensitivity runs that drop high-risk studies or key domains. Later, carry these judgments into your certainty summary so the main table reflects both the size of the effect and the trust you can place in it.

Compute Effect Sizes Correctly

Pick one effect metric per outcome family. State it in the protocol and stick to it unless the data force a change. Convert study results to that metric with the same formulas across studies. Store the effect and its standard error or its variance; that pair drives weights in most models. Where a paper gives medians with ranges, use accepted conversions or contact authors. For zero cells in two-by-two tables, apply a small, consistent continuity correction or use exact methods that avoid ad-hoc tweaks.

Dichotomous Outcomes

Risk ratio and odds ratio are common picks. Risk ratio is easier to read when the event is not rare, while odds ratio is handy for logistic outputs. Convert counts to the log effect and the standard error so pooling is straightforward. For cluster trials that fail to adjust, correct the standard error with an intracluster correlation if you can find one from similar work.

Continuous Outcomes

When scales match, use mean difference. When scales differ but measure the same concept, use standardized mean difference with a small-sample correction. Convert change scores and endpoint scores to the same basis to avoid mixing scales. If a paper omits a standard deviation, back-calculate from a confidence interval or a p-value when possible, and mark the method in your log.

Time-To-Event And Proportions

For survival data, hazard ratio on the log scale is the usual pick. For single-arm rates or proportions, transform to a scale that keeps variances stable across the range; logit or Freeman–Tukey can help in edge cases. State each choice and keep it the same across studies within an outcome.

Choose A Model And Pool The Effects

Fixed-effect models answer a narrow question: one true effect underlies all the studies, and chance drives spread. Random-effects models allow genuine between-study differences and give a mean of a distribution. Pick the frame that matches your question and design plan. For random-effects, REML or Paule-Mandel are common picks for tau-squared; DL is simple but can misstate uncertainty with few studies or high spread. Always show the interval around tau-squared or a prediction interval for the pooled effect so readers see the likely range in a new setting.

Heterogeneity And Weighting

Report Q, I², and tau², then explain what those values mean for your outcome. A modest I² can still matter if the effect moves across a decision threshold. Weighting follows the variance, so studies with tighter standard errors will pull harder. Multi-arm trials need special care so no study gets counted twice. Split shared comparators or use a multivariate model when arms overlap.

Model And Effect Cheat Sheet

Check Spread, Influence, And Fit

Look for odd studies that drive the result. Run leave-one-out checks, influence measures, and alternative estimators for tau-squared. If a single trial flips the sign or wipes the interval, say so and explain why that study differs. Subgroup runs should match the list in your protocol. Meta-regression is fine for patterns across studies when you have enough studies and a sensible covariate, but treat it as exploratory unless the plan named it as primary. Keep plots simple and label axes with units and scales so a reader can scan in seconds.

Study Size Bias And Missing Evidence

Small studies can inflate effects. Funnel plots can flag that pattern, and regression tests can add a p-value, but both need enough studies and a mix of sizes. Trim-and-fill is easy to run but can mislead when assumptions fail. Pair these checks with a narrative on search scope, non-English hits, preprints, and grey sources. If you find many registered but unreported trials, state how that gap could shift the needle. When outcome data are missing, list the pattern and try author contact or pre-set imputation rules. Mark each imputation in the dataset so readers can see which points are estimated.

From Numbers To Meaning: Certainty And Use

Effect size is half the story; certainty is the other half. Present a Summary of Findings table that ties the pooled effect to the trust you can place in it, with short notes on study limits, inconsistency, imprecision, and bias risks. Map your judgments to clear language about who might benefit, by how much, and at what risk. This is where method work meets real-world choices. Keep the prose straight and avoid hedging jargon. Readers should grasp the main point in one pass and find all the backup in tables and appendices.

Write For PRISMA And For Humans

Structure the manuscript around the reader’s workflow. Title and abstract state the question and the main numeric answer. Methods list the protocol record, full search strings, dates, and screening rules. Results show the PRISMA flow, a forest plot for each outcome, risk-of-bias plots, and any planned subgroup or meta-regression. Discussion states limits without hand-waving and lays out what the result means for decisions. Match each section to PRISMA items and link to a public repository with your forms, code, and data. That way anyone can retrace your steps and rerun the pool with updated trials later.

Steps For Conducting A Meta Analysis In A Systematic Review

Step-By-Step Checklist

1. Write a PICO and a one-line aim.
2. Draft and register a protocol with clear rules and planned analyses.
3. Build database strings, add registers, and set export formats.
4. Screen in pairs with masked votes; log every decision.
5. Extract in pairs; store raw and derived fields side by side.
6. Judge risk of bias by domain with quotes to support calls.
7. Pick a single effect metric per outcome and convert consistently.
8. Choose fixed-effect or random-effects to match the question.
9. Report Q, I², tau², and a prediction interval when you can.
10. Run planned subgroups; add leave-one-out and influence checks.
11. Probe size bias with plots and tests when study counts allow.
12. Summarize certainty and craft a clear Summary of Findings table.
13. Write to PRISMA with a crisp flow diagram and full search strings.
14. Share forms, code, and de-identified data in a public repository.

Practical Tips That Save Time And Headaches

Nail The Units And Time Points

Many disputes come from mixed scales and mismatched follow-up times. Pick a primary time point in the protocol and stick to it. If papers report several points, store them all but tag the one you will pool. Convert all units to a single set before computing effects, and show the mapping in an appendix.

Handle Multi-Arm Trials Without Double-Counting

When one study has two arms that both meet your rules, avoid giving it extra weight. Split the shared comparator or use a model that accounts for correlation. Spell out the rule in your protocol and log the math so a reader can check it in seconds.

Keep A Living Log

Version every major file and store changes with a short reason. Tag runs with a date and a commit hash if you use code. When you revise a number, do not overwrite the original; add a new field that records the fix. This habit turns panic into routine when a reviewer asks for a trace.

Figures And Tables That Carry The Load

A single forest plot with clear labels can do more than a page of prose. Sort studies by weight or by risk of bias to show patterns. Use consistent symbols for subgroups and a separate color or marker for high-risk studies. Add a prediction interval line on random-effects plots. For funnels, scale the y-axis to a measure of precision so the shape is easy to scan.

Limitations, Sensitivity, And What To Share

No dataset is perfect. Say where judgments could sway the result and show what happens when you change a rule. Share a fully runnable package that includes extraction forms, a codebook, the cleaned dataset, and scripts to rebuild every figure and table. That bundle turns your review from a one-off paper into a base other teams can update when new trials land.

Common Pitfalls And Easy Fixes

Mixing Endpoints

Pooling change scores with endpoint scores without adjustment will bend the result. Align the choice or convert with a reported correlation when possible.

Selective Time Windows

Picking the “best” time point after seeing the data invites bias. Pre-state the window and stick with it. Show other time points in a figure or appendix.

Hidden Unit Issues

Cluster trials that fail to adjust will look too precise. Fix the standard error with a design effect and explain the source of the intracluster value you used.

From Plan To Publish: A Short Template You Can Reuse

Abstract

Question, data sources, dates, main effect with interval, certainty, and one line on use.

Methods

Protocol link, full strings, screening rules, extraction form link, risk-of-bias tool, effect metrics, model choice, subgroups, and software.

Results

PRISMA flow, table of study features, forest plots, heterogeneity stats, subgroup and influence runs, and bias checks.

Discussion

Limits, fit with prior work, and practical takeaways for the target audience. Keep the tone calm and the claims tight.

Final Words: Make It Readable And Re-Runnable

Meta-analysis is not a black box. With a crisp protocol, a broad search, careful extraction, sound models, and open files, any trained reader can follow your steps and reach the same line items. That is the standard readers expect and the way to build reviews that stand up when the next wave of studies arrives.