How To Do Data Synthesis In Systematic Reviews | Plan Combine Report

Steps For Data Synthesis In A Systematic Review

Start with a protocol and stick to it. Define outcomes, time points, eligible designs, subgroup plans, and software. Map every choice before touching the numbers. That plan will anchor the work and ease reporting later under the PRISMA 2020 checklist.

Plan The Comparisons

List each contrast you will pool (e.g., intervention vs control). Link outcomes to the same contrast and time window. Decide which measures can be combined and which need their own lanes.

Decide The Synthesis Route

Pick one primary route for each outcome: meta-analysis when studies line up, or a structured narrative when they do not. Use alternative quantitative routes when scales differ or designs vary.

Doing Data Synthesis In Systematic Reviews: Start Smart

Extract Complete, Comparable Data

Capture the arm-level numbers you need for each effect measure: counts for dichotomous outcomes, means and SDs for continuous outcomes, and hazard ratios for time-to-event outcomes. Note analysis sets, time points, and any adjustments. Record unit conversions and imputed values so each step is auditable.

Harmonize Scales And Time Windows

Align units (mg/dL to mmol/L, minutes to hours), match follow-up windows, and pick a single direction where higher always means the same thing. When only medians are reported, use established conversions with caution and flag the choice in sensitivity checks.

Choose Effect Measures That Match The Data

Use risk ratio or odds ratio for binary outcomes. Use mean difference when all studies share a scale; switch to standardized mean difference when they do not. For survival data, stick with hazard ratios.

Pick Models And Quantify Between-Study Differences

Model choice comes from your question and the spread of effects. A fixed-effect model treats studies as estimates of one common effect. A random-effects model treats study effects as a distribution. Report your choice and why, and show heterogeneity stats alongside the pooled answer.

Core Heterogeneity Checks

Scan forest plots for overlap, then report I² and τ². Large I² means the spread is not just sampling noise; τ² expresses the variance of true effects. Don’t treat any single number as a pass/fail gate; use them to guide model choice and further checks.

Safer Random-Effects Inference

When you use random-effects, small sets of studies can give too-narrow intervals with classic methods. Use Hartung-Knapp (with REML for τ²) for more reliable intervals, and avoid leaning on DerSimonian–Laird unless you’ve shown it holds up.

When studies are few, present the fixed-effect result too so readers can see how much the answer shifts by model.

Run The Meta-Analysis

Document the software, exact functions, and settings. For each outcome, report the model, the effect measure, the pooled estimate, CI, prediction interval for random-effects, heterogeneity stats, and the study count. Keep the same order and phrasing across outcomes so readers can scan fast. The Cochrane Handbook gives clear, field-tested formulas and defaults.

Subgroups And Sensitivity Checks

Run only the subgroup splits you prespecified. Typical splits include risk of bias, dose, setting, and follow-up length. For sensitivity checks, swap models (fixed vs random), swap τ² estimators, drop outliers, or switch between RR and OR when both are defensible. Report each change and how the answer moved.

Meta-Regression With Restraint

Meta-regression can point to patterns across studies. Keep the number of covariates low, base the model on study-level inputs, and note that such patterns don’t prove causation. Treat them as context for the main pooled result.

When Meta-Analysis Doesn’t Fit

Sometimes pooling isn’t honest: outcomes are too mixed, designs vary too widely, or statistics are missing. In those cases, build a structured narrative that still answers the question with evidence, not anecdotes.

Build A Transparent Narrative

Group studies by design, population, or exposure level. Within each group, line up outcomes on the same scale or direction and summarize the pattern with text and simple visuals. Use effect-direction plots or harvest plots to show which way results point and how precise they are.

Avoid Common Traps

Don’t “vote count” p-values. Don’t treat lack of significance as lack of effect. Always link claims about patterns to the studies that back them, and include reasons for differences across results such as dose, time window, or risk of bias.

Check For Reporting Bias And Small-Study Effects

Map planned outcomes against what was reported. If many planned outcomes are missing, flag a risk of bias due to missing results. For meta-analyses with enough studies, add funnel plots and tests like Egger’s, but state their limits when heterogeneity is high or study counts are low.

Handle Missing Or Messy Statistics

Rebuild Standard Deviations When Needed

Many trials report SEs, CIs, or p-values instead of SDs. Rebuild SDs from those figures using standard formulas and show the path in an appendix. When authors provide medians and IQRs only, apply published conversion rules with caution and test the influence in sensitivity runs.

Avoid Double Counting

Multi-arm trials can tempt you to reuse a shared control. Split the shared group or use a multi-arm model so each participant contributes once. With cross-over designs, use paired analyses or extract first-period data if carryover clouds the picture.

Adjust Cluster Trials

When clusters are randomized but the paper reports at the person level, adjust the effective sample size using the intracluster correlation. If the ICC is missing, use a sensible value from similar work and flag this choice in sensitivity checks.

Align Designs And Units Of Analysis

Match Outcome Definitions

Write clear rules for what counts as an event, which scale anchors apply, and which time window qualifies as the main endpoint. Keep the same rules across studies so the pooled answer reflects like-for-like data.

Pick One Direction Per Outcome

Make sure higher always means better or worse across all trials in a synthesis set. Flip signs as needed so the forest plot reads the same way for every study.

Handle Repeated Measures

When outcomes are measured at several times, pick the prespecified window for the main synthesis. If you also pool a change-from-baseline and an end-of-study measure, state why and show both without mixing the two within one analysis.

Transparency, Code, And Data

Keep a machine-readable dataset with per-study inputs and derived values. Store scripts with locked seeds and versioned packages. Label each figure with the code name that produced it so others can rerun every step. These habits keep updates smooth and build trust in the pooled answer.

Rate Certainty And Present Findings

Readers need a clear view of how much trust to place in each pooled result. Use the GRADE approach to rate certainty across five domains and present outcomes in a compact table. The CDC’s ACIP handbook walks through the steps and levels used by GRADE, and matches well with health reviews.

Write For Reuse And PRISMA

State the synthesis route for each outcome, with the reasons. Name the model, estimator, and any small-sample tweaks. List subgroup and sensitivity rules as they appeared in the protocol. Add a “what would change the answer” note to help users judge transportability.

Close with tidy, repeatable outputs: the meta-analysis dataset, code, and a short guide to reproduce every figure. Clear reporting earns trust and speeds updates under the PRISMA 2020 checklist and Summary of Findings norms used with the GRADE approach.

Quick Reference: Measures, Models, And Graphics

Common Effect Measures

• Risk ratio (RR): Natural for incidence; stable under varied baseline risks.
• Odds ratio (OR): Handy for case-control designs; less intuitive for lay readers.
• Mean difference (MD): Same scale across trials; easy to read.
• Standardized mean difference (SMD): Different scales; expresses change in SD units.
• Hazard ratio (HR): Time-to-event data; assumes proportional hazards.

Model Picks That Age Well

• Report both fixed-effect and random-effects when there are three or fewer studies.
• For random-effects, use REML for τ² and Hartung-Knapp for test statistics and CIs.
• Show a prediction interval so readers can gauge the range expected in a new study.

Graphics That Speed Insight

• Forest plots with the same order across outcomes.
• Contour-enhanced funnel plots when you have at least ten studies.
• Effect-direction plots or harvest plots for narrative syntheses.

Steps To Report Data Synthesis Without Gaps

Methods To Spell Out

1. Eligibility for each synthesis and how studies were grouped.
2. Data prep rules: conversions, imputations, and outlier handling.
3. Exact effect measures and models, with software and versions.
4. Heterogeneity checks and how they shaped model choice.
5. Subgroups, meta-regression, and sensitivity runs.
6. Small-study and reporting bias checks.
7. Certainty ratings with a Summary of Findings layout.

Results To Share The Same Way Every Time

1. Study flow into each synthesis set.
2. Per-study effects with CIs and weights.
3. Pooled effects with CIs and, for random-effects, prediction intervals.
4. I², τ², and Q with p-value.
5. Notes on any departures from the protocol and why.

Common Pitfalls And Clean Fixes

Pooling Apples With Oranges

Don’t mix follow-up times, scales, or exposure windows. Split the outcome, convert the units, or park those studies in a narrative group.

Letting One Large Study Set The Tone

Weighting is good, swamping is not. When one study dominates, show influence plots, try a leave-one-out run, and add a prediction interval to keep the message balanced.

Chasing P-Values

Lead with effect sizes and CIs. P-values alone don’t tell the story readers need.

Make Updates Easy

Synthesis is not a one-time act. New trials appear, preprints finish peer review, and registries release results months later. Write your code so you can drop in new rows and rerun the lot. Keep a changelog that states what changed, why it changed, and how the headline numbers moved. Readers can track the arc of the evidence without guessing.

When an update lands, rerun the same sensitivity rules, refresh prediction interval, and revisit small-study checks. If the answer flips, say it and point to the data that drove the shift. Transparency beats spin, and readers will thank you for it.

Checklist Before You Publish

• Protocol loaded with clear rules for outcomes, windows, models, and subgroups.
• Data file with arm-level inputs, derived values, and audit notes.
• Figures labeled with code names and consistent ordering across outcomes.
• Both fixed-effect and random-effects runs when study counts are small.
• Heterogeneity stats reported next to the pooled effect.
• Small-study and reporting bias checks explained with limits.
• Summary of Findings table tied to a GRADE rating for each outcome.
• Links to the PRISMA 2020 checklist and methods that match the Cochrane Handbook.