Systematic Reviews and Meta-analysis

#A planned approach to the evidence

A systematic review addresses a defined question using explicit methods to find, select and assess relevant studies. Its purpose is to reduce arbitrary choices and provide a more complete picture than a handful of selected papers. A documented plan helps readers judge whether the process was consistent.

Important features include clear eligibility criteria, a sufficiently broad search and an assessment of study limitations. Reviewers also need to explain how information was extracted and disagreements resolved. Even a careful review can miss unavailable research, so the search date and access to unpublished findings matter.

#What pooling can and cannot do

Meta-analysis is a statistical method for combining results from multiple studies. It can improve precision and help explore variation between findings. A systematic review does not have to include one: combining results may be inappropriate when the studies ask substantially different questions or use incompatible measurements.

A pooled estimate depends on choices about which studies to include, how to weight them and which statistical model to use. Differences between studies, often called heterogeneity, may reflect populations, interventions, methods or chance. A single average can conceal meaningful differences rather than resolve them.

#Assess the certainty of the conclusion

Combining studies does not cancel out their biases. If the underlying research shares the same weakness, a pooled result may be precisely misleading. Selective publication can also distort the picture if studies with certain results are more likely to become available for inclusion.

#Read beyond the headline estimate

Readers should examine the range of findings, their uncertainty, study quality and relevance to the question. Conclusions may also depend heavily on one large study or on a particular analytical choice. Sensitivity analyses can show whether the main finding changes when those choices are varied.

Reviews can become outdated as new research appears. Their conclusions should state how confident readers can be, not simply whether a pooled result crossed a statistical threshold. A well-conducted review may reasonably conclude that the available evidence is too limited for a firm answer.

#Common misunderstandings

A systematic review is not simply a long reading list, and a meta-analysis is not automatically stronger evidence than every individual study. “Systematic” describes an explicit method for finding, selecting and assessing research. It does not guarantee that the available studies are reliable or that the review was conducted well.

A common misunderstanding is that every systematic review must include a meta-analysis. Sometimes studies differ too much in their participants, treatments or outcome measures for a combined number to be meaningful. Explaining those differences can be more useful than calculating an average.

Another misconception is that several reviews reaching the same conclusion provide separate confirmation. Reviews may draw on many of the same studies, so their conclusions are not necessarily independent. Likewise, a recently published review may rely on an older search. Its publication date alone does not show how current its evidence is. Missing or unpublished findings can also affect the picture.

#Questions worth asking a clinician

• What explicit methods did this systematic review use to find relevant studies and assess their risk of bias?
• Was combining these studies in a meta-analysis scientifically appropriate, given differences in participants, treatments and outcomes?
• Does the pooled result improve precision while still reflecting biases in the included studies?
• When was the review’s search last updated, and could newer or unpublished studies change its conclusions?
• How did differences between studies affect the findings and their relevance to my care?