Should Systematic Reviews Really be at the top of the Evidence Hierarchy?

Systematic reviews answer a specific research question through a systematic approach of searching for and analyzing research studies. Essentially, they are studies of research studies.

Systematic reviews are considered to be at the top of the hierarchy of evidence. But systematic reviews are only as good as the studies included in the systematic review.

In order to answer the question – “should systematic reviews really be at the top of the evidence hierarchy?” – let’s break down the strengths and limitations of systematic reviews.

Check out our previous blog on the hierarchy of evidence specifically in physiotherapy research.

What is a systematic review in physiotherapy research?

A systematic review in physiotherapy research aims to answer questions about the effectiveness of a specific physiotherapy intervention, like this example of a systematic review that was recently highlighted on PhysioNetwork.

Most often in physiotherapy research, the type of studies included in systematic reviews are clinical trials, in which one group received an intervention while the other group was a control or placebo. (In physiotherapy research, the control group will often receive an intervention that is standard clinical practice.)

A systematic review will synthesize several clinical trials in order to generate conclusions on the overall effectiveness of the intervention in question.

A meta-analysis is a specific type of systematic review that is considered to be even higher on the hierarchy. In addition to describing the quality of the evidence from the included studies, it quantifies the effect of the intervention.

What are the strengths of a systematic review?

• Systematic reviews allow us to draw conclusions about a specific intervention that has been heavily researched.
• They also help us synthesize studies that have conflicting results in order to draw one overall conclusion.
• They can be timesavers for clinicians trying to stay up to date on current research.

However, there are limitations of systematic reviews, especially in physiotherapy research, that must be considered. And clinicians must be savvy consumers of systematic reviews before implementing the content into clinical practice.

What are the limitations of a systematic review?

A recent review article published in the European Journal of Pain highlighted the limitations of systematic reviews in pain research (1). These also apply to physiotherapy research.

• Risk of bias:

Systematic reviews and meta-analyses of low quality can introduce bias thereby producing more positive conclusions. Therefore, it is very important that the reviewers used validated tools to assess quality of evidence and risk of bias of the included studies (2).

• Small sample size:

From the clinical trials included in a systematic review. The problem with small sample size is that it can lead to bias, meaning the results are less likely to be valid/reflective of the real world. This can inflate the study effect, making the intervention appear more effective than it truly is in the real world.

So… should systematic reviews really be at the top of the evidence hierarchy?

Yes, systematic reviews should be at the top of the hierarchy of evidence, BUT as long as they are of high quality with sufficient sample sizes and risk of bias reported using validated tools.

There has been debate about putting systematic reviews at the top of the hierarchy. Putting them at the top may give them too much clout, especially if the included trials were poor quality. Therefore, other hierarchies have been proposed. For example, this hierarchy casts systematic reviews as a “lens through which evidence is viewed” (3).

Tips for reading systematic reviews:

Use the following tips when reading and implementing the content of systematic reviews into your practice:

• Make sure the systematic review is up to date and includes the latest studies.
• When performing a literature search on a topic, first seek out systematic reviews published by non-biased entities, such as the Cochrane Collaboration.
• Check that it reviewed the quality of evidence and risk of bias using validated tools.
• Consider the limitations, including small sample size, of not only the systematic review, but also the included studies. Consult the individual studies for more information, as needed.
• After reviewing a systematic review, use your clinical judgment to determine if the intervention is worth implementing into your clinical practice.