Systematic Reviews

Systematic Reviews are a team effort and should not be conducted by solo researchers.

There are a variety of roles the team members need to fill, including:

Project Lead

• Supervises the overall progress of the project
• Responsibilities include:
  • Overseeing the creation of the research protocol
  • Assigning tasks
  • Monitoring progress
  • Setting deadlines
  • Managing writing of the manuscript
  • Managing dissemination of results

Content Specialists

• Subject matter experts
• Responsibilities include:
  • Screening articles against inclusion/exclusion criteria
  • Critical assessment of relevant studies
  • Data analysis of relevant studies
• To limit bias, teams should include at least two Content Specialists
• Discrepancies can be settled between the Content Specialists themselves or by a third party
• Project Leads may play this role if they are sufficiently familiar with the study's subject matter

Literature Search Expert

• Acquires project data
• Responsibilities include:
  • Selecting databases appropriate to the study's subject matter
  • Translating the research question into database search queries
  • Building search strategies to retrieve relevant publications
  • Documenting search strategies
  • Downloading search results for screening by Content Specialists
  • Updating search results
• Literature searches should be conducted by a single search expert to ensure comparable search strategies across databases
• This role should be filled by a librarian or information specialist

Statistician

• Conducts statistical analyses
• Responsibilities include:
  • Extracting data from relevant studies
  • Performing complex statistical calcuations
  • Producing meta-analyses
• This role should be filled by an experienced statistician or mathematician

Additional Information

There are a number of standards that are used to guide the process of conducting and reporting Systematic Reviews:

Cochrane Handbook for Systematic Reviews of Interventions

• Describes the process involved in preparing and maintaining Cochrane systematic reviews on the effects of healthcare interventions
• Provides direction on the standard methods involved in conducting a systematic review

Finding What Works in Health Care from the Institute of Medicine

• Recommends 21 standards for developing high-quality systematic reviews
• Addresses the entire systematic review process
• Proposes a framework for improving the quality of the science underpinning systematic reviews

Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA)

• An evidence-based minimum set of items for reporting in systematic reviews and meta-analyses
• Focuses on the reporting of reviews that evaluate the effects of interventions
• Can also be used for reporting systematic reviews with objectives other than evaluating interventions

Systematic reviews are comprised of multiple sequential components. These proceed from formulating the systematic review’s topic, scope, and research questions, to selecting and evaluating studies, extracting and synthesizing data, assessing the strength of evidence, and preparing and disseminating the report.

The main stages involved in producing a systematic review include the following steps:

1. Defining the purpose, topic, and scope of the systematic review

   • The purpose of a systematic review is the question it is trying to answer
   • The topic is the subject of the review
   • The scope describes the patients and problems to be included within the topic
   • These concepts need to be defined before you begin the systematic review process



2. Developing research questions, analytic frameworks, and protocols

   • The topic and scope are further defined by:
     • A set of structured research questions
     • An analytic framework (a schematic outline of the systematic review)
   • The research protocol incorporates:
     • The context and rationale for the systematic review
     • The research questions
     • The analytic framework
     • An outline of the proposed methodology



3. Building the team and managing the project

   • A competent systematic review team requires recruiting personnel with expertise in:
     • Clinical content areas relevant to the review
     • Systematic review methodology
     • Expertise in searching for relevant evidence
     • Data and statistical analysis methods
     • Research support
   • Identifying potential conflicts of interest among team members is essential to maintain the objectivity of the systematic review



4. Determining inclusion and exclusion criteria for studies

   • Eligibility criteria that define the studies containing the evidence that address the research questions
   • Defined by the systematic review team before selecting studies for inclusion



5. Conducting searches for relevant studies

   • Planning, designing, and implementing the searches for relevant studies
   • Selecting databases and appropriate search terms
   • Careful documentation of the strategies used



6. Selecting studies for inclusion

   • Selecting studies for inclusion in the systematic review against the inclusion/exclusion criteria
   • Assigning multiple team members to review each article can reduce errors and bias



7. Extracting data from studies and constructing evidence tables

   • Identifying and recording relevant data from selected articles
   • Entering data into evidence tables that provide a reference source of all included studies



8. Assessing quality and applicability of studies

   • Systematically assessing study quality using evaluation criteria defined in the initial phase
   • High-quality empirical studies are more likely to contain accurate and valid outcomes and thus return reliable conclusions



9. Qualitative analysis

   • Evaluating included studies in the context of their:
     • Clinical characteristics
     • Methodological characteristics
     • Strengths, limitations, and biases
     • Relevance to research questions and intended populations



10. Quantitative analysis

    • Data from multiple sources is typically synthesized via meta-analysis
    • Pooling data from two or more studies to improve statistical power and support conclusions
    • Combined data must be relatively comparable, with similar methodologies and population charateristics



11. Assessing and rating the strength of the body of evidence

    • The final step in the synthesis of a systematic review
    • Establishes the strength of the evidence collected
    • Evaluates the quality of the evidence against specific research questions and outcomes



12. Preparing and publishing the report

    • Details vary by type of review
    • The PRISMA checklist is an established guide for the final report

An alternate presentation of the Systematic Review process comes from Tsafnat, G., Glasziou, P., Choong, M.K. et al. Systematic review automation technologies. Syst Rev 3, 74 (2014). https://doi.org/10.1186/2046-4053-3-74. This figure is licensed CC-BY.