Systematic Review Process

Before you begin, think about:

  • Is a systematic review really necessary?
  • Do you have the time and the resources – both human and library support?

Conducting a review

Steps / How the librarian can help
Define the question / Assist with narrowing/refining the question.
Search systematic reviews to see if the question has already been addressed.
Assemble the team
Assemble a team with appropriate expertise (i.e., clinical content, SR methods, searching, quantitative methods, other as appropriate)
Create a review protocol
Produce review protocol per Cochrane, IOM, CRD or similar templates. For example:
  • Conceptual discussion of the problem
  • Review question
  • Search strategy
  • Study inclusion and exclusion criteria
  • Study quality assessment [see below]
  • Data extraction procedure [see below]
  • Data synthesis procedure
  • Record keeping
  • Project timetable
/ Assist in writing protocol. Especially:
  • Review question
  • Search strategy
  • Record keeping

Conduct a thorough search
Work with a librarian or other information specialist trained in performing SR to plan search strategy and conduct searches (IOM, Standard 3.1.1)
Consider core and specialized bibliographic databases and other sources of unpublished data [grey literature] selected in consultation with your medical librarian
Conduct handsearching of major journals; review reference lists of relevant articles; contact experts in the field
Select appropriate study design filter
Manage the search results [e.g. EndNote] / Construct answerable question
Select core and specialized databases
Select grey literature sources
Construct concept table
Identify search terms:
  • Select controlled vocabulary terms
  • Select appropriate textwords
  • Apply appropriate logic and positional operators
  • Apply appropriate study design filter
Execute scoping searches
Execute final search
Set up alerting services
Manage results [e.g., EndNote]
Select studies per protocol
Reduce by title/abstract first pass; obtain full-text of remainder
  • Have eligibility checked by more than one reviewer
  • Develop strategy to resolve disagreements
  • Keep log of excluded studies, with reasons for exclusions
/ Assist with obtaining full-text of requested studies
Appraise the quality of studies per protocol
Consider assessment by more than one reviewer (strongly recommended)
Use simple checklists and/or quality scales
Assess concealment of treatment allocation, blinding and handling of patient attrition
Consider blinding of reviewers to authors, institutions and journals
Extract data
Design and pilot data extraction form
Consider data extraction by more than one reviewer (strongly recommended)
Consider blinding of extractors to authors, institutions, and journals
Analyze and present results
Tabulate results from individual studies
Examine forest plot
Explore sources of heterogeneity
Consider meta-analysis of all trials or subgroups of trials
Perform sensitivity analyses; examine funnel plots
Make list of excluded studies available to interested readers
Interpret results
Consider limitations, including publication and related biases
Consider strength of evidence
Consider applicability
Consider economic implications
Consider implications for future research
Write up the report
Include all sections according to PRISMA or your template of choice / Write:
  • Details of the full search
  • “flow” of studies through the review process (including how many studies were excluded and why)

From:

IOM (Institute of Medicine). 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press.

The Stages of a Systematic Review
Created byDr David Denyer, Management Practice Fellow, Advanced Institute of Management Research (AIM), and Senior Lecturer in Organization Studies, Cranfield School of Management.

Steps in Conducting a Systematic Review (Box 2.1, page 25) in Systematic Reviews in Health Care: Meta-Analysis in Context. Mathhias Egger, George Davey Smith, and Douglas Altman, Eds. BMJ Books, 2001.

Dartmouth Biomedical Libraries

September 2012