There are a variety of issues to watch out for when evaluating systematic reviews that can indicate a particular publication may not meet standards for quality, strength of evidence, or limiting bias. When systematic reviews are published with these issues, it can be an indicator of lax quality standards on the part of journal editors and reviewers, and lack of awareness of methodologies on the part of authors.
Personnel Issues
- Only one author
- Search development did not involve a librarian or other literature search expert
- Only one screener
- Insufficient subject matter expertise
- Meta analysis did not involve a statistician or other mathematics expert
Methodology Issues
- No protocol
- Poorly defined inclusion/exclusion criteria
- Search strategy inadequately documented and not reproducible
- Search conducted in only one database
- Search omits key databases
- Database filters used without justification
- Search omits grey literature
- General failure to understand, apply, or follow accepted methodologies and reporting standards
- It is unfortunately common for systematic review publications to state that they were "conducted according to PRISMA". This is a mistake because PRISMA is a standard for reporting systematic reviews, not conducting them.
Additional Information
