Skip to Main Content

Research and Scholarly Activity Guide for Psychiatry Faculty, Residents, Fellows, and Students

What is the empirical evidence base?

The Department of Psychiatry's strategic plan for research includes this tactic:
Require a review of the empirical evidence base for any clinical topic presented in Grand Rounds.

So how do you identify the empirical evidence base?

Definition: empirical evidence is defined in the APA Dictionary of Psychology as "derived from or denoting experimentation or systematic observations as the basis for conclusion or determination" 

Approaches in narrowing searches or results to empirical evidence:

  • Choose a database that is limited to content based on empirical evidence, e.g. Cochrane Database of Systematic Reviews (aka Cochrane Library), OR.
  • Use Study Designs as part of your search terms (see Medical Subject Headings for examples for Epidemiologic Studies below) OR
  • Limit to Publication Type (see PubMed Publication Types as an example below) OR
  • Look at the article abstract for an indication of the Level of Evidence (see next section) if that is provided by the journal. 
  • other ways may vary by database

Levels of Evidence

Levels of evidence are assigned to studies based on the methodological quality of their design, validity, and applicability to patient care.

There are several well-known levels of evidence schemas: 

Evidence Pyramids

There are different "evidence pyramids" or paradigms for hierarchy of levels of evidence. 

Types of studies higher up the pyramid are generally considered to be stronger evidence than types of studies or resources lower down in the evidence pyramid. The pyramid below does not show where qualitative research fits into your evidence, but it is worthy for consideration as you review the literature. 

Evidence-Based Medicine (EBM) Pyramid

EBM Pyramid and EBM Page Generator, copyright 2006 Trustees of Dartmouth College and Yale University. All Rights Reserved. Produced by Jan Glover, David Izzo, Karen Odato and Lei Wang.

Another representation of the evidence-based pyramid

New York Medical College. Retrieved from https://guides.library.nymc.edu/c.php?g=944467&p=6807843 .

 

  • Meta-Analysis  A systematic review that uses quantitative methods to summarize the results.
  • Systematic Review    An article in which the authors have systematically searched for, appraised, and summarized all of the medical literature for a specific topic.
  • The systematic review or meta-analysis of randomized controlled trials (RCTs) and evidence-based practice guidelines are considered to be the strongest level of evidence on which to guide practice decisions. (Melnyk, 2004) The weakest level of evidence is the opinion from authorities and/or reports of expert committees.
  • Systematic reviews, meta-analyses, and critically-appraised topics/articles have all gone through an evaluation process: they have been "filtered". 
  • Information that has not been critically appraised is considered "unfiltered".
  • As you move up the pyramid, however, fewer studies are available; it's important to recognize that high levels of evidence may not exist for your clinical question.  If this is the case, you'll need to move down the pyramid if your quest for resources at the top of the pyramid is unsuccessful.
  • Critically Appraised Topic     Authors of critically-appraised topics evaluate and synthesize multiple research studies.
  • Critically Appraised Articles  Authors of critically-appraised individual articles evaluate and synopsize individual research studies.
  • Randomized Controlled Trials  RCT's include a randomized group of patients in an experimental group and a control group. These groups are followed up for the variables/outcomes of interest.
  • Cohort Study  Identifies two groups (cohorts) of patients, one which did receive the exposure of interest, and one which did not, and following these cohorts forward for the outcome of interest.
  • Case-Control Study  Involves identifying patients who have the outcome of interest (cases) and control patients without the same outcome, and looking to see if they had the exposure of interest.
  • Background Information / Expert Opinion   Handbooks, encyclopedias, and textbooks often provide a good foundation or introduction and often include generalized information about a condition.  While background information presents a convenient summary, often it takes about three years for this type of literature to be published.
  • Animal Research / Lab Studies  Information begins at the bottom of the pyramid: this is where ideas and laboratory research takes place. Ideas turn into therapies and diagnostic tools, which then are tested with lab models and
    animals.

Searching for Systematic Reviews and Evidence Syntheses

Search for the best evidence first.

Cochrane Library and Campbell Collaboration are databases focused on systematic reviews, links to both can be found below.

Systematic reviews can also be found by searching on your topic in PsycINFO or PubMed (MEDLINE) and using the filters in Additional Limits (PsycINFO) or on the left-hand side of the search results page (PubMed) to limit to Systematic Reviews. See links below.

You can also reference the A-Z list of databases listed below provided by the OHSU Library.

If a systematic review already exists on your question, evaluate the quality of this review and search for other articles on this topic that have been published since the existing systematic review came out. For example, if the most recent systematic review you have found was published in 2013, you would want to search for all articles on this topic published from then to the present and incorporate this new overview/synthesis into what the scientific community knew about this up to 2013, as represented by the systematic review.

If you cannot find any systematic review after looking at Cochrane, Campbell, and in PubMed, it is probable that one does not exist. This gives you the opportunity to be the person to conduct such a systematic review, thus providing a foundational overview on this topic to the field.

Evidence Critical Appraisal

If you do not find any systematic reviews where someone else has synthesized the evidence on your topic, you will need to do this evaluation or appraisal of the evidence directly from the individual research articles. Ways to critically appraise research studies are listed below:

When appraising research, keep the following three criteria in mind:

Quality 
Trials that are randomized and double blind, to avoid selection and observer bias, and where we know what happened to most of the subjects in the trial.

Validity
Trials that mimic clinical practice, or could be used in clinical practice, and with outcomes that make sense. For instance, in chronic disorders we want long-term, not short-term trials. We are [also] ... interested in outcomes that are large, useful, and statistically very significant (p < 0.01, a 1 in 100 chance of being wrong).

Size
Trials (or collections of trials) that have large numbers of patients, to avoid being wrong because of the random play of chance. For instance, to be sure that a number needed to treat (NNT) of 2.5 is really between 2 and 3, we need results from about 500 patients. If that NNT is above 5, we need data from thousands of patients.

These are the criteria on which we should judge evidence. For it to be strong evidence, it has to fulfill the requirements of all three criteria."

How to read a paper