Appraising Evidence

The best clinical practice requires one to appraise and use the research evidence available while making clinical decisions. The skill to appraising scientific evidence will become easy if one takes a methodical approach.

In this article, I will provide you with steps to follow while appraising a scientific article. It is key to know the structure of a research article before you attempt to appraise the evidence.

Most research articles will include the following sections:

  1. Title
  2. Abstract
  3. Introduction / Background (leading to the need for research)
  4. Aims
  5. Methods / Methodology
  6. Results / Findings
  7. Conclusion
  8. Discussion
  9. References

Consider asking the following questions while appraising a research article. This is not an exhaustive list of questions to ask while appraising evidence but can be used direct your assessment.

I. Title:

  • Is the title relevant to the aims of the research?
  • Does the title attract the readers while capturing the important theme of the study?
  • Does the title include the ‘PICO’ in a randomised controlled study?
    • Population / Problem / Patients (with disease in question)
    • Intervention / Exposure / Prognostic / Risk factor (under investigation)
    • Comparison / Control group intervention
    • Outcomes

II. Abstract:

  • Are the main features of all sections of the article mentioned briefly and clearly in the abstract?
    • Background: Is the ‘need for research’ mentioned?
    • Aims: Is the ‘main aim’ of the study mentioned?
    • Methods: Study design, randomisation and allocation methods, statistical analyses?
    • Results: Participant characteristics, results of the main hypothesis?
    • Conclusions: Description/explanation of the main results?
    • Discussion: Implication of the findings? How do these findings relate to the existing scientific literature?
  • Does the information packed in the abstract correspond to what is in the manuscript?

III. Introduction:

  • Does the introduction/background section clearly state what already is known about this topic?
  • Is the need for research clearly outlined?

IV. Aims:

  • Is the main aim of this study clearly stated?
  • Does the need for research from the background section and the main hypothesis (aim) of this study relate conceptually?

V. Methods:

  • Participant Selection Process: If the study participants are selected wrongly, the findings from the study risks having limited relevance and applicability to the population.
    • Was the participant selection process described clearly?
    • Are the inclusion and exclusion criteria listed?
    • Was the population from which the participants were selected defined clearly?
    • What is the participants’ likelihood of truly representing the population that they were selected from?
    • What proportion of selected participants completed the study?
    • Are the details of excluded participants, missing data, withdrawals, no-shows, lost to follow-up presented?
    • Can we compare the study sample to similar studies?
    • Can we generalise the study sample to other population?
  • Flow Diagram:
    • Is there a study flow diagram listing participants included, excluded, missed, withdrawn, lost to follow-up and who successfully completed the study (in both experimental & control groups).
  • Did the study have sources of sampling bias while selecting participants by following methods?
    • Convenience sampling (may favour/disfavour certain participants)
    • Volunteer / Self-selection sampling (participants may have similar characteristics)
    • Stratified sampling (may allow selection bias with unclear strata)
    • Random sampling (may result in unbalanced groups)
    • Other sampling methods: systematic, clustered, quota, snowball etc.,
  • Assessing Validity:
    • Did the study use measures/methods with adequate validity?
      • Face validity: Did the study use a measure/method that appears appropriate for the construct it is intending to assess?
      • Construct validity:
        • Did the study use a measure/method that estimates the theoretical construct adequately?
        • Does the measure/method used in the study correlate with other measures/methods that estimate the same theoretical construct adequately?
      • Content validity: Did the study use a measure/method that fully represents the construct it is intending to assess?
  • Assessing Reliability:
    • Did the study use measures (eg., bone mineral density by DEXA) with adequate reliability?
    • Did the study use methods (eg., using two radiographers for diagnosis) to increase the reliability of subjective assessments?
    • Did the study use methods (eg., taking serial MRIs) to increase the reliability of objective measurements (i.e., to reduce systematic/measurement errors)?

VI. Results:

  • Does this study require a statistical analysis? (Literature review vs Method study vs Empirical experiment)
  • Adequate statistics were done?
    • Did the authors justify their sample size?
    • Did the authors report the criteria used for allocation into groups?
    • Did the authors use a clear unit of measurement?
    • Do the data/results address the research question?
  • Presenting data:
    • Are tables and figures understandable without reading the text?
    • Is there a repetition of content between tables/figures and text?
    • Is there adherence to common conventions on units, symbols, format, rounding etc.,?
  • Ensuring validity of data:
    • Did the authors define and report statistical significance clearly?
    • Did the authors report clinically meaningful results using published cutoffs? 
    • Did the authors report minimally important difference in their results? 
    • Were the effect size, practically important differences explained?
    • Did the authors report 95% confidence intervals for their outcomes?
    • Are the data/measurement categories used for analyses clearly defined and are they valid/reliable?

VII. Conclusion:

  • Does the conclusion section report the answer to the main research question (aim) of this study?
  • Does the conclusion section describe/explain/interpret the results succinctly 

VIII. Discussion:

  • Are the main findings re-stated briefly?
  • Are the results interpreted and compared with current literature?
  • Do the findings contribute to the existing body of knowledge?
  • Are the implications of the research mentioned?
  • Are the limitations of the study stated?
  • Find out if the limitations of the study contain serious errors? eg., wrong study design.
  • Are the future research directions suggested?

IX. References: 

  • Did the authors use relevant articles for reference?
  • Did the authors use recent articles for reference?
  • Did the authors cite the referred articles correctly?
  • Can you find the cited articles in the online databases?
  • Did the authors fail to cite seminal articles for their area of research?

Leave a Reply

Your email address will not be published. Required fields are marked *