The best clinical practice requires one to appraise and use the research evidence available while making clinical decisions. The skill to appraising scientific evidence will become easy if one takes a methodical approach.

In this article, I will provide you with steps to follow while appraising a randomized controlled study. It is key to know the structure of a research article before you attempt to appraise the evidence.

Most articles that report a randomized controlled study will include the following sections:

1. Title
2. Abstract
3. Introduction / Background (leading to the need for research)
4. Aims
5. Methods / Methodology
6. Results / Findings
7. Conclusion
8. Discussion
9. References

Consider asking the following questions while appraising a research article. This is not an exhaustive list of questions to ask while appraising evidence but can be used direct your assessment.

I. Title:

• Does the title cover the specific theme or the main aim of the research?
• Is the title relevant to the aims of the research?
• Does the title attract the readers while capturing the important theme of the study?
• Does the title include the ‘PICO’ in a randomized controlled study?
  • Population / Problem / Patients (with disease in question)
  • Intervention / Exposure / Prognostic / Risk factor (under investigation)
  • Comparison / Control group intervention
  • Outcomes

II. Abstract:

• Are the main features of all sections of the article mentioned briefly and clearly in the abstract?
  • Background: Is the ‘need for research’ mentioned?
  • Aims: Is the ‘main aim’ of the study mentioned?
  • Methods: Study design, randomization/allocation methods, statistical analyses?
  • Results: Participant characteristics, results of the main hypothesis?
  • Conclusions: Description/explanation of the main results?
  • Discussion: Implication of the findings? How do these findings relate to the existing scientific literature?
• Does the information packed in the abstract correspond to what is in the manuscript?

III. Introduction:

• Does the introduction/background section clearly state what already is known about this topic?
• Is the need for research clearly outlined?

IV. Aims:

• Is the main aim of this study clearly stated?
• Does the need for research from the background section and the main hypothesis (aim) of this study relate conceptually?

V. Methods:

• Subject Selection Process: If the subjects are selected wrongly, the findings from the study risks having limited relevance and applicability to the population other than one sample.
  • Was the subject selection process described clearly?
  • Are the inclusion and exclusion criteria listed?
  • Was the population from which the subjects were selected defined clearly?
  • What is the subjects’ likelihood of truly representing the population that they were selected from?
  • What proportion of selected subjects completed the study?
  • Are the details of excluded subjects, missed cases, withdrawals, no-shows, lost to follow-up presented?
  • Can we compare the study sample to similar studies?
  • Can we generalize the study sample to other population?
• Flow Diagram:
  • Is there a study flow diagram listing subjects included, excluded, missed, withdrawn, lost to follow-up and who successfully completed the study (in both experimental & control groups).
• Did the study have sources of sampling bias while selecting subjects by following methods?
  • Convenience sampling (may favor/disfavor certain subjects)
  • Volunteer / Self-selection sampling (subjects may have similar characteristics)
  • Stratified sampling (may allow selection bias with unclear strata)
  • Random sampling (may result in unbalanced groups)
  • Other sampling methods: systematic, clustered, quota, snowball etc.,
• Assessing Validity:
  • Did the study use measures/methods with adequate validity?
    • Face validity: Did the study use a measure/method that appears appropriate for the construct it is intending to assess?
    • Construct validity:
      • Did the study use a measure/method that estimates the theoretical construct adequately?
      • Does the measure/method used in the study correlate with other measures/methods that estimate the same theoretical construct adequately?
    • Content validity: Did the study use a measure/method that fully represents the construct it is intending to assess?
• Assessing Reliability:
  • Did the study use measures (e.g. bone mineral density by DEXA) with adequate reliability?
  • Did the study use methods (e.g. using two radiographers for diagnosis) to increase the reliability of subjective assessments?
  • Did the study use methods (e.g. taking serial MRIs) to increase the reliability of objective measurements (i.e., to reduce systematic/measurement errors)?

VI. Results:

• Does this study require a statistical analysis? (Literature review vs Method study vs Empirical experiment)
• Adequate statistics were done?
  • Did the authors report sample size calculation?
  • Did they report criteria for allocation into groups?
  • Did they use a clear unit of measurement?
  • Do the data/results address the research question?
• Presenting data:
  • Are tables and figures understandable without reading the text?
  • Is there a repetition of content between tables/figures and text?
  • Is there adherence to common conventions on units, symbols, format, rounding etc.,?
• Ensuring validity of data:
  • Did authors define statistical significance clearly?
  • Were the effect size, practically important differences explained?
  • Are the data/measurement categories used for analyses clearly defined and are they valid/reliable?

VII. Conclusion:

• Does the conclusion section describe/explain/interpret the results succinctly?
• Does the conclusion answer the main research question (aim) of this study?

VIII. Discussion:

• Are the main findings re-stated briefly?
• Are the results interpreted and compared with current literature?
• Do the findings contribute to the existing body of knowledge?
• Are the implications of the research mentioned?
• Are the limitations of the study stated?
• Find out if the limitations of the study contain serious errors? e.g. wrong study design.
• Are the future research directions suggested?

IX. References:

• Did the authors use relevant articles for reference?
• Did the authors use recent articles for reference?
• Did the authors cite the referred articles correctly?
• Can you find the cited articles in the online databases?
• Did the authors fail to cite seminal articles for their area of research?