Critical Appraisal Workshop Created by Sebastian Leuschner Š
Why is it important? • Basis of medical practice – evidence-based medicine – Individual patient level – Local / National / International Guidelines
• Assessment • Interesting
How to... • • • •
Background Methods Results Conclusions
How to... Background • • • • •
What is the reason for this research? What is the research question? Is it important? How was it funded? Competing interests?
How to... Methods • • • • • •
Appropriate study design? Appropriate selection criteria? Ethical approval? Sample size & power? Withdrawal bias? Appropriate outcome measures?
• Statistics appropriate?
Study design Observational
Randomised controlled trial
•
• •
Researchers observe associations – Cohort • Prospective
– Case control • Typically retrospective
•
•
Most compelling evidence Randomised – Study subjects are randomly allocated to either receive intervention or no intervention/placebo Blinding – Subjects unaware which group they’re in – Double-blinding neither subject nor researchers aware – Best way to avoid bias Placebo-controlled
How to... Methods • RCT – Appropriate control? – Treatment equal in control & treatment group?
How to... Results • What was found? • Is it clear? • Do tables / figures reflect what is said in the text? • Results generalisable? • Look closely at this section!!
How to... Conclusions • • • • •
Has the research question been answered? Are the conclusions justified? Important limitations? What are the implications? Does it stimulate further research?
CFS & chocolate overview • Background – Poorly justified study • Evidence on cannabinoids weak at best
– – – – –
Research Q: Do flavonoids improve fatigue in CFS? It’s important Funding not acknowledged Conflict of interest: SB Head of Research for Nestlé None of the authors have an obvious clinical / research interest in CFS (article, google)
CFS & chocolate overview • Methods positives • Ethical approval gained, patient consent • Well designed cross-over trial with reasonable wash-out period • Randomisation to treatment • Self-reported questionnaire eliminate assessor bias • Treatment & control chocolate same in colour & composition except flavonoids • Statistics appropriate (power calculation, nonparametric analysis, intention-to-treat analysis)
CFS & chocolate overview • Method points needing consideration – CFS diagnostic criteria well-recognised? – Various assessment tools validated? – Sample representative of CFS patients with these exclusion criteria? – Chocolate taste really the same? No detail on “taste trial” – Other chocolate intake during trial not checked – (Small sample size)
CFS & chocolate overview • Results – Significant improvement with flavonoid choc* – Significant deterioration with control choc* – No significant weight gain – Mention of anecdotes inappropriate – Tables match text
CFS & chocolate overview • Conclusions – Research question answered – Some limitations acknowledged • Clearly stated only proof of concept
– Other limitations not stated • Control group
– Certain results unexplained • Lack of weight gain • Deterioration in control groups
– Use of “significantly”
CFS & chocolate overview • Judgements – Opens up avenues for further research – Unlikely to change clinical practice – Not generalisable • • • •
Conflict of interest Participants Control arm Small sample size
– Nestlé PR stunt rather than scientific research?
Eating chocolate may ease the symptoms of chronic fatigue syndrome , scientists claim Read more: http://www.metro.co.uk/news/848279-chocolate-may-ease-chronicfatigue-syndrome-scientists-claim#ixzz1FI1LjoTF
Chocolate could be the cure for chronic fatigue syndrome – Daily Mail
Others... • BBC • Student BMJ • Various health & chocolate websites • All claim chocolate may be the cure for CFS