1 / 33

Critical Appraisal

Critical Appraisal. Or Making Reading More Worthwhile www.bradfordvts.co.uk. The Problem. Vast and expanding literature. Limited time to read. Different reasons to read – mean different strategies. Keeping up to date. Answering specific clinical questions. Pursuing a research interest. .

Download Presentation

Critical Appraisal

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Critical Appraisal Or Making Reading More Worthwhile www.bradfordvts.co.uk

  2. The Problem • Vast and expanding literature. • Limited time to read. • Different reasons to read – mean different strategies. • Keeping up to date. • Answering specific clinical questions. • Pursuing a research interest. Bruce Davies

  3. Stages • Clarify your reasons for reading. • Specify your information need. • Identify relevant literature. • Critically appraise what you read. Bruce Davies

  4. Clarify Your Reasons for Reading. • Keeping up to date. • Skimming the main journals and summary bulletins. • Answering specific clinical questions. • Finding good quality literature on subject. • Pursuing a research interest. • Extensive literature searching. Bruce Davies

  5. Specify Your Information Need. • What kind of reports do I want? • How much detail do I need? • How comprehensive do I need to be? • How far back should I search? • The answers to these questions should flow from the reasons for reading. Bruce Davies

  6. Identify Relevant Literature. • There are many ways of finding literature. • Remember to ask a librarian – they are the experts. • Selectivity is the key to successful critical appraisal. Bruce Davies

  7. Critically Appraise What You Read. • Separating the wheat from the chaff. • Time is limited – you should aim to quickly stop reading the dross. • Others contain useful information mixed with rubbish. • Simple checklists enable the useful information to be identified. Bruce Davies

  8. Questions to Ask • Is it of interest? • Why was it done? • How was it done? • What has been found? • What are the implications? • What else is of interest? Bruce Davies

  9. Questions to Ask • Is it of interest? • Title, abstract, source. • Why was it done? • Introduction. • Should end with a clear statement of the purpose of the study. • The absence of such a statement can imply that the authors had no clear idea of what they were trying to find out. • Or they didn’t find anything but wanted to publish! Bruce Davies

  10. Questions to Ask • How was it done? • Methods. • Brief but should include enough detail to enable one to judge quality. • Must include who was studied and how they were recruited. • Basic demographics must be there. • An important guide to the quality of the paper. Bruce Davies

  11. Questions to Ask • What has it found? • Results. • The data should be there – not just statistics. • Are the aims in the introduction addressed in the results? • Look for illogical sequences, bland statements of results. • ? Flaws and inconsistencies. • All research has some flaws – this is not nit picking, the impact of the flaws need to assessed. Bruce Davies

  12. Questions to Ask • What are the implications? • Abstract / discussion. • The whole use of research is how far the results can be generalised. • All authors will tend to think their work is more important than the rest of us! • What is new here? • What does it mean for health care? • Is it relevant to my patients? Bruce Davies

  13. Questions to Ask • What else is of interest? • Introduction / discussion. • Useful references? • Important or novel ideas? • Even if the results are discounted it doesn’t mean there is nothing of value. Bruce Davies

  14. NOT JUST A FAULT FINDING EXERCISE ! Bruce Davies

  15. What Is the Method? • The first task – alternative check lists for different methods. • How was the study conducted to confirm the method? • Authors sometimes use the wrong words to describe their work! Bruce Davies

  16. Surveys • Describe how things are now. • Samples of populations or special groups. • The samples must be randomly selected. Bruce Davies

  17. Surveys • Do not have separate control or comparison groups. • Comparisons may be made between subgroups – but this is not control. • Use of the term survey should identify the method – beware the use in what is really a cohort study. Bruce Davies

  18. Surveys • Cross-sectional is seldom used to describe other methods. • There are many ways of selecting a sample: e.g: stratified, cluster and systematic. Bruce Davies

  19. Cohort Studies • Used to find out what happens to patients. • A group is identified and then watched to see what events befall them. • May have comparison or control groups – who must be identified from the start. • Not an essential feature tho.’ Bruce Davies

  20. Cohort Studies • Must have the element of time flowing forward, from the point at which they are identified. • This is sometimes called a retrospective cohort study. • The term cohort should be diagnostic. Bruce Davies

  21. Clinical Trials • Testing. • Always concerned with effectiveness. • The focus should always be on the outcome. • The outcomes may not be beneficial – in other words side effect trials. • Sometimes cohort trials are used to assess effectiveness. This is very poor research and can usually be dismissed. Bruce Davies

  22. Clinical Trials • When more than two things are compared it makes the study more complex and harder to get right. • The key words to look for are: random allocation, double blind, single blind, placebo-controlled. • The term outcome is sometimes used in cohort studies as well. Bruce Davies

  23. Case-control Studies • Ask what makes groups of patients different. • Select a set of patients with a characteristic – eg a disease. • The characteristics of this set are then compared with a control group who do not have the characteristic being studied – but all all other respects must be as identical as possible. Bruce Davies

  24. Case-control Studies • Case studies look backward – not forward as cohort studies do. • Other terms used are: case-referrent, case-comparator, case-comparison. • May also be called retrospective – as is used for some cohort studies. Bruce Davies

  25. The Results • The major mental challenge. • What do I think this really means? • CAUTION. • Large unexpected results are rare. • Flawed studies and misleading findings are common. Bruce Davies

  26. Statistics • A subject in itself. • Some general thoughts are worth emphasising. Size matters. Bruce Davies

  27. Probability • Is just that. • It is not proof. • Think of horse racing and the lottery. • Think of the odds. Bruce Davies

  28. Pitfalls • All statistical tests make assumptions about the raw data. • If there is no raw data presented you cannot know if the tests are meaningful. • Outliers. Bruce Davies

  29. Pitfalls • Skew. • Non-independence. • Serendipity masquerading as hypothesis – or data trawling! Bruce Davies

  30. Pitfalls • Black box analyses. Modern computers make statistical testing easy – the authors may not know what they are doing! • Bias – play devils advocate. • Confounding. • A very common problem in medicine. • Colour televisions do not cause increases in hypertension. Bruce Davies

  31. Checklists • Checklists for particular types of literature are a quick and easy way of learning critical appraisal. • They all have 3 stages: • Basic questions. • Essential appraisal. • Detailed appraisal. Bruce Davies

  32. BUT, BUT, BUT • Checklists do not tell you about the quality or usefulness. • This is still a subjective question. • All the lists do is enable a more structured and thoughtful response to the subjective question. Bruce Davies

  33. References • The Pocket guide to critical appraisal. • By: Iain Crombie. • Pub BMJ Books, 1996. • How to lie with statistics. • By: Darrell Huff. • Pub: Pelican, 1989. Bruce Davies

More Related