What frustrates me about evidence

1. What frustrates me about evidence and what changes I'd like to see! Jon Brassey, TRIP Database

2. Background • Worked in information support for over ten years • Started in primary care Q&A - answered over 10,000 clinical questions • Developed the TRIP Database –searched over 50 million times • Main interest – getting robust answers to clinicians queries!

3. Why am I annoyed? • Evidence is not focussed on supporting clinical practice. For me this can be broken down into: • Problems with the evidence • Problems with search

4. Problems with evidence • Frequently doesn’t answer clinical questions • Methodological purity seems more important than clinical usefulness

5. Analysis of dermatology questions • 357 Q&As analysed • How many questions were answered by systematic reviews? • >50% • 25-49% • 10-24% • <10%

6. Analysis of dermatology questions • Only 3 questions answered by a SR! • 63% therapy, 9% diagnosis, 9% etiology • 20% answered using secondary evidence alone • Not much focus, DUETs aside, at feeding in coal-face information needs with production of evidence

7. Evidence Updates • Run by Brian Haynes at McMaster via funding from the BMJ • Examine 120 core clinical journals and rate them on: • Quality • Relevance • Newsworthiness • Overall >95% rejected

8. Problems with search • 100 clinicians • Pain • NHS Evidence

9. Problems with search – 100 • Information seeking defined by the Google generation • Ask 100 clinicians the support they’d want, how many would say – ‘Give me a search box where I add terms and then return 10 results which I need to read in the hope that they will answer my question’.

10. Problems with search – pain Pain

11. Problems with search – NHSE • Nearly £25 million budget • Around £15 million on non-content, including £730,000 marketing • 385,000 searches per month or 4,620,000 annually (excludes CKS and SLs/SCs) • >£3 per search • TRIP Database < 1p

12. Summary of problems • Evidence not fit for purpose • Not findable • Usefulness medical information = (relevance x validity)/ work (Slawson & Shaughnessy)

13. Solutions – better evidence • Build on the work of DUETs and create a system to record clinical uncertainty • Better understand the clinical information needs of ‘normal’ clinicians • Better prioritisation of SRs and primary research • Less is more

14. Solutions – improve search • Work with clinicians and throw away the existing search paradigm • Work better with the search intentions • Break down full-text • Be ambitious and not be constrained by the Google ‘mentality’.

15. TRIP’s experiments • TILT – http://tilt.tripdatabase.com • Shared learning tool • Clinicians record their learning • Snippets of applicable evidence • Blitter – http://blitter.tripdatabase.com • Curated list of clinicians that blog or tweet • Index of content clinicians feel are noteworthy • Clinicians added based on speciality

16. Ultimately... • Need to give clinicians the: • The information they need • In the format they need • In the timeframe they need • To me answering their questions is vastly superior to what we currently have