Evidence-based focus on aphasia rehabilitation

1. Evidence-based focus on aphasia rehabilitation Elizabeth Rochon, Ph.D, Dept. of Speech-Language Pathology, University of Toronto Research, Toronto Rehabilitation Institute SWO Stroke SLP Network Lecture November 18, 2011

2. Outline • Different kinds of Research that informs clinical practice • Phases of Clinical Research and levels of evidence • Aphasia Therapy- where do we stand?

3. Research that Informs Clinical Practice Basic, Translational, Applied • Basic • Aims to increase scientific knowledge. • Is theoretical and aims to improve understanding of behaviour under study. • Lays the foundation for applied research but does not aim to solve or treat the phenomenon under study. • E.g., what is the underlying nature of paraphasic errors in naming in aphasia?

4. Research that Informs Clinical Practice Basic, Translational, Applied • Translational • A bridge between basic and clinical research. • Aims to promote a “bench-to-bedside” approach. (Raymer et al., 2008. Translational research in aphasia: from neuroscience to neurorehabilitation, JSHLR, 51, S259-S275)

5. Research that Informs Clinical Practice Basic, Translational, Applied • Applied • Its basis is basic research. • Aims to address and solve a practical problem. • E.g., What is the most effective treatment for naming disorders in aphasia?

6. Clinical (Translational and Applied) Research • Treatment Efficacy: • Best possible conditions (e.g., controlled environment, ideal study population, etc.) to produce the best possible outcome. • Requires “overt and systematic control over all aspects of the research.” (Robey, 2004, p. 402) • Treatment Effectiveness: • Benefit observed in the real world; is tested only after efficacy has been established. • Refers to benefit provided: “in a typical fashion by typical practitioners to typical patients in typical clinical settings”. (Robey, 2004, p. 402)

7. Clinical (Translational and Applied) Research • Phase I: Exploratory studies • Phase II: Feasibility studies • Phase III: Clinical Trials • Phase IV: Effectiveness Research • Phase V: Cost-effectiveness and consumer satisfaction (Robey et al. (2004). Report on the joint coordinating committee on evidence-based practice, ASHA)

8. Adaptation of 5 Phases for Communication Disorders & Sciences (Robey, 2004) • Phase I: • Selecting a therapeutic effect, establishing if present, estimating magnitude (i.e., therapeutic effect = dependent variable; treatment protocol = independent variable). • Benefit can be assessed at various levels of WHO model (a single study cannot assess all levels)! • Examples of Phase I research questions: • Test to detect effect and establish effect sizes; define population; delineate treatment protocol, estimate dose, etc.): • Examples of Phase I research designs: • Case and single-subject studies, small group pre-post studies, retrospective studies.

9. Adaptation of 5 Phases for Communication Disorders & Sciences (Robey, 2004) • Phase II: • “…Exploring the dimensions of the therapeutic effect and making the necessary preparations for conduction of a clinical trial.” (Robey, 2004, p. 404) • Examples of Phase II research tasks: • Establish presence and magnitude of efficacy; better define target population; refine protocol, determine optimal dosage, refine outcome measurements, etc. • Examples of Phase II research designs: • Hypothesis-testing single-subject studies, small group within-effect, case-control and cohort-control studies; measurement studies re: outcome measures.

10. Adaptation of 5 Phases for Communication Disorders & Sciences (Robey, 2004) • Phase III: • Efficacy is tested via a clinical trial. • Phase III research design: • A parallel-groups design (treatment condition vs. control condition). • Gold standard is randomized controlled trial (RCT). (e.g., treatment vs. control condition, random allocation, concealment, power calculation, etc.,)

11. Adaptation of 5 Phases for Communication Disorders & Sciences (Robey, 2004) • Phase IV: • Field research to determine applicability in clinical practice. • Examples of Phase IV research tasks: • Test effectiveness in target and sub-populations; test effectiveness of different service delivery models or variants of treatment protocol; meta-analyses. • Examples of Phase IV research designs: • Pre-post studies; parallel-groups and hypothesis-driven single-subject studies.

12. Adaptation of 5 Phases for Communication Disorders & Sciences (Robey, 2004) • Phase V: • Determine the benefits and the costs of the treatment. • Cost-effectiveness studies and cost-benefit analyses. • Audience at this stage includes administrators, policy makers, etc.

13. Current Best Evidence EBP Client/Patient Values Clinical Expertise What is Evidence-Based Practice (EBP)? • A process of clinical decision-making that entails: • “The integration of best research evidence with clinical expertise and patient values” (Sackett et al., 2000, in Johnson, 2006)

14. EBP Requires….. • A “show me the data” attitude i.e., sound scientific evidence might contradict opinions of authorities. • A focus on aspects of research relevant to practice i.e., clinically relevant questions using appropriate methodology. • Rigorous criteria to judge the quality of the evidence i.e., in the domains of validity; importance; precision. (Dollaghan, 2004 in Johnson, 2006)

15. 5 Key Steps in EBP • Pose an answerable question. • Search for the evidence. • Critically appraise the evidence for its validity and relevance. • Make a decision by integrating the evidence with clinical experience and patient/client values. • Evaluate performance after acting on the evidence. (from Johnson, 2006, p. 21, as per the Canadian Cochrane Network)

16. Levels of evidence for studies of treatment efficacy (From ASHA 2004 in Johnson, 2006)

17. Challenges to Implementation of EBP • Evidence may not be available or be of low quality. • Answers to complex clinical questions are never black and white. • Approaches and premises from medicine may not transfer easily to SLP. • Finding time to implement EBP. • Ensuring comprehensive searches. • Ensuring adequate critical appraisals. (see Johnson, 2006)

18. Levels of evidence for studies of treatment efficacy • Systematic reviews: • Identifies completed studies related to a specific research question • Evaluates results and arrive at a conclusion • Use a well-defined uniform approach • Useful for developing practice guidelines • Meta-analysis: • Statistical aspect of systematic review • Summary estimate and confidence interval calculated • Together individual study findings are combined to determine the effectiveness of an intervention

19. Aphasia Therapy: What have we learned from meta-analyses? I. Robey, 1998: The “weight of scientific evidence” bearing on: • The effectiveness/efficacy of aphasia treatments considering: pre-post effects and treatment vs. no treatment studies. • Improvement in therapy considering: amount and type of therapy; severity and type of aphasia. • N= 55 quasi-experimental studies

20. Aphasia Therapy: What have we learned from meta-analyses? Results (Robey, 1998) • Recovery is superior for treated vs. untreated individuals at all stages of recovery (especially acute). • For optimal gains, treatment dosage of 2 + hours per week is better than shorter treatment duration. • Large treatment gains found for persons with severe aphasia. • Schuell-Wepman-Darley multimodal stimulation approach resulted in a large effect size (too few examples of other specific treatment approaches to analyze).

21. Aphasia Therapy: What have we learned from meta-analyses? II. Results (Bhogal et al., 2002) • Intensive treatment (i.e., more hours per week) yields better outcomes). • E.g., half of the +ve studies provided an average of 8.8 hours/week for 11.2 weeks.; -ve studies provided approximately 1 hour/week for 22.9 weeks. • Total hours of therapy were greater in more intense studies. • “Intensive therapy delivered over 2 to 3 months is critical to maximizing aphasia recovery….” (p. 991). • (n=10 controlled trials, i.e., treatment vs. control condition)

22. Aphasia Therapy: What have we learned from meta-analyses? III. Cappa et al., 2005: • Class II, Class III and rigorous single-case studies indicate probable effectiveness : grade B recommendation. • A need for further investigation in the field. (European Federation of Neurological Societies, Task Force on Cognitive Rehabilitation)

23. Aphasia Therapy: What have we learned from a Cochrane Review? • Speech and language therapy for aphasia following stroke (Brady & Enderby): • Randomized controlled trials (RCTs) • Speech therapy vs. no speech therapy • Speech therapy vs. social support/stimulation • SLP intervention 1 vs. SLP intervention 2 (The Cochrane Collaboration, 2010) IV.

24. Aphasia Therapy: What have we learned from a Cochrane Review? • Trend favouring intensive over conventional therapy • But significantly more people withdrew from intensive vs. conventional therapy • When therapy is delivered by a therapist-trained and supervised volunteer appears to be as effective as when delivered by a professional. • Not enough evidence to adjudicate between therapy approaches. (The Cochrane Collaboration, 2010)

25. Aphasia Therapy: What have we learned from a Cochrane Review? • Authors note: • Much variability in quality of the evidence (i.e., many, if not most, studies not designed as true clinical trials). • E.g., randomization methodology and methods for concealing allocation, power size calculation, intention-to-treat analysis (The Cochrane Collaboration, 2010)

26. Aphasia Therapy: What have we learned from a Cochrane Review? • “..there may be a benefit from speech and language therapy but there was insufficient evidence to indicate the best approach to delivering speech and language therapy.” (p.2) • Evidence noted especially for functional communication, expressive language and the severity of aphasia. (The Cochrane Collaboration, 2010)

27. Aphasia Therapy: What have we learned from meta-analyses? V. Cicerone et al., 2011 • No advantage for intensive therapy in acute stage • CILT and training in everyday communication > CILT alone • Semantic & phonological treatment equally effective: treatment-specific effects demonstrated

28. Aphasia Therapy: What have we learned from meta-analyses? VI. Snell et al., 2010 • Results not linked to number of items given in therapy • Empirical study: severity of anomia need not determine how many items given in therapy

29. Aphasia Therapy: What have we learned from systematic reviews? VII. Farqi-Shah et al., 2010 • Treatment in L2 yields similar outcomes to treatment in L1. • Cross-language transfer occurred in most studies • Conclusion limited by methodological quality of studies (n=14)

30. Aphasia Therapy: What have we learned from systematic reviews? VIII. Simmons-Mackie et al., 2010 • Training communication partners improves their communication activities and/or participation. • Training communication partners is probably effective at improving communication activities and/or participation of persons with aphasia. • Insufficient evidence: acute aphasia; or, impact on language impairment, psychological adjustment, or QOL .

31. Aphasia Therapy: What have we learned from an evidence-based practice database (PsychBITE)? IX.

32. Aphasia Therapy: What have we learned from an evidence-based practice database (PsychBITE)? IX. • Psychological database for Brain Impairment Treatment Efficacy (PsycBITETM; www.psycbite.com) • (PsycBITETM is modeled on PEDro; used in physio, OT, and SLP via speechBITETM: www.speechbite.com • Includes 5 designs: • Systematic reviews (SR) • RCTs • Non-RCTs (NRCT) • Case series (CS) • Single-Subject designs (SSD) (Togher et al., 2009)

33. Aphasia Therapy: What have we learned from an evidence-based practice database (PsychBITE)? Levels of Evidence: • Class I: Evidence provided by one or more well designed RCTs • Class II: Evidence provided by 1 or more observational study with concurrent controls. • Class III: Evidence provided by expert opinion, case series, case reports.

34. Aphasia Therapy: What have we learned from an evidence-based practice database (PsychBITE)? • Up to September, 2007, n=339 aphasia treatment • Systematic reviews 3% • RCTs 7% • Non-RCTs 5% • Case series 15% • Single-Subject designs 70% (Togher et al., 2009)

35. Criteria Comprising The Pedro Scale(Herbert et al., 1998/99 in Togher et al., 2009) 1. Eligibility criteria of subjects were specified (not included in MQR Score) 2. Subjects were randomly allocated to interventions (in a crossover study, subjects were randomly allocated an order in which treatments were received) 3. Allocation was concealed 4. The intervention groups were similar at baseline regarding the most important outcome measures and prognostic indicators 5. There was blinding of >95% of subjects 6. There was blinding of >95% of therapists who administered the therapy

36. Criteria Comprising The Pedro Scale(Herbert et al., 1998/99 in Togher et al., 2009) 7. There was blinding of >95% of assessors who measured at least one key outcome 8. Measures of at least one key outcome were obtained from >85% of the subjects initially allocated to groups 9. All subjects for whom outcome measures were available received the treatment or control condition as allocated or, where this was not the case, data for at least one key outcome were analysed by ‘‘intention to treat’’ 10. The results of between-intervention-group statistical comparisons are reported for at least one key outcome 11. The study provides both point measures and measures of variability for at least one key outcome

37. Aphasia Therapy: What have we learned from an evidence-based practice database (PsychBITE)? • Mean MQR (methodological quality rating) based on PEDro scale: • RCTs 4.4/10 • Non-RCTs 2.6/10 (Maximum score: RCTs=10; NRCT=8; cannot rate CS and SSD)

38. Aphasia Therapy: What have we learned from an evidence-based practice database (PsychBITE)? Conclusion: • Methodological quality is modest • sources of bias are not sufficiently well controlled • Is this a problem with reporting or with the design and methodology? • Above criteria need to be taken into account when designing, executing and reporting aphasia treatment research. (Togher et al., 2009)

39. Aphasia Therapy: What have we learned from an evidence-based practice database (PsychBITE)? Examples of More Highly Rated Studies in PEDro: • Denes et al. (1996). Intensive versus regular speech therapy in global aphasia: a controlled study. • Doesborgh et al. (2004a). Cues on request: The efficacy of multicue, a computer program for word finidng therapy. • Doesborgh et al. (2004b). Effects of semantic treatment on verbal comunication and linguistic processing in aphasia after stroke: A randomized controlled trial.

40. Aphasia Therapy: What have we learned from an evidence-based practice database (PsychBITE)? Examples More Highly Rated Studies in PEDro: • Huber et al. (1997). Piracetam as an adjuvant to language therapy for aphasia: A randomized double-blind placebo-controlled pilot study. • Kagan et al. (2001). Training volunteers as conversation partners using “Supported conversation for adults with aphasia” (SCA): A controlled trial.

41. Aphasia Therapy: What have we learned from single-subject designs (SCED Scale)? X. • SCED scale developed by Tate et al. (2008) to determine quality of SSDs. • SSDs entail: systematic observation; manipulation of variables of interest, repeated measurement, data analysis. • Can be applied to clinical practice • Systematic approach to evaluation of SSDs would be useful for EBP!

42. The SCED ScaleRating Scale for Single Participant Designs 1. Clinical history was specified. Must include Age, Sex, Aetiology and Severity. 2. Target behaviours. Precise and repeatable measures that are operationally defined. Specify measure of target behaviour. 3. Design 1: 3 phases. Study must be either A-B-A or multiple baseline 4. Design 2: Baseline (pre-treatment phase). Sufficient sampling was conducted 5. Design 3: Treatment phase. Sufficient sampling was conducted http://www.psycbite.com/docs/The_SCED_Scale.pdf

43. The SCED ScaleRating Scale for Single Participant Designs 6. Design 4: Data record. Raw data points were reported 7. Observer bias: Inter-rater reliability was established for at least one measure of target behaviour 8. Independence of assessors 9. Statistical analysis 10. Replication: either across subjects, therapists or settings 11. Evidence for generalisation http://www.psycbite.com/docs/The_SCED_Scale.pdf

44. What have we learned from single-subject designs (SCED Scale)? • Tate et al. (2008) showed that SCED Scale is valid and reliable. • A standardized and systematic approach to evaluation of SSDs would be useful for EBP. • Scales can be used to evaluate quality of SSD articles you read!

45. What have we learned about Aphasia Therapy: Summary • Quality of studies is variable to poor • Some evidence for efficacy/effectiveness for: • Word finding treatment • Supported conversation • Benefits of intensive treatment • Treatment for severe aphasia

46. Conclusion • Treatment research needs to better adhere to conventions of clinical research, recognizing that: • Basic research is still necessary in the field to inform applied research. • Many studies are/will still be efficacy/Phase I or II studies. • Impediments exist to implementation (i.e., small numbers of appropriate subjects). • Single-subject designs will continue to provide good evidence. • Improvements will benefit our clients and the field!

47. Thank-you Questions?