ePRO Evolution and Emerging Best Practice

1. ePRO Evolution and Emerging Best Practice Adam Wood PDSM PRO Workshop 2010

2. Content • Defining PROs • Where are we now? • Why ePRO? • Obstacles • Emerging Best Practice • What about….? • For reference confidential

3. Defining PROs

4. Defining PROs • “A PRO is a measurement of any aspect of a patient’s health status that comes directly from the patient” - FDA • Patient diaries • Questionnaires • Disease specific measures • ePRO is any electronic implementation • IVRS • Hardware device (e.g. PDA, tablet) • Digital pen confidential

5. Where Are We Now?

6. PROs are all round us • CenterWatch estimate they are used in 75% of trials • Visit based measures more common that remote reported diaries • Some PROs are more important than others • Growing need for Health Economic data • Proving efficacy is one thing, persuading payers to fund your treatment can be quite another • Increasing trend to integrate Health Economics data alongside efficacy data in Ph II/III confidential

7. ePRO is Mainstream • A number of proven vendors with strong track records • Best practice is well established • Success is vastly more likely than failure • Regulators are positive about PROs • “Some treatment effects are known only to the patient”* • ePRO is an integral part of PROs • Not a separate, special case • And it’s not EDC * FDA PRO Guidance. confidential

8. ePRO and Regulators • Regulators in Europe and the US are keenly supportive of electronic PROs • invivodata regularly interacts with EMA and FDA • Drugs approved based on ePRO data worldwide • Japan (at least one) • Europe (multiple) • US (multiple) • invivodata inspected by regulators in all 3 confidential

9. FDA PRO Guidance • Finalised in 2009; drafted in 2006 • Defines new “rules of engagement” for use of PROs • Questions it triggers • Conceptual framework • Application of scale to patient population • Patient input into PRO design • Reliability, validity, ability to detect change • Translations • And, oh yes….mode of administration (p or e) confidential

10. FDA “Spoke and Wheel”* * Figure from: Patrick DL, Burke LB, Powers JH, Scott JA, Rock EP, Dawisha S, et al. FDA Paper Draft: Patient-Reported Outcomes to Support Medical Product Labeling Claims. Value in Health 2007 confidential

11. (Federal Register, Vol. 71, No. 23; Feb. 3, 2006) FDA on Patient Compliance “If a patient diary or some other form of unsupervised data entry is used, the FDA plans to review the protocol to determine what measures are taken to ensure that patients make entries according to the study design and not, for example, just before a clinic visit when their reports will be collected."  (Pg. 10; 334-337 – emphasis added) confidential

12. Why ePRO?

13. Missing data Ambiguous data Conflicting data Extraneous data Is This Familiar? From: “Technology Solutions for the collection of patient reported outcomes data”.  Bill Byrom. European Pharmaceutical Contractor, 2004. confidential

14. Benefits of ePRO • Who hasn’t had a paper diary nightmare? • Process benefits on the theme of cleaner data quicker • More complete – high compliance • Improved integrity – date:time stamps • Increased consistency – logic checks • No free or extraneous text • Automatically coded • Don’t forget better science - far greater compliance with protocol • Who’d want to defend paper diary data to a regulator? confidential

15. Instrumented Paper Diary* N = 40 invivodata eDiary† N = 40 *Contained hidden photosensor †Actual eDiary used in 2002 Study NCI & invivodata Diary Methods Study Stone et al. (2002): British Medical Journal eDiary v Paper confidential

16. 100 80 60 40 20 0 % Compliance Paper eDiary Reported compliance Actual compliance Results: eDiary vs. Paper Diary Stone et al. (2002): British Medical Journal eDiary v Paper • Paper Compliance • Reported: 90% • Actual: 11% • eDiary Compliance • Reported: 94% • Actual: 94% confidential

17. Obstacles

18. Obstacles and overcoming them • Fear of change • We’ve always done it this way • Precedent not acceptable to regulators confidential

19. Obstacles and overcoming them • Lack of awareness of modern ePRO • No longer something like this…. • Or this…. confidential

20. Device ePRO examples confidential

21. More obstacles • Not enough time? • Not enough time for what? • Not enough time to plan for success = planning for failure • Sites won’t like it? • Where’s the evidence for this? • Who pays them? • Doesn’t stop us using EDC widely confidential

22. PRO Scale Migration • Consensus has emerged on best practice • Documented by ISPOR “PRO Good Research Practices Task Force” • Published in Value in Health • Coons et al, 1098-3015/08 • “Recommendations on Evidence Needed to Support Measurement Equivalence between Electronic and Paper-Based Patient-Reported Outcome (PRO) Measures: ISPOR ePRO Good Research Practices Task Force Report” confidential

23. Levels of Modification Adapted from Shields et al. confidential

24. Examples of Modification Adapted from Shields et al. confidential

25. Evidence Required Adapted from Shields et al. confidential

26. Patients can’t use it? • Severe Parkinson’s Disease patient confidential

27. Living with the PRO Guidance

28. Emerging Best Practice • PRO Strategy • PRO Dossier • PRO input into protocol • An Endpoint Development Process confidential

29. “Begin with the end in mind” Labeling Claims Disease/Conceptual Model Measurement Strategy Instruments/Items Conceptual Framework Endpoint Model Endpoints confidential

30. A Few Words on Instruments • When is a PRO instrument adequate to support labeling claims? • “The adequacy of a PRO instrument as a measure to support medical product labeling claims depends on its documented measurement properties that demonstrate the instrument is ‘fit for purpose’” (Burke, 2008). • “Fit for purpose” means that the instrument is specific to and defined by the patient population and the specified disease, treatment, and selected endpoints. • In this circumstance, “instrument is defined as a means to capture data (i.e., a questionnaire) plus all the information and documentation that supports its use” (Burke, 2008). confidential

31. Impact of FDA PRO Guidance • Study Endpoints and Labeling Division (SEALD) are reviewing programs where labeling claims based upon PRO • Feedback from FDA specifically cites PRO Guidance • Expectations for PRO activity and documentation being driven by Guidance • PRO Submission/dossier template • Created a need for practical resources to better understand the nature of PROs • best practices that facilitate the effective development of PRO instruments. confidential

32. Common Pitfalls/Myths • We can go at risk and then just ask the FDA after-the-fact • Lack of patient-interview based data • Needed to support the importance and relevance of the concepts and items in the PRO instrument • The instrument is ‘valid’ in patient population A, therefore it is OK to use in patient population B • We can wait until Ph III to address the PRO instrument issues confidential

33. Ongoing Synthesis of Material Conceptual Model Development Concept Justification Measurement Strategy Instrument Identification Conceptual Framework Endpoint Model Endpoint Development Process • A tool for meeting scientific and regulatory needs confidential

34. Endpoint Development Process Ongoing Synthesis of Material Conceptual Model Development Concept Justification Measurement Strategy Instrument Identification Conceptual Framework Endpoint Model Identify Program Goals & Claims Expert Input Focus of Assessment Identify Existing Instruments Theoretical Validation Evidence/Plan Study Design Requirements Identify Relevant Population Patient Reports Interval of Assessment Modify Existing Instruments Psychometric Validation Evidence/Plan Construction of Endpoints Identify Relevant Drug Effects Empirical Evidence Recall Period Identify Alternate Measurement Strategies Identify Relevant PRO Concepts Timing & Schedule of Assessment Develop Novel Instruments confidential

35. Conceptual Model: Specifying & Substantiating Concepts Signs/ Symptoms General Impact Disease Related Impact Drug Action Concept Justification Impact 1A Patient Population Relevant Disease Process • Signs • Symptoms Impact 2A Impact 1B Diagnosis confidential

36. Concept Justification Yes indicates that this justification criterion has been satisfied No indicates that this justification criterion has not yet been satisfied TBD indicates To Be Determined confidential

37. Conceptual Framework PRO Concept Item 1 Item 2 PRO Concept Label Claim Item 3 PRO Concept Item 4 Item 5 Label Claim PRO Concept Item 6 confidential

38. PRO Strategy Pointers • Begin thinking of PRO strategy early in development • Interact with regulators • PRO EDP provides structure for elements of PRO Strategy and Dossier • Think about preparation for trial implementation confidential

39. What about….?

40. What about ePRO Design • ePRO Solution Design • Maximise value of new modalities • Don’t simply recreate the weaknesses of paper instruments • Less missing data • More precise measures • More frequent, reliable measures confidential

41. (Federal Register, Vol. 71, No. 23; Feb. 3, 2006) FDA’s view on PRO Design “PRO instruments that require patients to rely on memory … may threaten the accuracy of the PRO data. It is usually better to construct items that ask patients to describe their current state than to ask them to compare their current state with an earlier period or to attempt to average their experiences over a period of time.” (Pg. 11; 339-343 – emphasis added) confidential

42. What about…? • Psychometric validation • Luckily the FDA have written us an abridged text book on psychometrics • It’s called the FDA PRO Guidance • Applies to paper as well • Eliciting the patient perspective is a central tenet confidential

43. Psychometric Validation • Large proportion of the FDA PRO Guidance refers to psychometric aspects • With any PRO you need to show evidence that: • “The adequacy of a PRO instrument as a measure to support medical claims depends on its development history and demonstrated measurement properties” • Reliability • Validity • Ability to detect change • Interpretability • A specialised area that applies as much to paper as electronic confidential

44. What about Linguistic Validation? • Translations (to a high standard) • Full title: linguistic validation & cultural adaptation • Best practice well established • Plenty of vendors to choose from • Timelines typically 8 – 11 weeks • Applies to paper as well confidential

45. For Reference

46. Scientific Improvements • Patients are answering questions at the time of clinical interest • In other words as specified in the protocol and not just before a clinic visit • Patients give more complete answer sets so improving statistical power (ref: ACT 2004) • ePRO allows more sophisticated and targeted questioning • Competing compounds are now likely to be using ePRO • Who wants to be seen using inferior methods. confidential

47. Regulatory Motives • EMA and FDA regularly accept ePRO data for primary and secondary endpoints. • This is not something they view with suspicion. • Recently BfArM (the German regulatory authority) explicitly directed a sponsor to use ePRO rather than paper • The FDA PRO Guidance sets out clear expectations for the use of any form of PROs (paper or electronic) • http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM193282.pdf confidential

48. Features of Different ePRO Types confidential

49. Thank You Adam Wood e: awood@invivodata.com  +44 78 5492 8430