1 / 47

Randomized Controlled Trials in Health Services Research

Randomized Controlled Trials in Health Services Research . Morris Weinberger, PhD Senior Career Scientist, HSR&D Service Investigator, Center for Health Services Research, Durham VAMC VA Cyber-Seminar, January 12, 2009. Overview to Today’s Seminar. Overview of randomized controlled trials

geraldine
Download Presentation

Randomized Controlled Trials in Health Services Research

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Randomized Controlled Trials in Health Services Research Morris Weinberger, PhD Senior Career Scientist, HSR&D Service Investigator, Center for Health Services Research, Durham VAMC VA Cyber-Seminar, January 12, 2009

  2. Overview to Today’s Seminar • Overview of randomized controlled trials • Minimizing threats to internal validity • Targets of interventions • Analytical issues • Practical issues and advice

  3. Overview of Randomized Controlled Trials • Bias • Systematic (non-random) error • Bane of research, regardless of study design • Can invalidate study results • Can occur in any phase of research

  4. Overview of Randomized Controlled Trials • Most powerful research design to establish causality, including the effectiveness of interventions • Establishes causality by controlling for confounding factors • Well-designed experiments minimize bias • Not suitable for all research questions

  5. Overview of Randomized Controlled Trials • Subjects randomized to treatment groups • Follow subjects prospectively • Compare subjects across treatment groups on relevant outcomes

  6. Required CONSORT Figure

  7. Overview to Today’s Seminar • Overview of randomized trials • Minimizing threats to internal validity

  8. Validity • Internal validity: Can the observed differences between groups be attributed to the intervention? • Randomization • External validity: Are the observed differences in your study representative of patients/subjects in general? • Random sampling

  9. Threats to Internal Validity • Biased assignment of patients to groups • Biased outcome assessment • Non-compliance with treatment protocol • Dropouts • Co-intervention • Contamination

  10. Minimizing Threats to Internal Validity • Randomization • Blinding (Masking) • Intervention design • Study protocol

  11. Randomization • Random assignment of subjects to study groups: • Produces study groups comparable with respect to measured and unmeasured characteristics • Removes investigator bias in assigning patients to groups • Increases validity of statistical tests • If allocation of subjects to groups is predictable, it may lead to bias, e.g., decision to participate

  12. Minimizing Threats to Internal Validity • Randomization • Blinding (Masking)

  13. Minimizing Threats to Internal Validity • Randomization • Blinding (Masking) • Intervention design

  14. Considerations When Designing Interventions • What is the intervention? • Is it likely to be potent? • Is it ethical? • Is it practical and feasible in the “real world”? • Will it be acceptable to patients? • Is it effective?

  15. What is the Intervention?Standardization • Who did what to whom? • What was the dose? • How often? • For how long? • Administered under what conditions? • With what dose adjustments?

  16. Minimizing Threats to Internal Validity • Randomization • Blinding (Masking) • Intervention design • Study protocol

  17. Study Protocol • Choice of control group • Recruitment strategies • Retention strategies • Outcome measures • Evaluating effectiveness of the intervention • Compliance with treatment protocol • Co-intervention • Contamination

  18. Choosing the Control Group • Nothing • Usual Care • Placebo

  19. Recruitment Strategies • Recruitment sources • Community vs. patients • Volunteer bias • Barriers to recruitment • Interest in subject • Distrust of research • Distrust of contact from unknown persons • Transportation • Informed consent • Long enrollment visits

  20. Retention • Impact on external validity of the trial • Who completes the trial? • Impact on internal validity of the trial • What if there is differential dropout? • Impact on number of patients recruited • How will my sample size estimates be affected?

  21. Design Considerations:Choosing Measures • Properties of the measure • Validity and reliability • Floor and ceiling effects • Sensitivity to change • Pragmatic considerations • Setting • Respondent burden • Appropriateness for subjects • Cost

  22. Evaluating Effectiveness of the Intervention • Dose: Was the intervention delivered? • Contamination: Did the control group receive components of the intervention? • Co-intervention: Did the treatment group receive interventions other than what was intended? • Key: Measure what elements of the intervention were delivered to all study groups

  23. Overview to Today’s Seminar • Overview of randomized trials • Minimizing threats to internal validity • Targets of interventions

  24. Targets of Strategies to Improve Outcomes • Patients • Providers • System

  25. Patient-Level Strategies • Randomization is at the patient level • Simplifies the statistical analysis

  26. Provider-Level Strategies • Strategies to improve quality of care by intervening with providers • If outcomes are at physician level, issues are generally similar to patient-level interventions

  27. Provider-Level Strategies • Often, goal is to evaluate the impact of provider interventions on patient outcomes because • Expect intervention with provider to affect patient outcomes (e.g., improving patients’ glycemic control) • Concern that providing intervention to intervention patients will affect providers behavior with control patients (i.e. contamination) • It seems like effective sample size should be greater than the number of physicians randomized

  28. Provider-Level Strategies • Randomizing patients assumes balance on both measured and unmeasured variables • Analytically, assumes that patients are independently assigned to groups • When unit of randomization is not the unit of analysis: • Patients are not independent within physicians • Physicians not independent within setting (e.g., team, hospital) • Complicates sample size estimates

  29. Provider-Level Strategies:Summary • Often, interventions to improve outcomes target providers, but analyze patients • Reasonable, and perhaps only plausible strategy • Must provide reviewers with clear justification (i.e., minimizes threats to internal validity that would otherwise result) • Involve biostatisticians early, as the analyses (including sample size calculations) are complex

  30. Design Considerations: Biomedical vs. HSR Trials ConsiderationBiomedicalHSR • Patients • Eligibility Narrow Broad • Randomization Patient Patient, Physician, Clinic, Hospital • Intervention: - Components Single Multiple - Type Drug, device, Structure of care procedure - Uniformity High Low

  31. Design Considerations: Biomedical vs. HSR Trials ConsiderationBiomedicalHSR • Intervention • Control group Standard Usual Care • Assess compliance Easy-Moderate Difficult • Data Collection • Outcomes Events, test result Patient-centered • Process of care Easy-moderate Difficult • Cost Easy-moderate Difficult • Blinding Possible Not possible

  32. Overview to Today’s Seminar • Overview of randomized trials • Threats to internal validity • Targets of interventions • Analytical issues

  33. Analytical Issues • Between- versus within-group comparisons • Primary analysis is between-groups • Controlling for baseline differences • Randomization does not achieve balance on key factors

  34. Analytical Issues • Intention to Treat: Subjects analyzed as part of original group, regardless of compliance, dropout, or crossover • Question answered: What is benefit of treatment/intervention as given? • Per protocol: Focus on subjects who were compliant with protocol • Question answered: What is benefit for people who are compliant?

  35. Analytical Issues • Sub-group analyses: Focus on subjects with certain characteristics • Question answered: Does the treatment work for certain types of patients • Disaggregating complex interventions: Can we identify the effective component? • Unit of assignment versus unit of analysis

  36. Overview to Today’s Seminar • Overview of randomized trials • Threats to internal validity • Targets of interventions • Multi-site trials in health services research • Analytical issues • Practical issues and advice

  37. Practical Issues and Advice • What help do I need? • Am I ready to conduct a clinical trial? • Is the intervention worth evaluating? • Is the project feasible? • What outcomes can I reasonably expect to change?

  38. What Help Do I Need? • Colleagues with content and methodological expertise and/or other specialized skills • Biostatistician

  39. Biostatisticians Are Your Friends • Study design • Sample size • Measuring outcomes • Data management • Statistical analysis

  40. Am I Ready to Conduct a Clinical Trial? • Equipoise • Advance previous study • Good idea, flawed study • Application to another patient population • Application to another clinical venue • Relevance • Policy makers • Clinicians • Health care organizations

  41. Is the Intervention Worth Evaluating? • Is the study ethical? • Is the intervention feasible and practical? • If effective, will intervention be accepted and useful? • Health care organization • Physicians • Staff • Patients

  42. Is the Project Feasible?Pilot Study as Dress Rehearsal • Are there enough subjects? • Do I have a practical strategy to identify, enroll and retain subjects in the study? • Do I have time to complete the project? • Do I have resources to complete the study? • Can I measure the critical variables? • Are data collection forms reasonable? • Do I have buy-in from the organization and personnel (physicians, nurses, clerks, etc.)?

  43. What Outcomes Can I Reasonably Expect to Change? • What should my outcomes be? • Mortality and/or morbidity • Clinical parameters • Health-related quality of life • Health services utilization/cost

  44. Final Advice • Specify the hypotheses • Write early, write often: • methods • dummy tables • Monitor what is happening • Enrollment • Retention • Delivery of the intervention • Review CONSORT statement on reporting randomized trials

More Related