Identifying, Categorizing, and Evaluating Healthcare Efficiency Measures

1. Identifying, Categorizing, and Evaluating Healthcare Efficiency Measures Elizabeth A. McGlynn, Ph.D. Associate Director, RAND Health September 28, 2007

2. The Work Was Done by A Multidisciplinary Team • Peter Hussey • John Romley • Han de Vries • Margaret Wang • Paul Shekelle • Dana Goldman • Martha Timmer • Susan Chen • Jason Carter • Carlo Tringale

3. Overview of Talk • Highlight motivation for current work • Present RAND’s typology • Review existing measures • Examples • Concluding thoughts

4. Many Fortune 50 Companies Are Demanding Cost and Quality Metrics on Physicians Efficient Effective

5. Purchasers Are Using Efficiency Metrics In Several Ways • Public reporting – information to help consumers make more cost-conscious decisions • Pay-for-performance – financial rewards to providers with better performance • Tiering – differential co-payments to encourage patients to choose higher performing providers • Selective contracting – contracts limited to providers who perform at a certain level

6. But… • Little is known about what various stakeholders mean by “efficiency” • Considerable lack of common language, conceptual clarity • Link between content of metrics and proposed applications not always thought through • Little is known about the consequences (intended and unintended) of applying available metrics at different levels in the system • Can measures developed for one level (health plan) be applied at a different level (physician)?

7. Overview of Talk • Highlight motivation for current work • Present RAND’s typology • Review existing measures • Examples • Concluding thoughts

8. Definition of Efficiency Measure The relationship between a specific product of the health care system (output) and the resources used to create that product (inputs) • Maximize output produced for a given input • Minimize inputs used for a given output

9. Efficiency Measures Typology Overview Typology is organized in three tiers Who is asking what about whom, and why? 1. Perspective 2. Output(s) What is being produced? What resources are used to produce the output? 3. Inputs

10. Perspective • We identify several potential points of view: • Health care “firms” • Physicians • Hospitals • Nursing homes • “Agents” • Health plans • Employers • Individuals • Society

11. Context Matters in Efficiency Measurement

12. Use of Information Varies With Perspective • Health care firms can change the way the outputs are produced • Amount and/or mix of inputs • Service amenities associated with a product • Agents can principally change how much is paid, the conditions under which the product is purchased, or how the product is bundled • Individuals can principally change what they buy and from whom

13. Output: What’s Being Produced? • We define two major categories of outputs: • Health services • Health outcomes • Being explicit about the output is critical (and often not done) • Producers tend to define outputs • Financial flows (how the product is purchased) influence definitions, for example: • Hospital day vs. a discharge • Quality adjusted life year vs. a covered life

14. Examples of Outputs by Type

15. Inputs • We identify two main ways of measuring inputs: • Physical • Financial • These map to the economic definitions of technical (physical) and productive (financial) efficiency

16. Technical Efficiency Technical and Productive Efficiency MeasuresPoint to Different Root Causes of (In)Efficiency Productive Efficiency Inputs are put to good use Inputs are put to good use + Best mix of inputs chosen + Lowest prices are paid

17. Social Efficiency • Social efficiency is achieved when no member of society can be made better off without making another member worse off • Giving more resources to one person implies that those resources have been taken away from someone else • Appeal of “waste” is the notion that those resources do not benefit anyone currently

18. RAND’s Efficiency Typology Society Health Care Firms Perspective Health Plans Providers Purchasers Individuals Health Outcomes Health Services Output Input Physical Financial

19. Overview of Talk • Highlight motivation for current work • Present RAND’s typology • Review existing measures • Examples • Concluding thoughts

20. Identifying Existing Efficiency Measures • Peer-reviewed literature • Medline and EconLit search: 1990-2005 • Titles, abstracts, and articles reviewed by 2 independent reviewers; consensus resolution • Excluded non-U.S. studies • “Gray” literature review • Purposive, reputational sampling • 8 vendors identified & reviewed

21. Overview of Article Flow Titles (n=4,324)

22. Measures in Literature Dominated by Hospital Perspective & Service Outputs

23. 40 Different Physical Inputs Identified in Peer Reviewed Literature • Physician labor – number of physicians (usually FTEs) or hours worked • Nursing labor – number of nurses (usually FTEs) or hours worked • Administrative, technical, or other labor categories – number of personnel (usually FTEs) or hours worked • Beds – the most common indicator of capital stock • Depreciation of assets - a measure of capital, calculated in various ways Financial inputs appear almost as often as physical inputs

24. No measures using health services as outputs explicitly incorporated a measurement of qualityMethods for incorporating or accounting for quality are not well developed in peer reviewed literature

25. Different Worlds of Efficiency Measures • There is an almost total separation between the published studies of health care efficiency and the use of efficiency measures by providers, payers, and purchasers • Academic measures focus on multiple input/output models • Vendor measures focus on specifying outputs • Purchasers and health plans are generally using measures developed by vendors • Little independent testing of reliability and validity has been reported on these metrics

26. Vendor-Developed Measures • Episode-based: ETGs, MEGs, CCGs • Claims aggregated into clinically-defined episodes and attributed to different entities • Measure is cost per episode (productive efficiency) • Also can look at resource use per episode (technical efficiency)

27. Vendor-Developed Measures • Population-based: ACGs, DxCGs, CRGs, RRU, PPMS • Patient populations weighted by morbidity burden • Measure is cost per risk-adjusted patient per year (productive efficiency) • Also can look at resource use (technical efficiency)

28. Overview of Talk • Highlight motivation for current work • Present RAND’s typology • Review existing measures • Examples • Concluding thoughts

29. Some Examples of Efficiency Measures

30. Measures We Wouldn’t Classify as Efficiency

31. We Asked What the Evidence Was Regarding Whether the Measure Was… • Important • Scientifically sound • Feasible • Actionable

32. Summary of Findings

33. Overview of Talk • Highlight motivation for current work • Present RAND’s typology • Review existing measures • Examples • Concluding thoughts

34. Conclusion • Disconnect between academic world and vendors on efficiency measurement offers opportunity • Although steps have been taken to achieve consensus among stakeholders, more work needs to be done • Typology can assist in structuring more explicit discussions about key issues • Scientific soundness of both academic and vendor-developed measures is largely unknown • Ability of potential users to act on the results of these measures is not well understood

35. Some Priorities for the Future • Be explicit about the perspective, output, and inputs used in measure: • What’s the purpose, approach? • Develop measures to fill important gaps • When is it most important to incorporate quality measures? • Measures that use health outcomes as outputs • Measures for units of analysis other than hospital • Evaluate whether efficiency measures are scientifically sound, feasible, actionable