Identifying Peer Areas for Community Health Collaboration and Data Smoothing

1. Identifying Peer Areas for Community Health Collaboration and Data Smoothing Brian Paoli Utah Department of Health 6/6/2007 Utah Department of Health

2. Acknowledgments • Dr. Lois Haggard, UDOH • Dr. David Mason, Univ. of Utah • Mohammed Chaara, Univ. of Utah • Michael Friedrichs, UDOH • Kathryn Marti, UDOH Utah Department of Health

3. Outline • Background: • Why Peer Areas? • Data Smoothing • Previous Peer Area Attempts • Methodology and Procedures • Utah’s 61 Small Areas • Demographic Similarity • Producing Smoothed Estimates Utah Department of Health

4. Why Peer Areas? • Community Collaboration • Identify areas that are similar for purposes of comparison • Collaborate on strategies, interventions • Data Smoothing • “Borrow strength” from geographic areas that are similar. • Especially useful when multi-year trend data are not available Utah Department of Health

5. Data Smoothing • Why smooth? • We calculate measures, such as rates of death and disease, to assess the underlying disease risk in a population. • Measures from small populations are inherently erratic – subject to sampling variation. • Rare events such as infant mortality can vary widely from year to year. Utah Department of Health

6. Utah Department of Health

7. Data Smoothing • Most methods smooth data over time, requiring data from multiple years • Pool multiple years (e.g., 3- or 5-year averages) • Moving average • Combine areas to increase the number of cases Utah Department of Health

8. Goal • Produce reliable (smooth, not erratic) and timely estimates. • Make appropriate inferences about the underlying disease risk in each community • Method must be simple to apply –and easy to implement Utah Department of Health

9. 29 UT Counties Utah Department of Health

10. Utah Department of Health

11. Utah Small Areas • 61 small areas were defined using both ZIP code and County • Each area is either a ZIP code area, • two or more contiguous ZIP codes, or • a combination of ZIP code and county information. Utah Department of Health

12. 61 UT Small Areas Utah Department of Health

13. Utah Department of Health

14. Previous Attempts • Geographic adjacency • rooks’s adjacency • queen’s adjacency • geographic centroid • population centroid Utah Department of Health

15. Cluster analysis odd-sized groups odd-ball areas groups mutually exclusive (okay, but not necessary) Previous Attempts Utah Department of Health

16. Develop Methodology to: • Identify Peer Areas • Create “Demographic Distance” matrix • Smooth Data • Median? Pooled? Weighted? • Measure Our Success • Reliability of results • Appropriateness of making inference to index area from smoothed rates Utah Department of Health

17. 1. Identify Peer Areas • Demographic Characteristics • Use available demographic information from the U.S. Bureau of the Census • Use demographic variables that are associated with population health • Select a small number of these demographic variables • Produce a methodology others can replicate Utah Department of Health

18. 1. Identify Peer Areas • Factor analysis to reduce dimensionality. • Foreign born/Hispanic • Education • Income/Poverty • Employment • Age • Urban and Rural Utah Department of Health

19. 1. Identify Peer Areas • Selected 5 variables based on factor analysis and correlation with health outcomes (e.g., infant mortality, heart disease, etc.) • % Hispanic • % age 25+ with Bachelor’s degree • % children in poverty • % owner-occupied housing • % age 65+ Utah Department of Health

20. 1. Identify Peer Areas • Create distance matrix The distance d(x,y) between two areas with n dimensional observations x=[x1,x2,…,xn]’ and y=[y1,y2,…,yn]’ is: d(x,y)= ([x-y]’S-1[x-y])1/2 The matrix S contains the variances and covariances of the n variables. Utah Department of Health

21. Identify Peer Areas • Demographic distance Utah Department of Health

22. 1. Identify Peer Areas • Which are the Peer Areas for purposes of collaboration? • 3 (or some #) areas with smallest distances? • All areas within a certain distance? Utah Department of Health

23. 1. Identify Peer Areas • Which are the Peer Areas for purposes of data smoothing? • Same areas as for collaboration? • Need to think about the smoothing algorithm. Utah Department of Health

24. 2. Smooth the Data • Options • Weighted Median rate using a group of five areas • Pool a selected number of areas together and treat them as a single area (crude rate for the combined areas) • Pool all areas together and weight them by a function of their distances to the index area (closer areas -> more weight) Utah Department of Health

25. Distant Neighbors Close Neighbors Utah Department of Health

26. Areas that are distant contribute little to the smoothed rate Areas that are close contribute more to the smoothed rate Utah Department of Health

27. Utah Department of Health

28. Utah Department of Health

29. Utah Department of Health

30. Utah Department of Health

31. 3. Measure Our Success • Reliability • Did the data get smoother? • Intraclass Correlation Coefficient (ICC) • Ratio of the amount of variance between areas to the sum of the variance within and between areas • (MSbetween – MSwithin )/( MSbetween+(k-1)Mswithin ) • Range from 0 to 1 • 1 = perfectly smooth and level, only variance in the data is from one area to the next Utah Department of Health

32. 3. Measure Our Success • Appropriateness of Inference • Is it appropriate to infer that the smoothed rate represents the true underlying disease risk in the community? (Overall, are the smoothed scores in the ballpark?) • Sum of Squared Differences (SS) from smoothed data to original data. • Smoothed estimate should be close to the index area’s crude rate Utah Department of Health

33. 3. Measure Our Success • ICC – Want high scores, close to 1 • SS – Want low scores, given high ICC • HOW DID THE SMOOTHED RATES PERFORM? Utah Department of Health

34. Smoothed ICC=.901 Crude ICC=.835 Utah Department of Health

35. Smoothed ICC=.901 Crude ICC=.835 Utah Department of Health

36. Smoothed ICC=.901 Crude ICC=.835 Utah Department of Health

37. Smoothed ICC=.901 Crude ICC=.835 Utah Department of Health

38. Summary • A small number of demographic variables were identified • Capture the demographic variability • Related to health outcomes • Peer Areas were identified • Groupings seem intuitive Utah Department of Health

39. Summary • Smoothing algorithm was identified • Had characteristics we liked • Index area gets highest weight • Peer areas get high weights • Dissimilar areas weight=0 Utah Department of Health

40. Summary • Smoothed rates performed generally well • They were smooth (ICC ~ 1.0) • They represented the underlying risk in the index area (SS relatively small) Utah Department of Health

41. Summary • Easy to replicate? • Excel spreadsheet • You: • Enter your demographic variables • Enter health outcomes for the same areas • Change smoothing parameters (if desired) • Excel: • Calculates distance matrix • Generates smoothed rates • Generates performance measures Utah Department of Health

42. Challenges/Limitations • Demographic characteristics change, distance scores will need to be updated (decennial census years?) • How much smoothing to use is a subjective decision. • Smoothing may not seem credible to members of community • Peer Groups are not symmetric Utah Department of Health

43. Excel Spreadsheet The spreadsheet is free and the files can be downloaded from the IBIS website. Go to http://ibis.health.utah.gov Look for “Peer Area Analysis Tool” under the “News and System Enhancements” heading. Utah Department of Health

44. Contact Information: • Brian Paoli Office of Public Health Assessment Utah Department of Health 288 North 1460 West P.O. Box 14201 Salt Lake City, Utah 84114-2101 email: bpaoli@utah.gov Utah Department of Health