Measuring Public Health Agency Preparedness

1. Measuring Public Health Agency Preparedness Mary V. Davis, DrPH, MSPH December 20, 2006

2. What is the Role of Public Health Agencies in Disasters/Bioterrorism?

3. Feedback Poll Is it possible for me to know whether my health department is prepared? • Yes • No

4. Measuring Preparedness • What does it mean to be prepared? • How can preparedness be measured? • Definitions • Metrics • Instruments • What is the relationship between individual competency and agency preparedness?

5. Measurement 1.0 Evaluation of Public Health Response to Hurricanes Finds North Carolina Better Prepared for Public Health Emergencies January–June 2004

6. Purpose of NC Study Identify how BT funds improved public health capacity in North Carolina to prepare for and respond to bioterrorism events and other public health emergencies. Examine how improvements in public health capacity for a specific public health emergency can translate to all public health emergencies.

7. NC Context • Capitalize on “natural experiment” • 1999 Hurricane Floyd • 2002 Infusion of Federal funds to address bioterrorism • 2003 Hurricane Isabel • Strong partnership history between NC DPH and UNC SPH • No standard definition of “preparedness”

8. Evaluation Questions—Resources • Phases of Disaster Framework—Noji • Criteria for evaluating management of disasters—Quarentelli • Public Health Management of Disasters—Landesman

9. NC Hurricane Evaluation Study Handout Page 1 Evaluation Questions

10. Data Collection • Key informant interviews—Professional role involvement in Floyd and/or Isabel • Document review—state and federal protocols, documents, records

11. Results Floyd • Planning—no specific activities • Preparation • Communication with local health departments • Response • Command and Control • Surveillance and Assessment

12. Floyd and “The Box”

13. DPH Use of Federal Funds • Created Infrastructure • Office of Public Health Preparedness & Response • Public Health Command Center • Public Health Regional Surveillance Teams • Employed Staff • BT Coordinator • Office of Public Health Preparedness & Response staff • Public Health Regional Surveillance Team staff: 28 in 7 regions statewide • Public health epidemiologists: 12 statewide • Implemented Health Alert Network • Implemented NC Emergency Department Database

14. Actions (cont’d) • Enhanced Laboratory Capacity • Biosafety Level 3 and Level 2 • Purchased Equipment • 8 vehicles • Fixed vehicle-mounted and hand-held radios • Strengthened Communication • Bioterrorism Steering Committee, reps from multiple agencies • Provided Training • Emergency preparedness and core public health competencies • Exercises and drills

15. Resulted in Capacities Used in Isabel • Effective, centralized communication through Public Health Command Center • Between state agencies • With local public health departments • With community and academic partners • With citizens

16. Capacities (cont’d) • Command and Control Incident Management • Operational and functional plans • Systematic response • Decision making in real-time emergency conditions • Epidemiology and surveillance • Rapid needs assessment • Medical surveillance

17. Preparedness Logic Model Handout Page 2 NC Model

18. Preparedness Logic Model Handout Page 3

19. Preparedness Logic Model RAND Corporation. RAND Health. Learning from Experience, Page 9 • Goal: mitigate morbidity, mortality, stress • Objectives • Early identification • Early and effective response • Earliest return to normal function • Functional capabilities meet objectives • Capacity-building activities result in capabilities

20. Measurement 2.0 Mississippi Department of Health Hurricane Katrina After Action Report January–June 2006

21. Target Capabilities List • U.S. Department of Homeland Security Target Capabilities List version 1.1 • Defines 36 prevention, protection, response, and recovery capabilities (measures) • Capability measures: capacity (processes, infrastructure) • Performance measures (demonstrate task execution)

22. Why Target Capabilities? • Becoming benchmarks for state performance • Provide a framework for performance measures • Create a performance plan • Specific • Measurable • Obtainable

23. TCL Pilot Test • MS Department of Health Katrina After Action Review • Specific type of evaluation • Identifies strengths and weaknesses in response • Defines specific action plan for improvements • Examined response in 21 TCLs • Emergency Support Function 8—public health • General TCL’s—planning

24. Target Capabilities Examined in MDH Katrina AAR • Handout 4

25. NC Involvement • MS requested NC to construct a team to assess ESF 8 response • MDH planning with Region IV, specifically NC OEMS prior to Katrina • NC mobilized SMAT to Waveland, MS during Katrina response • NC personnel familiar with gulf communities and MDH structure

26. MDH Katrina AAR • Identify major points what worked/did not work well during MDH Katrina response Katrina in Emergency Support Function 8 • Data collection (all new instruments) • Responder Online Assessment • Responder Interviews • Community Assessment • Results used to: • Inform Performance Improvement Plan • Revise MS State Emergency Operations Plan

27. Online Survey Example

28. Online Survey and Interview Data • Feedback available for all TCLs • Least for Animal Health Emergency Support • Most for Planning, Mass Care • Substantive open-ended feedback from survey “no” responses • Interviewees provided substantive information on what worked and areas for improvement

29. AAR Recommendations • Overarching capability and specific TCL • Results from online survey, interviews, community assessment • Experience and knowledge of MDH and NC team • Federal guidelines

30. Limitations • First use of TCL Performance Measures as data collection instruments • Limited time to create and validate data collection instruments • Respondents lacked knowledge of performance measure • MDH identified participant samples • Challenge to digest the wealth of information from the results

31. Take Home Points • First use of Target Capability List as an AAR framework • First instance of one state government conducting an AAR for another state • NC team drew upon EM, OEMS, PHP&R, UNC SPH, and other expertise • Approach yields data in performance measure framework

32. Preparedness Measurement Next Steps • Modify Target Capability measurement instruments • Pilot in an After Action Review • Continuously Refine Measures • Use RAND Public Health Preparedness Logic Model as guide • Measures should be • specific about responder function • specific about activities and capabilities

33. Public Health Preparedness • Continuous process—constant quality improvement • Agencies need to test preparedness in real conditions • Exercises and drills • Outbreak investigations • Disaster, BT responses

34. Individual Competency and Agency Preparedness • Using RAND Public Health Preparedness Logic Model • Individual Competency • Conduct capacity-building activities • Demonstrate competency to perform functional capabilities • Agencies • Implement functional capabilities • Achieve preparedness objectives and goals

35. Resources • NC Hurricane Evaluation http://www2.sph.unc.edu/nccphp/hurricane_10_19_04.pdf Article in PH Reports in Jan/Feb 2007 issue • MDH AAR Report http://www.msdh.state.ms.us/msdhsite/_static/resources/1676.pdf • Contact mvdavis@email.unc.edu