Leisa J. Stanley, MS Healthy Start Coalition of Hillsborough County, Inc. David Darr, MD, MPH

1. A Graphical Interactive Tool for Evidenced Based Public Health Program Evaluation by Programmatic Personnel with Minimal Computer Skills Leisa J. Stanley, MS Healthy Start Coalition of Hillsborough County, Inc. David Darr, MD, MPH Lawton & Rhea Chiles Center, College of Public Health, University of South Florida 12th Annual MCH Epidemiology Conference Atlanta, Georgia December 6-8, 2006

2. Project Background • Healthy Start Coalitions – HSC of Hillsborough County • Established by legislative act, Chapter 383, FS., 1991 • Responsibilities for MCH System • Chiles Center for Healthy Mothers & Babies • Part of Department of Community & Family Health, College of Public Health, USF • The Chiles Center brings together experts in public health, health care, program design and evaluation, and maternal and child health from Florida universities and state, national and international programs. • Healthy Start Program • Countywide Program – 16,000 births in 2005 • In 2005-2006, served 6,000 pregnant women and 7,000 infants • Provides home visitation and care coordination services • 12 state and 11 local Performance Measures • 6 state Outcome Measures

3. Problem Statement • Question:Can data be prepared, so it is easily and reliably used by non-programmers for programmatic evaluation with software found on virtually all computers? • Issues: • Data Access: vital statistics, service data • Data Timeliness • Data Volume • Staff Training • System-Wide Issues • Increasing number of pregnant women without insurance • Rising Infant Mortality Rate • Racial Disparity

4. Purpose • Creation of database with historical data: • 1992 – forward • Vital Statistics Data • Healthy Start Services Data • Addition of provisional vital statistics data • Timeliness • Two Pathways • Research Database • Program Evaluation Database

5. Creation of a Database and Interactive Tool

6. Data Files • Florida Department of Health Office of Vital Statistics • Natality Record • Fetal Death Records • Infant Death Records • Provisional Vital Statistics Records • Florida Department of Health Office of Planning & Evaluation • Healthy Start Prenatal Screening Data • Healthy Start Infant Screening Data • Healthy Start Services Data • Hillsborough County Health Department • Local Healthy Start Prenatal Risk Screening Data • Healthy Start services data with supplemental local service codes

8. Data Manipulations • Converted all month/day/year variables into true SAS dates • Caculated needed variables • Kotelchuck Index • Interpregnancy Intervals • Created as many categorical variables as possible, with as many variations as possible to reduce the potential need for future reprogramming • Race/Ethnicity • Age of Mother • Birth Weight • Gestation

9. Data Manipulations • Created a variable to distinguish between a delivery (maternal data) and a birth (infant data) • Defined and created variables for service areas or targeted populations • Central Hillsborough Healthy Start • Intensive Teen Unit • Defined geographic regions • Created value-added variables (Yes/No) • Low Birth Weight • Preterm Birth • Infant Death • Early Entry into Prenatal Care

10. Data Manipulations • Created Pivot Charts in Excel • Data dictionary is available • Programming already done • Limitations: 256 variables; 65,563 records • Created macros in SAS to calculate from data and import back into Excel • Calculated Relative Risks • Agree on reference groups • Calculated Attributable Risk • Calculated Population Attributable Risk

12. Sample of a Pivot Table

14. Uses by Programmatic Personnel • Used by Federal Healthy Start Project • Allows complex questions to be answered easily • No programming experience is required • Minimal training is required • Questions • Racial Disparity Gap • Repeat Pregnancies for Teens • Entry into Prenatal Care • Prepare reports for funders and key stakeholders

15. Available 1998-2003 de-identified linked vital statistics data 1992-2004 de-identified unlinked vital statistics data 2001-2003 service data (incomplete) Needed Identified vital statistics data to link 1992-present complete services data Local service data codes Provisional vital statistics data Data Available versus Data Needed

16. Data Procurement • Apply for Institutional Review Board Approval • University of South Florida • Florida Department of Health • Requires execution of proprietary data usage agreement(s) with each entity owning the data desired for linkages • Requesting identified data and development of a research database • When we did not need to apply? • For programmatic evaluation

17. Provisional Vital Statistics Data • Specific program evaluation questions: • Issues of program delivery • Issues of access to care • Emerging areas in county • Do not use for: • Outcome evaluation • Be cautious with low numbers and impact on rates: Wide month to month variations are possible and small numbers preclude use in outcomes analyses

18. Lessons Learned • Initiate process as early as possible • Anticipate delays, especially in use of provisional vital statistics data • If identifiers are to be retained, yearly data usage agreements may be necessary • Understand limitations of provisional vital statistics data

19. Lessons Learned • Data Security • Assure original data owner that only de-identified data will be used by programmatic personnel for their evaluations • Assure original data owner that there will be no follow-back to any client • Assure original data owner that all results will be reported in aggregate only • Be very mindful of HIPAA rules, especially when reporting at zip code or census tract levels for rare conditions or events • Do not allow original data owner to have veto power over any reporting of data • Know and understand the data you are given

20. Leisa J. Stanley, MS lstanley@hstart.org David Darr, MD, MPH ddarr@health.usf.edu Contact Information