Monitoring and Evaluation: Child Health Programs

1. Monitoring and Evaluation:Child Health Programs

2. Session Objectives By the end of this session, participants will be able to: • Identify interventions and approaches to improve child health • Describe M&E challenges of specific child health program areas • Design an M&E framework for an identified intervention • Choose core output and outcome indicators for a specified intervention & recognize their strengths & limitations

3. Current Situation • Each year nearly 11 million children die before the age of five, 30,000 every day,largely from preventable causes. • 33% of all child deaths occur in the first month of life. • 41% of these deaths occur in Africa, which has only 10 % of the world’s under-five population. • 50% of these deaths occur in only six countries; 90% of these deaths occur in 42 of 192 countries.

5. Global Targets • Millennium Development Goal (2002): • To ensure a two-thirds reduction in under-five mortality by 2015 from the base year 1990 • Related targets • Target 2: Halve between 1990 and 2015, the proportion of people who suffer from hunger • Target 6: Reduce by three-quarters, between 1990 and 2015, the maternal mortality ratio • Target 7: Have halted by 2015 and begun to reverse the spread of HIV/AIDS • Target 8: Have halted by 2015 and begun to reverse the incidence of malaria and other major diseases • Target 10: Halve, by 2015, the proportion of people without sustainable access to safe drinking water and basic sanitation

6. Child Health Programs • Immunization • Control of diarrheal diseases & acute respiratory infections (ARI) • Prevention of HIV in infants and young children • Roll Back Malaria (RBM) • Integrated Management of Childhood Illness (IMCI) • Newborn health • Nutrition

7. Newborn Intervention Packages • Pre-pregnancy health • Care during pregnancy • Care during delivery • Postpartum care of the mother • Newborn care • Essential newborn care • Extra newborn care • Emergency newborn care

8. Improving family and community practices Improving health workers skills Improving Health System IMCI Strategy: Components

9. Exclusive breastfeeding Appropriate complementary feeding Micronutrient intake Providing stimulating environment Immunization Handwashing and hygiene Insecticide-treated nets HIV prevention and care Key Family & Community Practices I Growth Promotion and Development Disease Prevention

10. Increased fluids & continued feeding Appropriate home treatment Prevention/management of injuries/accidents Prevention of child abuse & neglect Male involvement in care provision Recognition of dangers signs Appropriate providers Treatment compliance Accepting referral Follow-up Adequate antenatal and delivery care Key Family & Community Practices II Appropriate Home Care Care Seeking

11. M&E Challenges

12. M&E Challenges: Immunization • Age groups differ among data sources Routine data • Estimation of denominators • Accuracy, completeness & timeliness of reporting Survey data • Card availability & reliability of recall • Sampling errors & sub-national estimates • Survey timing in relation to campaigns • Use of data for timely program action

13. Class Activity: Funny Numbers for Immunization Coverage • Example 1: For 6 months in a row, one health center recorded coverage over 100 percent. • Example 2: Last year, 51 out of 100 districts in the country reported higher figures for DTP3 than DTP1 coverage. • Example 3: As of May, most health facilities in the country were reporting 30% to 40% measles coverage, which is approximately what one would expect. One health center, however, reported 100% measles coverage.

14. M&E Challenges: Diarrhea, ARI, Fever • Seasonality of disease • Selection bias in health facility data • Difficulty of standardizing & comparing across different malaria transmission settings • Changing definitions (ARI) • Nonstandard indicators & survey questions on water and sanitation • Validation

15. Class Activity: Management of Childhood Diarrhea

16. M&E Challenges: PMTCT (I) • Wide range of interventions • Degree of integration • Non-standard output index • Replacement feeding M&E is complex • Impact indicators difficult to obtain • Follow-up of mother-infant pairs

17. M&E Challenges: PMTCT (II) • Estimating the population at risk • Political and ideological factors • Rapid scale-up of interventions • Ethical issues

18. M&E Challenges: Newborn Interventions (I) • Definitions of births, deaths, and newborn period • Data quality • Omission of still birth & early neonatal death • Heaping on age at death • Sampling errors on survey estimates • Sensitivity of perinatal (late fetal plus early neonatal) mortality rate to changes in data quality

19. M&E Challenges: Newborn Interventions (II) • Measurement of neonatal morbidity • Lack of standardized output indicators • Facility-level and outreach monitoring • Outcomes needed for both mother and baby • Attribution is difficult due to packaged services

20. Class Activity: Neonatal Health In Area A, there are 5000 live births, 100 neonatal deaths, and 20 of the deaths are due to neonatal asphyxia. In Area B, there also 100 neonatal deaths, and five deaths are due to neonatal asphyxia. In Area B, there are 2000 live births. What conclusions can you draw from these data?

21. M&E Challenges: Child Nutrition • Complexity of child nutrition programs • Accurate measurement of child’s age • Need for large sample sizes to detect change in breastfeeding • More complex survey questions • Interviewer or respondent error

22. M&E Challenges: Facility IMCI • Complexity of indicator measurement • Quality of health facility data • Observation bias • Courtesy bias • Selection bias

23. M&E Challenges: Disease Surveillance • Use of non-standard case definitions • Accuracy, completeness and timeliness of reporting • Representation • Errors in descriptive information about reported case • Lack of laboratory support for outbreak confirmation or patient management • Infrastructure and communication constraints

24. Class Activity: Disease Surveillance in Mirriah District • What might account for the increase in the number of new cases observed during the two most recent years? • How might you explain the discrepancy between the hospital cases and reported cases? • What is the effect of including the children without fever status recorded on the chart in the case definition? Adapted from CDC Case Study No. 891-903

25. M&E Challenges: Mortality • Reliability of various data sources • Measuring short-term changes • Data quality • Inconsistent definitions of newborn, infant and child death • Misclassification of cause of death • Demonstrating mortality impact

26. Class Activity: Mortality During the year 1997, data from birth registers showed that 8776 live births were recorded in the municipality of Mamfi, out of which almost 7744 were children of mothers residing in the area. One hundred and thirty-nine (139) fetal deaths were recorded also from mothers living in the area. Two hundred and seven (207) deaths of babies under one year of age were registered to residents in the area of Mamfi, with the age distribution shown on your handout. (1) Calculate the infant mortality rate (IMR) (2) Calculate the early neonatal mortality rate (ENMR) An investigation conducted in hospitals and maternity wards revealed that approximately 65% of births (alive and dead) took place in hospitals. Every one of such events was scrutinized, consulting the mothers’ medical charts and notes taken by the physician and nurses inside the labor room. Findings showed that thirty live births have been mistakenly declared as fetal deaths and were recorded as such. Twenty-two fetal deaths were erroneously declared as live births. (3) Calculate the accurate IMR & ENMR (4) How were the rates affected by errors in the definition of live births and fetal deaths? (5) To what extent are the corrected rates representative of the municipality of Mamfi?

27. ILLUSTRATIVE EXAMPLE IMCI

29. Logic Model for Improved Health Worker Skills • FUNCTIONAL OUTPUTS • No. of health workers trained in IMCI • No. of district supervisors trained in IMCI PROCESS Train first-level health workers & district supervisors in IMCI • OUTCOMES • Improved caretaker knowledge & practices • Early case management • Appropriate care seeking • Compliance with treatment • IMPACT • Reduced infant & child mortality • Improved infant & child health & nutrition • SERVICE OUTPUTS • Improved HW skills • Improved HW competence • Reduced missed opportunities for vacc. & treatment of childhood illness • Increased client satisfaction

30. WHO Priority Indicators of Health Worker Skills (I) • Assessment • Child checked for three danger signs • Child checked for the presence of cough, diarrhea, and fever • Child’s weight checked against a growth chart • Child’s vaccination status checked • Index of integrated assessment • Child under two years of age assessed for feeding practices

31. WHO Priority Indicators of Health Worker Skills (II) • Correct treatment and counseling • Child needing oral antibiotic and/or antimalarial is prescribed the drug(s) correctly • Child not needing antibiotic leaves the facility without antibiotic • Caretaker of sick child is advised to give extra fluids and continued feeding • Child needing vaccinations leaves facility with all needed vaccinations • Caretaker of child who is prescribed ORS and/or oral antibiotic and/or oral antimalarial know how to give the treatment • Correct management of severely ill children • Child needing referral is referred

32. Sources of Data: M&E of IMCI Training (I) • Health facility surveys (e.g. Service Provision Assessment • Inventory of resources and support services • Provider interview • Observation of services provided • Client exit interview

33. Sources of Data: M&E of IMCI Training (II) • Supervisory checklists • Review of national, district, and facility records • Demographic surveillance • Mortality survey • Vital registration (if 90+% complete) • Qualitative studies

34. Case Study Despite substantial efforts to improve infant and child health and nutrition in Lakkha District in the past decade, about one out of five babies born in a given year die before they reach their fifth birthday, many during the first year of life. In 2002, the Government recognized that improving the quality of care for sick children at the health facility alone would have a limited impact on reducing child mortality. Lakkha District was chosen, therefore, as a pilot site for the development and implementation of a household and community-based approach to promote key household practices for child survival, growth and development . Please refer to the handout for further instructions.

35. References • Arimond, Mary and Marie T. Ruel. 2003. Generating indicators of Appropriate Feeding of Children 6 through 23 Months from the KPC 2000+. Washington, D.C.: FANTA. • Bos, E. and A. Batson. 2000. Using Immunization Coverage Rates for Monitoring Health Sector Performance: Measurement and Interpretation Issues. HNP Discussion Paper. Washington, D.C.: World Bank. • Gage, Anastasia J., Disha Ali, and Chiho Suzuki. (Forthcoming). A Guide for Monitoring and Evaluating Child Health Programs. MEASURE Evaluation. Chapel Hill, NC: Carolina Population Center, University of North Carolina. • United Nations. 1983. Manual X: Indirect Techniques for Demographic Estimation. New York: United Nations. • WHO. 2001. Indicators for IMCI at First-level Facilities and Households. Geneva: WHO. • WHO. 2005. National Guide for Monitoring and Evaluating Programmes for the Prevention of HIV in Infants and Young Children. Geneva: WHO. • WHO and UNICEF. 2005. World Malaria Report 2005. Geneva: WHO.