Dena Ringold , Senior Economist, The World Bank APPAM - Moscow June 28, 2011

1. Making Services Work:Indicators, Assessments and Benchmarking of Governance in Health and Education Dena Ringold, Senior Economist, The World Bank APPAM - Moscow June 28, 2011

2. Outline • What is governance and why should it be measured? • A framework for measuring governance policies and performance; • Measuring dimensions of service delivery; • Examples and recent innovations.

3. Improved governance is needed to make services work • Governance: • Incentives to providers • Accountability for results Quality of social service delivery HD outcomes Demand for good governance

4. Why Measure Governance? • The Shock Factor: Measurement allows for benchmarking; • Both inter- and intra-national; • Does it work? Measurement allows for impact evaluation; • Measuring along the results chain to assess what works; • Look inside the “black box” of service delivery: • Intermediate service delivery indicators assess the “missing middle”; • Indicators are needed for results-based projects .

5. Governance Policies and Performance Governance has two measurable dimensions: • Governance policies, or “rules of the game,” that influence provider behavior; • E.g. human resource policies that establish hiring, firing, and rewarding of teachers and doctors; • Governance performance, or the extent to which governance policies work in practice; • E.g do teachers and doctors show up for work? • Governance policies and performance influence the quality of service delivery; • E.g quality at the point of service; quality measurement is elusive but there is progress.

6. Governance • Policies • Outcomes A Measurement Framework • Service Delivery • Completion rates • Learning outcomes • Morbidity • Mortality • HR policies • Budget and financial management • Information • Procurement rules • Rules for provider entry • Governance performance • Quality

7. Measuring Policies and Performance

8. Human Resources: SABER-Teachers • What is it? • Assessment tool for documenting 10 dimensions of teacher policies. • How does it work? • Expert survey based on an extensive questionnaire. • Currently applied in 30 countries including OECD; also at the state-level in India. • What about performance? • Absenteeism surveys can measure provider motivation, retention, etc.

9. Budget: ABI in the Philippines • What is it? • Expert survey of organizational performance piloted in the Department of Health, based on PEFA. • What does it cover? • 6 areas, including budget preparation; execution and reporting; organizational management and accountability; • Includes policy and performance indicators. • What are other instruments? • Public Expenditure Tracking Surveys; Public Expenditure Reviews (can look at the whole sector).

10. Information: Global Integrity Pilot • What is it? • An experiment to develop indicators of access to information in health and education, including policies and performance. • What does it cover? • 4 broad areas of access to information. • What has been learned so far? • Pilot in Macedonia suggests that it is possible to collect this type of information, now expanding to Ukraine and Kenya; • Big gaps between legislation and implementation.

11. Critical Inputs: WHO pharmaceuticals • What is it? • Expert survey for assessing governance policies for transparent pharmaceutical supply and delivery. • What does it cover? • 8 areas, including registration, licensing, inspections, controls of trials, procurement and distribution; • Applied in 25 developing countries. • What about performance? • Tests for counterfeit drugs (USAID quality database); • Stock-out rates (WHO/HAI database).

12. Provider Entry: Africa Health • What is it? • Recently published study of the private sector in health in Africa by the IFC. • What does it cover? • Survey includes the inclusion of the private sector in the national health information system and regulations governing the quality of private provision.

13. Looking forward • There is progress and space for applying tools more systematically, and developing new approaches; • Key questions are who should collect data and how often? • Technological innovations in data collection are promising (SMS for reporting stock-outs; Indaba software)

14. THANK YOU www.worldbank.org/hdchiefeconomist

15. Absence rates among teachers and health workers Note: Surveys were all fielded in 2002 or 2003. Sources: Chaudhury et al (2006) except for PNG, World Bank (2004) and Zambia, Das et al (2005).

16. Share of school grants and funds for health facilities that actually reached the intended recipient/facility

17. 10 Teacher Policy Areas Requirements for entering and remaining in the teaching profession Initial teacher preparation Recruitment and employment Teacher workloads and autonomy Professional development Compensation: salary and non-salary benefits Retirement rules and benefits Monitoring and evaluation of teacher quality Teacher representation and voice School leadership

18. 4 ‘Buckets’: Access to Information • Existence and usability of information • Is the Ministry of Education’s overall budget available to citizens? • Redress mechanisms to enforce accountability • Do formal processes exist to receive citizen complaints? • Availability of Fiscal/Budget information • Does information exist on whether school/medical facilities receive funds budgeted from state/local governments? • Influence of information availability on citizen participation in local decision making • Are citizens aware of basic rights to participate in formal community groups/forums?