Evaluation Capacity Building in South Asia: Experiences, Lessons Learned and Way Forward

1. Evaluation Capacity Building in South Asia: Experiences, Lessons Learned and Way Forward R.S. GoyalProfessor, Institute of Health Management Research, Jaipur, India(email: rsgoyal@iihmr.org)

2. Backdrop….. • Most South Asian countries now have monitoring and evaluationsystems in place, however, these are largely concerned with; theprogress of implementation, rather than assessing the social,economic and environmental impacts of projects. • Also, thereseem to be no systems that assess the impact of policy interventionsemerging from recent macro-level measures, such as; liberalization,privatization, and the preservation of women's rights.

3. Further, donor agencies have played a key role in planning, implementingand financing various socio-economic development programs andprojects in these countries. • In many cases, the outcomes of these interventionsdo not match the intended objectives. • It has also been observed thatdue to the lack of ongoing evaluation many governments failto learn, in time, the way a project is unfolding and the mannerin which it is generating benefits.

4. There are also many whosimply do not see the benefits of evaluation and, consider itto be a donor-driven activity of no management use. • Those donorswho do see evaluation as an important tool to improve investmentquality are now seeking to build evaluation as mandatory management style- result based management.

5. Success of these initiatives seems to have been constrained,among other things, by; • lack of a unified approach; • inadequateappreciation and analysis of governmental culture; • confusionabout concepts and methodologies; • lack of long-term commitment;and • lack of either interest or resources—or both—fromthe recipient governments. • Lack of professional evaluation capabilities to commission, manage, implement and use evaluation has also been sited as one of the major constrains.

6. Strengthening evaluation capacity in the South Asia region has therefore becomes critical to achieve results of development projects. • Institutionalization of evaluation training in academic institutions is identified as one of the most viable strategy. • With support from UNICEF ROSA, a group of select institutions in South Asia, embarked upon a path to offer professional evaluation training in a sustainable manner. • This paper discusses the process, outcomes, lessons learned and future directions of this initiative.

7. The Process….. • Assessment of demand for professional evaluation training. • A well published call for expression of interest sent by UNICEF ROSA indicated a widespread need of professional evaluators among donors and desire to participate in the capacity building process among leading institutions in the region. • The feasibility assessment led to identification of six institutions with essential facilities, expertise and commitment to participate in the capacity building program.

8. A key strategy of this initiative was to seek support from centers of excellence in evaluation training in the North and foster a North-South collaboration. • Among numerous Northern institutions showing interest; Carlton University (Canada) and London Metropolitan University (UK) were identified. • To promote ownership and cross fertilization of ideas among participating institutions, a consortium of institutions on evaluation training in South Asiawas formed. • To broaden the scope this initiative, the policy makers, commissioners of evaluation and evaluation professionals in South Asia were also brought in the loop.

9. It was agreed that training program would commence with an executive level course of one week duration. • The training curriculum was developed jointly by North and South institutions. • A training of trainers program was held in Dec. 07 at the author’s institute to test the training module and orient the faculty from South Asian institutions on delivering the course.

10. This program was attended by institutions from Bangladesh, India, Pakistan and Sri Lanka. • The feedback of this program was critically examined with reference to contents, training pedagogy and reading material provided. • The analysis shows that mode of delivery was good. However, contents need modification to suite the expectations of programs and professionals in the region.

11. The next step was setting up procedures by participating institutions to offer executive course on regular basis. • Several institutions managed this process efficiently. These include; BARC University, Dhaka, Institute of Health Management Research, Jaipur, Lahore University of Management Sciences, Lahore, Sri Jayawardanapura University, Colombo. Other institutions are in the process of making arrangements.

12. The participating institutions started offering evaluation trainings in their respective areas of specialization. • Author offered the first course on Evaluating Healthcare Interventions on Oct. 13-17, 08. It was attended by 18 participants from South Asian region. • The training focused on; evaluation design, ethics and standard of evaluation, result based management, types of evaluation and links to monitoring, developing terms of reference, process of carrying out evaluation and utilization of evaluation findings for program management.

13. The feedback of this training shows; • it met the expectations of participants to a large extent, • helped in enhancing the skills in planning evaluations, writing TOR, carrying out and utilizing evaluation findings, • further skill building was however, sought on research design and data analysis process. • In the light this feedback the training module was modified and the next executive level training was offered on March 2-6, 09.

14. This training attracted a very positive response and attended by 59 evaluation professionals from South Asia. • The feedback of second program was even more encouraging: • Over 95 percent participants observed that course was well structured and practical oriented. • Ninety five percent participants opinioned that course learning would help them in their immediate and future assignments. • Over 96 percent participants observed that course material was very relevant to their work. • Almost all participants wanted to extend the training period.

15. Lessons Learned….. • The regional approach adopted in this initiative has been a strength in its implementation and in moving it further. • It enabled adoption of common applicable and culturally acceptable curriculum and training pedagogy. • Coming together of institutions as a consortium for evaluation training has provided a region-wide forum and an authenticated body. • It facilitated the program to move forward as a body beyond the tenure of assistance of original donor agency and developing a plan for diploma program.

16. The collaboration between North and South Asian institutions played a vital role in the success of this initiative. • Theformer brought the state of art knowledge and approaches which later adapted to South Asian learning context and needs. • In developing diploma program in evaluation this experience will be taken further through a greater engagement of South Asian partners and advanced knowledge and experiences from the North.

17. The main success of this capacity building process was its sustainability. • The South Asian Institutions were able to run the course on a cost recovery basis. • Thus with infusion of a small start up funding together with catalytic actions, this intervention presents a viable model to address the need of professional education.

18. Future Directions….. • Planning has been initiated to offer a on campus and online diploma in evaluation. • Consolidation and standardization of training module of one week executive level course for wider replicability. • Evolution of a regional network of professional evaluators to support the evaluation activities.