Critical reflection and use of monitoring and Evaluation data

1. Critical reflection and use of monitoring and Evaluation data Capacity Development in Monitoring and Evaluation Dr. Rose Oluoch Rose_oluoch2002@yahoo.com

2. Critical reflection • What is it? • Why is it important? • Examples of critical reflection processes?

3. Definition: critical reflection • Questioning and analysing experiences, observations, theories, beliefs and/or assumptions. • Interpreting experiences and data to create new insights and agreement on actions. • Questioning what is normally taken for granted, particularly project assumptions: what did not work or is not working.

4. Importance of critical reflection • Without critical reflection, your M&E data will not help you to manage for impact, but only meet the bureaucratic demands of M&E. • It means moving beyond collecting, processing and reviewing data. • Critical reflection should lead to learning, by documenting and sharing decisions and ensuring these decisions are implemented.

5. Critical reflection processes and events • How will we make sense of the information gathered and used to make improvements? • Formally & informally • E.g. participatory project strategy review meetings, M&E development workshops, field visits & informal discussions with actors, quarterly progress reviews, periodic review workshops of key components, annual project review, preparation of supervision missions.

6. Key messages • M&E will only help if used in structured critical reflections with relevant stakeholders. • Critical reflection requires asking ‘why?’ ’so what?’ and ‘now what?’ after your M&E data show what has happened. • Reflections can happen in any forum. • Plan how to integrate a sequence of learning events to ensure clarity of insight and decisions.

7. Key messages • ‘Lessons learned’: • regular identification of lessons learned helps to systematise project experiences. • also critical to help others benefit from your problems & successes. • Annual project reviews with stakeholders: • redress problems and build on successes. • External events (e.g. supervisory missions & mid-termthereforeions or reviews): • identify strategic improvements.

8. Key questions for critical reflection • After asking ‘what is happening’, also ask regularly with stakeholders: • ‘Why is it happening?’ • ‘So what are the implications for the project?’ • ‘Now what do we do next?’

9. M&E made simple What Why So what Now what What has succeeded or failed Why have we had success or failure So what are the implications for the project? What action will we now take to make improvements

10. Documenting lessons learned - elements • Theme of ‘lessons learned’. • What was our original understanding or assumptions? • What is our revised understanding or assumptions? • One or two examples that substantiate the new understanding. • How the project came to this insight.

11. Learning at different levels • Individual & groups – field • Project level • Program level • Organizational level • Societal level (social learning) • Sectoral level

12. How to encourage critical reflection • Start with individual reflection. • Capture lessons learned with project stakeholders. • Plan for an integrated sequence of reflective events.

13. Individual reflection • If individuals do not reflect during their work on their own, they will probably find it difficult during group events, such as annual project reviews or monthly meetings with implementing partners. • Management style can influence this (e.g. group facilitator role).

14. The advantage of critical reflection in groups • To uncover new information. • To limit biases. • To build a clear picture of a situation / event / process and reach consensus. • To ensure well-reasoned, meaningful actions. • To facilitate action that has broad ownership.

15. Making M&E more reflective • Means planning more consciously when and how to deal critically with information. • Making project teams more reflective. • Reflecting with stakeholder groups. • Using steering groups for reflection. • Learning from your annual project review.

16. Making project teams more reflective • Before the meeting: • Decide who should be at project meetings. • Agree on scheduling: • long enough apart for new information to reflect on; • frequent enough for timely decision making; • Agree what M&E findings are to be discussed.

17. Making project teams more reflective • During the meeting • Ensure everyone has the same agenda and expectations are clear. • Share responsibilities, build skills and create a team spirit. • Ask staff to raise problems or dilemmas they are facing and invite everyone to find solutions. • Encourage analysis of a critical incident or issue of importance. • At regular intervals, include constructive feedback exercises. • Make sure outputs of the meeting are action-oriented.

18. Encouraging analysis of a critical incident or issue of importance • What did I/we do? • What does this mean? • Why did this happen? • How can I/we do things better in the future?

19. Learning from your annual project review (APR) • Overall purpose: to reach conclusions about achievements and failures in order to improve ongoing programme quality, and to share these conclusions. • Also: • To ensure the overall project goals, results and implementation strategy remain appropriate. • To assess progress towards planned impacts. • To review implementation to date and analyse reasons for any deviations. • To review the operational and management effectiveness and efficiency of implementation. • To review lessons and actions to improve next year’s implementation and performance (therefore input for AWPB).

20. During the APR • Organise group discussion around: • reviewing overall progress towards the intended outcomes and impacts. • reviewing specific work carried out over the previous years and identify constraints and lessons learned. • review, and if necessary improve, the overall project objective hierarchy and strategy. • Keep it participatory and keep people engaged through good facilitation and creating the right workshop conditions. • Ensure decisions for improved action are agreed upon before the end of the workshop.

21. After the APR • Ensure that any documentation from the APR is action-oriented, identifying who is responsible for what and by when. • Ask key stakeholders to verify the report and recommend changes before circulation. • Share the workshop report with key individuals and organisations. • Complete and check the work plan. • Make a detailed budget. • Produce the AWPB document.