1 / 11

Presentation 3:

Presentation 3: “Practical experiences gained from carrying out case studies in relation to different policy themes and/or overall context settings” Dr. Thomas Stumm Managing Director of EureConsult S.A. (Echternach, G.D. of Luxembourg).

eshepherd
Download Presentation

Presentation 3:

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Presentation 3: “Practical experiences gained from carrying out case studies in relation to different policy themes and/or overall context settings” Dr. Thomas Stumm Managing Director of EureConsult S.A. (Echternach, G.D. of Luxembourg)

  2. Evaluating Structural Funds interventions in one Member State:Telling the story from a country-specific perspective My experience confirmed to me that in-depth case studies have great potentials for unveiling a number of issues which are normally not perceivable at a first glance by any horizontal evaluation.  The precise but often also diverse nature of impacts resulting from a policy intervention in a given region (e.g. “What has been achieved in terms of direct/indirect effects and intended/unintended effects?).  The way in which a policy intervention is implemented and working in practice in the particular region being examined (e.g. “How has a policy intervention achieved the impact?).  The influence of contextual factors or overall developments in order to locate the effect of a particular intervention or the way how a policy intervention has been realised within these (“Why” is the “what” and “how” of a policy intervention like it is?).

  3. Evaluating Structural Funds interventions in one Member State:Telling the story from a country-specific perspective If multi-site case studies are realised for a specific country, an appropriate presentation format needs to be developed when elaborating a “general framework” for carrying out the case study work ( core teams of the main contractor). It has proved very useful presenting the overall outcome of multi-site case studies realised for a single Member State in a wider format (e.g. in form of a “country report”).  General context analysis preceding the actual case study part.  Subsequent in-depth analysis of the cases under review.  Final synthesis relating in-depth analysesto initial working hypotheses / the general patterns of the country context.

  4. Evaluating Structural Funds interventions in one Member State:Telling the story from a country-specific perspective A sound initial briefing of the field teams carrying out case study work at a later stage should be realised,e.g. in form of a one-day “kick-off workshop” bringing together all parties involved in an evaluation. The field teams / local experts need to get a clear understanding about the wider purpose and strategic objectives of the overall evaluation ( task of the client). The field teams / local experts must be made fully aware of the central purpose of the case study within the wider evaluation ( task of the core team). The field teams / local experts must become fully familiar with all aspects relating to the practical realisation of case studies ( task of the core team). Due to my positive experiences made with such briefings, I recommend this to be a mandatory first step in any evaluation involving extensive case study work.

  5. Evaluating Structural Funds interventions in one Member State:Telling the story from a country-specific perspective Field evaluators should realise a case study-based in-depth analysis with a certain degree of “distance” and “flexibility”.  As the preparation / elaboration of a good case study needs time, try to take some distance with respect to the inevitably occurring time-pressure & keep up professional standards.  Be flexible in the actual preparation & realisation of stakeholder interviews (e.g. drop questions or add new ones; carry out single-person interviews and structured group discussions)  When drawing up the case study report, try taking some distance with respect to an initially defined reporting template. Intuitively “co-determine” the importance given to a pre-defined item on ground of your actual knowledge & with respect to the actual relevance of the topic within the wider story to be told about a case.  Show intuitive flexibility while elaborating an informative and analytical but still appealing narrative which tells the story about a specific case (e.g. by focussing the story on a number of “critical incidents”).

  6. Evaluating territorial co-operation programmes & projects: Telling the story from a multi-country perspective The use of the case study method is particularly recommended for evaluating European Territorial Co-operation (ETC). This is due to the fact that the complexity of real-life interactions within co-operation processes is considerably higher if compared to those taking place in interventions carried out in a single-country perspective. Case studies can - in principle - be used at three different levels: (1) When evaluating Community-level funding schemes such as the “old” INTERREG Community Initiative or the new ETC-Objective. (2) When evaluating an individual cross-border, transnational or inter-regional co-operation programme. (3) When evaluating a specific co-operation project.

  7. Evaluating territorial co-operation programmes & projects: Telling the story from a multi-country perspective According to my practical experience, however, case studies on ETC are also a very demanding and particularly challenging task for each evaluator involved.  The evaluator has to be familiar with more than only one country-specific context (e.g. basic political-administrative settings, different legal & regulatory systems, different cultures & ways of “doing” and “thinking”).  The evaluator needs to have well-developed and appropriate language capacities (esp. in the context of cross-border co-operation).  The evaluator needs to have a well developed understanding about the previous history of co-operation (i.e. long-term development & core activities carried out in the past).  The the activities for preparing and realising an ETC-case study are much more complex as a greater number of key stakeholders originating from different countries and from different levels of co-operation need to be identified / contacted for telling a story which is comprehensive and well-balanced.

  8. Evaluating territorial co-operation programmes & projects: Telling the story from a multi-country perspective (1) An evaluation of Community-level funding schemes Compared to the previous INTERREG II ex-post evaluation, the current ex-post evaluation of the INTERREG III Community Initiative (2000-2006) has adopted a very different approach: Case studies play an important role in the entire evaluation process.  Based upon an extensive and aggregated analysis realised previously across all programmes, a typology of INTERREG III programmes was elaborated on ground of which a case-study sample was selected which well-reflected the diversity and types of existing programmes.  These 16 programme case studies, with each of them also involving the realisation of 5 shorter project-case studies, have shed more light on specific aspects which have already been analysed at a more aggregated level across all programmes.

  9. Evaluating territorial co-operation programmes & projects: Telling the story from a multi-country perspective (2) An evaluation of individual ETC-programmes Programme-level evaluations (e.g. the ongoing evaluations for the 2007-2013 period) should make more use of the case study method especially when it comes to assessing their project-level implementation.  This can provide the strategic “programme owners” with a much better understanding about the complex reality of the project-based implementation of their programme and about the real scope and nature of the effects generated by the approved projects.  The outcome of such project-level case studies is also of a long-term relevance, as it helps building up a permanent & precious “historical memory” about the client dimension of a programme. Another interesting perspective is the use of case studies in the context of future ex-ante evaluations for ETC-programmes after 2013.  Case studies can relate to various aspects in the process (i.e. problem assessment & strategy building and design of the programme-wide decision-making/management and implementations structures).  Case studies can adopt both a programme-internal focus (e.g. Learning from the past for the future) or a programme-external focus programme (e.g. Learning from others how they have dealt with a certain issue?).

  10. Evaluating territorial co-operation programmes & projects: Telling the story from a multi-country perspective (3) An evaluation of individual ETC-projects I am strongly convinced that the use of the case study method can also make an important contribution to raise the overall quality & depth of an individual project evaluation (if those are realised).  Project-focused evaluations are not yet realised very frequently: Reasons might be a not yet very broadly developed awareness about the associated benefits & the still rather weakly developed methods or techniques for evaluating ETC-projects.  An external evaluation should be carried out if a project has (1) a certain budget size (e.g. > € 2 million) and/or (2) if it has a rather complex implementation structure and realises particularly experimental actions bearing a certain risk of failure.  The most appropriate format to be adopted by a project evaluation should either be that of an ongoing evaluation or that of a “classical” mid-term evaluation.  The focus of an evaluation must serve the project & its (financial) scale must be reasonable. Individual project evaluations can elaborate “mini-case studies” which analyse all or certain operational elements of a project(e.g. an entire work package, a work process)or focus on issues of a particular common interest(e.g. depth & quality of project-level co-operation; emerging direct effects & unexpected direct/indirect effects etc.).

  11. Thank you for your attention!

More Related