PURPOSE

1. Evaluating the Effectiveness of OECD Economic SurveillanceJohn C. Whitehead School of Diplomacy and International RelationsSeton Hall University

2. PURPOSE • Our objective is to understand the overall impact of OECD economic surveillanceon member countries. • We analyzed economic surveys for randomly selected OECD Country Members across the three most recent review cycles. • We determined the extent to which OECD recommendations were implemented, and we also evaluated the content of these reports.

3. IMPORTANCE • The OECD is understudied relative to other international economic organizations. • Given the global economic crisis, surveillance is important. What are its strengths and weaknesses, and how can it be made better? • OECD surveillance is conducted every 18 months, and functions by peer review. How well does peer review work?

4. OUR APPROACH • Two project team members were randomly assigned to each of 24 OECD countries. • Each team member performed two tasks: • Assess the extent to which past recommendations by the OECD were adopted or pending in the two most recent Economic Surveys. • Assess the content of surveillance by completing a survey of the three most recent Economic Surveys. • Each country team initially worked independently, then collaborated to develop a composite assessment of surveillance effectiveness and surveillance content. • Project data is available on our website http://oecdproject.wordpress.com/

5. COUNTRIES IN STUDY • Twenty-Four randomly selected OECD Member Countries (out of 34 total)

6. MEASURING EFFECTIVENESS • Assessment of “Progress in Structural Reform” tables in two most recent Economic Surveys. • Coded actions taken by member countries as either • “adopted” (any official policy change in the spirit of OECD recommendations) and/or • “pending” (recommendations pending, proposed, or under review) • Total of 1680 unique recommendations by OECD (average of 70 per country)

7. OVERALL FINDINGS FOR EFFECTIVENESS • 52% ADOPTED (Median: 53%) • Range: Portugal 75%, Slovakia 25% • 28% PENDING (Median: 27%) • Range: UK 61%, Slovakia 15%

8. SPECIFIC FINDINGS FOR EFFECTIVENESS • We disaggregated these results in three ways: • Changes in effectiveness over time comparing the two most recent reviews • Comparing effectiveness in crisis and non-crisis countries • Comparing effectiveness across type of economic issue

9. CHANGES IN EFFECTIVENESS OVER TIME • Recommendation Adoption • 11 Improved, 12 Worsened, 1 Stayed the Same • Overall worsening with net average of -2% • Range: Turkey +32%, France -29% • Recommendations Pending • 10 Improved, 14 Worsened • Overall worsening with net average of -1% • Range: Italy +32%, Canada -25%

10. CRISIS VS. NON-CRISIS COUNTRIES • We classified countries in crisis based on whether they received IMF adjustment loans: • Greece (2011) • Hungary (2010) • Iceland (2009, 2011) • Ireland (2009, 2011) • Turkey (2008, 2010) • Mexico (2009-2011)

11. EFFECTIVENESS IN CRISIS AND NON-CRISIS COUNTRIES

12. CRISIS VS. NON-CRISIS COUNTRIES • We compare within-case variations for Greece and Hungary • These countries had Economic Surveys in years preceding crisis as well as crisis years. • Greece: Adopted 16% more recommendations in crisis years • Hungary: Adopted 14% more recommendations in crisis years • Implication: Surveillance can make a difference for countries in crisis.

13. EFFECTIVENESS ACROSS ISSUE AREAS *Social Policy includes education, healthcare, housing, labor, environment, immigration, human capital, pensions, family, aging, and infrastructure.

14. SURVEILLANCE CONTENT • Assessed three most recent OECD Economic Surveys • Coded each survey based on four questions: • 1). Is the political and social context of the country explained? • 2) Does the economic survey include a self-contained comprehensive statement assessing the effectiveness of past surveillance in the country? • 3) What reasons (if any) were provided where past OECD advice was not followed? • 4) Are dissenting views by the authorities clearly identified and explained?

15. OVERALL FINDINGS FOR SURVEILLANCE CONTENT • Question 1 • 91% of Surveys had political or social context. • Question 2 • 86% of Surveys had a comprehensive effectiveness statement. • Question 3 • 61% of Surveys contained reasons for past non-compliance. • Question 4 • 23% of Surveys contained dissenting views by state authorities.

16. SPECIFIC FINDINGS FOR SURVEILLANCE CONTENT • We disaggregated these results in two ways: • Changes in content over time comparing the two most recent reviews • Comparing content in crisis and non-crisis countries

17. CHANGES IN SURVEILLANCE CONTENT OVER TIME • Increased transparency over the span of the most three recent reports • Can improve surveillance by including more dissenting views which would provide a detailed explanation why past advice was not adopted

18. SURVEILLANCE CONTENT IN CRISIS AND NON-CRISIS COUNTRIES

19. CRISIS VS. NON-CRISIS COUNTRIES • More transparency about non-adoption of reforms in crisis countries. • Greece and Hungary had detailed discussions of dissents in pre-crisis years as well • Finding on Q4 for greater transparency about dissents should be tempered.

20. FINDINGS ON SURVEILLANCE CONTENT • Surveillance reports include contextual information. • Surveillance reports provide a benchmark for evaluating prior recommendations. • Explanations for why past recommendations were not adopted appear in only 3 out of 5 reports. • Discussions of non-adoption are more anecdotal than substantive.

21. FINDINGS ON SURVEILLANCE CONTENT • Surveillance reports include contextual information. • Surveillance reports provide a benchmark for evaluating prior recommendations. • Explanations for why past recommendations were not adopted appear in only 3 out of 5 reports. • Discussions of non-adoption are more anecdotal than substantive.

22. CAN BETTER CONTENT PRODUCE MORE EFFECTIVE SURVEILLANCE? • Adoption and pending percentages in the most recent Economic Survey by whether the prior survey had positive answers to Questions 3 and 4.

23. POLICY RECOMMENDATIONS • Economic Surveys should take a multi-year perspective to their recommendations. • The issue of non-adoption of reforms should receive greater attention. • More consideration should be given to the domestic political context when developing recommendations.

24. RECOMMENDATION 1: MULTI-YEAR PERSPECTIVE • Economic Surveys should take a multi-year perspective to their recommendations. • Helps to track whether pending recommendations become adopted in subsequent years • Helps call attention to potential for reform backsliding • Could be implemented thematically initially

25. RECOMMENDATION 2: ADDRESSING NON-ADOPTION • The issue of non-adoption of reforms should receive greater attention. • Economic Surveys should include a section on why prior recommendations have not been adopted. • More detail on whether countries dissent with recommendations would make dialogue more substantive • Consider developing a publicly available dataset (i.e. the IMF’s MONA data) • Use greater media outreach

26. RECOMMENDATION 3: TAKE POLITICS SERIOUSLY • More consideration should be given to the domestic political context when developing recommendations. • Consider prioritized recommendations • Consider tradeoffs between optimal reforms and politically feasible reforms

27. PROJECT TEAM MEMBERS

28. FOR MORE INFORMATION • Project Website: • http://oecdproject.wordpress.com/ • Email address for Project Team: • whiteheadOECDproject@gmail.com