1 / 26

CONCEPTUAL ISSUES IN CONSTRUCTING COMPOSITE INDICES

CONCEPTUAL ISSUES IN CONSTRUCTING COMPOSITE INDICES. Nadia Farrugia Department of Economics, University of Malta Paper prepared for the INTERNATIONAL CONFERENCE ON SMALL STATES AND ECONOMIC RESILIENCE Organised by The Islands and Small States Institute

falala
Download Presentation

CONCEPTUAL ISSUES IN CONSTRUCTING COMPOSITE INDICES

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CONCEPTUAL ISSUES IN CONSTRUCTING COMPOSITE INDICES Nadia Farrugia Department of Economics, University of Malta Paper prepared for the INTERNATIONAL CONFERENCE ON SMALL STATES AND ECONOMIC RESILIENCE Organised by The Islands and Small States Institute of the Foundation for International Studies at the University of Malta and the Commonwealth Secretariat, London Valletta, Malta 23 - 25 April 2007

  2. Presentation Outline • Introduction • Desirable Attributes for Developing Statistics and Composite Indices • Main Conceptual Issues • Indicator Selection • Dealing with Missing Data • Normalisation • Weighting and Aggregation • Testing and Reviewing the Results Obtained • Conclusion

  3. Introduction

  4. Definition • A composite index, • is a weighted (linear) aggregation of a number of variables • wj is a weight, with 0≤wj≤1 and ∑wj=1 • Xcj is the variable of country c in dimension j • for any country c the number of policy variables are equal to j=1,…,m.

  5. Uses • Describe complex phenomena in a single indicator • Cross-national comparisons of country performance • Benchmarking exercises • General trends • Policy priorities and performance targets • Several examples of renowned composite indices, stock market indices, RPI, GDP

  6. Strengths • Summarises complex and multi-dimensional issues • Helps set the direction for policymakers and to focus the discussion • Supports decision making • Helps disseminate information • Make stakeholders and the public more aware of certain problems • Generates academic discussion

  7. Weaknesses • Subjectivity in computation • May send misleading policy messages and can easily be misused • May conceal divergences between different components • Increase difficulty in identifying proper remedial action • Measurement problems

  8. Desirable Attributes

  9. Quality Frameworks • IMF – Data Quality Assurance Framework • Eurostat Framework • OECD – Quality Framework and Guidelines for OECD Statistics • Booysen – Dimensions for Classifying and Evaluating Development Indicators • Briguglio – Desirable Characteristics for Developing Vulnerability Indices • JRC-OECD – Handbook on Constructing Composite Indicators

  10. Desirable Attributes of Composite Indices • Accuracy • Simplicity and Ease of Comprehension • Methodological Soundness • Suitability for International and Temporal Comparisons • Transparency • Accessibility • Timeliness and Frequency • Flexibility

  11. Main Conceptual Issues

  12. Main Conceptual Issues • Indicator Selection • Dealing with Missing Data • Normalisation • Weighting and Aggregation • Testing and Reviewing the Results Obtained

  13. Indicator Selection • Define the concept • Select indicators which satisfy desirable attributes • Do not select variables which beg the question • Draft an initial indicator set and review the available data • Keep the number of variables as small as possible but not fewer than necessary (PCA, FA)

  14. Indicator Selection (Cont.) • Check for correlation between the variables or sub-indices (rank correlation test, Cronbach coefficient alpha, cluster and discriminant analysis) • Review the indicators selected and seek external advice and opinion

  15. Dealing with Missing Data • Exclude the country from the analysis • Imputation methods: Single or Multiple

  16. Single Imputation Methods • Case deletion • Mean/median/mode estimation • Hot deck imputation • Regression imputation

  17. Multiple Imputation Methods • Regression Method • Propensity Score Method • Markov Chain Monte Carlo Algorithm

  18. Quantifying Qualitative Data • Using a mapping (Likert) scale • Optimal spread of the scale • Permits non-linearity • Defect relates to subjectivity

  19. Normalisation • Rescaling • Standardisation (or z-scores) • Percentage differences over previous years • Ratios • Rankings • Measuring the relative position vis-à-vis a specified point

  20. Weighting and Aggregation • Equal Weighting • Differential Weighting • Country-Specific or Indicator-Specific Weights • Weights Over Time: Constant or Changing

  21. Differential Weighting • Weights Reflecting the Statistical Quality of the Data • Stochastic Weights • Participatory Methods • Precautionary Principle • Regression Method • Benefit-of-the-Doubt Weighting System

  22. Aggregation • Linear or geometric aggregation • Aggregation methods and weighting systems • Non-compensatory multi-criteria aggregation

  23. Testing and Reviewing the Results Obtained • Uncertainty and Sensitivity Analysis • Outliers • Expert Opinion • Analysing the Results Obtained

  24. Conclusion

  25. Conclusion • Composite indices have their pros and cons. • Hard to imagine that the debate on the use of composite indices will be ever settled. • Composite indices should be identified for what they are. • However, their importance should not be undermined. • Provided they are built on sound methodological considerations they are very useful to portray complex phenomena in a simple manner.

  26. Thank you! farend@onvol.net

More Related