1 / 24

Principles, criteria and methods Part 1

Quality management. Principles, criteria and methods Part 1. Produced in Collaboration between World Bank Institute and the Development Data Group (DECDG). Definitions of quality/I.

hastin
Download Presentation

Principles, criteria and methods Part 1

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Quality management Principles, criteria and methods Part 1 Produced in Collaboration between World Bank Institute and the Development Data Group (DECDG)

  2. Definitions of quality/I • Quality may be related either to subjective feelings or to objective facts • Objectively, the quality of something is the sum of its essential attributes or properties. • Quality has to do with user needs and satisfaction. • Users can not define the quality standards. • A minimum standard is to be transparent about the way in which concepts and methods are used.

  3. Definitions of quality/II • Subjectively, something has quality when it meets the expectations of the user • The International Organization for Standardization (ISO 8402) defines quality as: ‘Quality is the totality of features or characteristics of a product or service that bear on its ability to satisfy stated or implied needs of customers’. • Openness about the data shortcomings and needed data revisions can help in getting understanding and support.

  4. Quality frameworks • Some international organizations have developed quality frameworks for official statistics • These frameworks generally use the same kind of criteria, but there are slight differences • One of the most comprehensive quality framework for statistics is the Data Quality Assessment Framework (DQAF), developed by the IMF • Another useful international framework is the European Statistics Code of Practice (ESCP)

  5. DQAF Quality dimensions • The DQAF uses the following main dimensions of quality: • Prerequisites of quality • Assurances of integrity • Methodological soundness • Accuracy and reliability • Serviceability • Accessibility

  6. Prerequisites of quality • Prerequisites are not quality criteria, but refer to institutional quality elements: • Legal and institutional environment - the environment is supportive of statistics • Resources - resources are commensurate with needs of statistical programs • Relevance - statistics cover relevant information on the subject field • Quality management - quality is a cornerstone of statistical work

  7. Prerequisites – the indicators/1 • The responsibility for collecting, processing, and disseminating the statistics is clearly specified • Data sharing and coordination among data producing agencies are adequate • Individual reporters’ data are to be kept confidential and used for statistical purposes only • Statistical reporting is ensured through legal mandate and/or measures to encourage response • Staff, facilities, computing resources, and financing are commensurate with statistical programs • Measures to ensure efficient use of resources are implemented

  8. Prerequisites – the indicators/2 • The relevance and practical utility of existing statistics in meeting users’ needs are monitored and reported publicly • Processes are in place to focus on quality • Processes are in place to monitor the quality of the statistical program and reported publicly • Processes are in place to deal with quality considerations in planning the statistical program and reported publicly

  9. Assurances of integrity • Explanation: the principle of objectivity in the collection, processing, and dissemination of statistics is firmly adhered to • Elements are: • Professionalism • Transparency • Ethical standards

  10. Integrity – indicators/1 • Statistics are produced on an impartial basis • Choices of sources and statistical techniques as well as decisions about dissemination are informed solely by statistical considerations • The appropriate statistical entity is entitled to comment on erroneous interpretation and misuse of statistics • The terms and conditions under which statistics are collected, processed, and disseminated are available to the public

  11. Integrity – indicators/2 • Internal governmental access to statistics prior to their release is publicly identified • Products of statistical agencies/units are clearly identified as such • Advanced notice is given of major changes in methodology, source data, and statistical techniques • Guidelines for staff behavior are in place and are well known to the staff

  12. Methodological soundness • Explanation: The methodological basis for the statistics follows internationally accepted standards, guidelines, or good practices Elements: • Concepts and definitions have to be in accord with internationally accepted statistical frameworks • Scope must be in accord with internationally accepted standards, guidelines, or good practices • Classification/sectorization are in accord with internationally accepted standards, guidelines, or good practices • Basis for recording – Flows and stocks are valued and recorded according to internationally accepted standards, guidelines, or good practices

  13. Methodology – Indicators/1 • The overall approach to statistics, in terms of concepts and definitions follows internationally accepted standards, guidelines, or good practices, as well as the scope, and the classification system • The scope is broadly consistent with internationally accepted standards, guidelines, or good practices • Classification systems used are broadly consistent with internationally accepted standards, guidelines, or good practices

  14. Methodology – Indicators/2 • Market prices are used to value flows and stocks • Recording is done on an accrual basis • Grossing/netting procedures are broadly consistent with internationally accepted standards, guidelines, or good practices

  15. Accuracy and reliability/1 • Explanation: Source data and statistical techniques are sound and statistical outputs sufficiently portray reality Elements: • Source data available provide an adequate basis to compile statistics • Assessment of source data • Statistical techniques employed conform to sound statistical procedures • Assessment and validation of intermediate data and statistical outputs - • Correct application of data collection, processing and data analysis. • Correct sampling • Correct application of the mathematical tools.

  16. Accuracy and reliability/2 • Revision studies - Revisions, as a gauge of reliability, are tracked and mined for the information they may provide • New methods may be needed to be introduced and existing methods may be needed to be revised and changed. • Revisions also influence back-data.

  17. Accuracy and reliability – Indicators/1 Indicators of accuracy and reliability include several aspects related to source date: • Data collection programs take into account country-specific conditions • Source data reasonably approximate the definitions, scope, classifications, valuation, and time of recording required • Timeliness • Routinely assessment • Data compilation employs sound statistical techniques to deal with data sources • Other statistical procedures (e.g. data adjustments and transformations) employ sound statistical techniques

  18. Accuracy and reliability – Indicators / 2 • Intermediate results are validated against other information where applicable • Statistical discrepancies in intermediate data are assessed and investigated • Statistical discrepancies and other potential indicators or problems in statistical outputs are investigated • Studies and analyses of revisions are carried out routinely and used internally to inform statistical processes

  19. Serviceability/1 • Explanation: Statistics, with adequate periodicity and timeliness, are consistent and follow a predictable revisions policy Elements: • Periodicity and timeliness - Periodicity and timeliness follow internationally accepted dissemination standards

  20. Serviceability/2 • Consistency - Statistics are consistent within the dataset, over time, and with major datasets • Revision policy and practice - Data revisions follow a regular and publicized procedure

  21. Serviceability - Indicators • Periodicity and timeliness follows dissemination standards • Statistics are consistent within the dataset • Statistics are consistent or reconcilable over a reasonable period of time • Statistics are consistent or reconcilable with those obtained through other data sources • Revisions follow a regular and transparent schedule • Preliminary and/or revised data are clearly identified • Studies and analyses of revisions are made public

  22. Accessibility • Explanation: Data and metadata are easily available and assistance to users is adequate • Elements: • Data accessibility - Statistics are presented in a clear and understandable manner, forms of dissemination are adequate, and statistics are made available on an impartial basis • Metadata accessibility - Up-to-date and pertinent metadata are made available • Assistance to users - Prompt and knowledgeable support service is available

  23. Accessibility – Indicators/1 • Statistics are presented in a way that facilitates proper interpretation and meaningful comparisons (layout and clarity of text, tables, and charts) • Dissemination media and format are adequate • Statistics are released on a pre-announced schedule • Statistics are made available to all users at the same time • Statistics not routinely disseminated are made available upon request

  24. Accessibility – Indicators/2 • Documentation on concepts, scope, classifications, basis of recording, data sources, and statistical techniques is available, and differences from internationally accepted standards, guidelines, or good practices are annotated • Levels of detail are adapted to the needs of the intended audience • Contact points for each subject field are publicized • Catalogs of publications, documents, and other services, including information on any changes, are widely available

More Related