1 / 41

Innovation data collection: methodological procedures

Innovation data collection: methodological procedures. ECO - UIS Regional Workshop on Science, Technology and Innovation (STI) Indicators Tehran, Iran 8-10 December 2013. Luciana Marins, UIS. Ch. 8 OM - Survey procedures. Guidelines - collection and analysis of innovation data ;

eloise
Download Presentation

Innovation data collection: methodological procedures

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Innovation data collection:methodological procedures ECO - UIS Regional Workshop on Science, Technology and Innovation (STI) Indicators Tehran, Iran 8-10 December 2013 Luciana Marins, UIS

  2. Ch. 8 OM - Survey procedures • Guidelines - collection and analysis of innovation data; • Comparableresults over time and across countries; • Particular circumstances may require other methodology comparability.

  3. The survey approach • The “subject” approach: • Innovative behaviour and activities of the firm as a whole; • The “object” approach: • Specific innovations (“significant innovation” of some kind, firm’s main innovation).

  4. Populations (1) • The target population: • Innovation activities in the business enterprise sector (goods-producing and services industries); • Minimum: all statistical units with at least ten employees; • Classification by size: • Small: 10-49; • Medium: 50-249; • Large: 250 and above.

  5. Populations (2) • Statistical unit: • Size cut-off point: • Source: 2012 UIS Innovation Metadata Collection

  6. Populations (3) • The target population (cont.): • Classification by main economic activity: • (National industrial classification system); • ISIC; • NACE.

  7. Populations (4) • The frame population: • Units from which a survey sample or census is drawn; • Based on the last year of the observation period for surveys; • Ideal frame = up-to-date official business register NSOs; • If the register forms the basis for several surveys (innovation, R&D, general business), the information can be restricted to innovation.

  8. Survey methods (1) • Mandatory surveys increase response rates; • Census or sample surveys? • Sample surveys - representative of target population (industry, size, region) stratified sample; • Census - costly but unavoidable in some cases.

  9. Survey methods (2) • Completion: • Survey type: • Source: 2012 UIS Innovation Metadata Collection

  10. Survey methods (3) • Domains (sub-populations): • Subsets of the sampling strata; • Potential sub-populations: industry groupings, size classes, regions, units that engage in R&D and innovation-active; • Guidelines: • Same statistical units and classifications; • Consistence of the methods for results calculation; • Documentation of deviations in data treatment or differences in the quality of the results from the domains.

  11. Survey methods (4) • Sampling techniques: • Stratified sample surveys: size and principal activity; • Sampling fractions should not be the same for all strata; • Cross-sections: standard approach - new random sample for each innovation survey; • Panel data: alternative/supplementary approach.

  12. Survey methods (5) • Suitable respondents: • Methods: e.g., postal surveys, web-based questionnaires, personal interviews; • Unit’s most suitable respondent - very specialised questions that can be answered by only a few people; • Try to identify respondents by name before data collection starts.

  13. Survey methods (6) • Data collection method: • Source: 2012 UIS Innovation Metadata Collection

  14. Survey methods (7) • The questionnaire: • Pre-test; • Simple and short; • Order of the questions; • Questions on qualitativeindicators - binary or ordinal scale; • International innovation surveys - attention to translation and design; • Short-form questionnaires - units with little/no innovation activity previously reported.

  15. Survey methods (8) • Combination of Innovation and R&D surveys: • Reduction in the overall response burden; • Scope for analysing the relations between R&D and innovation activities; • Increase in the frequency of innovation surveys; • Country experiences - it is possible to obtain reliable results for R&D expenditures; • Longer questionnaire; • Units not familiar with the concepts of R&D and innovation may confuse them; • Different frames for the two surveys.

  16. Survey methods (9) • Survey combination: • Source: 2012 UIS Innovation Metadata Collection

  17. Survey methods (10) • Guidelines for conducting combined surveys: • Questionnaire: two distinct sections; • Smaller individual sections; • Comparison of results from combined and stand-alone surveys should be done with care - surveying methods should be reported; • Samplesextraction from a common business register.

  18. Estimation of results (1) • Weighting methods: • Weighting by the inverse of the sampling fractions of the sampling units, corrected by the unit non-response; • If a stratified sampling technique with different sampling fractions is used, weights should be calculated individually for each; • Based on the number of enterprises in a stratum; • International and other comparisons:same weighting method.

  19. Estimation of results (2) • Non-response: • Unit non-response: reporting unit does not reply at all; • Item non-response: response rate to a specific question - % of blank or missing answers; • Disregarding missing values + applying simple weighting procedures based on the responses received assumes that respondents andnon-respondents are distributed in the same way biased results; • Possibility: imputation methods.

  20. Estimation of results (3) • Item non-response: • Unit non-response: • Source: 2012 UIS Innovation Metadata Collection

  21. Presentation of results • Descriptive analysis: no generalisation of results; • Inferential analysis: conclusions about target population; • Variance for the results: (average) values for innovation indicators and their coefficients of variation and/or confidence intervals; • Results presentation:metadata (data collection procedure, sampling methods, procedures for dealing with non-response, quality indicators).

  22. Frequency of data collection • Every 2 years; • If not economically feasiblefrequency of 3 or 4 years; • Specify an observation period; • The length of the observation period for innovation surveys should not exceed 3 years nor be less than 1 year.

  23. Annex A - 5. Methodological issues for developing country contexts (1) • Information system specificities: • Relative weakness of statistical systems: • Absence of linkages between surveys and data sets; • Lack of official business registers; • Involvement of NSOs; • When lacking, basic variables about firms’ performance can be included in the innovation survey.

  24. Annex A - 5. Methodological issues for developing country contexts (2) • General methodological considerations: • Survey application: • In-person; • Trained personnel; • Questionnaire design: • Sections can be separated to allow different persons in the firm to reply them; • Guidance/definitions; • Language and translation of technical terms.

  25. Annex A - 5. Methodological issues for developing country contexts (3) • General methodological considerations: • Frequency: • Every 3 to 4 years (e.g., timed to CISrounds); • Update a minimum set of variables every year; • Purpose of surveys; • Clear questions; • Adequate legislative base; • The results should be published and distributed widely.

  26. Basic innovation indicators:examples

  27. Howdo we measure innovation? (1) • Indicators - definition: • Statistics and data, often gathered through specialised surveys, are the building blocks from which indicators are constructed; • An indicator can be defined as something that helps us understand where we are, where we are going and how far we are from a specific goal. Therefore it can be a sign, a number, a graphic; • An indicator quantifies and simplifies phenomena and helps us understand complex realities. Source: International Institute for Sustainable Development / Adapted from Blakley, W. (2012). Providing and calculating innovation indicators. Cape Town, South Africa. ASTII/HSRC/UIS Workshop. (PowerPoint Presentation)

  28. How do we measure innovation? (2) • Indicators - definition: • Basic indicators: based on “one question”; • Composite indicators: combine answers to several questions in order to examine a number of policy-relevant factors and better capture the diversity of innovative firms.

  29. Innovation indicators - examples (1) • Product or process innovation: • % of firms that implemented product innovation • % of firms that implemented process innovation • % of firms that implemented product or process innovation (innovative firms) • % of firms that developed in-house product or process innovation • % of firms that implemented new-to-market product innovation

  30. Innovation indicators - examples (2) • Product or process innovation: • % of firms that implemented product innovation

  31. Innovation indicators - examples (3) • Product or process innovation: • Source: 2011 UIS Pilot Data Collection of Innovation Statistics

  32. Innovation indicators - examples (4) • Marketing or organisational innovation: • % of firms that implemented marketing innovation • % of firms that implemented organisational innovation • % of firms that implemented marketing or organisational innovation

  33. Innovation indicators - examples (5) • Marketing or organisational innovation: • % of firms that implemented marketing or organisational innovation

  34. Innovation indicators - examples (6) • Inputs: • Total expenditures on innovation (as a % of total turnover) • Expenditure on innovation by type of expenditure (as a % of total expenditure on innovation) • % of firms that performed R&D • % of firms that performed R&D on a continuous basis

  35. Innovation indicators - examples (7) • Inputs: • % of firms that performed R&D

  36. Innovation indicators - examples (8) • Key policy-relevant characteristics: • % of firms that were active on international markets • % of firms that co-operated with foreign partners on innovations • % of firms that co-operated with universities or other higher education institutions • % of firms that received public financial support for innovation • % of firms that applied for one or more patents • % of R&D-performing firms that co-operated with other institutions

  37. Innovation indicators - examples (9) • Key policy-relevant characteristics: • % of firms that co-operated with universities or other higher education institutions

  38. Innovation indicators - examples (10) • Key policy-relevant characteristics: co-operation • Source: 2011 UIS Pilot Data Collection of Innovation Statistics

  39. Innovation indicators - examples (11) • Key policy-relevant characteristics: • % of R&D-performing firms that co-operated with other institutions

  40. Final remarks • Data collected with innovation surveys are a important component of comparativestudies about countries’competitiveperformance; • Strategically important for policy-makers; • Data confidentiality; • Data reliability.

  41. Thank you! http://www.uis.unesco.org l.marins@unesco.org

More Related