1 / 9

The Use of Surveys in Determining Outcomes of Research, Technology and Development Programs

The Use of Surveys in Determining Outcomes of Research, Technology and Development Programs. Presented at: Joint CES / AEA Conference Presented by: Suzanne Lafortune, Partner Performance Management Network Inc. Date: October 26, 2005 Tel: (613) 236-2320 Fax: (613) 236-8644

isleen
Download Presentation

The Use of Surveys in Determining Outcomes of Research, Technology and Development Programs

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Use of Surveys in Determining Outcomes of Research, Technology and Development Programs Presented at: Joint CES / AEA Conference Presented by: Suzanne Lafortune, Partner Performance Management Network Inc. Date: October 26, 2005 Tel: (613) 236-2320 Fax: (613) 236-8644 Email: suzanne.lafortune@pmn.net www.pmn.net

  2. Results &Resources versus Reach www.pmn.net The ‘clustering’ of services or activities by results, resources per user, andreachcan provide new perspective. policy development program funding enforcement and prosecution Intensive Problem Solving / R&D Technical Specialist Support specialized advisory investigations inspections and monitoring Results education and communication (workshops, speaking, trade shows) + Education Resources per user routine advisory dissemination of voluntary codes Information / Advice Information products mass media Awareness Building Reach Source: Steve Montague, The Three Rs of Performance: Core concepts for planning, measurement, and management, Performance Management Network Inc., 1997, p 120.

  3. Frequency and Type of Measurement Results & Resources versus Reach Intensive Problem Solving / R&D Results + Resources per user Technical Specialist Support Education Information / Advice Awareness Building Reach * Project related surveys ** Periodic surveys related to more in-depth impacts 3 Source: Steve Montague, The Three Rs of Performance: Core concepts for planning measurement and management, Performance Management Network Inc., 1997, p.20

  4. Why Use Surveys for Formative / Early Evaluations of R&D Programs? • Relevance of program design to R&D needs • Satisfaction with ways in which program is designed and delivered • Eligibility criteria • Application process • Use of advisory boards • Deadlines • Time of year (academic community versus private sector) • Speed of decision making • Feedback / advice / staff communications and support before project / in early phases of project • Technical competence / expertise of staff

  5. Why Use Surveys for Formative / Early Evaluations of R&D Programs? • Early outcomes – before project • Awareness building • Strategic alliances with partners • Leveraging • Early outcomes – during project • Active participation of partners • Knowledge gained • Job creation / maintenance within organization during project • Early outcomes – at the very end of the project • Improved technical capability • New / improved product / process / technology / service • Project incrementality • Important to measure early on, particularly for longer term R&D projects

  6. Why Use Surveys for Summative Evaluations of R&D Programs? • Relevance of program implementation to R&D needs • Satisfaction with ways in which program was implemented • Feedback / advice / staff communications and support • Technical competence / expertise of staff • Reporting requirements • Payment process

  7. Why Use Surveys for Summative Evaluations of R&D Programs? • Intermediate outcomes • Impact on recipient organizations • Revenues / sales • Productivity • Sustained / long term jobs • Competitiveness / increased market share • Increased profitability • Cost savings • Exports / new client base • Diversification • Expansion • Etc. • Impact on industry • Spin offs • Technology transfer

  8. Why Use Surveys for Summative Evaluations of R&D Programs? • Long term outcomes • Impact on the economy / on society • Socio-economic benefits • Reduced consumer costs • Employment • Improved quality of life • Etc. • Environmental benefits • Improved energy efficiency • Health and safety benefits • Etc.

  9. Types of Target Groups to Survey for R&D Evaluations • Non-recipients • Formative evaluation – relevance to needs, satisfaction, incrementality • Summative evaluation – relevance to needs, incrementality • Recipients • Project partners • Formative evaluation – impact of program on participation, early project outcomes / results on partner organizations • Summative evaluation – extent of participation, long-term collaborations, other outcomes / results on partner organizations • Beneficiaries • Summative evaluation – impacts of projects on beneficiaries

More Related