Download
1 / 64
640 likes | 775 Views
Students’ Green Fund. First Support Day | 06 September 2013. Agenda. 11am Welcome, context and purpose of today 11.15am Meeting your fellow projects 11.30am Introduction to M&E and how things are going to work today 11.45am Domain huddle 12.30pm Lunch 1.15pm Audiences huddle
E N D
Students’ Green Fund First Support Day | 06 September 2013
Agenda 11am Welcome, context and purpose of today 11.15am Meeting your fellow projects 11.30am Introduction to M&E and how things are going to work today 11.45am Domain huddle 12.30pm Lunch 1.15pm Audiences huddle 2pm Project framing huddle 2.30pm Getting you up to speed – 1) Behaviour change, 2) Research methods, 3) Carbon calculation 3.30pm Next steps and questions 4pm Close
Welcome Context Purpose of today
The successful projects 25 successful projects, 27 SUs / colleges Lancaster, UCLan, Bradford, Sheffield, Staffordshire, Exeter, Leicester, Southampton, Northampton, Cumbria, • Gloucestershire, Bedfordshire, Greenwich, Newcastle, Liverpool, Leeds, City, Bristol, Birmingham City, Worcester, Falmouth and Exeter, Roehampton, Brighton, Wigan and Leigh College, University of Sheffield, Sheffield Hallam University and Sheffield College
Meeting your fellow projects Find someone from a different project, explain your project to them as an ‘elevator pitch’, hear their pitch and move on. Try to fit in 3 different projects!
Monitoring and evaluating the SGF is important… M&E is an essential part of an effective project Without it, you won’t know what you’re achieving It will help you learn what is/isn’t working (so you can adapt) It will help others learn from what you are doing Finally, it will help others judge your effectiveness (and resource you accordingly) In short, M&E is primarily there to enable learning… and thereby deliver change: Learning and change are intertwined
Collectively we’re aiming for… • Impact • 4,000 tCO2/year saved • Increase in LiFE and Green League scores • Student adoption of pro-environmental behaviours continues beyond HE • Institutional leaders are more engaged in sustainability, resulting in a more holistic and mainstream approach to sustainability across the institution • Student governors, course reps, and academics become more engaged in sustainability, resulting in more courses with embedded sustainability content • Institutions become more receptive and collaborative to student opinion and demand on sustainability issues, and act accordingly • Institutions integrate sustainability into their graduate attributes and core purpose • Awareness and participation • Increase in student participation in pro-environmental actions • Increase in student awareness of sustainability initiatives • Increase of 10-15% adoption of pro-environmental behaviours • Graduate attributes include skills and knowledge to take positive action on sustainability • Improve graduate employability • Reach • 125,000 unique page views • 50,000 students engaged • 20,000 followers on social media • 5,000 staff engaged • 100% engagement of HE SU’s with SGF
How things will work today We want your help in developing the M&E strategy for the fund – this will ensure you it is appropriate to your project activities, and provides you with the information you need to understand how you project is working Hopefully you’ve all completed Task 2 circulated with the information about the event today. This sheet outlined three key areas of work, which we’re calling huddles. These are: Domain Audience Framing of your project / Approach
Example | Student Switch Off Students – accommodation type Skills and employability Energy Waste Competition Students – not the usual suspects Skills and employability
Forming the huddles The next three sessions of the day you will be working in the huddles you have identified to identify the areas of questioning that are most important to your project If your project spans several domains, please try to visit each one When you are at the huddle, please use the flip chart pads, post-its etc. provided to brainstorm the questions or areas of questioning that will help you track the performance of your project There are example questions on each table to prompt you if needed A member of the E&E team will also be visiting the huddles to help guide your discussions
Huddle 1 | Domain What are the key indicators that will help you track the effectiveness of your project? Are there particular behaviours or attitudes you need to assess? Do you need to know levels of awareness of these domains? What would project success in this domain look like?
Thinking about audiences First rule(s) of behaviour change: “Identify the audience group and the target behaviour” [AD, GSR Knowledge Review 2008] “Identify the barriers and benefits to an activity from the audience’s point of view” [McKenzie Mohr 2000] “Always segment” [Andreasen 1995]
Key principles for segmentation Segmentation is… 1. A practical tool Developed to subdivide large audiences into well-defined subgroups, in order to target them more effectively If a behaviour change tool: segment on the behaviour(s) in question, OR the most proximal determinants of those behaviours 2. An iterative process Make this as transparent as possible but keep it flexible 3. As much an art as a science Focus on practical purposes of the model for diverse stakeholders: more heads are better than one (steering group, advisory group etc)
Potential uses of segmentation models • Segmentation is less good for… • Accurately profiling individuals • Re-classifying existing subgroups – especially where relevant ‘communities’ already exist which can be targeted effectively • (eg. localities; faith groups; age bands) • Segmentation is good for subdividing large samples in order to… • Build insight • Develop strategy • Design interventions • Devise KPIs, and evaluation frameworks • Target subgroups more effectively with mass communications/intervention • Develop a common language for audiences (collaborate on all the above)
Defra Pro-Environmental Segmentation Model (2008-) • It would embarrass me if my friends thought my lifestyle was purposefully environmentally friendly • Being green is an alternative lifestyle it's not for the majority • I find it hard to change my habits to be more environmentally-friendly • It's only worth doing environmentally-friendly things if they save you money • The effects of climate change are too far in the future to really worry me • It's not worth me doing things to help the environment if others don't do the same • It's not worth Britain trying to combat climate change, because other countries will just cancel out what we do • Which of these best describes how you feel about your current lifestyle and the environment? 17 Golden Questions… • I would only travel by bus if I had no other choice • For the sake of the environment, car users should pay higher taxes • People who fly should bear the cost of the environmental damage that air travel causes • I don't pay much attention to the amount of water I use at home • People have a duty to recycle • We are close to the limit of the number of people the earth can support • The Earth has very limited room and resources • If things continue on their current course, we will soon experience a major environmental disaster • The so-called 'environmental crisis' facing humanity has been greatly exaggerated
Huddle 2 | Audience What are the key indicators that will help you track the effectiveness of your project? Are there particular behaviours or attitudes you need to assess? Do you need to know levels of awareness, attitudes and behaviours amongst these audiences? What would project success each audience look like?
Huddle 3 | Framing of your project What are the key indicators that will help you track the effectiveness of your project? Are there particular behaviours or attitudes you need to assess? What would project success in this framing / approach look like?
What is behaviour change? Definitions are scarce – but here are two alternatives... • For policymakers: An intervention to encourage individuals to change their behaviour in a way that will help Government achieve its policy goals...incorporating a better understanding of behaviour (NAO 2011) • For practitioners: A way of working based on the understanding of behaviours and audiences which results in learning and change (Darnton 2012) Implications include that we may not need to target individuals in order to change behaviour
Spanning 3 Schools of Behavioural Theory… Behavioural Economics Social Psychology Sociology
…with 3 different views of people Behavioural Economics Social Psychology Sociology Individual as Rational Man Individual as Social Animal Individual as Actor
Introducing ISM Individual Social Material Each model/discipline suggests different avenues for intervention, and when faced with complex problems, we need to draw on as many as we can: AD’s GSR Review 2008: “There is no one winning model” BUT, faced with myriad models, practitioners ask ‘Which one should I use?’ ISM devised to bridge this divide: one single multi-disciplinary model Predominantly a practical tool (also, theoretical objections) ISM argues that for substantive and lasting behaviour change we must: Work across multiple contexts Draw on multiple disciplines Involve multiple stakeholders Combine into a package of interventions (or multiple at the same time)
ISM applied to…recycling SOCIAL CONTEXT - NORMS INDIVIDUAL CONTEXT - ATTITUDES MATERIAL CONTEXT - SCHEDULES
ISM applied to…mobile phone driving SOCIAL CONTEXT - MEANINGS MATERIAL CONTEXT – TIME & SCHEDULES INDIVIDUAL CONTEXT – COSTS & BENEFITS
Implications from behavioural theory for SGF M&E Overall, measure success vs ‘reach, outcomes and impacts’ But to understand (and potentially replicate) effectiveness, also measure change in the target factors and the target behaviour If using ISM as overarching evaluation framework, implication is that multidisciplinary ISM requires multiple research methods, eg. Costs & Benefits – experiments (inc. RCTs) Values, Beliefs & Attitudes – primary methods (quali; quant) Habits – (self) monitoring inc. pagers/diaries, SRHI Meanings – discourse analysis Infrastructure – (street) audits Time & Schedules – time use studies Outcome: story of success across ISM contexts/labels (qv. recycling), plus factor-specific insights
k k
Monitoring and evaluation is all about… Understanding your achievements and the changes occurring as a result of your project activities. Therefore we need to know: What the current situation is, including what your audiences are thinking and doing (baseline research) What the situation is at the end of the funding period, what your audiences are thinking and doing then (follow-up research) Carrying out mid-point research provides an update half way through, allowing you to adapt your work accordingly
Two main research techniques… Qualitative • Concerned with the ‘how’ and ‘why’ • Common methods include in-depth interviews or focus groups. • Particularly useful when investigating why people behave in certain ways. Quantitative • Concerned with ‘hard’ data, for example ‘what?’ and ‘how much or how many?’ • Therefore often numerical (QUANTity) • Methods include face-to-face questionnaires and online surveys.
Surveys / Questionnaires Generally considered a quantitative research technique But, open-ended questions can obtain qualitative data Online surveys are increasingly popular, there are advantages and disadvantages with this method…
Online surveys Advantages • Can reach large groups of people in a short amount of time. • Can gather facts, opinions, feedback and attitudes • Easily repeatable • Can cover more stigmatised or sensitive topics. • Possibility of interviewer bias is also reduced • They are also fast to deploy and report upon Disadvantages • Self-selection bias • Possible technical issues • ‘Netiquette’ • Survey fatigue • May be less suitable for complex issues • Possibility for respondents to misinterpret questions • May require follow-up research to unpack findings
Minimising disadvantages of online surveys Don’t promote the survey as a green/environmental/sustainability survey to encourage those without a particular interest in this area to take part Provide an incentive for completing the survey Test your survey before you send it out, making sure instructions to respondents are clear and easy to understand
Samples and populations Total population • For example, distributing the survey through an ‘all-student’ mailing list. • Sending to all students/staff allows you to gather data on people who have been taking part in your projects as well as those who haven’t. • Keep track of who is answering the survey so you can carry out extra promotion with sectors of the population who might be underrepresented. • If you’re just surveying your total project population, consider finding a control group to enable you to assess wider changes. Sampling • Only surveying a particular proportion of the population. • Normally selected randomly. • Need to remember that smaller groups are less likely to be selected. • Should be representative of the total population. • Online tools are available to help you calculate sample sizes.
Focus groups Disadvantages • Can be time consuming and costly to run and analyse • Some groups are difficult to moderate so you may not get the information you are looking for • Some participants may not feel confident expressing their opinions in a group situation (though online groups can be a solution to this) Advantages • Used to explore complex ideas and behaviours • A great tool for collective idea generation, evaluating services and exploring experiences. • Discover how different groups think and interact • Save time and money compared to individual interviews.
Focus group top tips Groups normally include 6-8 participants (thought its always good to invite a couple more to account for drop outs) Run groups across your project audiences, and invite a range of people to get a cross-section of views Ask open-ended, probing questions and try to involve all members of the group (even the quiet ones) The facilitator should remain neutral and simply guide the discussion rather than controlling it
M&E Guidance | Handbook and on-going support Monitoring and evaluation handbook distributed next week with further top tips and guidance Review your reach, outcome and impact statements and make sure you’re able to gather data on all of these Get in touch if you have any queries Use the ‘how to’ research guides that are available on NUS Connect
Carbon target Its about more than carbon, but… Weneed to collectively and quantifiably save 4,000 tCO2/year = 8,000 tCO2 Fyi average carbon for UK citizen c 11 tCO2/yr, so target like 727 people Electricity, gas, water, travel, (food) Only 12 projects have given carbon targets in bids. Total of just 3,000 tCO2. Need your help!
