1 / 119

DLHE advanced

DLHE advanced . 2012. Objectives. Methodology Recap and reflect on the methodology Questionnaire and data Review the questionnaire and data whilst learning how to overcome common issues and challenges Data submission and quality checking

upton-wong
Download Presentation

DLHE advanced

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. DLHE advanced 2012

  2. Objectives Methodology Recap and reflect on the methodology Questionnaire and data Review the questionnaire and data whilst learning how to overcome common issues and challenges Data submission and quality checking Understand the validation kit and changes to data collection Preparing for audit Find out about the audit process and findings from HEFCE DLHE and KIS Learn how to use the downloadable files

  3. Where to begin?

  4. The review • The review of DLHE took place ahead of 2011/12 • The sector was consulted through DLHE contacts who were invited to respond to the proposals • Changes were subsequently introduced for 2011/12 and communicated through seminars, circulars and the coding manual Today we recap and reflect on the changes and aim to draw on institutional experience to improve on procedures, practices and dealing with challenges…

  5. Coverage

  6. Coverage

  7. DLHE Coverage Flowchart

  8. So much to do in so little time… • HESA recognise that some institutions will now be surveying a higher volume of leavers in comparison to earlier years • The contact period has not been increased • Resource and budget will be impacted • It is important to note • Surveying a wider range of students generates a more diverse set of results andwill potentially provide more data…

  9. DLHE 2011/12

  10. Target response rates

  11. How did we do last time?

  12. How did we do last time?

  13. The KIS agenda • Key Information Set • Course level information for prospective students • Uses DLHE data At course level, there must be at least 23 or more responses and for salary information, at least 50% of the leavers (if 23+) who are employed full-time must have provided salary information

  14. Promoting DLHE Evidence shows there is a link between those institutions who promote DLHE and those who exceed target response rates Why not consider graduation ceremonies, graduate contact, internal publicity or offering prizes?

  15. Surveying the correct students

  16. What if it goes wrong? • Missed entirely (cohort) • Contact institutional liaison for further information • Missed entirely (individual) • Resurvey using correct questionnaire for intended census These coverage errors can be corrected at a later date but are best avoided

  17. What if it goes wrong • Dormant • Students sometimes reported as dormant if in the final stages of postgraduate study at the start of reporting period 2. This is a student reporting discrepancy and will result in the institutions own record systems not identifying students for DLHE purposes • Wrong census • Some institutions own record systems only allow for one end date to be recorded. HESA defines the end date as at the point the structured part of the course finishes whereas some institutions consider this to be the date they are awarded. If the end date is recorded as the award date then the student could be potentially surveyed for the wrong census These coverage errors usually go unmissed and cannot be easily corrected at a later date and therefore must be avoided

  18. Methodology

  19. Methodology • 2011/12 saw the introduction of a new more relaxed approach to DLHE methodology • Institution’s now decide when and how to contact leavers during the contact period (initial and follow-up no longer exist) • Using predetermined methods to contact leavers was viewed as restrictive…

  20. It’s now up to you • The new methodology now allows for the use of as many methods as possible during the contact period • With the view that by increasing the lines of communication, the number of responses will increase • Institutions are advised to review previous collection response rates for an indication of methods which attract the highest response rates

  21. Methods of contact • Postal mailing • Telephone questionnaire • PDF version (e-mail) • Centrally-hosted online questionnaire

  22. Paper questionnaire • HESA provide paper questionnaires for sending out by post • Questionnaires are also available in Welsh • Institutions are contacted by HESA with a set of questions about the size of the DLHE target population and the planned surveying arrangements in January (for April) and June (for January)

  23. Common issues Pros Cons Third party questions are not identified Coding boxes have been removed • One version of the questionnaire simplifies the DLHE process, is more cost effective and easier to manage Suggestions • Laminated copies showing third party questions • Create coding reference material

  24. PDF questionnaire • A PDF version of the questionnaire is available in both English and Welsh • Available to download from the HESA website • PDF questionnaires are emailed to leavers who are then required to print, complete, sign, date and return the form back to the institution • Institutions should consider using postage-paid returns to increase likelihood of responses. Leavers will not respond well to having to pay postage

  25. What are the chances Smartphone anyone? Do you open your emails from your smartphone? Will you go back to emails read from your smartphone? Is your smartphone remotely connected to a printer? • Whilst PDF remains a fairly problem free approach to surveying leavers, it is worth keeping in mind that response rates to emailed PDFs may be lower than other methods • Therefore this method should only be considered in combination with one or more other methods

  26. DLHE online system • Online version of the questionnaire (Welsh version also available) • Institutions provide link www.dlhe.ac.uk to leavers • HESA provide free of charge to institutions who register for system • Also know as ‘Centrally-hosted online DLHE survey’ • Has in-built validation to ensure questions and sections are answered • Leavers can complete it at their own accord and at their own pace

  27. More than just… An online DLHE survey tool Why not use the (free, maintained and updated by HESA) DLHE online survey to input responses? Inbuilt validation will ensure all valid questions are both answered and internally consistent Remember to adjust the STATUS field to reflect how the information was captured!

  28. Telephone • Often the most effective method – accounts for majority of responses but requires the most significant amount of resource • Callers require perseverance, an excellent telephone manner and a sound grasp of the requirements. Some institutions recruit students as callers and multilingual speakers are sometimes used

  29. Telephone best practice When to call Preparation Time is of the essence - know your leaver before calling (create a profile) Note the qualification awarded to the leaver vs. what they had originally aimed for Do not block your telephone number Ensure permissions are set to allow calls to mobiles Have the correct questionnaire for the survey period • Experience shows calling outside of standard working hours yields better results • Specifically between 5.30pm and 8.30pm is often considered the best time to call • Also consider employing staff to contact leavers at the weekend • A cut off time should be agreed and adhered to • Log the times of calls to establish peak or optimum times for future surveys

  30. Third parties • If a leaver is unobtainable, a third party can be used as an alternative source in providing the required information on behalf of the leaver • Most commonly this scenario will take place during a telephone call (some third parties have been known to also return paper questionnaires) • Third party information is considered legitimate on the basis they are fully confident the answers are correct (and relate to the applicable census date) • Third parties should only be asked a specific subset of the questions

  31. Other methods • Direct contact • During the field work period e.g. they visit the careers office or during the graduation ceremony • Academic departments Last resort • Can also be used as a source of information as they may have on-going interactions with the leaver(s) • Employers Last resort • If employer is known to the institution they too may be contacted

  32. Covering letter and email It is mandatory for all leavers to receive specific information in connection with DLHE. The covering letter, email or verbal confirmation provided during a telephone call; It is possible to include the covering letter or email for all methods with the exception of using telephone. If the institution opts to use the telephone as their first method it is important to remember to confirm this information verbally as part of the introduction

  33. Covering letter and email • Printed on headed paper (covering letter) • Do not amend HESA text • Include institutions own data protection notice • Include literature about the careers service, forthcoming alumni events • Branding helps

  34. Interactive task

  35. Discussion questions • Map the DLHE process within your institution -consider what additional tasks you might do as an institution • What barriers exist when undertaking DLHE? • How is the DLHE data used internally within your institution?

  36. The questionnaire

  37. Understanding the questionnaire • Only one version of the questionnaire is available and this is to be used for all methods • Specific questions must be answered for a valid response! Institution’s commonly follow up incomplete or illegible responses • Remember, third parties should only be asked set questions. There are no markings on the form to indicate these questions • The need to ‘pad out’ (‘X’ or default codes) the data/answers for questions which are not required or returned is no longer necessary due to the format (XML)

  38. Official use only

  39. STATUS • The top boxes are used to record the dates of attempted contact • These are not used to indicate how many times an institution should attempt contact • Many institutions attach their own labels to include more dates • The boxes on the bottom row are used to record the STATUS of the response: • (2) Telephone survey: HE leaver • (3) Telephone survey: third party • (5) Own institution’s student record • (6) Institutional third party • (7) Deceased • (8) Explicit refusal NEW

  40. Section A Activities

  41. Section A: Q1

  42. ALLACT and MIMPACT • Required for a valid response • To be asked third parties • Ensures remainder of information returned is internally consistent • If the leaver does not complete any activities or completes more than one activity but does not identify which is most important, in the first instance try to contact the leaver. If this is not possible use professional judgement to code ALLACT and MIMPACT based on completed section(s)

  43. A new concept… • The leaver (or third party) must decide which one activity is ‘most important’ • This doesn’t necessarily mean it has to be the job which takes up the most time or pays the most salary, or any further study, training or research which takes up the most time. Some leavers may opt to select the activity which will support their future circumstances/aspirations

  44. Section B Employment

  45. Section B: Q2

  46. JOBSNO • Not required for a valid response • To be asked third parties • Will be used to understand the extent of portfolio careers • No requirement to return less than 2 jobs and only able to return up to 9 jobs

  47. A new concept… • If JOBSNO is more than one, leavers are asked to provide details about what they consider to their main job • This doesn’t necessarily mean it has to be the job which takes up the most time or pays the most salary! Some leavers may opt to choose ‘on an internship’ or ‘Developing a professional portfolio/creative practice’ above full-time, paid work

  48. Section B: Q3 and Q4

  49. JOBTITLE and JOBDUTIES • Required for a valid response (at least one of the two) • To be asked third parties • Determines occupations that recent leavers from HE have entered and identifies leavers who are employed on postdoctoral research contracts • Must code up to highest level of detail. SOCDLHE2010 allows for up to 5 digits of detail with 423 codes available in total!

  50. SOCDLHE2010 and KIS • SOCDLHE2000 replaced with SOCDLHE2010, a more developed coding frame reflecting the changing nature and scope of employment activities (423 codes) • A new version of CASCOT has been released to reflect the move to SOCDLHE2010 and is downloaded from Aardvark • KIS will include a proportion of those leavers who go into occupations classified within SOC groups 1-3 ‘Professional/Managerial’

More Related