Enhancing Online Assessment Support for Distance Learners

1. Using technology to support our online distance learners taking assessments Karen Barton Amanda Jefferies Jonathan Meere Sulanie Peramunagama Andrew Pyper StilianosVidalis Amanda Yip

2. Background • University of Hertfordshire involved early on in the use of online test invigilation (proctoring) • Mariana Lilley, Jonathan Meere and Amanda Yip pioneered the use of live proctoring • Work provided a basis for European funding bid

3. Background – online proctoring • Record and Review, • Record and Review (with rear camera feed),  • Record and Review with Live Authentication,  • Live Authentication and Invigilation.  • Live Invigilation with UH Invigilators (Deployment)

4. OP4RE • ERASMUS+ funding £450,000 for 7 European HEI partners and 1 commercial provider • 30+ associate partners across Europe and globally • 3 year project from September 2016 -August 2019 • 5 intellectual outputs being worked on by different partners

5. Aims • 5 Intellectual outputs- ours is Security and anti-fraud legislation protocols relating to online inviglation (remote proctoring) • Specifically: • develop protocols to support the validity of tests • establish the underlying security of the proctoring platform

6. Methodology • Test for vulnerabilities in the actual system • Identify the threats to validity in the process of conducting the test

7. Test for vulnerabilities in the actual system

8. Methodology- vulnerability test • To test for holes or vulnerabilities in the security of the proctoring provider’s platform. • Penetration Testing Execution Standard (PTES) • Open Web Application Security Project (OWASP) methodologies

9. Results- vulnerability test • No unauthorised access was possible • a denial of service attack did not succeed • No information leakage was detected.

10. Identify the threats to validity in the process of conducting the test

11. Methodology- identifying threats to validity • Analysis of the issues identified by the proctoring service itself • Additional analysis performed by members of the research team • Focus group with UH Exam invigilators

12. Interim results - analysis • The analysis indicates significant shortcomings in the record and review process with one camera • Limited number of issues logged by proctoring system when compared to research team members.

13. Interim results – focus group with invigilators • What do invigilators look for in exam hall contexts? • How do they respond to suspicious behaviour? • Forms of cheating they have detected • To what extent are they confident that they can they detect suspicious behaviour given an example record and review test?

14. Interim results – focus group with invigilators • Communication concerns • Environmental concerns • Undetected help (external to the exam environment itself) • Overall, not confident about outcome of test with standard one-camera record and review protocol • Caveat- open book may be ok

15. Appropriate allocation of test protocols

16. An early version of the model

17. Thanks Any questions? Please contact Professor Amanda Jefferies  a.l.jefferies@herts.ac.uk