‘Evaluating the Quality of eLearning in a Classical University: The practical case of the University of Granada’ Edinbu

1. Coimbra Group e-Learning Workshop “Quality in e-Learning” ‘Evaluating the Quality of eLearning in a Classical University: The practical case ofthe University of Granada’ Edinburgh, 14 March 2005 Isabel Pérez Torres Virtual Learning Centre of the University of Granada

2. Coimbra Group e-Learning Workshop “Quality in e-Learning” • Past and Present of elearning at the University of Granada (UGR). • The CEVUG virtual learning approach • The Gui@FORTIC. • A real-world example: the UGR blended courses.

3. Past & Present of eLearning at the UGR(1) • The experience of the University of Granada (UGR) in e-learning from 1995-2001 • Participation in several international projectsunder the umbrella of the Coimbra Group of Universities and in the EUNITE alliance. • Teaching staff training programme on the use of ICT and the development of virtual authoring tools. • Experiences were decentralized, as many different actors from different Vice-Rectorades took part.

4. Past & Present of eLearning at the UGR(2) • The Secretariat for Information Technologies in Education (STAD) and the Virtual Learning Centre (CEVUG) (2001-) • Generation of under and post-graduatevirtual courses. • Methodological and technical training for content authors and on-line tutors. • Provision of guides for content authors, on-line tutors, evaluation, LMS use, etc. • Maintenance of our own computer system structure. • Technical and methodological problem solving service. • Participation in several international e-learning projects.

5. The CEVUG virtual learning approach (1) • From the traditional instructional paradigm: • Teaching oriented. • Centred on the teacher’s knowledge. • To the new personal paradigm: • Learning oriented. • Student centred approach.

6. The CEVUG virtual learning approach (2) • The student centred approach is reflected in: • The way the teacher presents the information to the students. • The references and resources to be consulted. • The teacher-students interaction. • The student autonomous & self-learning experience.

7. The CEVUG virtual learning approach (3) • To optimise this learning model, we should take into account two crucial aspects: • The pedagogical quality of the materials and methodology. • The personalized support. • Technological aspects: promoting accessibility, taking advantage of the web characteristics (interactivity, global and up-to-date information, etc.). • Teaching methods: emphasizing constructivism, self-learning processes and group work.

8. The CEVUG virtual learning approach (4) • Different e-learning scenarios at Granada University: • Face-to face learning. • Blended learning: • training courses for teachers • undergraduate courses, pre-service teachers (CAP) • Virtual learning: • Post-graduate courses

9. Quality in e-Learning • The integration of the virtual model proposed by the CEVUG must be supported by the same quality standards applied to the rest of teaching and learning activities at the University of Granada.

10. Guía@Fortic(1) • Guía@ForticA Guide for the Evaluation of Formative Actions Based on Information and Communication Technologies. • A regional research project financed by the UCUA (Quality Unit for Andalusian Universities). • Granada University and six other universities. • Aim: to provide a tool for assessing the quality of e-learning actions. • The guide is based on the EFQM(European Foundation for Quality Model)

11. Guía@Fortic(2) • The process of evaluation Three phases: I. Planning the evaluation process. II. Developing the evaluation process. III. Elaborating the evaluation report.

12. I. Planning the evaluation process • Constitute the evaluation team (ET) • The ET will prepare a work plan including: • the organisational and functional structure, • a plan to collect information, • a plan to analyse data, • the internal actions, • the external actions, • the human and material resources, • the timetable.

13. I. Planning the evaluation process • How to design the evaluation of a formative action: Content, methodology and dissemination: • Identify the formative action. • Define the evaluation (determining criteria and subcriteria). • Plan how to collect information (questionnaires, interviews, etc.). • Plan the analysis and assessment of data (qualitative or quantitative methods; descriptive or comparative techniques). • Plan the evaluation report (structure, number of reports, etc.).

14. II. Developing the evaluation process • Evaluation criteria and subcriteria. Aspects to evaluate in each formative action (a): 1. Planning:identity data, plan structure, management procedures, follow up and improvement. 2. Programme:structure, objectives and contents; pedagogical materials and bibliography; activities and competencies to develop; communication tools; time schedule; assessment.

15. II. Developing the evaluation process • Aspects to evaluate in each formative action (b): 3. Resources: human resources (teachers, students, and supporting staff); technical and financial resources. 4. Process development:methodology; supporting and evaluation system; tutorial action. 5. Quality of the results: those deriving from the process and the satisfaction of the participants (students, teaching and technical staff).

16. II. Criteria and subcriteria • Each criteria is analysed taking into account: • aspects to considers (subcriteria); • evidences of fulfilment of the criteria and subcriteria • questions to help the evaluation. • In order to be more accurate criteria and subcriteria have been weighed => Some of them are considered more relevant than others.

17. III. Elaborating the evaluation report • General layout: • Title. • Content index. • Summary (overall design of evaluation, strong and weak points). • Introduction (description of the formative action, objectives, purpose of the evaluation, context, etc.). • Methodology (evaluation tools, data collection, etc.). • Evaluation results (findings, conclusions & recommendations). • Appendices (tables, references, glossary, etc.).

18. A real-world example: the UGR blended courses • The CEVUG virtual courses programme (PVA) aims at supporting an increasing number of blended courses each year. • Free credit choice subjects (6). • Optional subjects (6). • Online teaching should be up to 75%. • Teachers prepare the materials and resources. • The courses are tutored online. • They should be given for three years. • They undergo a selection process.

19. A real-world example: the UGR blended courses • The selection process: 1. Teachers submit a proposal signed by the department and the CEVUG. 2. The evaluation commission decides which subjects will take part in the programme. 3. Teachers are given technical and methodological training. 4. The teachers design the materials.

20. A real-world example: the UGR blended courses • The selection process: 5. The materials have to pass two processes of evaluation: • An internal evaluation by the STAD-CEVUG staff. • An external evaluation by other university experts. The evaluation is made according to: • The overall project quality. • The methodology and resources proposed. • The availability of previous digital resources. • The teacher’s qualifications in ICT.

21. A real-world example: the UGR blended courses • A questionnaire of standards o quality. • 1. Usability • Structure and navegation; layout; multimedia elements; language use; accesability. • 2. Pedagogical use • Objectives, content, activities, resources, materials, timetable, evaluation.

22. A real-world example: the UGR blended courses • The selection process: 6. The evaluation commission either • Accept the course => it will be taught as a blended course, or • Reject it => it will remain as a face-to-face course. • After the course: • Overall evaluation by the students: • materials, methodology, tutorial action, results.

23. 3w Thanks for your attention! http://cevug.ugr.es