Analyzing and Improving College Teaching: Here’s an IDEA

1. Analyzing and Improving College Teaching: Here’s an IDEA Alan C. Lacy, Associate Dean College of Applied Science and Technology Illinois State University

2. Overview • Assessment of Teaching • Components of Teaching • Sources of Evaluation • The IDEA Student Rating System • Mission of IDEA • Administration of IDEA • Using the results from IDEA

3. ASSESSMENT OF TEACHING

5. How well did I teach it? • What are the components of effective teaching? • Who should assess these components?

6. Components of Teaching • Content Expertise • Course Design • Instructional Delivery • Course Management

7. Sources of Assessment • Students • Peers • Administrators • Self

8. 1 = not important 2 = of minor importance3 = somewhat important 4 = very important CJ- Cannot judge • The appropriateness of an instructors course objectives • Faculty member’s contributions to course/curriculum development • Severity, laxness, and consistency of grading standards

9. 1 = not important 2 = of minor importance3 = somewhat important 4 = very important CJ- Cannot judge • The degree to which subject matter presented is balanced and current • Relevance and quality of assessment and testing methods • Faculty member’s disciplinary knowledge/skills

10. Students are not qualified to assess: • Faculty expertise • Appropriateness of goals, content, and organization of course • Materials used in delivery • How student work is evaluated, including grading practices

11. Nor are students qualified to assess: • Support for departmental efforts • Assistance to colleagues • Contributing to a positive atmosphere

12. Student are qualified to assess: • Instructional Delivery • Course Design • Instructor Availability • Progress To Course Objectives

13. Assessment Matrix

14. THE IDEA STUDENT RATING SYSTEM

15. What is IDEA? • Individual Development & Educational Assessment (IDEA) Student Rating System The IDEA Center Kansas State University Manhattan, Kansas http://www.idea.ksu.edu/

16. Mission of the IDEA Center • To serve colleges and universities committed to improving learning, teaching, and leadership performance

17. IDEA Formal Student Evaluation • Developed at Kansas State University • Based on research on teaching • Has national norms • Is tailored to instructor and course • Very detailed and helpful reports (Formative/Summative) • Currently utilized at 275 institutions • (8 years ago – 110)

18. IDEA – Improvement of Student Learning • Diagnostic tool to guide improvement • Evaluative tool to assess teaching effectiveness

19. Improvement of Student Learning • Student ratings have positive impact if…. • The instrument: • Is learning focused • Provides a diagnostic • Emphasis on summative evaluation is appropriate • 30-50% of overall evaluation of teaching • 6-8 representative classes (more if n<10) • Faculty trust the process

20. Improvement of Student Learning • Campus and college culture has great impact. • Is teaching excellence a top priority? • Are student ratings given an appropriate weight in overall evaluation of teaching? • Does the institution provide resources to aid those seeking to improve instruction? • Is credible support for improvement available?

21. Other Teaching Activities • Teaching Lab Classes • Curriculum Development • Mentoring Students • Supervising Field Experiences

22. Student Learning Model Assumptions • Types of learning must reflect instructor’s purpose • Effectiveness determined by student progress on objectives stressed by the instructor • Specific teaching behaviors influence certain types of student progress under certain circumstances

23. IDEA – Assessment of Teaching • Pilot tested at ISU in 2001-2002 • 40 Tenure/Tenure track Faculty • 975 students • Fully implemented now

24. Administration of the IDEA System • Complete Faculty Information Forms (FIF) • Administering Student Course Evaluations • Ship Evaluations to IDEA • Distribute IDEA Results to Departments

25. Costs for IDEA • Staff time • Monetary • $0.13 per course evaluation form • Shipping costs • $6 per class • Small charges for reports

26. Fall 2008 Administration • Assessed 400 classes in 7 departments • 11,500 student response forms • $3,224 total cost • Purchase of forms • Processing fees

27. Rationale for Use • Shows commitment to teaching • Provides validated course evaluation instrument based on research • Provides consistency across the College • Summative aspects aid personnel decisions • Formative aspects aid professional development

30. Using the IDEA Results • Fall 2008 College Data Compared to the IDEA Database

31. Adjusted Scores • Accounts for factors beyond instructor’s control • Uses regression equation • Student work habits (#43) • Student motivation (#39) • Class size (enrollment, FIF) • Student effort (multiple items) • Course difficulty (multiple items) Nothing like this in homegrown instruments

32. Using the Results • Outcomes • Student Learning • Process • Teaching Methods • Instructional Approach • Course and Curriculum Emphasis • Program Assessment Inputs • Student Background • Student Characteristics

33. Guiding Principles to Formal Evaluation Decisions • Comprehensive and systematic • Valid, summative information • Formative information for improvement • Consider teaching components • Multiple sources of assessment • Consider scholarship on best practices • Fair and equitable

34. Thank you for your attention! Any Questions?