1 / 60

Assessment Using O nline A ssignment M anagement Systems

Assessment Using O nline A ssignment M anagement Systems. Glenn Ledder Department of Mathematics University of Nebraska-Lincoln gledder@math.unl.edu. Assessment Using O nline A ssignment M anagement Systems. Defining the terms OAM advantages and disadvantages

luke-ware
Download Presentation

Assessment Using O nline A ssignment M anagement Systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Assessment Using Online Assignment ManagementSystems Glenn Ledder Department of Mathematics University of Nebraska-Lincoln gledder@math.unl.edu

  2. Assessment Using Online Assignment ManagementSystems • Defining the terms • OAM advantages and disadvantages • Low-maintenance administration • Accurate and reproducible results • Getting meaningful results

  3. Defining the Terms

  4. “Assessment” • Conflicting meanings

  5. “Assessment” • Conflicting meanings • General meaning: measuring academic success in any context

  6. “Assessment” • Conflicting meanings • General meaning: measuring academic success in any context • Specific meaning: measuring academic success in a broad context

  7. “Assessment” • Conflicting meanings • General meaning: measuring academic success in any context • Specific meaning: measuring academic success in a broad context (larger than a single course)

  8. Assessment vs Evaluation • Dictionary definition: • “assess”: to determine the importance, size, or value of • “evaluate”: to determine or fix the value of

  9. Assessment vs Evaluation • Dictionary definition: • “assess”: to determine the importance, size, or value of • “evaluate”: to determine or fix the value of • Connotation: • “assessment” usually emphasizes the making of summary judgments • “evaluation” usually emphasizes the measurement of quantitative value

  10. Assessment vs Evaluation • Assessment: • measuring knowledge and understanding • Evaluation: • measuring achievement

  11. Assessment vs Evaluation • Assessment: • measuring knowledge and understanding • used for placement • Evaluation: • measuring achievement • used to determine course grades

  12. Assessment vs Evaluation • Assessment: • measuring knowledge and understanding • used for placement • Evaluation: • measuring achievement • used to determine course grades • Test: an instrument used for assessment, evaluation, or both

  13. OAM Advantages and Disadvantages

  14. OAM Advantages/Disadvantages + Immediate grading (no wait for grading or machine scoring)

  15. OAM Advantages/Disadvantages + Immediate grading + General purpose testing center (no need for direct faculty involvement)

  16. OAM Advantages/Disadvantages + Immediate grading + General purpose testing center – Administrative difficulties (How do you maintain the database?)

  17. OAM Advantages/Disadvantages + Immediate grading + General purpose testing center – Administrative difficulties + Algorithmic Q’s and symbolic A’s (individual answers, no prompts)

  18. OAM Advantages/Disadvantages + Immediate grading + General purpose testing center – Administrative difficulties + Algorithmic Q’s and symbolic A’s – Difficulties in writing questions (How to avoid tricky, picky, sticky?)

  19. Low-MaintenanceAdministration

  20. Low-MaintenanceAdministration • Lots of instructors and proctors • Should require minimal knowledge

  21. Low-MaintenanceAdministration • Lots of instructors and proctors • Should require minimal knowledge • One or two faculty manage the system • Should require minimal time

  22. Low-MaintenanceAdministration • Lots of instructors and proctors • Should require minimal knowledge • One or two faculty manage the system • Should require minimal time • Questions are “permanent.” • Student records are “transient.”

  23. The Math 106 EDU folder structure • 106 COURSE • Question banks • Gateway exam • Practice assignments The 106 COURSE data is “permanent.” The only regular changes are to the Gateway exam dates. Maintained by faculty manager.

  24. The Math 106 EDU folder structure • 106 COURSE • Question banks • Gateway exam • Practice assignments • 106 CLASS • Question banks • Gateway exam • Practice assignments • Student records The 106 COURSE data is “permanent.” The only regular changes are to the Gateway exam dates. Maintained by faculty manager. Students register for their own 106CLASS. Tests are inherited. Student records are local. Maintained by individual instructors/proctors.

  25. Accurate and Reproducible Results

  26. Reproducibility • Tests should measure something. • The test has to yield consistent results in multiple administrations for a single student.

  27. Reproducibility • Tests should measure something. • The test has to yield consistent results in multiple administrations for a single student. • But questions must change when a test is given at different times. • This is especially important for evaluations.

  28. Reproducibility • Use template problems to get a great variety of answers. • Use template problems to get uniformity of content and difficulty. • Group problems into categories that are consistent in content and difficulty.

  29. The Math 106 Gateway Exam 10 questions, 8 correct to pass • Elementary functions: xn, sin(ax), cos(ax), tan(ax), eax, ln x, nx 2. Products 3. Quotients 4. Compositions 5. Compositions of compositions 6. Products with a composite factor 7. Compositions of products 8. Quotients with an embedded composition 9. Quotients with an embedded product 10. Functions defined by equations

  30. Category 4 - Compositions A, C, N > 0; B ≠ 0; K ≠ 0,1; The algorithm chooses 5 random integers satisfying certain requirements.

  31. Category 4 - Compositions A, C, N > 0; B ≠ 0; K ≠ 0,1; X=t,u,v,w,x,y,z; The algorithm randomly chooses an independent variable.

  32. Category 4 - Compositions X=t,u,v,w,x,y,z; A,C,N>0; B≠0; K≠0,1 P=XN+B,XN+BX Q=AXN+B,AXN+BX,sqrt(X)+B S=sinAX,cosAX,tanAX T=e-CX+B,eKX+BX U=Ae-CX+B,AeKX+BX,AlnX,ANX The algorithm randomly creates functions P, Q, S, T, U, using some of the chosen integers.

  33. Category 4 - Compositions X=t,u,v,w,x,y,z; A,C,N>0; B≠0; K≠0,1 P=XN+B,XN+BX Q=AXN+B,AXN+BX,sqrt(X)+B S=sinAX,cosAX,tanAX T=e-CX+B,eKX+BX U=Ae-CX+B,AeKX+BX,AlnX,ANX F1=sqrt(P),sqrt(S),sqrt(T) F2=SN,TN F3=lnQ,lnCS F4=eQ,eCS F5=sinQ,cosQ,sinU,cosU The algorithm chooses one of 5 functions based on prior elements. There are 38templates, each with 7independent variables and at least one parameter

  34. Reproducibility • Tests should measure something. • The test has to yield consistent results in multiple administrations for a single student.

  35. Accuracy • Tests should measure what they were intended to measure.

  36. Accuracy • Tests should measure what they were intended to measure. • Questions should have to be done right to be counted right.

  37. Accuracy • Tests should measure what they were intended to measure. • Questions should have to be done right to be counted right. • Students who can do what the test is supposed to measure should get a good score.

  38. Counted right→Done right • Savvy test takers can do well on multiple choice tests even when they don’t understand the material.

  39. Counted right→Done right • Savvy test takers can do well on multiple choice tests even when they don’t understand the material. • For calculations, students should have to type in the answer.

  40. Counted right→Done right • Savvy test takers can do well on multiple choice tests even when they don’t understand the material. • For calculations, students should have to type in the answer. • For conceptual questions, use multiple selection. (Which of the following…)

  41. Can do→Good score • Tricky: Misinterpretation or small mistakes account for many wrong answers.

  42. Can do→Good score • Tricky: Misinterpretation or small mistakes account for many wrong answers. Find the derivative ofcose2x and Find the derivative ofe-2cos x are tricky

  43. Can do→Good score • Tricky: Misinterpretation or small mistakes account for many wrong answers. Find the derivative ofcose2x and Find the derivative ofe-2cos x are tricky, compared to Find the derivative ofe2cos x

  44. Can do→Good score • Tricky: Misinterpretation or small mistakes account for many wrong answers. Find the derivative ofcose2x and Find the derivative ofe-2cos x are tricky, compared to Find the derivative ofe2cos x The problem is meant to test the chain rule, not elementary algebra.

  45. Can do→Good score • Picky: Too many details connect minor errors with wrong answers. Find the derivative of cos2(2x+3)+4sin x. Find the derivative of 4x5-2xcos(ex2).

  46. Can do→Good score • Picky: Too many details connect minor errors with wrong answers. Find the derivative of cos2(2x+3)+4sin x. Find the derivative of cos2(2x+3). Find the derivative of 4x5-2xcos(ex2). Find the derivative of ex5cos(ex2).

  47. Can do→Good score • Picky: Too many details connect minor errors with wrong answers. Find the derivative of cos2(2x+3)+4sin x. Find the derivative of cos2(2x+3). The simpler problems are hard enough. Find the derivative of 4x5-2xcos(ex2). Find the derivative of ex5cos(ex2).

  48. Can do→Good score • Sticky: Difficulties in transferring answer from paper to computer. 4x(x+3)-2x2 (x+3)2 2x2 x+3 Find the derivative of —–. ————

  49. Can do→Good score • Sticky: Difficulties in transferring answer from paper to computer. 4x(x+3)-2x2 (x+3)2 2x2 x+3 Find the derivative of —–. ———— 32 25 2x2 x+3 Find the derivative of —– at x=2. —

  50. Can do→Good score • Sticky: Difficulties in transferring answer from paper to computer. 4x(x+3)-2x2 (x+3)2 2x2 x+3 Find the derivative of —–. ———— 32 25 2x2 x+3 Find the derivative of —– at x=2. — 2x2 (x+3)2 Type4x(x+3)-2x2/(x+3)2, get 4x(x+3)-———

More Related