Download
1 / 47
470 likes | 494 Views
How to Prepare Exams?. Hussain Ali Alkharusi , Ph.D Associate Professor of Measurement & Evaluation, College of Education Dean of Admissions & Registration Sultan Qaboos University April 22, 2018. Outline. Definition of assessment Quality of assessment Learning outcomes
E N D
How to Prepare Exams? Hussain Ali Alkharusi, Ph.D Associate Professor of Measurement & Evaluation, College of Education Dean of Admissions & Registration Sultan Qaboos University April 22, 2018
Outline • Definition of assessment • Quality of assessment • Learning outcomes • Principles of assessment • Alignment of assessment with learning outcomes • Assessment blueprint • General guidelines for exam’s development • Types of exam’s items • Guidelines for writing exam’s items • Time requirements for assessment tasks
Assessment • Assessment is a systematic process of collecting, analyzing, and interpreting information to determine how well student performance matches expected learning outcomes.
Quality Assessment Validity • The extent to which the assessment measures what it is supposed to measure and the extent to which inferences made on the basis of the assessment results are accurate. Reliability • The degree to which the results of the assessment are consistent across raters, or across different assessments that measure the same thing.
Learning Outcomes • Learning outcomes are operational statements describing what the student will able to know or do at the end of the course. Students will DO WHAT (how)
Examples of Course Learning Outcomes • By the end of the course, the student is expected to: • Understand basic concepts related to educational assessment. • Know basic principles of tests, their purposes, and classifications. • Prepare a table of specifications for a test on one or more units of his/her teaching subject area. • Construct various good test questions. • Analyze test items to verify their quality. • Prepare an evaluation report of student performance in accordance with the MoE’s regulations. • Create scoring rubrics and performance criteria. • Use at least one software program of item analysis.
Principles of Assessment • Assessment should enhance student learning. • Students should be aware of assessment criteria at the beginning of the course. • Marking should be accurate and consistent and based on pre-defined criteria. • Grading should be criterion-based rather than norm-referenced. • Feedback should be timely, meaningful, and helpful.
Principles of Assessment • Assessment should be linked to course learning outcomes. • Each course syllabus should include a matrix connecting course learning outcomes with the assessment methods to be used to test the outcomes. • Assessment should address different levels of attainment, including knowledge, comprehension and higher cognitive skills.
Assessment Blueprint • At least the major assessment components of the course (e.g., The Final Exam)should have a blueprint. • An assessment blueprint, also known as table of specifications, consists of a matrix representing the number of questions in the exam within each topic and level of learning outcome. It identifies the learning outcomes and topics that are to be assessed as well as the relative weight on the exam given to each.
General Guidelines for Exam’s Development (1-3) The test should reflect both the content and objectives in proportion to their importance and emphasis in the course. The items should be phrased so that the content rather than the form of the statement will determine the answer. The difficulty level of the items should be appropriate to the group of examinees. The items should reflect several taxonomy levels. Avoid items that reveal the answer to another item.
General Guidelines for Test Development (2-3) Tests should be power tests, not speed tests. The reading level should be relatively easy. It is often desirable to use more than one type of item. All items of a particular kind should ordinarily be placed together in the test. The directions to the students should be as clear, complete, and concise as possible. Before the actual scoring begins, answer keys should be prepared and scoring procedures outlines
General Guidelines for Test Development (3-3) Use case-based questions that use a common text to which a set of items refers Present practical or real-world situations to the students Present the student with a diagram of equipment and ask for higher order thinking questions Use pictorial materials that require students to apply principles and concepts Have diagrams, tables, charts, and texts above the item using the information Items should follow an easy to difficult progression
Types of Exam’s Items A. Essay type: • Restricted response (one-half page) • Extended response (2 pages) B. Objective type: • Multiple-choice • True-false • Short-answer • Completion • Matching
Writing Essay Questions: General Suggestions • Do not provide optional essay questions, except when you are assessing writing ability where a choice of questions is desirable. • Restrict the use of the essay questions to those learning outcomes that cannot be measured satisfactorily by objective items (e.g.; problem-solving, interpretation…etc.). • Use several shorter questions rather than fewer longer questions. • Phrase the question so that the students task is clearly defined.
Multiple-Choice Items The multiple-choice item consists of two parts: (1) the stem, which contains the problem stated as a direct question or an incomplete statement; and (2) a list of suggested answers (responses or options). The correct response is called the key. The incorrect responses are called distracters.
An Example of a Multiple-Choice Item Directions: Underline the letter of the option that represents the best answer for the following statement: In what type of test is the “halo” effect is more operative? A. Essay B. Matching C. True-False D. Short-answer E. Multiple-choice
True-False Items The true-false item is essentially a two-response multiple-choice item in which only one of the answers is presented and the student judges the truth or falsity of the statement.
Matching Questions A typical matching exercise consists of two columns so that each stem in the first column is to be paired with an alternative in the second column.
An Example of a Matching Question • Directions: Famous inventions are listed in the left-hand column below. In the right-hand column are names of famous inventors. Place the letter corresponding to the inventor in the space before the invention for which he is famous.
Short-Answer Items Short-answer items present a direct question that requires a single word, number, or phrase to be supplied by the student.
An Example of a Short-Answer Item What instrument is used to measure the resistance in an electrical circuit?
Completion Items The completion item is a special form of the short-answer item, where one or more blanks are inserted in one or more sentences for the student to fill in.
An Example of a Completion Item The first experimental psychology laboratory was located in the city of ……….
Poor : south America: A is a flat, arid country. B imports coffee from the United Stated. C has a larger population than the United Stated. D was settled mainly by colonists from Spain. Better: Most of south America was settled by colonists from…: A England B France C Holland D Spain 1- The stem of the item should be meaningful by itself and should present a definite problem.
2- The item stem should include as much of the item as possible and should be free of irrelevant material. Poor: Most of South America was settled by colonists from Spain. How would you account for the large number of Spanish colonists settling there? A they were adventurous B they were in search of wealth C they wanted lower taxes D they were seeking religious freedom Better: Why did Spanish colonists settle most of South America? A they were adventurous B they were in search of wealth C they wanted lower taxes D they were seeking religious freedom
Best : Spanish colonists settled most of South America in search of ……: A adventure B wealth C lower taxes D religious freedom
3- Use negatively stated stem only when significant learning outcomes require it. Better : Which one of the following states is located in Al-Batinahsouth Region? A Sohar B Saham CSuwaiq D Musena’a Poor: Which one of the following states is not located in al-Batinah north region? A Sohar B Saham CSuwaiq D Musena’a
Poor: Which one of the following is not a safe driving practice on icy roads? A accelerating slowly B Jamming on the breaks C holding the weal firmly D slowing down gradually Better : All of the following are safe driving practices on icy roads except: A accelerating slowly B Jamming on the breaks C holding the weal firmly D slowing down gradually
4- All the alternatives should be grammatically consistent with the stem of the item. Poor: An electric transformer can be used A for storing electricity B to increase the voltage of alternating current C it coverts electrical energy into mechanical energy D alternating current is changed to direct current Better: An electric transformer can be used to A store electricity B increase the voltage of alternating current C covert electrical energy into mechanical energy D change alternating current is changed to direct current
5- An item should contain only one correct or clearly best answer. Poor:Oman has borders with…: A UAE B Kuwait C KSA D Bahrain Poor: Oman has borders with A UAE T F B Kuwait T F C KSA T F D Bahrain T F
Poor:Which of the following is the best source of heat for home use? A Coal B Electricity C Gas D Oil Better: In the Midwestern part of the United States, which one of the following is the most economical source of heat for home use? A Coal B Electricity C Gas D Oil
6. Verbal associations between the stem and the correct answer should be avoided. • Poor: Which one of the following agencies should you contact to find out about a tornado warning in your locality?? • State farm bureau • Local radio station √ • United States post office • United States Weather Bureau • Better: Which one of the following agencies should you contact to find out about a tornado warning in your locality?? • A. Local farm bureau • Nearest radio station √ • Local post office • United States Weather Bureau
7. The relative length of the alternatives should not provide a clue to the answer. • Poor: What is the major purpose of the United Nations? • To maintain peace among the people of the world • To establish international law • To provide military control • To form new governments • Better: The major purpose of the United Nations is to….: • maintain peace among the people of the world • develop a new system of international law • provide military control of nations • maintain democratic governments in newly formed nations.
8. Avoid repetition of words in the alternatives • Poor:“The study of the price system narrows down to an analysis of these two sets of prices and the interrelationships between them.” The two sets of prices referred to are: • those for consumption goods and those for capital goods. • those for consumption goods and those for productive services. • those for labor and those for the other productive factors. • those for economic and those for noneconomic goods. • Better: “The study of the price system narrows down to an analysis of these two sets of prices and the interrelationships between them.” The two sets of prices referred to are those for: • consumption goods and capital goods. • consumption goods and productive services. • labor and the other productive factors. • economic and noneconomic goods.
Other Guidelines for Constructing Multiple-Choice Items Distracters should not overlap, subsume, or be synonymous with one another Example: A substance that in its pure form is a good conductors of electricity is: a) Waterb) Silver c) H2S D) H2O Alternatives should be homogenous, attractive, and, arranged in a logical order. The omissions in complete statements should usually occur toward the end of the stem. Avoid specific determiners such as “always”, “never”, “none”...etc. Use numerals for items and letters for options. Use the “none of the above” option sparingly, if at all. Avoid regular, recurring patterns of correct responses.
Special Guidelines for Constructing True-False Items Avoid a disproportionate number of either true or false statements. Avoid trick statements. Examples: “The Raven” was written by Edgar Allen Poe. (notice the misspelling of Allan) The Battle of Hastings was fought in 1066 BC. (notice “BC” instead of “AD”) True-false items must be based on statements that are clearly true or clearly false. Avoid the exact wording of the textbook. Avoid excess use of negative words and phrases Example: 1. On true-false tests negative phrases should not be disallowed
Special Guidelines for Constructing Matching Questions In matching questions, place items on the left and options on the right. Place all the items and options for a matching question on a single page. Limit a matching question to not more than 10 to 15 items. In matching questions, avoid having an equal number of premises and responses. In matching questions, explain clearly the basis on which the match is to be made. Include only homogeneous material in each matching question
Special Guidelines for Constructing Completion and Short-Answer Items In completion items, avoid overly mutilated statements. Example: Poor: The ….. is obtained by dividing the …. by the …. Better: The ratio IQ is obtained by dividing the …. by the … Prepare a scoring key that contains all anticipated acceptable answers. Avoid grammatical and other clues to the correct answer Example: Poor: The authors of the first verbal intelligence test were … Better: Who prepared the first verbal intelligence test?
