1 / 25

Measuring Change: Evaluation Plan for Middle Ground Project

Measuring Change: Evaluation Plan for Middle Ground Project. Educational Outfitters (2011). Educational Outfitters. Shanna Futral Evaluation Field Director Todd Braeger Senior Evaluator and Analyst. Who We Are & What We Do. Why do evaluate?. Future US DOE Funding.

armand-hudson
Download Presentation

Measuring Change: Evaluation Plan for Middle Ground Project

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Measuring Change: Evaluation Plan forMiddle Ground Project Educational Outfitters (2011)

  2. Educational Outfitters Shanna Futral Evaluation Field Director Todd Braeger Senior Evaluator and Analyst Who We Are & What We Do

  3. Why do evaluate?

  4. Future US DOE Funding Results from grant-funded projects guide the development of, and allocation to, existing and new US DOE discretionary grants (including the Presidential Academies grant that funds the Middle Ground Project).

  5. Government Performance and Results Act (GPRA) & Continuation Award RequirementsIn return for receiving a PA grant, US DOE requires rigorous evaluation.Therefore, your completion of the assessments is required.

  6. Continuous Improvement Cycle YOU are Middle Ground! Teachers’ data will aid Middle Ground staff in improving project offerings: training topics, presenters, materials, support activities, etc.

  7. All Submitted Data is Confidential No one other than the Project Staff and Evaluation Team will ever see your individual data.

  8. What are we measuring? • Attendance (required) • Expectations and Satisfaction with Project Trainings and Materials • Knowledge of History and Civics (passing score and 20% gain each year) • Instructional Approach (20% gain each year)

  9. Teaching History & Civics Experts agree that thinking skills must be integrated into the teaching of history and civics, not developed as a separate skill. Last 30 years (big push in last 10), “historical thinking skills” or “Habits of the Mind” have been popping up in curriculum standards and assessments.

  10. “History is about facts in much the same way that reading is about the alphabet: Facts (and letters) are essential building blocks; without them you cannot do history (or read). But, just as reading necessitates looking at how the letters and words stand in relation to one another (so that you see the difference between a horse chestnut and a chestnut horse), thinking historically requires going beyond chronology or chronicle and looking at the relations that the facts bear to one another.” - Spoehr & Spoehr, 1994, p. 72

  11. For federal reports:All participating teachers’ data are lumped together for analysis and reporting purposes. Project-Level Changes notIndividual Changes

  12. Measuring Change Traditional MCQs • Pre/Baseline • Post Guided Self-Assessments • Pre/Baseline • Post: Then/Now

  13. Content • Primary & Secondary Sources Quiz (traditional pre/post) • Multiple Choice Answer questions (traditional pre/post) • Essay (guided self-assessment) • Core Content Statements (guided self-assessment) • Completed by Teacher

  14. Essay Instructions Understanding and Responsiveness Analysis and Reasoning Supporting Evidence Sequencing (5 paragraph format)

  15. Essay Instructions PRE • Type your essay in Word. Save to your hard-drive. Then copy and paste it into the online system. • You will need your original essay on hand to complete the post-assessment. • Do not spend too much time on this!

  16. Essay Instructions THEN/NOW Do you feel that your original essay needs improvement OR does it well reflect how you would currently answer the essay question? (Check the ONE statement below that best describes how you feel.) (1) I AM satisfied with my original essay and changing or modifying it would NOT demonstrate an improvement. Explain why. (2) I am NOT satisfied with my original essay and changing or modifying it WOULD demonstrate an improvement in my understanding. Please revise your essay.

  17. Content Statements’ Rating Scales PRE and THEN/NOW: • To what extent do you teach this content concept? • What constraints, if any, influence the level to which you teach this content concept? • How comfortable are you with teaching this content concept? Only NOW: • Which statement below best describes what your current (NOW) lesson plans reflect or include related to this content concept? • Which statements below are true about your current (NOW) understanding of this content concept and, if you teach it, your integration of it into your lessons?

  18. Note about Rating Scales Numbers are relative, not points. For example: To what extent do you teach this content concept? (Check ONE.) 3=I dedicated enough time to this material to give my students a good understanding of it. 2=I briefly discussed/covered it, but more as a side note or point of interest. 1=I am planning to teach it to my students. 0=I have not taught it to my students.

  19. Instructional Approach • Guided self-assessment completed by Teacher • Guided assessment completed by Instructional Specialist/Mentor for each Teacher

  20. Instructional Statements • Specific, observable behavior(s) the Teacher engages in as part of his/her regular classroom instruction. • For example:Student’s Background: I consider each student’s educational, cultural, and community background knowledge, skills, experiences, and perceptions. Examples include: Each student’s background knowledge and skill-level; potential misconceptions and bias of the subject matter; and their ability to use technology.

  21. Instructional Statements’ Rating Scales PRE and THEN/NOW: • The level of the teacher's skills and knowledge to effectively use this strategy. • The teacher's level of classroom implementation of this strategy.

  22. Schedule of Assessments COHORT BASELINE (Now) (Pre) Participant Information Self-Assessment of Instructional Approach (“Skills Assessment”)Content and Thinking Assessment (“Content Assessment” )END-OF-SUMMER (Post: Then/Now)Content and Thinking Assessment End-of-Course Follow-Up QuestionnaireAFTER EACH TRAINING SESSIONTeacher Expectation & Satisfaction Survey (TESS)END-OF-YEAR (January 2012) (Post: Then/Now)Content and Thinking Assessment End-of-Course Follow-Up QuestionnaireSkills Assessment End-of-Course Follow-Up Questionnaire Mentor’s Log & Assessment of Teacher’s Instructional Approach

  23. Instructions for Completing Online Assessments • Summer URLs are listed on the handout. • Throughout the year, some URLs will be emailed directly to you from grantsherpa@yahoo.com. • If emailed: URL is uniquely tied to each email address. Do not forward. • If you experience difficulties accessing the online form, delete the “cookies” from your Internet browser. • Must be completed in one sitting. • Do not use any resources (i.e., books, websites, notes, peers). • Push “Done” at the bottom of each survey. A “Thank You” will pop up if it was submitted successfully.

  24. Assessment Window 60 to 120-minutes window to complete assessments when done on-site. OR 2-week window to complete each assessment when emailed to you. It is very important that you complete the assessment on time in order for your gains to be accurately and fully documented.

  25. Need Assistance? If you have technical difficulties, email Shanna: grantsherpa@yahoo.com. Type “MGP Evaluation Question” as the email subject line.

More Related