SLCI Scores- ten brave Utah Valley Conference Participants

1. SLCI Scores- ten brave Utah Valley Conference Participants

2. Metacognitive Tools For Expert Learners * Evaluated through rubrics

3. Unifying Concepts - Metadiscipline of Science • Science explains physical phenomena based upon testable information about the physical world. • In modern life, science literacy is important to both personal and collective decisions that involve science content and reasoning. • Doubt plays necessary roles in advancing science. • Scientists use evidence-based reasoning to select which among several competing working hypotheses best explains a physical phenomenon. • A theory in science is a unifying explanation for observations that result from testing several hypotheses. • Peer review generally leads to better understanding of physical phenomena than can the unquestioned conclusions of involved investigators. • Science can test certain kinds of hypotheses through controlled experiments. • All science rests on fundamental assumptions about the physical world. • Science differs from technology. • Scientific knowledge is discovered, and some discoveries require an important history. • Science employs modeling as a method for understanding the physical world. • Scientific knowledge imparts power that must be used ethically.

4. Metadisciplinary Concepts Converted to Outcomes STUDENTS WILL BE ABLE TO… • Define the domain of science and determine whether a statement constitutes a hypothesis that can be resolved within that domain. • Describe through example how science literacy is important in everyday life to an educated person. • Explain why the attribute of doubt has value in science. • Explain how scientists select which among several competing working hypotheses best explains a physical phenomenon. • Explain how "theory" as used and understood in science differs from "theory "as commonly used and understood by the general public. • Explain why peer review generally improves our quality of knowing within science. • Explain how science employs the method of reproducible experiments to understand and explain the physical world. • Articulate how science’s way of knowing rests on some assumptions. • Distinguish between science and technology by examples of how these are different frameworks of reasoning. • Cite a single major theory from one of the science disciplines and explain its historical development. • Explain and provide an example of how modeling is used in science. • Explain why ethical decision-making becomes increasingly important to a society as it becomes increasingly advanced in science.

5. DRAFT: Metadisciplinary Outcomes for the Arts Students should be able to… • Explain the significance of creative expression and art to the human experience. • Discern objective vs. subjective scholarship, criticism and analysis of the arts. • Articulate in his/her own words a definition for what constitutes the arts. • Communicate ideas and emotions through the practice and study of the arts. • Recognize and value creative expression from various cultural and historical perspectives. • Explain in his/her own words reasons why critical thinking and problem solving have value in the arts. • Describe, using at least two specific examples, how art literacy is important in everyday life.

6. A realization that should arise from becoming educated: every metadiscipline offers a valuable way of knowing • Articulate concepts • Rewrite concepts as outcomes • Develop assessment instruments • Teach the outcomes • Use assessment results to try improve learning

7. OK…Can we assess this stuff?

8. The Instrument • For each outcome • Construct several concept inventory items. • Use established methods for drafting items that have been developed in other concept inventories. • In addition… • Test reasoning, not factual knowledge. • Administer the inventory under the conditions in which a citizen will use common information.

9. …initial instrument constructed 2008-2010 through the collegial efforts of • Edward Nuhfer, Faculty Development & Geology, Channel Islands • Jerry Clifford, Physics, Channel Islands • Christopher Cogan, Environmental Sciences & Resource Management, Channel Islands • Anya Goodman, Biochemistry, San Luis Obispo • Carl Kloock, Biology, Bakersfield • Beth Stoeckly, Physics, Channel Islands • Christopher Wheeler, Geology, Channel Islands • Gregory Wood, Physics, Channel Islands • Natalie Zayas, Science Education & Environmental Sciences, Monterey Bay

10. Science Literacy Concept Inventory • Incorporates 25 validated items that map to the twelve concepts • Reliability of .85 • Tested on over 8000 students in about 30 institutions

11. Outcome. Student can Define the domain of science and determine whether a statement constitutes a hypothesis that can be resolved within that domain. Concept Some Misconceptions Science explains physical phenomena based upon testable information about the physical world. Science is on a mission to refute religion; scientists study the paranormal; untestable statements are like scientific hypotheses.

12. Which of the following statements presents a hypothesis that science can now easily resolve? A. Warts can be cured by holding quartz crystals on them daily for a week. B. A classmate sitting in the room can see the auras of other students. C. Radio City Music Hall in New York is haunted by several spirits. D. People with chronic illnesses have them as punishment for past misdeeds. 0, 1, or 2?

13. Which of the following statements presents a hypothesis that science can now easily resolve? A. Warts can be cured by holding quartz crystals on them daily for a week. B. A classmate sitting in the room can see the auras of other students. C. Radio City Music Hall in New York is haunted by several spirits. D. People with chronic illnesses have them as punishment for past misdeeds.

14. What did we learn that made this so interesting? • Starting with…Do experts (professors) outscore novices (students)?

15. YES! They do! Also, students on average do come to us with some science literacy: zero literacy = about 25% (random guessing).

16. Does knowing “stuff” advantage a person?

17. Professors in Metadisciplines by Mean Score %

18. We can learn about our students’ science literacy

19. Our current GE science courses don’t produce science literacy…

20. Development seems more apparent long term through ranks…

21. We can learn much about our students

22. Gender-equality (n = 12,120)

23. Science Major or Not: 73% vs 67%

24. Science Major or Not: difference generally increases through undergraduate ranks Freshmen Sophomores Juniors Seniors

25. First Generation Students: 67% vs 72%

26. English as a First Language: 63% vs 71%

27. Cumulative first-generation and ESL:9 point disadvantage • 64% versus 73% for those who have English as a first language AND are not first-generation students. • That may be another way of showing that economic status and struggles of students and their families infuse differences.

28. SLCI by Institutions with Other Measures As Eric Gaze and the Quantitative Literacy test researchers have noted, these little literacy tests may be powerful predictors of student success.

29. Institutions

30. Given that our GE science courses don’t produce much increase in reasoning of science literacy…

31. What are some ways that we can convey citizen competency in science literacy to our students?

32. Metacurriculum for Metacognition

33. Metacognitive Tools For Expert Learners * Evaluated through rubrics

34. Knowledge Surveys • Example Survey Item • I can outline Piaget’s four main stages of cognitive development, and comment on how children’s thinking changes during these four stages. Response Options 2 = I have current ability to address this challenge very well. 1 = I can now only partially address the challenge. 0 = I currently have insufficient skill/knowledge to address the challenge.

35. Knowledge Surveys • Metacognition • If I think I can do this, how well can I do this? • Can I look at the items and start to distinguish which emphasize knowledge, skills or reasoning? • Am I able to do this at the targeted level of understanding? • For construction, consult the tutorials at http://elixr.merlot.org/assessment-evaluation/knowledge-surveys

36. Correlations between performance and several metacognitive self-assessments (n = 1011)

37. Stages of Intellectual Development • Level 1 & 2 thinkers believe that all problems have right and wrong answers, that all answers can be furnished by authority (usually the teacher), and that ambiguity is a needless nuisance that obstructs getting at right answers. • Level 3 thinkers realize that authority is fallible and doesn't have all answers. They respond by concluding that all opinions are equally valid, that arguments are just about proponents thinking differently. Evidence doesn't change this. • Level 4 thinkers recognize that not all challenges have right or wrong answers, but they do not yet recognize frameworks through which to resolve how evidence best supports one among several competing arguments. • Level 5 thinkers can use evidence and begin to accept that evaluations that lead to best solutions can be relative to the context in which a problem occurs. • Level 6 thinkers appreciate ambiguity as a legitimate quality of many issues, can use evidence well to explore alternative viewpoints. They recognize that the most reasonable answers often depend upon context and value systems. • Levels 7, 8 and 9 thinkers incorporate metacognitive reflection in their reasoning, and they increasingly perceive how their personal values act alongside context and evidence to influence chosen decisions and actions. William J. Perry Jr. (1968) Forms of Intellectual and Ethical Development in the College Years

38. “Critical Thinking” and Metadisciplinary Reasoning * From Brookfield, 2012 **Modified from Carole Huston, San Diego University

39. What might a rubric look like… • To guide development of a sophisticated written teaching philosophy for a faculty member • To guide development of a sophisticated written learning philosophy… • For anyone?

40. Reflective Exercises & Learning Journals….

41. Strategic Reading • Expert Readers: • read with a purpose and in “extensive mode” • accomplished in use of prior knowledge • utilize a wide variety of strategies for monitoring and comprehension (e.g., prediction, integration, self-questioning, reflecting) • Novice Readers: • focus on decoding single words/phrases • fail to adjust for different texts/purposes • seldom use text-processing strategies Paris et al. (1996)

42. Reading and Reflecting • Reading Reflections: • Completed after each reading assignment • Short responses to a few questions • Submitted online before class • Credit awarded for “reflective” submissions • What is the main point of this reading? • What did you find surprising? Why? • What did you find confusing? Why?

43. Metacognitive Goals of Reflections • What is the main point of this reading? • Summarizing (Anderson & Thiede, 2008) • Keywords (Thiede et al., 2005) • What did you find surprising? Why? • Misconceptions (Bransford et al., 2000) • Affect (Winne & Hadwin, 1998; Pintrich and Zusho, 2007) • What did you find confusing? Why? • Monitoring, Evaluation, and Reflection • (Ertmer and Newby, 1996; Zimmerman, 2002)

44. Early Results Reading Reflections vs. Course Grades MACALESTER GEOLOGY Pearson = 0.842 p-value = <0.001 HAMLINE ECONOMICS Pearson = 0.779 p-value = <0.001

45. Classroom Activities Effect Size = 1.35 (Large) Clicker Questions Effect Size = 1.08 (Large)

47. Exam “Wrapper” • Self Evaluation • Preparation Strategies • Performance Analysis • Planning Achacoso (2004) Lovett (2008)

48. Backwards Design Worth being familiar with Important to know and do Highest Priority Wiggins and McTighe (1998)

49. We Must Act Beyond the Scale of Courses

50. A Guide for Learning to Learn… • Purpose of Education • Levels of Thinking • Affective Domain • Significant Learning • Meanings of Learning • Research on The Brain • Intellectual Development • Critical Thinking & Reasoning • Metacognition • Learning Styles • Behavioral Dimensions of Grades