The Peer Review Process in graduate level online coursework. “None of us is as smart as all of us”

1. The Peer Review Process in graduate level online coursework. “None of us is as smart as all of us” Tim Molseed, Ed. D. Black Hills State University, South Dakota EXAMINING THREADED DISCUSSIONS IN ONLINE GRADUATE COURSES

2. Examining Threaded Discussion • Purpose of study: • To better facilitate engagement as a learning community in supporting, motivating and encouraging each other in the process of developing capstone portfolios. • To better facilitate graduate students to engage as a community of learners in analyzing, critiquing, and making suggestions regarding their capstone portfolios.

3. Examining Threaded Discussion • Study construction: • In threaded discussion, examination of social/emotional responses defined as either a general opinion/evaluative or anecdotal/reflective statement, phrase, or sentence that stated the reviewers thoughts, feelings, related experiences, judgments and/or ideas concerning the writers’ submission but with no reference to specific content or style issues. E.g. “I think you did really well”, “Your submission reminded me of when I taught 3rd grade”. • In threaded discussion, examining task responses defined as either content (the adding, deleting, clarifying of ideas, theories, experiences and/or research) or style (arrangement of content, tables, context and/or issues of grammar, punctuation and format).

4. Examining Threaded Discussion • Threaded Discussion Peer Review Response Type sub-categories: Social/Emotional Task Opinion/ Anecdotal/ Content Style Evaluative Reflective

5. Examining Threaded Discussion Hypotheses: H1 - The number of social/emotional peer review response types will differ significantly from the task oriented peer review response types. H2 – Of the task oriented response types that deal specifically with issues of content will differ significantly from those dealing with issues of style.

6. Examining Threaded Discussion The Study Sample & Process: * Two sections of graduate master capstone classes with total of 36 graduate students resulting in 569 available response comments, sentences, or phrases. * Using researcher constructed instrument (developed in consultation with trained psychologist faculty member), responses were classified into one of the four response types. (Instrument was piloted and juried by six graduate faculty and found to have a 93% inter-rater reliability prior to and during the course of the study.)

7. Examining Threaded Discussion Findings:

8. Examining Threaded Discussion Findings –

9. Examining Threaded Discussion Findings: * Results of chi-square analysis of the social/emotional and task responses (n = 529), indicated a significant difference, [X2 (1, 529) = 4.54, p = 0.033, effect size .089] * A significant difference was demonstrated between the task sub categories of content and style comments (n = 289), [X2 (1, 289) = 26.19, p= 0.0001, effect size of .31]

10. Examining Threaded Discussion Results: H1 - The number of social/emotional peer review response types will differ significantly from the task oriented peer review response types. Hypothesis accepted. A statistically significant difference presented with a higher number of task vs. social/emotional response types though noting that effect size of .089 would suggest a weak relationship

11. Examining Threaded Discussion Results: H2 – Of the task oriented response types that deal specifically with issues of content will differ significantly from those dealing with issues of style. Hypothesis accepted. A statistically significant difference did present with a much higher number of style responses vs. content though noting that an effect size of .31 would only suggest a medium to strong relationship.

12. Examining Threaded Discussion Discussion: *The higher number of task vs. social/emotional type responses may reflect a maturity level of students in the online cohort having been together for two years. *The significantly higher number of style related task responses would suggest students feeling more confident addressing issues of grammar, punctuation, layout, and overall writing construction other than issues involved with content issues of adding, deleting, clarifying of ideas, theories, experiences and/or research.

13. Examining Threaded Discussion Suggestions for instruction: *Need for the intentional building of social/emotional interactions balanced with group maturity and history. *Defining “meaningful responses” to engage students in task comments (e.g. students adding to the thoughts of the discussion, and/or extend the discussion building on the other points, clarifying and/or summarizing the posting, and/or present alternative ideas, and/or ask further questions that might emerge from the thinking of others). *Modeling of expectations using both public and private forums.