KNOWLEDGE KNOWS NO BOUNDARIES.

1. STEM Learning Communities End of the Year Report Sponsored by Title V Presented by: Oralia De los Reyes, Ph.D. Director of Title V Michele Hansen, Ph.D. External Evaluators from IUPUI KNOWLEDGE KNOWS NO BOUNDARIES.

2. PRESENTATION OVERVIEW • Project Goals • Program Results by Grant Objectives • 1) Professional Development • 2) Learning Communities • 3) Integration of Technology • 4) Intervention for Math Readiness • Conclusions and Implications • BUILDING INFRASTRUCTURE TO: • Offer Professional Development • Institutionalize three High Impact Educational Practices: Learning Communities, First Year Seminar and Service Learning Components • Establish a Culture of Evidence • Maximize use of Blackboard Outcomes System

3. PROJECT GOALS • Restructure curricula and academic support • to achieve improved SLO’s • Implement a culture of evidence that identifies and aligns best practices for SLO’s • Study the effects of redesigned curriculum and assessment measures in STEM learning communities • TARGET POPULATIONS • Students in STEM fields • Increase the number of students interested in STEM fields • Increase the success rate of students pursuing STEM fields • Faculty • Increase the number of faculty who integrate instructional innovations for improved SLO’s • Increase the use of assessment of SLO’s to guide curricula improvements

4. Title V Objectives • To centralize a comprehensive faculty development program to encourage faculty to integrate instructional course innovations for improved learning outcomes • To provide structured, integrated learning communities (LCs) for students pursuing STEM fields • To integrate the use of technology into course redesign and assessment of student learning outcomes to guide curricular improvements based on evidence.

5. Comprehensive Evaluation Plan Designed to Address The Following: • Has the centralized, comprehensive faculty development program encouraged faculty to integrate instructional course innovations for improved learning outcomes? (Related to Objective 1) • In what ways have the structured, integrated learning communities (LCs) for STEM students and intrusive support interventions made distinguishable impacts on student learning and academic success outcomes? (Related to Objective 2) • To what extent have faculty integrated the use of technology into course redesign and assessment of student learning outcomes to guide curricular improvements based on evidence? (Related to Objective 3).

6. Summative and Formative Evaluation: Annual Faculty Survey Student and Faculty Focus Groups Student Academic Achievement Retention and Graduation Rates Critical Thinking Assessment Test (CAT) Attitudes Toward Science Assessment Instrument (ATSI) National Survey of Student Engagement (NSSE) Course Embedded Assessment - Actual Student Work (papers and assignments designed to assess critical thinking and communication skills) Blackboard Outcomes Systems- Program & Course Level Assessment

7. PROFESSIONAL DEVELOPMENT

8. PROFESSIONAL DEVELOPMENT Source: Professional Development Database, compiled by Title V, Fall 2011.

9. PROFESSIONAL DEVELOPMENT Source: Professional Development Database, compiled by Title V, Fall 2011.

10. Annual Faculty Survey • The UTB/TSC Annual Faculty Survey was designed to assess: • Participation in professional development activities, • How faculty spend their time inside and outside the classroom, • Pedagogical strategies faculty are using to engage students, • Use of technology and assessment planning, • How faculty perceive institutional priorities.

11. Faculty Survey Design • The Foundations Of Excellence In The First College Year Project Faculty Survey, the Higher Education Research Institute Faculty Survey and the goals of the Title V evaluation plans were used as guides to develop the survey.

12. Annual Faculty Survey Results • The survey was administered to 615 Faculty members in spring 2011. • 277 responded and completed the survey. • The response rate was 45%.

13. Faculty Professional Development Participation

14. Overall Faculty Satisfaction With Professional Development at UTB/TSC

15. Using Assignments To Improve General Education Outcomes Note: N/A removed from analyses N=227

16. Faculty Who Participated in Title V Summer Institute were Significantly More Likely to Improve Student Learning through… • Redesigned existing course • Integrated assignments that require students to apply knowledge and skills learned in other courses • Assignments that assess students' critical thinking levels • Assignments that assess students' communication skills • Rubrics for assessing assignments • Based on independent samples test results p<.05 • Participating N=37 and Not Participating N=220

17. LEARNING COMMUNITIES

18. STUDENTS ENROLLED IN STEM LEARNING COMMUNITIES Source: Learning Communities Student Tracking Database, compiled by Title V, Fall 2011.

19. LEARNING COMMUNITY STUDENTSBY COLLEGE, N=544 CSMT CLA Source: Datatel Database, Management and Reporting Office, compiled by Title V, Fall 2011.

20. PERFORMANCE MEASURES: LC STUDENTS’ RETENTION BY COHORT FALL 2009 TO FALL 2011 UTB FT Freshman Fall ‘09 to Fall ‘10 Retention 55.4% Pilots

21. CONTROL AND COMPARISON GROUPS The CONTROL GROUP was drawn from a database provided by the office of Institutional Effectiveness and was defined by random selection of non LC students... with same classification taking same courses in the same semester The COMPARISON GROUP was drawn from 2009 FoE Freshmen Cohort, DFWI Grade Distribution Report, published by the office of Data Management and Reporting, Fall 2010

22. FALL 2010-SPRING 2011 LC STUDENTS’ DEMOGRAPHICS NOTE: Control Group was defined by Student Classification and Non LC students taking same courses during the same semester Source: Learning Communities Student Tracking Database, compiled by Title V, Fall 2011.

23. GRADE DISTRIBUTION: FALL 2010, n=526 8 pts. Comparison Group Source:Office of Data Management and Reporting Database, Fall 2011

24. GRADE DISTRIBUTION: FALL 2010, PRE-CALC, n=21 14 pts. 33 pts. Comparison Group Source:Office of Data Management and Reporting Database, Fall 2011

25. GRADE DISTRIBUTION: ENGL 1301-FALL 2010, n=152 10 pts. 20 pts. 6 pts. Comparison Group Source:Office of Data Management and Reporting Database, Fall 2011

26. GRADE DISTRIBUTION: FALL 2010 AND SPRING 2011 Comparison Group Source: DFWI Grade Distribution of FY2009 Freshmen Cohort (Fall 2008, Spring and Summer 2009) for FoE, Office of Data Management and Reporting, Spring 2010

27. HIGH PERFORMANCE LC’SFALL 2010-SPRING 2011: COLLEGE ALGEBRA Comparison Group Source: DFWI Grade Distribution of FY2009 Freshmen Cohort (Fall 2008, Spring and Summer 2009) for FoE, Office of Data Management and Reporting, Spring 2010

28. HIGH PERFORMANCE LC’SFALL 2010-SPRING 2011: CALCULUS I Comparison Group Source: DFWI Grade Distribution of FY2009 Freshmen Cohort (Fall 2008, Spring and Summer 2009) for FoE, Office of Data Management and Reporting, Spring 2010

29. HIGH PERFORMANCE LC’SFALL 2010-SPRING 2011: CHEMISTRY I 11 pts. Comparison Group Source: DFWI Grade Distribution of FY2009 Freshmen Cohort (Fall 2008, Spring and Summer 2009) for FoE, Office of Data Management and Reporting, Spring 2010

30. Classroom Community Scale (CCS) (Rovai, 2002) • The 20-item Scale measures sense of community in a learning environment. • Research results have suggested that the Classroom Community Scale (CCS) is a valid and reliable measure of classroom community and learning.

31. CCS Results: Fall 2011How Did you Hear About the Learning Communities N=154

32. CCS Results: Fall 2011 (Rank Ordered by Mean)1=Strongly Disagree – 5=Strongly Agree N=154

33. CCS Results Continued: Fall 2011 N=154

34. Learning Communities Student Survey • Designed to assess students’ satisfaction levels with their experiences in learning community courses. • Administered Fall 2010, Spring 2011, and Fall 2011 • Responses on Likert-Type Scale where 1=Very Little, to 5=Very Much • N = 321 Students

35. Learning Community Experience Satisfaction

36. LC Survey Top Rated Items N=312 to 315

37. LC Survey Results Continued… N=314

38. Learning Community Experiences • 58% Participated in a Campus Activity or Event Outside of Class • 43% Participated in an Activity or Event in the Community • 40% Participated in a Community Service or Volunteer Activity

39. The NSSE assesses the extent to which undergraduate students are involved in educational practices empirically linked to high levels of learning and development. Random sample of first-year students and seniors. Oversampled first-year students and administered to students in the LCs in the spring. Strength of NSSE is that it is aligned with many of the Title V learning objectives and allows for comparisons with internal peers (UTB/TSC students not enrolled in LCs), peer institutions, and the NSSE national sample. National Survey of Student Engagement (NSSE)

40. NSSE Items Significantly Different (p. < .05) N=54 to 165

41. NSSE Items Significantly Different (p. < .10) N=49 to 164

42. LC Student Focus Groups • A total of 9 students were recruited to voluntarily participate in an hour long semi-structured interview regarding learning communities. • The interview was tape-recorded and subsequently content analyzed. • The qualitative results were analyzed for content and coded into individual thematic theme categories.

43. Importance of Scorpion-tation/Orientation • In Scorpion-tation they had the different booths and the people... Looked pretty interesting… They explained to me like the different classes and having two difficult classes there would be offered tutoring -- I think that helped a lot.” • “I was taking both so it was already there when I was at Scorpion-tation because I know I needed to take Bio and History… And they were like OK this is a linked course you want to take it? And I was like – Yeah Sure!” • “Mine was through Orientation as well. My wife suggested – so she suggested I take the learning communities.”

44. High Praise to the Tutoring Support • “They (tutors) explained everything. They had patience.” • “They helped to introduce good study habits.” • “The tutors they really helped in helping us become individual(s) off of them. Like they kind of helped us to get our sea legs you could say.” • “Chem. and Pre-Cal. Their pretty hard classes, and even Bio, knowing that you have the option for tutors and knowing that there’s a set time for you to study…. I think that helps a lot.” • “At the time when you would go (tutoring) – they would always go over a chapter they would give you like an overview on every chapter. That was really helpful too. And they would always ask us like if you guys want during lecture -- write questions that you want to know-- bring it to the tutoring session -- and we’ll answer them there.” • “Linked actually makes it more easier since the tutoring is there. If you take them separately having like that time setup for studying makes it more difficult.”

45. Improving Critical Thinking • “I also liked the fact that I had to look at it through two different philosophies. I mean the philosophy of English and Biology. I felt that it helped with a deeper understanding. It just reinforced it.” •  “Seeing them spending more time with both of them (Chemistry and Math). You’re understanding more. You’re not just understanding Chemistry better your understanding Chemistry and Math.” •  “I don’t know if this one counts but the big thing that framed the beginning part of our semester –was - What is science? What do we consider science? And what is considered science? And we basically went through the scientific process as well which eventually led to us forming our own opinion of what science (is).”

46. Developing a Sense of Community • “For me I think for both classes we had the same classmates… and I know that’s a lot of fun because you form like a little community amongst each other. So once you get that confidence of like knowing everybody for both classes you feel more free to like ask more questions or ask each other questions that we don’t understand.” • “This is my first semester so like I didn’t know anybody and just being in the learning community and meeting new people and having them there with me - helping me… Instead of those classes that I didn’t have with other students that I know. I think it just helped me more.” • “Having the same students and actually going to the other classes with those students was extremely helpful. And I’m a pretty open person but in other classes you could see people shut down because they’re afraid…because they don’t know their peers. And so they’re afraid to ask questions.”

47. TECHNOLOGY INTEGRATION AND ASSESSMENT

48. INTEGRATION OF TECHNOLOGY Use of Technology Facts: 23 faculty are using tablet PC in their classrooms to grade papers and engage students 330 TurningPoint clickers are being used by 17 faculty in 29 sections COW for Mathematics LC’s and tutorials 10 sections Mobile Computers for faculty trainings 83 workshops Development of ESO’s and SLO’s Blackboard Outcomes templates Trained 101 faculty on Technology related topics A group of faculty, the “Select 40” were trained by Blackboard and piloted the Blackboard Content Management 9.1 Service Pak in Spring 2011 30 faculty got Laser Pointers/Podium remote controls

49. Annual Faculty Survey Results: Integration of Technology Note: N/A removed from analyses N=227

50. Annual Faculty Survey Results: Supporting a Culture of Evidence N=277, Note: N/A removed from analyses