Does Technology Improve Student Learning?

1. Does Technology Improve Student Learning? Michelle Barton, Palomar College Darla Cooper, Oxnard College Kelvin Leeds, Santa Ana College Susan Longacre, City College of San Francisco John Whitmer, @ONE Project, Evergreen Valley College

2. Outline • Overview (John) • Researchers (Darla & Michelle) • The Effect of Tablet PCs on Learning and Collaboration (Kelvin) • Developing Critical Thinking in the Nursing Student Using Simulated Online Clinical Scenarios (Sue) • Q & A

3. Overview (1) • @ONE Project • Vision: evaluate effectiveness of technology in CCC classrooms • Goals of Scholar Program: • To create and share original research on the impact of instructional technology on student learning in California Community College classrooms • To support the improvement of faculty members’ research abilities, technology skills, and pedagogical approaches

4. Overview (2) • Faculty-designed research project (competitive application), using Classroom Action Research methodology • Ongoing program: 6 Scholars 04-05, 10 Scholars 05-06, 10 Scholars 06-07 • Research support: added to 05-06 cohort through Center for Student Success • Support team: Pedagogical Support, Research Support, Instructional Tech Support, Logistics • Program Elements: Summer Retreat, Winter Retreat, Spring Graduation (06-07 only), Virtual Group Meetings, individual support

5. 2005-06 @ONE Carnegie Scholars (1) • Gregory Beyrer - Distance Education and Media Services, Cosumnes River College"Online Student Success": Making a Difference • Carolyn Brown - Graphic Design, Foothill CollegeStudent engagement and outcome in online courses; what can we learn from face-to-face learning communities? • Cheryl Carter - Counseling, Diablo Valley CollegeNon-Traditional Student Participation in Synchronous Online Workshops • Bobby Hutchison - Business, Behavioral and Social Sciences, Modesto Junior CollegeThe Impact of Online Human Sexuality Education on Percpetions of Risk, Worry and Knowledge • Kelvin Leeds - Mathematics, Santa Ana CollegeTechnology: Fad or Fixture

6. 2005-06 @ONE Carnegie Scholars (2) • Patrick Len - Physical Sciences, Cuesta CollegeAnalyzing Individual and Cooperative Electronic Response Systems to Improve Student Learning and Attitudes in Astronomy • Susan Longacre, - Registered Nursing, City College of San FranciscoUtilizing Online Simulated Clinical Scenarios to Develop Critical Thinking in the Nursing Student • Scott Lukas - Anthropology/Sociology, Lake Tahoe CollegeAn Assessment of Reflexive Student Learning in Online Social Science Courses • Frances Rice-Farrand - Nursing, Los Angeles City CollegeEducation for Future • Denise Stiglich - Math and Science, Antelope Valley CollegeInteractive PowerPoint and Online Homework Programs in the Classroom

7. The Researchers The Odd Couple

8. Overview • What Did We Do? • How Did We Do It? • Why Did We Do It? • Would We Do It Again and What Would We Do Differently? • What Did the Scholars Study and How Did They Do It?

9. What Did We Do? • Worked with @ONE staff • Retreats • Worked with the scholars

10. Worked with @ONE Staff • Helped select the scholars • Met regularly to review program • Met regularly with scholars to get updates on their progress • Discussed drafts of monographs • Helped plan for next year

11. Retreats • Helped plan the retreats • Prepared and gave presentations at the retreats • Met individually with each scholar to review: • Research design and methodology (1st retreat) • Data analysis and interpretation (2nd retreat)

12. Worked with the Scholars • Helped refine and solidify research question, design and methodology • Reviewed and helped develop survey instruments, databases, and definition files • Helped with data analysis and interpretation • Reviewed monographs and provided feedback to scholars • Helped with data presentation in the monograph • On call for help

13. How Did We Do It? • TEAMWORK!! • Communication (Lots of it!) • Humor • Patience

14. Why Did We Do It? • To do research again • To learn about technology and its impact on learning in different disciplines • To do research that matters and directly impacts students

15. Would We Do It Again and What Would We Do Differently? • YES! YES! YES! • Lessons learned • Focus is key • Data storage and organization • Data presentation • Review instruments • Submission of written methodology

16. What Did the Scholars Study andHow Did They Do It? • Impact of collaborative use of “clickers” on student learning and attitudes in astronomy (Mixed design, pre-test post-test & two groups) • Comparison of student perceptions and experiences regarding participation in face-to-face and online discussion groups (Questionnaire and interview) • Impact of an online preparation course on student performance in online courses (Analysis of historical data and questionnaire)

17. What Did the Scholars Study andHow Did They Do It? • Participation and satisfaction of non-traditional students in synchronous online student services workshops (Questionnaire) • Student perceptions of learning using Tablet PCs (Questionnaire and interview) • Effect of online clinical scenarios on the development of critical thinking skills (Mixed design, pre-test post-test with experimental and comparison groups)

18. The Effect of Tablet PCs on Learning and Collaboration: A Study of Student Perceptions Kelvin Leeds Santa Ana College April 11, 2006

19. Background • Technology is changing the classroom setting • Keyboards are not conducive for Math and Science Type classes • Tablet PCs may be more suitable for Math and Science classes

20. Technology Uses • Tablet PC • Use Tablet PC as virtual paper to take notes • Synchroneyes • Software program that enables the instuctor to view student computer screens • Blackboard • Online portal used to post course materials • Whiteboard • Software program that allows students to form virtual groups

21. Research Questions • Do students think the use of Tablet PCs in the classroom enhances their learning when compared to traditional methods? • Do students think the use of Tablet PCs during class increases collaboration between them and their classmates? • Are students hesitant to share their classroom work since others in class may know whose work is whose?

22. Methodology • MESA Pre-Calculus Course at Santa Ana College, Fall 2005 Semester, 23 students • Developed two questionnaires: • Background Questionnaire • Student Perception Questionnaire • Follow up interviews

23. 11 questions designed to investigate students’ knowledge/experience with computers, 22 students Overall computer use Course work and computer use Computer access Background Questionnaire

24. Background Questionnaire Sample Question In general, how often do you use a computer during scheduled class time at this college? a.) I never use computers during class time b.) Less than half of the time c.) Half of the time d.) More than half of the time

25. Background Questionnaire Key Findings for Background Questionnaire • Overall Computer Use • Virtually all students use computers outside of class • Half of students use computers for more advanced tasks beyond word processing/email/Internet • Course Work and Computer Use • Most students use computers outside of class to complete course related materials • Half of students use computers in class to complete course related materials • Computer Access • Most students have access to a computer at home or work • Half of students use computers on campus, the other half have access to a more convenient computer

26. Student Perception Questionnaire • 18 Questions designed to investigate students’ perceptions about using Tablet PCs during class • Instructor's Notes • Students’ Notes • Homework • General Comments • Two Administrations, 14 students • Middle and end of the term

27. Student Perception Questionnaire Sample Question When the instructor uses Synchroneyes to present my notes on the screen in front of the class: a.) I feel very comfortable b.) I feel somewhat comfortable c.) I feel somewhat uncomfortable d.) I feel very uncomfortable e.) It does not bother me one way or the other f.) Not Applicable

28. Student Perception Questionnaire Key Findings for Instructor’s Notes • All students felt it was helpful to have instructors notes available on the Internet • Most students access these notes at least half of the time

29. Student Perception Questionnaire Key Findings for Students’ Notes • More than half of students felt using the Tablet PC to take notes was more helpful than using traditional pencil/paper • Most students feel comfortable when their notes are presented in front of the class using Synchroneyes • Most students think using Synchroneyes to present other students’ notes is helpful

30. Student Perception Questionnaire Key Findings for Homework • Most students think using Whiteboard to review homework is more helpful than traditional paper/pencil • Most students think using Whiteboard to review homework creates more collaboration than traditional paper/pencil

31. Student Perception Questionnaire Key Findings for General Comments • Most students think that using computers in non-computer related courses during regular class time is more helpful than not using computers in class • More than half of students think that using computers during class time improves their performance during class

32. Follow up Interviews • Students feel it is helpful to use Synchroneyes since they see more than one way to solve a problem • Some students do not take notes because this allows them to focus on what the instructor says, knowing they will have access to the instructor’s notes later

33. Overall Results • More than half of students felt that using computers in class improved their performance during class • More than half of students felt that they collaborate more using Whiteboard and the Tablet PC rather than traditional paper/pencil • Most students felt comfortable when their work was presented to class using Synchroneyes with the Tablet PC

34. Future Research • Do students think it is worth the time and effort that it takes to learn to use the Tablet PC? • Do students think that using the Tablet PCs in combination with blackboard helps to keep course information more organized? • Students have access and knowledge about computers, thus the background survey is not necessary

35. Implications • Although it takes time both to learn and use in the classroom, students think that using computers during class improves their classroom performance • Having access to a copy of the instructors notes online is helpful to students • Tablets PCs can be used to increase collaboration amongst students in other classes

36. Developing Critical Thinking in the Nursing Student Using Simulated Online Clinical Scenarios Sue Longacre, RN, MS Registered Nursing Department City College of San Francisco

37. Problem Statement “The new nurses can detect there is a problem; they may be able to determine that it is urgent, but then all they do is call the doctor. They cannot diagnose what is wrong, even in instances where the problem is as common as an oncoming stroke or heart attack.” Betty and Gordon Moore Foundation Focus Group Study, 2005

38. Critical Thinking Definition Critical thinking is a purposeful, results-oriented process that focuses on decision making, problem solving, and clinical judgment. Brunt (2005)

39. Critical Thinking Competencies Faccione’s (1990) Critical Thinking Competencies • Interpretation • Analysis • Evaluation • Inference • Explanation • Self-Regulation • .

40. Course Content • Maternal/Newborn Nursing • Simulated online clinical scenarios • Multi-media and text • Asynchronous learning, available via WebCT • Students completed written responses via WebCT • Assignments returned with faculty comments via WebCT

41. Research Hypothesis Students who complete the online clinical scenarios will demonstrate a larger increase in their post-test critical thinking skills over their pre-test critical thinking skills than those students who do not complete the online clinical scenarios.

42. Study Design X = Critical Thinking Assessment Instrument O = Simulated Online Clinical Scenarios Formal survey administered on the last day of class to experimental students to gather feedback on students’ perceptions of the online assignments.

43. Demographics by Research Group • Gender: predominantly female • Ethnicity: experimental group Asian/Filipino (68%), comparison group Caucasian (68%) • Median age 32 • Direct patient care experience: 5 experimental and 4 comparison students

44. Clinical Scenario AssignmentExample

45. Patient Assessment • The patient is a 25 year old Gravida 2 Para 1 @ 28 weeks gestation. • Her first pregnancy and delivery were normal. • She takes prenatal vitamins. • Today’s fundal height measurement is 28 centimeters. Sample Online Clinical Scenario slide using picture and text.

46. Pregnancy Weight Gain The patient’s BMI before pregnancy was 21. Her prepregnant weight was 59K (130#). Her weight today is 68K (150#) Sample Online Clinical Scenario slide using a graph and text.

47. Patient’s Question The patient is complaining of frequent ‘leg cramps’ of her calf area. The patient asks the nurse, “What can I do to stop this muscle cramping of my legs?” Sample Online Clinical Scenario slide with patient’s question.

48. Assignment You are the nurse caring for this patient today in the clinic. Based on the information provided; • How would you answer the patient’s question? • What would be pertinent nursing diagnoses, starting with the highest priority and why? (This may include risk for...) • Describe appropriate nursing actions and/or interventions for the patient. Describe your rationale for these actions and/or interventions. .

49. Critical Thinking Assessment Instrument • DEVELOPMENT • Standardized Critical Thinking Instruments not adequate for nursing • Instrument developed to measure critical thinking competencies in a nursing environment • Reviewed by @ONE staff and nursing faculty • Piloted on previous students

50. Critical Thinking Assessment Instrument • SCORING • Scored by two faculty blind to purpose of research project • Reviewers agreed with majority of scores • When scores differed, a 3rd faculty assigned final score, always agreed with one of previous scores