1 / 41

Three Colleges Collaborate to Assess “Study Abroad” – Influential Factors, Experiences and Outcomes

Three Colleges Collaborate to Assess “Study Abroad” – Influential Factors, Experiences and Outcomes. Christine Brooks Cote, Bowdoin College James C. Fergerson, Bates College Mark Freeman, Colby College North East Association for Institutional Research Portsmouth, NH November 2004.

lawson
Download Presentation

Three Colleges Collaborate to Assess “Study Abroad” – Influential Factors, Experiences and Outcomes

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Three Colleges Collaborate to Assess “Study Abroad” – Influential Factors, Experiences and Outcomes Christine Brooks Cote, Bowdoin College James C. Fergerson, Bates College Mark Freeman, Colby College North East Association for Institutional Research Portsmouth, NH November 2004

  2. Off-Campus Study Survey Participants Bates Bowdoin Colby Amherst Smith

  3. Project Development • Genuine interest in understanding the impact upon students of a semester or a year spent abroad • A need to evaluate the Colby-Bates-Bowdoin Off-Campus Study Programs (CBB) in light of the goals presented to the Mellon Foundation • Seized the opportunity to use the instrument in a collaborative assessment project supported by NEASC and funded by the Mellon Foundation

  4. With Support From: “Motivating Inquiry Regarding Teaching and Learning at Critical Transition Points in Liberal Education”

  5. “Motivating Inquiry Regarding Teaching and Learning at Critical Transition Points in Liberal Education”20 Liberal Arts Colleges Participated • First Year Seminar study • Senior/Capstone Experience study • Off-Campus Study survey • “Faculty Fellows” interviews of students about focusing on an area of study Institutions could opt in or out of each study

  6. Comparing Study Away Programs

  7. Survey Development • Based upon a survey designed at Bowdoin • Survey was reviewed and revised by the participating institutions (IR offices, off-campus study offices, faculty, deans, NEASC) • Flexible administration: Common “core” of questions with some local questions; language was localized • Colby/Bates/Bowdoin also used the survey for a final evaluation its CBB Consortium programs in London, Cape Town, and Quito

  8. Survey Administration • Bowdoin: Done in conjunction with senior exit interviews • Bates: Stand-alone Web survey • Colby: Web survey in conjunction with Senior Survey • HTML code shared, but some questions and wording were “localized”; data collected locally • Institutions sent data to Colby, for processing/clean-up; SPSS file returned

  9. Bates: “Calendar Wars” & “Working the Web” Paper announcement with 5 e-mail reminders . Survey appears as “to do” link in student’s personal academic affairs account until submitted. L.L. Bean gift certificates raffled as “incentives”.

  10. Survey Components • Students’ motivations for off-campus study choices • Preparation and advising for study away experience • Student experiences while away • Perceptions of the impact on academic and personal growth • Senior academic activities that built upon the study abroad experience • Several “free-response” questions

  11. Major Survey Components Students’ motivations for off-campus study choices Very strong influence No influence on choices on choices • My desire to learn or become more fluent in the language of this country or region of the world • My desire to study in a country or region of the world that is very different from the one I have lived in most of my life • My desire to have a lighter academic load than I would normally have at [Bates or Bowdoin or Colby]

  12. Motivations for Study Away (All Institutions) * Not asked by Bowdoin ** “Academic Advisors or Professors” at Bowdoin

  13. Major Survey Components Students’ experiences while studying away Strongly agree Strongly disagree • While away, I was part of an intellectual community that was invigorating and thought-provoking. • Most of my time outside of class was spent with people who called the country or region of the world their home. • While away, I nearly always spoke a language other than English.

  14. Major Survey Components Students’ perception of impact of off-campus study experience Strongly agree Strongly disagree • The experiences I had while away caused me to change my understanding of how people around the world view the United States. • My understanding of the culture, country or region of the world I was in changed significantly by my living there for a time. • The experiences I had while away taught me a great deal about gender issues facing people in other parts of the world. • Returning to [Bates or Bowdoin or Colby], I engaged in research or independent study that directly built upon the work I did while away.

  15. Key Experiences/Outcomes Questions • Did students gain the skills that are desirable in a liberal arts context? • How did outcomes vary by type of student, field of study, or by institution? • Did study away change personal, academic, and career goals? • Did students link study away experiences to other academic work? • How do they compare with those who didn’t study away?

  16. Major Study Away Outcomes Overview

  17. Other Study Away Outcomes Overview

  18. Study Away Reasons - Cluster Analysis • Reasons for studying abroad items are the best candidates for a priori or “causal” variables. • If Reasons presuppose certain outcomes then a model focusing on them has diagnostic value in practice. • Cluster analysis into “types” to simply interpretation. • A fourcluster solution fit the five-school data best

  19. Cluster names are subjective interpretations of item response patterns Cluster characteristics Selected Cluster Solution

  20. Does Cluster Membership Predict OCS Experiences? Ay-uh… • Well-Roundedsand Experientialistsstrong on cultural immersion, language learning experiences; weakest of the four clusters on “adjusting to academic structure” • Narrow Academics stand out on “keeping in touch with professors back home” and “course learning”, “part of intellectual community”; weakest on cultural immersion experiences • Club Medsweak pretty much everywhere, especially traditional academics – but more cultural immersion than Narrow Academics

  21. Does Cluster Membership Predict OCS Outcomes? Ay-uh… • Well-Roundedsstrongest on most outcomes, virtually across the board – “academic integration” as well as “personal growth” and “cross-cultural understanding” outcomes • Narrow Academics weakest on “personal growth” and “cross-cultural understanding” – almost as strong as Well-Roundeds in “integration with post-OCS academics” • Club Medsbring up the rear virtually across the board, especially on post-OCS academic integration. They do, however, report that that the experience was “rewarding”

  22. Comparing Students By College By Cluster

  23. Academic Division and Clusters - Bates

  24. Location of Study – By Cluster (Bates)

  25. Cumulative Average GPAs – By Cluster (Bates)

  26. Changes in Goals – Bates

  27. Core Changes – By Division

  28. Linking OCS Survey Data With Senior Survey Outcomes • Responses were linked with HEDS senior survey data (Colby only) – can leverage the OCS survey results against other institution-wide outcomes • And can compare outcomes with non-study abroad graduates

  29. Regression Analysis: Are Other Factors Linked To Outcomes? • So, “reasons for studying abroad” account for a healthy portion of the variance in outcomes. • What of other factors? • Sex • Major • Country of study • Does school “uniquely” account for any variation in outcomes? • If so, may help

  30. Regression analysis

  31. Survey Data Useful for CBB Program Evaluation • CBB Off-Campus Study Programs were funded by the Mellon Foundation • Two critical goals of the programs were to generally enhance the academic quality of study abroad experiences and improve the integration of study abroad experiences with the educational program at the home college.

  32. Survey Data Useful for CBB Program Evaluation Motivations The academic opportunities within the program or country support my major CBB 61% All Others 47% My desire to have a lighter academic load than I would normally have at [Bates or Bowdoin or Colby] CBB 5% All Others 13%

  33. Survey Data Useful for CBB Program Evaluation Experiences and Perception of Impact While away, I was part of an intellectual community that was invigorating and thought-provoking. CBB 55% All Others 43% I learned a great deal from the courses I took while away. CBB 81% All Others 61%

  34. Personal Growth / Reflection “Study abroad gave me the opportunity to have my own thoughts; it provided space from academics that was filled by reflection. That's very important in college because students are confronted by so much information that is difficult to synthesize. With study abroad, there is time and space to reflect on Bates, on the USA, on yourself.” -- Anthropology major, Ghana It is difficult to put in words. I learned so much - and it is still sinking in after a year. I learned to interview powerful political figures in a different language. I learned to live - and enjoy living in a home without indoor plumbing. I even learned a lot from being mugged, getting parasites, and getting sick - all of it opened me up to a different reality from life at Bates. -- Political Science major, Madagascar

  35. Connections to Research “Broadly speaking, it was a lesson on life, what it means to be human, and how to interact and communicate cross-culturally. I worked so hard with subsistence campesino potato farmers to produce our own food or kill our own animals, and that has made me much more conscious of the flow of food globally. At Bates I took a Social Justice Spanish class, I was rewarded an Otis Fellowship to return to Bolivia, and I understood my thesis with greater depth. The mix of Bates and Off Campus help me confront my own assumptions and romanticisms about the life in Bolivia, Vermont, and so on. --Environmental Studies Major, Bolivia

  36. Senior Thesis

  37. Senior Thesis / Research Links “My senior thesis related directly to my experience in Tasmania. The first week of my stay there was the most influential in my thesis. I returned to Tasmania the following summer to do research for my thesis, and having already lived there, made connections and gotten to know the culture and area was very helpful. I could hit the ground running.” -- Environmental Studies major “My thesis was not about Ecuador, but I extensively worked with Spanish texts, so that was important. Although I did not like Ecuador very much, the experience provided me with confidence to apply for a Phillips Fellowship, which allowed me to do pre-thesis research in Cuba.” -- History major

  38. Uses of Collaborative Assessment – Advantages • Labor-saving: Divides up workload and tasks • Survey design involved faculty and other administrators • Improved analysis by sharing technical expertise and discussing results • Promoted other data-sharing – Your First College Year results shared • Importance of comparative data – why are we different, and what does that mean?

  39. Uses of Collaborative Assessment -- Concerns • Different campus cultures and analytical approaches • Timing • Campus calendars / office work schedules • Assembling the right team(s) • Getting “Buy-in” to exchange data • FERPA concerns (exchanging non-survey outcomes data) • Integrating data from different collection methods

  40. Policy Discussions and Faculty Reactions • “This is really interesting!” • At Bowdoin, the faculty Off-Campus Study Committee has reviewed the entire report, including qualitative data from one-on-one and group interviews. • Data from the report will be used in our final CBB report to the Mellon foundation. • Important assessment discussions would focus on reasons behind college differences...yet to happen.

  41. Questions? • Contacts: • Christine Brooks Cote, ccote@bowdoin.edu • Jim Fergerson, jfergers@bates.edu • Mark Freeman, mfreeman@colby.edu

More Related