1 / 21

USING THE CLA TO INFORM CAMPUS PLANNING

USING THE CLA TO INFORM CAMPUS PLANNING. Anne L. Hafner Campus WASC Faculty Coordinator Winter 2008. BACKGROUND. Collegiate Learning Assessment ( CLA) created by Council on Aid to Education, non-profit in NY

myron
Download Presentation

USING THE CLA TO INFORM CAMPUS PLANNING

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. USING THE CLA TO INFORM CAMPUS PLANNING Anne L. Hafner Campus WASC Faculty Coordinator Winter 2008

  2. BACKGROUND • Collegiate Learning Assessment ( CLA) created by Council on Aid to Education, non-profit in NY • Performance assessment of critical thinking, analytic reasoning and written communication (see example) • Assesses “value added” by a college • Value added: standardized gain score that controls for students’ academic abilities(and for selectivity and SES)

  3. CLA MEASURES I • Analytic writing task: Make an argument: “in our time, specialists of all kinds are highly overrated. We need more generalists - people who ca provide broad perspectives.” • Critique an argument

  4. CLA Measures - Performance Task • Provide students with a real world scenario. Students have 90 minutes to advise the mayor on crime reduction strategies & evaluate potential policies: a.Invest in a drug treatment program b. Put more police on the street • Students provided w document library including memos, statistics, data tables, news articles, etc.

  5. Performance Task • Students are required to use an integrated set of critical thinking, analytical reasoning, problem solving, and written communication skills

  6. DETAILS I • Common models: cross sectional (freshman and senior cohorts) or longitudinal (follow same kids) • Students’ SAT scores are collected to enable comparison of student’s “expected” CLA score vs. student’s actual CLA score

  7. DETAILS II • Both freshmen & seniors are tested. After seniors were tested, a gain score from freshman to senior class can be estimated. This informs the campus how much “value” was added by attending CSULA • Because sample of 100 drawn may not be representative of the class, CAE makes an adjustment so that a school’s “actual” CLA score is compared to its “expected” CLA score (based on students’ SATs)

  8. INTERPRETING DIFFERENCES • Differences between actual and expected scores are reported in 2 ways: by points on CLA scale and by standard errors in terms of 5 performance levels (well below expected, below expected, at expected, above expected, and well above expected) At expected is between -1 and +1 standard errors from expected.

  9. CSULA Fall 06 Administration • Fall 2006 - 110 students tested • Problems with getting students to participate, paid incentives • One half of students took performance task, one half took writing task • CAE sent report back with some interpretation, also sent data set

  10. ANALYSIS: HOW DID OUR FRESHMEN SCORE? • After taking into account their abilities (SAT), our freshmen scored 1.4 s.e. higher thanexpected overall or above expected. • Students performed at expected on performance task and above expected on writing tasks. • CLA sample scored higher than freshman population on SAT and ELM, same on EPT

  11. How CSULA Freshmen Did • Our college performed in the middle of the group of all schools nationwide that participated in CLA in 2006 (5th -6th decile)

  12. Correlations • With dataset, correlations were run • CLA scale scores moderately to highly intercorrelated • Scale scores correlated moderately with EPT, SAT, HSGPA • Overall CLA performance level did not have any positive correlations other than with CLA scale scores

  13. CONCLUSIONS I • CLA sample not equivalent to 2006 freshman class (CAE: not a problem) • CLA sample’s demographics similar to campus demographics • CSULA sample scored higher than expected on tasks • Overall, CLA performance level was not highly correlated with other tests

  14. CONCLUSIONS II • Students with English as primary language do not score significantly higher than non-English speakers on CLA • CLA appears to be measuring something different from knowledge & skills measured by the SAT, EPT, ELM and grades

  15. FINDINGS Summer 2007 • In summer 07, we received our senior findings and “gain” score • Seniors scored “at expected” • Gain was 55 pts (less than expected) Campus is at 4th decile (between 30th-40th percentile nationwide) • CAE can examine if improvement in average student performance between entry and graduation is in line with similar gains of comparable students at other colleges

  16. FINDINGS II • Seniors did less well than freshmen on performance tasks, and on critique an argument tasks. • CSULA’s 6 year graduation rate was “at expected’ 32%

  17. MAJOR PROBLEMS • Staff time to recruit students and administer the test • High cost of enrollment incentives • Time required to take test (90 minutes) • Cost to take the test is inexpensive ($20)

  18. STRENGTHS • Value added is major focus, rather than absolute performance • CLA is well developed and assesses two key skills: critical thinking/reasoning, and writing skills • Strong psychometric properties, valid and reliable, standardized gain scores enable comparisons • Can work with small sample

  19. CRITICISMS • Issue of junior transfers not dealt with in models; different population may need to be also assessed • Uncertainty about validity (consequential) and reliability • Heavy reliance on SAT score as measure of academic ability

  20. THE FUTURE • It may be preferable to embed CLA tasks into a freshman required course such as Intro to College or orientation • Senior level: could be embedded in capstone courses or made graduation requirement • CLA could be a useful measure of some GE outcomes

  21. THE FUTURE II • Pressure on universities from US DOE for accountability led to Voluntary System of Accountability (VSA) • 2007: CSU Chancellor mandated that all campuses use CLA and report annually for the VSA • CAE is working to create performance standards for CLA (e.g “Proficient” level); could be useful to colleges • CAE is working with CSU to deal with issue of junior transfers

More Related