Assessing Students Preparedness to Compete and

1. College of Technology and Computer Science Closing the LoopAssessing Students’ Use of Modern Engineering Tools Ed Howard and Evelyn BrownDepartment of Engineering Background ABET specifies a set of learning outcomes that every accredited engineering program must assess. One of these outcomes is the effective use of modern engineering tools. This learning outcome is assessed directly using work samples from selected courses and indirectly using the results of student surveys. Work samples may include homework or laboratory assignments, targeted exam questions, or project work. Student surveys include senior surveys and end of course surveys in which students assess their understanding of course objectives. Process Faculty members serve as outcome coordinators, with primary responsibility for annually evaluating assessment data for their assigned outcome. The learning outcome related to students’ ability to effectively use modern engineering tools is assessed via direct and indirect measures in a number of courses, as described in the table below. Result Reporting At the end of each academic year, each outcome is given an overall achievement rating by its outcome coordinator. The overall rating is determined by examining all related student work samples and survey questions. A sample outcome report from 2009-10 is provided below. Faculty meet to discuss the outcome reports, with particular attention focused on outcomes whose achievement rating is below 3. Continuous Improvement In 2007-08, this outcome received an overall achievement rating of 3 (out of 5). The faculty noted opportunities to strengthen two freshmen level courses (ENGR 1012 and ENGR 2050). Additional inclusion of modern engineering tools was incorporated into those courses. In 2008-09, this outcome received an overall achievement rating of 4 (out of 5). No actions or changes were recommended or implemented. In 2009-10, this outcome received an overall achievement rating of 3.5 (out of 5). It was postulated that the moving of more computing into the freshmen year had created a gap between the learning of the tools and the application of the tools, which occurs in the junior year. More computing was added to existing sophomore-level courses, including the differential equations course, with cooperation and assistance from the ECU Department of Mathematics. In 2019-11, preliminary results indicate the modifications have led to improvement in achievement of this outcome. • Task Specific –Instructional Memo to an English as a Second Language Audience • Reinforces communication across cultures. • Integrates graphics for multi-cultural audiences. • Task Specific Evaluation • How well did technology students adopt the strategies and suggestions for communicating across cultures? • Did each student meet the minimum criteria for a passing grade? • Development Across Tasks –Multiple Successive Assignments • Emphasizes recognition of cultural variances impacting readers’ perceptions. • Focuses on language choices and variant meanings in graphics due to color and symbol usage. • A review loop used for student growth. • Development across Tasks Evaluation • How well did technology students communicate clearly with multiple audiences, including those from other cultures? • Did each student meet the minimum criteria for a passing grade across all assignments? Results for 2010-2011 Data were gathered during the 2010-11 academic year and summer 2011. Students were identified by major, and the grades for the memo and the course were compiled . In all, data characterizing 105 students comprising 6 different majors were summarized—see Table 1. • Outcome: Graduates of the Engineering program will demonstrate an ability to use the techniques, skills, and modern engineering tools necessary for engineering practice • OVERALL RATING OF OUTCOME ACHIEVEMENT: 3.5 • 5 = Well above expectations: All measures exceed target levels Target levels for Assessment and Evaluation Plan: • 4 = Above expectations: Most measures at or above target levels Capstone Assessment: 70% of samples at 3 (satisfactory) or 4 (superior) level • 3 = Outcome achieved: Most measures at or near target levels Student Work Samples: 70% of samples at 3 (satisfactory) or 4 (superior) level 2 = Outcome not achieved: Most measures below target levels Targeted Exam Questions: 60% correct for each question • 1 = Well below expectations: All measures below target levels Student Surveys: 70% 4 or 5 responses for each related course outcome • Comments: •   Capstone Assessment: 60% of the projects were rated at 3 or 4 level, slightly below the target level. • Student Work Samples: • ENGR 1012: 80% of student SolidWorks portfolios rated 3 or 4. • ENGR 3014: 55% of student projects utilizing MATLAB to model and analyze a circuit rated 3 of 4. • ENGR 3050: 79.5% of student projects utilizing MATLAB to control a coupled-tank system rated 3 or 4. • Student Surveys: For ENGR 1012, 2050, and 4000, all related course objectives rated 79% or above 3 and 4. • Targeted Exam Questions: ENGR 2050: 74% overall correct; individual questions ranged from 29-97%. • Senior Exit Survey: On a 7-point scale, graduates rated the ability to apply modern tools specific to their concentration as 5.69 average; ability to apply engineering skills 5.62 average. • Overall, these results show an acceptable achievement of the outcome. Freshman students demonstrate a mastery of solid modeling, and do well on FE-style questions regarding spreadsheets and interpreting flow charts. Student score slightly less well on MATLAB questions, but their MATLAB programming skills are expanded in the junior year in ENGR 3014 and 3050. The lower scores in 3014 may be related to the fact that there are little or no programming exercises in the sophomore year (although MATLAB exercises are being added to the differential equations course by the math department). For the capstone projects, hardware and software was generally used correctly, but students missed opportunities to use some appropriate tools. • Recommendations: •  Recommend adding more computer exercises to sophomore-level classes. Recommend having students consider appropriate tools in their capstone project planning. Assessing Students Preparedness to Compete and Succeed in a Global Economy Through Written Communications Robert A. Chin & Carolyn Dunn Donna Hollar Department of Technology Systems Department of Construction Management Desired Outcome The problem was to prepare technology students to compete and succeed in a global economy ; students who can assess global issues and events from multiple perspectives and apply critical thinking skills to address global challenges. A two prong approach was employed within sections of ITEC 3290 Technical Writing during the 2010-11 academic year and summer 2011. Measures • Table 1. Performance for Each Measure by 105 Students Grouped by Major • Discussion • The current data is not comprehensive and representative enough at this point to indicate a reliable trend upon which to base a course of action. Respective program coordinators, the course coordinator, and the director of program assessment and accreditation, however, shall review the results for improvement opportunities and to ensure excellence is sustained. Lessons on intercultural communication continue to be updated and stressed throughout the course. • Conclusions • Accessing student achievement of the desired outcome using a writing intensive course serving six majors has provided multiple benefits: • Inter-major interaction and discussions provided expanded opportunities to appreciate the need for preparedness in a global economy. • Comparative analysis across majors aids in identifying “best practices” that can be utilized within degree programs.