1 / 10

Assessment , Evaluation, & Statistics: A Brief Overview

Assessment , Evaluation, & Statistics: A Brief Overview. WISTPC 2014 Florida International University Miami, FL Steven J. Condly , PhD United States Military Academy at West Point scondly@gmail.com. Definitions. Measurement ( What do you know? )

mya
Download Presentation

Assessment , Evaluation, & Statistics: A Brief Overview

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Assessment, Evaluation, & Statistics:A Brief Overview WISTPC 2014 Florida International University Miami, FL Steven J. Condly, PhD United States Military Academy at West Point scondly@gmail.com

  2. Definitions • Measurement (What do you know?) • Assigning numbers to things, events, people, actions, etc. • Assessment (How do you know?) • Measurements, actions, processes, data that answer the question. • Evaluation (How are we doing?) • Comparing results and observations with goals and objectives (implied or otherwise).

  3. Implications • Measurement (assigning #s) • Need for consistency, accuracy, and precision. • Assessment (collected data) • Should relate to what it is purportedly describing. • Evaluation (comparing results to standards) • Not entirely objective, but should be reasonable and logical.

  4. Types of Evaluation • Formative • Serves to strengthen or improve the program being evaluated • In-process • Summative • Examines the effect(s) program • Examines how well program goals and objectives were met • Contains implications for corrective action

  5. Formative Evaluation • How well identified and defined is the problem? • How well does the program deal with the problem? • How well does the program progress? [feedback loop]

  6. Summative Evaluation • Similar to Formative, but looks back over the entire life of the program. • Reaches conclusions regarding effectiveness, cost, overall success, and likelihood of generalizability (or moving on to the next level). • Easier to perform if formative evaluations are being performed and data/results collected.

  7. Types of Assessment • Assessment is data collection with a purpose • Really only two ways to do it: • Question • Observe • Raw data have to be processed (statistics)

  8. Statistics • Select a good comparison criterion or group • Standard statistical techniques are alright for Likert-scaled survey data • Don’t use p-values (NHST) • Strongly influenced by sample size • Small p does not necessarily indicate a stronger relationship or effect, or practical significance • What people think it is: P(H0=0|sample) • What it actually is: P(sample|H0=0) • How much there is there?

  9. Effect Size Statistics • For Likert or interval-level data, when comparing two groups, use Cohen’s d • M1 - M2 / [(s1 + s2) / 2] • For ordinal data, when comparing two groups, use Probability of Superiority • MWU / (n1n2) • For correlations between two groups, use r2 • (r) (r) x 100 gives % of variance explained

  10. Websites • http://oerl.sri.com/ccli_resources.html • www.socialresearchmethods.net/kb/contents.php • http://www.uccs.edu/~lbecker/ • Grissom, R. J. (1994). The probability of the superior outcome of one treatment over another. Journal of Applied Psychology, 79(2), 314-316.

More Related