1 / 27

CONCEPTS TO BE INCLUDED

Learn how to compute the reliability of a scale score using Classical Test Theory and Cronbach's alpha. Explore the factors influencing scale reliability, such as item correlation and the number of items.

mbussell
Download Presentation

CONCEPTS TO BE INCLUDED

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CONCEPTS TO BE INCLUDED • Classical test theory • Scale reliability • Cronbach’s alpha

  2. RELIABILITY: CRONBACH’s ALPHA

  3. AIM • Know how to compute the reliability of a scale score • Understand why scale reliability depends on • The correlation among items • The number of items

  4. STEPS IN SCALE CONSTRUCTION Scale analysis Conceptualization Pilot testing 3 4 5 6 1 2 Validation Collection of data Item writing

  5. SCALE ANALYSIS: RELIABILITY Reliability = --------------------------- = ------------------------------- truevariation truevariation (truevariation + measurement error) totalvariation

  6. RELIABILITY Reliability is repeatability IQ score Susan IQ score Vera Test version A on January 23: 122 133 Test version A on January 24: 119 136 Test version B on January 25: 120 134 Individualdifferences error error

  7. CALCULATING RELIABILITY • Hans 120 • Monique 122 • Vera 134 • Peter 101

  8. CLASSICAL TEST THEORY (CTT) • Observed score = latentscore + measurement error

  9. CTT: EXAMPLE • Intelligent test with 10 questions • Patrick has 8 questions correct • Measurementerrors? • Unintelligent but lucky? 8 = 7 + 1 • Intelligent but unlucky? 8 = 9 – 1 • No measurement error? 8 = 8 + 0 Measurement error Correct

  10. CALCULATING MEASUREMENT ERROR VARIANCE Patrick’sSum score : item 1 score … + item 10 score = 1 + 1 + 0 + 0 + 1 + 1 + 1 + 1 + 1 + 1 = 8 In general we have variables: Sumscore = : item 1 score + … item 10 score

  11. CALCULATING VARIANCE OF SUMS • Variance(A + B) = Variance(A) + Variance(B) + 2 * Covariance(A, B) • Sumscore = itemA + itemB • Variance(sumscore) = Variance(itemA+itemB) = Variance(itemA) + Variance(itemB) + 2 * Covariance(itemA, itemB)

  12. VARIANCE OF SUMS: EXAMPLE • Var(itemA) = 1 • Var(itemB) = 1 • Covariance = 0.9 (correlation = 0.9) • Variance(sumscore) = Var (itemA) + Var (itemB) + 2 * Cov (itemA, itemB) = 1 + 1 + 2 * 0.9 = 2 + 1.8 = 3.8

  13. Measurement error item A Item A score True score

  14. Measurement error item A Item A score True score Item B score Measurement error item B

  15. Measurement error item A Item A score True score Correlation Item B score Measurement error item B

  16. CALCULATING RELIABILITY Reliability = --------------------------- = ------------------------------- truevariation truevariation (truevariation + measurement error) totalvariation

  17. VARIANCE OF SUMS: EXAMPLE • Var(itemA) = 1 • Var(itemB) = 1 • Covariance = 0.9 (correlation = 0.9) • Variance(sumscore) = Var (itemA) + Var (itemB) + 2 * Cov (itemA, itemB) = 1 + 1 + 2 * 0.9 = 2 + 1.8 = 3.8 Reliability = 1.8 / 3.8 = 0.47

  18. VARIANCE OF SUMS: ANOTHER EXAMPLE • Var(itemA) = 1 • Var(itemB) = 1 • Covariance = 0.1 (correlation = 0.1) • Variance(sumscore) = Var (itemA) + Var (itemB) + 2 * Cov (itemA, itemB) = 1 + 1 + 2 * 0.1 = 2 + 0.2 = 2.2 Reliability = 0.2 / 2.2 = 0.09

  19. VARIANCE OF SUMS: ANOTHER EXAMPLE • Example A: Covariance = 0.9 (Correlation = 0.9) Reliability = 0.47 • Example B: Covariance = 0.1 (Correlation = 0.1) Reliability = 0.09 • The higerthecorrelationbetween items, thehigherthereliability

  20. VARIANCE OF SUMS: ANOTHER EXAMPLE • Example A: Covariance = 0.9 (Correlation = 0.9) Reliability = 0.47 • Example B: Covariance = 0.1 (Correlation = 0.1) Reliability = 0.09  The higherthecorrelationbetween items, thehigherthereliability  The more items you have, thehigherthereliability

  21. VARIANCE OF SUMS: ANOTHER EXAMPLE • Variance(A + B) = Variance(A) + Variance(B) + 2 * Covariance(A, B) • Variance(A+B+C) = Var(A) + Var(B) + Var(C) + 2Cov(A,B) + 2Cov(A,C) + 2Cov(B,C) • Variance(A+B+C+D) = Var(A) + Var(B) + Var(C) + Var(D) + 2Cov(A,B) + 2Cov(A,C) + 2Cov(A,D) + 2Cov(B,C) + 2Cov(B,D) + 2Cov(C,D)

  22. LET’S PUT IN NUMBERS • Assumeallvariances are 1 and allcovariances are 0.2 • Variance(A + B) = Variance(A) + Variance(B) + 2 * Covariance(A, B) 2 + 0.4=2.4  reliability is 0.4/2.4 = 0.17 • Variance(A+B+C) = Var(A) + Var(B) + Var(C) + 2Cov(A,B) + 2Cov(A,C) + 2Cov(B,C) 3 + 1.2 = 4.2  reliability is 1.2 / 4.2 = 0.29 • Variance(A+B+C+D) = Var(A) + Var(B) + Var(C) + Var(D)+ 2Cov(A,B) + 2Cov(A,C) + 2Cov(A,D) + 2Cov(B,C) + 2Cov(B,D) + 2Cov(C, D) 4 + 2.4  reliability is 2.4 / 6.4 = 0.38

  23. reliability reliability Correlationbetween items = 0.5 Correlationbetween items = 0.1 Number of items Number of items

  24. CRONBACH’S ALPHA • Is a way of calculatingreliability • K = number of items • Chronbach’salpha = reliability * K / (K-1)

  25. LET’S CALCULATE CRONBACH’S ALPHA • Assumeallvariances are 1 and allcovariances are 0.2 • Variance(A+B+C) = Var(A) + Var(B) + Var(C) + 2Cov(A,B) + 2Cov(A,C) + 2Cov(B,C) = 3 + 1.2 = 4.2  reliability is 1.2 / 4.2 = 0.29 • Cronbach’salpha: reliability * K / (K-1) • In this case there are 3 items, so K = 3  Cronbach’salpha = 0.29 * 3/(3-1) = 0.48

  26. THIS MICROLECTURE • Correlations between items reflect the extent to which items measure the same thing reliably • This ‘same thing’, can only be the true score (without error) • The higher the correlations between the items on the scale, the higher the reliability of the scale score • The more items the scale consists of, the higher the reliability of the scale score • Cronbach’s alpha is one of many ways to compute the reliability of the scale score

More Related