Robustness of the Unidimensional IRT Model

1. Robustness of the Unidimensional IRT Model Wes Bonifay Quantitative Psychology Advanced Quantitative Methods in Education Research University of California, Los Angeles

2. Robustness • Commonly applied unidimensional latent variable measurement models in structural equation modeling (SEM) and item response theory (IRT) assume that item response data are locally dependent based on a single latent factor • Local independence (LI): there are no relationships among the observed items other than those explained by the underlying common factor

3. Robustness • Most psychological constructs are substantively complex, and thus require heterogeneous item content • This heterogeneous content essentially guarantees at least some multidimensionality • If a test contains construct-relevant multidimensionality, then a single common factor will not be able to account for the relationships among the entire set of observed item responses

4. Robustness • In the real world, strict unidimensionality is patently unachievable • “Such a case will not occur in application of theory” – McDonald (1981) • “The enthusiasm for IRT has been tempered by the realization that the validity with which these methods can be applied to realistic data sets (e.g., small numbers of items and examinees, multidimensional data) is often poorly documented” – Harwell, Stone, Hsu, & Kirisci (1996)

5. Robustness • When multidimensional data are forced into a unidimensional structure, then the model is misspecified • Such misspecification may lead to biased measurement and structural model parameter estimates • How much multidimensionality can exist in the data when the goal is to fit a unidimensional model?

6. Robustness • It has been argued that IRT is robust to LI violations so long as the data are “unidimensional enough”: • characterized by a strong common dimension (Harrison, 1986) • exhibits “essential unidimensionality” (Stout, 1990) • verges on similar approximations to unidimensionality (see Hattie (1985) for a review of unidimensionality measures)

7. Robustness • Relaxing the LI assumption: • Strong LI • relationships between items are identically zero • Weaker LI • Stout’s concept of “essential independence” • “A test can be considered essentially unidimensional if LI approximately holds in a sample of test takers who are approximately equal on the latent trait” – Embretson & Reise (2000) • Weakest LI • Ip’s concept of “Weak LI in Expectation”

8. Robustness • In assessing the degree to which the data are “unidimensional enough” to use a unidimensional measurement model, researchers routinely utilize indices from various statistical domains • Nonparametric approaches: • DETECT • Mokken Scale Analysis • Factor analytic approaches: • ratio of 1st to 2nd eigenvalue • explained common variance • coefficient omega • SEM goodness-of-fit indices: • SRMR • RMSEA • CFI

9. Discussion • If we accept that multidimensionality is always present in real data, then: • A) Is it worthwhile to detect or characterize local independence violations? • B) Should we instead focus simply on determining whether the data are “unidimensional enough”? • Is characterization of LD necessary in order to assess robustness?