1 / 52

Testing Model Fit

Testing Model Fit. SOC 681 James G. Anderson, PhD. Limitations of Fit Indices. Values of fit indices indicate only the average or overall fit of a model. It is thus possible that some parts of the model may poorly fit the data.

dutch
Download Presentation

Testing Model Fit

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Testing Model Fit SOC 681 James G. Anderson, PhD

  2. Limitations of Fit Indices • Values of fit indices indicate only the average or overall fit of a model. It is thus possible that some parts of the model may poorly fit the data. • Becauwe a single index reflects only a particul;ar aspect oof model fit, a favorable value of that indesx does not by itself indicate good fit. That is why model fit is assessed based on the values of multiple indices.

  3. Limitations of Fit Statistics • Fit indices do not indicate whether the results are theoretically meaningful. • Values of fit indices that suggest adequate fit do not indicate the predictive power of the model is also high. • The sampling distribution of many fit indices are unknown.

  4. Assessment of Model Fit • Examine the parameter estimates • Examine the standard errors and significance of the parameter estimates. • Examine the squared multiple correlation coefficients for the equations • Examine the fit statistics • Examine the standardized residuals • Examine the modification indices

  5. Measures of Fit • Measures of fit are provided for three models: • Default Model – this is the model that you specified • Saturated Model – This is the most general model possible. No constraints are placed on the population moments It is guaranteed to fit any set of data perfectly. • Independence Model – The observed variables are assumed to be uncorrelated with one another.

  6. Overall measures of Fit • NPAR is the number of parameters being estimated (q) • CMIN is the minimum value of the discrepancy function between the sample covariance matrix and the estimated covariance matrix. • DF is the number of degrees of freedom and equals the p-q • p=the number of sample moments • q= the number of parameters estimated

  7. Overall measures of Fit • CMIN is distributed as chi square with df=p-q • P is the probability of getting as large a discrepancy with the present sample • CMIN/DF is the ratio of the minimum discrepancy to degrees of freedom. Values should be close to 1.0 for correct models.

  8. Chi Square: 2 • Best for models with N=75 to N=100 • For N>100, chi square is almost always significant since the magnitude is affected by the sample size • Chi square is also affected by the size of correlations in the model: the larger the correlations, the poorer the fit

  9. Chi Square to df Ratio: 2/df • There are no consistent standards for what is considered an acceptable model • Some authors suggest a ratio of 2 to 1 • In general, a lower chi square to df ratio indicates a better fitting model

  10. Transforming Chi Square to Z Z = (2*2) - (2*df-1)

  11. CMIN

  12. RMR, GFI • RMR is the Root Mean Square Residual. It is the square root of the average amount that the sample variances and covariances differ from their estimates. Smaller values are better • GFI is the Goodness of Fit Index. GFI is between 0 and 1 where 1 indicates a perfect fit. Acceptable values are above 0.90.

  13. GFI and AGFI (LISREL measures) • The AGFI takes into consideration the df available to test the model. • Values close to .90 reflect a good fit. • These indices are affected by sample size and can be large for poorly specified models. • These are usually not the best measures to use.

  14. RMR, GFI • AGFI is the Adjusted Goodness of Fit Index. It takes into account the degrees of freedom available for testing the model. Acceptable values are above 0.90. • PGFI is the Parsimony Goodness of Fit Index. It takes into account the degrees of freedom available for testing the model. Acceptable values are above 0.90.

  15. RMR, GFI

  16. Comparisons to a Baseline Model • NFI is the Normed Fit Index. It compares the improvement in the minimum discrepancy for the specified (default) model to the discrepancy for the Independence model. A value of the NFI below 0.90 indicates that the model can be improved.

  17. Bentler-Bonett Index or Normed Fit Index (NFI) • Define null model in which all correlations are zero: 2 (Null Model) - 2 (Proposed Model) 2 (Null Model) • Value between .90 and .95 is acceptable; above .95 is good • A disadvantage of this index is that the more parameters, the larger the index.

  18. Comparisons to a Baseline Model • RFI is the Relative Fit Index This index takes the degrees of freedom for the two models into account. • IFI is the incremental fit index. Values close to 1.0 indicate a good fit. • TLI is the Tucker-Lewis Coefficient and also is known as the Bentler-Bonett non-normed fit index (NNFI). Values close to 1.0 indicate a good fit. • CFI is the Comparative Fit Index and also the Relative Noncentrality Iindex (RNI). Values close to 1.0 indicate a good fit.

  19. Tucker Lewis Index or Non-normed Fit Index (NNFI) • Value: 2/df(Null Model) - 2/df(Proposed Model) 2/df(Null Model) • If the index is greater than one, it is set to1. • Values close to .90 reflects a good model fit. • For a given model, a lower chi-square to df ratio (as long as it is not less than one) implies a better fit.

  20. Comparative Fit Index (CFI) • If D= 2 - df, then: D(Null Model) - D(Proposed Model) D(Null Model) • If index > 1, it is set to 1; if index <0, it is set to 0 • A lower value for D implies a better fit • If the CFI < 1, then it is always greater than the TLI • The CFI pays a penalty of one for every parameter estimated

  21. Baseline Comparisons

  22. Parsimony Adjusted Measures • PRATIO is the Parsimony Ratio. It is the number of constraints in the model being evaluated as a fraction of the number of constraints in the independence model. • PNFI is the result of applying the PRATIO to the NFI. • PCFI is the result of applying parsimony adjustments to the CFI.

  23. Parsimony-Adjusted Measures

  24. Measures Based on the Population Discrepancy • NCP is an estimate of the noncentrality parameter obtained by fitting a model to the population moments rather than to the sample moments. • The 90% confidence interval is also computed for the NCP.

  25. NCP

  26. The Minimum Sample Discrepancy Function • FMIN is the minimum value of the discrepancy .

  27. FMIN

  28. Root Mean Square Error of Approximation (RMSEA) • Value: [(F0pop-F0est)/n]/df • F0 is the minimum vale of the discrepancy function. • If 2 < df for the model, RMSEA is set to 0 • Good models have values of < .05; values of > .10 indicate a poor fit. • It is a parsimony-adjusted measure. • Amos provides upper and lower limits of a 90% confidence interval for the RMSEA

  29. PCLOSE • PCLOSE is the probability for testing the null hypothesis that the population RMSEA is no greater than 0.05.

  30. RMSEA

  31. Information Theoretic Measures • These indices are composite measures of badness of fit and complexity. • Simple models that fit well receive low scores. Complicated poorly fitting models get high scores. • These indices are used for model comparison not to evaluate a single model.

  32. AIC

  33. Akaike Information Criterion (AIC) • Value: 2 + k(k-1) - 2(df) where k= number of variables in the model • A better fit is indicated when AIC is smaller • Not standardized and not interpreted for a given model. • For two models estimated from the same data, the model with the smaller AIC is preferred.

  34. Information Theoretic Measures • BCC is the Browne-Cudeck Criterion • BIC is Bayes Information Criterion. • CAIC is the consistent AIC • ECVI except for a constant scale factor it is the same as AIC. • MECVI except for a scale factor is the same as the BCC.

  35. ECVI

  36. Nonhierarchical Models

  37. Hierarchical Models

  38. Difference in Chi Square Value: X2diff = X2model 1 -X2 model 2 DFdiff = DF model 1 –DFmodel 2

  39. Miscellaneous Measures • HOELTER is the largest sample size for which one would accept the hypothesis that a model is correct.

  40. Hoelter Index • Value: (N-1)*2(crit) + 1 2 Where 2 (crit) is the critical value for the chi-square statistic • The index should only be calculated if chi square isstatistically significant.

  41. Hoelter Index (2) • If the critical value is unknown, can approximate: [ (1.645 + (2df-1) ]2 + 1 2 2/ (N-1) • For both formulas, one rounds down to the nearest integer • The index states the sample size at which the chi square would not be significant

  42. Hoelter Index (3) • In other words, how small one’s sample size would have to be for chi square to no longer be significant • Hoelter Recommends values of at least 200 • Values < 75 indicate poor fit

  43. HOELTER

More Related